A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Observability] log sub process killing in process manager force kill (#52314)
Ning Xie committed 22 days ago
[Core] Bound UniProc EngineCore startup threads to available CPUs (#58946)
Andreas Karatzas committed 22 days ago
[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268)
Harshal Adhav committed 22 days ago
Ting SUN committed 22 days ago
[XPU] Use encoder-only model runner for EC producer instances (#59320)
liuzhenwei committed 22 days ago
[Bugfix][Model] Fix DiffusionGemma silently freezing attention mask under CUDA graph replay (#51994)
Fabian Joswig committed 22 days ago
[Bugfix][Frontend] Report named Anthropic tool calls as tool_use (#47598)
Ting SUN committed 22 days ago
[Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses (#59173)
Shijie Lyu committed 22 days ago
[Agents] Add API compatibility checking skill (#58003)
Flora Feng committed 22 days ago
[Refactor] Remove dead tests code (#58916)
Wentao Ye committed 22 days ago
[Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036)
Lucas Wilkinson committed 22 days ago
[Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling (#59107)
Mia Ojeda committed 22 days ago
[Benchmark] Add sweep warmup and failure recovery (#57305)
Louie Tsai committed 22 days ago
support rl feature : weight checker (#51350)
shiyuan680 committed 22 days ago
[Bugfix][MiniMax M3] Share target embeddings with MTP under PP (#58648)
xiaozhoupy committed 22 days ago
[Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235)
Matthew Bonanni committed 22 days ago
[CI] Fix mock type narrowing in LoRA serving test (#59344)
Kevin H. Luu committed 22 days ago
[Dependency] Upgrade FlashInfer to 0.7.0.post1 (#59323)
Wei Zhao committed 22 days ago
[Rust Frontend] Use the MiMo structural-tag builder (#59148)
Bugen Zhao committed 22 days ago
[Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286)
Nick Hill committed 22 days ago