A high-throughput and memory-efficient inference and serving engine for LLMs
main
[PD][NixlPush] Skip extra `add_remote_agent` step in D->P handshake (#49345)
Nicolò Lucchesi committed 3 months ago
[Bugfix] Enhance extra_config handling for layer name suffix matching (#48589)
Xin He committed 3 months ago
[Elastic EP] Async preparation (#47288)
Itay Alroy committed 3 months ago
[Build] Fix DeepEP CUDA driver stub linking (#50103)
Kevin H. Luu committed 3 months ago
[Rust Frontend] Align sampling validation with Python (#47494)
Reid committed 3 months ago
[Rust Frontend] Fix finish reason for named tool choices (#49496)
Reid committed 3 months ago
[CI] Increase Qwen3.5 MTP GSM8K generation length (#49881)
Jiangyun Zhu committed 3 months ago
[CI] Add PyTorch stable ABI audit check (#48164)
Chris Leonard committed 3 months ago
[Test] Skip ROCm AITER MLA prefill tests on non-ROCm platforms (#49945)
Liangliang Ma committed 3 months ago
[Rust Frontend] Add ordinary-text tokenizer encoding (#49992)
Bugen Zhao committed 3 months ago
[Kimi-K3] Add AttnRes kernels (#50090)
Thien Tran committed 3 months ago
[CI][ROCm] Soft fail LoRA mirror (#50086)
Andreas Karatzas committed 3 months ago
[Bugfix][Spec Decode] Preserve draft buffers across level-2 sleep (#49774)
aoshen02 committed 3 months ago
[Bugfix] Respect cgroup memory limits on all platforms (#49966)
Chauncey committed 3 months ago
[Core][Frontend] Add weight version tagging for RL rollouts (#49040)
Shuolei Wang committed 3 months ago
[Bugfix] Fix multi-modal support on CPU MRV2 (#50073)
Li, Jiang committed 3 months ago
Remove triton per group quant [ROCm] [Bugfix] (#49621)
afriedri committed 3 months ago
[Build] Fix CUDA arch detection producing kernel-less builds on SM121 (#49904)
Ayushman Singh committed 3 months ago
[Bugfix] Only pad transformers backend `value` when it is narrower (#50060)
Nick Hill committed 3 months ago
[Bugfix][KV Offload][OBJ] Preserve job completion during cleanup (#49947)
MINJUN GIL committed 3 months ago