A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Kernel][SM100] Add a CuTeDSL fused query kernel (#49792)
xiaozhoupy committed 3 months ago
[ci] Update CI notify workflow with PR write permissions (#51079)
Kevin H. Luu committed 3 months ago
[CI] Stabilize GLM-5.2 PCP evaluation (#51015)
Kevin H. Luu committed 3 months ago
[MoE][Humming] Support SiTU activation for Kimi-K3 (#50510)
Julian Huang committed 3 months ago
[Kernel][Inkling] Fuse shared-expert partial addition into the Lamport collective (#50697)
Canlin Guo committed 3 months ago
[Spec Decode] Enable fused non-causal TokenSpeed MLA for DSpark (#50911)
NVShreyas committed 3 months ago
Wentao Ye committed 3 months ago
[MM][CG] Support ViT full CUDA graph for Kimi-K2.5 (#50929)
Linkun committed 3 months ago
[Core] Explicitly manage torch CPU threads in workers (#49919)
Nick Hill committed 3 months ago
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection (#50157)
amitz-nv committed 3 months ago
[Kimi-K3][AMD] Fuse AttnRes state updates and norms (#50593)
yinfengLiu committed 3 months ago
[Bugfix][MoE] Filter packed expert weights during EP loading (#49558)
aoshen02 committed 3 months ago
fix: fuse weightless RMSNorms at their declared width (#50867)
Anuj Bolewar committed 3 months ago
[Model Runner v2] E/P/D disaggregation support (#38390)
Wentao Ye committed 3 months ago
[Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580)
Bugen Zhao committed 3 months ago
[Mamba] enable prefix cache by default (#50991)
Jiangyun Zhu committed 3 months ago
[Misc] Avoid importing `nixl_ep` on every `vllm serve` config (#50879)
Nicolò Lucchesi committed 3 months ago
[Bugfix][CPU] Fix macOS build: std::sqrt is not constexpr under libc++ (#50915)
Harjoth Khara committed 3 months ago
[Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906)
yimdev committed 3 months ago
[Bugfix] Flatten >2D multimodal embeddings, not just 3D (#50250)
Michał Ganczarenko committed 3 months ago