A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393)
Divakar Verma committed 27 days ago
[Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020)
김재억 committed 27 days ago
[Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679)
Juntian Liu committed 27 days ago
[Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954)
Matej Sirovatka committed 27 days ago
[Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526)
Thien Tran committed 27 days ago
[Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527)
Thien Tran committed 27 days ago
[Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434)
Nick Hill committed 27 days ago
[CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB MIG (#58496)
Kevin H. Luu committed 27 days ago
[Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371)
Samuel Kim committed 27 days ago
SIDDARTHA REDDY committed 27 days ago
Yifan Qiao committed 27 days ago
[Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626)
Samyabrata Maji committed 27 days ago
[Bugfix][KVConnector] Finalize saves on steps without a forward (#57775)
Yifan Qiao committed 27 days ago
Release prompt_embeds tensor when its InputBatch slot is freed (#57988)
Khushali Desai committed 27 days ago
[KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472)
Zijing Liu committed 27 days ago
[Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582)
Misha Goin committed 27 days ago
[Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368)
Netanel Haber committed 27 days ago
[Bugfix][Quantization] Give LM heads standard linear metadata (#58444)
Misha Goin committed 27 days ago
[ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558)
stefankoncarevic committed 27 days ago
[Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288)
Sense_wang committed 27 days ago