A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459)
Robert Shaw committed 28 days ago
[CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369)
stefankoncarevic committed 28 days ago
[Test][Determinism] Cover chunked prefill in the batch-invariance suite (#55612)
Bob Ok committed 28 days ago
[Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215)
Juntian Liu committed 28 days ago
[CI] Share BF16 baselines across quantization comparison tests (#58469)
Aarushi Jain committed 28 days ago
[5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected (#58455)
Kevin H. Luu committed 28 days ago
[Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit (#58462)
Nick Hill committed 28 days ago
[CI] Disable JIT warmup by default in VllmRunner (#58452)
Misha Goin committed 28 days ago
[CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm (#58465)
Kevin H. Luu committed 28 days ago
[ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts (#58419)
djramic committed 28 days ago
[ROCm][Test] Cover MoRI graph replay and output lifetime (#58093)
Andreas Karatzas committed 29 days ago
[Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) (#55936)
陈志谦 committed 29 days ago
[Bugfix] Pick a KV block size supported by every attention backend (#49845)
Divy committed 29 days ago
[CI] Select one GPU for the H200 initialized snapshot E2E step (#58351)
Thang Nguyen committed 29 days ago
Lio Einaudi committed 29 days ago
Doc: add DiffusionGemma to supported models (#46466)
CBP committed 29 days ago
[Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing (#57923)
Simon Danielsson committed 29 days ago
[Bugfix][NIXL] Restore successful push completion reporting (#58188)
Dao007forever committed 29 days ago
[CI][ROCm] Add an MI355 Kimi-K3 unit test group (#58012)
Oxana Korzh committed 29 days ago
[ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy (#58095)
Andreas Karatzas committed 29 days ago