A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Hardware][AMD][CI] Use Triton-based AITER MHA for LM Eval Qwen-3.5 Models Tests (#46691)
Matt committed 4 months ago
[CI] Depend GPQA Eval DGX Spark job on arm64 image build (#46736)
Michael Goin committed 4 months ago
[Kernel] Vectorized fp32 `moe_sum` reduction and support any topk (#46643)
Michael Goin committed 4 months ago
[ROCm][CI] Fine-tuning queues and test names (#39238)
Andreas Karatzas committed 4 months ago
Fix P/D with DP Supervisor (#46628)
Robert Shaw committed 4 months ago
[Perf] Remove redundant clone for GLM, Deepseek etc (#46651)
Wentao Ye committed 4 months ago
[Kernel][MoE] Tune block-FP8 fused MoE for low-batch decode (#46642)
Michael Goin committed 4 months ago
[ROCm]: Bump aiter to 0.1.16.post2 (#46692)
Rohan Potdar committed 4 months ago
[CI] Re-enable skipped glm and seedoss parser tests (#46671)
Flora Feng committed 4 months ago
[Bugfix] FLASHINFER_MLA_SPARSE_SM120 compatibility with GLM-5 NVFP4 (#46506)
Gabriel Wu committed 4 months ago
[Bugfix] Fix NVFP4+MTP crash: force unquantized mtp.fc for Qwen3Next (#46316)
Ranran committed 4 months ago
[Core] Avoid mixed length specdec batches via padding (#45237)
Yiliu Dong committed 4 months ago
[ROCM] [Communication] Add INT3 quantization method for quickreduce (#45666)
haoyangli0109 committed 4 months ago
[ROCm][CI] rm duplicate Distributed Torchrun ci test (#46729)
Divakar Verma committed 4 months ago
[Pooling] Fix Cohere embed billed image token accounting for mixed-content inputs (#46093)
Taneem Ibrahim committed 4 months ago
[CI] Pin GitHub Actions to commit hashes in macos-smoke-test.yml (#38290)
Russell Bryant committed 4 months ago
[MoE] [MoE Refactor] Add moe kernel oracle abc 37753 (#43461)
Qiuyang Yue committed 4 months ago
Matt committed 4 months ago
[NIXL][Mamba] Add Mamba1 support to NIXL P/D disaggregation (#45019)
Asaf Gardin committed 4 months ago
[CPU][CI/Build] Allow more CPU CI agents (#46702)
Li, Jiang committed 4 months ago