A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][Compile] Fuse AITER static FP8 attention output (#58099)
Andreas Karatzas committed 29 days ago
[Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732)
aoshen02 committed 29 days ago
[Mooncake] Address review nits from #56855 (#57174)
Nicolò Lucchesi committed 29 days ago
[ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors (#58281)
Andreas Karatzas committed 29 days ago
[ROCm][CI] Mirror the three TurboQuant evaluation groups on MI355 (#58282)
Andreas Karatzas committed 29 days ago
[Quantization] Select per-token NVFP4 MoE backends explicitly (#57176)
Matej Sirovatka committed 29 days ago
[Tests] Select V2 for diffusion scheduler unit tests (#58272)
Andreas Karatzas committed 29 days ago
[ROCm][Perf] Use wvSplitK for single-output GEMMs (#53283)
00CC committed 29 days ago
[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303)
Ting SUN committed 29 days ago
[MoE] Use GateLinear for all MoE models (#58234)
Jee Jee Li committed 29 days ago
[XPU][CI] Deselect tests/v1/spec_decode/test_mtp.py::test_glm_mtp_defers_lm_head (#58237)
xiangdong committed 29 days ago
[Spec decode] Support variable-length decode for Kimi-K3 adaptive ver (#52988)
Albert Cheng committed 29 days ago
[Compilation] Fix QuTLASS compilation with PyTorch 2.13 (#58173)
Wentao Ye committed 29 days ago
[XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb (#55721)
Michał Ganczarenko committed 29 days ago
[Bugfix] Set worker runtime threads before profiling and compilation (#55891)
Andreas Karatzas committed 29 days ago
[ROCm][CI] Stage G gating (#50922)
Andreas Karatzas committed 29 days ago
[CI][Bugfix] Update IPC test caller for #57312's _apply_entries signature (#58107)
vllm-agent committed 29 days ago
[ROCm][Compile] Support BF16 AsyncTP fusion (#58098)
Andreas Karatzas committed 29 days ago
[ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451)
Fangzhou Ai committed 29 days ago
[EPD] Support metadata-only audio inputs (#57887)
Tianyu Guo committed 29 days ago