A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Misc] Use VLLMValidationError in pooling input validation (#51931)
Frank committed 2 months ago
[Model][LoRA] Add tower/connector LoRA support for Ultravox (#48215)
Yangyi Gao committed 2 months ago
Remove NIXL reinstall step (#51882)
ovidiusm committed 2 months ago
[Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing (#51772)
Yongye Zhu committed 2 months ago
[XPU] update UMD to 26.27 (#50513)
Yan Ma committed 2 months ago
Revert "[Perf][ROCm] Dual-stream decode with hipgraphs" (#52024)
Simon Danielsson committed 2 months ago
[BugFix] Reserve the bonus query slot in DFlash scheduling budget (#51256)
kx committed 2 months ago
[Bugfix] Bound KV block zeroing launch geometry (#52058)
Lucas Wilkinson committed 2 months ago
[Core] Clearer comments in `BlockPool.free_blocks()` (#52076)
Nick Hill committed 2 months ago
[Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" (#52003)
Wentao Ye committed 2 months ago
[Bugfix][KV Offload] Emit self-describing CPU events at KV-group block granularity (#51614)
Ziqi Fan committed 2 months ago
[KV Offload] Expose data-parallel topology to offloading backends (#51879)
Ziqi Fan committed 2 months ago
fix and test EPLB balancedness calculation (#51813)
Julien Debache committed 2 months ago
[Hardware][Power] Unqualized MoE Backend for Power (VSX) (#51624)
Akash kaothalkar committed 2 months ago
[CI] Mirror external test assets in vLLM S3 (#52064)
Kevin H. Luu committed 2 months ago
[Bugfix][ROCm][CI] Restore the DeepSeek-V4 input GEMM override point (#51821)
stefankoncarevic committed 2 months ago
Yifan Jiang committed 2 months ago
[CI] Force source builds for hybrid dependencies (#52043)
Andreas Karatzas committed 2 months ago
[ROCm] [bugfix] Chunked prefill paged decode masked load perf (#50017)
afriedri committed 2 months ago
[CI Bug] Fix ci moe test (#52009)
Wentao Ye committed 2 months ago