A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][Frontend] Check EC requirements for each metadata item (#56070)
shaohuaxi committed about 1 month ago
[Bugfix][Multimodal] Restore cached audio inputs with UUIDs (#56310)
Jiatai Wang committed about 1 month ago
[Bugfix][ROCm] Create linear layer biases with `requires_grad=False` (#56161)
Micah Williamson committed about 1 month ago
[ROCm][Bugfix] Fix profiler in TheRock image (#56190)
Matt committed about 1 month ago
[KV Connector] Support symmetric DCP disagg for hybrid mamba models (#55531)
Wei Zhao committed about 1 month ago
[Bugfix] Avoid MistralCommonBackend for HF tokenizers (#54192)
Julien Denize committed about 1 month ago
[ROCm][Feature] Support KV connectors with ROCM_AITER_UNIFIED_ATTN (#53695)
Simon Danielsson committed about 1 month ago
[Core] MRV2 support for fast-prefill (#56145)
Nick Hill committed about 1 month ago
[ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM (#51692)
Simon Danielsson committed about 1 month ago
[Docs]: quote variable-bearing wheel URLs (#56286)
Josiah Davis committed about 1 month ago
[DCP][Kernel][Perf] Fuse the empty-shard LSE mask into the A2A pack kernel (#54889)
Rita Brugarolas committed about 1 month ago
[Bugfix][MoE] Fix batched CUTLASS workspace overallocation (#55579)
Itay Alroy committed about 1 month ago
[Bugfix] Honor explicit empty and zero CLI arguments (#55710)
Cenab Batu Bora committed about 1 month ago
[CI/Build][CPU] Fix flaky rust downloads, broken prune flag, and triton-cpu cache coupling (#56247)
Li, Jiang committed about 1 month ago
[Perf] Use UVA-backed contents for MRV2 apply_write (#55819)
bjf-frz committed about 1 month ago
[Bugfix] Qwen3-VL(-MoE): pass architectures to with_hf_config for pipeline parallelism (#43272)
Alexander Crettenand committed about 1 month ago
[CI] Enable ruff `INP` to require `__init__.py` under `vllm/` (#56264)
Harry Mellor committed about 1 month ago
[Mypy] Fix typing for R/S models (#54157)
Taneem Ibrahim committed about 1 month ago
[Fast Start] Support fp4 (#55465)
siyu committed about 1 month ago
[Feature][Spec Decode] MTP with separate (possibly quantized) lm head for nemotron (#54574)
YoavMiron committed about 1 month ago