A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Fix Nixl+Offloading PD edge cases on TheRock image (#57249)
Matt committed about 1 month ago
[Rust Frontend] Bump vllm-proto to 0.3.0 (#57233)
Alec committed about 1 month ago
[Bugfix][KV Offload] Compact canonical MLA rows (#56799)
Itay Etelis committed about 1 month ago
[CI] Support `FORCE_COLOR` env var; enable colors in CPU CI output (#56913)
Nick Hill committed about 1 month ago
[EC Connector] Add Metrics Collection (#54960)
omerpaz95 committed about 1 month ago
[Core] Don't issue a blocking collective RPC during engine handshake (#57226)
Oxana Korzh committed about 1 month ago
[Scheduler] Add --max-num-active-seqs to cap RUNNING admission (#56758)
Chuan (Richard) Li committed about 1 month ago
[Test][Core] Add hybrid model prefix cache hit-rate coverage (#57127)
Jiangyun Zhu committed about 1 month ago
Use skip-redaction option for crcr-report (#56532)
Andrey Talman committed about 1 month ago
[Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533)
Alec committed about 1 month ago
[ROCm][Bugfix] Ignore descales for unquantized AITER caches (#56726)
dongluw committed about 1 month ago
[A2A] Add DeepEPv2 to SP Supported List (#57210)
Robert Shaw committed about 1 month ago
[Rust Frontend] Normalize native renderer reasoning controls (#56998)
Bugen Zhao committed about 1 month ago
[ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934)
Rohan Potdar committed about 1 month ago
Divakar Verma committed about 1 month ago
[Bugfix][Metrics][MFU] Size activation traffic from the model dtype (#57070)
Thillai Chithambaram committed about 1 month ago
[Bugfix][Spec Decode] Fix EAGLE and dense draft startup with EP (#56930)
kyleliang-nv committed about 1 month ago
[ROCm][CI] Shard MI300 Multimodal Processor (#57056)
Aarushi Jain committed about 1 month ago
[Tests] Delete deprecate torchao tests for v1 configs (#52956)
andrewor14 committed about 1 month ago
[Frontend] Add per-request metrics to Responses API (#55084)
xinnywinne committed about 1 month ago