A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Misc] Add Transformers version upper bound in requirements (#59614)
Isotr0py committed 19 days ago
[Rust Frontend] Expose gRPC port in Python vllm serve (#59659)
Alec committed 19 days ago
[Bugfix] Fix minimax-m3 multimodal processor compatability with Transformers v5.18 (#59613)
Isotr0py committed 19 days ago
[ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm (#58344)
kliuae committed 19 days ago
[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Lucas Wilkinson committed 19 days ago
[Model][Spec Decode] Enable EAGLE3/DSpark pipeline parallelism for Sarvam MLA (#55902)
Mohit Singla committed 19 days ago
[Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates (#59450)
Matthew Bonanni committed 19 days ago
[Doc] Document ECMooncakeConnector for EPD disaggregation (#59196)
Teng Ma committed 19 days ago
[Perf][PCP] Shard decode requests across PCP ranks (#52162)
Qiu Chunshuo committed 19 days ago
[Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500)
Guanxin Li committed 19 days ago
[Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Martin Hickey committed 19 days ago
[Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend (#59679)
Harry Mellor committed 19 days ago
[Bugfix][Bench] Clean up synthetic video files and writers (#59529)
Prudhvi Vuda committed 19 days ago
[Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Yoni Gozlan committed 19 days ago
[Feature] Add native ModelExpress weight transfer backend (#58399)
Hyunjae Woo committed 19 days ago
[Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623)
Flora Feng committed 19 days ago
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387)
Harry Mellor committed 19 days ago
[Feature] Release WorkspaceManager scratch on sleep (#59156)
aoshen02 committed 19 days ago
[Bugfix][Rust Frontend] Preserve whitespace in GLM string arguments (#59654)
ai-jz committed 19 days ago
[Bugfix][CPU] Fix macOS build on Apple Clang < 17: structured binding… (#58932)
chowdmia committed 19 days ago