A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][Docker] Pin AINIC apt repo to snapshot 1.117.5-a-77 (#56459)
djramic committed about 1 month ago
[Metrics] Consolidate Prometheus histogram bucket defaults into a single module (#48866)
Guy Stone committed about 1 month ago
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len (#56446)
Harry Mellor committed about 1 month ago
[Bugfix] Fix Qwen3-VL and Cosmos3-Edge text architectures for CPU and pipeline parallelism (#53699)
labAxiaoming committed about 1 month ago
[Bugfix][Scoring] Warn when serving original Qwen3 reranker without chat template (#56017)
김재억 committed about 1 month ago
[Bugfix] Fix GLM-OCR MTP position masking during CUDA graph capture (#56447)
Yuxuan Zhang committed about 1 month ago
[Bugfix][Multimodal] Parse decoded video frame lists as a single video (#55326)
waizuichougou committed about 1 month ago
[Bugfix][Responses] Fix browser.find action type (#55305)
Nicole LiHui 🥜 committed about 1 month ago
[ROCm][Bugfix] Fix AITER preshuffled FP8 block-scale kernel (#56433)
Matt committed about 1 month ago
[Bugfix][Pooling] Restore token limits for offline Jina scoring (#56415)
shaohuaxi committed about 1 month ago
[Bugfix][Rust Frontend][Renderer] Align DeepSeek tool-call arguments with deepseek-recipe (#56260)
JinYan Su committed about 1 month ago
[Misc] Log FlashInfer allreduce workspace init failure as error (#55127)
Nicolò Lucchesi committed about 1 month ago
[ROCm][CI] Add HY-V4 generation coverage (#55667)
Andreas Karatzas committed about 1 month ago
[Model] Support DeepSeek-V4.1-Flash (#56214)
Yongye Zhu committed about 1 month ago
[XPU][CI] Add decord to test requirements (#56355)
Tony Lin committed about 1 month ago
[Bugfix] Pin EPLB and MLA host-to-device transfer buffers (#56138)
Kevin H. Luu committed about 1 month ago
[Bugfix] Fall back to full decode graphs for noncompiled models (#55095)
Andreas Karatzas committed about 1 month ago
[Bugfix][Bench] Fix bench mm-processor crash in shared request sampling (#56300)
Nicolò Lucchesi committed about 1 month ago
[CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365)
JulienDarve committed about 1 month ago
[CPU] Speedup LM Head on Arm CPUs (#55352)
Fadi Arafeh committed about 1 month ago