A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm] Pad non-aligned AITER MLA heads (#51647)
yinfengLiu committed 2 months ago
[Bugfix][Structured Output] Stop XGrammar token batches at termination (#52805)
Flora Feng committed 2 months ago
[Bugfix][DP] Don't assume the engines started when forwarding a wake (#51481)
aoshen02 committed 2 months ago
[Rust Frontend][RL] add routed expert prompt offset (#52703)
Biswa Panda committed 2 months ago
[CI] Standardize test job labels by device (#52659)
Kevin H. Luu committed 2 months ago
[DBO][CI] Increase the coverage of prefill DBO in test_dbo.py (#48628)
Sage Moore committed 2 months ago
[Bugfix] Detect all attention-spelling variants in ModelConfig.is_hybrid (#52161)
Michał Ganczarenko committed 2 months ago
[CI] Upgrade huggingface-hub to 1.28.0 (#52797)
Andreas Karatzas committed 2 months ago
[Rust Frontend] Wait for all utility calls to finish (#52671)
Connor Carpenter committed 2 months ago
[Bugfix][MLA] Do not use Dense MHA for GLM-5.2 (#52512)
Woosuk Kwon committed 2 months ago
[ROCm][Perf] Enable fused KDA decode on gfx942 (MI325X) (#52293)
Matvei Pashkovskii committed 2 months ago
[ROCm] Gate Torch FP8 scaled-MM on architecture support (#51021)
Strahinja Stamenkovic committed 2 months ago
Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend (#52182)
Misha Goin committed 2 months ago
[MOE] Standardize and abstract fused shared expert optimization selection (#51695)
fxmarty-amd committed 2 months ago
[Bugfix][PaliGemma] Remove stale image embedding scaling (#52692)
Jikui Xie committed 2 months ago
[Model] Enable LoRA support for tower and connector in LlavaNextForConditionalGeneration (#49788)
karthik committed 2 months ago
[Multimodal] Reorganize video decoder backends (#49155)
Isotr0py committed 2 months ago
Upgrade Flashinfer version to 0.6.17 (#52681)
Wei Zhao committed 2 months ago
[Bugfix][Gemma4] Align parser enable_thinking default with template (#52430)
Alexander Lee committed 2 months ago
[CI] Register CPU CI "VLLM_CPU_CI_ENV" environment variable (#52633)
Taneem Ibrahim committed 2 months ago