A high-throughput and memory-efficient inference and serving engine for LLMs
main
[AMD][BugFix] Add gpu_sync_allowed to ROCm AITER FA backend (#53641)
rasmith committed about 2 months ago
[Bugfix][GPT-OSS] Fix strict tool-call grammar to accept Harmony renders (#52222)
yzong-rh committed about 2 months ago
[Bugfix] Restore portable all2all backend default (#53952)
vllm-agent committed about 2 months ago
[Bugfix][Rust Frontend] Fix LogprobsTensors wire schema mismatch (#53939)
Jung Jiyu committed about 2 months ago
[XPU][CI] increase timeout of extract_hidden_states tp2 (#53862)
liuzhenwei committed about 2 months ago
[Doc] Add Granite 3.1 series to batch invariance tested models (#53650)
Shenglei Fu committed about 2 months ago
Wentao Ye committed about 2 months ago
[Bugfix] Guard tool call argument JSON parsing in chat message postprocessing (#48922)
Venkat Balaji S committed about 2 months ago
[XPU][TEST]Move LoRA Multimodal to B70 in Intel GPU CI (#53841)
xiangdong committed about 2 months ago
[Model] Remove unused DeepSeek V4 top-k buffer helper (#53697)
Woosuk Kwon committed about 2 months ago
[ROCm][K3] Extend FP8 asm MLA prefill to non-divisor small head counts (#51040)
xiaohuguo2023 committed about 2 months ago
[Bugfix][DP] Synchronize the device on pause completion (#52914)
aoshen02 committed about 2 months ago
[Bugfix][ROCm][Disagg] Fix MoRIIO shared KV memory region registration (#53698)
Avinash Paul committed about 2 months ago
[Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644)
Konstantin Dunas committed about 2 months ago
[Bugfix] Make Gemma4 MTP suppress_tokens masking CUDA-graph-safe (#53884)
Luciano Martins committed about 2 months ago
[ROCm][DSV4][Perf] Fuse DeepSeek V4 C4 compressor GEMMs (#53838)
Fangzhou Ai committed about 2 months ago
[Bugfix][ROCm] Capture CUDA graphs on the current stream (#53818)
fanxingran committed about 2 months ago
[Model] Pixtral: use packed multimodal encoder attention (#52185)
Oliver Holworthy committed about 2 months ago
[Mypy Fix] Mypy fix for "vllm/model_executor/models/[tT]" (#53466)
Jensen Chen committed about 2 months ago
[Bugfix] Handle empty FlatLogprobs slices and delta output (#53704)
Tony committed about 2 months ago