A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][Quantization] Add Quark W4A8 (INT4-FP8) MoE CI coverage (#48050)
amd-sourjya committed 3 months ago
[Docs] Fix broken anchor links in serving/pooling/MoE docs (#49654)
Euisuh Jeong committed 3 months ago
[ROCm][CI] Prepare AMD mirrors for regating (#49270)
Andreas Karatzas committed 3 months ago
[BugFix][LoRA] Skip marlin-backend gpt-oss LoRA tests on XPU (#49385)
Chaojun Zhang committed 3 months ago
[CI][PD] Add hybrid SSM P_TP>D_TP accuracy sweep entry (#49593)
Nicolò Lucchesi committed 3 months ago
[CI] Disable reasoning in Responses smoke test (#49511)
Andreas Karatzas committed 3 months ago
[ROCm][CI] Language Models tests tiny-mixtral with aiter fix (#49551)
music-dino committed 3 months ago
[Bugfix][KV cache] Support sparse-MLA targets with SWA drafts (#48776)
Michael Goin committed 3 months ago
[Bug] Fix batch invariance rms norm comparison (#49603)
Wentao Ye committed 3 months ago
[Bugfix] Restore structured output logger initialization (#49626)
Chang Guo committed 3 months ago
[Core] Update PyTorch to 2.13.0, torchvision to 0.28.0, triton to 3.7.1 (#48155)
Andrey Talman committed 3 months ago
[CI] Bump PyTorch Compilation Unit Tests timeout to 150 min (#49606)
Andrey Talman committed 3 months ago
[CI][Bugfix] Fix test isolation in block_int8/ptpc_fp8 MoE kernel tests (#49609)
Nick Hill committed 3 months ago
[CI] Isolate cudagraph tests in child processes (#49510)
Andreas Karatzas committed 3 months ago
[DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case (#49486)
Wentao Ye committed 3 months ago
[CI] Use explicit devices in quantization tests (#49512)
Andreas Karatzas committed 3 months ago
Revert "[MRV2] Always build attn metadata at capture time" (#49364) (#49451)
vllm-agent committed 3 months ago
[Bugfix][Structured Output][Spec Decode] Advance grammar across reasoning boundary (#44993)
yue.yu committed 3 months ago
Fix GPTQ quantized Qwen3.5 MTP weight loading with spec decode (#48816)
vllmellm committed 3 months ago
[Perf] Defer MM embeds loading off the event loop (#49477)
Guan-Ming Chiu committed 3 months ago