A high-throughput and memory-efficient inference and serving engine for LLMs
main
[TEST][CI] Fix serve rlhf tests subprocess import errors (#59622)
Miłosz Grunwald committed 20 days ago
[CI][XPU] Skip CUDA-IPC weight sync metrics test on Intel (#59556)
liuzhenwei committed 20 days ago
[Metrics][P/D] Add KV-fetch stage gauges for async KV loads (#58874)
Zijing Liu committed 20 days ago
[ROCm][BugFix] Revert AITER PA gluon decode from ROCM_AITER_FA (#54805)
ukannika committed 20 days ago
[ci] Update mergify to rebase if behind by 100 commits (#59710)
Kevin H. Luu committed 20 days ago
[Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks (#57693)
Anurag Singh committed 20 days ago
[Bugfix][CI] Widen DBO+DP+EP GSM8K accuracy margin on ROCm (#59700)
Divakar Verma committed 20 days ago
[Bugfix][CLI] Include inherited field docstrings in get_attr_docs (#49821)
hcl committed 20 days ago
[ROCm][CI] Raise the MI355 DeepSeek-R1 GSM8K startup wait to 1800s (#59666)
Aarushi Jain committed 20 days ago
[Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321)
Flora Feng committed 20 days ago
[Perf][HiSparse] Avoid repeated prefix scans and residency updates (#57930)
Matej Sirovatka committed 20 days ago
[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path (#59300)
Yongye Zhu committed 20 days ago
[KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875)
Zijing Liu committed 20 days ago
[ROCm][CI] Drop two no-GPU AMD mirrors from the CPU test areas (#59593)
stefankoncarevic committed 20 days ago
[Bugfix][Engram] Fix intermittent Triton 3.8 crash in the lookup kernel (#59639)
djramic committed 20 days ago
[MyPy] Fix mypy errors in `vllm/model_executor/models/[kK]*` (#59402)
Harry Mellor committed 20 days ago
[Bugfix][Responses] Use standard reasoning content-part events (#59652)
ai-jz committed 20 days ago
[ROCm][Triton] Migrate Kimi-K3 kernels from make_block_ptr to tensor … (#58769)
Jaden Mathias committed 20 days ago
[KV Connector][NIXL] Coalesce host-buffer KV copies across cache groups (#54483)
Chanbin Lim committed 20 days ago
[Perf] Integrate flash-maxsim Triton kernels for late-interaction scoring (#40337)
roipony committed 20 days ago