A high-throughput and memory-efficient inference and serving engine for LLMs
main
[multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Brandon Pelfrey committed 3 months ago
Stabilize GPU memory teardown between ROCm CI tests (#49242)
Aarushi Jain committed 3 months ago
[CI] fix compile test | refactor VLLM_DISABLE_COMPILE_CACHE for tests (#49770)
Divakar Verma committed 3 months ago
[ROCm][CI] Force native compile caches onto local disk (#49763)
Aarushi Jain committed 3 months ago
[Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of the KV-head dedup (#49499)
Yifan Qiao committed 3 months ago
[XPU] add warning for xpu graph limitations (#49419)
liuzhenwei committed 3 months ago
[BUGFIX] Fix log capture in KV test (#49655)
liuzhenwei committed 3 months ago
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768)
Woosuk Kwon committed 3 months ago
[ROCm][CI] Wait for ROCm VRAM to settle between compiled and eager LL… (#49739)
Aarushi Jain committed 3 months ago
[ROCm][Docker] Drop MORI_GPU_ARCHS so MoRI autodetects the device arch (#49737)
Rohan Potdar committed 3 months ago
[ROCm][CI] Fix XPASS(strict) on mixed audio embeds test (#49733)
djramic committed 3 months ago
[KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flaky api-server test (#49734)
Harry Mellor committed 3 months ago
[Model] Remove Plamo2 (#49729)
Harry Mellor committed 3 months ago
[CI] Stabilize memory-sensitive compile and structured output tests (#49749)
Jiangyun Zhu committed 3 months ago
[Bugfix] Support non-uniform page sizes in KVBlockZeroer (#49704)
Elvir Crnčević committed 3 months ago
[AMD][Bugfix][EPLB] Fix elastic EP scaling accuracy on ROCm (#47206)
Oxana Korzh committed 3 months ago
[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321)
Tyler Michael Smith committed 3 months ago
[Bugfix] Register axk1 config to fix A.X-K1 init (#49727)
djramic committed 3 months ago
Fix GLM-4.1V video placeholder token ID handling. (#49484)
Aarushi Jain committed 3 months ago
[CI] Avoid unnecessary Hugging Face metadata requests (#49508)
Andreas Karatzas committed 3 months ago