A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Refactor] Derive is_reasoning_end from the engine grammar (#56200)
Flora Feng committed about 1 month ago
[Bugfix][Rust Frontend] Honor `add_generation_prompt` in DeepSeek V3.2/V4/V4.1 renderers (#56593)
Yeonwoo Sung committed about 1 month ago
[Perf][DSpark] Collapse DeepSeek-V4.1 draft states before SP all-gather (#56903)
Juntian Liu committed about 1 month ago
[CI] Shard V1 KV Connectors 1→4 (#56324)
Thang Nguyen committed about 1 month ago
[CI] Fix NIXL push worker test stub after failure deferral (#56955)
Kevin H. Luu committed about 1 month ago
[Frontend] Use XGrammar schema constraints for DeepSeek V4.1 (#56408)
Yixin Dong committed about 1 month ago
[Feature][PCP] Support decode-only FULL CUDA graphs (#53867)
Qiu Chunshuo committed about 1 month ago
[Bugfix][Frontend] Handle aborted requests in beam search (#56249)
shaohuaxi committed about 1 month ago
[CI] Extend Arm CPU kernel shard timeout (#56601)
Kevin H. Luu committed about 1 month ago
[Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention (#56876)
Yongye Zhu committed about 1 month ago
[KV Offload] Add KVCR secondary-tier adapter (#53624)
Moein Khazraee committed about 1 month ago
[Bugfix][SparseMLA] Fix piecewise cudagraph capture crash in index group (#56825)
Jared Wen committed about 1 month ago
[Doc] Clarify ITL vs TPOT Prometheus metrics (#55283)
Jyotirmoy Roy committed about 1 month ago
[Bugfix][Kimi-K3] Keep transient checkpoints out of prefix-cache eviction (#56794)
Sy03 committed about 1 month ago
[CI] Select the supported DCP backend for PCP eval (#56677)
Kevin H. Luu committed about 1 month ago
cjackal committed about 1 month ago
[CI] Extend DSv4 engine readiness timeout on main (#56735)
Kevin H. Luu committed about 1 month ago
[Bugfix] Fix stale HPC QK-norm weights after weight refit (#56706)
aoshen02 committed about 1 month ago
[Cleanup] Remove vestigial `tpu_input_batch.py` (#56898)
Nick Hill committed about 1 month ago
[Model Runner V2] Tolerate lack of pinned memory support (#56908)
Nick Hill committed about 1 month ago