A high-throughput and memory-efficient inference and serving engine for LLMs
main
[XPU][UT] skip GLM-5.3-Flash test on XPU (#55266)
Qiming Zhang committed about 2 months ago
[Docs] Add example for Renderer.render_cmpl() usage (#54265)
Shaurya Pratap Srivastava committed about 2 months ago
[Rust Frontend] Record Mooncake/NIXL KV-connector metrics (#52755)
Ilya Markov committed about 2 months ago
[Bugfix] Reject tokenizer-less Qwen VL processor init (#54886)
Luyi Xiao committed about 2 months ago
docs: note that enforce_eager also disables torch.compile (#55271)
Anshul Desai committed about 2 months ago
[Bugfix][Docs] Package glm5next nvidia subtree and fix its docstrings (#55214)
Harry Mellor committed about 2 months ago
[Bugfix][KV Connector] Fix DecodeBenchConnector prefix block selection (#54878)
Jeff (Junze) Ma committed about 2 months ago
[ROCm][CI] Bump ROCk base to ROCm 10.0 (#55246)
Rohan Potdar committed about 2 months ago
[EC Connector] P2P NIXL + CPU EC Connector (#47941)
omerpaz95 committed about 2 months ago
[Performance][DSv4] Size dequant gather launch grid by rows (#55061)
aoshen02 committed about 2 months ago
[Bugfix][PD] Pad resumed speculative decode requests (#55126)
Ziming Huang committed about 2 months ago
[KVConnector] Add retention interval to OffloadingConnector (#51886)
bnellnm committed about 2 months ago
[ROCm][CI] Build and publish TheRock nightly docker images (#55014)
Rohan Potdar committed about 2 months ago
[Core][KV Events] Echo session_id on GPU BlockStored events (#51381)
guangli_Liu committed about 2 months ago
[Bugfix][DCP] Materialize prefill keys on non-owner ranks (#54908)
Yunxiao Ning committed about 2 months ago
aoshen02 committed about 2 months ago
[Rust Frontend] Report reasoning tokens in chat completion usage (#54883)
Bugen Zhao committed about 2 months ago
[ROCm] Bump AITER to 0.1.21.post1 (#52826)
Rohan Potdar committed about 2 months ago
[Perf] Prefetch the weight before the PDL wait in fused_q_kv_rmsnorm (#55020)
Yongye Zhu committed about 2 months ago
[Perf][Kimi-K3] Cut MLA decode concat/cache epilogue latency (#54896)
Yongye Zhu committed about 2 months ago