A high-throughput and memory-efficient inference and serving engine for LLMs
main
[PD][Bugfix] Fix NIXL hybrid MLA+mamba heterogeneous TP (#49297)
Ziming Huang committed 3 months ago
[Renderer] Offload derender CPU work to renderer thread pool (#49396)
Guan-Ming Chiu committed 3 months ago
[ROCm] Remove redundant AITER fused_qk_rmsnorm probe (avoids config-time HIP init) (#47992)
stefankoncarevic committed 3 months ago
Add auto label for xpu relate issue (#49398)
Kunshang Ji committed 3 months ago
[ROCm] Upgrade NIXL and UCX (#49251)
Andreas Karatzas committed 3 months ago
[MRV2] Always build attn metadata at capture time (#49364)
Woosuk Kwon committed 3 months ago
[Bugfix][Attention] Ignore empty MLA context chunks during merge (#49294)
Lucas Wilkinson committed 3 months ago
bnellnm committed 3 months ago
[Misc] Move PyNvVideoCodec stuff out of gpu worker (#49322)
Isotr0py committed 3 months ago
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178)
aoshen02 committed 3 months ago
[CI][Bugfix] Fix ROCm FP8 KV cache dtype in attention backend test (#49380)
peizhang56 committed 3 months ago
[CI][Bugfix] Fix and wire streaming-input tests (#49356)
Nick Hill committed 3 months ago
[Misc] Fix terminal output logo coloring (#49344)
Nick Hill committed 3 months ago
stefankoncarevic committed 3 months ago
skip cudagraph/DP padding in topk (#48979)
gnovack committed 3 months ago
[CI] Bump timeout of `entrypoints-integration-api-server-openai-part-2` (#49359)
Nick Hill committed 3 months ago
[Bugfix] DFlash fc sized wrong when num_target_layers != num_hidden_layers (#48524)
Michael Goin committed 3 months ago
[CI] Add gemma-4-E4B-it-assistant to CI gsm8k for GemmaMTP (#49243)
Michael Goin committed 3 months ago
[Bugfix] fix cutalss version upgrade bug, need update MSG new commit (#49016)
rongfu.leng committed 3 months ago