A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Build] Update DeepGEMM pin to deepseek-ai nv_dev tip (#52035)
Yongye Zhu committed 2 months ago
[Bugfix] Pin DeepEP by its full commit hash (#52028)
Tyler Michael Smith committed 2 months ago
[Bugfix][Kernel] Fix persistent top-k histogram reuse after short rows (#49139)
xiao feng committed 2 months ago
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Lucas Wilkinson committed 2 months ago
[K3 Perf] Flash kda out kernel for prefill, 1.1~1.4x kernel performance improvement (#51311)
Wentao Ye committed 2 months ago
kiroxu committed 2 months ago
[Bugfix][ROCm][MoE] Update AITER MXFP4 W4A16 tests to the renamed expert_mask (#51980)
stefankoncarevic committed 2 months ago
[CI Bug] Fix ci qwen3.5 (#52007)
Wentao Ye committed 2 months ago
Hongxia Yang committed 2 months ago
[ROCm] update triton in base docker for gluon compatibility (#51464)
Hongxia Yang committed 2 months ago
[Bugfix] Bound Anthropic stop sequences (#51997)
Taneem Ibrahim committed 2 months ago
[Docs] Warn that --api-key does not gate all endpoints (#51999)
Russell Bryant committed 2 months ago
[Model] Add native Dots3 NOTE multimodal support (#51255)
范裕达 committed 2 months ago
Avoid long-blocking H2D copies in ViT (#51841)
Max Hu committed 2 months ago
[XPU][CI/Release][2/N] add triton shim in xpu requirements (#51935)
Kunshang Ji committed 2 months ago
[Bugfix] Disable fine-grained prefix-cache hits for incompatible hybrid KV layouts (#51843)
Michael Goin committed 2 months ago
[Bugfix][Multimodal] Invalidate retained PyNvVideoCodec decoder after failure (#51139)
dmai-afk committed 2 months ago
[Bugfix][Frontend] Return 400 for invalid PyNvVideoCodec video input (#51120)
dmai-afk committed 2 months ago
[Bugfix] Fix MiniMax-M3 compressed-tensors FP8 MoE SwiGLU params (#46845)
Tan Pin Siang committed 2 months ago
[CI] Add registry layer cache to x86 CPU image build (#51911)
Li, Jiang committed 2 months ago