A high-throughput and memory-efficient inference and serving engine for LLMs
main
[5/N] Share HiSparse host cache across TP ranks (reopens #52760) (#56629)
Lucas Wilkinson committed about 1 month ago
[PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157)
PatrykSaffer committed about 1 month ago
[ROCm][Bugfix] Fix elastic EP scaling deadlock (#56610)
Matt committed about 1 month ago
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Raphaël Rialland committed about 1 month ago
[CI] Update DeepSeek V4.1 MegaMoE routing test (#56599)
Kevin H. Luu committed about 1 month ago
[CI] Fix Qwen3 Omni DSpark load test config (#56600)
Kevin H. Luu committed about 1 month ago
[Bugfix] Fix DeepGEMM FP8 warmup coverage (#56452)
Ayushman Singh committed about 1 month ago
[4/N] Expose HiSparse cache metrics via KV connector stats (#56061)
Nicolò Lucchesi committed about 1 month ago
[Bugfix][CI] skip conftest for NPU compatibility test (#55799)
Zeyu Yang committed about 1 month ago
[CI/Build] Add DSv4.1 auto-label rules and narrow DSv4 (#56332)
jcotant-inferact committed about 1 month ago
Add @arpera to CODEOWNERS of Structured Output (#56501)
Artem Perevedentsev committed about 1 month ago
[Bugfix][CI] Update CuTeDSL indexer Q sentinel test for migrated wrappers (#56594)
Roberto L. Castro committed about 1 month ago
[Frontend][Rust] Reject empty structured-output values (#54821)
Clinton Thomas committed about 1 month ago
[Elastic EP] Reuse CUDA graphs across reconfiguration (#54985)
Itay Alroy committed about 1 month ago
[Perf] Fuse DSV4.1 input metadata preparation with Triton (#56562)
Woosuk Kwon committed about 1 month ago
[XPU][CI] Skip ROCm test on non-ROCm platforms (#56555)
liuzhenwei committed about 1 month ago
[Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522)
JartX committed about 1 month ago
[Bugfix] Initialize data parser in Nano-Nemotron audio test (#56401)
Taneem Ibrahim committed about 1 month ago
[Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings (#56398)
Taneem Ibrahim committed about 1 month ago
[XPU][CI] Remove pip install dependency in test yaml files (#55171)
Kunshang Ji committed about 1 month ago