A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Rust Frontend] Support HTTP RL weight synchronization (#56567)
Zupeng Wang committed about 1 month ago
[CI][Bugfix] Complete FSE fixture R-SWA contract (#56710)
Kevin H. Luu committed about 1 month ago
[Fast Start] Fast loader support nnode>1 (#55468)
siyu committed about 1 month ago
[Bugfix][Frontend] Support Responses text types in DeepSeek V4.1 (#56299)
CedricHuang committed about 1 month ago
[ROCm][Perf] Tune MiniMax-M3 decode top-k for short contexts (#55235)
Fangzhou Ai committed about 1 month ago
Zhewen Li committed about 1 month ago
[XPU][CI] update test requirements (#56704)
liuzhenwei committed about 1 month ago
[Perf] Parallelize mHC pre-norm JIT warmup (#56683)
Woosuk Kwon committed about 1 month ago
Wentao Ye committed about 1 month ago
[Performance][EPD] Reduce Python proxy serialization overhead (#56657)
Tianyu Guo committed about 1 month ago
[Attention] Remove DCP indexer interleave guard and test TP1 output parity (#55802)
stu-cao committed about 1 month ago
[Warmup] Gemma 4 de-JITification (#55768)
Roberto L. Castro committed about 1 month ago
[Bugfix][PCP][DCP] Respect interleave in indexer KV gather mapping (#56715)
Lucas Wilkinson committed about 1 month ago
[NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645)
Lucas Wilkinson committed about 1 month ago
[Perf] Avoid redundant conversions in FP8 dummy initialization (#56688)
Woosuk Kwon committed about 1 month ago
[CI] Initialize warmup registry in V2 QSA runner fixture (#56707)
Lucas Wilkinson committed about 1 month ago
[Bugfix] Skip Triton autotune inspection without Triton (#56676)
Kevin H. Luu committed about 1 month ago
[ROCm][Performance] Avoid blocking MiniMax M3 scalar upload (#56170)
Fangzhou Ai committed about 1 month ago
[Bugfix] Detect unloaded NVFP4 weight scales with a NaN sentinel (#52501)
Pavel Zakharov committed about 1 month ago
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305)
Matthew Bonanni committed about 1 month ago