A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Rust Frontend] Add support for TLS in render server (#54999)
Wen Zhou committed about 2 months ago
[CI] Zen5 image build (#50314)
Andy Linfoot committed about 2 months ago
[CI] Avoid logging test server environment values (#54379)
Taneem Ibrahim committed about 2 months ago
[ROCm][Perf] Optimize MiniMax-M3 decode indexer and top-k (#54682)
Fangzhou Ai committed about 2 months ago
[warmup] overlap renderer warmup and engine core initialization (#54557)
Ning Xie committed about 2 months ago
Fix DeepSeek V4 FlashMLA auto KV cache dtype (#45091)
elehayym committed about 2 months ago
[Rust Frontend] Support `--lora-modules` for static adapter loading (#54837)
Will Eaton committed about 2 months ago
[Rust Frontend] Use token-attributed text in reasoning and unified parsers (#54884)
Bugen Zhao committed about 2 months ago
[ROCm][Perf] Add low-M FP32 router GEMM for gfx950 (#54845)
Fangzhou Ai committed about 2 months ago
[Fusion] Manual `ActivationQuantFusionPass` initial application (#51415)
Misha Goin committed about 2 months ago
[Bugfix][Model] Fix FP8 PLE loading in mixed ModelOpt checkpoints (#54882)
sychen52 committed about 2 months ago
[Kernel] Enable Kimi-K3 SiTU on the CuteDSL MoE backend and the SM107 low-latency GEMM plan (#54606)
Bolin Sun committed about 2 months ago
[Model] add GLM-5.3-Flash support (#53906)
Jiangyun Zhu committed about 2 months ago
Yifan Jiang committed about 2 months ago
[Bugfix][KV Connector] Populate SimpleCPUOffload BlockStored metadata (#54325)
mevince committed about 2 months ago
[Core] Deprecate "all" mamba cache mode (#55041)
Nick Hill committed about 2 months ago
[Bugfix][KV Connector] Safely fill circular buffers in DecodeBench (#54879)
Jeff (Junze) Ma committed about 2 months ago
Optimize PLE MTP metadata transfers (#55054)
bhsueh_NV committed about 2 months ago
[Core] Triton kernel for small-batch top-p only masking (#54651)
Nick Hill committed about 2 months ago
feat: Add support for reasoning_token_count to reasoning parser (#54982)
JasonCohere committed about 2 months ago