A high-throughput and memory-efficient inference and serving engine for LLMs
main
[CI] Reuse ColQwen3 models across pooling tests (#55889)
Andreas Karatzas committed about 1 month ago
[Core] Let SWA layers take the primary block size to avoid inflating the KV block LCM (#53007)
bnellnm committed about 1 month ago
[CI] Split long misc test groups by command (#52346)
Kevin H. Luu committed about 1 month ago
[Bugfix] Handle null RoPE parameters for NoPE layers (#55949)
Andreas Karatzas committed about 1 month ago
[2/N][KV Connector][NIXL] Support per-region transfer geometry (#53780)
Matthew Bonanni committed about 1 month ago
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809)
Roderick Wu committed about 1 month ago
[ROCm] [Docker] Upgrade default AINIC repo to ship libionic 54.0-187-1 (#54112)
shikamd123 committed about 1 month ago
[Bugfix] speedup nvfp4 kv for FMHA (#55031)
sychen52 committed about 1 month ago
[Test] Dequantize NVFP4 KV cache scales in the layout the kernel writes (#55908)
Raya Elena Solano committed about 1 month ago
[Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction (#55643)
Jiangyun Zhu committed about 2 months ago
[Refactor] Remove utils dead code (#53941)
Wentao Ye committed about 2 months ago
[Core] Scope PCP-DP validation to GPU manager (#54523)
Qiu Chunshuo committed about 2 months ago
[Qwen3.8-Flash-Next] Tune FP8 TP2/TP4 Triton MoE on B200 (#55890)
Canlin Guo committed about 2 months ago
[LoRA] Clarify target module matching logic (#55865)
linitra24 committed about 2 months ago
[Kimi Bug] Fix kimi k3 startup cuda graph issue with recoverSSM (#55774)
Wentao Ye committed about 2 months ago
[4/N][warmup][DSv4] Migrate common attention kernels (#50176)
Roberto L. Castro committed about 2 months ago
[Bugfix] Fix Kimi K3 loading with interleaved weight streams (#53379)
Sherif Waly committed about 2 months ago
[Model] Enable torch.compile for Sarvam MLA (#55817)
Mohit Singla committed about 2 months ago
[Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume (#53945)
akshaver committed about 2 months ago
[Bugfix] Apply attention sinks in the Transformers backend (#52156)
Thomas Parnell committed about 2 months ago