A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix] Fix Llama4ForCausalLM initialization test failure (#46341)
liuzhenwei committed 4 months ago
Temporarily skip M3 on CI (#46352)
Roger Wang committed 4 months ago
[Doc] Update MiniMax-M3 (#45940)
Jee Jee Li committed 4 months ago
[Rust Frontend] Support thinking_token_budget for chat and completions (#46137)
Chao-Ju Chen committed 4 months ago
[feature][kv_offload] Self-describing KV events for OffloadingConnector (#43468)
Change72 committed 4 months ago
[Model] Remove MiniMaxText01, MiniMaxVL01, MiniMaxForCausalLM (#45993)
Tiezhen WANG committed 4 months ago
[CPUOffloadingManager] Maintain evictable list in LRUCachePolicy (#46216)
Varun Sundar Rabindranath committed 4 months ago
[1/N][Core] add partial prefix cache primitives (#45939)
Jiangyun Zhu committed 4 months ago
[Frontend] Refactor ServingTokenization entrypoint. (#46022)
wang.yuqi committed 4 months ago
[Bugfix] [Rust Frontend] Fix stop string truncation with repeated matches (#46113)
Reid committed 4 months ago
[DSV4][XPU] Pass gemm1_clamp_limit to XpuFusedMoe (#44517)
Ma Jian committed 4 months ago
[SpecDecode] Support DFlash with FlashInfer (#43081)
gq112 committed 4 months ago
[XPU][CI] Add agent_tags for Intel GPU CI (#45768)
xiangdong committed 4 months ago
[ROCm][CI] Enable kv_connector unit tests on ROCm (#45955)
Micah Williamson committed 4 months ago
[CI] Fix missing `tp_size` attribute on `RoutedExperts` (#46163)
fxmarty-amd committed 4 months ago
[Hardware][AMD][CI] Fix Spec Decode Eagle test group (#46018)
Matt committed 4 months ago
[Hardware][AMD][CI] Fix Kernels Attention test groups (#46080)
Matt committed 4 months ago
[ROCm][CI] skip test_double_aiter_rms_quant_fusion (#45967)
Charlie Fu committed 4 months ago
[Hardware][AMD][CI] Fix gfx942 Kernels MoE test group (#46298)
Matt committed 4 months ago
[Bugfix][Qwen3-VL] Fix multi-video crash with list-valued fps/num_frames (#46305)
Ting SUN committed 4 months ago