[kernel] Integrate FlashInfer BF16 CuTeDSL Low Latency GEMM (#50572)

Jiahan Chang (Cyrus) committed about 2 months ago

Add support for loading dflash2 model in speculators format (#53797)

Fynn Schmitt-Ulms committed about 2 months ago

[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)

Ronald committed about 2 months ago

Upgrade tpu-inference to v0.28.0 (#54020)

Dennis Yeh committed about 2 months ago

[Attention] Enable masked MHA for GLM-5 head dimensions (#53785)

Matthew Bonanni committed about 2 months ago

[Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens (#54089)

Mikhail Podvitskii committed about 2 months ago

[Perf][DSv4] Use native CUDA SwiGLU clamp kernel for Humming MoE (throughput +1.40%) (#53685)

Chauncey committed about 2 months ago

[Kimi Perf] Tune hopper low latency gemm kernel, 4%~97% performance improvement (#54088)

Wentao Ye committed about 2 months ago

[Bugfix] Avoid TCPStore port collision for co-located non-DP Ray engines (#53666)

Jeffrey Wang committed about 2 months ago

[Rust Frontend] Align OpenAI request and response edge cases (#53218)

Bugen Zhao committed about 2 months ago

[Model Runner V2] Use MRV2 for all models by default (#53183)

Nick Hill committed about 2 months ago

[Doc] Add EXAONE-4.0-1.2B to batch invariance tested models (#53839)

Paarth Sharma committed about 2 months ago

Fix Humming MoE activation_output aliasing (#54056)

Elvir Crnčević committed about 2 months ago

[Bugfix] Handle malformed namespace tools (#53763)

Taneem Ibrahim committed about 2 months ago

[Attention][DCP] Use FlashInfer native CP for MLA decode (#54012)

Summer Yang committed about 2 months ago

[ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540)

Shanshan Shen committed about 2 months ago

[CI/Build][Hardware][NVIDIA] Add opt-in Rubin Docker builds (#53443)

Shang Wang committed about 2 months ago

[Perf][GLM5.2] Fuse sparse MLA Q concatenation with head padding (#53878)

Chauncey committed about 2 months ago

[Bugfix][Scheduler] Don't pad spec decode up to `max_model_len` (#53962)

Nick Hill committed about 2 months ago

[Bugfix] Preserve parallel HY-V3 calls delivered in one streaming delta (#53965)

Taneem Ibrahim committed about 2 months ago