[CI] Fix LM Eval Large Models (H100) (#32423)

Matthew Bonanni committed 10 months ago

Add thread_n=64 support to Marlin MoE (#32360)

Michael Goin committed 10 months ago

[Feat] Support non-gated MoE with Marlin, NVFP4 CUTLASS, FP8, INT8, compressed-tensors (#32257)

TomerBN-Nvidia committed 10 months ago

[Refactor] Remove unused file (#32422)

Wentao Ye committed 10 months ago

[MoE Refactor][17/N] Apply Refactor to Bf16 (#31827)

Yongye Zhu committed 10 months ago

[ROCM] DSfp4 mla projection gemms weight dynamic quantization (#32238)

Aleksandr Malyshev committed 10 months ago

[BugFix] Python file source reading can fail on UnicodeDecodeError (#32416)

Richard Zou committed 10 months ago

[ROCm] [CI] [Release] Rocm wheel pipeline with sccache (#32264)

TJian committed 10 months ago

[UX] Use kv_offloading_backend=native by default (#32421)

Michael Goin committed 10 months ago

[BugFix] Fix `assert x_s.shape[-1] == x_q.shape[-1] // group_shape[1]` in Blackwell Quantized MoE Test (#32362)

Lucas Wilkinson committed 10 months ago

[Attention][AMD] Make flash-attn optional (#30361)

Matthias Gehre committed 10 months ago

fixing podman build issue (#32131)

smit kadvani committed 10 months ago

[Feature] Support async scheduling + PP (#32359)

Wentao Ye committed 10 months ago

[Model Runner V2] Support FlashInfer backend & Fix CUDA Graph bug [1/2] (#32348)

Woosuk Kwon committed 10 months ago

[ROCm][Bugfix] Disable hip sampler to fix deepseek's accuracy issue on ROCm (#32413)

Pleaplusone committed 10 months ago

[ROCm][Perf] Enable shuffle kv cache layout and assembly paged attention kernel for `AiterFlashAttentionBackend` (#29887)

Pleaplusone committed 10 months ago

[Quant] Support MXFP4 W4A16 for compressed-tensors MoE models (#32285)

Dipika Sikka committed 10 months ago

[Attention][MLA] Make `FLASHINFER_MLA` the default MLA backend on Blackwell, and TRTLLM the default prefill (#32339)

Matthew Bonanni committed 10 months ago

[ROCm] Improve error handling while loading quantized model on gfx120… (#31715)

brian033 committed 10 months ago

[3/N] Group together media-related code (#32406)

Cyrus Leung committed 10 months ago