[ROCm] Run init test engine in-process to avoid KV-cache OOM (#48527)

djramic committed 3 months ago

[Bugfix][Spec Decode] Support heterogeneous QK fusion geometry (#48671)

aoshen02 committed 3 months ago

[Bugfix] Set kv_quant_mode on the generic MLA KV-cache spec (#48379)

drakosha committed 3 months ago

Build with ABI stable FlashMLA (#48174)

Jane (Yuan) Xu committed 3 months ago

[Security] Replace diskcache to eliminate pickle deserialization (#44549)

Russell Bryant committed 3 months ago

[LoRA][1/N] Integrate flashinfer MoE LoRA for BF16 model (#48632)

Jee Jee Li committed 3 months ago

[Quant] Enable humming w[2-7]a[4,8] inference with compressed-tensors (#46390)

HDCharles committed 3 months ago

[CI][AMD] Configure MI300 tests for native execution without DinD (#48387)

Andreas Karatzas committed 3 months ago

[Core][LoRA] Support fp32 lm_head (head_dtype) on the LoRA path (#48525)

Karthik Kothuri committed 3 months ago

[ROCm][CI] Remove mxfp4 test skips after `amd-quark` 0.12 release (#47330)

Micah Williamson committed 3 months ago

add pad-aware reduce path (#48385)

gnovack committed 3 months ago

[Perf] Remove redundant repeat and copy for dsv4, 1.8% E2E TPOT improvement. (#48137)

Wentao Ye committed 3 months ago

[Bugfix] Preserve unloaded non-persistent buffers during layerwise reload (#44371)

Joan committed 3 months ago

[Bugfix][R3] Exclude draft routers from expert capture (#48622)

aoshen02 committed 3 months ago

fix: size FlashInfer prefill workspace to batch head footprint (#48428)

Joe Rowell committed 3 months ago

[Bugfix][CI] Fix test_head_dtype quant_method test on ROCm (#48654)

Micah Williamson committed 3 months ago

[Bugfix] MoRIIO toy P/D proxy: add /health (#45222)

chaeminlim-mb committed 3 months ago

Log fully resolved pooling config at startup (#48030)

Taneem Ibrahim committed 3 months ago

[CI][Bugfix] Fix FlashAttention reported MLA dimension support (#48631)

Matthew Bonanni committed 3 months ago

[ROCm][CI] fix flashinfer import check (#48647)

Divakar Verma committed 3 months ago