[MM][CG] Support ViT CG for Qwen2-VL (#41736)

John Calderon committed 6 months ago

[Spec Decode] Support hybrid attention models in extract_hidden_states (#39949)

Michael Goin committed 6 months ago

[Bugfix][Model] Gemma4 MoE routing closure captures per_expert_scale, breaking functional_call substitution (#42250)

Noelia Bentancor committed 6 months ago

[Feature] Support custom callable proposer backend for speculative decoding (#39487)

CynicDora committed 6 months ago

[Feature] Support compile mode for batch invariance on SM80 (#42456)

Wentao Ye committed 6 months ago

[kv_offload] Add multi-tier KV cache offloading framework (#40020)

Ronen Schaffer committed 6 months ago

[Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support (#41946)

TJian committed 6 months ago

[XPU] [CT] Enable CT W4A4MxFp4 path and add xpu kernel (#38896)

zofia committed 6 months ago

[CI] Re-enable Nemotron Parse parity test and switch testing to nemotron-parse v1.2 (#42498)

Marek Wawrzos committed 6 months ago

[AMD] skip machete tests for rocm (#42326)

hissu-hyvarinen committed 6 months ago

[Bugfix][Quark] Fix W8A8 INT8 garbage outputs on Step-3.5-Flash (and other 3-key fused-MoE Quark exports) (#41892)

PikaPikachu committed 6 months ago

Triton attention: add USE_TD constexpr for tensor descriptor Q/K/V load/store (#40327)

Artur Fierka committed 6 months ago

[kv_offload] Add req_id to ReqContext for per-request tracking (#42507)

Ronen Schaffer committed 6 months ago

[Bugfix][Qwen3-VL] Fix pipeline-parallel deepstack initialization (#42394)

SILONG ZENG committed 6 months ago

[Bugfix] Fix scipy audio resampling ratio (#42233)

JooHo Lee committed 6 months ago

[ROCm][CI] Skip ROCm batch invalid-input test pending torch fix (#41572)

Andreas Karatzas committed 6 months ago

[ROCm] Run AITER RMSNorm pad fusion before AR RMS fusion (#42411)

akii96 committed 6 months ago

[Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate (#42329)

hallerite committed 6 months ago

[CI] Use uv with Python 3.12 for PyPI wheel upload (#42470)

Kevin H. Luu committed 6 months ago

[Feat][KVConnector] Add `bind_gpu_block_pool()` to KVConnectorBase_V1 (#39654)

Yifan Qiao committed 6 months ago