[ROCm][Perf] Fix RMSNorm+Quant fusion for gfx950 (non-fnuz) (#41825)

frida-andersson committed 6 months ago

[Bugfix] Fix int32 overflow in DeepGEMM SiLU/mul FP8 Triton kernel (#42201)

Vensen committed 6 months ago

[Docs] Add Apple Silicon documentation for vLLM-Metal GPU support (#41987)

Alexa Griffith committed 6 months ago

Add documentation about vLLM FIPS compliance (#42190)

Vinay R Damodaran committed 6 months ago

[Model Runner V2] Fix `seq_lens_cpu_upper_bound` (#42202)

Nick Hill committed 6 months ago

[ROCm][DSv4] implement flash sparse mla with triton kernels (#41812)

Hexiang Wang committed 6 months ago

[Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement (#40408)

Wentao Ye committed 6 months ago

[Bug] Fix kimi dtype issue with `mm_projector_forward` (#42081)

Wentao Ye committed 6 months ago

[Bugfix] Accept canonicalized `modelopt_*` quant_method in `_extract_modelopt_quant_algo` (#42181)

Vadim Gimpelson committed 6 months ago

[kv_offload] Move `FilterReusedOffloadingManager` logic to `CPUOffloadingManager` (#41727)

Martin Hickey committed 6 months ago

[Refactor] Cleanup batch invariant dead code (#41993)

Wentao Ye committed 6 months ago

[ROCm] Clean up a bit the AITER FA backend (#41942)

pschlan-amd committed 6 months ago

[DSv4] Improved dequant gather K cache kernel (#42236)

Thien Tran committed 6 months ago

[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392)

Rohan Potdar committed 6 months ago

[kv_offload] Set offloading connector to prefer HND layout (#41928)

Martin Hickey committed 6 months ago

Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming (#42244)

Noa Neria committed 6 months ago

[Frontend]Responses API supports chat_template_kwargs (#42272)

Chauncey committed 6 months ago

Fix EXAONE-4.5 to align with Transformers update (#42246)

Kyungmin Lee committed 6 months ago

[CI] Make Python-only Installation optional (#42293)

haosdent committed 6 months ago

[KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy (#40364)

Nicolò Lucchesi committed 6 months ago