add(v1): RequestStatesStats to RequestOutput (#24947)

HUIJONG JEONG committed about 1 year ago

[Perf] Optimize `reshape_and_cache` CUDA Kernel (#25955)

Jiangyun Zhu committed about 1 year ago

[Input] Remove unused `prompt` field (#26097)

Cyrus Leung committed about 1 year ago

[Misc] Remove typing.List (#26150)

Varun Sundar Rabindranath committed about 1 year ago

[BUG] Reorder model config creation (#26124)

ahao-anyscale committed about 1 year ago

`FusedMoE` support for the Transformers backend (#22650)

Harry Mellor committed about 1 year ago

[Model] Use `merge_by_field_config` for MM models (G) (#26117)

Cyrus Leung committed about 1 year ago

[Bugfix] Fix import `gemm_afp4wfp4` failure on AMD (#26068)

Zhewen Li committed about 1 year ago

[ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (#26104)

TJian committed about 1 year ago

[Build/CI] Revert back to Ubuntu 20.04, install python 3.12 with uv (#26103)

Tyler Michael Smith committed about 1 year ago

[gpt-oss] disable tool server initialization if no tool in request (#25790)

Andrew Xia committed about 1 year ago

[Bug]: Limit num_reqs in dummy_run when max_num_seqs is small (#26144)

Benjamin Chislett committed about 1 year ago

[DeepSeek] Improve performance of DS MLA cache kernel (#26132)

Matthew Bonanni committed about 1 year ago

[CI/Build] Conditionally register cutlass_fp4_group_mm to fix building on Hopper (#26138)

Michael Goin committed about 1 year ago

[Attention] Move Backend enum into registry (#25893)

Matthew Bonanni committed about 1 year ago

[Bug][Benchmark] Fix duplicate req in oversampling (#26140)

Ekagra Ranjan committed about 1 year ago

[Log] Optimize DeepGEMM Missing Log (#26106)

Wentao Ye committed about 1 year ago

[Bugfix] Disable cascade attention with FlashInfer (#26130)

Michael Goin committed about 1 year ago

Fix MTP with deepep_low_latency (#25904)

Matthew Bonanni committed about 1 year ago

[Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696)

ElizaWszola committed about 1 year ago