[bugfix] do not add extra linebreak for score/rerank with chat template (#38617)

Augusto Yao committed 7 months ago

[Bugfix][MLA] Add logits size budget to sparse indexer prefill chunking (#36178)

Lucas Wilkinson committed 7 months ago

[1/N][Cleanup] Standardize on use of `is_quantized_kv_cache` (#38659)

Matthew Bonanni committed 7 months ago

[Perf] Optimize mean pooling using chunks and index_add, 5.9% E2E throughput improvement (#38559)

Wentao Ye committed 7 months ago

Fix priority preemption regression test in scheduler (#37051)

HarshRathva committed 7 months ago

[ROCm][perf] Shuffle KV cache to use paged_attention_common (#32914)

Samu Tamminen committed 7 months ago

[Misc] Add 20 regression tests for 11 tool parser bug fixes (#38172)

Ben Browning committed 7 months ago

[vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825)

Luka Govedič committed 7 months ago

Fix NaN from stale FP4 scale padding in create_fp4_scale_tensor (#38148)

Elvir Crnčević committed 7 months ago

[Feat][v1] Simple yet General CPU KV Cache Offloading (#37160)

Yifan Qiao committed 7 months ago

[ROCm][perf] fix Aiter sparse MLA with MTP>1 (#37887)

Stig-Arne Grönroos committed 7 months ago

Generative Scoring (#34539)

Vedant V Jhaveri committed 7 months ago

feat(grpc): add periodic stats logging and servicer log forwarding (#38333)

Chang Su committed 7 months ago

[Quantization] Consolidate dummy format logic into DummyModelLoader (#38637)

Asaf Gardin committed 7 months ago

[Kernel] [Helion] [17/N] Add Helion kernel torch.compile support (#38592)

Yanan Cao committed 7 months ago

[Refactor] Remove dead code in kv connector and model runner (#38383)

Wentao Ye committed 7 months ago

[Perf] Fix DBO overlap: capture DeepEP event before yield (#38451)

czhu-cohere committed 7 months ago

[MoE Refactor] Migrate Unquantized to Full Oracle Flow (#36286)

yzong-rh committed 7 months ago

[fix] Remove trtllm ragged mla prefills (#36540)

Olya Kozlova committed 7 months ago

[Model] Sync upstream BT=chunk_size fix for GDN chunk_fwd_kernel_o, simplify warmup to single pass (#38343)

Xu Jinyang committed 7 months ago