[Voxtral] Add new streaming arch (#32861)

Patrick von Platen committed 10 months ago

[CI/Build][CPU] Fix failed pooling tests and macos smoke test (#32907)

Li, Jiang committed 10 months ago

[Misc] Add `get_name` to missing AttentionBackends (#32698)

Nicolò Lucchesi committed 10 months ago

[CI][Models] Add VLM Support for Sequence Classification Conversion (#32885)

Andreas Karatzas committed 10 months ago

[Bugfix] Fix _CPU_MOE_ACT AssertionError when vLLM config not set (#32777)

Karan Bansal committed 10 months ago

[CI] Fix mypy for `vllm/v1/structured_output` (#32722)

Wentao Ye committed 10 months ago

[torch.compile] Compile `CustomOp.forward_native` for `SiluAndMul` and `QuantFP8` to avoid raw torch ops inside opaque custom ops (#32806)

Luka Govedič committed 10 months ago

[BugFix] deepseek_v32_encoding: Replace asserts with proper exceptions (#32884)

Rishabh Saini committed 10 months ago

[Misc] Log vLLM logo when starting server (#32796)

Nick Hill committed 10 months ago

[MoE Refactor] Move `select_experts` from `FusedMoEQuantMethod` -> `FusedMoE` (#31996)

bnellnm committed 10 months ago

[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration (#32855)

Fadi Arafeh committed 10 months ago

[Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper (#32619)

Xin Yang committed 10 months ago

[Refactor] Remove unused tpu files (#32610)

Wentao Ye committed 10 months ago

Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141)

Eldar Kurtić committed 10 months ago

[Bugfix][Attention] Explicitly report support for kv_cache_dtype bfloat16 (#32795)

Matthew Bonanni committed 10 months ago

[CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across NUMA domains on Arm (#32792)

Fadi Arafeh committed 10 months ago

[CI][Attention] Add more CI dependencies for attention tests (#32487)

Matthew Bonanni committed 10 months ago

[Feature] Add --ssl-ciphers CLI argument for TLS cipher control (#30937)

RickyChen / 陳昭儒 committed 10 months ago

Support custom URI schemes and trace handlers for profiler (#32393)

David Ramon Prados committed 10 months ago

[UX] Default api_server_count to dp_size if not specified (#32525)

Tyler Michael Smith committed 10 months ago