[Bugfix] Disable TRTLLM FP8 MoE if router_logits_dtype==float32 and routing_method!=DeepSeekV3 (#33613)

Michael Goin committed 9 months ago

[Voxtral Realtime] Change name (#33716)

Patrick von Platen committed 9 months ago

[MISC] Fix Tensor Parallelism for Quantized Mamba Models with n_groups=1 (#33257)

Vadim Gimpelson committed 9 months ago

Turn `@config` into a `dataclass_transform` (#31541)

Harry Mellor committed 9 months ago

[torch.compile] Significantly speed up cold start times (#33641)

Richard Zou committed 9 months ago

[Attention][FA3] Update FA3 to include new swizzle optimization (#23465)

Lucas Wilkinson committed 9 months ago

[P/D] rework mooncake connector and introduce its bootstrap server (#31034)

dtc committed 9 months ago

[Bugfix] Fix startup hang for Granite Speech (#33699)

Cyrus Leung committed 9 months ago

[Voxtral models] Skip warm-up to skip confusing error message in warm-up (#33576)

Patrick von Platen committed 9 months ago

[MM] Pass `prefix` parameter to MMEncoderAttention (#33674)

Shanshan Shen committed 9 months ago

[Bugfix] Do not add extra \n for image-only cases when constructing multimodal text prompts. (#33647)

wang.yuqi committed 9 months ago

Feat/add nemotron nano v3 tests (#33345)

shaharmor98 committed 9 months ago

[Bugfix][Async][Connector] avoid vllm-side double free during async scheduling + request abort + async KV cache transfer (#33377)

Kuntai Du committed 9 months ago

Document NixlConnector backend selection via kv_connector_extra_config (#33552)

Krish Gupta committed 9 months ago

Fix Gemma3n audio encoder for Transformers v5 (#33673)

Harry Mellor committed 9 months ago

[Models] Intern-S1-Pro (#33636)

zxy committed 9 months ago

Fix Gemma3 GGUF for Transformers v5 (#33683)

Harry Mellor committed 9 months ago

Fix offline test for Transformers v5 (#33682)

Harry Mellor committed 9 months ago

[Bugfix] fix qwen3-asr response error (#33644)

Song Zhixin committed 9 months ago

[Misc] Update default image format of `encode_base64` (#33656)

Cyrus Leung committed 9 months ago