tuned fused configs for B300 (#30629)

navmarri14 committed 11 months ago

[Perf] enable flashinfer rotary_embedding custom ops in DeepSeek rotary (#30729)

jiahanc committed 11 months ago

[BugFix]Reclaim resources to prevent memory leaks when use LMCacheMPConnector (#30745)

wz1qqx committed 11 months ago

[XPU] allow custom workers (e.g. vllm-omni workers) to be used on XPU (#30935)

Fanli Lin committed 11 months ago

[MM Encoder]: Migrate legacy ViT `MultiHeadAttention` to new `MMEncoderAttention` interface (#30684)

Isotr0py committed 11 months ago

[BugFix] Add sleep to fix tight loop and release GIL (#29476)

Alec committed 11 months ago

[Bugfix] fix DP-aware routing in OpenAI API requests (#29002)

inkcherry committed 11 months ago

[Cleanup] Remove unused ModelRunner V1 `InputBatch.num_tokens` field (#30218)

Nick Hill committed 11 months ago

fix fp8 online quantization streaming with tp > 1 (#30900)

Vasiliy Kuznetsov committed 11 months ago

[LoRA] Set default MXFP4 LoRA backend to Marlin (#30598)

Xin Yang committed 11 months ago

[Doc] Add Sophgo TPU Support (#30949)

wzyrrr committed 11 months ago

[Bugfix][torch2.10] Fix test_qwen2_5_vl_compilation with 2.10 RC (#30822)

Lucas Kabela committed 11 months ago

adds jais 2 support (#30188)

sarathc-cerebras committed 11 months ago

[Bug] Fix batch invariant in torch 2.10 (#30907)

Wentao Ye committed 11 months ago

[Misc] support nsys profile for bench latency (#29776)

zhrrr committed 11 months ago

Filter safetensors files to download if .safetensors.index.json exists (#30537)

Michael Goin committed 11 months ago

[ROCm] Serving Fails on Radeon Due to AITER Dtype Import (#30952)

vllmellm committed 11 months ago

[Bugfix][CPU] Fix Mac CPU build (#30955)

Li, Jiang committed 11 months ago

[moe] Use enable_chunking func (to support disabling chunking) (#29935)

Ming Yang committed 11 months ago

[ROCm][Bugfix] Fix `fa_version` argument error in `flash_attn_maxseqlen_wrapper` for ROCm without aiter (#30909)

Andreas Karatzas committed 11 months ago