A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][NIXL] Fix DCP pulls across MLA cache regions (#57389)
Lucas Wilkinson committed about 1 month ago
[docs] Fix legacy hf CLI references (vllm) (#57958)
Lucain committed about 1 month ago
[Bugfix] Fix external LB DP rank handling when replicas share nodes (#53743)
Tony Lin committed about 1 month ago
[ROCm][DSv4] Enable DSpark adaptive verification (#52362)
Tuukka Sarvi committed about 1 month ago
[Bugfix][Frontend] Accept diarized transcription responses in run-batch (#57948)
sfigs committed about 1 month ago
[Pooling] Fix normalization of chunked long-text embeddings (#57498)
Taneem Ibrahim committed about 1 month ago
[MM] Move `supports_multimodal_inputs` and cache out of registry (#57913)
Cyrus Leung committed about 1 month ago
[ROCm][Tests] Reduce host memory when downcasting FP32 HF references (#57869)
Andreas Karatzas committed about 1 month ago
[Frontend] Add streaming parity tests and docs for derender (#57922)
Martin Hickey committed about 1 month ago
[Bugfix][Multimodal] Tolerate malformed EXIF in Molmo 2 image preprocessing (#57234)
Hotragn Pettugani committed about 1 month ago
[CI][XPU] Deselect Ray UT in XPU V1 test Job (#57889)
Chaojun Zhang committed about 1 month ago
[Security] Prefer fresh multimodal payloads over a stale receiver cache (#57833)
Juan Pérez de Algaba committed about 1 month ago
[CI][ROCm] Add ten AMD parity groups (#57876)
Andreas Karatzas committed about 1 month ago
[Test] Organize structured output utility tests (#57545)
CaiJohn committed about 1 month ago
[Bugfix][ROCm][DSv4.1] Disable SWA bounded replay on ROCm (#57906)
Fangzhou Ai committed about 1 month ago
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Hongxin Xu committed about 1 month ago
[CI][ROCm] Increase timeout for MI300 Distributed DP Extended (#57903)
djramic committed about 1 month ago
[CI][ROCm] Mirror remaining portable suites without duplicate coverage (#57877)
Andreas Karatzas committed about 1 month ago
[AMD][Minimax-M3][perf] Enable packed LBHNC AITER QK-norm fusion for MiniMax-M3 on ROCm (#54535)
weitliao committed about 1 month ago
[CI][ROCm] Make Python-only installation failures blocking (#57870)
Andreas Karatzas committed about 1 month ago