Add ORCA endpoint load metrics support (#24905)

Misha Efimov committed about 1 year ago

[XPU]Refine Dockerfile.xpu, avoid oneccl dependency issue (#27964)

Kunshang Ji committed about 1 year ago

[Bugfix] Padded Eagle Specdec with Chunked Prefill (#26263)

Rémi Delacourt committed about 1 year ago

[Hybrid] Pass kernel block size to builders (#27753)

Thomas Parnell committed about 1 year ago

[Frontend] Align finish_reason when tool is called with OpenAI (#25054)

Sungyoon Jeong committed about 1 year ago

[BugFix][LoRA] use adapter_id instead of id field of lora_request (#27728)

Biswa Panda committed about 1 year ago

Fix(llm): Abort orphaned requests when llm.chat() batch fails Fixes #26081 (#27420)

Vensen committed about 1 year ago

[Misc] Provide Siglip2 chat template (#27939)

Cyrus Leung committed about 1 year ago

[V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377)

Asaf Joseph Gardin committed about 1 year ago

Performance fix MistralTokenizer: cache special ids and tokens (#27925)

Julien Denize committed about 1 year ago

[Bugfix] Fix Qwen Omni audio inference (#27920)

Cyrus Leung committed about 1 year ago

[CI/Build] Fix flaky test_transcription_validation.py::test_basic_audio_gemma (#27924)

Ben Browning committed about 1 year ago

[KV offload] Offloading connector async scheduling support (#27648)

Yue Zhang committed about 1 year ago

Adds anthropic /v1/messages endpoint to openai api_server (#27882)

Benjamin Bartels committed about 1 year ago

[BugFix] Fix mixed penalties batch with async scheduling (#27910)

Nick Hill committed about 1 year ago

[Core][TPU] Support TPU Data Parallalism (#27365)

wenxindongwork committed about 1 year ago

[Bugfix] Python 3.10 compatibility for `Self` (#27918)

Cyrus Leung committed about 1 year ago

[Bugfix] DeepSeek V3.2 MTP metadata & CUDA graph issues (#26779)

Haco committed about 1 year ago

[V0 deprecation] Remove VLLM_USE_V1 usage in platform and v1 module (#27798)

wangxiyuan committed about 1 year ago

[Docs] Mock all imports for docs (#27873)

Harry Mellor committed about 1 year ago