[v1] Support multiple KV cache groups in GPU model runner (#17945)

Chen Zhang committed over 1 year ago

[Bugfix]: make most of `test_openai_schema.py` pass (#17664)

David Xia committed over 1 year ago

Upload vllm index for the rc builds (#18173)

Andrey Talman committed over 1 year ago

[V1] Support multiple kv connectors (#17564)

Michael Goin committed over 1 year ago

[CI] Fix race condition in test_kv_cache_events test (#18169)

Russell Bryant committed over 1 year ago

Add support for loading torchao models with `AOPerModuleConfig` (#17826)

Jerry Zhang committed over 1 year ago

[V1] Structured Outputs + Thinking compatibility (#16577)

Aaron Pham committed over 1 year ago

[Kernel] Have rotary embeddings support tensors (#18046)

Lucas Wilkinson committed over 1 year ago

[Frontend] decrease import time of vllm.multimodal (#18031)

David Xia committed over 1 year ago

[CI] Disable Failing Tests (#18165)

Robert Shaw committed over 1 year ago

Modularize fused experts and integrate PPLX kernels (#15956)

bnellnm committed over 1 year ago

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)

Ekagra Ranjan committed over 1 year ago

[Doc] Update prefix cache metrics to counting tokens (#18138)

Chen Zhang committed over 1 year ago

[KVConnector] Keep KVTransferParams as a dict (#18033)

Nick Hill committed over 1 year ago

[Bugfix] Fix chat utils tests (#18139)

Cyrus Leung committed over 1 year ago

Update deprecated type hinting in `platform`, `plugins`, `triton_utils`, `vllm_flash_attn` (#18129)

Harry Mellor committed over 1 year ago

Update deprecated type hinting in `vllm/device_allocator` and `vllm/distributed` (#18126)

Harry Mellor committed over 1 year ago

Update deprecated type hinting in `vllm/lora` (#18128)

Harry Mellor committed over 1 year ago

[doc] add missing import (#18133)

Reid committed over 1 year ago

[Model] GritLM supports other attention backends (#18109)

Cyrus Leung committed over 1 year ago