A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
main
[V1] Override `mm_counts` for dummy data creation (#15703)
Cyrus Leung committed over 1 year ago
pansicheng committed over 1 year ago
[doc] update conda to usage link in installation (#15761)
Reid committed over 1 year ago
[Bugfix] Fix Mistral guided generation using xgrammar (#15704)
Julien Denize committed over 1 year ago
[CI] xgrammar structured output supports Enum. (#15757)
Chauncey committed over 1 year ago
[Bugfix] Fix Mllama interleaved images input support (#15564)
Isotr0py committed over 1 year ago
[V1][Minor] Simplify rejection sampler's parse_output (#15741)
Woosuk Kwon committed over 1 year ago
[V1] Support interleaved modality items (#15605)
Roger Wang committed over 1 year ago
[Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore (#12957)
shangmingc committed over 1 year ago
[V1] [Feature] Collective RPC (#15444)
wwl2755 committed over 1 year ago
[FEAT] [ROCm] Add AITER int8 scaled gemm kernel (#15433)
TJian committed over 1 year ago
Reid committed over 1 year ago
[Kernel][TPU][ragged-paged-attn] vLLM code change for PR#8896 (#15659)
yarongmu-google committed over 1 year ago
[TPU][V1][Bugfix] Fix w8a8 recompiilation with GSM8K (#15714)
Nicolò Lucchesi committed over 1 year ago
[Bugfix] set VLLM_WORKER_MULTIPROC_METHOD=spawn for vllm.entrypoionts.openai.api_server (#15700)
Jinzhen Lin committed over 1 year ago
Use numba 0.61 for python 3.10+ to support numpy>=2 (#15692)
cyyever committed over 1 year ago
[CI] Speed up V1 structured output tests (#15718)
Russell Bryant committed over 1 year ago
[Bugfix] LoRA V1: add and fix entrypoints tests (#15715)
Varun Sundar Rabindranath committed over 1 year ago
[Misc][V1] Misc code streamlining (#15723)
Nick Hill committed over 1 year ago
[Docs] Document v0 engine support in reasoning outputs (#15739)
Ce Gao committed over 1 year ago