A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI]Add transformers_utils to Async Engine, Inputs, Utils, Worker Test (#24615)
Xingyu Liu committed about 1 year ago
[Docs] Use 1-2-3 list for deploy steps in deployment/frameworks/ (#24633)
Michael Yao committed about 1 year ago
[Docs] Update V1 doc to reflect whisper support (#24606)
Russell Bryant committed about 1 year ago
[Docs] Revise frameworks/anything-llm.md (#24489)
Michael Yao committed about 1 year ago
[Bugfix] Fix incorrect import of CacheConfig (#24631)
Cyrus Leung committed about 1 year ago
Add the support for the qwen3 next model (a hybrid attention model). (#24526)
Tao He committed about 1 year ago
[torchao] Support quantization configs using module swap (#21982)
Jerry Zhang committed about 1 year ago
Remove redundant all gather + split (#23441)
Chenxi Yang committed about 1 year ago
[Model] New model support for Motif-1-Tiny (#23414)
TaehyunKim committed about 1 year ago
[Doc]: fixing doc typos (#24635)
Didier Durand committed about 1 year ago
shengshiqi-google committed about 1 year ago
Enable --profile in 'vllm bench throughput' (#24575)
Tomas Ruiz committed about 1 year ago
Kimi K2 Fused MoE kernels Optimization configs (#24597)
Saman A. Pour committed about 1 year ago
[Engine][Chore] use local variable and remove output var assignment (#24554)
Guy Stone committed about 1 year ago
[BugFix] Fix pipeline parallel (#24621)
Nick Hill committed about 1 year ago
[Bugfix] Add missing VIT backend dispatch on CPU (#24623)
Li, Jiang committed about 1 year ago
[Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131)
Peter Salas committed about 1 year ago
[Bug] [Spec Decode] Fix model_initialization test and mismatch in aux_hidden_layers (#24613)
Wenlong Wang committed about 1 year ago
Add @chaunceyjiang to codeowner for reasoning Reasoning and Tool parser (#24406)
Chauncey committed about 1 year ago
[distributed] update known issues (#24624)
youkaichao committed about 1 year ago