A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[Build] skip renaming files for release wheels pipeline (#9671)
Simon Mo committed about 2 years ago
[Misc] Consolidate pooler config overrides (#10351)
Cyrus Leung committed about 2 years ago
[Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350)
Sky Lee committed about 2 years ago
[Model] Support Qwen2 embeddings and use tags to select model tests (#10184)
Cyrus Leung committed about 2 years ago
[Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308)
Tyler Michael Smith committed about 2 years ago
[bugfix] Fix static asymmetric quantization case (#10334)
Luka Govedič committed about 2 years ago
[Bugfix] Fix unable to load some models (#10312)
Cyrus Leung committed about 2 years ago
[Tool parsing] Improve / correct mistral tool parsing (#10333)
Patrick von Platen committed about 2 years ago
[Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583)
Zijin Xiao committed about 2 years ago
[Perf] Reduce peak memory usage of llama (#10339)
Murali Andoorveedu committed about 2 years ago
Support Roberta embedding models (#9387)
Maximilien de Bayser committed about 2 years ago
[Docs] Publish meetup slides (#10331)
Woosuk Kwon committed about 2 years ago
[CI/Build] Make shellcheck happy (#10285)
Cyrus Leung committed about 2 years ago
[CI/Build] Fix CPU CI online inference timeout (#10314)
Isotr0py committed about 2 years ago
[ci][distributed] disable hanging tests (#10317)
youkaichao committed about 2 years ago
[Model] Add BNB quantization support for Idefics3 (#10310)
B-201 committed about 2 years ago
Guillaume Calmettes committed about 2 years ago
[Frontend] Pythonic tool parser (#9859)
Mike Depinet committed about 2 years ago
[Misc] format.sh: Simplify tool_version_check (#10305)
Russell Bryant committed about 2 years ago
[misc] error early for old-style class (#10304)
youkaichao committed about 2 years ago