A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[Bugfix] Fix lm_head weights tying with lora for llama (#9227)
Isotr0py committed about 2 years ago
sroy745 committed about 2 years ago
[Bugfix] Machete garbage results for some models (large K dim) (#9212)
Lucas Wilkinson committed about 2 years ago
[Bugfix] Fix Weight Loading Multiple GPU Test - Large Models (#9213)
Michael Goin committed about 2 years ago
[misc] improve model support check in another process (#9208)
youkaichao committed about 2 years ago
[Core] Fix invalid args to _process_request (#9201)
Russell Bryant committed about 2 years ago
[CI/Build] Update Dockerfile install+deploy image to ubuntu 22.04 (#9130)
Michael Goin committed about 2 years ago
[CI/Build] mypy: check vllm/entrypoints (#9194)
Russell Bryant committed about 2 years ago
[Hardware][CPU] Support AWQ for CPU backend (#7515)
Li, Jiang committed about 2 years ago
Add Dependabot configuration for GitHub Actions updates (#1217)
Ewout ter Hoeven committed about 2 years ago
[Bugfix] Access `get_vocab` instead of `vocab` in tool parsers (#9188)
Cyrus Leung committed about 2 years ago
[Bugfix] Fix lora loading for Compressed Tensors in #9120 (#9179)
Ahmad Fahadh Ilyas committed about 2 years ago
[Doc] Fix VLM prompt placeholder sample bug (#9170)
Jiangtao Hu committed about 2 years ago
[ci][test] use load dummy for testing (#9165)
youkaichao committed about 2 years ago
[Bugfix] fix composite weight loading and EAGLE weight loading (#9160)
Cyrus Leung committed about 2 years ago
[Frontend] Log the maximum supported concurrency (#8831)
AlpinDale committed about 2 years ago
[Model] Remap FP8 kv_scale in CommandR and DBRX (#9174)
Hui Liu committed about 2 years ago
[Bugfix][Doc] Report neuron error in output (#9159)
Joe Rowell committed about 2 years ago
[Misc] Improve validation errors around best_of and n (#9167)
Travis Johnson committed about 2 years ago
Update link to KServe deployment guide (#9173)
Yuan Tang committed about 2 years ago