A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[Bugfix] Fix Phi-3v crash when input images are of certain sizes (#7840)
zifeitong committed about 2 years ago
[ci][test] fix RemoteOpenAIServer (#7838)
youkaichao committed about 2 years ago
[ci][test] exclude model download time in server start time (#7834)
youkaichao committed about 2 years ago
[misc][core] lazy import outlines (#7831)
youkaichao committed about 2 years ago
[Frontend] add json_schema support from OpenAI protocol (#7654)
Tyler Rockwood committed about 2 years ago
[Frontend] Publish Prometheus metrics in run_batch API (#7641)
Pooya Davoodi committed about 2 years ago
[Bugfix] Fix run_batch logger (#7640)
Pooya Davoodi committed about 2 years ago
[Core] Add multi-step support to LLMEngine (#7789)
Alexander Matveev committed about 2 years ago
Bump version to v0.5.5 (#7823)
Simon Mo committed about 2 years ago
[Misc] Update `marlin` to use vLLMParameters (#7803)
Dipika Sikka committed about 2 years ago
[github][misc] promote asking llm first (#7809)
youkaichao committed about 2 years ago
[BugFix] Fix server crash on empty prompt (#7746)
Maximilien de Bayser committed about 2 years ago
[misc] Add Torch profiler support for CPU-only devices (#7806)
Jie Fu (傅杰) committed about 2 years ago
[Hardware][Intel GPU] refactor xpu_model_runner for tp (#7712)
Kunshang Ji committed about 2 years ago
[Ray backend] Better error when pg topology is bad. (#7584)
SangBin Cho committed about 2 years ago
[Frontend] error suppression cleanup (#7786)
Joe Runde committed about 2 years ago
[Misc] fix typo in triton import warning (#7794)
Siyuan Liu committed about 2 years ago
Kevin H. Luu committed about 2 years ago
[Bugfix] Use LoadFormat values for `vllm serve --load-format` (#7784)
Michael Goin committed about 2 years ago
[Doc] Fix incorrect docs from #7615 (#7788)
Peter Salas committed about 2 years ago