A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[Bugfix] fix lora_dtype value type in arg_utils.py (#5398)
Ali Panahi committed over 2 years ago
[Bugfix] Fix `MultiprocessingGPUExecutor.check_health` when world_size == 1 (#5254)
Junichi Sato committed over 2 years ago
[Docs] [Spec decode] Fix docs error in code example (#5427)
Cade Daniel committed over 2 years ago
[Doc] Add an automatic prefix caching section in vllm documentation (#5324)
Kuntai Du committed over 2 years ago
[Speculative decoding] Initial spec decode docs (#5400)
Cade Daniel committed over 2 years ago
SangBin Cho committed over 2 years ago
[Misc] Remove VLLM_BUILD_WITH_NEURON env variable (#5389)
Woosuk Kwon committed over 2 years ago
[Doc][Typo] Fixing Missing Comma (#5403)
Roger Wang committed over 2 years ago
[Doc] add debugging tips (#5409)
youkaichao committed over 2 years ago
[Bugfix][Frontend] Cleanup "fix chat logprobs" (#5026)
Cyrus Leung committed over 2 years ago
[Bugfix] OpenAI entrypoint limits logprobs while ignoring server defined --max-logprobs (#5312)
maor-ps committed over 2 years ago
[Misc] Various simplifications and typing fixes (#5368)
Nick Hill committed over 2 years ago
[ci] Fix Buildkite agent path (#5392)
Kevin H. Luu committed over 2 years ago
[Doc] Add documentation for FP8 W8A8 (#5388)
Michael Goin committed over 2 years ago
Bump version to v0.5.0 (#5384)
Simon Mo committed over 2 years ago
[Docs] Alphabetically sort sponsors (#5386)
Woosuk Kwon committed over 2 years ago
[Docs] Add Docs on Limitations of VLM Support (#5383)
Roger Wang committed over 2 years ago
[ci] Mount buildkite agent on Docker container to upload benchmark results (#5330)
Kevin H. Luu committed over 2 years ago
[ci] Use small_cpu_queue for doc build (#5331)
Kevin H. Luu committed over 2 years ago
[Bugfix] Fix LLaVA-NeXT (#5380)
Cyrus Leung committed over 2 years ago