A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
main
[Bugfix][Kernel] allow non-power-of-2 for prefix prefill with alibi (#4573)
DefTruth committed over 2 years ago
[CI] Make mistral tests pass (#4596)
SangBin Cho committed over 2 years ago
[Core] Optimize sampler get_logprobs (#4594)
SangBin Cho committed over 2 years ago
[Core][Distributed] support cpu&device in broadcast tensor dict (#4660)
youkaichao committed over 2 years ago
[Bug fix][Core] fixup ngram not setup correctly (#4551)
leiwen83 committed over 2 years ago
[Core][Optimization] change copy-on-write from dict[int, list] to list (#4648)
youkaichao committed over 2 years ago
[Bugfix] Fixed error in slice_lora_b for MergedQKVParallelLinearWithLora (#4609)
Austin Veselka committed over 2 years ago
[Build/CI] Fixing 'docker run' to re-enable AMD CI tests. (#4642)
Alexei-V-Ivanov-AMD committed over 2 years ago
[Core][Optimization] change python dict to pytorch tensor (#4607)
youkaichao committed over 2 years ago
[Kernel] Make static FP8 scaling more robust (#4570)
Philipp Moritz committed over 2 years ago
Update lm-format-enforcer to 0.10.1 (#4631)
Noam Gat committed over 2 years ago
[CI] Add retry for agent lost (#4633)
Cade Daniel committed over 2 years ago
[CI] use ccache actions properly in release workflow (#4629)
Simon Mo committed over 2 years ago
[Bugfix] Fix `asyncio.Task` not being subscriptable (#4623)
Cyrus Leung committed over 2 years ago
Disable cuda version check in vllm-openai image (#4530)
zhaoyang-star committed over 2 years ago
[CI] Reduce wheel size by not shipping debug symbols (#4602)
Simon Mo committed over 2 years ago
bump version to v0.4.2 (#4600)
Simon Mo committed over 2 years ago
[Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937)
DearPlanet committed over 2 years ago
[CI] check size of the wheels (#4319)
Simon Mo committed over 2 years ago
Michael Goin committed over 2 years ago