A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
Make vllm compatible with verl (#12824)
ZSL98 committed almost 2 years ago
[Bugfix] FA2 illegal memory access (#12848)
Lucas Wilkinson committed almost 2 years ago
[misc] Revert # 12833 (#12857)
Kevin H. Luu committed almost 2 years ago
[MISC] Check space in the file names in the pre commit checks (#12804)
Lu Fang committed almost 2 years ago
Yu Chin Fabian Lim committed almost 2 years ago
Varun Sundar Rabindranath committed almost 2 years ago
[Bugfix] Fix unsupported FA version check for Turing GPU (#12828)
Isotr0py committed almost 2 years ago
[Doc] double quote cmake package in build.inc.md (#12840)
Jitse Klomp committed almost 2 years ago
[torch.compile] PyTorch 2.6 and nightly compatibility (#12393)
youkaichao committed almost 2 years ago
[Kernel] Make rotary_embedding ops more flexible with input shape (#12777)
Isotr0py committed almost 2 years ago
[Misc] Remove unnecessary decode call (#12833)
Cyrus Leung committed almost 2 years ago
[misc] Reduce number of config file requests to HuggingFace (#12797)
Kevin H. Luu committed almost 2 years ago
[Attention] Use FA3 for MLA on Hopper (#12807)
Lucas Wilkinson committed almost 2 years ago
[Docs] Add Google Cloud Slides (#12814)
Simon Mo committed almost 2 years ago
[Misc] Update w2 scale loading for GPTQMarlinMoE (#12757)
Dipika Sikka committed almost 2 years ago
[Misc] Improve error message for incorrect pynvml (#12809)
youkaichao committed almost 2 years ago
[Misc] Remove duplicated DeepSeek V2/V3 model definition (#12793)
Michael Goin committed almost 2 years ago
Improve `TransformersModel` UX (#12785)
Harry Mellor committed almost 2 years ago
[Bugfix] Fix the test_ultravox.py's license (#12806)
Lu Fang committed almost 2 years ago
[Model] LoRA Support for Ultravox model (#11253)
Sumit Vij committed almost 2 years ago