A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
main
Update Ray version requirements (#2636)
Simon Mo committed almost 3 years ago
Small async_llm_engine refactor (#2618)
Murali Andoorveedu committed almost 3 years ago
Speed up Punica compilation (#2632)
Woosuk Kwon committed almost 3 years ago
[Minor] Fix warning on Ray dependencies (#2630)
Woosuk Kwon committed almost 3 years ago
Implement custom all reduce kernels (#2192)
Hanzhi Zhou committed almost 3 years ago
Use head_dim in config if exists (#2622)
Xiang Xu committed almost 3 years ago
AWQ: Up to 2.66x higher throughput (#2566)
Casper committed almost 3 years ago
Don't build punica kernels by default (#2605)
Philipp Moritz committed almost 3 years ago
Support for Stable LM 2 (#2598)
dakotamahan-stability committed almost 3 years ago
[ROCm] add support to ROCm 6.0 and MI300 (#2274)
Hongxia Yang committed almost 3 years ago
use a correct device when creating OptionalCUDAGuard (#2583)
Vladimir committed almost 3 years ago
fix names and license for Qwen2 (#2589)
Junyang Lin committed almost 3 years ago
Support Batch Completion in Server (#2529)
Simon Mo committed almost 3 years ago
Fix the syntax error in the doc of supported_models (#2584)
LastWhisper committed almost 3 years ago
Added `include_stop_str_in_output` and `length_penalty` parameters to OpenAI API (#2562)
Federico Galatolo committed almost 3 years ago
[Bugfix] fix crash if max_tokens=None (#2570)
Nikola Borisov committed almost 3 years ago
lint: format all python file instead of just source code (#2567)
Simon Mo committed almost 3 years ago
[Experimental] Add multi-LoRA support (#1804)
Antoni Baum committed almost 3 years ago
[Feature] Simple API token authentication and pluggable middlewares (#1106)
Erfan Al-Hossami committed almost 3 years ago
Jason Zhu committed almost 3 years ago