A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
main
[Minor] Fix a typo in .pt weight support (#2160)
Woosuk Kwon committed almost 3 years ago
Make sampler less blocking (#1889)
Antoni Baum committed almost 3 years ago
[Minor] Add Phi 2 to supported models (#2159)
Woosuk Kwon committed almost 3 years ago
[Minor] Fix xformers version (#2158)
Woosuk Kwon committed almost 3 years ago
[Minor] Add more detailed explanation on `quantization` argument (#2145)
Woosuk Kwon committed almost 3 years ago
Temporarily enforce eager mode for GPTQ models (#2154)
Woosuk Kwon committed almost 3 years ago
[Docs] Add CUDA graph support to docs (#2148)
Woosuk Kwon committed almost 3 years ago
Remove dependency on CuPy (#2152)
Woosuk Kwon committed almost 3 years ago
Pin PyTorch & xformers versions (#2155)
Woosuk Kwon committed almost 3 years ago
Fix all-reduce memory usage (#2151)
Woosuk Kwon committed almost 3 years ago
[Minor] Delete Llama tokenizer warnings (#2146)
Woosuk Kwon committed almost 3 years ago
Optimize model execution with CUDA graph (#1926)
Woosuk Kwon committed almost 3 years ago
Simplify weight loading logic (#2133)
Roy committed almost 3 years ago
[ROCm] Temporarily remove GPTQ ROCm support (#2138)
Woosuk Kwon committed almost 3 years ago
[Docs] Add supported quantization methods to docs (#2135)
Woosuk Kwon committed almost 3 years ago
CHU Tianxiang committed almost 3 years ago
Add a flag to include stop string in output text (#1976)
Yunfeng Bai committed almost 3 years ago
Avoid multiple redefinition (#1817)
Mingcan Xiang committed almost 3 years ago
TJian committed almost 3 years ago
Fix typing in AsyncLLMEngine & add toml to requirements-dev (#2100)
mezuzza committed almost 3 years ago