A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
add support for tokenizer revision (#1163)
Federico Cassano committed about 3 years ago
TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181)
Zhuohan Li committed about 3 years ago
[Minor] Fix type annotations (#1238)
Woosuk Kwon committed about 3 years ago
support sharding llama2-70b on more than 8 GPUs (#1209)
Zhuohan Li committed about 3 years ago
Fix boundary check in paged attention kernel (#1241)
Liang committed about 3 years ago
Added `dtype` arg to benchmarks (#1228)
kg6-sleipnir committed about 3 years ago
fixing typo in `tiiuae/falcon-rw-7b` model name (#1226)
Usama Ahmed committed about 3 years ago
Bump up the version to v0.2.0 (#1212)
Woosuk Kwon committed about 3 years ago
Provide default max model length (#1224)
Woosuk Kwon committed about 3 years ago
Fix OOM in attention kernel test (#1223)
Woosuk Kwon committed about 3 years ago
Add Mistral to supported model list (#1221)
Woosuk Kwon committed about 3 years ago
[Fix] Remove false assertion (#1222)
Woosuk Kwon committed about 3 years ago
Woosuk Kwon committed about 3 years ago
[Mistral] Mistral-7B-v0.1 support (#1196)
Chris Bamford committed about 3 years ago
Add rope_scaling to Qwen (#1210)
Qing committed about 3 years ago
Add `skip_special_tokens` sampling params (#1186)
Dan Lord committed about 3 years ago
Use standard extras for uvicorn (#1166)
Danilo Peixoto committed about 3 years ago
Automatically configure `max_num_batched_tokens` (#1198)
Woosuk Kwon committed about 3 years ago
Qing committed about 3 years ago
Wang Ran (汪然) committed about 3 years ago