A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
Zhuohan Li committed over 3 years ago
Change the name to vLLM (#150)
Woosuk Kwon committed over 3 years ago
Rename servers to engines (#152)
Zhuohan Li committed over 3 years ago
[Minor] Fix benchmark_throughput.py (#151)
Woosuk Kwon committed over 3 years ago
Rename servers and change port numbers to reduce confusion (#149)
Zhuohan Li committed over 3 years ago
Add script for benchmarking serving throughput (#145)
Woosuk Kwon committed over 3 years ago
Remove redundant code in ColumnParallelLinear (#146)
Woosuk Kwon committed over 3 years ago
Non-streaming simple fastapi server (#144)
Zhuohan Li committed over 3 years ago
Add docstrings for LLMServer and related classes and examples (#142)
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
[PyPI] Packaging for PyPI distribution (#140)
Woosuk Kwon committed over 3 years ago
[Docs] Write the `Adding a New Model` section (#138)
Woosuk Kwon committed over 3 years ago
Fix various issues of async servers (#135)
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Document supported models (#127)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Add throughput benchmarking script (#133)
Woosuk Kwon committed over 3 years ago
Enable LLaMA fast tokenizer (#132)
Woosuk Kwon committed over 3 years ago
Add a doc for installation (#128)
Woosuk Kwon committed over 3 years ago
Improve setup script & Add a guard for bfloat16 kernels (#130)
Woosuk Kwon committed over 3 years ago