A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
Raise error for long prompt (#273)
Lily Liu committed over 3 years ago
[Fix] Weight loading for GPTBigCode (#313)
Zhuohan Li committed over 3 years ago
[Fix] Do not pin memory when in WSL (#312)
Zhuohan Li committed over 3 years ago
Add news for the vllm+skypilot example (#314)
Zhanghao Wu committed over 3 years ago
Bayang committed over 3 years ago
[Tokenizer] Add tokenizer mode (#298)
Woosuk Kwon committed over 3 years ago
remove floats == 0 comparison (#285)
Lily Liu committed over 3 years ago
[Tokenizer] Add an option to specify tokenizer (#284)
Woosuk Kwon committed over 3 years ago
Jishnu Ray Chowdhury committed over 3 years ago
Cody Yu committed over 3 years ago
expand coverage of gpt2 model loading (#271)
twaka committed over 3 years ago
[Fix] Fix default port number in benchmark scripts (#265)
Zhuohan Li committed over 3 years ago
[BugFix] Fix a bug in counting running sequences (#266)
Woosuk Kwon committed over 3 years ago
[Doc] Documentation for distributed inference (#261)
Zhuohan Li committed over 3 years ago
[Bug] Fix the OOM condition for CPU cache (#260)
Zhuohan Li committed over 3 years ago
Compatible with Decapoda Research llama hf version (#251)
BasicCoder committed over 3 years ago
Lianmin Zheng committed over 3 years ago
fix wrong using getattr to get dict value (#232)
metacryptom committed over 3 years ago
[Docs] Add GPTBigCode to supported models (#213)
Woosuk Kwon committed over 3 years ago
GPTBigCode (StarCoder, SantaCoder Support) (#209)
Michael Feil committed over 3 years ago