A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
[Model] Add support for GPT-J (#226)
Andre Slavescu committed over 3 years ago
Don't try to load training_args.bin (#373)
Fazlul Shahriar committed over 3 years ago
Add trust_remote_code arg to get_config (#405)
Woosuk Kwon committed over 3 years ago
Sort the outputs before return (#402)
Woosuk Kwon committed over 3 years ago
avoid python list copy in sequence initialization (#401)
Lily Liu committed over 3 years ago
Add trust-remote-code flag to handle remote tokenizers (#364)
codethazine committed over 3 years ago
[Fix] Change /generate response-type to json for non-streaming (#374)
Nicolas Frenay committed over 3 years ago
fix: only response [DONE] once when streaming response. (#378)
Ricardo Lu committed over 3 years ago
Woosuk Kwon committed over 3 years ago
akxxsb committed over 3 years ago
[Server] Add option to specify chat template for chat endpoint (#345)
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
[Model] Add support for MPT (#334)
Woosuk Kwon committed over 3 years ago
Fix an endless loop issue when engine_step throws a RuntimeError (#339)
coolcloudcol committed over 3 years ago
[Quality] Add CI for formatting (#343)
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
[Quality] Add code formatter and linter (#326)
Zhuohan Li committed over 3 years ago
[Fix] Better error message for batched prompts (#342)
Zhuohan Li committed over 3 years ago
Allow send list of str for the Prompt on openai demo endpoint /v1/completions (#323)
Michele Catalano committed over 3 years ago
feat: add ChatCompletion endpoint in OpenAI demo server. (#330)
Ricardo Lu committed over 3 years ago