A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
Bump up version to 0.1.1 (#204)
Zhuohan Li committed over 3 years ago
[Fix] Better error message when there is OOM during cache initialization (#203)
Zhuohan Li committed over 3 years ago
[Bugfix] Fix a bug in RequestOutput.finished (#202)
Woosuk Kwon committed over 3 years ago
Zhuohan Li committed over 3 years ago
Use slow tokenizer for open llama models (#168)
Woosuk Kwon committed over 3 years ago
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
[Minor] Fix icons in doc (#165)
Woosuk Kwon committed over 3 years ago
Zhuohan Li committed over 3 years ago
Fix repo & documentation URLs (#163)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Add and list supported models in README (#161)
Zhuohan Li committed over 3 years ago
[PyPI] Fix package info in setup.py (#158)
Woosuk Kwon committed over 3 years ago
Add logo and polish readme (#156)
Zhuohan Li committed over 3 years ago
Add performance figures for dark mode (#160)
Woosuk Kwon committed over 3 years ago
Remove benchmark_async_llm_server.py (#155)
Zhuohan Li committed over 3 years ago
[Minor] Fix CompletionOutput.__repr__ (#157)
Woosuk Kwon committed over 3 years ago
Add comments on swap space (#154)
Woosuk Kwon committed over 3 years ago
Write README and front page of doc (#147)
Woosuk Kwon committed over 3 years ago
Reduce GPU memory utilization to make sure OOM doesn't happen (#153)
Zhuohan Li committed over 3 years ago