A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
wenjun93 committed about 3 years ago
Fix for breaking changes in xformers 0.0.21 (#834)
Woosuk Kwon committed about 3 years ago
Fix typo of Aquila in README.md (#836)
ldwang committed about 3 years ago
Implement approximate GELU kernels (#828)
Woosuk Kwon committed about 3 years ago
Add compute capability 8.9 to default targets (#829)
Woosuk Kwon committed about 3 years ago
fix: revert code to avoid no attribute problem (#827)
Wen Sun committed about 3 years ago
Update Supported Model List (#825)
Zhuohan Li committed about 3 years ago
shunxing1234 committed about 3 years ago
Fix mqa is false case in gpt_bigcode (#806)
zhaoyang-star committed about 3 years ago
set default coompute capability according to cuda version (#773)
Xudong Zhang committed about 3 years ago
[Feature | CI] Added a github action to build wheels (#746)
Daniel committed about 3 years ago
Fix typo in sampling_params.py (#788)
wangcx18 committed over 3 years ago
Xinyu Yang committed over 3 years ago
Align with huggingface Top K sampling (#753)
Abraham-Xu committed over 3 years ago
Fix typo in tokenizer.py (#750)
Ikko Eltociear Ashimine committed over 3 years ago
Uranus committed over 3 years ago
Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715)
WanMok committed over 3 years ago
[Fix] unwantted bias in InternLM Model (#740)
WRH committed over 3 years ago
Check the max prompt length for the OpenAI completions API (#472)
Nicolas Basile committed over 3 years ago
Jia Guoqing committed over 3 years ago