A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
[DOCS] Add engine args documentation (#1741)
Casper committed almost 3 years ago
Add stop_token_ids in SamplingParams.__repr__ (#1745)
陈序 committed almost 3 years ago
[FIX] Fix the case when `input_is_parallel=False` for `ScaledActivation` (#1737)
Zhuohan Li committed almost 3 years ago
[BugFix] Fix TP support for AWQ (#1731)
Woosuk Kwon committed almost 3 years ago
fix RAM OOM when load large models in tensor parallel mode. (#1395)
boydfd committed almost 3 years ago
Rewrite torch.repeat_interleave to remove cpu synchronization (#1599)
ljss committed almost 3 years ago
[Fix] Fix bugs in scheduler (#1727)
Zhuofan committed almost 3 years ago
Fix hanging in the scheduler caused by long prompts (#1534)
陈序 committed almost 3 years ago
[BugFix] Fix a bug in loading safetensors (#1732)
Woosuk Kwon committed almost 3 years ago
[FIX] Update the doc link in README.md (#1730)
Zhuohan Li committed almost 3 years ago
Migrate linter from `pylint` to `ruff` (#1665)
Simon Mo committed almost 3 years ago
[Docs] Fix the code block's format in deploying_with_docker page (#1722)
Wen Sun committed almost 3 years ago
Documentation about official docker image (#1709)
Simon Mo committed almost 3 years ago
Add instructions to install vLLM+cu118 (#1717)
Woosuk Kwon committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
Fix warning msg on quantization (#1715)
Woosuk Kwon committed almost 3 years ago
[Optimization] Implement fused add rmsnorm (#1667)
ljss committed almost 3 years ago
Add AWQ support for all models (#1714)
Woosuk Kwon committed almost 3 years ago
use get_tensor in safe_open (#1696)
twaka committed almost 3 years ago
Support download models from www.modelscope.cn (#1588)
liuyhwangyh committed almost 3 years ago