@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 14f9c72bfd
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

Update Supported Model List (#825)

Zhuohan Li committed Aug 22, 2023 at 11:51 UTC 14f9c72bfdd6c90cbcb52bcfa34f33f56cbe323f
2 files changed +12
README.md
+3
@@ -42,6 +42,7 @@ vLLM is flexible and easy to use with:
42
43 vLLM seamlessly supports many Huggingface models, including the following architectures:
44
45 +- Aqualia (`BAAI/Aquila-7B`, `BAAI/AquilaChat-7B`, etc.)
46 - Baichuan (`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.)
47 - BLOOM (`bigscience/bloom`, `bigscience/bloomz`, etc.)
48 - Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
@@ -49,9 +50,11 @@ vLLM seamlessly supports many Huggingface models, including the following archit
50 - GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
51 - GPT-J (`EleutherAI/gpt-j-6b`, `nomic-ai/gpt4all-j`, etc.)
52 - GPT-NeoX (`EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
53 +- InternLM (`internlm/internlm-7b`, `internlm/internlm-chat-7b`, etc.)
54 - LLaMA & LLaMA-2 (`meta-llama/Llama-2-70b-hf`, `lmsys/vicuna-13b-v1.3`, `young-geng/koala`, `openlm-research/open_llama_13b`, etc.)
55 - MPT (`mosaicml/mpt-7b`, `mosaicml/mpt-30b`, etc.)
56 - OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
57 +- Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
58
59 Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/getting_started/installation.html#build-from-source):
60
docs/source/models/supported_models.rst
+9
@@ -14,6 +14,9 @@ Alongside each architecture, we include some popular models that use it.
14 * - Architecture
15 - Models
16 - Example HuggingFace Models
17 + * - :code:`AquilaForCausalLM`
18 + - Aqualia
19 + - :code:`BAAI/Aquila-7B`, :code:`BAAI/AquilaChat-7B`, etc.
20 * - :code:`BaiChuanForCausalLM`
21 - Baichuan
22 - :code:`baichuan-inc/Baichuan-7B`, :code:`baichuan-inc/Baichuan-13B-Chat`, etc.
@@ -35,6 +38,9 @@ Alongside each architecture, we include some popular models that use it.
38 * - :code:`GPTNeoXForCausalLM`
39 - GPT-NeoX, Pythia, OpenAssistant, Dolly V2, StableLM
40 - :code:`EleutherAI/gpt-neox-20b`, :code:`EleutherAI/pythia-12b`, :code:`OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5`, :code:`databricks/dolly-v2-12b`, :code:`stabilityai/stablelm-tuned-alpha-7b`, etc.
41 + * - :code:`InternLMForCausalLM`
42 + - InternLM
43 + - :code:`internlm/internlm-7b`, :code:`internlm/internlm-chat-7b`, etc.
44 * - :code:`LlamaForCausalLM`
45 - LLaMA, LLaMA-2, Vicuna, Alpaca, Koala, Guanaco
46 - :code:`meta-llama/Llama-2-13b-hf`, :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`young-geng/koala`, :code:`JosephusCheung/Guanaco`, etc.
@@ -44,6 +50,9 @@ Alongside each architecture, we include some popular models that use it.
50 * - :code:`OPTForCausalLM`
51 - OPT, OPT-IML
52 - :code:`facebook/opt-66b`, :code:`facebook/opt-iml-max-30b`, etc.
53 + * - :code:`OPTForCausalLM`
54 + - Qwen
55 + - :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
56
57 If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
58 Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.