@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ac1fbf7fd2
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

[Doc] Shorten README by removing supported model list (#4796)

Zhuohan Li committed May 13, 2024 at 16:23 UTC ac1fbf7fd2d1fdddc7b4953eeb3acae35c62766f
2 files changed +28 -44
README.md
+9 -38
@@ -51,41 +51,14 @@ vLLM is flexible and easy to use with:
51 - (Experimental) Prefix caching support
52 - (Experimental) Multi-lora support
53
54 -vLLM seamlessly supports many Hugging Face models, including the following architectures:
55 -
56 -- Aquila & Aquila2 (`BAAI/AquilaChat2-7B`, `BAAI/AquilaChat2-34B`, `BAAI/Aquila-7B`, `BAAI/AquilaChat-7B`, etc.)
57 -- Baichuan & Baichuan2 (`baichuan-inc/Baichuan2-13B-Chat`, `baichuan-inc/Baichuan-7B`, etc.)
58 -- BLOOM (`bigscience/bloom`, `bigscience/bloomz`, etc.)
59 -- ChatGLM (`THUDM/chatglm2-6b`, `THUDM/chatglm3-6b`, etc.)
60 -- Command-R (`CohereForAI/c4ai-command-r-v01`, etc.)
61 -- DBRX (`databricks/dbrx-base`, `databricks/dbrx-instruct` etc.)
62 -- DeciLM (`Deci/DeciLM-7B`, `Deci/DeciLM-7B-instruct`, etc.)
63 -- Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
64 -- Gemma (`google/gemma-2b`, `google/gemma-7b`, etc.)
65 -- GPT-2 (`gpt2`, `gpt2-xl`, etc.)
66 -- GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
67 -- GPT-J (`EleutherAI/gpt-j-6b`, `nomic-ai/gpt4all-j`, etc.)
68 -- GPT-NeoX (`EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
69 -- InternLM (`internlm/internlm-7b`, `internlm/internlm-chat-7b`, etc.)
70 -- InternLM2 (`internlm/internlm2-7b`, `internlm/internlm2-chat-7b`, etc.)
71 -- Jais (`core42/jais-13b`, `core42/jais-13b-chat`, `core42/jais-30b-v3`, `core42/jais-30b-chat-v3`, etc.)
72 -- LLaMA, Llama 2, and Meta Llama 3 (`meta-llama/Meta-Llama-3-8B-Instruct`, `meta-llama/Meta-Llama-3-70B-Instruct`, `meta-llama/Llama-2-70b-hf`, `lmsys/vicuna-13b-v1.3`, `young-geng/koala`, `openlm-research/open_llama_13b`, etc.)
73 -- MiniCPM (`openbmb/MiniCPM-2B-sft-bf16`, `openbmb/MiniCPM-2B-dpo-bf16`, etc.)
74 -- Mistral (`mistralai/Mistral-7B-v0.1`, `mistralai/Mistral-7B-Instruct-v0.1`, etc.)
75 -- Mixtral (`mistralai/Mixtral-8x7B-v0.1`, `mistralai/Mixtral-8x7B-Instruct-v0.1`, `mistral-community/Mixtral-8x22B-v0.1`, etc.)
76 -- MPT (`mosaicml/mpt-7b`, `mosaicml/mpt-30b`, etc.)
77 -- OLMo (`allenai/OLMo-1B-hf`, `allenai/OLMo-7B-hf`, etc.)
78 -- OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
79 -- Orion (`OrionStarAI/Orion-14B-Base`, `OrionStarAI/Orion-14B-Chat`, etc.)
80 -- Phi (`microsoft/phi-1_5`, `microsoft/phi-2`, etc.)
81 -- Phi-3 (`microsoft/Phi-3-mini-4k-instruct`, `microsoft/Phi-3-mini-128k-instruct`, etc.)
82 -- Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
83 -- Qwen2 (`Qwen/Qwen1.5-7B`, `Qwen/Qwen1.5-7B-Chat`, etc.)
84 -- Qwen2MoE (`Qwen/Qwen1.5-MoE-A2.7B`, `Qwen/Qwen1.5-MoE-A2.7B-Chat`, etc.)
85 -- StableLM(`stabilityai/stablelm-3b-4e1t`, `stabilityai/stablelm-base-alpha-7b-v2`, etc.)
86 -- Starcoder2(`bigcode/starcoder2-3b`, `bigcode/starcoder2-7b`, `bigcode/starcoder2-15b`, etc.)
87 -- Xverse (`xverse/XVERSE-7B-Chat`, `xverse/XVERSE-13B-Chat`, `xverse/XVERSE-65B-Chat`, etc.)
88 -- Yi (`01-ai/Yi-6B`, `01-ai/Yi-34B`, etc.)
54 +vLLM seamlessly supports most popular open-source models on HuggingFace, including:
55 +- Transformer-like LLMs (e.g., Llama)
56 +- Mixture-of-Expert LLMs (e.g., Mixtral)
57 +- Multi-modal LLMs (e.g., LLaVA)
58 +
59 +Find the full list of supported models [here](https://docs.vllm.ai/en/latest/models/supported_models.html).
60 +
61 +## Getting Started
62
63 Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/getting_started/installation.html#build-from-source):
64
@@ -93,9 +66,7 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
66 pip install vllm
67 ```
68
96 -## Getting Started
97 -
98 -Visit our [documentation](https://vllm.readthedocs.io/en/latest/) to get started.
69 +Visit our [documentation](https://vllm.readthedocs.io/en/latest/) to learn more.
70 - [Installation](https://vllm.readthedocs.io/en/latest/getting_started/installation.html)
71 - [Quickstart](https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html)
72 - [Supported Models](https://vllm.readthedocs.io/en/latest/models/supported_models.html)
docs/source/models/supported_models.rst
+19 -6
@@ -16,13 +16,21 @@ Alongside each architecture, we include some popular models that use it.
16 - Example HuggingFace Models
17 - :ref:`LoRA <lora>`
18 * - :code:`AquilaForCausalLM`
19 - - Aquila
19 + - Aquila & Aquila2
20 - :code:`BAAI/Aquila-7B`, :code:`BAAI/AquilaChat-7B`, etc.
21 - ✅︎
22 + * - :code:`ArcticForCausalLM`
23 + - Arctic
24 + - :code:`Snowflake/snowflake-arctic-base`, :code:`Snowflake/snowflake-arctic-instruct`, etc.
25 + -
26 * - :code:`BaiChuanForCausalLM`
23 - - Baichuan
27 + - Baichuan & Baichuan2
28 - :code:`baichuan-inc/Baichuan2-13B-Chat`, :code:`baichuan-inc/Baichuan-7B`, etc.
29 - ✅︎
30 + * - :code:`BloomForCausalLM`
31 + - BLOOM, BLOOMZ, BLOOMChat
32 + - :code:`bigscience/bloom`, :code:`bigscience/bloomz`, etc.
33 + -
34 * - :code:`ChatGLMModel`
35 - ChatGLM
36 - :code:`THUDM/chatglm2-6b`, :code:`THUDM/chatglm3-6b`, etc.
@@ -39,10 +47,6 @@ Alongside each architecture, we include some popular models that use it.
47 - DeciLM
48 - :code:`Deci/DeciLM-7B`, :code:`Deci/DeciLM-7B-instruct`, etc.
49 -
42 - * - :code:`BloomForCausalLM`
43 - - BLOOM, BLOOMZ, BLOOMChat
44 - - :code:`bigscience/bloom`, :code:`bigscience/bloomz`, etc.
45 - -
50 * - :code:`FalconForCausalLM`
51 - Falcon
52 - :code:`tiiuae/falcon-7b`, :code:`tiiuae/falcon-40b`, :code:`tiiuae/falcon-rw-7b`, etc.
@@ -135,6 +139,15 @@ Alongside each architecture, we include some popular models that use it.
139 - StableLM
140 - :code:`stabilityai/stablelm-3b-4e1t/` , :code:`stabilityai/stablelm-base-alpha-7b-v2`, etc.
141 -
142 + * - :code:`Starcoder2ForCausalLM`
143 + - Starcoder2
144 + - :code:`bigcode/starcoder2-3b`, :code:`bigcode/starcoder2-7b`, :code:`bigcode/starcoder2-15b`, etc.
145 + -
146 + * - :code:`XverseForCausalLM`
147 + - Xverse
148 + - :code:`xverse/XVERSE-7B-Chat`, :code:`xverse/XVERSE-13B-Chat`, :code:`xverse/XVERSE-65B-Chat`, etc.
149 + -
150 +
151
152 If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
153 Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.