@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 175c43eca4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[Doc] Reorganize Supported Models by Type (#6167)

Roger Wang committed Jul 5, 2024 at 22:59 UTC 175c43eca4e6a50e160c386c6668ae4645c0b5d1
2 files changed +36 -18
docs/source/models/supported_models.rst
+34 -17
@@ -7,6 +7,8 @@ vLLM supports a variety of generative Transformer models in `HuggingFace Transfo
7 The following is the list of model architectures that are currently supported by vLLM.
8 Alongside each architecture, we include some popular models that use it.
9
10 +Decoder-only Language Models
11 +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
12 .. list-table::
13 :widths: 25 25 50 5
14 :header-rows: 1
@@ -95,14 +97,6 @@ Alongside each architecture, we include some popular models that use it.
97 - LLaMA, Llama 2, Meta Llama 3, Vicuna, Alpaca, Yi
98 - :code:`meta-llama/Meta-Llama-3-8B-Instruct`, :code:`meta-llama/Meta-Llama-3-70B-Instruct`, :code:`meta-llama/Llama-2-13b-hf`, :code:`meta-llama/Llama-2-70b-hf`, :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`01-ai/Yi-6B`, :code:`01-ai/Yi-34B`, etc.
99 - ✅︎
98 - * - :code:`LlavaForConditionalGeneration`
99 - - LLaVA-1.5
100 - - :code:`llava-hf/llava-1.5-7b-hf`, :code:`llava-hf/llava-1.5-13b-hf`, etc.
101 - -
102 - * - :code:`LlavaNextForConditionalGeneration`
103 - - LLaVA-NeXT
104 - - :code:`llava-hf/llava-v1.6-mistral-7b-hf`, :code:`llava-hf/llava-v1.6-vicuna-7b-hf`, etc.
105 - -
100 * - :code:`MiniCPMForCausalLM`
101 - MiniCPM
102 - :code:`openbmb/MiniCPM-2B-sft-bf16`, :code:`openbmb/MiniCPM-2B-dpo-bf16`, etc.
@@ -143,10 +137,6 @@ Alongside each architecture, we include some popular models that use it.
137 - Phi-3-Small
138 - :code:`microsoft/Phi-3-small-8k-instruct`, :code:`microsoft/Phi-3-small-128k-instruct`, etc.
139 -
146 - * - :code:`Phi3VForCausalLM`
147 - - Phi-3-Vision
148 - - :code:`microsoft/Phi-3-vision-128k-instruct`, etc.
149 - -
140 * - :code:`QWenLMHeadModel`
141 - Qwen
142 - :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
@@ -172,14 +162,40 @@ Alongside each architecture, we include some popular models that use it.
162 - :code:`xverse/XVERSE-7B-Chat`, :code:`xverse/XVERSE-13B-Chat`, :code:`xverse/XVERSE-65B-Chat`, etc.
163 -
164
165 +.. note::
166 + Currently, the ROCm version of vLLM supports Mistral and Mixtral only for context lengths up to 4096.
167 +
168 +.. _supported_vlms:
169 +
170 +Vision Language Models
171 +^^^^^^^^^^^^^^^^^^^^^^^
172 +
173 +.. list-table::
174 + :widths: 25 25 50 5
175 + :header-rows: 1
176 +
177 + * - Architecture
178 + - Models
179 + - Example HuggingFace Models
180 + - :ref:`LoRA <lora>`
181 + * - :code:`LlavaForConditionalGeneration`
182 + - LLaVA-1.5
183 + - :code:`llava-hf/llava-1.5-7b-hf`, :code:`llava-hf/llava-1.5-13b-hf`, etc.
184 + -
185 + * - :code:`LlavaNextForConditionalGeneration`
186 + - LLaVA-NeXT
187 + - :code:`llava-hf/llava-v1.6-mistral-7b-hf`, :code:`llava-hf/llava-v1.6-vicuna-7b-hf`, etc.
188 + -
189 + * - :code:`Phi3VForCausalLM`
190 + - Phi-3-Vision
191 + - :code:`microsoft/Phi-3-vision-128k-instruct`, etc.
192 + -
193
194 If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
177 -Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.
195 +Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` and :ref:`Adding a New Multimodal Model <adding_a_new_multimodal_model>`
196 +for instructions on how to implement support for your model.
197 Alternatively, you can raise an issue on our `GitHub <https://github.com/vllm-project/vllm/issues>`_ project.
198
180 -.. note::
181 - Currently, the ROCm version of vLLM supports Mistral and Mixtral only for context lengths up to 4096.
182 -
199 .. tip::
200 The easiest way to check if your model is supported is to run the program below:
201
@@ -210,8 +226,9 @@ Alternatively, you can raise an issue on our `GitHub <https://github.com/vllm-pr
226 output = llm.generate("Hello, my name is")
227 print(output)
228
229 +
230 Model Support Policy
214 ----------------------
231 +=====================
232
233 At vLLM, we are committed to facilitating the integration and support of third-party models within our ecosystem. Our approach is designed to balance the need for robustness and the practical limitations of supporting a wide range of models. Here’s how we manage third-party model support:
234
docs/source/models/vlm.rst
+2 -1
@@ -3,7 +3,8 @@
3 Using VLMs
4 ==========
5
6 -vLLM provides experimental support for Vision Language Models (VLMs). This document shows you how to run and serve these models using vLLM.
6 +vLLM provides experimental support for Vision Language Models (VLMs). See the :ref:`list of supported VLMs here <supported_vlms>`.
7 +This document shows you how to run and serve these models using vLLM.
8
9 .. important::
10 We are actively iterating on VLM support. Expect breaking changes to VLM usage and development in upcoming releases without prior deprecation.