A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
[Doc] Section for Multimodal Language Models (#7719)
Roger Wang committed
Aug 20, 2024 at 23:24 UTC
450664121277240b1c52611dbb7381ef2cc75355
1 file changed
+13
-3
docs/source/models/supported_models.rst
+13
-3
@@ -177,51 +177,61 @@ Decoder-only Language Models
177
178
.. _supported_vlms:
179
180
-Vision Language Models
181
-^^^^^^^^^^^^^^^^^^^^^^^
180
+Multimodal Language Models
181
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
182
183
.. list-table::
184
- :widths: 25 25 50 5
184
+ :widths: 25 25 25 25 5
185
:header-rows: 1
186
187
* - Architecture
188
- Models
189
+ - Supported Modality(ies)
190
- Example HuggingFace Models
191
- :ref:`LoRA <lora>`
192
* - :code:`Blip2ForConditionalGeneration`
193
- BLIP-2
194
+ - Image
195
- :code:`Salesforce/blip2-opt-2.7b`, :code:`Salesforce/blip2-opt-6.7b`, etc.
196
-
197
* - :code:`ChameleonForConditionalGeneration`
198
- Chameleon
199
+ - Image
200
- :code:`facebook/chameleon-7b` etc.
201
-
202
* - :code:`FuyuForCausalLM`
203
- Fuyu
204
+ - Image
205
- :code:`adept/fuyu-8b` etc.
206
-
207
* - :code:`InternVLChatModel`
208
- InternVL2
209
+ - Image
210
- :code:`OpenGVLab/InternVL2-4B`, :code:`OpenGVLab/InternVL2-8B`, etc.
211
-
212
* - :code:`LlavaForConditionalGeneration`
213
- LLaVA-1.5
214
+ - Image
215
- :code:`llava-hf/llava-1.5-7b-hf`, :code:`llava-hf/llava-1.5-13b-hf`, etc.
216
-
217
* - :code:`LlavaNextForConditionalGeneration`
218
- LLaVA-NeXT
219
+ - Image
220
- :code:`llava-hf/llava-v1.6-mistral-7b-hf`, :code:`llava-hf/llava-v1.6-vicuna-7b-hf`, etc.
221
-
222
* - :code:`PaliGemmaForConditionalGeneration`
223
- PaliGemma
224
+ - Image
225
- :code:`google/paligemma-3b-pt-224`, :code:`google/paligemma-3b-mix-224`, etc.
226
-
227
* - :code:`Phi3VForCausalLM`
228
- Phi-3-Vision
229
+ - Image
230
- :code:`microsoft/Phi-3-vision-128k-instruct`, etc.
231
-
232
* - :code:`MiniCPMV`
233
- MiniCPM-V
234
+ - Image
235
- :code:`openbmb/MiniCPM-V-2` (see note), :code:`openbmb/MiniCPM-Llama3-V-2_5`, :code:`openbmb/MiniCPM-V-2_6`, etc.
236
-
237