A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
[Doc][V1] Add V1 support column for multimodal models (#10998)
Signed-off-by: Roger Wang <ywang@roblox.com>
Roger Wang committed
Dec 8, 2024 at 22:29 UTC
af7c4a92e654684066e61518d6ed90feda983635
1 file changed
+25
-1
docs/source/models/supported_models.rst
+25
-1
@@ -495,7 +495,7 @@ Text Generation
495
---------------
496
497
.. list-table::
498
- :widths: 25 25 15 25 5 5
498
+ :widths: 25 25 15 20 5 5 5
499
:header-rows: 1
500
501
* - Architecture
@@ -504,47 +504,55 @@ Text Generation
504
- Example HF Models
505
- :ref:`LoRA <lora>`
506
- :ref:`PP <distributed_serving>`
507
+ - V1
508
* - :code:`AriaForConditionalGeneration`
509
- Aria
510
- T + I
511
- :code:`rhymes-ai/Aria`
512
-
513
- ✅︎
514
+ -
515
* - :code:`Blip2ForConditionalGeneration`
516
- BLIP-2
517
- T + I\ :sup:`E`
518
- :code:`Salesforce/blip2-opt-2.7b`, :code:`Salesforce/blip2-opt-6.7b`, etc.
519
-
520
- ✅︎
521
+ -
522
* - :code:`ChameleonForConditionalGeneration`
523
- Chameleon
524
- T + I
525
- :code:`facebook/chameleon-7b` etc.
526
-
527
- ✅︎
528
+ -
529
* - :code:`FuyuForCausalLM`
530
- Fuyu
531
- T + I
532
- :code:`adept/fuyu-8b` etc.
533
-
534
- ✅︎
535
+ -
536
* - :code:`ChatGLMModel`
537
- GLM-4V
538
- T + I
539
- :code:`THUDM/glm-4v-9b` etc.
540
- ✅︎
541
- ✅︎
542
+ -
543
* - :code:`H2OVLChatModel`
544
- H2OVL
545
- T + I\ :sup:`E+`
546
- :code:`h2oai/h2ovl-mississippi-800m`, :code:`h2oai/h2ovl-mississippi-2b`, etc.
547
-
548
- ✅︎
549
+ -
550
* - :code:`Idefics3ForConditionalGeneration`
551
- Idefics3
552
- T + I
553
- :code:`HuggingFaceM4/Idefics3-8B-Llama3` etc.
554
- ✅︎
555
+ -
556
-
557
* - :code:`InternVLChatModel`
558
- InternVL 2.5, Mono-InternVL, InternVL 2.0
@@ -552,96 +560,112 @@ Text Generation
560
- :code:`OpenGVLab/InternVL2_5-4B`, :code:`OpenGVLab/Mono-InternVL-2B`, :code:`OpenGVLab/InternVL2-4B`, etc.
561
-
562
- ✅︎
563
+ - ✅︎
564
* - :code:`LlavaForConditionalGeneration`
565
- LLaVA-1.5
566
- T + I\ :sup:`E+`
567
- :code:`llava-hf/llava-1.5-7b-hf`, :code:`TIGER-Lab/Mantis-8B-siglip-llama3` (see note), etc.
568
-
569
- ✅︎
570
+ - ✅︎
571
* - :code:`LlavaNextForConditionalGeneration`
572
- LLaVA-NeXT
573
- T + I\ :sup:`E+`
574
- :code:`llava-hf/llava-v1.6-mistral-7b-hf`, :code:`llava-hf/llava-v1.6-vicuna-7b-hf`, etc.
575
-
576
- ✅︎
577
+ -
578
* - :code:`LlavaNextVideoForConditionalGeneration`
579
- LLaVA-NeXT-Video
580
- T + V
581
- :code:`llava-hf/LLaVA-NeXT-Video-7B-hf`, etc.
582
-
583
- ✅︎
584
+ -
585
* - :code:`LlavaOnevisionForConditionalGeneration`
586
- LLaVA-Onevision
587
- T + I\ :sup:`+` + V\ :sup:`+`
588
- :code:`llava-hf/llava-onevision-qwen2-7b-ov-hf`, :code:`llava-hf/llava-onevision-qwen2-0.5b-ov-hf`, etc.
589
-
590
- ✅︎
591
+ -
592
* - :code:`MiniCPMV`
593
- MiniCPM-V
594
- T + I\ :sup:`E+`
595
- :code:`openbmb/MiniCPM-V-2` (see note), :code:`openbmb/MiniCPM-Llama3-V-2_5`, :code:`openbmb/MiniCPM-V-2_6`, etc.
596
- ✅︎
597
- ✅︎
598
+ -
599
* - :code:`MllamaForConditionalGeneration`
600
- Llama 3.2
601
- T + I\ :sup:`+`
602
- :code:`meta-llama/Llama-3.2-90B-Vision-Instruct`, :code:`meta-llama/Llama-3.2-11B-Vision`, etc.
603
-
604
-
605
+ -
606
* - :code:`MolmoForCausalLM`
607
- Molmo
608
- T + I
609
- :code:`allenai/Molmo-7B-D-0924`, :code:`allenai/Molmo-72B-0924`, etc.
610
-
611
- ✅︎
612
+ - ✅︎
613
* - :code:`NVLM_D_Model`
614
- NVLM-D 1.0
615
- T + I\ :sup:`E+`
616
- :code:`nvidia/NVLM-D-72B`, etc.
617
-
618
- ✅︎
619
+ - ✅︎
620
* - :code:`PaliGemmaForConditionalGeneration`
621
- PaliGemma
622
- T + I\ :sup:`E`
623
- :code:`google/paligemma-3b-pt-224`, :code:`google/paligemma-3b-mix-224`, etc.
624
-
625
- ✅︎
626
+ -
627
* - :code:`Phi3VForCausalLM`
628
- Phi-3-Vision, Phi-3.5-Vision
629
- T + I\ :sup:`E+`
630
- :code:`microsoft/Phi-3-vision-128k-instruct`, :code:`microsoft/Phi-3.5-vision-instruct` etc.
631
-
632
- ✅︎
633
+ - ✅︎
634
* - :code:`PixtralForConditionalGeneration`
635
- Pixtral
636
- T + I\ :sup:`+`
637
- :code:`mistralai/Pixtral-12B-2409`, :code:`mistral-community/pixtral-12b` etc.
638
-
639
- ✅︎
640
+ - ✅︎
641
* - :code:`QWenLMHeadModel`
642
- Qwen-VL
643
- T + I\ :sup:`E+`
644
- :code:`Qwen/Qwen-VL`, :code:`Qwen/Qwen-VL-Chat`, etc.
645
- ✅︎
646
- ✅︎
647
+ -
648
* - :code:`Qwen2AudioForConditionalGeneration`
649
- Qwen2-Audio
650
- T + A\ :sup:`+`
651
- :code:`Qwen/Qwen2-Audio-7B-Instruct`
652
-
653
- ✅︎
654
+ -
655
* - :code:`Qwen2VLForConditionalGeneration`
656
- Qwen2-VL
657
- T + I\ :sup:`E+` + V\ :sup:`E+`
658
- :code:`Qwen/Qwen2-VL-2B-Instruct`, :code:`Qwen/Qwen2-VL-7B-Instruct`, :code:`Qwen/Qwen2-VL-72B-Instruct`, etc.
659
- ✅︎
660
- ✅︎
661
+ -
662
* - :code:`UltravoxModel`
663
- Ultravox
664
- T + A\ :sup:`E+`
665
- :code:`fixie-ai/ultravox-v0_3`
666
-
667
- ✅︎
668
+ -
669
670
| :sup:`E` Pre-computed embeddings can be inputted for this modality.
671
| :sup:`+` Multiple items can be inputted per text prompt for this modality.