A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
[Docs] Document Deepseek V3 support (#11535)
Signed-off-by: simon-mo <simon.mo@hey.com>
Simon Mo committed
Dec 26, 2024 at 16:21 UTC
82d24f7aacf79bbccb6413333dff6303fbbb44b9
2 files changed
+7
-2
README.md
+1
-1
@@ -60,7 +60,7 @@ vLLM is flexible and easy to use with:
60
61
vLLM seamlessly supports most popular open-source models on HuggingFace, including:
62
- Transformer-like LLMs (e.g., Llama)
63
-- Mixture-of-Expert LLMs (e.g., Mixtral)
63
+- Mixture-of-Expert LLMs (e.g., Mixtral, Deepseek-V2 and V3)
64
- Embedding Models (e.g. E5-Mistral)
65
- Multi-modal LLMs (e.g., LLaVA)
66
docs/source/models/supported_models.md
+6
-1
@@ -137,6 +137,11 @@ See [this page](#generative-models) for more information on how to use generativ
137
- :code:`deepseek-ai/DeepSeek-V2`, :code:`deepseek-ai/DeepSeek-V2-Chat` etc.
138
-
139
- ✅︎
140
+ * - :code:`DeepseekV3ForCausalLM`
141
+ - DeepSeek-V3
142
+ - :code:`deepseek-ai/DeepSeek-V3-Base`, :code:`deepseek-ai/DeepSeek-V3` etc.
143
+ -
144
+ - ✅︎
145
* - :code:`ExaoneForCausalLM`
146
- EXAONE-3
147
- :code:`LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct`, etc.
@@ -676,7 +681,7 @@ See [this page](#generative-models) for more information on how to use generativ
681
- PaliGemma, PaliGemma 2
682
- T + I\ :sup:`E`
683
- :code:`google/paligemma-3b-pt-224`, :code:`google/paligemma-3b-mix-224`, :code:`google/paligemma2-3b-ft-docci-448`, etc.
679
- -
684
+ -
685
- ✅︎
686
-
687
* - :code:`Phi3VForCausalLM`