@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 82d24f7aac
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[Docs] Document Deepseek V3 support (#11535)

Signed-off-by: simon-mo <simon.mo@hey.com>

Simon Mo committed Dec 26, 2024 at 16:21 UTC 82d24f7aacf79bbccb6413333dff6303fbbb44b9
2 files changed +7 -2
README.md
+1 -1
@@ -60,7 +60,7 @@ vLLM is flexible and easy to use with:
60
61 vLLM seamlessly supports most popular open-source models on HuggingFace, including:
62 - Transformer-like LLMs (e.g., Llama)
63 -- Mixture-of-Expert LLMs (e.g., Mixtral)
63 +- Mixture-of-Expert LLMs (e.g., Mixtral, Deepseek-V2 and V3)
64 - Embedding Models (e.g. E5-Mistral)
65 - Multi-modal LLMs (e.g., LLaVA)
66
docs/source/models/supported_models.md
+6 -1
@@ -137,6 +137,11 @@ See [this page](#generative-models) for more information on how to use generativ
137 - :code:`deepseek-ai/DeepSeek-V2`, :code:`deepseek-ai/DeepSeek-V2-Chat` etc.
138 -
139 - ✅︎
140 + * - :code:`DeepseekV3ForCausalLM`
141 + - DeepSeek-V3
142 + - :code:`deepseek-ai/DeepSeek-V3-Base`, :code:`deepseek-ai/DeepSeek-V3` etc.
143 + -
144 + - ✅︎
145 * - :code:`ExaoneForCausalLM`
146 - EXAONE-3
147 - :code:`LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct`, etc.
@@ -676,7 +681,7 @@ See [this page](#generative-models) for more information on how to use generativ
681 - PaliGemma, PaliGemma 2
682 - T + I\ :sup:`E`
683 - :code:`google/paligemma-3b-pt-224`, :code:`google/paligemma-3b-mix-224`, :code:`google/paligemma2-3b-ft-docci-448`, etc.
679 - -
684 + -
685 - ✅︎
686 -
687 * - :code:`Phi3VForCausalLM`