A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
[FIX] Add Gemma model to the doc (#2966)
Zhuohan Li committed
Feb 21, 2024 at 09:46 UTC
a9c821289582747c57f149017678a282f5e788e4
2 files changed
+4
README.md
+1
@@ -60,6 +60,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
60
- ChatGLM (`THUDM/chatglm2-6b`, `THUDM/chatglm3-6b`, etc.)
61
- DeciLM (`Deci/DeciLM-7B`, `Deci/DeciLM-7B-instruct`, etc.)
62
- Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
63
+- Gemma (`google/gemma-2b`, `google/gemma-7b`, etc.)
64
- GPT-2 (`gpt2`, `gpt2-xl`, etc.)
65
- GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
66
- GPT-J (`EleutherAI/gpt-j-6b`, `nomic-ai/gpt4all-j`, etc.)
docs/source/models/supported_models.rst
+3
@@ -32,6 +32,9 @@ Alongside each architecture, we include some popular models that use it.
32
* - :code:`FalconForCausalLM`
33
- Falcon
34
- :code:`tiiuae/falcon-7b`, :code:`tiiuae/falcon-40b`, :code:`tiiuae/falcon-rw-7b`, etc.
35
+ * - :code:`GemmaForCausalLM`
36
+ - Gemma
37
+ - :code:`google/gemma-2b`, :code:`google/gemma-7b`, etc.
38
* - :code:`GPT2LMHeadModel`
39
- GPT-2
40
- :code:`gpt2`, :code:`gpt2-xl`, etc.