@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a9c8212895
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

[FIX] Add Gemma model to the doc (#2966)

Zhuohan Li committed Feb 21, 2024 at 09:46 UTC a9c821289582747c57f149017678a282f5e788e4
2 files changed +4
README.md
+1
@@ -60,6 +60,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
60 - ChatGLM (`THUDM/chatglm2-6b`, `THUDM/chatglm3-6b`, etc.)
61 - DeciLM (`Deci/DeciLM-7B`, `Deci/DeciLM-7B-instruct`, etc.)
62 - Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
63 +- Gemma (`google/gemma-2b`, `google/gemma-7b`, etc.)
64 - GPT-2 (`gpt2`, `gpt2-xl`, etc.)
65 - GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
66 - GPT-J (`EleutherAI/gpt-j-6b`, `nomic-ai/gpt4all-j`, etc.)
docs/source/models/supported_models.rst
+3
@@ -32,6 +32,9 @@ Alongside each architecture, we include some popular models that use it.
32 * - :code:`FalconForCausalLM`
33 - Falcon
34 - :code:`tiiuae/falcon-7b`, :code:`tiiuae/falcon-40b`, :code:`tiiuae/falcon-rw-7b`, etc.
35 + * - :code:`GemmaForCausalLM`
36 + - Gemma
37 + - :code:`google/gemma-2b`, :code:`google/gemma-7b`, etc.
38 * - :code:`GPT2LMHeadModel`
39 - GPT-2
40 - :code:`gpt2`, :code:`gpt2-xl`, etc.