A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
[Docs] Add GPTBigCode to supported models (#213)
Woosuk Kwon committed
Jun 22, 2023 at 15:05 UTC
665c48963be11b2e5cb7209cd25f884129e5c284
2 files changed
+5
-1
README.md
+2
-1
@@ -42,7 +42,8 @@ vLLM is flexible and easy to use with:
42
vLLM seamlessly supports many Huggingface models, including the following architectures:
43
44
- GPT-2 (`gpt2`, `gpt2-xl`, etc.)
45
-- GPTNeoX (`EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
45
+- GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
46
+- GPT-NeoX (`EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
47
- LLaMA (`lmsys/vicuna-13b-v1.3`, `young-geng/koala`, `openlm-research/open_llama_13b`, etc.)
48
- OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
49
docs/source/models/supported_models.rst
+3
@@ -17,6 +17,9 @@ Alongside each architecture, we include some popular models that use it.
17
* - :code:`GPT2LMHeadModel`
18
- GPT-2
19
- :code:`gpt2`, :code:`gpt2-xl`, etc.
20
+ * - :code:`GPTBigCodeForCausalLM`
21
+ - StarCoder, SantaCoder, WizardCoder
22
+ - :code:`bigcode/starcoder`, :code:`bigcode/gpt_bigcode-santacoder`, :code:`WizardLM/WizardCoder-15B-V1.0`, etc.
23
* - :code:`GPTNeoXForCausalLM`
24
- GPT-NeoX, Pythia, OpenAssistant, Dolly V2, StableLM
25
- :code:`EleutherAI/gpt-neox-20b`, :code:`EleutherAI/pythia-12b`, :code:`OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5`, :code:`databricks/dolly-v2-12b`, :code:`stabilityai/stablelm-tuned-alpha-7b`, etc.