@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 415d109527
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

[Fix] Update Supported Models List (#1690)

Zhuohan Li committed Nov 16, 2023 at 14:47 UTC 415d1095278d896135728a54b4307dde3bc83243
2 files changed +11
README.md
+2
@@ -49,6 +49,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
49 - Aquila & Aquila2 (`BAAI/AquilaChat2-7B`, `BAAI/AquilaChat2-34B`, `BAAI/Aquila-7B`, `BAAI/AquilaChat-7B`, etc.)
50 - Baichuan (`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.)
51 - BLOOM (`bigscience/bloom`, `bigscience/bloomz`, etc.)
52 +- ChatGLM (`THUDM/chatglm2-6b`, `THUDM/chatglm3-6b`, etc.)
53 - Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
54 - GPT-2 (`gpt2`, `gpt2-xl`, etc.)
55 - GPT BigCode (`bigcode/starcoder`, `bigcode/gpt_bigcode-santacoder`, etc.)
@@ -61,6 +62,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
62 - OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
63 - Phi-1.5 (`microsoft/phi-1_5`, etc.)
64 - Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
65 +- Yi (`01-ai/Yi-6B`, `01-ai/Yi-34B`, etc.)
66
67 Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/getting_started/installation.html#build-from-source):
68
docs/source/models/supported_models.rst
+9
@@ -20,6 +20,9 @@ Alongside each architecture, we include some popular models that use it.
20 * - :code:`BaiChuanForCausalLM`
21 - Baichuan
22 - :code:`baichuan-inc/Baichuan-7B`, :code:`baichuan-inc/Baichuan-13B-Chat`, etc.
23 + * - :code:`ChatGLMModel`
24 + - ChatGLM
25 + - :code:`THUDM/chatglm2-6b`, :code:`THUDM/chatglm3-6b`, etc.
26 * - :code:`BloomForCausalLM`
27 - BLOOM, BLOOMZ, BLOOMChat
28 - :code:`bigscience/bloom`, :code:`bigscience/bloomz`, etc.
@@ -53,9 +56,15 @@ Alongside each architecture, we include some popular models that use it.
56 * - :code:`OPTForCausalLM`
57 - OPT, OPT-IML
58 - :code:`facebook/opt-66b`, :code:`facebook/opt-iml-max-30b`, etc.
59 + * - :code:`PhiForCausalLM`
60 + - Phi-1.5
61 + - :code:`microsoft/phi-1_5`, etc.
62 * - :code:`QWenLMHeadModel`
63 - Qwen
64 - :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
65 + * - :code:`YiForCausalLM`
66 + - Yi
67 + - :code:`01-ai/Yi-6B`, :code:`01-ai/Yi-34B`, etc.
68
69 If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
70 Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.