A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
[Doc] Add Baichuan 13B to supported models (#656)
Zhuohan Li committed
Aug 2, 2023 at 16:45 UTC
f7389f4763c37579d249d0f9d80917e2ecfc4ead
3 files changed
+5
-4
README.md
+1
-1
@@ -42,7 +42,7 @@ vLLM is flexible and easy to use with:
42
43
vLLM seamlessly supports many Huggingface models, including the following architectures:
44
45
-- Baichuan-7B (`baichuan-inc/Baichuan-7B`)
45
+- Baichuan (`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.)
46
- BLOOM (`bigscience/bloom`, `bigscience/bloomz`, etc.)
47
- Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
48
- GPT-2 (`gpt2`, `gpt2-xl`, etc.)
docs/source/models/supported_models.rst
+2
-2
@@ -15,8 +15,8 @@ Alongside each architecture, we include some popular models that use it.
15
- Models
16
- Example HuggingFace Models
17
* - :code:`BaiChuanForCausalLM`
18
- - Baichuan-7B
19
- - :code:`baichuan-inc/Baichuan-7B`.
18
+ - Baichuan
19
+ - :code:`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.
20
* - :code:`BloomForCausalLM`
21
- BLOOM, BLOOMZ, BLOOMChat
22
- :code:`bigscience/bloom`, :code:`bigscience/bloomz`, etc.
vllm/model_executor/models/__init__.py
+2
-1
@@ -1,4 +1,5 @@
1
-from vllm.model_executor.models.baichuan import BaiChuanForCausalLM, BaichuanForCausalLM
1
+from vllm.model_executor.models.baichuan import (BaiChuanForCausalLM,
2
+ BaichuanForCausalLM)
3
from vllm.model_executor.models.bloom import BloomForCausalLM
4
from vllm.model_executor.models.falcon import FalconForCausalLM
5
from vllm.model_executor.models.gpt2 import GPT2LMHeadModel