@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f7389f4763
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

[Doc] Add Baichuan 13B to supported models (#656)

Zhuohan Li committed Aug 2, 2023 at 16:45 UTC f7389f4763c37579d249d0f9d80917e2ecfc4ead
3 files changed +5 -4
README.md
+1 -1
@@ -42,7 +42,7 @@ vLLM is flexible and easy to use with:
42
43 vLLM seamlessly supports many Huggingface models, including the following architectures:
44
45 -- Baichuan-7B (`baichuan-inc/Baichuan-7B`)
45 +- Baichuan (`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.)
46 - BLOOM (`bigscience/bloom`, `bigscience/bloomz`, etc.)
47 - Falcon (`tiiuae/falcon-7b`, `tiiuae/falcon-40b`, `tiiuae/falcon-rw-7b`, etc.)
48 - GPT-2 (`gpt2`, `gpt2-xl`, etc.)
docs/source/models/supported_models.rst
+2 -2
@@ -15,8 +15,8 @@ Alongside each architecture, we include some popular models that use it.
15 - Models
16 - Example HuggingFace Models
17 * - :code:`BaiChuanForCausalLM`
18 - - Baichuan-7B
19 - - :code:`baichuan-inc/Baichuan-7B`.
18 + - Baichuan
19 + - :code:`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.
20 * - :code:`BloomForCausalLM`
21 - BLOOM, BLOOMZ, BLOOMChat
22 - :code:`bigscience/bloom`, :code:`bigscience/bloomz`, etc.
vllm/model_executor/models/__init__.py
+2 -1
@@ -1,4 +1,5 @@
1 -from vllm.model_executor.models.baichuan import BaiChuanForCausalLM, BaichuanForCausalLM
1 +from vllm.model_executor.models.baichuan import (BaiChuanForCausalLM,
2 + BaichuanForCausalLM)
3 from vllm.model_executor.models.bloom import BloomForCausalLM
4 from vllm.model_executor.models.falcon import FalconForCausalLM
5 from vllm.model_executor.models.gpt2 import GPT2LMHeadModel