A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
Add Mistral to supported model list (#1221)
Woosuk Kwon committed
Sep 28, 2023 at 14:33 UTC
202351d5bf7a625852316516c0e1d00745bf88ab
2 files changed
+4
README.md
+1
@@ -64,6 +64,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
64
- GPT-NeoX (`EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
65
- InternLM (`internlm/internlm-7b`, `internlm/internlm-chat-7b`, etc.)
66
- LLaMA & LLaMA-2 (`meta-llama/Llama-2-70b-hf`, `lmsys/vicuna-13b-v1.3`, `young-geng/koala`, `openlm-research/open_llama_13b`, etc.)
67
+- Mistral (`mistralai/Mistral-7B-v0.1`, `mistralai/Mistral-7B-Instruct-v0.1`, etc.)
68
- MPT (`mosaicml/mpt-7b`, `mosaicml/mpt-30b`, etc.)
69
- OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
70
- Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
docs/source/models/supported_models.rst
+3
@@ -44,6 +44,9 @@ Alongside each architecture, we include some popular models that use it.
44
* - :code:`LlamaForCausalLM`
45
- LLaMA, LLaMA-2, Vicuna, Alpaca, Koala, Guanaco
46
- :code:`meta-llama/Llama-2-13b-hf`, :code:`meta-llama/Llama-2-70b-hf`, :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`young-geng/koala`, etc.
47
+ * - :code:`MistralForCausalLM`
48
+ - Mistral, Mistral-Instruct
49
+ - :code:`mistralai/Mistral-7B-v0.1`, :code:`mistralai/Mistral-7B-Instruct-v0.1`, etc.
50
* - :code:`MPTForCausalLM`
51
- MPT, MPT-Instruct, MPT-Chat, MPT-StoryWriter
52
- :code:`mosaicml/mpt-7b`, :code:`mosaicml/mpt-7b-storywriter`, :code:`mosaicml/mpt-30b`, etc.