@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 0b32a987dd
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

Add and list supported models in README (#161)

Zhuohan Li committed Jun 20, 2023 at 10:57 UTC 0b32a987ddb0d73dbf366c51eb5a0b7daf0d3ec0
3 files changed +15 -1
README.md
+7
@@ -39,6 +39,13 @@ vLLM is flexible and easy to use with:
39 - Streaming outputs
40 - OpenAI-compatible API server
41
42 +vLLM seamlessly supports many Huggingface models, including the following architectures:
43 +
44 +- GPT-2 (e.g., `gpt2`, `gpt2-xl`, etc.)
45 +- GPTNeoX (e.g., `EleutherAI/gpt-neox-20b`, `databricks/dolly-v2-12b`, `stabilityai/stablelm-tuned-alpha-7b`, etc.)
46 +- LLaMA (e.g., `lmsys/vicuna-13b-v1.3`, `young-geng/koala`, `openlm-research/open_llama_13b`, etc.)
47 +- OPT (e.g., `facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
48 +
49 Install vLLM with pip or [from source](https://llm-serving-cacheflow.readthedocs-hosted.com/en/latest/getting_started/installation.html#build-from-source):
50
51 ```bash
docs/source/conf.py
+2
@@ -53,7 +53,9 @@ copybutton_prompt_is_regexp = True
53 #
54 html_title = project
55 html_theme = 'sphinx_book_theme'
56 +html_logo = 'assets/logos/vllm-logo-text-light.png'
57 html_theme_options = {
58 + 'logo_only': True,
59 'path_to_docs': 'docs/source',
60 'repository_url': 'https://github.com/WoosukKwon/vllm',
61 'use_repository_button': True,
docs/source/models/supported_models.rst
+6 -1
@@ -8,19 +8,24 @@ The following is the list of model architectures that are currently supported by
8 Alongside each architecture, we include some popular models that use it.
9
10 .. list-table::
11 - :widths: 25 75
11 + :widths: 25 25 50
12 :header-rows: 1
13
14 * - Architecture
15 - Models
16 + - Example HuggingFace Models
17 * - :code:`GPT2LMHeadModel`
18 - GPT-2
19 + - :code:`gpt2`, :code:`gpt2-xl`, etc.
20 * - :code:`GPTNeoXForCausalLM`
21 - GPT-NeoX, Pythia, OpenAssistant, Dolly V2, StableLM
22 + - :code:`EleutherAI/gpt-neox-20b`, :code:`EleutherAI/pythia-12b`, :code:`OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5`, :code:`databricks/dolly-v2-12b`, :code:`stabilityai/stablelm-tuned-alpha-7b`, etc.
23 * - :code:`LlamaForCausalLM`
24 - LLaMA, Vicuna, Alpaca, Koala, Guanaco
25 + - :code:`openlm-research/open_llama_13b`, :code:`lmsys/vicuna-13b-v1.3`, :code:`young-geng/koala`, :code:`JosephusCheung/Guanaco`, etc.
26 * - :code:`OPTForCausalLM`
27 - OPT, OPT-IML
28 + - :code:`facebook/opt-66b`, :code:`facebook/opt-iml-max-30b`, etc.
29
30 If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
31 Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.