@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f8c688d746
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

[Minor] Add Phi 2 to supported models (#2159)

Woosuk Kwon committed Dec 17, 2023 at 02:54 UTC f8c688d746b32a54c249c4faf6fc73b83f53952a
3 files changed +5 -5
README.md
+1 -1
@@ -65,7 +65,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
65 - Mixtral (`mistralai/Mixtral-8x7B-v0.1`, `mistralai/Mixtral-8x7B-Instruct-v0.1`, etc.)
66 - MPT (`mosaicml/mpt-7b`, `mosaicml/mpt-30b`, etc.)
67 - OPT (`facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc.)
68 -- Phi-1.5 (`microsoft/phi-1_5`, etc.)
68 +- Phi (`microsoft/phi-1_5`, `microsoft/phi-2`, etc.)
69 - Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
70 - Yi (`01-ai/Yi-6B`, `01-ai/Yi-34B`, etc.)
71
docs/source/models/supported_models.rst
+2 -2
@@ -60,8 +60,8 @@ Alongside each architecture, we include some popular models that use it.
60 - OPT, OPT-IML
61 - :code:`facebook/opt-66b`, :code:`facebook/opt-iml-max-30b`, etc.
62 * - :code:`PhiForCausalLM`
63 - - Phi-1.5
64 - - :code:`microsoft/phi-1_5`, etc.
63 + - Phi
64 + - :code:`microsoft/phi-1_5`, :code:`microsoft/phi-2`, etc.
65 * - :code:`QWenLMHeadModel`
66 - Qwen
67 - :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
tests/models/test_models.py
+2 -2
@@ -15,12 +15,12 @@ MODELS = [
15 "EleutherAI/pythia-70m",
16 "bigscience/bloom-560m",
17 "mosaicml/mpt-7b",
18 - "microsoft/phi-1_5",
18 + "microsoft/phi-2",
19 ]
20
21
22 @pytest.mark.parametrize("model", MODELS)
23 -@pytest.mark.parametrize("dtype", ["half"])
23 +@pytest.mark.parametrize("dtype", ["float"])
24 @pytest.mark.parametrize("max_tokens", [128])
25 def test_models(
26 hf_runner,