@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 9ba0bd6aa6
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

Add `lm-eval` directly to requirements-test.txt (#9161)

Michael Goin committed Oct 8, 2024 at 21:22 UTC 9ba0bd6aa6a9a3cefa5c320800ea736a0abbaf36
5 files changed +4 -6
.buildkite/lm-eval-harness/run-lm-eval-gsm-hf-baseline.sh
+1 -1
@@ -2,7 +2,7 @@
2 # We can use this script to compute baseline accuracy on GSM for transformers.
3 #
4 # Make sure you have lm-eval-harness installed:
5 -# pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git@9516087b81a61d0e220b22cc1b75be76de23bc10
5 +# pip install lm-eval==0.4.4
6
7 usage() {
8 echo``
.buildkite/lm-eval-harness/run-lm-eval-gsm-vllm-baseline.sh
+1 -1
@@ -3,7 +3,7 @@
3 # We use this for fp8, which HF does not support.
4 #
5 # Make sure you have lm-eval-harness installed:
6 -# pip install lm-eval==0.4.3
6 +# pip install lm-eval==0.4.4
7
8 usage() {
9 echo``
.buildkite/test-pipeline.yaml
-3
@@ -98,7 +98,6 @@ steps:
98 - vllm/
99 commands:
100 - pip install -e ./plugins/vllm_add_dummy_model
101 - - pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git@a4987bba6e9e9b3f22bd3a6c1ecf0abd04fd5622#egg=lm_eval[api]
101 - pytest -v -s entrypoints/llm --ignore=entrypoints/llm/test_lazy_outlines.py --ignore=entrypoints/llm/test_generate.py --ignore=entrypoints/llm/test_generate_multiple_loras.py --ignore=entrypoints/llm/test_guided_generate.py
102 - pytest -v -s entrypoints/llm/test_lazy_outlines.py # it needs a clean process
103 - pytest -v -s entrypoints/llm/test_generate.py # it needs a clean process
@@ -278,7 +277,6 @@ steps:
277 - csrc/
278 - vllm/model_executor/layers/quantization
279 commands:
281 - - pip install lm-eval
280 - export VLLM_WORKER_MULTIPROC_METHOD=spawn
281 - bash ./run-tests.sh -c configs/models-small.txt -t 1
282
@@ -492,6 +490,5 @@ steps:
490 - csrc/
491 - vllm/model_executor/layers/quantization
492 commands:
495 - - pip install lm-eval
493 - export VLLM_WORKER_MULTIPROC_METHOD=spawn
494 - bash ./run-tests.sh -c configs/models-large.txt -t 4
docs/source/quantization/fp8.rst
+1 -1
@@ -106,7 +106,7 @@ Install ``vllm`` and ``lm-evaluation-harness``:
106
107 .. code-block:: console
108
109 - $ pip install vllm lm_eval==0.4.3
109 + $ pip install vllm lm-eval==0.4.4
110
111 Load and run the model in ``vllm``:
112
requirements-test.txt
+1
@@ -22,6 +22,7 @@ timm # required for internvl test
22 transformers_stream_generator # required for qwen-vl test
23 matplotlib # required for qwen-vl test
24 datamodel_code_generator # required for minicpm3 test
25 +lm-eval[api]==0.4.4 # required for model evaluation test
26
27 # TODO: Add this after fully implementing llava(mantis)
28 # git+https://github.com/TIGER-AI-Lab/Mantis.git # required for llava(mantis) test