A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
Add `lm-eval` directly to requirements-test.txt (#9161)
Michael Goin committed
Oct 8, 2024 at 21:22 UTC
9ba0bd6aa6a9a3cefa5c320800ea736a0abbaf36
5 files changed
+4
-6
.buildkite/lm-eval-harness/run-lm-eval-gsm-hf-baseline.sh
+1
-1
@@ -2,7 +2,7 @@
2
# We can use this script to compute baseline accuracy on GSM for transformers.
3
#
4
# Make sure you have lm-eval-harness installed:
5
-# pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git@9516087b81a61d0e220b22cc1b75be76de23bc10
5
+# pip install lm-eval==0.4.4
6
7
usage() {
8
echo``
.buildkite/lm-eval-harness/run-lm-eval-gsm-vllm-baseline.sh
+1
-1
@@ -3,7 +3,7 @@
3
# We use this for fp8, which HF does not support.
4
#
5
# Make sure you have lm-eval-harness installed:
6
-# pip install lm-eval==0.4.3
6
+# pip install lm-eval==0.4.4
7
8
usage() {
9
echo``
.buildkite/test-pipeline.yaml
-3
@@ -98,7 +98,6 @@ steps:
98
- vllm/
99
commands:
100
- pip install -e ./plugins/vllm_add_dummy_model
101
- - pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git@a4987bba6e9e9b3f22bd3a6c1ecf0abd04fd5622#egg=lm_eval[api]
101
- pytest -v -s entrypoints/llm --ignore=entrypoints/llm/test_lazy_outlines.py --ignore=entrypoints/llm/test_generate.py --ignore=entrypoints/llm/test_generate_multiple_loras.py --ignore=entrypoints/llm/test_guided_generate.py
102
- pytest -v -s entrypoints/llm/test_lazy_outlines.py # it needs a clean process
103
- pytest -v -s entrypoints/llm/test_generate.py # it needs a clean process
@@ -278,7 +277,6 @@ steps:
277
- csrc/
278
- vllm/model_executor/layers/quantization
279
commands:
281
- - pip install lm-eval
280
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
281
- bash ./run-tests.sh -c configs/models-small.txt -t 1
282
@@ -492,6 +490,5 @@ steps:
490
- csrc/
491
- vllm/model_executor/layers/quantization
492
commands:
495
- - pip install lm-eval
493
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
494
- bash ./run-tests.sh -c configs/models-large.txt -t 4
docs/source/quantization/fp8.rst
+1
-1
@@ -106,7 +106,7 @@ Install ``vllm`` and ``lm-evaluation-harness``:
106
107
.. code-block:: console
108
109
- $ pip install vllm lm_eval==0.4.3
109
+ $ pip install vllm lm-eval==0.4.4
110
111
Load and run the model in ``vllm``:
112
requirements-test.txt
+1
@@ -22,6 +22,7 @@ timm # required for internvl test
22
transformers_stream_generator # required for qwen-vl test
23
matplotlib # required for qwen-vl test
24
datamodel_code_generator # required for minicpm3 test
25
+lm-eval[api]==0.4.4 # required for model evaluation test
26
27
# TODO: Add this after fully implementing llava(mantis)
28
# git+https://github.com/TIGER-AI-Lab/Mantis.git # required for llava(mantis) test