@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d76084c12f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[ CI ] Re-enable Large Model LM Eval (#6031)

Robert Shaw committed Jul 1, 2024 at 12:40 UTC d76084c12f0400a8f8364883a841405c903b61f6
1 file changed +9
.buildkite/test-pipeline.yaml
+9
@@ -198,6 +198,15 @@ steps:
198 - export VLLM_WORKER_MULTIPROC_METHOD=spawn
199 - bash ./run-tests.sh -c configs/models-small.txt -t 1
200
201 +- label: LM Eval Large Models
202 + gpu: a100
203 + num_gpus: 4
204 + working_dir: "/vllm-workspace/.buildkite/lm-eval-harness"
205 + commands:
206 + - pip install lm-eval
207 + - export VLLM_WORKER_MULTIPROC_METHOD=spawn
208 + - bash ./run-tests.sh -c configs/models-large.txt -t 4
209 +
210 - label: Documentation Build
211 working_dir: "/vllm-workspace/test_docs/docs"
212 no_gpu: True