@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 45c35f0d58
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[CI/Build] Reducing CPU CI execution time (#5241)

Li, Jiang committed Jun 5, 2024 at 01:26 UTC 45c35f0d58f4508bf43bd6af1d3d0d0ec0c915e6
1 file changed +1 -1
.buildkite/run-cpu-test.sh
+1 -1
@@ -11,7 +11,7 @@ trap remove_docker_container EXIT
11 remove_docker_container
12
13 # Run the image
14 -docker run -itd -v ~/.cache/huggingface:/root/.cache/huggingface --network host -e HF_TOKEN --env VLLM_CPU_KVCACHE_SPACE=4 --name cpu-test cpu-test
14 +docker run -itd -v ~/.cache/huggingface:/root/.cache/huggingface --cpuset-cpus=48-95 --cpuset-mems=1 --network host -e HF_TOKEN --env VLLM_CPU_KVCACHE_SPACE=4 --name cpu-test cpu-test
15
16 # offline inference
17 docker exec cpu-test bash -c "python3 examples/offline_inference.py"