@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2ba80bed27
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[Bugfix] Update Dockerfile.cpu to fix NameError: name 'vllm_ops' is not defined (#5009)

Letian Li committed May 23, 2024 at 17:08 UTC 2ba80bed2732edf42b1014ea4e34757849fc93d0
2 files changed +3 -1
.buildkite/run-cpu-test.sh
+1 -1
@@ -11,4 +11,4 @@ trap remove_docker_container EXIT
11 remove_docker_container
12
13 # Run the image and launch offline inference
14 -docker run --network host --env VLLM_CPU_KVCACHE_SPACE=1 --name cpu-test cpu-test python3 examples/offline_inference.py
14 +docker run --network host --env VLLM_CPU_KVCACHE_SPACE=1 --name cpu-test cpu-test python3 vllm/examples/offline_inference.py
Dockerfile.cpu
+2
@@ -17,4 +17,6 @@ RUN pip install -v -r requirements-cpu.txt --extra-index-url https://download.py
17
18 RUN VLLM_TARGET_DEVICE=cpu python3 setup.py install
19
20 +WORKDIR /workspace/
21 +
22 CMD ["/bin/bash"]