@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 08c5bdecae
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[Bugfix][TPU] Fix outlines installation in TPU Dockerfile (#6256)

Woosuk Kwon committed Jul 9, 2024 at 02:56 UTC 08c5bdecae5c5186c39a1d1ff444c3bf01f7fa0e
1 file changed +7 -3
Dockerfile.tpu
+7 -3
@@ -2,11 +2,8 @@ ARG NIGHTLY_DATE="20240601"
2 ARG BASE_IMAGE="us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:nightly_3.10_tpuvm_$NIGHTLY_DATE"
3
4 FROM $BASE_IMAGE
5 -
5 WORKDIR /workspace
7 -COPY . /workspace/vllm
6
9 -ENV VLLM_TARGET_DEVICE="tpu"
7 # Install aiohttp separately to avoid build errors.
8 RUN pip install aiohttp
9 # Install the TPU and Pallas dependencies.
@@ -14,6 +11,13 @@ RUN pip install torch_xla[tpu] -f https://storage.googleapis.com/libtpu-releases
11 RUN pip install torch_xla[pallas] -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html -f https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html
12
13 # Build vLLM.
14 +COPY . /workspace/vllm
15 +ENV VLLM_TARGET_DEVICE="tpu"
16 RUN cd /workspace/vllm && python setup.py develop
17
18 +# Re-install outlines to avoid dependency errors.
19 +# The outlines version must follow requirements-common.txt.
20 +RUN pip uninstall outlines -y
21 +RUN pip install "outlines>=0.0.43"
22 +
23 CMD ["/bin/bash"]