@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / aea6ad629f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 08:15 UTC

Add hf_transfer to testing image (#10096)

Michael Goin committed Nov 8, 2024 at 03:35 UTC aea6ad629ff92f072a11b21dcdb1105677744007
1 file changed +5
Dockerfile
+5
@@ -191,6 +191,11 @@ ADD . /vllm-workspace/
191 RUN --mount=type=cache,target=/root/.cache/pip \
192 python3 -m pip install -r requirements-dev.txt
193
194 +# enable fast downloads from hf (for testing)
195 +RUN --mount=type=cache,target=/root/.cache/pip \
196 + python3 -m pip install hf_transfer
197 +ENV HF_HUB_ENABLE_HF_TRANSFER 1
198 +
199 # Copy in the v1 package for testing (it isn't distributed yet)
200 COPY vllm/v1 /usr/local/lib/python3.12/dist-packages/vllm/v1
201