@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4f0e0ea131
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

Add FlashInfer to default Dockerfile (#6172)

Simon Mo committed Jul 8, 2024 at 13:38 UTC 4f0e0ea131ef40654faa26fa21196031754df53a
1 file changed +3
Dockerfile
+3
@@ -167,6 +167,9 @@ RUN --mount=type=bind,from=build,src=/workspace/dist,target=/vllm-workspace/dist
167 RUN --mount=type=bind,from=mamba-builder,src=/usr/src/mamba,target=/usr/src/mamba \
168 --mount=type=cache,target=/root/.cache/pip \
169 python3 -m pip install /usr/src/mamba/*.whl --no-cache-dir
170 +
171 +RUN --mount=type=cache,target=/root/.cache/pip \
172 + python3 -m pip install https://github.com/flashinfer-ai/flashinfer/releases/download/v0.0.8/flashinfer-0.0.8+cu121torch2.3-cp310-cp310-linux_x86_64.whl
173 #################### vLLM installation IMAGE ####################
174
175