@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a63a4c6341
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 20:45 UTC

[Misc] Use 0.0.9 version for flashinfer (#6447)

Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com>

Pernekhan Utemuratov committed Jul 15, 2024 at 10:10 UTC a63a4c634174de7a6b966dd6996f9f8d2ae86827
1 file changed +1 -1
Dockerfile
+1 -1
@@ -172,7 +172,7 @@ RUN --mount=type=bind,from=mamba-builder,src=/usr/src/mamba,target=/usr/src/mamb
172 python3 -m pip install /usr/src/mamba/*.whl --no-cache-dir
173
174 RUN --mount=type=cache,target=/root/.cache/pip \
175 - python3 -m pip install https://github.com/flashinfer-ai/flashinfer/releases/download/v0.0.8/flashinfer-0.0.8+cu121torch2.3-cp310-cp310-linux_x86_64.whl
175 + python3 -m pip install https://github.com/flashinfer-ai/flashinfer/releases/download/v0.0.9/flashinfer-0.0.9+cu121torch2.3-cp310-cp310-linux_x86_64.whl
176 #################### vLLM installation IMAGE ####################
177
178