@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / cfba685bd4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[CI/Build] Add examples folder into Docker image so that we can leverage the templates*.jinja when serving models (#8758)

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>

Peter Pan committed Oct 9, 2024 at 00:37 UTC cfba685bd462f360994da7ac0d33f9759589506e
1 file changed +1
Dockerfile
+1
@@ -182,6 +182,7 @@ RUN --mount=type=bind,from=build,src=/workspace/dist,target=/vllm-workspace/dist
182 RUN --mount=type=cache,target=/root/.cache/pip \
183 . /etc/environment && \
184 python3 -m pip install https://github.com/flashinfer-ai/flashinfer/releases/download/v0.1.6/flashinfer-0.1.6+cu121torch2.4-cp${PYTHON_VERSION_STR}-cp${PYTHON_VERSION_STR}-linux_x86_64.whl
185 +COPY examples examples
186 #################### vLLM installation IMAGE ####################
187
188