@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 7bc94a0fdd
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

add ccache to docker build image (#3704)

Simon Mo committed Mar 28, 2024 at 22:14 UTC 7bc94a0fddcd62d20b40390a7efb69c7a221ae5b
1 file changed +6 -1
Dockerfile
+6 -1
@@ -35,6 +35,9 @@ COPY requirements-build.txt requirements-build.txt
35 RUN --mount=type=cache,target=/root/.cache/pip \
36 pip install -r requirements-build.txt
37
38 +# install compiler cache to speed up compilation leveraging local or remote caching
39 +RUN apt-get update -y && apt-get install -y ccache
40 +
41 # copy input files
42 COPY csrc csrc
43 COPY setup.py setup.py
@@ -56,7 +59,9 @@ ENV NVCC_THREADS=$nvcc_threads
59 # make sure punica kernels are built (for LoRA)
60 ENV VLLM_INSTALL_PUNICA_KERNELS=1
61
59 -RUN python3 setup.py build_ext --inplace
62 +ENV CCACHE_DIR=/root/.cache/ccache
63 +RUN --mount=type=cache,target=/root/.cache/ccache \
64 + python3 setup.py build_ext --inplace
65 #################### EXTENSION Build IMAGE ####################
66
67 #################### FLASH_ATTENTION Build IMAGE ####################