@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c614cfee58
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

Update dockerfile with ModelScope support (#3429)

ifsheldon committed Mar 20, 2024 at 01:54 UTC c614cfee5861e5715a023fa501e432d4acf910fe
1 file changed +1 -1
Dockerfile
+1 -1
@@ -122,7 +122,7 @@ RUN --mount=type=bind,from=flash-attn-builder,src=/usr/src/flash-attention-v2,ta
122 FROM vllm-base AS vllm-openai
123 # install additional dependencies for openai api server
124 RUN --mount=type=cache,target=/root/.cache/pip \
125 - pip install accelerate hf_transfer
125 + pip install accelerate hf_transfer modelscope
126
127 COPY --from=build /workspace/vllm/*.so /workspace/vllm/
128 COPY vllm vllm