@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 47a0b615b4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

Add ray[default] to wget to run distributed inference out of box (#11265)

Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>

Jiaxin Shan committed Dec 20, 2024 at 13:54 UTC 47a0b615b45efd0a9ed57049d8ca6eff1c249844
2 files changed +2 -2
Dockerfile
+1 -1
@@ -163,7 +163,7 @@ RUN PYTHON_VERSION_STR=$(echo ${PYTHON_VERSION} | sed 's/\.//g') && \
163 RUN echo 'tzdata tzdata/Areas select America' | debconf-set-selections \
164 && echo 'tzdata tzdata/Zones/America select Los_Angeles' | debconf-set-selections \
165 && apt-get update -y \
166 - && apt-get install -y ccache software-properties-common git curl sudo vim python3-pip \
166 + && apt-get install -y ccache software-properties-common git curl wget sudo vim python3-pip \
167 && apt-get install -y ffmpeg libsm6 libxext6 libgl1 \
168 && add-apt-repository ppa:deadsnakes/ppa \
169 && apt-get update -y \
requirements-cuda.txt
+1 -1
@@ -2,7 +2,7 @@
2 -r requirements-common.txt
3
4 # Dependencies for NVIDIA GPUs
5 -ray >= 2.9
5 +ray[default] >= 2.9
6 nvidia-ml-py >= 12.560.30 # for pynvml package
7 torch == 2.5.1
8 # These must be updated alongside torch