@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / cca61642e0
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 15:45 UTC

[Bugfix] Fix 3.12 builds on main (#8510)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

Joe Runde committed Sep 16, 2024 at 18:01 UTC cca61642e0484212e6cd78b35b4789afed8d19c6
2 files changed +1 -4
Dockerfile
-4
@@ -182,10 +182,6 @@ FROM vllm-base AS test
182 ADD . /vllm-workspace/
183
184 # install development dependencies (for testing)
185 -# A newer setuptools is required for installing some test dependencies from source that do not publish python 3.12 wheels
186 -# This installation must complete before the test dependencies are collected and installed.
187 -RUN --mount=type=cache,target=/root/.cache/pip \
188 - python3 -m pip install "setuptools>=74.1.1"
185 RUN --mount=type=cache,target=/root/.cache/pip \
186 python3 -m pip install -r requirements-dev.txt
187
requirements-common.txt
+1
@@ -29,4 +29,5 @@ importlib_metadata
29 mistral_common >= 1.4.0
30 pyyaml
31 six>=1.16.0; python_version > '3.11' # transitive dependency of pandas that needs to be the latest version for python 3.12
32 +setuptools>=74.1.1; python_version > '3.11' # Setuptools is used by triton, we need to ensure a modern version is installed for 3.12+ so that it does not try to import distutils, which was removed in 3.12
33 einops # Required for Qwen2-VL.