@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 098f94de42
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 08:15 UTC

[CI/Build] Drop Python 3.8 support (#10038)

Signed-off-by: Russell Bryant <rbryant@redhat.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

Russell Bryant committed Nov 6, 2024 at 09:31 UTC 098f94de42859f8251fe920f87adb88336129c53
8 files changed +11 -18
.readthedocs.yaml
+1 -1
@@ -6,7 +6,7 @@ version: 2
6 build:
7 os: ubuntu-22.04
8 tools:
9 - python: '3.9'
9 + python: "3.12"
10
11 sphinx:
12 configuration: docs/source/conf.py
CMakeLists.txt
+1 -1
@@ -31,7 +31,7 @@ install(CODE "set(CMAKE_INSTALL_LOCAL_ONLY TRUE)" ALL_COMPONENTS)
31 # Supported python versions. These versions will be searched in order, the
32 # first match will be selected. These should be kept in sync with setup.py.
33 #
34 -set(PYTHON_SUPPORTED_VERSIONS "3.8" "3.9" "3.10" "3.11" "3.12")
34 +set(PYTHON_SUPPORTED_VERSIONS "3.9" "3.10" "3.11" "3.12")
35
36 # Supported NVIDIA architectures.
37 set(CUDA_SUPPORTED_ARCHS "7.0;7.5;8.0;8.6;8.9;9.0")
docs/source/getting_started/amd-installation.rst
-2
@@ -13,8 +13,6 @@ Requirements
13 * GPU: MI200s (gfx90a), MI300 (gfx942), Radeon RX 7900 series (gfx1100)
14 * ROCm 6.2
15
16 -Note: PyTorch 2.5+/ROCm6.2 dropped the support for python 3.8.
17 -
16 Installation options:
17
18 #. :ref:`Build from source with docker <build_from_source_docker_rocm>`
docs/source/getting_started/installation.rst
+1 -1
@@ -66,7 +66,7 @@ If you want to access the wheels for previous commits, you can specify the commi
66 $ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
67 $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
68
69 -Note that the wheels are built with Python 3.8 ABI (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about ABI), so **they are compatible with Python 3.8 and later**. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual versions of wheels are contained in the wheel metadata.
69 +Note that the wheels are built with Python 3.9 ABI (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about ABI), so **they are compatible with Python 3.9 and later**. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual versions of wheels are contained in the wheel metadata.
70
71 Another way to access the latest code is to use the docker images:
72
docs/source/getting_started/neuron-installation.rst
+1 -1
@@ -11,7 +11,7 @@ Requirements
11 ------------
12
13 * OS: Linux
14 -* Python: 3.8 -- 3.11
14 +* Python: 3.9 -- 3.11
15 * Accelerator: NeuronCore_v2 (in trn1/inf2 instances)
16 * Pytorch 2.0.1/2.1.1
17 * AWS Neuron SDK 2.16/2.17 (Verified on python 3.8)
docs/source/getting_started/quickstart.rst
+1 -1
@@ -12,7 +12,7 @@ This guide will help you quickly get started with vLLM to:
12 Prerequisites
13 --------------
14 - OS: Linux
15 -- Python: 3.8 - 3.12
15 +- Python: 3.9 -- 3.12
16 - GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
17
18 Installation
setup.py
+4 -8
@@ -55,12 +55,6 @@ def is_ninja_available() -> bool:
55 return which("ninja") is not None
56
57
58 -def remove_prefix(text, prefix):
59 - if text.startswith(prefix):
60 - return text[len(prefix):]
61 - return text
62 -
63 -
58 class CMakeExtension(Extension):
59
60 def __init__(self, name: str, cmake_lists_dir: str = '.', **kwa) -> None:
@@ -197,8 +191,10 @@ class cmake_build_ext(build_ext):
191 os.makedirs(self.build_temp)
192
193 targets = []
200 - target_name = lambda s: remove_prefix(remove_prefix(s, "vllm."),
201 - "vllm_flash_attn.")
194 +
195 + def target_name(s: str) -> str:
196 + return s.removeprefix("vllm.").removeprefix("vllm_flash_attn.")
197 +
198 # Build all the extensions
199 for ext in self.extensions:
200 self.configure(ext)
vllm/distributed/parallel_state.py
+2 -3
@@ -89,12 +89,11 @@ def _get_unique_name(name: str) -> str:
89 return newname
90
91
92 -_groups: Dict[str, Callable[[], "GroupCoordinator"]] = {}
92 +_groups: Dict[str, Callable[[], Optional["GroupCoordinator"]]] = {}
93
94
95 def _register_group(group: "GroupCoordinator") -> None:
96 - # looks like Python 3.8 does not understand `ReferenceType`
97 - _groups[group.unique_name] = weakref.ref(group) # type: ignore
96 + _groups[group.unique_name] = weakref.ref(group)
97
98
99 if supports_custom_op():