A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[bugfix] fix blackwell deepep installation (#22255)
youkaichao committed
Aug 6, 2025 at 01:26 UTC
59a0b8554bf0e8a9902e14e3d0e564fea38157b6
2 files changed
+12
-6
tools/ep_kernels/README.md
+5
-5
@@ -13,16 +13,16 @@ All scripts accept a positional argument as workspace path for staging the build
13
14
## Usage
15
16
-### Single-node
17
-
16
```bash
19
-bash install_python_libraries.sh
17
+# for hopper
18
+TORCH_CUDA_ARCH_LIST="9.0" bash install_python_libraries.sh
19
+# for blackwell
20
+TORCH_CUDA_ARCH_LIST="10.0" bash install_python_libraries.sh
21
```
22
22
-### Multi-node
23
+Additional step for multi-node deployment:
24
25
```bash
25
-bash install_python_libraries.sh
26
sudo bash configure_system_drivers.sh
27
sudo reboot # Reboot is required to load the new driver
28
```
tools/ep_kernels/install_python_libraries.sh
+7
-1
@@ -29,6 +29,12 @@ if [ -z "$CUDA_HOME" ]; then
29
exit 1
30
fi
31
32
+# assume TORCH_CUDA_ARCH_LIST is set correctly
33
+if [ -z "$TORCH_CUDA_ARCH_LIST" ]; then
34
+ echo "TORCH_CUDA_ARCH_LIST is not set, please set it to your desired architecture."
35
+ exit 1
36
+fi
37
+
38
# disable all features except IBGDA
39
export NVSHMEM_IBGDA_SUPPORT=1
40
@@ -95,7 +101,7 @@ clone_repo "https://github.com/ppl-ai/pplx-kernels" "pplx-kernels" "setup.py"
101
cd pplx-kernels
102
# see https://github.com/pypa/pip/issues/9955#issuecomment-838065925
103
# PIP_NO_BUILD_ISOLATION=0 disables build isolation
98
-PIP_NO_BUILD_ISOLATION=0 TORCH_CUDA_ARCH_LIST=9.0a+PTX pip install -vvv -e .
104
+PIP_NO_BUILD_ISOLATION=0 pip install -vvv -e .
105
popd
106
107
# build and install deepep, require pytorch installed