@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 205b94942e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

[CI/Build] fix TORCH_CUDA_ARCH_LIST in wheel build (#3801)

youkaichao committed Apr 2, 2024 at 11:54 UTC 205b94942e97f668c01d8f4b476da36277df4c59
1 file changed +2 -1
.github/workflows/scripts/build.sh
+2 -1
@@ -15,6 +15,7 @@ $python_executable -m pip install -r requirements.txt
15 export MAX_JOBS=1
16 # Make sure punica is built for the release (for LoRA)
17 export VLLM_INSTALL_PUNICA_KERNELS=1
18 -
18 +# Make sure release wheels are built for the following architectures
19 +export TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6 8.9 9.0+PTX"
20 # Build
21 $python_executable setup.py bdist_wheel --dist-dir=dist