A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
[CI Bugfix] Specify same TORCH_CUDA_ARCH_LIST for flashinfer aot and install (#20772)
Signed-off-by: mgoin <mgoin64@gmail.com>
Michael Goin committed
Jul 11, 2025 at 10:16 UTC
cf75cd2098f6a3f0bc38d92d1669810c084dab9b
1 file changed
+2
-1
docker/Dockerfile
+2
-1
@@ -408,7 +408,8 @@ RUN --mount=type=cache,target=/root/.cache/uv bash - <<'BASH'
408
409
# Needed to build AOT kernels
410
pushd flashinfer
411
- python3 -m flashinfer.aot
411
+ TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
412
+ python3 -m flashinfer.aot
413
TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
414
uv pip install --system --no-build-isolation .
415
popd