@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 739c350c19
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

[Minor Fix] Use cupy-cuda11x in CUDA 11.8 build (#3256)

陈序 committed Mar 14, 2024 at 00:43 UTC 739c350c1926682f435316294491aa54661849b6
1 file changed +6
setup.py
+6
@@ -431,6 +431,12 @@ def get_requirements() -> List[str]:
431 else:
432 with open(get_path("requirements.txt")) as f:
433 requirements = f.read().strip().split("\n")
434 + if nvcc_cuda_version <= Version("11.8"):
435 + # replace cupy-cuda12x with cupy-cuda11x for cuda 11.x
436 + for i in range(len(requirements)):
437 + if requirements[i].startswith("cupy-cuda12x"):
438 + requirements[i] = "cupy-cuda11x"
439 + break
440 return requirements
441
442