@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 523e30ea0c
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

[BugFix] Hot fix in setup.py for neuron build (#3537)

Zhuohan Li committed Mar 20, 2024 at 17:59 UTC 523e30ea0c5abcb447763dcd9a77b54d5c5f3239
1 file changed +1 -1
setup.py
+1 -1
@@ -168,7 +168,7 @@ class cmake_build_ext(build_ext):
168
169
170 def _is_cuda() -> bool:
171 - return torch.version.cuda is not None
171 + return torch.version.cuda is not None and not _is_neuron()
172
173
174 def _is_hip() -> bool: