@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 1ecc645b8f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[doc] backward compatibility for 0.6.4 (#11359)

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao committed Dec 19, 2024 at 21:33 UTC 1ecc645b8f5431f1404551ad24721a63f01aea4e
1 file changed +5
docs/source/getting_started/debugging.rst
+5
@@ -86,6 +86,11 @@ If GPU/CPU communication cannot be established, you can use the following Python
86 from vllm.distributed.device_communicators.pynccl import PyNcclCommunicator
87
88 pynccl = PyNcclCommunicator(group=gloo_group, device=local_rank)
89 + # pynccl is enabled by default for 0.6.5+,
90 + # but for 0.6.4 and below, we need to enable it manually.
91 + # keep the code for backward compatibility when because people
92 + # prefer to read the latest documentation.
93 + pynccl.disabled = False
94
95 s = torch.cuda.Stream()
96 with torch.cuda.stream(s):