@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 105a40f53a
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

[Minor] Fix false warning when TP=1 (#2674)

Woosuk Kwon committed Jan 30, 2024 at 14:39 UTC 105a40f53a8001ce7f4282462a4636b31383f393
1 file changed +4
vllm/model_executor/parallel_utils/custom_all_reduce.py
+4
@@ -29,6 +29,10 @@ def init_custom_ar() -> None:
29 return
30 rank = get_tensor_model_parallel_rank()
31 world_size = get_tensor_model_parallel_world_size()
32 + if world_size == 1:
33 + # No need to initialize custom allreduce for single GPU case.
34 + return
35 +
36 if world_size not in _SUPPORTED_WORLD_SIZES:
37 logger.warn(
38 "Custom allreduce is disabled due to an unsupported world size: "