@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 559eb852f8
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

[Core] init_distributed_environment align with init_process_group(#4014)

[Core][Distributed] make init_distributed_environment compatible with init_process_group (#4014)

youkaichao committed Apr 11, 2024 at 14:00 UTC 559eb852f83fe7867390dd2986b4f93a6572cf10
1 file changed +3 -3
vllm/distributed/parallel_state.py
+3 -3
@@ -39,9 +39,9 @@ _PIPELINE_GLOBAL_RANKS = None
39
40
41 def init_distributed_environment(
42 - world_size: int,
43 - rank: int,
44 - distributed_init_method: Optional[str] = None,
42 + world_size: int = -1,
43 + rank: int = -1,
44 + distributed_init_method: str = "env://",
45 local_rank: int = -1,
46 backend: str = "nccl",
47 ):