A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] add backend to doc string of initialize_model_parallel (#22142)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Ning Xie committed
Aug 5, 2025 at 10:36 UTC
29b97c09950fb6970756f5e2cfd4a3d7c1f4d72e
1 file changed
+1
vllm/distributed/parallel_state.py
+1
@@ -1013,6 +1013,7 @@ def initialize_model_parallel(
1013
parallelism.
1014
pipeline_model_parallel_size: number of GPUs used for pipeline model
1015
parallelism.
1016
+ backend: name of torch distributed communication backend.
1017
1018
Let's say we have a total of 8 GPUs denoted by g0 ... g7 and we
1019
use 2 GPUs to parallelize the model tensor, and 4 GPUs to parallelize