@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 772a66732d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 07:00 UTC

[platforms] restore xpu check for parallel config (#10479)

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao committed Nov 20, 2024 at 09:13 UTC 772a66732d0ff58a43dbd1ae79c0d165659aa96d
1 file changed +10
vllm/platforms/xpu.py
+10
@@ -55,3 +55,13 @@ class XPUPlatform(Platform):
55 "CUDA graph is not supported on XPU, fallback to the eager "
56 "mode.")
57 model_config.enforce_eager = True
58 +
59 + # check and update parallel config
60 + parallel_config = vllm_config.parallel_config
61 + if (parallel_config.distributed_executor_backend is not None
62 + and parallel_config.distributed_executor_backend != "ray"):
63 + logger.warning(
64 + "%s is not supported on XPU, fallback to ray distributed"
65 + " executor backend.",
66 + parallel_config.distributed_executor_backend)
67 + parallel_config.distributed_executor_backend = "ray"