@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 16c3e295a8
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

fix(ray_utils): ignore re-init error (#465)

Massimiliano Pronesti committed Jul 20, 2023 at 02:01 UTC 16c3e295a877580867c8b90ace86acf3d1841455
1 file changed +1 -1
vllm/engine/ray_utils.py
+1 -1
@@ -44,7 +44,7 @@ def initialize_cluster(
44 "Ray is not installed. Please install Ray to use distributed "
45 "serving.")
46 # Connect to a ray cluster.
47 - ray.init(address=ray_address)
47 + ray.init(address=ray_address, ignore_reinit_error=True)
48
49 if not parallel_config.worker_use_ray:
50 # Initialize cluster locally.