@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c0d00f5be6
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

[Fix] fix import error of RayWorker (#604) (#605)

Xudong Zhang committed Jul 28, 2023 at 14:37 UTC c0d00f5be6d3ed390534dd909c82b639baf2d359
1 file changed +1
vllm/engine/ray_utils.py
+1
@@ -27,6 +27,7 @@ try:
27 except ImportError:
28 ray = None
29 TorchDistributedWorker = None
30 + RayWorker = None # pylint: disable=invalid-name
31
32 if TYPE_CHECKING:
33 from ray.util.placement_group import PlacementGroup