@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 59edd0f134
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 19:30 UTC

[Bugfix][CI] Import ray under guard (#7486)

Woosuk Kwon committed Aug 13, 2024 at 17:12 UTC 59edd0f1340a85d585a477fc687a16cfc5cf5276
1 file changed +1 -1
vllm/distributed/device_communicators/tpu_communicator.py
+1 -1
@@ -1,4 +1,3 @@
1 -import ray
1 import torch
2 import torch.distributed as dist
3 from torch.distributed import ProcessGroup
@@ -6,6 +5,7 @@ from torch.distributed import ProcessGroup
5 from vllm.platforms import current_platform
6
7 if current_platform.is_tpu():
8 + import ray
9 import torch_xla.core.xla_model as xm
10 import torch_xla.runtime as xr
11 from torch_xla._internal import pjrt