@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / aee8ef661a
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Miner fix of type hint (#2340)

ljss committed Jan 4, 2024 at 13:27 UTC aee8ef661a227bfeb0c886b2b3892b0fc56bbd47
1 file changed +1 -1
vllm/engine/ray_utils.py
+1 -1
@@ -55,7 +55,7 @@ def initialize_cluster(
55 parallel_config: ParallelConfig,
56 engine_use_ray: bool = False,
57 ray_address: Optional[str] = None,
58 -) -> Tuple[str, Optional["PlacementGroup"]]:
58 +) -> Optional["PlacementGroup"]:
59 """Initialize the distributed cluster probably with Ray.
60
61 Args: