@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 74d8d77626
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Remove unused const TIMEOUT_TO_PREVENT_DEADLOCK (#2321)

Ronen Schaffer committed Jan 4, 2024 at 01:49 UTC 74d8d77626763bf7c4a2dd227231c69bb4638e29
1 file changed -1
vllm/entrypoints/api_server.py
-1
@@ -12,7 +12,6 @@ from vllm.sampling_params import SamplingParams
12 from vllm.utils import random_uuid
13
14 TIMEOUT_KEEP_ALIVE = 5 # seconds.
15 -TIMEOUT_TO_PREVENT_DEADLOCK = 1 # seconds.
15 app = FastAPI()
16 engine = None
17