@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ff9ddbceee
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756)

zifeitong committed Jun 21, 2024 at 20:33 UTC ff9ddbceee63efba6ba1f8d4dc66a92f1191da04
1 file changed -4
vllm/entrypoints/openai/run_batch.py
-4
@@ -1,5 +1,4 @@
1 import asyncio
2 -import sys
2 from io import StringIO
3 from typing import Awaitable, List
4
@@ -137,9 +136,6 @@ async def main(args):
136 output_buffer.seek(0)
137 await write_file(args.output_file, output_buffer.read().strip())
138
140 - # Temporary workaround for https://github.com/vllm-project/vllm/issues/4789
141 - sys.exit(0)
142 -
139
140 if __name__ == "__main__":
141 args = parse_args()