@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 10585e035e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

Removed Extraneous Print Message From OAI Server (#3440)

Robert Shaw committed Mar 15, 2024 at 19:35 UTC 10585e035ec564cd376146c3fe5ffe427a43c92c
1 file changed -3
vllm/entrypoints/openai/serving_completion.py
-3
@@ -309,10 +309,7 @@ class OpenAIServingCompletion(OpenAIServing):
309 except ValueError as e:
310 # TODO: Use a vllm-specific Validation Error
311 data = self.create_streaming_error_response(str(e))
312 - print("yield", f"data: {data}\n\n")
312 yield f"data: {data}\n\n"
314 -
315 - print("yield", "data: [DONE]\n\n")
313 yield "data: [DONE]\n\n"
314
315 def request_output_to_completion_response(