@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 90fbf12540
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

fix relative import path of protocol.py (#3134)

Co-authored-by: huohuarong <huohuarong@zuoshouyisheng.com>

Huarong committed Mar 2, 2024 at 03:42 UTC 90fbf12540da089fcc7dc825ce2ceb7ea3a3df33
1 file changed +1 -1
vllm/entrypoints/openai/serving_completion.py
+1 -1
@@ -5,7 +5,7 @@ from typing import AsyncGenerator, AsyncIterator, Callable, List, Optional, Dict
5 from vllm.logger import init_logger
6 from vllm.utils import random_uuid
7 from vllm.engine.async_llm_engine import AsyncLLMEngine
8 -from .protocol import (
8 +from vllm.entrypoints.openai.protocol import (
9 CompletionRequest,
10 CompletionResponse,
11 CompletionResponseChoice,