@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 54ca1ba71d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

docs: add description (#1553)

lots-o committed Nov 4, 2023 at 01:14 UTC 54ca1ba71deda7eb54f6c9d5002db43039e9edf3
1 file changed +1
vllm/outputs.py
+1
@@ -53,6 +53,7 @@ class RequestOutput:
53 request_id: The unique ID of the request.
54 prompt: The prompt string of the request.
55 prompt_token_ids: The token IDs of the prompt.
56 + prompt_logprobs: The log probabilities to return per prompt token.
57 outputs: The output sequences of the request.
58 finished: Whether the whole request is finished.
59 """