A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 12:15 UTC
[Docs] Improve `priority` parameter documentation (#29572)
Signed-off-by: maang <maang_h@163.com> Signed-off-by: maang-h <55082429+maang-h@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
maang-h committed
Nov 27, 2025 at 18:09 UTC
51906c8c559f1d7c23efa667fcb3b7ed79f7fa25
1 file changed
+3
vllm/entrypoints/llm.py
+3
@@ -405,6 +405,9 @@ class LLM:
405
lora_request: LoRA request to use for generation, if any.
406
priority: The priority of the requests, if any.
407
Only applicable when priority scheduling policy is enabled.
408
+ If provided, must be a list of integers matching the length
409
+ of `prompts`, where each priority value corresponds to the prompt
410
+ at the same index.
411
412
Returns:
413
A list of `RequestOutput` objects containing the