@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / df3dcdf49d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Bugfix] Fix priority in multiprocessing engine (#9277)

Sebastian Schoennenbeck committed Oct 11, 2024 at 17:35 UTC df3dcdf49dccfa4914d825fa08b74de8ae050e1e
1 file changed +2 -1
vllm/engine/multiprocessing/engine.py
+2 -1
@@ -285,7 +285,8 @@ class MQLLMEngine:
285 params=request.params,
286 lora_request=request.lora_request,
287 trace_headers=request.trace_headers,
288 - prompt_adapter_request=request.prompt_adapter_request)
288 + prompt_adapter_request=request.prompt_adapter_request,
289 + priority=request.priority)
290
291 if self.log_requests:
292 logger.info("Added request %s.", request.request_id)