@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / cf25b93bdd
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Core] Fix invalid args to _process_request (#9201)

Signed-off-by: Russell Bryant <rbryant@redhat.com>

Russell Bryant committed Oct 10, 2024 at 00:10 UTC cf25b93bddb607077e52cbe4681332ca61aff189
1 file changed +2 -1
vllm/engine/multiprocessing/client.py
+2 -1
@@ -606,7 +606,8 @@ class MQLLMEngineClient:
606 and request_id is not None)
607
608 return self._process_request(prompt, pooling_params, request_id,
609 - lora_request, trace_headers, priority)
609 + lora_request, trace_headers, None,
610 + priority)
611
612 async def _process_request(
613 self,