@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 1fe0a4264a
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Bugfix] Fix Token IDs Reference for MiniCPM-V When Images are Provided With No Placeholders (#8991)

Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>

Alex Brooks committed Oct 1, 2024 at 03:52 UTC 1fe0a4264aa94ceeccc7e8d99ac0d72f0560f541
1 file changed +1 -1
vllm/model_executor/models/minicpmv.py
+1 -1
@@ -274,8 +274,8 @@ def input_processor_for_minicpmv(ctx: InputContext, llm_inputs: LLMInputs):
274 get_slice_image_placeholder(image_size, num_image)
275
276 prompt = llm_inputs.get("prompt")
277 + token_ids = llm_inputs.get("prompt_token_ids")
278 if prompt is None:
278 - token_ids = llm_inputs.get("prompt_token_ids")
279 prompt = tokenizer.decode(token_ids)
280
281 pattern = "(<image>./</image>)"