@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 969da7d70b
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[V1][VLM] Fix edge case bug for InternVL2 (#11165)

Signed-off-by: Roger Wang <ywang@roblox.com>

Roger Wang committed Dec 13, 2024 at 03:09 UTC 969da7d70bc0539f6be12027b71bef758325a61a
1 file changed +4 -1
vllm/model_executor/models/internvl.py
+4 -1
@@ -669,8 +669,11 @@ class InternVLChatModel(nn.Module, SupportsMultiModal, SupportsPP):
669 image_embeds = self.extract_feature(image_input["data"])
670
671 patches_per_image = image_input["patches_per_image"]
672 +
673 + # Only one image in the current batch
674 if len(patches_per_image) == 1:
673 - image_embeds = image_embeds.unsqueeze(0)
675 + image_embeds = image_embeds.view(
676 + -1, self.config.text_config.hidden_size).unsqueeze(0)
677 return image_embeds
678
679 # NOTE: Image embeddings are split into separate tensors for each image