@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / dd2b5633dd
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[V1][Bugfix] Skip hashing empty or None mm_data (#11386)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Woosuk Kwon committed Dec 21, 2024 at 14:22 UTC dd2b5633dd5fb0ecb5fb7247351ebedc1d69d054
1 file changed +4
vllm/v1/engine/mm_input_mapper.py
+4
@@ -180,6 +180,10 @@ class MMHasher:
180 return None
181
182 mm_data = prompt["multi_modal_data"]
183 + if not mm_data:
184 + # mm_data can be None or an empty dict.
185 + return None
186 +
187 image_inputs = mm_data["image"]
188
189 return self.hash_images(image_inputs)