@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 985b4a2b19
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[Bugfix] Fix num video tokens calculation for Qwen2-VL (#13148)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Cyrus Leung committed Feb 12, 2025 at 19:55 UTC 985b4a2b1989b2879809d6a3b84c11ac9e1171a3
1 file changed +5 -1
vllm/model_executor/models/qwen2_vl.py
+5 -1
@@ -800,7 +800,11 @@ class Qwen2VLProcessingInfo(BaseProcessingInfo):
800 preprocessed_size = ImageSize(width=image_width,
801 height=image_height)
802
803 - grid_t = max(num_frames // temporal_patch_size, 1)
803 + # NOTE: Frames are padded to be divisible by `temporal_patch_size`
804 + # https://github.com/huggingface/transformers/blob/v4.48.3/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L294
805 + padded_num_frames = num_frames + num_frames % temporal_patch_size
806 +
807 + grid_t = max(padded_num_frames // temporal_patch_size, 1)
808 grid_h = preprocessed_size.height // patch_size
809 grid_w = preprocessed_size.width // patch_size
810