@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 306923da82
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[Bugfix] Fix Qwen2_5_VLForConditionalGeneration packed_modules_mapping (#12905)

Jee Jee Li committed Feb 8, 2025 at 13:02 UTC 306923da82535593bc508cc4e039bdec55159e9f
1 file changed +5 -2
vllm/model_executor/models/qwen2_5_vl.py
+5 -2
@@ -760,9 +760,12 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal,
760 "q_proj",
761 "k_proj",
762 "v_proj",
763 - ]
763 + ],
764 + "gate_up_proj": [
765 + "gate_proj",
766 + "up_proj",
767 + ],
768 }
765 -
769 # LoRA specific attributes, TODO: double check
770 supported_lora_modules = [
771 "qkv_proj",