@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 176fcb1c71
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 08:15 UTC

[Bugfix] Fix QwenModel argument (#10262)

Signed-off-by: Jie Fu <jiefu@tencent.com>

Jie Fu (傅杰) committed Nov 13, 2024 at 00:36 UTC 176fcb1c71655d825d2363e5f1468fa248fe783b
1 file changed +2 -2
vllm/model_executor/models/qwen.py
+2 -2
@@ -1068,7 +1068,7 @@ class QWenLMHeadModel(QWenBaseModel, SupportsLoRA):
1068 config = vllm_config.model_config.hf_config
1069 # Initialize VL
1070 if hasattr(config, "visual"):
1071 - return QWenVL(vllm_config)
1071 + return QWenVL(vllm_config=vllm_config)
1072 # Initialize LLM
1073 else:
1074 - return QWenLLM(vllm_config)
1074 + return QWenLLM(vllm_config=vllm_config)