@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / e7a8341c7c
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 07:00 UTC

[Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Cyrus Leung committed Nov 22, 2024 at 02:09 UTC e7a8341c7c7481a0c797d50ead7a698255ac8a9f
1 file changed +1 -1
vllm/model_executor/models/qwen2_audio.py
+1 -1
@@ -212,7 +212,7 @@ def input_processor_for_qwen2_audio(
212
213 return token_inputs(
214 prompt_token_ids=new_input_ids,
215 - prompt=inputs['prompt'],
215 + prompt=inputs.get("prompt"),
216 multi_modal_data=multi_modal_data,
217 )
218