A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 16:30 UTC
[Doc] Show that `use_audio_in_video` is supported in docs (#30837)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Jan 6, 2026 at 15:27 UTC
da71d444107dbec33b597faebece6b816fab64eb
4 files changed
-8
docs/models/supported_models.md
-3
@@ -767,9 +767,6 @@ Some models are supported only via the [Transformers modeling backend](#transfor
767
The official `openbmb/MiniCPM-V-2` doesn't work yet, so we need to use a fork (`HwwwH/MiniCPM-V-2`) for now.
768
For more details, please see: <https://github.com/vllm-project/vllm/pull/4087#issuecomment-2250397630>
769
770
-!!! note
771
- For Qwen2.5-Omni and Qwen3-Omni, reading audio from video pre-processing (`--mm-processor-kwargs '{"use_audio_in_video": true}'`) is currently work in progress and not yet supported.
772
-
770
#### Transcription
771
772
Speech2Text models trained specifically for Automatic Speech Recognition.
examples/offline_inference/qwen2_5_omni/README.md
-1
@@ -10,7 +10,6 @@ python examples/offline_inference/qwen2_5_omni/only_thinker.py \
10
-q mixed_modalities
11
12
# Read vision and audio inputs from a single video file
13
-# NOTE: V1 engine does not support interleaved modalities yet.
13
python examples/offline_inference/qwen2_5_omni/only_thinker.py \
14
-q use_audio_in_video
15
vllm/model_executor/models/qwen2_5_omni_thinker.py
-2
@@ -1128,8 +1128,6 @@ class Qwen2_5OmniThinkerForConditionalGeneration(
1128
multimodal_embeddings += tuple(audio_embeddings)
1129
return multimodal_embeddings
1130
1131
- # TODO (ywang96): support overlapping modality embeddings so that
1132
- # `use_audio_in_video` will work on V1.
1131
def embed_input_ids(
1132
self,
1133
input_ids: torch.Tensor,
vllm/model_executor/models/qwen3_omni_moe_thinker.py
-2
@@ -1371,8 +1371,6 @@ class Qwen3OmniMoeThinkerForConditionalGeneration(
1371
return inputs_embeds
1372
1373
deepstack_input_embeds = None
1374
- # TODO (ywang96): support overlapping modalitiy embeddings so that
1375
- # `use_audio_in_video` will work on V1.
1374
# split the feat dim to obtain multi-scale visual feature
1375
has_vision_embeddings = [
1376
embeddings.shape[-1] != self.config.text_config.hidden_size