A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 01:00 UTC
[Doc] Add Realtime Transcription section to supported_models.md (#39845)
Signed-off-by: Ziying Tao <tzzying@outlook.com>
z1ying committed
Apr 17, 2026 at 20:26 UTC
d0697cc7b6825da0ba92aff93b05ea85b4725018
2 files changed
+19
-1
docs/models/supported_models.md
+18
@@ -682,6 +682,24 @@ Speech2Text models trained specifically for Automatic Speech Recognition.
682
!!! note
683
`VoxtralForConditionalGeneration` requires `mistral-common[audio]` to be installed.
684
685
+#### Realtime Transcription
686
+
687
+Speech models that support streaming transcription via the
688
+[`/v1/realtime`](../serving/openai_compatible_server.md#realtime-api)
689
+WebSocket endpoint.
690
+
691
+| Architecture | Models | Example HF Models | [LoRA](../features/lora.md) | [PP](../serving/parallelism_scaling.md) |
692
+| ------------ | ------ | ----------------- | -------------------- | ------------------------- |
693
+| `VoxtralRealtimeGeneration` | Voxtral Realtime | `mistralai/Voxtral-Mini-4B-Realtime-2602` | | |
694
+| `Qwen3ASRRealtimeGeneration` | Qwen3-ASR Realtime | `Qwen/Qwen3-ASR-0.6B` | | |
695
+
696
+!!! note
697
+ `VoxtralRealtimeGeneration` requires `mistral-common[audio]` to be installed, and must be served with `--tokenizer-mode mistral`.
698
+
699
+ `Qwen3ASRRealtimeGeneration` is not auto-detected from `config.json`.
700
+ You must pass `--hf-overrides '{"architectures":["Qwen3ASRRealtimeGeneration"]}'`
701
+ when serving.
702
+
703
## Pooling Models
704
705
See [this page](pooling_models/README.md) for more information on how to use pooling models.
docs/serving/openai_compatible_server.md
+1
-1
@@ -60,7 +60,7 @@ We currently support the following OpenAI APIs:
60
- [Translation API](#translations-api) (`/v1/audio/translations`)
61
- Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#transcription).
62
- [Realtime API](#realtime-api) (`/v1/realtime`)
63
- - Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#transcription).
63
+ - Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#realtime-transcription).
64
65
In addition, we have the following custom APIs:
66