A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] Fix Markdown Pre-commit Error (#24670)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Wentao Ye committed
Sep 11, 2025 at 12:05 UTC
4984a291d55bf6ec6b60c198cd1398315a9a2958
1 file changed
+5
-6
docs/contributing/model/transcription.md
+5
-6
@@ -37,7 +37,7 @@ It is assumed you have already implemented your model in vLLM according to the b
37
- The `supported_languages` mapping is validated at init time.
38
- Set `supports_transcription_only=True` if the model should not serve text generation (eg Whisper).
39
40
-- Provide an ASR configuration via [get_speech_to_text_config][vllm.model_executor.models.interfaces.SupportsTranscription.get_speech_to_text_config].
40
+- Provide an ASR configuration via [get_speech_to_text_config][vllm.model_executor.models.interfaces.SupportsTranscription.get_speech_to_text_config].
41
This is for controlling general behavior of the API when serving your model:
42
43
??? code
@@ -65,7 +65,7 @@ It is assumed you have already implemented your model in vLLM according to the b
65
66
- Implement the prompt construction via [get_generation_prompt][vllm.model_executor.models.interfaces.SupportsTranscription.get_generation_prompt]. The server passes you the resampled waveform and task parameters; you return a valid [PromptType][vllm.inputs.data.PromptType]. There are two common patterns:
67
68
- #### A. Multimodal LLM with audio embeddings (e.g., Voxtral, Gemma3n)
68
+### A. Multimodal LLM with audio embeddings (e.g., Voxtral, Gemma3n)
69
70
Return a dict containing `multi_modal_data` with the audio, and either a `prompt` string or `prompt_token_ids`:
71
@@ -102,7 +102,7 @@ It is assumed you have already implemented your model in vLLM according to the b
102
103
For further clarification on multi modal inputs, please refer to [Multi-Modal Inputs](../../features/multimodal_inputs.md).
104
105
- #### B. Encoder–decoder audio-only (e.g., Whisper)
105
+### B. Encoder–decoder audio-only (e.g., Whisper)
106
107
Return a dict with separate `encoder_prompt` and `decoder_prompt` entries:
108
@@ -142,7 +142,6 @@ It is assumed you have already implemented your model in vLLM according to the b
142
return cast(PromptType, prompt)
143
```
144
145
-
145
- (Optional) Language validation via [validate_language][vllm.model_executor.models.interfaces.SupportsTranscription.validate_language]
146
147
If your model requires a language and you want a default, override this method (see Whisper):
@@ -177,7 +176,6 @@ It is assumed you have already implemented your model in vLLM according to the b
176
return int(audio_duration_s * stt_config.sample_rate // 320) # example
177
```
178
180
-
179
## 2. Audio preprocessing and chunking
180
181
The API server takes care of basic audio I/O and optional chunking before building prompts:
@@ -264,10 +262,11 @@ Once your model implements `SupportsTranscription`, you can test the endpoints (
262
-F "model=$MODEL_ID" \
263
http://localhost:8000/v1/audio/translations
264
```
265
+
266
Or check out more examples in <gh-file:examples/online_serving>.
267
268
!!! note
269
270
- If your model handles chunking internally (e.g., via its processor or encoder), set `min_energy_split_window_size=None` in the returned `SpeechToTextConfig` to disable server-side chunking.
271
- Implementing `get_num_audio_tokens` improves accuracy of streaming usage metrics (`prompt_tokens`) without an extra forward pass.
273
-- For multilingual behavior, keep `supported_languages` aligned with actual model capabilities.
\ No newline at end of file
272
+- For multilingual behavior, keep `supported_languages` aligned with actual model capabilities.