A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] Add usage of implicit text-only mode (#22561)
Signed-off-by: Roger Wang <hey@rogerw.me> Co-authored-by: Flora Feng <4florafeng@gmail.com>
Roger Wang committed
Aug 8, 2025 at 23:04 UTC
23472ff51cdf25c2f9c9bf9afa50a8d3cc6cc1d8
1 file changed
+3
docs/models/supported_models.md
+3
@@ -583,6 +583,9 @@ See [this page](../features/multimodal_inputs.md) on how to pass multi-modal inp
583
584
**This is no longer required if you are using vLLM V1.**
585
586
+!!! tip
587
+ For hybrid-only models such as Llama-4, Step3 and Mistral-3, a text-only mode can be enabled by setting all supported multimodal modalities to 0 (e.g, `--limit-mm-per-prompt '{"image":0}`) so that their multimodal modules will not be loaded to free up more GPU memory for KV cache.
588
+
589
!!! note
590
vLLM currently only supports adding LoRA to the language backbone of multimodal models.
591