A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 21:48 UTC
[Documentation] Fix typo in `docs/design/torch_compile_multimodal.md` (#32741)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Lucas Kabela committed
Jan 20, 2026 at 23:54 UTC
c80f92c14d5e6c52691f586052af68d1495aac74
1 file changed
+5
-5
docs/design/torch_compile_multimodal.md
+5
-5
@@ -11,14 +11,14 @@ to new models to improve performance.
11
12
## Overview
13
14
-We have recently enabled the `@supports_torch_compile` decorator to work for multiple nn module components within a model type; this enables
14
+We have recently enabled the `@support_torch_compile` decorator to work for multiple nn module components within a model type; this enables
15
turning compile on for multimodal encoders, bringing performance improvements to additional components of the stack.
16
17
When applied to the vision block of [`Qwen2_5_vl`](https://github.com/vllm-project/vllm/pull/23207) we observe ~4.5% e2e perf improvements with
18
some increase in compilation time
19
20
This feature is off by default, but can be enabled by setting `compile_mm_encoder: true` in the compilation config when models have the
21
-`@supports_torch_compile` decorator.
21
+`@support_torch_compile` decorator.
22
23
## How Compilation Works for Multimodal Components
24
@@ -26,7 +26,7 @@ This feature is off by default, but can be enabled by setting `compile_mm_encode
26
27
To compile a multimodal component such as an encoder, we follow the same mechanism as the LLM text backbone, with a few additional scaffoldings:
28
29
-1. The `@supports_torch_compile` decorator should include `enable_if=should_torch_compile_mm_vit`. This will gate the compilation behind our
29
+1. The `@support_torch_compile` decorator should include `enable_if=should_torch_compile_mm_vit`. This will gate the compilation behind our
30
`compile_mm_encoder` configuration
31
32
2. `with set_model_tag("<component_name>", is_encoder=True)` context manager should be used around the nn.Module's instantiation. Since torch.compile
@@ -44,9 +44,9 @@ this for more configuration in the future.
44
45
## Applying torch.compile to a New Multimodal Model/Component
46
47
-To apply `supports_torch_compile` to a new general nn.Module, we advise following the same steps in [`debug_vllm_compile`](./debug_vllm_compile.md); this includes:
47
+To apply `support_torch_compile` to a new general nn.Module, we advise following the same steps in [`debug_vllm_compile`](./debug_vllm_compile.md); this includes:
48
49
-1. Applying `supports_torch_compile` on initially small modules (such as basic MLP layers), then raising to more general modules until one reaches a good performance
49
+1. Applying `support_torch_compile` on initially small modules (such as basic MLP layers), then raising to more general modules until one reaches a good performance
50
tradeoff
51
52
2. Leveraging [`tlparse`](https://github.com/meta-pytorch/tlparse) to identify and eliminate the source of recompiles and graph breaks