A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 1, 18:45 UTC
[Doc] Fix multimodal torch.compile troubleshooting to not use removed VLLM_TORCH_COMPILE_LEVEL (#44378)
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Daoyuan Li committed
Jun 7, 2026 at 00:34 UTC
15652a6b703f059767d75e5a511c4a5accf969ad
1 file changed
+1
-1
docs/design/torch_compile_multimodal.md
+1
-1
@@ -88,7 +88,7 @@ If compilation fails for a multimodal model:
88
89
1. **Disable and test**: First verify the model works without compilation:
90
```bash
91
- VLLM_TORCH_COMPILE_LEVEL=0 vllm serve <model> --compilation-config='{"compile_mm_encoder":"false"}'
91
+ vllm serve <model> --compilation-config='{"mode":0,"compile_mm_encoder":"false"}'
92
```
93
94
2. **Check logs**: Enable debug logging to see compilation details: