A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 12:30 UTC
[docs] Improve tlparse section (#33211)
Signed-off-by: angelayi <yiangela7@gmail.com>
Angela Yi committed
Jan 27, 2026 at 18:07 UTC
fb7abfc1d03c93d07813946d84368002b2cac833
1 file changed
+12
-7
docs/design/debug_vllm_compile.md
+12
-7
@@ -39,10 +39,9 @@ For more details on the design, please see the following resources:
39
40
## Use tlparse
41
42
-Use [tlparse](https://github.com/meta-pytorch/tlparse) to acquire torch.compile logs. These logs show all stages of the compilation process,
43
-including the fused kernels that torch.compile produces.
44
-If you can, we recommend sending these or pieces of these along with any bug reports --
45
-they are very helpful.
42
+Use [tlparse](https://github.com/meta-pytorch/tlparse) to view torch.compile
43
+logs. These logs show all stages of the compilation process, including the fused
44
+kernels that torch.compile produces.
45
46
Install tlparse:
47
@@ -50,11 +49,16 @@ Install tlparse:
49
pip install tlparse
50
```
51
52
+To enable the torch.compile logs, you can set the envvar `TORCH_TRACE=<dir>`.
53
+During tracing, a file per rank will be created inside of that directory, with
54
+each file containing the artifacts during compilation. If you can, we recommend
55
+sending these log files along with bug reports -- they are very helpful.
56
+
57
Usage (offline inference)
58
59
```sh
60
TORCH_TRACE=~/trace_dir python my_script.py
57
-tlparse ~/trace_dir/<the_first_log_file>
61
+tlparse ~/trace_dir/<rank_0_log_file>
62
```
63
64
Usage (serving)
@@ -62,10 +66,11 @@ Usage (serving)
66
```sh
67
TORCH_TRACE=~/trace_dir vllm serve
68
# ctrl-c out of the server
65
-tlparse ~/trace_dir/<the_first_log_file>
69
+tlparse ~/trace_dir/<rank_0_log_file>
70
```
71
68
-The `tlparse` command outputs some HTML files (perhaps into e.g. `./tl_out/index.html`).
72
+Given one of the log files, the `tlparse` command outputs some HTML files
73
+(perhaps into e.g. `./tl_out/index.html`).
74
Open it to see the logs. It'll look something like the following:
75
76
