A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 08:15 UTC
[UX] Enable torch_profiler_with_stack (#37571)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Jee Jee Li committed
Mar 20, 2026 at 19:17 UTC
dd20ee4e3e873364bd79983dcbb30d2189c96507
2 files changed
+8
-4
docs/contributing/profiling.md
+4
@@ -3,6 +3,10 @@
3
!!! warning
4
Profiling is only intended for vLLM developers and maintainers to understand the proportion of time spent in different parts of the codebase. **vLLM end-users should never turn on profiling** as it will significantly slow down the inference.
5
6
+!!! tip "Choosing a profiler"
7
+ - Use **Nsight Systems** for low-overhead, performance-critical profiling.
8
+ - Use **PyTorch Profiler** for medium-overhead profiling with richer debugging information (e.g., stack traces, memory, shapes). Note that enabling these features adds overhead and is not recommended for benchmarking.
9
+
10
## Profile with PyTorch Profiler
11
12
We support tracing vLLM workers using different profilers. You can enable profiling by setting the `--profiler-config` flag when launching the server.
vllm/config/profiler.py
+4
-4
@@ -45,10 +45,10 @@ class ProfilerConfig:
45
worker's traces (CPU & GPU) will be saved under this directory. Note that
46
it must be an absolute path."""
47
48
- torch_profiler_with_stack: bool = False
49
- """If `True`, enables stack tracing in the torch profiler. Disabled by default
50
- to reduce overhead. Can be enabled via VLLM_TORCH_PROFILER_WITH_STACK=1 env var
51
- or --profiler-config.torch_profiler_with_stack=true CLI flag."""
48
+ torch_profiler_with_stack: bool = True
49
+ """If `True`, enables stack tracing in the torch profiler. Enabled by default
50
+ as it is useful for debugging. Can be disabled via
51
+ --profiler-config.torch_profiler_with_stack=false CLI flag."""
52
53
torch_profiler_with_flops: bool = False
54
"""If `True`, enables FLOPS counting in the torch profiler. Disabled by default."""