A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 22:15 UTC
[doc] clarify profiling is only for developers (#13554)
Signed-off-by: youkaichao <youkaichao@gmail.com>
youkaichao committed
Feb 19, 2025 at 20:55 UTC
52ce14d31f0645efe51a87de29b5a861056e44db
1 file changed
+4
-4
docs/source/contributing/profiling/profiling_index.md
+4
-4
@@ -1,15 +1,15 @@
1
# Profiling vLLM
2
3
+:::{warning}
4
+Profiling is only intended for vLLM developers and maintainers to understand the proportion of time spent in different parts of the codebase. **vLLM end-users should never turn on profiling** as it will significantly slow down the inference.
5
+:::
6
+
7
We support tracing vLLM workers using the `torch.profiler` module. You can enable tracing by setting the `VLLM_TORCH_PROFILER_DIR` environment variable to the directory where you want to save the traces: `VLLM_TORCH_PROFILER_DIR=/mnt/traces/`
8
9
The OpenAI server also needs to be started with the `VLLM_TORCH_PROFILER_DIR` environment variable set.
10
11
When using `benchmarks/benchmark_serving.py`, you can enable profiling by passing the `--profile` flag.
12
9
-:::{warning}
10
-Only enable profiling in a development environment.
11
-:::
12
-
13
Traces can be visualized using <https://ui.perfetto.dev/>.
14
15
:::{tip}