A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
Document multi-proc method selection for profiling (#23802)
Signed-off-by: jdebache <jdebache@nvidia.com>
Julien Debache committed
Sep 1, 2025 at 15:28 UTC
41c80698b3849969dcb5c5e40d0991b0eb4821cc
1 file changed
+2
docs/contributing/profiling.md
+2
@@ -73,6 +73,8 @@ apt install nsight-systems-cli
73
74
### Example commands and usage
75
76
+When profiling with `nsys`, it is advisable to set the environment variable `VLLM_WORKER_MULTIPROC_METHOD=spawn`. The default is to use the `fork` method instead of `spawn`. More information on the topic can be found in the [Nsight Systems release notes](https://docs.nvidia.com/nsight-systems/ReleaseNotes/index.html#general-issues).
77
+
78
#### Offline Inference
79
80
For basic usage, you can just append `nsys profile -o report.nsys-rep --trace-fork-before-exec=true --cuda-graph-trace=node` before any existing script you would run for offline inference.