A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 22:45 UTC
[Docs] Add profiler user docs for http request (#31370)
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
rongfu.leng committed
Dec 26, 2025 at 23:48 UTC
887e900b773172ddec085149c9bdc702640b1fa4
1 file changed
+23
docs/contributing/profiling.md
+23
@@ -54,6 +54,29 @@ vllm bench serve \
54
--num-prompts 2
55
```
56
57
+Or use http request:
58
+
59
+```shell
60
+# We need first call /start_profile api to start profile.
61
+$ curl -X POST http://localhost:8000/start_profile
62
+
63
+# Call model generate.
64
+curl -X POST http://localhost:8000/v1/chat/completions \
65
+ -H "Content-Type: application/json" \
66
+ -d '{
67
+ "model": "meta-llama/Llama-3.1-8B-Instruct",
68
+ "messages": [
69
+ {
70
+ "role": "user",
71
+ "content": "San Francisco is a"
72
+ }
73
+ ]
74
+ }'
75
+
76
+# After need call /stop_profile api to stop profile.
77
+$ curl -X POST http://localhost:8000/stop_profile
78
+```
79
+
80
## Profile with NVIDIA Nsight Systems
81
82
Nsight systems is an advanced tool that exposes more profiling details, such as register and shared memory usage, annotated code regions and low-level CUDA APIs and events.