A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 06:30 UTC
[Doc] Fix GPU Worker count in Process Count Summary (#36000)
Signed-off-by: simone-dotolo <simonedotolo@libero.it> Signed-off-by: simone-dotolo <84937474+simone-dotolo@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
simone-dotolo committed
Mar 4, 2026 at 18:03 UTC
e86221deb6859c28325097f4568e6d553ae92e8d
1 file changed
+1
-1
docs/design/arch_overview.md
+1
-1
@@ -122,7 +122,7 @@ For a deployment with `N` GPUs, `TP` tensor parallel size, `DP` data parallel si
122
|---|---|---|
123
| API Server | `A` (default `DP`) | Handles HTTP requests and input processing |
124
| Engine Core | `DP` (default 1) | Scheduler and KV cache management |
125
-| GPU Worker | `N` (= `DP x TP`) | One per GPU, executes model forward passes |
125
+| GPU Worker | `N` (= `DP x PP x TP`) | One per GPU, executes model forward passes |
126
| DP Coordinator | 1 if `DP > 1`, else 0 | Load balancing across DP ranks |
127
| **Total** | **`A + DP + N` (+ 1 if DP > 1)** | |
128