A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs] Fix typos in EP deployment doc (#24669)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Harry Mellor committed
Sep 11, 2025 at 17:07 UTC
51d41265ad841a3b6efea665c83cdc5d54eb7c1d
1 file changed
+2
-2
docs/serving/expert_parallel_deployment.md
+2
-2
@@ -158,10 +158,10 @@ vllm serve Qwen/Qwen3-30B-A3B \
158
159
### Memory Footprint Overhead
160
161
-EPLB uses redundant experts to that need to fit in GPU memory. This means that EPLB may not be a good fit for memory constrained environments or when KV cache space is at a premium.
161
+EPLB uses redundant experts that need to fit in GPU memory. This means that EPLB may not be a good fit for memory constrained environments or when KV cache space is at a premium.
162
163
This overhead equals `NUM_MOE_LAYERS * BYTES_PER_EXPERT * (NUM_TOTAL_EXPERTS + NUM_REDUNDANT_EXPERTS) ÷ NUM_EP_RANKS`.
164
-For DeepSeekV3, this is approximately `2.4 GB` for one redundant expert per rank.
164
+For DeepSeekV3, this is approximately `2.4 GB` for one redundant expert per EP rank.
165
166
### Example Command
167