@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 7c04779afa

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 14, 05:30 UTC

[Doc]: fix various spelling issues in multiple files (#23636)

Signed-off-by: Didier Durand <durand.didier@gmail.com>

Didier Durand committed Aug 26, 2025 at 16:05 UTC 7c04779afa7d0811dba3e1ec98c0ac1bc56570be

6 files changed +10 -10

.buildkite/nightly-benchmarks/README.md

+1 -1

@@ -141,7 +141,7 @@ When run, benchmark script generates results under `benchmark/results` folder, a

141 `compare-json-results.py` compares two `benchmark_results.json` files and provides performance ratio e.g. for Output Tput, Median TTFT and Median TPOT.

142 If only one benchmark_results.json is passed, `compare-json-results.py` compares different TP and PP configurations in the benchmark_results.json instead.

143

144 -Here is an example using the script to compare result_a and result_b with Model, Dataset name, input/output lenght, max concurrency and qps.

144 +Here is an example using the script to compare result_a and result_b with Model, Dataset name, input/output length, max concurrency and qps.

145 `python3 compare-json-results.py -f results_a/benchmark_results.json -f results_b/benchmark_results.json`

146

benchmarks/README.md

+1 -1

@@ -749,7 +749,7 @@ vllm serve Qwen/Qwen2.5-VL-3B-Instruct \

749

750 Benchmark. It is recommended to use the flag `--ignore-eos` to simulate real responses. You can set the size of the output via the arg `random-output-len`.

751

752 -Ex.1: Fixed number of items and a single image resolutionm, enforcing generation of approx 40 tokens:

752 +Ex.1: Fixed number of items and a single image resolution, enforcing generation of approx 40 tokens:

753

754 ```bash

755 vllm bench serve \

docs/configuration/optimization.md

+2 -2

@@ -168,7 +168,7 @@ llm = LLM(

168 Batch-level DP is not to be confused with API request-level DP

169 (which is instead controlled by `data_parallel_size`).

170

171 -The availablilty of batch-level DP is based on model implementation.

171 +The availability of batch-level DP is based on model implementation.

172 Currently, the following models support `mm_encoder_tp_mode="data"`:

173

174 - Llama4 (<gh-pr:18368>)

@@ -205,7 +205,7 @@ vllm serve Qwen/Qwen2.5-VL-3B-Instruct --api-server-count 4 -dp 2

205

206 !!! note

207 [Multi-modal processor cache](#processor-cache) is disabled when API server scale-out is enabled

208 - because it requires a one-to-one correspondance between API and engine core processes.

208 + because it requires a one-to-one correspondence between API and engine core processes.

209

210 ## Multi-Modal Caching

211

docs/configuration/tpu.md

+1 -1

@@ -70,7 +70,7 @@ For example, max_model_len=512, padding_gap=64, the buckets will be [16, 32, 64,

71 The fewer tokens we pad, the less unnecessary computation TPU does, the better performance we can get. For example, if num_tokens=300, with exponential padding, we pad to 512, with the bucket_padding above, we pad to 320.

73 -However, you need to be careful to choose the padding gap. If the gap is too small, it means the number of buckets is large, leading to increased warmup (precompile) time and higher memory to store the compiled graph. Too many compilaed graphs may lead to HBM OOM. Conversely, an overly large gap yields no performance improvement compared to the default exponential padding.

73 +However, you need to be careful to choose the padding gap. If the gap is too small, it means the number of buckets is large, leading to increased warmup (precompile) time and higher memory to store the compiled graph. Too many compiled graphs may lead to HBM OOM. Conversely, an overly large gap yields no performance improvement compared to the default exponential padding.

75 #### Quantization

docs/design/fused_moe_modular_kernel.md

+3 -3

@@ -133,7 +133,7 @@ class FusedMoEModularKernel:

133 Typically a FusedMoEPrepareAndFinalize type is backed by an All2All Dispatch & Combine implementation / kernel. For example,

134

135 * PplxPrepareAndFinalize type is backed by Pplx All2All kernels,

136 -* DeepEPHTPrepareAndFinalize type is backed by DeepEP High-Throughtput All2All kernels, and

136 +* DeepEPHTPrepareAndFinalize type is backed by DeepEP High-Throughput All2All kernels, and

137 * DeepEPLLPrepareAndFinalize type is backed by DeepEP Low-Latency All2All kernels.

138

139 #### Step 1: Add an All2All manager

@@ -183,7 +183,7 @@ implementations that input `FusedMoEActivationFormat.Standard` support chunking

183

184 #### maybe_make_prepare_finalize

185

186 -The `maybe_make_prepare_finalize` method is responsbile for constructing an instance of `FusedMoEPrepareAndFinalize` when appropriate based on the current all2all backend, e.g. when EP + DP is enabled. The base class method currently constructs all the `FusedMoEPrepareAndFinalize` objects for the EP+DP case. Derived classes can override this method to construct prepare/finalize objects for different scenarios, e.g. `ModelOptNvFp4FusedMoE` can construct a `FlashInferCutlassMoEPrepareAndFinalize` for the EP+TP case.

186 +The `maybe_make_prepare_finalize` method is responsible for constructing an instance of `FusedMoEPrepareAndFinalize` when appropriate based on the current all2all backend, e.g. when EP + DP is enabled. The base class method currently constructs all the `FusedMoEPrepareAndFinalize` objects for the EP+DP case. Derived classes can override this method to construct prepare/finalize objects for different scenarios, e.g. `ModelOptNvFp4FusedMoE` can construct a `FlashInferCutlassMoEPrepareAndFinalize` for the EP+TP case.

187 Please refer to the implementations in,

188

189 * `ModelOptNvFp4FusedMoE`

@@ -198,7 +198,7 @@ Please refer to the implementations in,

198 * `CompressedTensorsW8A8Fp8MoECutlassMethod`

199 * `Fp8MoEMethod`

200 * `ModelOptNvFp4FusedMoE`

201 -dervied classes.

201 +derived classes.

202

203 #### init_prepare_finalize

204

vllm/distributed/kv_transfer/README.md

+2 -2

@@ -2,7 +2,7 @@

2 # Distributed KV cache transfer

4 This folder implements distributed KV cache transfer across vLLM instances.

5 -Currently the main usecase is for disaggregated prefilling.

5 +Currently the main use case is for disaggregated prefilling.

7 ## Abstractions

@@ -14,7 +14,7 @@ The KV cache transfer contains three layer of abstractions:

15 Why we need KV lookup buffer: FIFO pipe itself is not enough as prefill vLLM worker may process requests in a different order compared to decode vLLM worker. Say the QPS is really high, prefill worker may handle requests in order A -> B -> C, but the decode worker may process request C first. This is not the case that can be naturally handled by FIFO pipe, so we provide KV lookup buffer to help translate a FIFO pipe to a lookup buffer.

17 -NOTE: KV pipe layer is bypassible: you can skip this layer if your distributed

17 +NOTE: KV pipe layer is bypassable: you can skip this layer if your distributed

18 communication service already supports key-value-based lookup (like redis or

19 RDMA database).