@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 63fed55506

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 13, 02:00 UTC

[Doc]: fix typos in various files (#28811)

Signed-off-by: Didier Durand <durand.didier@gmail.com>

Didier Durand committed Nov 16, 2025 at 15:30 UTC 63fed5550609b96b578d2512aefced09efe76e1e

7 files changed +10 -10

docs/contributing/benchmarks.md

+1 -1

@@ -983,7 +983,7 @@ each document has close to 512 tokens.

983

984 Please note that the `/v1/rerank` is also supported by embedding models. So if you're running

985 with an embedding model, also set `--no_reranker`. Because in this case the query is

986 -treated as a individual prompt by the server, here we send `random_batch_size - 1` documents

986 +treated as an individual prompt by the server, here we send `random_batch_size - 1` documents

987 to account for the extra prompt which is the query. The token accounting to report the

988 throughput numbers correctly is also adjusted.

989

docs/design/cuda_graphs.md

+1 -1

@@ -128,7 +128,7 @@ A [CUDAGraphWrapper][vllm.compilation.cuda_graph.CUDAGraphWrapper] instance wrap

128 3. Otherwise, i.e., the runtime_mode matches the mode of the wrapper, the wrapper will perform CUDA Graphs capture (if key does not exist, create

129 a new entry and cache it) or replay (if key exists in the cache).

130

131 -The above steps are based on the assumption that the CUDA Graphs wrapper would directly trust what’s in the forward context (controlled by the dispatcher). This lets us simplify and cenralize the logic, reducing the complexity as well as the risk of mismatched state between the wrappers and the dispatcher. It also allows reusing the wrapper class for both `FULL` and `PIECEWISE` runtime modes. See the implementation [here](https://github.com/vllm-project/vllm/blob/f751e50b7a2aae3110d83ed0d88202fc91b3e78a/vllm/compilation/cuda_graph.py#L106).

131 +The above steps are based on the assumption that the CUDA Graphs wrapper would directly trust what’s in the forward context (controlled by the dispatcher). This lets us simplify and centralize the logic, reducing the complexity as well as the risk of mismatched state between the wrappers and the dispatcher. It also allows reusing the wrapper class for both `FULL` and `PIECEWISE` runtime modes. See the implementation [here](https://github.com/vllm-project/vllm/blob/f751e50b7a2aae3110d83ed0d88202fc91b3e78a/vllm/compilation/cuda_graph.py#L106).

132

133 #### Nested Wrapper design

134

docs/features/custom_arguments.md

+1 -1

@@ -5,7 +5,7 @@ You can use vLLM *custom arguments* to pass in arguments which are not part of t

5 Custom arguments can be useful if, for example, you want to use a [custom logits processor](./custom_logitsprocs.md) without modifying the vLLM source code.

7 !!! note

8 - Make sure your custom logits processor have implemented `validate_params` for custom arguments. Otherwise invalid custom arguments can cause unexpected behaviour.

8 + Make sure your custom logits processor have implemented `validate_params` for custom arguments. Otherwise, invalid custom arguments can cause unexpected behaviour.

10 ## Offline Custom Arguments

docs/features/custom_logitsprocs.md

+4 -4

@@ -71,7 +71,7 @@ Logits processor `update_state()` implementations should assume the following mo

72 * **"Condense" the batch to be contiguous:** starting with the lowest-index empty slot (which was caused by a Remove), apply a Unidirectional Move from the current highest non-empty slot in the batch to fill the empty slot. Proceed with additional Unidirectional Move operations in order of increasing empty slot destination index and decreasing non-empty slot source index until the batch is contiguous

74 - * **Shrink the batch:** a side-effect of condensing the batch is that empty slots resulting from Remove operations are grouped in a contiguous block at the end of the batch array. Thus, after condensing, update `BatchUpdate.batch_size` to reflect the number of non-empty slots

74 + * **Shrink the batch:** a side effect of condensing the batch is that empty slots resulting from Remove operations are grouped in a contiguous block at the end of the batch array. Thus, after condensing, update `BatchUpdate.batch_size` to reflect the number of non-empty slots

76 5. Reorder the batch for improved efficiency. Depending on the attention backend implementation and the current characteristics of the batch, zero or more Swap Move operations may be applied to reorder the batch

@@ -286,7 +286,7 @@ Once you have created a custom subclass (like `WrappedPerReqLogitsProcessor`) wh

286

287 ## Ways to Load Your Custom Logits Processor in vLLM

288

289 -Logits processors are loaded at initialization. Critically, the set of loaded logits processors cannot be modified after the vLLM engine finishes loading, and new logits logits processors cannot be loaded on-demand for individual requests.

289 +Logits processors are loaded at initialization. Critically, the set of loaded logits processors cannot be modified after the vLLM engine finishes loading, and new logits processors cannot be loaded on-demand for individual requests.

290

291 This section details different ways of making your logits processor visible to vLLM and triggering vLLM to load your logits processor.

292

@@ -438,7 +438,7 @@ The examples below show how a user would pass a custom argument (`target_token`)

438

439 ## Best Practices for Writing Custom Logits Processors

440

441 -Once vLLM loads a logits processor during initialization, then vLLM will invoke `update_state()` and `apply()` against that logits processor in every engine step. Both methods operate on all requests which currently reside in the vLLM persistent batch. Thus it is important to implement these methods efficiently.

441 +Once vLLM loads a logits processor during initialization, then vLLM will invoke `update_state()` and `apply()` against that logits processor in every engine step. Both methods operate on all requests which currently reside in the vLLM persistent batch. Thus, it is important to implement these methods efficiently.

442

443 * Write efficient `apply()` and `update_state()` implementations in light of the fact that logits processors operate at batch granularity

444 * For example, you may be able to use efficient vectorized operations to implement `apply()` or update internal state vectors in `update_state()`

@@ -465,4 +465,4 @@ Once vLLM loads a logits processor during initialization, then vLLM will invoke

465

466 * **Note:** for wrapped per-request logits processors, the `AdapterLogitsProcessor` base-class handles this by default

467

468 -* `is_argmax_invariant()` can be hard-coded to `True` or `False` if the logits processor has consistent behavior. However the argmax invariance may also be determined programmatically (i.e. if your logits processor is user-customizable in some way that impacts whether the logits processor is argmax invariant). For this reason, `is_argmax_invariant()` is not a class method

468 +* `is_argmax_invariant()` can be hard-coded to `True` or `False` if the logits processor has consistent behavior. However, the argmax invariance may also be determined programmatically (i.e. if your logits processor is user-customizable in some way that impacts whether the logits processor is argmax invariant). For this reason, `is_argmax_invariant()` is not a class method

docs/getting_started/installation/cpu.md

+1 -1

@@ -104,7 +104,7 @@ Currently, there are no pre-built CPU wheels.

104

105 ### Which `dtype` should be used?

106

107 -- Currently vLLM CPU uses model default settings as `dtype`. However, due to unstable float16 support in torch CPU, it is recommended to explicitly set `dtype=bfloat16` if there are any performance or accuracy problem.

107 +- Currently, vLLM CPU uses model default settings as `dtype`. However, due to unstable float16 support in torch CPU, it is recommended to explicitly set `dtype=bfloat16` if there are any performance or accuracy problem.

108

109 ### How to launch a vLLM service on CPU?

110

docs/getting_started/installation/cpu.s390x.inc.md

+1 -1

@@ -2,7 +2,7 @@

3 vLLM has experimental support for s390x architecture on IBM Z platform. For now, users must build from source to natively run on IBM Z platform.

5 -Currently the CPU implementation for s390x architecture supports FP32 datatype only.

5 +Currently, the CPU implementation for s390x architecture supports FP32 datatype only.

7 !!! warning

8 There are no pre-built wheels or images for this device, so you must build vLLM from source.

docs/getting_started/installation/cpu.x86.inc.md

+1 -1

@@ -83,7 +83,7 @@ uv pip install dist/*.whl

83 !!! example "Troubleshooting"

84 - **NumPy ≥2.0 error**: Downgrade using `pip install "numpy<2.0"`.

85 - **CMake picks up CUDA**: Add `CMAKE_DISABLE_FIND_PACKAGE_CUDA=ON` to prevent CUDA detection during CPU builds, even if CUDA is installed.

86 - - `AMD` requies at least 4th gen processors (Zen 4/Genoa) or higher to support [AVX512](https://www.phoronix.com/review/amd-zen4-avx512) to run vLLM on CPU.

86 + - `AMD` requires at least 4th gen processors (Zen 4/Genoa) or higher to support [AVX512](https://www.phoronix.com/review/amd-zen4-avx512) to run vLLM on CPU.

87 - If you receive an error such as: `Could not find a version that satisfies the requirement torch==X.Y.Z+cpu+cpu`, consider updating [pyproject.toml](https://github.com/vllm-project/vllm/blob/main/pyproject.toml) to help pip resolve the dependency.

88 ```toml title="pyproject.toml"

89 [build-system]