@tduncan / Nvidia-Optimization-Stack-VLLM / commits / 529d010351

[Doc] Fix typos in speculative decoding docs (#51500)

Signed-off-by: lkm2835 <lkm2835@gmail.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Kyungmin Lee committed Aug 11, 2026 at 17:17 UTC 529d0103511429f0086a308b6ef2698a87fec4b9

4 files changed +8 -8

docs/features/speculative_decoding/README.md

+4 -4

@@ -1,6 +1,6 @@

1 # Speculative Decoding

3 -This document shows how to use [Speculative Decoding](https://arxiv.org/pdf/2302.01318) with vLLM to reduce inter-token latency under medium-to-low QPS (query per second), memory-bound workloads.

3 +This document shows how to use [Speculative Decoding](https://arxiv.org/pdf/2302.01318) with vLLM to reduce inter-token latency under medium-to-low QPS (queries per second), memory-bound workloads.

5 To train your own draft models for optimized speculative decoding, see [vllm-project/speculators](speculators.md) for seamless training and integration with vLLM.

@@ -86,7 +86,7 @@ only apply to model-based methods such as `draft_model`, `mtp`, `eagle3`, and

89 - | `use_heterogeneous_vocab` | `boolean` | `false` | Allow draft and target models with different vocabularies. Builds a token-level intersection at initialisation and constrains draft logits to shared tokens only. Only compatible with `method=draft_model`. Probabilistic draft sampling (`draft_sample_method='probabilistic'`) is not yet supported when this option is enabled. |

89 +| `use_heterogeneous_vocab` | `boolean` | `false` | Allow draft and target models with different vocabularies. Builds a token-level intersection at initialisation and constrains draft logits to shared tokens only. Only compatible with `method=draft_model`. Probabilistic draft sampling (`draft_sample_method='probabilistic'`) is not yet supported when this option is enabled. |

91 !!! note

92 Gemma 4 assistant checkpoints are handled as Gemma 4 MTP speculators, not

@@ -200,7 +200,7 @@ speculative decoding, breaking down the guarantees into three key areas:

200 > distribution. [View Test Code](https://github.com/vllm-project/vllm/blob/47b65a550866c7ffbd076ecb74106714838ce7da/tests/samplers/test_rejection_sampler.py#L252)

201 > - **Greedy Sampling Equality**: Confirms that greedy sampling with speculative decoding matches greedy sampling

202 > without it. This verifies that vLLM's speculative decoding framework, when integrated with the vLLM forward pass and the vLLM rejection sampler,

203 - > provides a lossless guarantee. Almost all of the tests in [tests/spec_decode/e2e](../../../tests/v1/spec_decode).

203 + > provides a lossless guarantee. Almost all of the tests in [tests/spec_decode/e2e](../../../tests/v1/spec_decode)

204 > verify this property using [this assertion implementation](https://github.com/vllm-project/vllm/blob/b67ae00cdbbe1a58ffc8ff170f0c8d79044a684a/tests/spec_decode/e2e/conftest.py#L291)

205

206 3. **vLLM Logprob Stability**

@@ -220,7 +220,7 @@ For mitigation strategies, please refer to the FAQ entry *Can the output of a pr

220 ## Known Feature Incompatibility

221

222 1. Pipeline parallelism is not composable with speculative decoding as of `vllm<=0.15.0`

223 -2. Speculative decoding with a draft models is not supported in `vllm<=0.10.0`

223 +2. Speculative decoding with draft models is not supported in `vllm<=0.10.0`

224

225 ## Resources for vLLM contributors

226

docs/features/speculative_decoding/draft_model.md

+2 -2

@@ -38,7 +38,7 @@ vllm serve Qwen/Qwen3-4B-Thinking-2507 \

38 --speculative-config '{"model": "Qwen/Qwen3-0.6B", "num_speculative_tokens": 5, "method": "draft_model"}'

39 ```

41 -The code used to request as completions as a client remains unchanged:

41 +The code used to request completions as a client remains unchanged:

43 ??? code

@@ -80,7 +80,7 @@ The code used to request as completions as a client remains unchanged:

81 By default, vLLM requires the draft and target models to share the same vocabulary. Setting `use_heterogeneous_vocab: true` enables the **Token-Level Intersection (TLI)** algorithm, which allows draft models from a different model family with a different tokenizer.

83 - Currently,`use_heterogeneous_vocab` currently requires `draft_sample_method='greedy'` (the default). Probabilistic draft sampling is not yet supported and will be added in a

83 + Currently, `use_heterogeneous_vocab` requires `draft_sample_method='greedy'` (the default). Probabilistic draft sampling is not yet supported and will be added in a

84 future release.

86 ```python

docs/features/speculative_decoding/mlp.md

+1 -1

@@ -1,6 +1,6 @@

1 # MLP Draft Models

3 -The following code configures vLLM to use speculative decoding where proposals are generated by draft models that condition draft predictions on both context vectors and sampled tokens. For more information see [The Hitchhiker's Guide to Speculative Decoding](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) and [IBM Research's Technical Report](https://arxiv.org/abs/2404.19124).

3 +The following code configures vLLM to use speculative decoding where proposals are generated by draft models that condition draft predictions on both context vectors and sampled tokens. For more information, see [The Hitchhiker's Guide to Speculative Decoding](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) and [IBM Research's Technical Report](https://arxiv.org/abs/2404.19124).

5 ## MLP Drafter Example

docs/features/speculative_decoding/n_gram.md

+1 -1

@@ -1,7 +1,7 @@

1 # N-Gram Speculation

3 The following code configures vLLM to use speculative decoding where proposals are generated by

4 -matching n-grams in the prompt. For more information read [this thread.](https://x.com/joao_gante/status/1747322413006643259)

4 +matching n-grams in the prompt. For more information, read [this thread](https://x.com/joao_gante/status/1747322413006643259).

6 ```python

7 from vllm import LLM, SamplingParams