A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
[doc] fix ordered list issue (#20819)
Signed-off-by: reidliu41 <reid201711@gmail.com>
Reid committed
Jul 11, 2025 at 21:49 UTC
6fb162447bb2eba84cb23c59dfa37321ba89cb53
1 file changed
+6
-6
docs/features/spec_decode.md
+6
-6
@@ -256,12 +256,12 @@ speculative decoding, breaking down the guarantees into three key areas:
256
2. **Algorithmic Losslessness**
257
\- vLLM’s implementation of speculative decoding is algorithmically validated to be lossless. Key validation tests include:
258
259
- > - **Rejection Sampler Convergence**: Ensures that samples from vLLM’s rejection sampler align with the target
260
- > distribution. [View Test Code](https://github.com/vllm-project/vllm/blob/47b65a550866c7ffbd076ecb74106714838ce7da/tests/samplers/test_rejection_sampler.py#L252)
261
- > - **Greedy Sampling Equality**: Confirms that greedy sampling with speculative decoding matches greedy sampling
262
- > without it. This verifies that vLLM's speculative decoding framework, when integrated with the vLLM forward pass and the vLLM rejection sampler,
263
- > provides a lossless guarantee. Almost all of the tests in <gh-dir:tests/spec_decode/e2e>.
264
- > verify this property using [this assertion implementation](https://github.com/vllm-project/vllm/blob/b67ae00cdbbe1a58ffc8ff170f0c8d79044a684a/tests/spec_decode/e2e/conftest.py#L291)
259
+ > - **Rejection Sampler Convergence**: Ensures that samples from vLLM’s rejection sampler align with the target
260
+ > distribution. [View Test Code](https://github.com/vllm-project/vllm/blob/47b65a550866c7ffbd076ecb74106714838ce7da/tests/samplers/test_rejection_sampler.py#L252)
261
+ > - **Greedy Sampling Equality**: Confirms that greedy sampling with speculative decoding matches greedy sampling
262
+ > without it. This verifies that vLLM's speculative decoding framework, when integrated with the vLLM forward pass and the vLLM rejection sampler,
263
+ > provides a lossless guarantee. Almost all of the tests in <gh-dir:tests/spec_decode/e2e>.
264
+ > verify this property using [this assertion implementation](https://github.com/vllm-project/vllm/blob/b67ae00cdbbe1a58ffc8ff170f0c8d79044a684a/tests/spec_decode/e2e/conftest.py#L291)
265
266
3. **vLLM Logprob Stability**
267
\- vLLM does not currently guarantee stable token log probabilities (logprobs). This can result in different outputs for the