@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 40d0e7411d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 08:15 UTC

[Doc] Update FAQ links in spec_decode.rst (#9662)

Signed-off-by: whyiug <whyiug@hotmail.com>

whyiug committed Nov 8, 2024 at 12:44 UTC 40d0e7411dbeb276befd33c4485115ac3d4d7f2a
1 file changed +2 -2
docs/source/models/spec_decode.rst
+2 -2
@@ -182,7 +182,7 @@ speculative decoding, breaking down the guarantees into three key areas:
182 3. **vLLM Logprob Stability**
183 - vLLM does not currently guarantee stable token log probabilities (logprobs). This can result in different outputs for the
184 same request across runs. For more details, see the FAQ section
185 - titled *Can the output of a prompt vary across runs in vLLM?* in the `FAQs <../serving/faq.rst>`_.
185 + titled *Can the output of a prompt vary across runs in vLLM?* in the `FAQs <../serving/faq>`_.
186
187
188 **Conclusion**
@@ -197,7 +197,7 @@ can occur due to following factors:
197
198 **Mitigation Strategies**
199
200 -For mitigation strategies, please refer to the FAQ entry *Can the output of a prompt vary across runs in vLLM?* in the `FAQs <../serving/faq.rst>`_.
200 +For mitigation strategies, please refer to the FAQ entry *Can the output of a prompt vary across runs in vLLM?* in the `FAQs <../serving/faq>`_.
201
202 Resources for vLLM contributors
203 -------------------------------