A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] Update the doc for log probs + prefix caching (#23399)
Signed-off-by: Chen Zhang <zhangch99@outlook.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Chen Zhang committed
Aug 22, 2025 at 06:20 UTC
a073be6d87c6480ecd725bd475cc4f30fd747aa4
1 file changed
+1
-1
docs/usage/v1_guide.md
+1
-1
@@ -166,7 +166,7 @@ Processed means the values after applying all processors, including temperature
166
167
##### Prompt Logprobs with Prefix Caching
168
169
-Currently prompt logprobs are only supported when prefix caching is turned off via `--no-enable-prefix-caching`. In a future release, prompt logprobs will be compatible with prefix caching, but a recomputation will be triggered to recover the full prompt logprobs even upon a prefix cache hit. See details in [RFC #13414](gh-issue:13414).
169
+Logprobs are not cached. For a request requiring prompt logprobs, the engine will ignore the prefix cache and recompute the prefill of full prompt to generate the logprobs.
170
171
#### Deprecated Features
172