A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
[Doc] remove visible token in doc (#17884)
Signed-off-by: yan <yanma1@habana.ai>
Yan Ma committed
May 9, 2025 at 16:21 UTC
ff8c40050234f3319ad4e8afd45fd70e50cb50b3
1 file changed
+1
-1
docs/source/design/v1/prefix_caching.md
+1
-1
@@ -86,7 +86,7 @@ To improve privacy in shared environments, vLLM supports isolating prefix cache
86
{"role": "user", "content": "Here is a document with details about the world series: ..."},
87
{"role": "user", "content": "Who won the world series in 2020?"}
88
],
89
- "cache_salt": "Z3V2bmV3aGxza3ZubGFoZ3Zud3V3ZWZ2bmd0b3V2bnZmc2xpZ3RoZ2x2aQ=="
89
+ "cache_salt": "your-cache-salt"
90
}
91
```
92