A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 21:00 UTC
prefix_caching.md: Fixed typo (#14293)
Signed-off-by: Daivid Savernin-Frenk <daivid.frank@TurboNext.ai>
DaividFrank committed
Mar 5, 2025 at 16:43 UTC
8f808cf86e7c050c8597ce0ea88e5c1e0023bb4b
1 file changed
+1
-1
docs/source/design/v1/prefix_caching.md
+1
-1
@@ -183,7 +183,7 @@ When a request is finished, we free all its blocks if no other requests are usin
183
184
When the head block (least recently used block) of the free queue is cached, we have to evict the block to prevent it from being used by other requests. Specifically, eviction involves the following steps:
185
186
-1. Pop the block from the head of the free queue. This is the LRU black to be evicted.
186
+1. Pop the block from the head of the free queue. This is the LRU block to be evicted.
187
2. Remove the block ID from the Cache Block.
188
3. Remove the block hash.
189