A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
[Doc] Remove outdated comment to avoid misunderstanding (#9287)
homeffjy committed
Oct 12, 2024 at 02:02 UTC
1a1823871d76b9ce54f1c5bc0b61257aa9c53295
1 file changed
+3
-4
vllm/core/block_manager_v2.py
+3
-4
@@ -24,9 +24,8 @@ class BlockSpaceManagerV2(BlockSpaceManager):
24
autoregressively-generated tokens, and other advanced features such as
25
prefix caching, forking/copy-on-write, and sliding-window memory allocation.
26
27
- The current implementation is partial; in particular prefix caching and
28
- sliding-window are not feature complete. This class implements the design
29
- described in https://github.com/vllm-project/vllm/pull/3492.
27
+ This class implements the design described in
28
+ https://github.com/vllm-project/vllm/pull/3492.
29
30
Lookahead slots
31
The block manager has the notion of a "lookahead slot". These are slots
@@ -190,7 +189,7 @@ class BlockSpaceManagerV2(BlockSpaceManager):
189
190
assert (request_id
191
not in self.cross_block_tables), \
193
- "block table already exists"
192
+ "block table already exists"
193
194
check_no_caching_or_swa_for_blockmgr_encdec(self, seq_group)
195