@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 1a1823871d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Doc] Remove outdated comment to avoid misunderstanding (#9287)

homeffjy committed Oct 12, 2024 at 02:02 UTC 1a1823871d76b9ce54f1c5bc0b61257aa9c53295
1 file changed +3 -4
vllm/core/block_manager_v2.py
+3 -4
@@ -24,9 +24,8 @@ class BlockSpaceManagerV2(BlockSpaceManager):
24 autoregressively-generated tokens, and other advanced features such as
25 prefix caching, forking/copy-on-write, and sliding-window memory allocation.
26
27 - The current implementation is partial; in particular prefix caching and
28 - sliding-window are not feature complete. This class implements the design
29 - described in https://github.com/vllm-project/vllm/pull/3492.
27 + This class implements the design described in
28 + https://github.com/vllm-project/vllm/pull/3492.
29
30 Lookahead slots
31 The block manager has the notion of a "lookahead slot". These are slots
@@ -190,7 +189,7 @@ class BlockSpaceManagerV2(BlockSpaceManager):
189
190 assert (request_id
191 not in self.cross_block_tables), \
193 - "block table already exists"
192 + "block table already exists"
193
194 check_no_caching_or_swa_for_blockmgr_encdec(self, seq_group)
195