@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 870c37481e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[V1][Minor] Remove outdated comment (#12968)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Woosuk Kwon committed Feb 8, 2025 at 12:48 UTC 870c37481e4d9dbcd548344e1eee6bd83993388a
1 file changed -2
vllm/v1/core/kv_cache_manager.py
-2
@@ -205,8 +205,6 @@ class KVCacheManager:
205 # Should not exceed the maximum number of blocks per request.
206 # This is especially because the block table has the shape
207 # [..., max_num_blocks_per_req].
208 - # TODO(woosuk): Check and reject requests if
209 - # num_prompt_tokens + max_tokens > max_model_len.
208 self.max_num_blocks_per_req - len(req_blocks),
209 )
210 assert num_new_blocks > 0