@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / b21f0f9d17
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[V1][Minor] Remove outdated comment (#12928)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Woosuk Kwon committed Feb 7, 2025 at 19:07 UTC b21f0f9d173e5094791815380cb213f9eed44bad
1 file changed +1 -3
vllm/v1/core/kv_cache_manager.py
+1 -3
@@ -299,9 +299,7 @@ class KVCacheManager:
299
300 While all scheduled requests must be in the RUNNING state, the inverse
301 is not necessarily true. There may be RUNNING requests that are not
302 - scheduled in the current step. As of 1/1/2025, the scheduler does not
303 - allow this case, but it is possible in the future, as we allow more
304 - flexible scheduling.
302 + scheduled in the current step.
303
304 This can result in an edge case where the number of common prefix blocks
305 is 0, even though all scheduled requests share a common prefix. This