A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
[V1][Minor] Remove outdated comment (#12928)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Woosuk Kwon committed
Feb 7, 2025 at 19:07 UTC
b21f0f9d173e5094791815380cb213f9eed44bad
1 file changed
+1
-3
vllm/v1/core/kv_cache_manager.py
+1
-3
@@ -299,9 +299,7 @@ class KVCacheManager:
299
300
While all scheduled requests must be in the RUNNING state, the inverse
301
is not necessarily true. There may be RUNNING requests that are not
302
- scheduled in the current step. As of 1/1/2025, the scheduler does not
303
- allow this case, but it is possible in the future, as we allow more
304
- flexible scheduling.
302
+ scheduled in the current step.
303
304
This can result in an edge case where the number of common prefix blocks
305
is 0, even though all scheduled requests share a common prefix. This