A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Misc] rename interval to max_recent_requests (#24229)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Ning Xie committed
Sep 15, 2025 at 17:18 UTC
59e17dd4a0f058e0bef38a371316b87fee3e882e
1 file changed
+2
-2
vllm/v1/core/kv_cache_utils.py
+2
-2
@@ -116,8 +116,8 @@ class PrefixCachingMetrics:
116
This function is called with information gathered when new requests
117
are being scheduled and are looking for computed blocks.
118
119
- When there are more than `interval` requests, the oldest set of
120
- requests are removed from the metrics.
119
+ When there are more than `max_recent_requests` requests, the oldest set
120
+ of requests are removed from the metrics.
121
122
Args:
123
stats: The prefix cache stats.