@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d4acf518d0

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 13, 02:00 UTC

[Metrics] Fix KV cache usage percent metric multiproc (#28792)

The `vllm:kv_cache_usage_perc` Gauge metric is missing `multiprocess_mode="mostrecent"` and ends up returning ``` vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="277"} 0.0 vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="275"} 0.0 vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="273"} 0.6530455880475035 ... ``` The deprecated `vllm:gpu_cache_usage_perc` Gauge metric has `multiprocess_mode="mostrecent"`. Signed-off-by: Jae-Won Chung <jwnchung@umich.edu>

Jae-Won Chung committed Nov 17, 2025 at 04:54 UTC d4acf518d09515560e1082a80b8a4d6550e20d9b

1 file changed +1

vllm/v1/metrics/loggers.py

@@ -494,6 +494,7 @@ class PrometheusStatLogger(AggregateStatLoggerBase):

494 gauge_kv_cache_usage = self._gauge_cls(

495 name="vllm:kv_cache_usage_perc",

496 documentation="KV-cache usage. 1 means 100 percent usage.",

497 + multiprocess_mode="mostrecent",

498 labelnames=labelnames,

499 )

500 self.gauge_kv_cache_usage = make_per_engine(