A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
[V1][Metrics] Updated list of deprecated metrics in v0.8 (#14695)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Mark McLoughlin committed
Mar 14, 2025 at 16:45 UTC
9d2b4a70f43e91f49c3d2fd449d586ff8f25e31f
1 file changed
+10
-1
docs/source/serving/metrics.md
+10
-1
@@ -39,7 +39,16 @@ The following metrics are exposed:
39
40
The following metrics are deprecated and due to be removed in a future version:
41
42
-- *(No metrics are currently deprecated)*
42
+- `vllm:num_requests_swapped`, `vllm:cpu_cache_usage_perc`, and
43
+ `vllm:cpu_prefix_cache_hit_rate` because KV cache offloading is not
44
+ used in V1.
45
+- `vllm:gpu_prefix_cache_hit_rate` is replaced by queries+hits
46
+ counters in V1.
47
+- `vllm:time_in_queue_requests` because it duplicates
48
+ `vllm:request_queue_time_seconds`.
49
+- `vllm:model_forward_time_milliseconds` and
50
+ `vllm:model_execute_time_milliseconds` because
51
+ prefill/decode/inference time metrics should be used instead.
52
53
Note: when metrics are deprecated in version `X.Y`, they are hidden in version `X.Y+1`
54
but can be re-enabled using the `--show-hidden-metrics-for-version=X.Y` escape hatch,