@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 87f545ba6f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

[Misc] Fix logger format typo (#4396)

Roy committed Apr 27, 2024 at 13:45 UTC 87f545ba6fdbbbe9813736fc398874563e2604a7
1 file changed +2 -2
vllm/engine/metrics.py
+2 -2
@@ -230,8 +230,8 @@ class StatLogger:
230 "Avg prompt throughput: %.1f tokens/s, "
231 "Avg generation throughput: %.1f tokens/s, "
232 "Running: %d reqs, Swapped: %d reqs, "
233 - "Pending: %d reqs, GPU KV cache usage: %.1f%, "
234 - "CPU KV cache usage: %.1f%",
233 + "Pending: %d reqs, GPU KV cache usage: %.1f%%, "
234 + "CPU KV cache usage: %.1f%%",
235 prompt_throughput,
236 generation_throughput,
237 stats.num_running,