@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 260 lines 9.62 KB
Rendered Raw
1 # Logging Configuration
2
3 vLLM leverages Python's `logging.config.dictConfig` functionality to enable
4 robust and flexible configuration of the various loggers used by vLLM.
5
6 For `vllm serve`, configure logging with CLI arguments:
7
8 - Use the built-in configuration, optionally with `--log-level`.
9 - Emit structured JSON with `--logging-config.formatter json`.
10 - Use a custom Python logging configuration file with
11 `--logging-config.pylogging_config_file`.
12 - Disable vLLM logging configuration with
13 `--logging-config.configure_logging false`.
14
15 ## CLI logging configuration
16
17 `--logging-config` accepts a JSON object. Its fields are `log_level`,
18 `formatter`, `configure_logging`, and `pylogging_config_file`. For example:
19
20 ```bash
21 vllm serve mistralai/Mistral-7B-v0.1 \
22 --logging-config '{"log_level":"DEBUG","configure_logging":true}'
23 ```
24
25 Fields can also be set individually with dotted arguments:
26
27 ```bash
28 vllm serve mistralai/Mistral-7B-v0.1 \
29 --logging-config.log_level DEBUG
30 ```
31
32 The built-in `text` formatter is selected by default. To emit one JSON object
33 per vLLM and Uvicorn log record, select the built-in `json` formatter:
34
35 ```bash
36 vllm serve mistralai/Mistral-7B-v0.1 \
37 --logging-config.formatter json
38 ```
39
40 The built-in JSON output includes `asctime`, `levelname`, `name`,
41 `processName`, `process`, `message`, and `vllm_process_name`. `processName` is
42 Python's process name, `vllm_process_name` identifies vLLM's logical process
43 (including worker ranks where applicable), and `process` is the
44 operating-system PID.
45
46 The built-in profile formats records emitted through vLLM's Python loggers and
47 Uvicorn's server and access loggers when using `vllm serve`. It does not convert
48 direct writes to `stdout` or `stderr`. Use a custom configuration to format
49 other loggers.
50
51 `--log-level` is a shortcut for `--logging-config.log_level` and takes
52 precedence if both are supplied. It sets the level of vLLM's built-in logging
53 configuration. `--log-config-file` is deprecated and will be removed in
54 v0.33.0; use `--logging-config.pylogging_config_file` in new commands.
55
56 If `configure_logging` is `false`, vLLM does not apply a logging
57 configuration. It cannot be combined with `pylogging_config_file`. This has
58 the same effect as the legacy `VLLM_CONFIGURE_LOGGING=0` setting.
59
60 The custom logging configuration file must be JSON following Python's [logging
61 configuration dictionary
62 schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details).
63
64 !!! note "Custom configurations override built-in formatter settings"
65 When `pylogging_config_file` is set, vLLM loads that JSON file and replaces
66 its built-in `dictConfig`; it does not merge the two. Therefore, `log_level`
67 and `formatter` apply only when no custom configuration file is provided.
68 Set levels and formatters in the custom file itself. vLLM applies the
69 resolved configuration in its child processes as well.
70
71 ## Environment variables
72
73 The CLI configuration is recommended for `vllm serve`. The legacy
74 `VLLM_CONFIGURE_LOGGING`, `VLLM_LOGGING_LEVEL`, and
75 `VLLM_LOGGING_CONFIG_PATH` variables remain supported as defaults. The default
76 handler's stream, prefix, and color are currently controlled only through
77 environment variables. Values from a YAML `--config` file or the command line
78 override those defaults. See [Environment Variables](https://docs.vllm.ai/en/latest/configuration/env_vars/)
79 for those settings.
80
81 ## Examples
82
83 ### Example 1: Customize vLLM root logger
84
85 For more control over fields and handlers, configure the vLLM root logger and
86 Uvicorn's server and access loggers with
87 [`python-json-logger`](https://github.com/nhairs/python-json-logger)
88 (which is part of the container image) to log to STDOUT in JSON format at
89 `INFO` level.
90
91 To begin, first, create an appropriate JSON logging configuration file:
92
93 ??? note "/path/to/logging_config.json"
94
95 ```json
96 {
97 "formatters": {
98 "json": {
99 "class": "pythonjsonlogger.jsonlogger.JsonFormatter",
100 "format": "%(asctime)s %(levelname)s %(name)s %(vllm_process_name)s %(process)d %(message)s"
101 }
102 },
103 "handlers": {
104 "console": {
105 "class" : "logging.StreamHandler",
106 "formatter": "json",
107 "level": "INFO",
108 "stream": "ext://sys.stdout"
109 }
110 },
111 "loggers": {
112 "vllm": {
113 "handlers": ["console"],
114 "level": "INFO",
115 "propagate": false
116 },
117 "uvicorn": {
118 "handlers": ["console"],
119 "level": "INFO",
120 "propagate": false
121 },
122 "uvicorn.error": {
123 "handlers": ["console"],
124 "level": "INFO",
125 "propagate": false
126 },
127 "uvicorn.access": {
128 "handlers": ["console"],
129 "level": "INFO",
130 "propagate": false
131 }
132 },
133 "disable_existing_loggers": false,
134 "version": 1
135 }
136 ```
137
138 Finally, run vLLM with the custom logging configuration JSON file:
139
140 ```bash
141 vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
142 --logging-config.pylogging_config_file /path/to/logging_config.json
143 ```
144
145 With this custom configuration, each vLLM and Uvicorn log record is one JSON
146 object. The example selects `vllm_process_name` and `process` explicitly; both
147 fields are also present in the built-in JSON output described above. vLLM
148 avoids altering `stdout` or `stderr`, so no text is prepended to JSON log
149 records.
150
151 This applies to records emitted through the configured Python loggers. A JSON
152 formatter cannot convert unrelated output into JSON, such as a third-party
153 library writing directly to `stdout` or `stderr`; configure or route such
154 output separately when a consumer requires every collected line to be JSON.
155
156 When serving, vLLM also passes this file to Uvicorn as its logging
157 configuration. The `uvicorn`, `uvicorn.error`, and `uvicorn.access` entries
158 format its server and access logs as JSON.
159
160 ### Example 2: Silence a particular vLLM logger
161
162 To silence a particular vLLM logger, it is necessary to provide custom logging
163 configuration for the target logger that configures the logger so that it won't
164 propagate its log messages to the root vLLM logger.
165
166 When custom configuration is provided for any logger, it is also necessary to
167 provide configuration for the root vLLM logger since any custom logger
168 configuration overrides the built-in default logging configuration used by vLLM.
169
170 First, create an appropriate JSON logging configuration file that includes
171 configuration for the root vLLM logger and for the logger you wish to silence:
172
173 ??? note "/path/to/logging_config.json"
174
175 ```json
176 {
177 "formatters": {
178 "vllm": {
179 "class": "vllm.logging_utils.NewLineFormatter",
180 "datefmt": "%m-%d %H:%M:%S",
181 "format": "%(levelname)s %(asctime)s %(filename)s:%(lineno)d] %(message)s"
182 }
183 },
184 "handlers": {
185 "vllm": {
186 "class" : "logging.StreamHandler",
187 "formatter": "vllm",
188 "level": "INFO",
189 "stream": "ext://sys.stdout"
190 }
191 },
192 "loggers": {
193 "vllm": {
194 "handlers": ["vllm"],
195 "level": "DEBUG",
196 "propagate": false
197 },
198 "vllm.example_noisy_logger": {
199 "propagate": false
200 }
201 },
202 "version": 1
203 }
204 ```
205
206 Finally, run vLLM with the custom logging configuration JSON file:
207
208 ```bash
209 vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
210 --logging-config.pylogging_config_file /path/to/logging_config.json
211 ```
212
213 ### Example 3: Disable vLLM default logging configuration
214
215 To disable vLLM's default logging configuration and silence vLLM log output,
216 set `--logging-config.configure_logging false` when running vLLM. This prevents
217 vLLM from configuring the root vLLM logger, which in turn silences other vLLM
218 loggers unless the application or Python root logger configures a handler.
219
220 ```bash
221 vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
222 --logging-config.configure_logging false
223 ```
224
225 For legacy launch scripts, `VLLM_CONFIGURE_LOGGING=0` has the same effect.
226
227 ### Example 4: Disable access logs for health check endpoints
228
229 In production environments, health check endpoints like `/health`, `/metrics`,
230 and `/ping` are frequently called by load balancers and monitoring systems,
231 generating a large volume of repetitive access logs. To reduce log noise while
232 keeping logs for other endpoints, use the `--disable-access-log-for-endpoints`
233 option.
234
235 **Disable access logs for health and metrics endpoints:**
236
237 ```bash
238 vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
239 --disable-access-log-for-endpoints /health,/metrics,/ping
240 ```
241
242 **Common endpoints to consider filtering:**
243
244 | Endpoint | Description | Typical Caller |
245 | ---------- | ---------------------- | ---------------------------------------------------- |
246 | `/health` | Health check | Kubernetes liveness/readiness probes, load balancers |
247 | `/metrics` | Prometheus metrics | Prometheus scraper (every 15-60s) |
248 | `/ping` | SageMaker health check | SageMaker infrastructure |
249 | `/load` | Server load metrics | Custom monitoring |
250
251 **Notes:**
252
253 - This option only affects uvicorn access logs, not vLLM application logs
254 - Specify multiple endpoints by separating them with commas (no spaces)
255 - The filter uses exact path matching, query parameters are ignored (e.g., `/health?verbose=true` matches `/health`)
256 - If you need to completely disable all access logs, use `--disable-uvicorn-access-log` instead
257
258 ## Additional resources
259
260 - [`logging.config` Dictionary Schema Details](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details)