@tduncan / Nvidia-Optimization-Stack-VLLM / commits / 45d56c0baf

[Core] Model console logging as CLI configuration (#57205)

Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com>

Mark McLoughlin committed Sep 25, 2026 at 11:36 UTC 45d56c0baf454f2816af514011ae27c2857ee312
36 files changed +440 -135
examples/features/logging_configuration.md
+57 -43
@@ -3,46 +3,60 @@
3 vLLM leverages Python's `logging.config.dictConfig` functionality to enable
4 robust and flexible configuration of the various loggers used by vLLM.
5
6 -vLLM offers two environment variables that can be used to accommodate a range
7 -of logging configurations that range from simple-and-inflexible to
8 -more-complex-and-more-flexible.
6 +For `vllm serve`, configure logging with CLI arguments:
7
10 -- No vLLM logging (simple and inflexible)
11 - - Set `VLLM_CONFIGURE_LOGGING=0` (leaving `VLLM_LOGGING_CONFIG_PATH` unset)
12 -- vLLM's default logging configuration (simple and inflexible)
13 - - Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1`
14 -- Fine-grained custom logging configuration (more complex, more flexible)
15 - - Leave `VLLM_CONFIGURE_LOGGING` unset or set `VLLM_CONFIGURE_LOGGING=1` and
16 - set `VLLM_LOGGING_CONFIG_PATH=<path-to-logging-config.json>`
8 +- Use the built-in configuration, optionally with `--log-level`.
9 +- Use a custom Python logging configuration file with
10 + `--logging-config.pylogging_config_file`.
11 +- Disable vLLM logging configuration with
12 + `--logging-config.configure_logging false`.
13
18 -## Logging Configuration Environment Variables
14 +## CLI logging configuration
15
20 -### `VLLM_CONFIGURE_LOGGING`
16 +`--logging-config` accepts a JSON object. Its fields are `log_level`,
17 +`configure_logging`, and `pylogging_config_file`. For example:
18
22 -`VLLM_CONFIGURE_LOGGING` controls whether or not vLLM takes any action to
23 -configure the loggers used by vLLM. This functionality is enabled by default,
24 -but can be disabled by setting `VLLM_CONFIGURE_LOGGING=0` when running vLLM.
19 +```bash
20 +vllm serve mistralai/Mistral-7B-v0.1 \
21 + --logging-config '{"log_level":"DEBUG","configure_logging":true}'
22 +```
23 +
24 +Fields can also be set individually with dotted arguments:
25
26 -If `VLLM_CONFIGURE_LOGGING` is enabled and no value is given for
27 -`VLLM_LOGGING_CONFIG_PATH`, vLLM will use built-in default configuration to
28 -configure the root vLLM logger. By default, no other vLLM loggers are
29 -configured and, as such, all vLLM loggers defer to the root vLLM logger to make
30 -all logging decisions.
26 +```bash
27 +vllm serve mistralai/Mistral-7B-v0.1 \
28 + --logging-config.log_level DEBUG
29 +```
30
32 -If `VLLM_CONFIGURE_LOGGING` is disabled and a value is given for
33 -`VLLM_LOGGING_CONFIG_PATH`, an error will occur while starting vLLM.
31 +`--log-level` is a shortcut for `--logging-config.log_level` and takes
32 +precedence if both are supplied. It sets the level of vLLM's built-in logging
33 +configuration. `--log-config-file` is deprecated and will be removed in
34 +v0.33.0; use `--logging-config.pylogging_config_file` in new commands.
35
35 -### `VLLM_LOGGING_CONFIG_PATH`
36 +If `configure_logging` is `false`, vLLM does not apply a logging
37 +configuration. It cannot be combined with `pylogging_config_file`. This has
38 +the same effect as the legacy `VLLM_CONFIGURE_LOGGING=0` setting.
39
37 -`VLLM_LOGGING_CONFIG_PATH` allows users to specify a path to a JSON file of
38 -alternative, custom logging configuration that will be used instead of vLLM's
39 -built-in default logging configuration. The logging configuration should be
40 -provided in JSON format following the schema specified by Python's [logging
40 +The custom logging configuration file must be JSON following Python's [logging
41 configuration dictionary
42 schema](https://docs.python.org/3/library/logging.config.html#dictionary-schema-details).
43
44 -If `VLLM_LOGGING_CONFIG_PATH` is specified, but `VLLM_CONFIGURE_LOGGING` is
45 -disabled, an error will occur while starting vLLM.
44 +!!! note "Custom configurations override `--log-level`"
45 + When `pylogging_config_file` is set, vLLM loads that JSON file and replaces
46 + its built-in `dictConfig`; it does not merge the two. Therefore,
47 + `--log-level` applies only when no custom configuration file is provided.
48 + Set logger and handler levels in the custom file itself. vLLM applies the
49 + resolved configuration in its child processes as well.
50 +
51 +## Environment variables
52 +
53 +The CLI configuration is recommended for `vllm serve`. The legacy
54 +`VLLM_CONFIGURE_LOGGING`, `VLLM_LOGGING_LEVEL`, and
55 +`VLLM_LOGGING_CONFIG_PATH` variables remain supported as defaults. The default
56 +handler's stream, prefix, and color are currently controlled only through
57 +environment variables. Values from a YAML `--config` file or the command line
58 +override those defaults. See [Environment Variables](https://docs.vllm.ai/en/latest/configuration/env_vars/)
59 +for those settings.
60
61 ## Examples
62
@@ -83,12 +97,11 @@ To begin, first, create an appropriate JSON logging configuration file:
97 }
98 ```
99
86 -Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
87 -to the path of the custom logging configuration JSON file:
100 +Finally, run vLLM with the custom logging configuration JSON file:
101
102 ```bash
90 -VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
91 - vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
103 +vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
104 + --logging-config.pylogging_config_file /path/to/logging_config.json
105 ```
106
107 ### Example 2: Silence a particular vLLM logger
@@ -137,26 +150,27 @@ configuration for the root vLLM logger and for the logger you wish to silence:
150 }
151 ```
152
140 -Finally, run vLLM with the `VLLM_LOGGING_CONFIG_PATH` environment variable set
141 -to the path of the custom logging configuration JSON file:
153 +Finally, run vLLM with the custom logging configuration JSON file:
154
155 ```bash
144 -VLLM_LOGGING_CONFIG_PATH=/path/to/logging_config.json \
145 - vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
156 +vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
157 + --logging-config.pylogging_config_file /path/to/logging_config.json
158 ```
159
160 ### Example 3: Disable vLLM default logging configuration
161
150 -To disable vLLM's default logging configuration and silence all vLLM loggers,
151 -simple set `VLLM_CONFIGURE_LOGGING=0` when running vLLM. This will prevent vLLM
152 -for configuring the root vLLM logger, which in turn, silences all other vLLM
153 -loggers.
162 +To disable vLLM's default logging configuration and silence vLLM log output,
163 +set `--logging-config.configure_logging false` when running vLLM. This prevents
164 +vLLM from configuring the root vLLM logger, which in turn silences other vLLM
165 +loggers unless the application or Python root logger configures a handler.
166
167 ```bash
156 -VLLM_CONFIGURE_LOGGING=0 \
157 - vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048
168 +vllm serve mistralai/Mistral-7B-v0.1 --max-model-len 2048 \
169 + --logging-config.configure_logging false
170 ```
171
172 +For legacy launch scripts, `VLLM_CONFIGURE_LOGGING=0` has the same effect.
173 +
174 ### Example 4: Disable access logs for health check endpoints
175
176 In production environments, health check endpoints like `/health`, `/metrics`,
tests/conftest.py
+8 -4
@@ -1430,16 +1430,20 @@ def temporary_enable_log_propagate():
1430 import logging
1431
1432 logger = logging.getLogger("vllm")
1433 + previous_propagate = logger.propagate
1434 logger.propagate = True
1435 yield
1435 - logger.propagate = False
1436 + logger.propagate = previous_propagate
1437
1438
1439 @pytest.fixture()
1440 def caplog_vllm(temporary_enable_log_propagate, caplog):
1440 - # To capture vllm log, we should enable propagate=True temporarily
1441 - # because caplog depends on logs propagated to the root logger.
1442 - yield caplog
1441 + # caplog depends on logs propagated to the root logger. The vLLM logger
1442 + # inherits this INFO level until runtime initialization configures it.
1443 + import logging
1444 +
1445 + with caplog.at_level(logging.INFO):
1446 + yield caplog
1447
1448
1449 @pytest.fixture()
tests/entrypoints/launchers/test_cli_args.py
+19
@@ -8,6 +8,7 @@ import pytest
8
9 import vllm.entrypoints.launchers.cli_args as cli_args_module
10 from tests.utils import VLLM_PATH
11 +from vllm.engine.arg_utils import AsyncEngineArgs
12 from vllm.entrypoints.launchers.cli_args import (
13 make_arg_parser,
14 validate_parsed_serve_args,
@@ -69,6 +70,24 @@ def test_config_arg_parsing(serve_parser, cli_config_file):
70 assert args.port == 9000
71
72
73 +def test_logging_config_cli_args(serve_parser):
74 + with pytest.warns(UserWarning, match="--log-config-file is deprecated"):
75 + args = serve_parser.parse_args(
76 + [
77 + "--logging-config",
78 + '{"log_level":"WARNING","pylogging_config_file":"/tmp/json.json"}',
79 + "--log-level",
80 + "DEBUG",
81 + "--log-config-file",
82 + "/tmp/flat.json",
83 + ]
84 + )
85 +
86 + config = AsyncEngineArgs.from_cli_args(args).create_logging_config()
87 + assert config.log_level == "DEBUG"
88 + assert config.pylogging_config_file == "/tmp/flat.json"
89 +
90 +
91 ### Tests for LoRA module parsing
92 def test_valid_key_value_format(serve_parser):
93 # Test old format: name=path
tests/test_logger.py
+60 -9
@@ -16,11 +16,13 @@ from uuid import uuid4
16
17 import pytest
18
19 +from vllm.config import LoggingConfig
20 from vllm.logger import (
21 _DATE_FORMAT,
22 _FORMAT,
23 _configure_vllm_root_logger,
24 _use_color,
25 + configure_logging,
26 enable_trace_function_call,
27 init_logger,
28 )
@@ -50,16 +52,27 @@ def test_trace_function_call():
52 os.remove(path)
53
54
55 +def test_caplog_vllm_captures_info_before_runtime_logging_is_configured(caplog_vllm):
56 + message = "Capture this unconfigured INFO record"
57 + logger = init_logger(f"vllm.test_logger.{uuid4()}")
58 +
59 + logger.info(message)
60 +
61 + assert message in caplog_vllm.text
62 +
63 +
64 def test_default_vllm_root_logger_configuration(monkeypatch):
65 """This test presumes that VLLM_CONFIGURE_LOGGING (default: True) and
66 VLLM_LOGGING_CONFIG_PATH (default: None) are not configured and default
67 behavior is activated."""
68 monkeypatch.setenv("VLLM_LOGGING_COLOR", "0")
58 - _configure_vllm_root_logger()
69 + early_logger = init_logger(f"vllm.test_logger.{uuid4()}")
70 + configure_logging(LoggingConfig())
71
72 logger = logging.getLogger("vllm")
73 assert logger.level == logging.INFO
74 assert not logger.propagate
75 + assert early_logger.isEnabledFor(logging.INFO)
76
77 handler = logger.handlers[0]
78 assert isinstance(handler, logging.StreamHandler)
@@ -74,6 +87,29 @@ def test_default_vllm_root_logger_configuration(monkeypatch):
87 assert formatter.datefmt == _DATE_FORMAT
88
89
90 +def test_offline_llm_configures_logging_before_logging_args(monkeypatch):
91 + import vllm.entrypoints.llm as llm_module
92 +
93 + class StopInitialization(Exception):
94 + pass
95 +
96 + configured = False
97 +
98 + def configure_logging(_):
99 + nonlocal configured
100 + configured = True
101 +
102 + def log_args(_):
103 + assert configured
104 + raise StopInitialization
105 +
106 + monkeypatch.setattr(llm_module, "configure_logging_if_needed", configure_logging)
107 + monkeypatch.setattr(llm_module, "log_non_default_args", log_args)
108 +
109 + with pytest.raises(StopInitialization):
110 + llm_module.LLM(model="facebook/opt-125m")
111 +
112 +
113 def test_use_color_force_color(monkeypatch):
114 """FORCE_COLOR forces colored logs without a TTY, while NO_COLOR and an
115 explicit VLLM_LOGGING_COLOR=0 take precedence over it."""
@@ -133,7 +169,7 @@ def test_logger_configuring_can_be_disabled(monkeypatch):
169 monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
170
171 with patch("vllm.logger.dictConfig") as dict_config_mock:
136 - _configure_vllm_root_logger()
172 + configure_logging(LoggingConfig(configure_logging=False))
173 dict_config_mock.assert_not_called()
174
175
@@ -214,9 +250,11 @@ def test_custom_logging_config_is_parsed_and_used_when_provided(monkeypatch):
250 with NamedTemporaryFile(encoding="utf-8", mode="w") as logging_config_file:
251 logging_config_file.write(json.dumps(valid_logging_config))
252 logging_config_file.flush()
217 - monkeypatch.setenv("VLLM_LOGGING_CONFIG_PATH", logging_config_file.name)
253 + monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
254 with patch("vllm.logger.dictConfig") as dict_config_mock:
219 - _configure_vllm_root_logger()
255 + configure_logging(
256 + LoggingConfig(pylogging_config_file=logging_config_file.name)
257 + )
258 dict_config_mock.assert_called_with(valid_logging_config)
259
260
@@ -237,13 +275,18 @@ def test_custom_logging_config_causes_an_error_if_configure_logging_is_off(monke
275 with NamedTemporaryFile(encoding="utf-8", mode="w") as logging_config_file:
276 logging_config_file.write(json.dumps(valid_logging_config))
277 logging_config_file.flush()
240 - monkeypatch.setenv("VLLM_LOGGING_CONFIG_PATH", logging_config_file.name)
278 + monkeypatch.delenv("VLLM_LOGGING_CONFIG_PATH", raising=False)
279 with pytest.raises(RuntimeError) as ex_info:
242 - _configure_vllm_root_logger()
280 + configure_logging(
281 + LoggingConfig(
282 + configure_logging=False,
283 + pylogging_config_file=logging_config_file.name,
284 + )
285 + )
286 assert ex_info.type is RuntimeError
287 expected_message_snippet = (
245 - "VLLM_CONFIGURE_LOGGING evaluated to false, but "
246 - "VLLM_LOGGING_CONFIG_PATH was given."
288 + "Logging configuration is disabled, but a Python logging config "
289 + "file was given."
290 )
291 assert expected_message_snippet in str(ex_info)
292
@@ -297,6 +340,8 @@ test_logger = init_logger("vllm.test_logger")
340
341 def mp_function(**kwargs):
342 # This function runs in a subprocess
343 + if logging_config := kwargs.pop("logging_config", None):
344 + configure_logging(logging_config)
345
346 test_logger.warning("This is a subprocess: %s", kwargs.get("a"))
347 test_logger.error("This is a subprocess error.")
@@ -328,7 +373,13 @@ def test_caplog_mp_spawn(caplog_mp_spawn):
373 p = ctx.Process(
374 target=mp_function,
375 name=f"SubProcess{1}",
331 - kwargs={"a": "AAAA", "b": "BBBBB"},
376 + kwargs={
377 + "a": "AAAA",
378 + "b": "BBBBB",
379 + "logging_config": LoggingConfig(
380 + pylogging_config_file=os.environ["VLLM_LOGGING_CONFIG_PATH"]
381 + ),
382 + },
383 )
384 p.start()
385 p.join()
tests/v1/engine/test_async_llm.py
+3 -1
@@ -11,7 +11,7 @@ import pytest
11 import vllm.v1.engine.async_llm as async_llm_module
12 from vllm import SamplingParams
13 from vllm.assets.image import ImageAsset
14 -from vllm.config import VllmConfig
14 +from vllm.config import LoggingConfig, VllmConfig
15 from vllm.engine.arg_utils import AsyncEngineArgs
16 from vllm.entrypoints.openai.chat_completion.protocol import (
17 ChatCompletionRequest,
@@ -89,6 +89,7 @@ def _mock_async_llm_dependencies(monkeypatch: pytest.MonkeyPatch):
89
90 def test_cuda_profiler_requests_reach_engine_core(monkeypatch: pytest.MonkeyPatch):
91 vllm_config = MagicMock()
92 + vllm_config.logging_config = LoggingConfig()
93 vllm_config.observability_config.otlp_traces_endpoint = None
94 vllm_config.scheduler_config.stream_interval = 1
95 vllm_config.profiler_config.profiler = "cuda"
@@ -111,6 +112,7 @@ def test_cuda_only_torch_profiler_skips_frontend_cpu_trace(
112 monkeypatch: pytest.MonkeyPatch,
113 ):
114 vllm_config = MagicMock()
115 + vllm_config.logging_config = LoggingConfig()
116 vllm_config.observability_config.otlp_traces_endpoint = None
117 vllm_config.scheduler_config.stream_interval = 1
118 vllm_config.profiler_config.profiler = "torch"
vllm/compilation/breakable_cudagraph.py
+2 -1
@@ -25,6 +25,7 @@ from __future__ import annotations
25 import dataclasses
26 import functools
27 import gc
28 +import logging
29 import threading
30 import weakref
31 from collections.abc import Callable
@@ -285,7 +286,7 @@ class BreakableCUDAGraphWrapper:
286 self.runtime_mode = runtime_mode
287 self.compilation_config = vllm_config.compilation_config
288 self.graph_pool = current_platform.get_global_graph_pool()
288 - self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
289 + self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
290
291 self.entries: dict[BatchDescriptor, _BreakableEntry] = {}
292 BreakableCUDAGraphWrapper._all_instances.add(self)
vllm/compilation/cuda_graph.py
+3 -3
@@ -2,6 +2,7 @@
2 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
4 import dataclasses
5 +import logging
6 import weakref
7 from collections import Counter
8 from collections.abc import Callable
@@ -11,7 +12,6 @@ from unittest.mock import patch
12
13 import torch
14
14 -import vllm.envs as envs
15 from vllm.compilation.counter import compilation_counter
16 from vllm.compilation.monitor import validate_cudagraph_capturing_enabled
17 from vllm.config import CUDAGraphMode, VllmConfig
@@ -164,7 +164,7 @@ class CUDAGraphWrapper:
164 assumption on the dynamic shape (batch size) of the runtime inputs, as a
165 trade-off for staying orthogonal to compilation logic. Nevertheless,
166 tracing and checking the input addresses to be consistent during replay is
167 - guaranteed when VLLM_LOGGING_LEVEL == "DEBUG".
167 + guaranteed when vLLM debug logging is enabled.
168 """
169
170 _all_instances: ClassVar[weakref.WeakSet["CUDAGraphWrapper"]] = weakref.WeakSet()
@@ -188,7 +188,7 @@ class CUDAGraphWrapper:
188 self.compilation_config = vllm_config.compilation_config
189
190 self.first_run_finished = False
191 - self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
191 + self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
192 self._runnable_str = str(runnable) if self.is_debugging_mode else None
193
194 # assert runtime_mode is not NONE(no cudagraph), otherwise, we don't
vllm/config/__init__.py
+3
@@ -20,6 +20,7 @@ from vllm.config.kernel import KernelConfig
20 from vllm.config.kv_events import KVEventsConfig
21 from vllm.config.kv_transfer import KVTransferConfig
22 from vllm.config.load import LoadConfig
23 +from vllm.config.logging import LoggingConfig
24 from vllm.config.lora import LoRAConfig
25 from vllm.config.mamba import MambaConfig
26 from vllm.config.model import (
@@ -97,6 +98,8 @@ __all__ = [
98 "KVTransferConfig",
99 # From vllm.config.load
100 "LoadConfig",
101 + # From vllm.config.logging
102 + "LoggingConfig",
103 # From vllm.config.lora
104 "LoRAConfig",
105 # From vllm.config.mamba
vllm/config/logging.py new
+46
@@ -0,0 +1,46 @@
1 +# SPDX-License-Identifier: Apache-2.0
2 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3 +
4 +from typing import Literal, cast
5 +
6 +from pydantic import Field
7 +
8 +import vllm.envs as envs
9 +from vllm.config.utils import config
10 +
11 +LogLevel = Literal["CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG", "NOTSET"]
12 +
13 +
14 +@config
15 +class LoggingConfig:
16 + """vLLM logging. Default ``dictConfig`` adds a formatted stream handler.
17 +
18 + Supply a JSON object with ``--logging-config`` or individual fields with
19 + dotted arguments such as ``--logging-config.log_level DEBUG`` and
20 + ``--logging-config.pylogging_config_file logging.json``. ``--log-level``
21 + and legacy ``--log-config-file`` override their matching fields. Set
22 + ``configure_logging`` to false to skip applying a ``dictConfig``.
23 + """
24 +
25 + log_level: LogLevel = Field(
26 + default_factory=lambda: cast(LogLevel, envs.VLLM_LOGGING_LEVEL)
27 + )
28 + """Log level used when no custom logging configuration is provided."""
29 +
30 + configure_logging: bool = Field(default_factory=lambda: envs.VLLM_CONFIGURE_LOGGING)
31 + """Whether to apply a Python logging configuration.
32 +
33 + With no ``pylogging_config_file``, the default config adds a formatted
34 + stream handler to the ``vllm`` logger. A custom Python logging configuration
35 + file also requires this to be enabled.
36 + """
37 +
38 + pylogging_config_file: str | None = Field(
39 + default_factory=lambda: envs.VLLM_LOGGING_CONFIG_PATH
40 + )
41 + """Path to a Python logging JSON file using
42 + [``dictConfig`` schema](https://docs.python.org/3/library/logging.config.html#configuration-file-format).
43 +
44 + A custom ``dictConfig`` is authoritative over ``log_level`` for logger,
45 + handler, and formatter settings.
46 + """
vllm/config/vllm.py
+3
@@ -40,6 +40,7 @@ from .kernel import KernelConfig
40 from .kv_events import KVEventsConfig
41 from .kv_transfer import KVTransferConfig
42 from .load import LoadConfig
43 +from .logging import LoggingConfig
44 from .lora import LoRAConfig
45 from .mamba import MambaBackendEnum, MambaConfig
46 from .model import ModelConfig
@@ -402,6 +403,8 @@ class VllmConfig:
403 default_factory=ObservabilityConfig
404 )
405 """Observability configuration."""
406 + logging_config: LoggingConfig = Field(default_factory=LoggingConfig)
407 + """Logging configuration."""
408 quant_config: QuantizationConfig | None = None
409 """Quantization configuration."""
410 compilation_config: CompilationConfig = Field(default_factory=CompilationConfig)
vllm/distributed/device_communicators/pynccl_allocator.py
+2 -1
@@ -2,6 +2,7 @@
2 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3 import atexit
4 import contextlib
5 +import logging
6 import tempfile
7 from typing import Any
8
@@ -83,7 +84,7 @@ def compile_nccl_allocator():
84 cpp_sources=nccl_allocator_source,
85 with_cuda=True,
86 extra_ldflags=ldflags,
86 - verbose=envs.VLLM_LOGGING_LEVEL == "DEBUG",
87 + verbose=logger.isEnabledFor(logging.DEBUG),
88 is_python_module=False,
89 build_directory=out_dir,
90 extra_include_paths=nccl_include_paths,
vllm/engine/arg_utils.py
+64 -1
@@ -8,6 +8,7 @@ import functools
8 import json
9 import os
10 import sys
11 +import warnings
12 from collections.abc import Callable
13 from dataclasses import MISSING, asdict, dataclass, fields, is_dataclass
14 from types import UnionType
@@ -50,6 +51,7 @@ from vllm.config import (
51 KVEventsConfig,
52 KVTransferConfig,
53 LoadConfig,
54 + LoggingConfig,
55 LoRAConfig,
56 MambaConfig,
57 ModelConfig,
@@ -84,6 +86,7 @@ from vllm.config.kernel import (
86 SparseIndexerTopkBackend,
87 )
88 from vllm.config.load import SafetensorsLoadStrategy
89 +from vllm.config.logging import LogLevel
90 from vllm.config.lora import MaxLoRARanks
91 from vllm.config.mamba import MambaBackendEnum, MambaSSUAlgorithm
92 from vllm.config.model import (
@@ -113,7 +116,7 @@ from vllm.config.scheduler import SchedulerPolicy
116 from vllm.config.utils import get_field
117 from vllm.config.vllm import OptimizationLevel, PerformanceMode
118 from vllm.config.watermarking import WatermarkConfig
116 -from vllm.logger import init_logger, suppress_logging
119 +from vllm.logger import configure_logging_if_needed, init_logger, suppress_logging
120 from vllm.platforms import CpuArchEnum, current_platform
121 from vllm.plugins import load_general_plugins
122 from vllm.ray.lazy_utils import is_in_ray_actor, is_ray_initialized
@@ -443,6 +446,20 @@ def get_kwargs(cls: ConfigType) -> dict[str, dict[str, Any]]:
446 return copy.deepcopy(_compute_kwargs(cls))
447
448
449 +_LOG_CONFIG_FILE_DEPRECATION_MESSAGE = (
450 + "--log-config-file is deprecated and will be removed in v0.33.0. "
451 + "Use --logging-config.pylogging_config_file instead."
452 +)
453 +
454 +
455 +class DeprecatedLogConfigFileAction(argparse.Action):
456 + """Warn when the legacy ``--log-config-file`` option is used."""
457 +
458 + def __call__(self, parser, namespace, values, option_string=None):
459 + warnings.warn(_LOG_CONFIG_FILE_DEPRECATION_MESSAGE, stacklevel=1)
460 + setattr(namespace, self.dest, values)
461 +
462 +
463 @dataclass
464 class EngineArgs:
465 """Arguments for vLLM engine."""
@@ -733,6 +750,10 @@ class EngineArgs:
750
751 profiler_config: ProfilerConfig = get_field(VllmConfig, "profiler_config")
752
753 + logging_config: LoggingConfig | None = None
754 + log_level: LogLevel | None = None
755 + log_config_file: str | None = None
756 +
757 kv_transfer_config: KVTransferConfig | None = None
758 kv_events_config: KVEventsConfig | None = None
759
@@ -1551,6 +1572,34 @@ class EngineArgs:
1572 **lora_kwargs["enable_moe_shared_loras"],
1573 )
1574
1575 + # Logging arguments
1576 + logging_group = parser.add_argument_group(
1577 + title="LoggingConfig",
1578 + description=LoggingConfig.__doc__,
1579 + )
1580 + logging_config_kwargs = get_kwargs(VllmConfig)["logging_config"]
1581 + logging_config_kwargs["default"] = argparse.SUPPRESS
1582 + logging_group.add_argument("--logging-config", **logging_config_kwargs)
1583 + logging_group.add_argument(
1584 + "--log-level",
1585 + choices=get_args(LogLevel),
1586 + default=argparse.SUPPRESS,
1587 + help=(
1588 + "Shortcut for --logging-config.log_level. "
1589 + "Overrides that field if both are specified."
1590 + ),
1591 + )
1592 + logging_group.add_argument(
1593 + "--log-config-file",
1594 + dest="log_config_file",
1595 + default=argparse.SUPPRESS,
1596 + metavar="PYLOGGING_CONFIG_FILE",
1597 + action=DeprecatedLogConfigFileAction,
1598 + help=_LOG_CONFIG_FILE_DEPRECATION_MESSAGE,
1599 + )
1600 + # Retain the warning in v0.31.0 and v0.32.0. Remove this option and its
1601 + # compatibility mapping in create_logging_config() in v0.33.0.
1602 +
1603 # Observability arguments
1604 observability_kwargs = get_kwargs(ObservabilityConfig)
1605 observability_group = parser.add_argument_group(
@@ -2081,6 +2130,16 @@ class EngineArgs:
2130 jit_monitor_verbose=self.jit_monitor_verbose,
2131 )
2132
2133 + def create_logging_config(self) -> LoggingConfig:
2134 + config = self.logging_config or LoggingConfig()
2135 + if self.log_level is not None:
2136 + config = dataclasses.replace(config, log_level=self.log_level)
2137 + if self.log_config_file is not None:
2138 + config = dataclasses.replace(
2139 + config, pylogging_config_file=self.log_config_file
2140 + )
2141 + return config
2142 +
2143 def create_engine_config(
2144 self,
2145 usage_context: UsageContext | None = None,
@@ -2090,6 +2149,9 @@ class EngineArgs:
2149
2150 NOTE: If VllmConfig is incompatible, we raise an error.
2151 """
2152 + logging_config = self.create_logging_config()
2153 + configure_logging_if_needed(logging_config)
2154 +
2155 current_platform.pre_register_and_update()
2156
2157 device_config = DeviceConfig(device=cast(Device, current_platform.device_type))
@@ -2721,6 +2783,7 @@ class EngineArgs:
2783 diffusion_config=diffusion_config,
2784 structured_outputs_config=self.structured_outputs_config,
2785 observability_config=observability_config,
2786 + logging_config=logging_config,
2787 compilation_config=compilation_config,
2788 kv_transfer_config=self.kv_transfer_config,
2789 kv_events_config=self.kv_events_config,
vllm/entrypoints/cli/main.py
+4 -2
@@ -9,7 +9,7 @@ import importlib.metadata
9 import sys
10 from importlib.util import find_spec
11
12 -from vllm.logger import init_logger
12 +from vllm.logger import configure_logging_from_args, init_logger
13
14 logger = init_logger(__name__)
15
@@ -110,7 +110,9 @@ def main():
110 cmds[cmd.name] = cmd
111 args = parser.parse_args()
112 if args.subparser in cmds:
113 - cmds[args.subparser].validate(args)
113 + cmd = cmds[args.subparser]
114 + configure_logging_from_args(args)
115 + cmd.validate(args)
116
117 if hasattr(args, "dispatch_function"):
118 args.dispatch_function(args)
vllm/entrypoints/launchers/api_server/entry.py
+2 -1
@@ -15,7 +15,7 @@ from typing import Any
15 import vllm.envs as envs
16 from vllm.engine.arg_utils import AsyncEngineArgs
17 from vllm.engine.protocol import EngineClient
18 -from vllm.logger import init_logger
18 +from vllm.logger import configure_logging_from_args, init_logger
19 from vllm.reasoning import ReasoningParserManager
20 from vllm.tool_parsers import ToolParserManager
21 from vllm.usage.usage_lib import UsageContext
@@ -216,6 +216,7 @@ def main():
216 )
217 parser = make_arg_parser(parser)
218 args = parser.parse_args()
219 + configure_logging_from_args(args)
220 validate_parsed_serve_args(args)
221
222 uvloop.run(run_server(args))
vllm/entrypoints/launchers/cli_args.py
-2
@@ -134,8 +134,6 @@ class BaseFrontendArgs:
134 The `demo` Python tool executes model-generated code in Docker without
135 network isolation by default. See the security guide for more
136 information."""
137 - log_config_file: str | None = envs.VLLM_LOGGING_CONFIG_PATH
138 - """Path to logging config JSON file for both vllm and uvicorn"""
137 max_log_len: int | None = None
138 """Max number of prompt characters or prompt ID numbers being printed in
139 log. The default of None means unlimited."""
vllm/entrypoints/launchers/dp_supervisor.py
+4 -1
@@ -23,7 +23,7 @@ import uvloop
23 from fastapi import FastAPI, Response
24
25 import vllm.envs as envs
26 -from vllm.logger import init_logger
26 +from vllm.logger import configure_logging_from_args, init_logger
27 from vllm.utils.system_utils import (
28 decorate_logs,
29 kill_process_tree,
@@ -251,6 +251,8 @@ def _run_rust_vllm_dp_server(child_args: argparse.Namespace) -> None:
251
252 def _run_vllm_dp_server(child_args: argparse.Namespace) -> None:
253 """Entrypoint function for the vLLM DP Server."""
254 + configure_logging_from_args(child_args)
255 +
256 # Create a fresh process group for the vLLM DP Server,
257 # so that CTRL-C is propagated cleanly.
258 os.setpgrp()
@@ -558,4 +560,5 @@ class DPSupervisor:
560
561
562 def run_dp_supervisor(args: argparse.Namespace) -> None:
563 + configure_logging_from_args(args)
564 uvloop.run(DPSupervisor(args).run())
vllm/entrypoints/launchers/grpc_server.py
+2 -1
@@ -44,7 +44,7 @@ import uvloop
44 from vllm import envs
45 from vllm.engine.arg_utils import AsyncEngineArgs
46 from vllm.entrypoints.serve.utils.api_utils import log_version_and_model
47 -from vllm.logger import init_logger
47 +from vllm.logger import configure_logging_from_args, init_logger
48 from vllm.usage.usage_lib import UsageContext
49 from vllm.utils.argparse_utils import FlexibleArgumentParser
50 from vllm.v1.engine.async_llm import AsyncLLM
@@ -195,6 +195,7 @@ def main():
195 parser = AsyncEngineArgs.add_cli_args(parser)
196
197 args = parser.parse_args()
198 + configure_logging_from_args(args)
199
200 # Run server
201 try:
vllm/entrypoints/launchers/render/entry.py
+6 -2
@@ -8,7 +8,7 @@ from argparse import Namespace
8
9 from vllm import AsyncEngineArgs, envs
10 from vllm.config import VllmConfig
11 -from vllm.logger import init_logger
11 +from vllm.logger import configure_logging_from_args, init_logger
12
13 from ..app import build_app
14 from ..launcher import serve_http, setup_server
@@ -87,7 +87,10 @@ async def run_launch_fastapi(args: argparse.Namespace) -> None:
87 # cache space warning from CpuPlatform.check_and_update_config.
88 envs.VLLM_CPU_KVCACHE_SPACE = 0
89
90 - vllm_config = VllmConfig(model_config=model_config)
90 + vllm_config = VllmConfig(
91 + model_config=model_config,
92 + logging_config=engine_args.create_logging_config(),
93 + )
94 shutdown_task = await build_and_serve_renderer(
95 vllm_config, listen_address, sock, args
96 )
@@ -115,6 +118,7 @@ if __name__ == "__main__":
118 )
119 parser = make_arg_parser(parser)
120 args = parser.parse_args()
121 + configure_logging_from_args(args)
122 validate_parsed_serve_args(args)
123
124 uvloop.run(run_launch_fastapi(args))
vllm/entrypoints/launchers/run_batch.py
+2 -1
@@ -61,7 +61,7 @@ from vllm.entrypoints.speech_to_text.translation.protocol import (
61 TranslationResponseVerbose,
62 )
63 from vllm.exceptions import VLLMValidationError
64 -from vllm.logger import init_logger
64 +from vllm.logger import configure_logging_from_args, init_logger
65 from vllm.reasoning import ReasoningParserManager
66 from vllm.utils import random_uuid
67 from vllm.utils.argparse_utils import FlexibleArgumentParser
@@ -289,6 +289,7 @@ def make_arg_parser(parser: FlexibleArgumentParser):
289 def parse_args():
290 parser = FlexibleArgumentParser(description="vLLM OpenAI-Compatible batch runner.")
291 args = make_arg_parser(parser).parse_args()
292 + configure_logging_from_args(args)
293
294 # Backward compatibility: If --url is set, use it for host
295 url_explicit = any(arg == "--url" or arg.startswith("--url=") for arg in sys.argv)
vllm/entrypoints/launchers/utils/server_utils.py
+7 -1
@@ -38,7 +38,13 @@ def get_uvicorn_log_config(args: Namespace) -> dict | None:
38 3. Otherwise, return None (use uvicorn defaults)
39 """
40 # First, try to load from file if specified
41 - log_config = load_log_config(args.log_config_file)
41 + logging_config = getattr(args, "logging_config", None)
42 + log_config_file = (
43 + logging_config.pylogging_config_file
44 + if logging_config is not None
45 + else getattr(args, "log_config_file", None)
46 + )
47 + log_config = load_log_config(log_config_file)
48 if log_config is not None:
49 return log_config
50
vllm/entrypoints/llm.py
+2 -1
@@ -42,7 +42,7 @@ from vllm.entrypoints.generate.beam_search.offline import BeamSearchOfflineMixin
42 from vllm.entrypoints.pooling.offline import PoolingOfflineMixin
43 from vllm.entrypoints.serve.utils.api_utils import log_non_default_args
44 from vllm.inputs import PromptType
45 -from vllm.logger import init_logger
45 +from vllm.logger import configure_logging_if_needed, init_logger
46 from vllm.lora.request import LoRARequest
47 from vllm.model_executor.layers.quantization import QuantizationMethods
48 from vllm.outputs import PoolingRequestOutput, RequestOutput
@@ -338,6 +338,7 @@ class LLM(BeamSearchOfflineMixin, PoolingOfflineMixin, OfflineInferenceMixin):
338 **kwargs,
339 )
340
341 + configure_logging_if_needed(engine_args.create_logging_config())
342 log_non_default_args(engine_args)
343
344 self.llm_engine = LLMEngine.from_engine_args(
vllm/entrypoints/serve/utils/request_logger.py
+1 -1
@@ -31,7 +31,7 @@ class RequestLogger:
31 "`--enable-log-requests` is set but "
32 "the minimum log level is higher than DEBUG. "
33 "Only limited information will be logged to minimize overhead. "
34 - "To view more details, set `VLLM_LOGGING_LEVEL=DEBUG`."
34 + "To view more details, set `--log-level DEBUG`."
35 )
36
37 def log_inputs(
vllm/logger.py
+87 -33
@@ -9,16 +9,21 @@ import os
9 import sys
10 from collections.abc import Generator, Hashable
11 from contextlib import contextmanager
12 +from copy import deepcopy
13 +from dataclasses import replace
14 from functools import lru_cache, partial
15 from logging import Logger
16 from logging.config import dictConfig
17 from os import path
18 from types import MethodType
17 -from typing import Any, Literal, cast
19 +from typing import TYPE_CHECKING, Any, Literal, cast
20
21 import vllm.envs as envs
22 from vllm.logging_utils import ColoredFormatter, NewLineFormatter
23
24 +if TYPE_CHECKING:
25 + from vllm.config.logging import LoggingConfig
26 +
27 _FORMAT = (
28 f"{envs.VLLM_LOGGING_PREFIX}%(levelname)s %(asctime)s "
29 "[%(fileinfo)s:%(lineno)d] %(message)s"
@@ -40,15 +45,18 @@ def _use_color() -> bool:
45
46 DEFAULT_LOGGING_CONFIG: dict[str, dict[str, Any] | Any] = {
47 "formatters": {
48 + # The "()" factory form forwards custom kwargs such as log_level.
49 "vllm": {
44 - "class": "vllm.logging_utils.NewLineFormatter",
50 + "()": "vllm.logging_utils.NewLineFormatter",
51 "datefmt": _DATE_FORMAT,
52 "format": _FORMAT,
53 + "log_level": envs.VLLM_LOGGING_LEVEL,
54 },
55 "vllm_color": {
49 - "class": "vllm.logging_utils.ColoredFormatter",
56 + "()": "vllm.logging_utils.ColoredFormatter",
57 "datefmt": _DATE_FORMAT,
58 "format": _FORMAT,
59 + "log_level": envs.VLLM_LOGGING_LEVEL,
60 },
61 },
62 "handlers": {
@@ -71,6 +79,8 @@ DEFAULT_LOGGING_CONFIG: dict[str, dict[str, Any] | Any] = {
79 "disable_existing_loggers": False,
80 }
81
82 +_last_configured_logging_config: "LoggingConfig | None" = None
83 +
84
85 @lru_cache
86 def _print_debug_once(logger: Logger, msg: str, *args: Hashable) -> None:
@@ -165,36 +175,46 @@ _METHODS_TO_PATCH = {
175 }
176
177
168 -def _configure_vllm_root_logger() -> None:
178 +def _configure_vllm_root_logger(config: "LoggingConfig | None" = None) -> None:
179 + """Configure logging from explicit config or bootstrap environment values."""
180 logging_config: dict[str, dict[str, Any] | Any] = {}
170 -
171 - if not envs.VLLM_CONFIGURE_LOGGING and envs.VLLM_LOGGING_CONFIG_PATH:
181 + if config is None:
182 + configure_logging = envs.VLLM_CONFIGURE_LOGGING
183 + log_level = envs.VLLM_LOGGING_LEVEL
184 + log_config_file = envs.VLLM_LOGGING_CONFIG_PATH
185 + else:
186 + configure_logging = config.configure_logging
187 + log_level = config.log_level
188 + log_config_file = config.pylogging_config_file
189 +
190 + if not configure_logging and log_config_file:
191 raise RuntimeError(
173 - "VLLM_CONFIGURE_LOGGING evaluated to false, but "
174 - "VLLM_LOGGING_CONFIG_PATH was given. VLLM_LOGGING_CONFIG_PATH "
175 - "implies VLLM_CONFIGURE_LOGGING. Please enable "
176 - "VLLM_CONFIGURE_LOGGING or unset VLLM_LOGGING_CONFIG_PATH."
192 + "Logging configuration is disabled, but a Python logging config "
193 + "file was given. pylogging_config_file requires "
194 + "configure_logging to be enabled."
195 )
196
179 - if envs.VLLM_CONFIGURE_LOGGING:
180 - logging_config = DEFAULT_LOGGING_CONFIG
197 + if configure_logging:
198 + logging_config = deepcopy(DEFAULT_LOGGING_CONFIG)
199
200 vllm_handler = logging_config["handlers"]["vllm"]
201 # Refresh these values in case env vars have changed.
184 - vllm_handler["level"] = envs.VLLM_LOGGING_LEVEL
202 + vllm_handler["level"] = log_level
203 vllm_handler["stream"] = envs.VLLM_LOGGING_STREAM
204 vllm_handler["formatter"] = "vllm_color" if _use_color() else "vllm"
205
206 vllm_loggers = logging_config["loggers"]["vllm"]
189 - vllm_loggers["level"] = envs.VLLM_LOGGING_LEVEL
207 + vllm_loggers["level"] = log_level
208 + for formatter in logging_config["formatters"].values():
209 + formatter["log_level"] = log_level
210
191 - if envs.VLLM_LOGGING_CONFIG_PATH:
192 - if not path.exists(envs.VLLM_LOGGING_CONFIG_PATH):
211 + if log_config_file:
212 + if not path.exists(log_config_file):
213 raise RuntimeError(
214 "Could not load logging config. File does not exist: %s",
195 - envs.VLLM_LOGGING_CONFIG_PATH,
215 + log_config_file,
216 )
197 - with open(envs.VLLM_LOGGING_CONFIG_PATH, encoding="utf-8") as file:
217 + with open(log_config_file, encoding="utf-8") as file:
218 custom_config = json.loads(file.read())
219
220 if not isinstance(custom_config, dict):
@@ -212,11 +232,57 @@ def _configure_vllm_root_logger() -> None:
232 if logging_config:
233 dictConfig(logging_config)
234
235 + # Transformers uses httpx to access the Hugging Face Hub. httpx is quite verbose,
236 + # so we set its logging level to WARNING when vLLM's logging level is INFO.
237 + # httpx2 is the successor huggingface_hub switches to in its 2.x releases.
238 + if log_level == "INFO":
239 + logging.getLogger("httpx").setLevel(logging.WARNING)
240 + logging.getLogger("httpx2").setLevel(logging.WARNING)
241 +
242 +
243 +def configure_logging(config: "LoggingConfig") -> None:
244 + """Apply a logging configuration in the current process."""
245 + _configure_vllm_root_logger(config)
246 + global _last_configured_logging_config
247 + _last_configured_logging_config = config
248 + _log_platform_warnings(config)
249 +
250 +
251 +def _log_platform_warnings(config: "LoggingConfig") -> None:
252 + """Emit platform diagnostics only after an enabled config is active."""
253 + if not config.configure_logging:
254 + return
255 +
256 + # Import lazily because platform modules use init_logger during import.
257 + from vllm.platforms import current_platform
258 +
259 + current_platform.log_warnings()
260 +
261 +
262 +def configure_logging_if_needed(config: "LoggingConfig") -> None:
263 + """Apply a logging configuration unless it is already active in this process."""
264 + if config != _last_configured_logging_config:
265 + configure_logging(config)
266 +
267 +
268 +def configure_logging_from_args(args: Any) -> "LoggingConfig":
269 + """Apply parsed logging arguments and retain them for child processes."""
270 + from vllm.config.logging import LoggingConfig
271 +
272 + config = getattr(args, "logging_config", None) or LoggingConfig()
273 + if hasattr(args, "log_level"):
274 + config = replace(config, log_level=args.log_level)
275 + if hasattr(args, "log_config_file"):
276 + config = replace(config, pylogging_config_file=args.log_config_file)
277 +
278 + configure_logging_if_needed(config)
279 + args.log_config_file = config.pylogging_config_file
280 + args.logging_config = config
281 + return config
282 +
283
284 def init_logger(name: str) -> _VllmLogger:
217 - """The main purpose of this function is to ensure that loggers are
218 - retrieved in such a way that we can be sure the root vllm logger has
219 - already been configured."""
285 + """Retrieve a logger and add vLLM's convenience logging methods."""
286 logger = logging.getLogger(name)
287
288 for method_name, method in _METHODS_TO_PATCH.items():
@@ -246,18 +312,6 @@ def current_formatter_type(logger: Logger) -> Literal["color", "newline", None]:
312 return None
313
314
249 -# The root logger is initialized when the module is imported.
250 -# This is thread-safe as the module is only imported once,
251 -# guaranteed by the Python GIL.
252 -_configure_vllm_root_logger()
253 -
254 -# Transformers uses httpx to access the Hugging Face Hub. httpx is quite verbose,
255 -# so we set its logging level to WARNING when vLLM's logging level is INFO.
256 -# httpx2 is the successor huggingface_hub switches to in its 2.x releases.
257 -if envs.VLLM_LOGGING_LEVEL == "INFO":
258 - logging.getLogger("httpx").setLevel(logging.WARNING)
259 - logging.getLogger("httpx2").setLevel(logging.WARNING)
260 -
315 logger = init_logger(__name__)
316
317
vllm/logging_utils/formatter.py
+4 -6
@@ -4,16 +4,14 @@
4 import logging
5 from pathlib import Path
6
7 -from vllm import envs
8 -
7
8 class NewLineFormatter(logging.Formatter):
9 """Adds logging prefix to newlines to align multi-line messages."""
10
13 - def __init__(self, fmt, datefmt=None, style="%"):
11 + def __init__(self, fmt, datefmt=None, style="%", *, log_level: str | None = None):
12 super().__init__(fmt, datefmt, style)
13
16 - self.use_relpath = envs.VLLM_LOGGING_LEVEL == "DEBUG"
14 + self.use_relpath = log_level == "DEBUG"
15 if self.use_relpath:
16 self.root_dir = Path(__file__).resolve().parent.parent.parent
17
@@ -95,7 +93,7 @@ class ColoredFormatter(NewLineFormatter):
93 GREY = "\033[90m" # Grey for timestamp and file info
94 RESET = "\033[0m"
95
98 - def __init__(self, fmt, datefmt=None, style="%"):
96 + def __init__(self, fmt, datefmt=None, style="%", *, log_level: str | None = None):
97 # Inject grey color codes into format string for timestamp and file info
98 if fmt:
99 # Wrap %(asctime)s with grey
@@ -107,7 +105,7 @@ class ColoredFormatter(NewLineFormatter):
105 )
106
107 # Call parent __init__ with potentially modified format string
110 - super().__init__(fmt, datefmt, style)
108 + super().__init__(fmt, datefmt, style, log_level=log_level)
109
110 def format(self, record):
111 # Store original levelname to restore later (in case record is reused)
vllm/platforms/cuda.py
+2 -7
@@ -298,10 +298,6 @@ class CudaPlatformBase(Platform):
298 def is_fully_connected(cls, device_ids: list[int]) -> bool:
299 raise NotImplementedError
300
301 - @classmethod
302 - def log_warnings(cls):
303 - pass
304 -
301 @classmethod
302 def is_pin_memory_available(cls) -> bool:
303 if in_wsl():
@@ -1009,11 +1005,12 @@ class NvmlCudaPlatform(CudaPlatformBase):
1005 len(set(device_names)) > 1
1006 and os.environ.get("CUDA_DEVICE_ORDER") != "PCI_BUS_ID"
1007 ):
1012 - logger.warning(
1008 + logger.warning_once(
1009 "Detected different devices in the system: %s. Please"
1010 " make sure to set `CUDA_DEVICE_ORDER=PCI_BUS_ID` to "
1011 "avoid unexpected behavior.",
1012 ", ".join(device_names),
1013 + scope="process",
1014 )
1015
1016
@@ -1065,5 +1062,3 @@ finally:
1062 pynvml.nvmlShutdown()
1063
1064 CudaPlatform = NvmlCudaPlatform if nvml_available else NonNvmlCudaPlatform
1068 -
1069 -CudaPlatform.log_warnings()
vllm/platforms/interface.py
+4
@@ -147,6 +147,10 @@ class Platform:
147 # empty string means the device does not support ray
148 ray_device_key: str = ""
149
150 + @classmethod
151 + def log_warnings(cls) -> None:
152 + """Log platform-specific diagnostics after logging is configured."""
153 +
154 # platform-agnostic way to specify the device control environment variable,
155 # .e.g. CUDA_VISIBLE_DEVICES for CUDA.
156 # hint: search for "get_visible_accelerator_ids_env_var" in
vllm/utils/system_utils.py
+3 -2
@@ -4,6 +4,7 @@
4 from __future__ import annotations
5
6 import contextlib
7 +import logging
8 import multiprocessing
9 import os
10 import signal
@@ -63,7 +64,7 @@ def suppress_stdout():
64 """Suppress stdout from C libraries at the file descriptor level.
65
66 Only suppresses stdout, not stderr, to preserve error messages.
66 - Suppression is disabled when VLLM_LOGGING_LEVEL is set to DEBUG.
67 + Suppression is disabled when vLLM debug logging is enabled.
68
69 Example:
70 with suppress_stdout():
@@ -72,7 +73,7 @@ def suppress_stdout():
73
74 """
75 # Don't suppress if logging level is DEBUG
75 - if envs.VLLM_LOGGING_LEVEL == "DEBUG":
76 + if logger.isEnabledFor(logging.DEBUG):
77 yield
78 return
79
vllm/v1/engine/async_llm.py
+3 -1
@@ -28,7 +28,7 @@ from vllm.exceptions import (
28 VLLMValidationError,
29 )
30 from vllm.inputs import EngineInput, PromptType
31 -from vllm.logger import init_logger
31 +from vllm.logger import configure_logging_if_needed, init_logger
32 from vllm.lora.request import LoRARequest
33 from vllm.multimodal import MULTIMODAL_REGISTRY, MultiModalRegistry
34 from vllm.outputs import STREAM_FINISHED, PoolingRequestOutput, RequestOutput
@@ -122,6 +122,8 @@ class AsyncLLM(EngineClient):
122 None
123
124 """
125 + configure_logging_if_needed(vllm_config.logging_config)
126 +
127 # Ensure we can serialize custom transformer configs
128 maybe_register_config_serialize_by_value()
129
vllm/v1/engine/coordinator.py
+11 -3
@@ -9,8 +9,8 @@ import weakref
9 import msgspec.msgpack
10 import zmq
11
12 -from vllm.config import ParallelConfig
13 -from vllm.logger import init_logger
12 +from vllm.config import LoggingConfig, ParallelConfig
13 +from vllm.logger import configure_logging, init_logger
14 from vllm.utils.network_utils import make_zmq_socket
15 from vllm.utils.system_utils import get_mp_context, set_process_title
16 from vllm.v1.engine import EngineCoreOutputs, EngineCoreRequestType
@@ -77,7 +77,10 @@ class DPCoordinator:
77 zmq_addr_pipe.close()
78
79 def __init__(
80 - self, parallel_config: ParallelConfig, enable_wave_coordination: bool = True
80 + self,
81 + parallel_config: ParallelConfig,
82 + enable_wave_coordination: bool = True,
83 + logging_config: LoggingConfig | None = None,
84 ):
85 dp_size = parallel_config.data_parallel_size
86 assert dp_size > 1, "Coordinator only used for data parallel"
@@ -108,6 +111,7 @@ class DPCoordinator:
111 "back_publish_address": back_publish_address,
112 "zmq_addr_pipe": child_zmq_addr_pipe,
113 "enable_wave_coordination": enable_wave_coordination,
114 + "logging_config": logging_config,
115 },
116 daemon=True,
117 )
@@ -167,7 +171,11 @@ class DPCoordinatorProc:
171 zmq_addr_pipe=None,
172 min_stats_update_interval_ms: int = 100,
173 enable_wave_coordination: bool = True,
174 + logging_config: LoggingConfig | None = None,
175 ):
176 + if logging_config is not None:
177 + configure_logging(logging_config)
178 +
179 coordinator = DPCoordinatorProc(
180 engine_count=engine_count,
181 min_stats_update_interval_ms=min_stats_update_interval_ms,
vllm/v1/engine/core.py
+8 -2
@@ -28,7 +28,7 @@ from vllm.distributed import (
28 stateless_destroy_torch_distributed_process_group,
29 )
30 from vllm.envs import enable_envs_cache
31 -from vllm.logger import init_logger
31 +from vllm.logger import configure_logging, init_logger
32 from vllm.logging_utils.dump_input import dump_engine_exception
33 from vllm.lora.request import LoRARequest
34 from vllm.multimodal.cache import (
@@ -1350,6 +1350,10 @@ class EngineCoreProc(EngineCore):
1350 @staticmethod
1351 def run_engine_core(*args, dp_rank: int = 0, local_dp_rank: int = 0, **kwargs):
1352 """Launch EngineCore busy loop in background process."""
1353 + vllm_config: VllmConfig = kwargs["vllm_config"]
1354 + if logging_config := getattr(vllm_config, "logging_config", None):
1355 + configure_logging(logging_config)
1356 +
1357 # Ensure we can serialize transformer config after spawning
1358 maybe_register_config_serialize_by_value()
1359
@@ -1357,7 +1361,6 @@ class EngineCoreProc(EngineCore):
1361 signal_callback: SignalCallback | None = None
1362 clean_shutdown = False
1363 try:
1360 - vllm_config: VllmConfig = kwargs["vllm_config"]
1364 parallel_config: ParallelConfig = vllm_config.parallel_config
1365 data_parallel = parallel_config.data_parallel_size > 1 or dp_rank > 0
1366 if data_parallel:
@@ -2473,6 +2476,9 @@ class EngineCoreActorMixin:
2476 dp_rank: int = 0,
2477 local_dp_rank: int = 0,
2478 ):
2479 + if logging_config := getattr(vllm_config, "logging_config", None):
2480 + configure_logging(logging_config)
2481 +
2482 # Initialize tracer for distributed tracing if configured.
2483 maybe_init_worker_tracer(
2484 instrumenting_module_name="vllm.engine_core",
vllm/v1/engine/llm_engine.py
+2 -1
@@ -16,7 +16,7 @@ from vllm.distributed import stateless_destroy_torch_distributed_process_group
16 from vllm.distributed.parallel_state import get_dp_group
17 from vllm.engine.arg_utils import EngineArgs
18 from vllm.inputs import EngineInput, PromptType
19 -from vllm.logger import init_logger
19 +from vllm.logger import configure_logging_if_needed, init_logger
20 from vllm.lora.request import LoRARequest
21 from vllm.multimodal import MULTIMODAL_REGISTRY, MultiModalRegistry
22 from vllm.outputs import PoolingRequestOutput, RequestOutput
@@ -60,6 +60,7 @@ class LLMEngine:
60 mm_registry: MultiModalRegistry = MULTIMODAL_REGISTRY,
61 multiprocess_mode: bool = False,
62 ) -> None:
63 + configure_logging_if_needed(vllm_config.logging_config)
64 self.vllm_config = vllm_config
65 self.model_config = vllm_config.model_config
66 self.observability_config = vllm_config.observability_config
vllm/v1/engine/utils.py
+1
@@ -1128,6 +1128,7 @@ def launch_core_engines(
1128 coordinator = DPCoordinator(
1129 parallel_config,
1130 enable_wave_coordination=vllm_config.model_config.is_moe,
1131 + logging_config=vllm_config.logging_config,
1132 )
1133
1134 addresses.coordinator_input, addresses.coordinator_output = (
vllm/v1/executor/multiproc_executor.py
+4 -1
@@ -42,7 +42,7 @@ from vllm.distributed.parallel_state import (
42 model_parallel_is_initialized,
43 )
44 from vllm.envs import enable_envs_cache
45 -from vllm.logger import init_logger
45 +from vllm.logger import configure_logging, init_logger
46 from vllm.platforms import current_platform
47 from vllm.tracing import instrument, maybe_init_worker_tracer
48 from vllm.utils import numa_utils
@@ -853,6 +853,9 @@ class WorkerProc:
853 def worker_main(*args, **kwargs):
854 """Worker initialization and execution loops.
855 This runs a background process"""
856 + if logging_config := getattr(kwargs["vllm_config"], "logging_config", None):
857 + configure_logging(logging_config)
858 +
859 # Signal handler used for graceful termination.
860 # SystemExit exception is only raised once to allow this and worker
861 # processes to terminate without error
vllm/v1/executor/ray_executor_v2.py
+4 -1
@@ -16,7 +16,7 @@ from vllm.distributed.device_communicators.shm_broadcast import (
16 Handle,
17 MessageQueue,
18 )
19 -from vllm.logger import init_logger
19 +from vllm.logger import configure_logging, init_logger
20 from vllm.platforms import current_platform
21 from vllm.utils.network_utils import (
22 get_distributed_init_method,
@@ -113,6 +113,9 @@ class RayWorkerProc(WorkerProc):
113 is_driver_worker: bool,
114 is_driver_node: bool = False,
115 ):
116 + if logging_config := getattr(vllm_config, "logging_config", None):
117 + configure_logging(logging_config)
118 +
119 # Defer WorkerProc.__init__ until GPU IDs are known.
120 self._is_driver_node = is_driver_node
121 self._parallel_config = vllm_config.parallel_config
vllm/v1/utils.py
+5
@@ -520,6 +520,11 @@ def run_api_server_worker_proc(
520 listen_address, sock, args, client_config=None, **uvicorn_kwargs
521 ) -> None:
522 """Entrypoint for individual API server worker processes."""
523 + if logging_config := getattr(args, "logging_config", None):
524 + from vllm.logger import configure_logging
525 +
526 + configure_logging(logging_config)
527 +
528 from vllm.entrypoints.launchers.api_server.entry import run_server_worker
529
530 client_config = client_config or {}
vllm/v1/worker/gpu_ubatch_wrapper.py
+2 -2
@@ -1,6 +1,7 @@
1 # SPDX-License-Identifier: Apache-2.0
2 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
4 +import logging
5 import threading
6 from collections.abc import Callable
7 from dataclasses import dataclass
@@ -8,7 +9,6 @@ from typing import Any
9
10 import torch
11
11 -import vllm.envs as envs
12 from vllm.compilation.cuda_graph import CUDAGraphWrapper
13 from vllm.config import CUDAGraphMode, VllmConfig
14 from vllm.distributed.device_communicators.pynccl_allocator import set_graph_pool_id
@@ -89,7 +89,7 @@ class UBatchWrapper:
89
90 self.sm_control = create_sm_control_context(vllm_config.parallel_config)
91 self.device = device
92 - self.is_debugging_mode = envs.VLLM_LOGGING_LEVEL == "DEBUG"
92 + self.is_debugging_mode = logger.isEnabledFor(logging.DEBUG)
93 self._runnable_str = str(runnable) if self.is_debugging_mode else None
94
95 @property