| 1 | # yamllint disable rule:line-length |
| 2 | --- |
| 3 | id: 'service-discovery-docker' |
| 4 | meta: |
| 5 | kind: 'docker' |
| 6 | name: 'Docker' |
| 7 | tagline: 'Running containers on the local Docker daemon.' |
| 8 | link: 'https://www.docker.com/' |
| 9 | icon_filename: 'docker.svg' |
| 10 | keywords: |
| 11 | - 'service discovery' |
| 12 | - 'sd' |
| 13 | - 'docker' |
| 14 | - 'containers' |
| 15 | - 'discovery' |
| 16 | overview: |
| 17 | description: | |
| 18 | Netdata can automatically discover running Docker containers on the local Docker daemon and generate collector jobs for the services running inside them. The discoverer queries the Docker API on a fixed interval, builds one target per container port, and applies your `services:` rules to render collector job YAML — typically picking the right go.d module from the container image (nginx, postgres, redis, …). |
| 19 | |
| 20 | This page covers Docker-specific setup. For the broader Service Discovery model and the shared template-helper reference, see [Service Discovery](/src/collectors/SERVICE-DISCOVERY.md). |
| 21 | how_it_works: | |
| 22 | Each discovery cycle, the discoverer: |
| 23 | |
| 24 | 1. **Calls** `ContainerList` on the Docker API at the configured `address`. |
| 25 | 2. **Builds one target per `(container, network, port)` triple** for every container that has at least one network and at least one published port. Containers running in `network: host` mode are intentionally skipped — those are picked up by the [`net_listeners`](/src/go/plugin/go.d/discovery/sdext/discoverer/netlistensd/README.md) discoverer instead. |
| 26 | 3. **Exposes** target fields: `.Name`, `.Image`, `.Command`, `.Labels`, `.PrivatePort`, `.PublicPort`, `.PublicPortIP`, `.PortProtocol`, `.NetworkMode`, `.NetworkDriver`, `.IPAddress`, `.Address` (the convenience `IPAddress:PrivatePort`). |
| 27 | 4. **Runs the `services:` rules** against each target. The default stock conf carries curated rules for ~40 popular images (nginx, postgres, redis, rabbitmq, etc.) keyed on `.Image` patterns. |
| 28 | 5. **Reconciles** disappeared containers — when a container exits, its target is removed and the corresponding collector job stops on the next reconcile. |
| 29 | limitations: | |
| 30 | - Containers in **`network: host` mode** are not produced as Docker targets. Configure the `net_listeners` discoverer to pick them up via the host's process table. |
| 31 | - Only **TCP** ports are typically useful; the stock conf's first rule explicitly skips non-TCP, missing-port, and IPv6-mapped entries. |
| 32 | - Only **published ports** appear as targets. A container that exposes ports only inside a Docker network without `-p` mapping still produces a target via its private port and network IP. |
| 33 | - The discoverer reads the live container list; it does not inspect image manifests, healthcheck output, or process tables inside the container. Anything beyond labels/image/ports must be inferred via service rules. |
| 34 | - Only the **local Docker daemon** is supported (Unix socket or TCP). There is no docker-swarm or remote-cluster discovery mode. |
| 35 | setup: |
| 36 | prerequisites: |
| 37 | list: |
| 38 | - title: 'Access to the Docker socket' |
| 39 | description: | |
| 40 | The Netdata Agent must be able to reach the Docker daemon. The default `address` is `unix:///var/run/docker.sock`. If you run Netdata in a container, mount the socket: `-v /var/run/docker.sock:/var/run/docker.sock:ro`. The Netdata user (or the container) must have read access to the socket. |
| 41 | - title: 'Discovery is enabled by default' |
| 42 | description: | |
| 43 | The stock conf at `/etc/netdata/go.d/sd/docker.conf` ships with `disabled: no` and a curated set of `services:` rules covering ~40 popular images. To turn discovery off, set `disabled: yes` at the top of the file. |
| 44 | configuration: |
| 45 | file: |
| 46 | name: 'go.d/sd/docker.conf' |
| 47 | options: |
| 48 | description: | |
| 49 | The configuration file has two top-level blocks: `discoverer:` (the options below) and `services:` (rules that turn discovered containers into collector jobs — see [Service Rules](#service-rules)). |
| 50 | |
| 51 | After editing the file, restart the Netdata Agent to load the updated discovery pipeline. |
| 52 | folding: |
| 53 | title: 'Discoverer options' |
| 54 | enabled: false |
| 55 | list: |
| 56 | - name: 'address' |
| 57 | description: 'Docker daemon address.' |
| 58 | default_value: 'unix:///var/run/docker.sock' |
| 59 | required: false |
| 60 | detailed_description: | |
| 61 | Supports both Unix-socket (`unix:///var/run/docker.sock`) and TCP (`tcp://hostname:2375`) endpoints. |
| 62 | |
| 63 | If unset, Netdata also honors the `DOCKER_HOST` environment variable when present. |
| 64 | - name: 'timeout' |
| 65 | description: 'Maximum time to wait for a Docker API response (per request).' |
| 66 | default_value: '2s' |
| 67 | required: false |
| 68 | examples: |
| 69 | folding: |
| 70 | title: 'Configuration examples' |
| 71 | enabled: true |
| 72 | list: |
| 73 | - name: 'Default (Unix socket)' |
| 74 | description: 'Use the default local Docker socket and the stock services rules.' |
| 75 | config: | |
| 76 | disabled: no |
| 77 | discoverer: |
| 78 | docker: |
| 79 | address: unix:///var/run/docker.sock |
| 80 | services: |
| 81 | # See the stock conf for the full curated rule set. |
| 82 | - id: skip |
| 83 | match: | |
| 84 | {{ or (eq .NetworkMode "host") (not (eq .PortProtocol "tcp")) (empty .PrivatePort) }} |
| 85 | - id: nginx |
| 86 | match: '{{ match "sp" .Image "nginx nginx:*" }}' |
| 87 | config_template: | |
| 88 | name: docker_{{.Name}} |
| 89 | url: http://{{.Address}}/stub_status |
| 90 | - name: 'Remote daemon over TCP' |
| 91 | description: 'Point the discoverer at a remote Docker daemon. TLS is not yet wired into the discoverer; either expose the daemon on a trusted internal network or use a stunnel/socat proxy.' |
| 92 | config: | |
| 93 | disabled: no |
| 94 | discoverer: |
| 95 | docker: |
| 96 | address: tcp://docker.internal:2375 |
| 97 | timeout: 5s |
| 98 | services: |
| 99 | - id: skip |
| 100 | match: '{{ or (eq .NetworkMode "host") (not (eq .PortProtocol "tcp")) (empty .PrivatePort) }}' |
| 101 | - id: redis |
| 102 | match: '{{ match "sp" .Image "redis redis:* */redis */redis:*" }}' |
| 103 | config_template: | |
| 104 | name: docker_{{.Name}} |
| 105 | address: redis://@{{.Address}} |
| 106 | services: |
| 107 | description: | |
| 108 | A `services:` rule turns each discovered container target into one or more collector jobs. Most rules match on `.Image` (using the `match "sp"` simple-pattern helper for the typical `image image:* */image */image:*` family), some also gate on `.PrivatePort`, and a few use `.Labels` to honor user intent. |
| 109 | |
| 110 | The shared rule model — function reference (`match`, `glob`, sprig, `toYaml`), `config_template` rendering rules, and the `missingkey=error` failure semantics — lives on the [Service Discovery](/src/collectors/SERVICE-DISCOVERY.md) hub page. The notes below are Docker-specific. |
| 111 | evaluation: |
| 112 | description: | |
| 113 | Quick reference — see [Rule evaluation semantics](/src/collectors/SERVICE-DISCOVERY.md#rule-evaluation-semantics) on the hub page for the full model. |
| 114 | list: |
| 115 | - name: 'The first rule in the stock conf is a skip rule' |
| 116 | description: 'It drops targets that are unreachable or uninteresting (host networking, non-TCP, missing port, IPv6-mapped public IP). Keep it as the first rule — every subsequent rule assumes it has filtered out the noise.' |
| 117 | - name: 'Match on .Image with `match "sp"`' |
| 118 | description: | |
| 119 | The simple-patterns matcher (`match "sp" .Image "nginx nginx:* */nginx */nginx:*"`) is the idiomatic way to handle the four-form image family (bare, tagged, namespaced, namespaced-tagged). Use `glob` if you only need shell-style globbing without the simple-patterns engine. |
| 120 | - name: 'Module inference from rule id' |
| 121 | description: 'For Docker, set `id: <module-name>` (e.g. `id: nginx`) so the rendered job inherits the module name automatically. Use a different `id` only when you also include `module:` explicitly in the template.' |
| 122 | template_variables: |
| 123 | description: 'Available inside both `match` expressions and `config_template` bodies for Docker targets.' |
| 124 | list: |
| 125 | - name: '.Name' |
| 126 | type: 'string' |
| 127 | description: 'Container name (without the leading slash).' |
| 128 | - name: '.Image' |
| 129 | type: 'string' |
| 130 | description: 'Container image as reported by Docker (e.g. `nginx:1.25`, `myorg/redis:6`).' |
| 131 | - name: '.Command' |
| 132 | type: 'string' |
| 133 | description: 'Container command line.' |
| 134 | - name: '.Labels' |
| 135 | type: 'map' |
| 136 | description: 'All container labels. Use `index .Labels "key"` or `hasKey .Labels "key"` to read individual entries.' |
| 137 | - name: '.IPAddress' |
| 138 | type: 'string' |
| 139 | description: 'IP of the container on the matched network.' |
| 140 | - name: '.Address' |
| 141 | type: 'string' |
| 142 | description: 'Convenience `IPAddress:PrivatePort` — the canonical address used in most stock rule templates.' |
| 143 | - name: '.PrivatePort' |
| 144 | type: 'string' |
| 145 | description: 'Container-side port.' |
| 146 | - name: '.PublicPort' |
| 147 | type: 'string' |
| 148 | description: 'Host-side port (empty when the container does not publish a host mapping for this port).' |
| 149 | - name: '.PublicPortIP' |
| 150 | type: 'string' |
| 151 | description: 'Host IP that the container port is bound to (empty when no public mapping).' |
| 152 | - name: '.PortProtocol' |
| 153 | type: 'string' |
| 154 | description: 'Port protocol — `tcp` or `udp`. Stock rules typically gate on `eq .PortProtocol "tcp"`.' |
| 155 | - name: '.NetworkMode' |
| 156 | type: 'string' |
| 157 | description: 'Container network mode (`bridge`, `host`, `overlay`, custom network names, …). The stock skip rule drops `host` mode.' |
| 158 | - name: '.NetworkDriver' |
| 159 | type: 'string' |
| 160 | description: 'Driver of the matched network.' |
| 161 | - name: '.ID' |
| 162 | type: 'string' |
| 163 | description: 'Container ID (full hex).' |
| 164 | examples: |
| 165 | description: 'Each example shows one or more entries from the `services:` array. Order matters — see [How rules are evaluated](#how-rules-are-evaluated).' |
| 166 | list: |
| 167 | - name: 'Skip rule for unreachable / uninteresting targets' |
| 168 | description: 'The first rule in the stock conf. Drops `host` networking (those are local-listener targets), non-TCP ports, ports without a private side, and IPv6-mapped public IPs. Place it first.' |
| 169 | config: | |
| 170 | - id: skip |
| 171 | match: | |
| 172 | {{ $netNOK := eq .NetworkMode "host" -}} |
| 173 | {{ $protoNOK := not (eq .PortProtocol "tcp") -}} |
| 174 | {{ $portNOK := empty .PrivatePort -}} |
| 175 | {{ $addrNOK := or (empty .IPAddress) (glob .PublicPortIP "*:*") -}} |
| 176 | {{ or $netNOK $protoNOK $portNOK $addrNOK }} |
| 177 | - name: 'Nginx — module inferred from rule id' |
| 178 | description: | |
| 179 | Match the four common image-name forms with `match "sp"`. `id: nginx` makes the module name infer to `nginx` automatically — no `module:` line needed in the template. |
| 180 | config: | |
| 181 | - id: nginx |
| 182 | match: '{{ match "sp" .Image "nginx nginx:*" }}' |
| 183 | config_template: | |
| 184 | - name: docker_{{.Name}} |
| 185 | url: http://{{.Address}}/stub_status |
| 186 | - name: docker_{{.Name}} |
| 187 | url: http://{{.Address}}/basic_status |
| 188 | - name: docker_{{.Name}} |
| 189 | url: http://{{.Address}}/nginx_status |
| 190 | - name: docker_{{.Name}} |
| 191 | url: http://{{.Address}}/status |
| 192 | - name: 'Postgres — explicit module override' |
| 193 | description: | |
| 194 | When the rule `id` is something other than the target module name, set `module:` explicitly inside the rendered job. |
| 195 | config: | |
| 196 | - id: postgres |
| 197 | match: '{{ or (eq .PrivatePort "5432") (match "sp" .Image "postgres postgres:* */postgres */postgres:* */postgresql */postgresql:*") }}' |
| 198 | config_template: | |
| 199 | module: postgres |
| 200 | name: docker_{{.Name}} |
| 201 | dsn: postgres://netdata:postgres@{{.Address}}/postgres |
| 202 | - name: 'Label-driven custom matching' |
| 203 | description: | |
| 204 | Use container labels to override behaviour without changing rules — e.g. opt a container in or out of monitoring, or pick a non-default endpoint. The example below requires the operator to set the label `netdata.go.d/module=mymodule` on the container. |
| 205 | config: | |
| 206 | - id: label-routed |
| 207 | match: '{{ and (hasKey .Labels "netdata.go.d/module") (eq (index .Labels "netdata.go.d/module") "mymodule") }}' |
| 208 | config_template: | |
| 209 | module: mymodule |
| 210 | name: docker_{{.Name}} |
| 211 | url: http://{{.Address}}/metrics |
| 212 | verify: |
| 213 | description: 'After enabling the discoverer, confirm it is finding containers and producing jobs.' |
| 214 | checks: |
| 215 | list: |
| 216 | - name: 'Confirm containers are being listed' |
| 217 | description: | |
| 218 | Watch the agent log for Docker discoverer messages. With systemd: |
| 219 | |
| 220 | ```bash |
| 221 | journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep "discoverer=docker" |
| 222 | ``` |
| 223 | |
| 224 | On a healthy daemon you should see the agent successfully calling `ContainerList`. If the log shows `error on creating docker client` or permission errors, the agent cannot reach `/var/run/docker.sock`. |
| 225 | - name: 'Confirm jobs are being created' |
| 226 | description: | |
| 227 | In the Netdata UI go to `Collectors -> go.d -> <module>` for whatever modules your service rules target (nginx, redis, postgres, …) — each container that matched a rule should appear as a `docker_<container-name>` job. |
| 228 | - name: 'Confirm metrics are being collected' |
| 229 | description: | |
| 230 | If a job was created but no charts appear, the rendered `config_template` produced a config the collector module rejected (wrong DSN, unreachable URL, missing credential). Check the collector's log. |
| 231 | troubleshooting: |
| 232 | problems: |
| 233 | list: |
| 234 | - name: 'Permission denied on docker.sock' |
| 235 | description: | |
| 236 | The Netdata user must be able to read the Docker socket. On a typical Linux host: |
| 237 | |
| 238 | ```bash |
| 239 | sudo usermod -aG docker netdata |
| 240 | systemctl restart netdata |
| 241 | ``` |
| 242 | |
| 243 | In containers, mount the socket read-only and verify the file is readable from inside. |
| 244 | - name: 'No targets discovered for containers in `host` networking' |
| 245 | description: | |
| 246 | `host`-mode containers are intentionally skipped by the Docker discoverer. Enable the `net_listeners` discoverer instead — it picks up locally-listening processes, which includes `host`-mode containers. |
| 247 | - name: 'Wrong module picked for an image' |
| 248 | description: | |
| 249 | Stock rules match on `.Image` patterns. Custom forks or in-house image names won't match. Add a rule above the stock catch-alls keyed on your own image name (`match "sp" .Image "myorg/nginx myorg/nginx:*"`) or use a `.Labels`-driven rule. |
| 250 | - name: 'Generated jobs fail to start' |
| 251 | description: | |
| 252 | Common causes: the rendered URL is not reachable from the agent (different network, firewall); credentials baked into the template are wrong; the module's port is not the one Docker reported. Check the rendered job YAML in the agent's debug output. |