| 1 | <!--startmeta |
| 2 | custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/network-flows/configuration.md" |
| 3 | sidebar_label: "Configuration" |
| 4 | learn_status: "Published" |
| 5 | learn_rel_path: "Network Flows" |
| 6 | keywords: ['configuration', 'netflow.yaml', 'tuning', 'retention', 'listener'] |
| 7 | endmeta--> |
| 8 | |
| 9 | <!-- markdownlint-disable-file --> |
| 10 | |
| 11 | # Configuration |
| 12 | |
| 13 | The netflow plugin reads its configuration from `netflow.yaml`. The defaults are good for initial validation; production deployments usually tune the listener address and retention once the observed flow rate is known. This page documents every option, with its real default and the file that defines it. |
| 14 | |
| 15 | ## Where the file lives |
| 16 | |
| 17 | | Path | Purpose | |
| 18 | |---|---| |
| 19 | | `/etc/netdata/netflow.yaml` | Your configuration. Edits here survive package upgrades. | |
| 20 | | `/usr/lib/netdata/conf.d/netflow.yaml` | The stock file shipped with the package. Reference only. | |
| 21 | |
| 22 | The plugin reads the user file when it exists, and the stock file otherwise. To start customising, copy the stock file: |
| 23 | |
| 24 | ```bash |
| 25 | sudo cp /usr/lib/netdata/conf.d/netflow.yaml /etc/netdata/netflow.yaml |
| 26 | ``` |
| 27 | |
| 28 | ## Three things to know before you edit |
| 29 | |
| 30 | 1. **Restart required.** There is no live-reload for plugin configuration. After saving the file, run `sudo systemctl restart netdata`. Only the GeoIP databases reload on a timer; everything else needs a restart. |
| 31 | 2. **Strict YAML.** Every section refuses unknown keys. A misspelled key fails the plugin at startup with an error in the journal. If you see "the plugin won't start after my edit", check for a typo before anything else. |
| 32 | 3. **CLI flags vs YAML.** When the plugin runs as a Netdata Agent plugin (the normal case), only the YAML is read — CLI flags do nothing. The CLI flags shown below apply only if you run the binary directly outside of Netdata. |
| 33 | |
| 34 | ## Top-level layout |
| 35 | |
| 36 | ```yaml |
| 37 | enabled: true # global on/off |
| 38 | listener: { ... } # UDP socket and journal sync |
| 39 | protocols: { ... } # which protocols to accept; decapsulation; timestamps |
| 40 | journal: { ... } # tier directories, retention, query guardrails |
| 41 | enrichment: { ... } # GeoIP, classifiers, ASN, BMP, BioRIS, network sources |
| 42 | ``` |
| 43 | |
| 44 | The YAML form is strictly nested — every key lives inside its section as shown above. The flat form is only valid for CLI flags (e.g. `--netflow-listen 0.0.0.0:2055`), which the plugin exposes for one-off invocation; the YAML schema rejects unknown top-level keys. |
| 45 | |
| 46 | ## `enabled` |
| 47 | |
| 48 | ```yaml |
| 49 | enabled: true |
| 50 | ``` |
| 51 | |
| 52 | Set to `false` to turn the entire flow plugin off. The plugin still loads but does nothing. Default: `true`. |
| 53 | |
| 54 | ## `listener` |
| 55 | |
| 56 | Controls the UDP socket and the journal write cadence. |
| 57 | |
| 58 | ```yaml |
| 59 | listener: |
| 60 | listen: "0.0.0.0:2055" |
| 61 | max_packet_size: 9216 |
| 62 | sync_every_entries: 1024 |
| 63 | sync_interval: "1s" |
| 64 | ``` |
| 65 | |
| 66 | | Key | CLI flag | Default | Notes | |
| 67 | |---|---|---|---| |
| 68 | | `listen` | `--netflow-listen` | `0.0.0.0:2055` | Address and port for the UDP socket. Same socket handles NetFlow v5/v7/v9, IPFIX, and sFlow. | |
| 69 | | `max_packet_size` | `--netflow-max-packet-size` | `9216` | Maximum UDP datagram in bytes. Increase for jumbo sFlow datagrams or routers that send oversized IPFIX. | |
| 70 | | `sync_every_entries` | `--netflow-sync-every-entries` | `1024` | Flush the raw journal to disk after this many records, regardless of `sync_interval`. | |
| 71 | | `sync_interval` | `--netflow-sync-interval` | `1s` | Maximum time between forced flushes. | |
| 72 | |
| 73 | ### UDP buffer tuning is not in this file |
| 74 | |
| 75 | If you receive a high flow rate, the kernel UDP receive buffer matters more than `max_packet_size`. Tune at the kernel level: |
| 76 | |
| 77 | ```bash |
| 78 | sudo sysctl -w net.core.rmem_max=33554432 |
| 79 | sudo sysctl -w net.core.rmem_default=8388608 |
| 80 | sudo sysctl -w net.core.netdev_max_backlog=250000 |
| 81 | ``` |
| 82 | |
| 83 | Persist these in `/etc/sysctl.d/99-netflow.conf`. The plugin does not call `setsockopt(SO_RCVBUF)` itself; whatever the kernel default is, that's what the listener gets. |
| 84 | |
| 85 | ## `protocols` |
| 86 | |
| 87 | ```yaml |
| 88 | protocols: |
| 89 | v5: true |
| 90 | v7: true |
| 91 | v9: true |
| 92 | ipfix: true |
| 93 | sflow: true |
| 94 | decapsulation_mode: none |
| 95 | timestamp_source: input |
| 96 | ``` |
| 97 | |
| 98 | | Key | CLI flag | Default | Values | |
| 99 | |---|---|---|---| |
| 100 | | `v5` | `--netflow-enable-v5` | `true` | Boolean. NetFlow v5. | |
| 101 | | `v7` | `--netflow-enable-v7` | `true` | Boolean. NetFlow v7 (Catalyst). | |
| 102 | | `v9` | `--netflow-enable-v9` | `true` | Boolean. NetFlow v9. | |
| 103 | | `ipfix` | `--netflow-enable-ipfix` | `true` | Boolean. IPFIX. | |
| 104 | | `sflow` | `--netflow-enable-sflow` | `true` | Boolean. sFlow v5. | |
| 105 | | `decapsulation_mode` | `--netflow-decapsulation-mode` | `none` | `none`, `srv6`, `vxlan`. Strips outer headers from the data-link section, surfaces the inner 5-tuple. | |
| 106 | | `timestamp_source` | `--netflow-timestamp-source` | `input` | Which timestamp is stored as `_SOURCE_REALTIME_TIMESTAMP`. See below. | |
| 107 | |
| 108 | You must keep at least one protocol enabled or the plugin refuses to start. |
| 109 | |
| 110 | ### `timestamp_source` values |
| 111 | |
| 112 | - **`input`** (default) — the time the plugin received the datagram. |
| 113 | - **`netflow_packet`** — the time the exporter put in the NetFlow/IPFIX header. |
| 114 | - **`netflow_first_switched`** — the time the flow actually started, from the per-record first-switched field when the exporter provides it. |
| 115 | |
| 116 | The Network Flows view still uses journal entry time, which is the time the Netdata Agent received the datagram, for query windows and tier selection. `timestamp_source` controls the stored source timestamp metadata; it does not make the dashboard time picker query by exporter timestamps. |
| 117 | |
| 118 | ## `journal` |
| 119 | |
| 120 | This is the section most operators tune. It controls where flow data lives, how much of it lives, and how the query engine guardrails its work. |
| 121 | |
| 122 | ```yaml |
| 123 | journal: |
| 124 | journal_dir: flows |
| 125 | query_max_groups: 50000 |
| 126 | tiers: |
| 127 | raw: { size_of_journal_files: 50GB, duration_of_journal_files: 24h } |
| 128 | minute_1: { size_of_journal_files: 5GB, duration_of_journal_files: 14d } |
| 129 | minute_5: { size_of_journal_files: 5GB, duration_of_journal_files: 30d } |
| 130 | hour_1: { size_of_journal_files: 5GB, duration_of_journal_files: 365d } |
| 131 | ``` |
| 132 | |
| 133 | ### Journal directory |
| 134 | |
| 135 | | Key | Default | Notes | |
| 136 | |---|---|---| |
| 137 | | `journal_dir` | `flows` | Relative paths resolve under `NETDATA_CACHE_DIR` (typically `/var/cache/netdata/flows`). Absolute paths are used as-is. | |
| 138 | |
| 139 | ### Per-tier retention |
| 140 | |
| 141 | Each tier has its own size and duration budget, configured under `tiers:` only — YAML has no global retention knobs. Raw and rollup tiers have very different storage and access patterns; they should be sized independently. |
| 142 | |
| 143 | ```yaml |
| 144 | tiers: |
| 145 | raw: |
| 146 | size_of_journal_files: 50GB |
| 147 | duration_of_journal_files: 24h |
| 148 | minute_1: |
| 149 | size_of_journal_files: 5GB |
| 150 | duration_of_journal_files: 14d |
| 151 | minute_5: |
| 152 | size_of_journal_files: 5GB |
| 153 | duration_of_journal_files: 30d |
| 154 | hour_1: |
| 155 | size_of_journal_files: 5GB |
| 156 | duration_of_journal_files: 365d |
| 157 | ``` |
| 158 | |
| 159 | | YAML name | Aliases | On-disk directory | |
| 160 | |---|---|---| |
| 161 | | `raw` | — | `flows/raw/` | |
| 162 | | `minute_1` | `1m`, `minute-1`, `minute1` | `flows/1m/` | |
| 163 | | `minute_5` | `5m`, `minute-5`, `minute5` | `flows/5m/` | |
| 164 | | `hour_1` | `1h`, `hour-1`, `hour1` | `flows/1h/` | |
| 165 | |
| 166 | The on-disk directory names are short (`1m`, `5m`, `1h`); the YAML keys are explicit (`minute_1`, `minute_5`, `hour_1`). Mind the difference if you go look at the disk. |
| 167 | |
| 168 | Per-tier values: |
| 169 | |
| 170 | | Key | Default per tier | Notes | |
| 171 | |---|---|---| |
| 172 | | `size_of_journal_files` | `10GB` | Disk budget for this tier. Minimum `100MB`. Set to `null` to disable size-based retention on this tier. | |
| 173 | | `duration_of_journal_files` | `7d` | Time budget for this tier. Set to `null` to disable time-based retention on this tier. | |
| 174 | |
| 175 | Either limit triggers rotation. The tier expires whichever is hit first. At least one of the two must be set per tier (validation enforces this). |
| 176 | |
| 177 | If you omit a tier entry entirely, that tier uses the built-in defaults (`10GB` / `7d`). If you provide a tier entry but omit one of the two knobs, the omitted knob falls back to its built-in default. Setting either to `null` explicitly disables that limit on that tier. |
| 178 | |
| 179 | Standalone CLI runs still accept the legacy uniform retention flags: |
| 180 | `--netflow-retention-size-of-journal-files` and |
| 181 | `--netflow-retention-duration-of-journal-files`. They apply the same value to |
| 182 | all tiers and exist only for standalone/CLI compatibility; production |
| 183 | configuration should use the per-tier YAML shape above. |
| 184 | |
| 185 | The example block at the top of this section is a typical production profile: 24 hours of raw, 2 weeks at 1-minute, 30 days at 5-minute, 1 year at 1-hour. Detailed forensics for the last day; long-term trends for the year. |
| 186 | |
| 187 | ### Rotation |
| 188 | |
| 189 | Each tier rotates files at `size_of_journal_files / 20`, clamped between 5 MB and 200 MB. Time-based rotation is fixed at one hour per file. You don't configure these directly. If `size_of_journal_files` is set to `null` on a tier (size-based retention disabled), the rotation size falls back to 100 MB so files still rotate cleanly. |
| 190 | |
| 191 | ### Query guardrails |
| 192 | |
| 193 | | Key | Default | What it limits | |
| 194 | |---|---|---| |
| 195 | | `query_max_groups` (alias: `query-max-groups`) | `50000` | Maximum number of distinct group keys a single aggregation query can build. When exceeded, additional groups are folded into a synthetic `__overflow__` bucket and the response carries a warning. Protects the query worker from memory blow-up on accidentally wide group-by combinations. | |
| 196 | |
| 197 | The tier the planner uses for a given query is decided automatically from the time window and the query view (Sankey / time-series / map / etc.) — the planner aligns to the coarser tier when the window allows, and falls back to a finer tier for the unaligned head/tail. There are no separate "max window per tier" knobs. |
| 198 | |
| 199 | ## `enrichment` |
| 200 | |
| 201 | Enrichment is a large topic and lives in dedicated pages. The top-level enable/disable knobs: |
| 202 | |
| 203 | ```yaml |
| 204 | enrichment: |
| 205 | # default_sampling_rate: 1024 # single rate, or per-prefix map |
| 206 | # default_sampling_rate: { 10.1.0.0/16: 1024 } # per-prefix form is also valid |
| 207 | # override_sampling_rate: { 10.1.0.0/16: 1024 } # per-prefix override map |
| 208 | default_sampling_rate: ~ |
| 209 | override_sampling_rate: ~ |
| 210 | metadata_static: { exporters: {} } |
| 211 | geoip: { asn_database: [], geo_database: [], optional: false } |
| 212 | networks: {} |
| 213 | network_sources: {} |
| 214 | exporter_classifiers: [] |
| 215 | interface_classifiers: [] |
| 216 | classifier_cache_duration: 5m |
| 217 | asn_providers: [flow, routing, geoip] |
| 218 | net_providers: [flow, routing] |
| 219 | routing_static: { prefixes: {} } |
| 220 | routing_dynamic: |
| 221 | bmp: { enabled: false } |
| 222 | bioris: { enabled: false } |
| 223 | ``` |
| 224 | |
| 225 | A note on `default_sampling_rate` vs. `override_sampling_rate`: both keys accept either a single integer (applied to every record from every exporter) or a per-prefix map. The intended split is "default" for `rate=1` records that lack a sampling rate, and "override" for replacing a known-wrong rate; the schema does not enforce that intent — either knob can take either form. |
| 226 | |
| 227 | `enrichment.geoip.optional` (default `false`) decides what happens when an MMDB file declared in `asn_database` / `geo_database` is missing at startup: `false` aborts the plugin, `true` logs a warning and continues without that database. |
| 228 | |
| 229 | For the cross-cutting picture — order of evaluation, the `asn_providers` and `net_providers` chains, the MMDB shared mechanism, the static-vs-dynamic composition rules — see the [Enrichment](/docs/network-flows/enrichment.md) page. Per-method configuration details (URLs, refresh cadence, license, vendor commands) live on the integration cards under flows.enrichment-methods: |
| 230 | |
| 231 | - IP intelligence (MMDB): [DB-IP](/src/crates/netflow-plugin/integrations/db-ip_ip_intelligence.md), [MaxMind GeoIP / GeoLite2](/src/crates/netflow-plugin/integrations/maxmind_geoip_-_geolite2.md), [IPtoASN](/src/crates/netflow-plugin/integrations/iptoasn.md), [Custom MMDB](/src/crates/netflow-plugin/integrations/custom_mmdb_database.md). |
| 232 | - BGP routing: [BMP](/src/crates/netflow-plugin/integrations/bmp_bgp_monitoring_protocol.md), [bio-rd RIS](/src/crates/netflow-plugin/integrations/bio-rd_-_ripe_ris.md). |
| 233 | - Network sources: [AWS IP Ranges](/src/crates/netflow-plugin/integrations/aws_ip_ranges.md), [Azure IP Ranges](/src/crates/netflow-plugin/integrations/azure_ip_ranges.md), [GCP IP Ranges](/src/crates/netflow-plugin/integrations/gcp_ip_ranges.md), [NetBox](/src/crates/netflow-plugin/integrations/netbox.md), [Generic JSON-over-HTTP IPAM](/src/crates/netflow-plugin/integrations/generic_json-over-http_ipam.md). |
| 234 | - YAML-defined: [Static Metadata](/src/crates/netflow-plugin/integrations/static_metadata.md), [Classifiers](/src/crates/netflow-plugin/integrations/classifiers.md), [Decapsulation](/src/crates/netflow-plugin/integrations/decapsulation.md). |
| 235 | - Operational: [Enrichment Intel Downloader](/docs/network-flows/intel-downloader.md) — the bundled refresh tool for MMDB providers. |
| 236 | |
| 237 | The enrichment section has no CLI flag — it is YAML-only. |
| 238 | |
| 239 | ## Common edits |
| 240 | |
| 241 | ### Listen on a different port |
| 242 | |
| 243 | ```yaml |
| 244 | listener: |
| 245 | listen: "0.0.0.0:9995" |
| 246 | ``` |
| 247 | |
| 248 | ### Bind to a specific address |
| 249 | |
| 250 | ```yaml |
| 251 | listener: |
| 252 | listen: "10.0.0.10:2055" |
| 253 | ``` |
| 254 | |
| 255 | ### Disable a protocol you don't use |
| 256 | |
| 257 | ```yaml |
| 258 | protocols: |
| 259 | v5: false |
| 260 | ``` |
| 261 | |
| 262 | ### Move the journal directory |
| 263 | |
| 264 | ```yaml |
| 265 | journal: |
| 266 | journal_dir: /var/lib/netflow |
| 267 | ``` |
| 268 | |
| 269 | Absolute paths are used as-is. Relative paths resolve under `NETDATA_CACHE_DIR`. |
| 270 | |
| 271 | ### Strip VXLAN tunnel headers |
| 272 | |
| 273 | ```yaml |
| 274 | protocols: |
| 275 | decapsulation_mode: vxlan |
| 276 | ``` |
| 277 | |
| 278 | The plugin reads the inner 5-tuple from `dataLinkFrameSection` records (IPFIX IE 315) when the exporter ships them. |
| 279 | |
| 280 | ### Production retention profile |
| 281 | |
| 282 | ```yaml |
| 283 | journal: |
| 284 | tiers: |
| 285 | raw: |
| 286 | size_of_journal_files: 200GB |
| 287 | duration_of_journal_files: 24h |
| 288 | minute_1: |
| 289 | size_of_journal_files: 20GB |
| 290 | duration_of_journal_files: 14d |
| 291 | minute_5: |
| 292 | size_of_journal_files: 20GB |
| 293 | duration_of_journal_files: 30d |
| 294 | hour_1: |
| 295 | size_of_journal_files: 20GB |
| 296 | duration_of_journal_files: 365d |
| 297 | ``` |
| 298 | |
| 299 | The built-in defaults (10GB / 7d on every tier) are intended for first validation and small deployments. Most production deployments should size retention from observed flow rate. This profile gives you 24 hours of full-detail forensics, 14 days of 1-minute trends, 30 days of 5-minute snapshots, and a year of hourly aggregates. Storage required scales with your flow rate — see [Sizing and Capacity Planning](/docs/network-flows/sizing-capacity.md). |
| 300 | |
| 301 | ## Things that go wrong |
| 302 | |
| 303 | - **The plugin doesn't start.** Check `journalctl --namespace netdata --since "5 minutes ago" | grep netflow`. The most common cause is a typo in a YAML key (strict mode rejects unknowns). |
| 304 | - **Edits don't take effect.** Restart Netdata. There is no DynCfg integration for the plugin's configuration. |
| 305 | - **CLI flags I added don't do anything.** When running under Netdata, only the YAML is read. |
| 306 | - **Tiers fill up faster than expected.** Each tier has its own size/duration. Set per-tier values that match how long you actually need each tier. |
| 307 | - **Queries time out at 30 seconds.** Function calls have a hard 30s timeout in the plugin. If your query is too wide, narrow the time range or add filters that let a higher tier serve it. |
| 308 | - **`__overflow__` appears in results.** A group-by exceeded `query_max_groups` (default 50 000). Either narrow the filter, reduce the number of group-by fields, or raise the limit. |
| 309 | |
| 310 | ## What's next |
| 311 | |
| 312 | - [Retention and Querying](/docs/network-flows/retention-querying.md) — How the four tiers work and how the dashboard picks one. |
| 313 | - [Sizing and Capacity Planning](/docs/network-flows/sizing-capacity.md) — How much disk and CPU you need. |
| 314 | - [Validation and Data Quality](/docs/network-flows/validation.md) — How to confirm the data is right. |
| 315 | - [Troubleshooting](/docs/network-flows/troubleshooting.md) — When things break. |