| 1 | <!--startmeta |
| 2 | custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/integrations/db-ip_ip_intelligence.md" |
| 3 | meta_yaml: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/metadata.yaml" |
| 4 | sidebar_label: "DB-IP IP Intelligence" |
| 5 | learn_status: "Published" |
| 6 | learn_rel_path: "Network Flows/Enrichment Methods" |
| 7 | keywords: ['geoip', 'asn', 'dbip', 'db-ip', 'mmdb', 'ip intelligence', 'flow enrichment'] |
| 8 | message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE FLOWS' metadata.yaml FILE" |
| 9 | endmeta--> |
| 10 | |
| 11 | <!-- markdownlint-disable-file --> |
| 12 | |
| 13 | # DB-IP IP Intelligence |
| 14 | |
| 15 | |
| 16 | <img src="https://netdata.cloud/img/network-wired.svg" width="150"/> |
| 17 | |
| 18 | |
| 19 | Plugin: netflow-plugin |
| 20 | Module: dbip |
| 21 | |
| 22 | <img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> |
| 23 | |
| 24 | ## Overview |
| 25 | |
| 26 | Enrich network flows with ASN and geographic context from DB-IP Lite MMDB |
| 27 | databases. DB-IP IP Intelligence is the default IP intelligence source for the |
| 28 | Netdata netflow plugin: when neither `enrichment.geoip.asn_database` nor |
| 29 | `enrichment.geoip.geo_database` is configured, the plugin auto-detects the |
| 30 | DB-IP-built MMDB files at startup and uses them. Two MMDB files are involved: |
| 31 | an ASN database (`topology-ip-asn.mmdb`) and a geographic database |
| 32 | (`topology-ip-geo.mmdb`). Native packages (DEB, RPM) ship a stock copy under |
| 33 | `/usr/share/netdata/topology-ip-intel/`; the bundled refresh tool fetches updates |
| 34 | into `/var/cache/netdata/topology-ip-intel/`. |
| 35 | |
| 36 | DB-IP publishes the Lite databases monthly under a Creative Commons Attribution 4.0 |
| 37 | International licence (https://db-ip.com/db/lite.php). The MMDB format is the MaxMind |
| 38 | binary database; lookups are in-process with no per-flow network call. |
| 39 | |
| 40 | Populates these flow record fields when a lookup succeeds: |
| 41 | |
| 42 | - From the **geo MMDB**: `SRC_COUNTRY`, `DST_COUNTRY`, `SRC_GEO_STATE`, `DST_GEO_STATE`, |
| 43 | `SRC_GEO_CITY`, `DST_GEO_CITY`, `SRC_GEO_LATITUDE`, `DST_GEO_LATITUDE`, |
| 44 | `SRC_GEO_LONGITUDE`, `DST_GEO_LONGITUDE`. Country and state survive into all four |
| 45 | journal tiers; city, latitude, and longitude are kept only in the raw tier so the |
| 46 | rollups stay cardinality-bounded. |
| 47 | - From the **ASN MMDB**: `SRC_AS_NAME`, `DST_AS_NAME`. The DB-IP-built ASN database also |
| 48 | tags address ranges as private/reserved (RFC 1918, link-local, RFC 6598, etc.); when |
| 49 | the resolved AS number is `0` and that flag is set, the plugin renders the AS name as |
| 50 | `AS0 Private IP Address Space` instead of `AS0 Unknown ASN`. The AS *number* itself |
| 51 | comes from the wider asn_providers chain, not directly from the MMDB. |
| 52 | |
| 53 | For the cross-provider mechanics (lookup pass ordering, the 30-second hot-reload poll, |
| 54 | the auto-detect path order, multi-database composition, IPv4/IPv6 handling, the failure |
| 55 | modes shared by every MMDB provider), see |
| 56 | [IP Intelligence](https://learn.netdata.cloud/docs/network-flows/enrichment). |
| 57 | |
| 58 | |
| 59 | Files are read on plugin start and reloaded automatically when their mtime or size |
| 60 | changes. Auto-detection scans `${NETDATA_CACHE_DIR}/topology-ip-intel/` first |
| 61 | (typically `/var/cache/netdata/topology-ip-intel/`), falling back to the stock copy |
| 62 | under `${NETDATA_STOCK_DATA_DIR}/topology-ip-intel/` (typically |
| 63 | `/usr/share/netdata/topology-ip-intel/`). When databases are auto-detected they are |
| 64 | treated as optional, so a missing or unreadable file does not abort plugin startup. |
| 65 | |
| 66 | |
| 67 | This integration is only supported on the following platforms: |
| 68 | |
| 69 | - Linux |
| 70 | |
| 71 | This integration runs as a single instance per Netdata Agent. |
| 72 | |
| 73 | |
| 74 | ### Default Behavior |
| 75 | |
| 76 | #### Auto-Detection |
| 77 | |
| 78 | Native packages ship the stock DB-IP MMDB files; the plugin auto-detects them at startup. No configuration required for the default install. |
| 79 | |
| 80 | #### Limits |
| 81 | |
| 82 | Lookup coverage and freshness depend on the DB-IP Lite files installed on disk. Native packages provide a stock copy; schedule the downloader if you need monthly refreshes. |
| 83 | |
| 84 | #### Performance Impact |
| 85 | |
| 86 | Lookups are local MMDB reads with no per-flow network call. Memory use is mostly the mapped database files and the kernel page cache needed to keep active pages hot. |
| 87 | |
| 88 | ## Setup |
| 89 | |
| 90 | |
| 91 | ### Prerequisites |
| 92 | |
| 93 | #### DB-IP MMDB files |
| 94 | |
| 95 | Native DEB / RPM packages ship the stock DB-IP MMDB files under |
| 96 | `/usr/share/netdata/topology-ip-intel/`. For source builds, or to get a fresher copy |
| 97 | than the one bundled with the package, run `topology-ip-intel-downloader` once when |
| 98 | the binary is available (packaged 32-bit installs do not include it) to populate |
| 99 | `/var/cache/netdata/topology-ip-intel/`: |
| 100 | |
| 101 | ```bash |
| 102 | sudo /usr/sbin/topology-ip-intel-downloader |
| 103 | ``` |
| 104 | |
| 105 | See the [Enrichment Intel Downloader](https://learn.netdata.cloud/docs/network-flows/enrichment-intel-downloader) |
| 106 | page for downloader options and how to schedule periodic refreshes. DB-IP Lite |
| 107 | data is published monthly, so a monthly cron of the downloader is the right |
| 108 | cadence -- more frequent runs will not produce newer data. |
| 109 | |
| 110 | |
| 111 | #### Licence acknowledgement |
| 112 | |
| 113 | DB-IP Lite databases are distributed under the Creative Commons Attribution 4.0 |
| 114 | International licence (https://db-ip.com/db/lite.php). Attribution is required |
| 115 | when redistributing the data or derivative dashboards. |
| 116 | |
| 117 | |
| 118 | |
| 119 | ### Configuration |
| 120 | |
| 121 | #### Options |
| 122 | |
| 123 | Configure DB-IP under `enrichment.geoip` in `netflow.yaml`. Empty `asn_database` |
| 124 | and `geo_database` enable auto-detection. |
| 125 | |
| 126 | |
| 127 | <details open><summary>Config options</summary> |
| 128 | |
| 129 | |
| 130 | |
| 131 | | Option | Description | Default | Required | |
| 132 | |:-----|:------------|:--------|:---------:| |
| 133 | | enrichment.geoip.asn_database | List of MMDB paths providing AS data. Empty = auto-detect under cache/stock dirs. | [] (auto-detect) | no | |
| 134 | | enrichment.geoip.geo_database | List of MMDB paths providing geo data. Empty = auto-detect. | [] (auto-detect) | no | |
| 135 | | enrichment.geoip.optional | When true, missing or unreadable MMDBs are warnings, not fatal. Auto-detected files default to optional. | false (true when auto-detected) | no | |
| 136 | |
| 137 | |
| 138 | </details> |
| 139 | |
| 140 | |
| 141 | |
| 142 | #### via File |
| 143 | |
| 144 | The configuration file name for this integration is `netflow.yaml`. |
| 145 | |
| 146 | |
| 147 | You can edit the configuration file using the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#edit-configuration-files) script from the |
| 148 | Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#locate-your-config-directory). |
| 149 | |
| 150 | ```bash |
| 151 | cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata |
| 152 | sudo ./edit-config netflow.yaml |
| 153 | ``` |
| 154 | |
| 155 | ##### Examples |
| 156 | |
| 157 | ###### Default (auto-detect stock files) |
| 158 | |
| 159 | Native package install. No explicit configuration; the plugin finds the stock or cache copy automatically. |
| 160 | |
| 161 | ```yaml |
| 162 | enrichment: |
| 163 | geoip: |
| 164 | asn_database: [] |
| 165 | geo_database: [] |
| 166 | optional: true |
| 167 | |
| 168 | ``` |
| 169 | ###### Explicit DB-IP paths |
| 170 | |
| 171 | Override auto-detection by pointing to specific DB-IP MMDBs (for example, after running the downloader to a non-standard location). |
| 172 | |
| 173 | <details open><summary>Config</summary> |
| 174 | |
| 175 | ```yaml |
| 176 | enrichment: |
| 177 | geoip: |
| 178 | asn_database: |
| 179 | - /var/cache/netdata/topology-ip-intel/topology-ip-asn.mmdb |
| 180 | geo_database: |
| 181 | - /var/cache/netdata/topology-ip-intel/topology-ip-geo.mmdb |
| 182 | optional: false |
| 183 | |
| 184 | ``` |
| 185 | </details> |
| 186 | |
| 187 | |
| 188 | |
| 189 | ### Private IPs have empty GeoIP fields |
| 190 | |
| 191 | GeoIP databases normally have no country, city, or coordinate entry for RFC 1918 / |
| 192 | private space. The DB-IP-built ASN database tags private ranges so `*_AS_NAME` |
| 193 | renders as `AS0 Private IP Address Space`, while geographic fields stay empty and |
| 194 | private addresses do not appear on maps. Declare your internal CIDRs under |
| 195 | `enrichment.networks` when you want internal labels -- see |
| 196 | [Static metadata](https://learn.netdata.cloud/docs/network-flows/enrichment). |
| 197 | |
| 198 | |
| 199 | ### Stale databases |
| 200 | |
| 201 | The plugin does not alert on staleness. Check file mtime: |
| 202 | `ls -la /var/cache/netdata/topology-ip-intel/`. DB-IP Lite is published monthly, |
| 203 | so a monthly cron of `/usr/sbin/topology-ip-intel-downloader` keeps you on the |
| 204 | upstream cadence when the downloader is installed; running it more often will not |
| 205 | produce fresher data. |
| 206 | |
| 207 | |
| 208 | ### Map renders empty over a long time window |
| 209 | |
| 210 | `SRC_GEO_CITY`, `DST_GEO_CITY`, `SRC_GEO_LATITUDE`, `DST_GEO_LATITUDE`, |
| 211 | `SRC_GEO_LONGITUDE`, and `DST_GEO_LONGITUDE` are stored only in the raw journal |
| 212 | tier; the 1-minute, 5-minute, and 1-hour rollups drop them to keep cardinality |
| 213 | bounded. A query that auto-falls back to a rollup tier therefore renders an empty |
| 214 | city map. Narrow the time range so the query fits the raw tier, or use the |
| 215 | country / state map (those survive into rollups). |
| 216 | |
| 217 | |
| 218 |