| 1 | <!--startmeta |
| 2 | custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/integrations/classifiers.md" |
| 3 | meta_yaml: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/metadata.yaml" |
| 4 | sidebar_label: "Classifiers" |
| 5 | learn_status: "Published" |
| 6 | learn_rel_path: "Network Flows/Enrichment Methods" |
| 7 | keywords: ['classifiers', 'rules', 'akvorado', 'expression', 'exporter', 'interface', 'boundary', 'connectivity', 'provider', 'tagging'] |
| 8 | message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE FLOWS' metadata.yaml FILE" |
| 9 | endmeta--> |
| 10 | |
| 11 | <!-- markdownlint-disable-file --> |
| 12 | |
| 13 | # Classifiers |
| 14 | |
| 15 | |
| 16 | <img src="https://netdata.cloud/img/network-wired.svg" width="150"/> |
| 17 | |
| 18 | |
| 19 | Plugin: netflow-plugin |
| 20 | Module: classifiers |
| 21 | |
| 22 | <img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> |
| 23 | |
| 24 | ## Overview |
| 25 | |
| 26 | Annotate network flows with exporter and interface labels derived from reusable |
| 27 | classification rules. Where [static metadata](https://learn.netdata.cloud/docs/network-flows/enrichment) |
| 28 | forces you to enumerate every exporter and every ifIndex by hand, classifiers |
| 29 | let you express the network design once -- "anything matching `^edge-` is the |
| 30 | edge tier", "any interface with `BACKBONE-LUMEN` in its description is on |
| 31 | Lumen", "any interface at 100Gbps is a core uplink" -- and apply that labelling |
| 32 | across the whole flow stream. |
| 33 | |
| 34 | The plugin ships two rule lists, evaluated in YAML order: |
| 35 | |
| 36 | - `enrichment.exporter_classifiers` -- runs once per exporter (cached). Sees the |
| 37 | exporter's IP and friendly name, and any classification slots already filled |
| 38 | by static metadata or by earlier rules. Can set |
| 39 | `EXPORTER_GROUP / ROLE / SITE / REGION / TENANT`. |
| 40 | - `enrichment.interface_classifiers` -- runs once per `(exporter, interface)` |
| 41 | pair, applied **twice per flow record** (once for the input interface, once |
| 42 | for the output). Sees |
| 43 | everything an exporter rule sees plus `Interface.Index / Name / Description / |
| 44 | Speed / VLAN`. Can set `IN_IF_PROVIDER / OUT_IF_PROVIDER`, |
| 45 | `IN_IF_CONNECTIVITY / OUT_IF_CONNECTIVITY`, `IN_IF_BOUNDARY / OUT_IF_BOUNDARY` |
| 46 | (1=external, 2=internal), and override `IN_IF_NAME / DESCRIPTION` / |
| 47 | `OUT_IF_NAME / DESCRIPTION`. |
| 48 | |
| 49 | The expression language is **Akvorado-compatible for the documented operators |
| 50 | and actions**. It implements a subset of Akvorado's `expr-lang`-derived grammar. Akvorado rules |
| 51 | using only equality, comparison, `in`, `contains`, `startsWith`, `endsWith`, |
| 52 | `matches`, `&&`, `||`, `!`, parentheses, and the documented `Classify*` / |
| 53 | `Reject` / `Format` actions will work; arithmetic, ternaries, lambdas, and |
| 54 | arbitrary `expr-lang` features are not supported. |
| 55 | |
| 56 | Output values written by `Classify*` actions are **lowercased and stripped to |
| 57 | ASCII alphanumerics + `.` + `+` + `-`** before they reach the flow record. So |
| 58 | `ClassifyRegion("EU West")` becomes `euwest`. Use `SetName` / `SetDescription` |
| 59 | when you want to preserve case and whitespace -- those write directly without |
| 60 | normalisation. |
| 61 | |
| 62 | For the cross-cutting Enrichment concept (where classifiers sit in the merge |
| 63 | order vs static metadata, GeoIP, IPAM, BGP routing), see |
| 64 | [Enrichment](https://learn.netdata.cloud/docs/network-flows/enrichment). |
| 65 | |
| 66 | |
| 67 | Each rule is a single boolean expression; an action with no condition (e.g. |
| 68 | `Classify("edge")` at top level) is treated as always-true and always fires. |
| 69 | Rules are AND/OR-composed, so the typical shape is `condition && Classify*(...)`. |
| 70 | The plugin evaluates the list top to bottom, **first-write-wins per slot**: |
| 71 | once `EXPORTER_GROUP` is set, no later rule can change it. Order rules from |
| 72 | most-specific to least-specific. |
| 73 | |
| 74 | Two short-circuit rules end the loop early. For exporter rules, the loop stops |
| 75 | when `group + role + site + region + tenant` are all non-empty. For |
| 76 | interface rules, the loop stops when `connectivity + provider + boundary` are |
| 77 | all set. `SetName` / `SetDescription` / |
| 78 | `Reject` do not contribute to short-circuit. |
| 79 | |
| 80 | A rule that throws at runtime (e.g. comparing a string with `>`) breaks out of |
| 81 | the loop for that record and keeps whatever was set so far. Use `matches`, `startsWith`, or `contains` |
| 82 | on string fields instead of `>` / `<` to avoid this. |
| 83 | |
| 84 | **Akvorado parity**: if `metadata_static` already filled any classification |
| 85 | slot for the target, the matching classifier list does **not run** for that |
| 86 | target -- operator-provided classification has priority and the rules cannot |
| 87 | override it. Don't try to mix static and rule-based labelling on the same |
| 88 | exporter or interface; pick one tool per target. |
| 89 | |
| 90 | Results are cached. The exporter cache keys on `ExporterInfo (ip + name)`. The |
| 91 | interface cache keys on `(exporter, exporter_classification, interface)` -- so |
| 92 | when the exporter's classification changes (for example after you push new |
| 93 | static metadata and restart) the interface caches naturally invalidate. The |
| 94 | cache TTL is `enrichment.classifier_cache_duration` (default 5 minutes). It is |
| 95 | a last-access TTL so entries live as long as they're queried. |
| 96 | |
| 97 | |
| 98 | This integration is only supported on the following platforms: |
| 99 | |
| 100 | - Linux |
| 101 | |
| 102 | This integration runs as a single instance per Netdata Agent. |
| 103 | |
| 104 | |
| 105 | ### Default Behavior |
| 106 | |
| 107 | #### Auto-Detection |
| 108 | |
| 109 | Disabled by default. Both rule lists are empty; populate `enrichment.exporter_classifiers` and / or `enrichment.interface_classifiers` to enable. |
| 110 | |
| 111 | #### Limits |
| 112 | |
| 113 | Resource use scales with rule count and the number of distinct exporters and interfaces. The classifier cache limits repeat evaluation for stable exporter/interface inventories. |
| 114 | |
| 115 | #### Performance Impact |
| 116 | |
| 117 | Rules run at decode time, in the flow-pipeline hot path, so cost matters. |
| 118 | The cache absorbs nearly all of it: per (exporter, interface) the rule list |
| 119 | evaluates only on cache miss. Tune |
| 120 | `enrichment.classifier_cache_duration` upwards (15-60 minutes) for very |
| 121 | high-cardinality exporter / interface pools where the default 5 minutes |
| 122 | still yields visible misses; tune downwards (30-60 seconds) when iterating |
| 123 | on rule changes during a config session. |
| 124 | |
| 125 | |
| 126 | ## Setup |
| 127 | |
| 128 | |
| 129 | ### Prerequisites |
| 130 | |
| 131 | #### Know what to classify |
| 132 | |
| 133 | Classifiers shine when there is a pattern to match -- exporter naming |
| 134 | conventions (`edge-...`, `core-...`), management-IP subnets per site, |
| 135 | SNMP interface descriptions that follow a template (`BACKBONE-<carrier>`, |
| 136 | `TRANSIT-...`, `IX-...`), or 100Gbps-equals-core conventions. If your |
| 137 | fleet has no such pattern, [static metadata](https://learn.netdata.cloud/docs/network-flows/enrichment) |
| 138 | is the better fit -- it lets you list each exporter and ifIndex by hand. |
| 139 | |
| 140 | |
| 141 | #### Configure interface metadata first if you want interface rules |
| 142 | |
| 143 | The plugin does **not** poll SNMP itself, so `Interface.Name`, |
| 144 | `Interface.Description`, and `Interface.Speed` are populated only from |
| 145 | `enrichment.metadata_static` (the static-metadata integration card). If |
| 146 | you have not configured `interfaces:` under `metadata_static.exporters`, |
| 147 | those identifiers will be empty strings / zero, and any rule that |
| 148 | matches against them will never fire. `Interface.Index` and |
| 149 | `Interface.VLAN` come from the flow record itself and are always available. |
| 150 | |
| 151 | |
| 152 | |
| 153 | ### Configuration |
| 154 | |
| 155 | #### Options |
| 156 | |
| 157 | Both lists live under `enrichment:`. Each entry is a free-form string |
| 158 | containing a single rule expression. The cache TTL is one global setting. |
| 159 | |
| 160 | |
| 161 | <details open><summary>Config options</summary> |
| 162 | |
| 163 | |
| 164 | |
| 165 | | Option | Description | Default | Required | |
| 166 | |:-----|:------------|:--------|:---------:| |
| 167 | | enrichment.exporter_classifiers | Ordered list of rules applied per exporter. Each rule is a string expression. Available identifiers: `Exporter.IP`, `Exporter.Name`, `CurrentClassification.Group / .Role / .Site / .Region / .Tenant`. Available actions: `Classify` / `ClassifyGroup`, `ClassifyRole`, `ClassifySite`, `ClassifyRegion`, `ClassifyTenant`, plus the `*Regex(input, pattern, template)` variants of each, plus `Reject()`. Interface-only actions (`ClassifyProvider`, `ClassifyConnectivity`, `ClassifyExternal` / `ClassifyInternal`, `SetName`, `SetDescription`) fail at runtime if used here. | [] | no | |
| 168 | | enrichment.interface_classifiers | Ordered list of rules applied per `(exporter, interface)` pair. Sees everything an exporter rule sees, plus `Interface.Index`, `Interface.Name`, `Interface.Description`, `Interface.Speed` (bits per second), `Interface.VLAN`, and the per-interface `CurrentClassification.Connectivity / .Provider / .Boundary / .Name / .Description`. Available actions: `ClassifyProvider`, `ClassifyConnectivity`, `ClassifyExternal()`, `ClassifyInternal()`, `SetName`, `SetDescription`, `Reject()`, plus the `*Regex` variants of provider / connectivity. Exporter-only `Classify*` actions fail at runtime if used here. | [] | no | |
| 169 | | enrichment.classifier_cache_duration | Last-access TTL for both classifier caches (exporter and interface). Values below 1 second are rejected. The cache prunes opportunistically -- entries idle longer than the TTL are dropped on the next prune pass, capped at one prune every TTL or 30 seconds, whichever is smaller. Restart the plugin to clear caches outright when you change rules. | 5m | no | |
| 170 | |
| 171 | |
| 172 | </details> |
| 173 | |
| 174 | |
| 175 | |
| 176 | #### via File |
| 177 | |
| 178 | The configuration file name for this integration is `netflow.yaml`. |
| 179 | |
| 180 | |
| 181 | You can edit the configuration file using the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#edit-configuration-files) script from the |
| 182 | Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#locate-your-config-directory). |
| 183 | |
| 184 | ```bash |
| 185 | cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata |
| 186 | sudo ./edit-config netflow.yaml |
| 187 | ``` |
| 188 | |
| 189 | ##### Examples |
| 190 | |
| 191 | ###### Exporter classification by name pattern |
| 192 | |
| 193 | Tag exporters by the prefix of their friendly name -- the simplest and |
| 194 | most common pattern. Falls back to a regex capture for the region code |
| 195 | when the name encodes one. The final `Reject()` rule drops a test |
| 196 | exporter from collection entirely. |
| 197 | |
| 198 | |
| 199 | ```yaml |
| 200 | enrichment: |
| 201 | exporter_classifiers: |
| 202 | # Group by name prefix. |
| 203 | - 'Exporter.Name startsWith "edge-" && Classify("edge")' |
| 204 | - 'Exporter.Name startsWith "core-" && Classify("core")' |
| 205 | - 'Exporter.Name startsWith "agg-" && Classify("aggregation")' |
| 206 | |
| 207 | # Site by management-IP subnet. |
| 208 | - 'Exporter.IP startsWith "10.1." && ClassifySite("ny-dc1")' |
| 209 | - 'Exporter.IP startsWith "10.2." && ClassifySite("par-dc1")' |
| 210 | |
| 211 | # Region from a name suffix like "edge-fra-01" -> "fra". |
| 212 | - 'ClassifyRegionRegex(Exporter.Name, "-([a-z]{3})-[0-9]+$", "$1")' |
| 213 | |
| 214 | # Drop a lab exporter entirely. |
| 215 | - 'Exporter.IP startsWith "192.0.2." && Reject()' |
| 216 | |
| 217 | ``` |
| 218 | ###### Interface classification from SNMP descriptions |
| 219 | |
| 220 | Encode the boundary, the provider, and the connectivity tier from the |
| 221 | interface description that your network team already maintains. The |
| 222 | `(?i)` regex flag is the Rust regex inline-case-insensitive prefix. |
| 223 | |
| 224 | |
| 225 | <details open><summary>Config</summary> |
| 226 | |
| 227 | ```yaml |
| 228 | enrichment: |
| 229 | interface_classifiers: |
| 230 | # Provider tag from a description prefix. |
| 231 | - 'Interface.Description startsWith "BACKBONE-LUMEN" && ClassifyProvider("Lumen")' |
| 232 | - 'Interface.Description startsWith "BACKBONE-COGENT" && ClassifyProvider("Cogent")' |
| 233 | - 'Interface.Description startsWith "BACKBONE-NTT" && ClassifyProvider("NTT")' |
| 234 | |
| 235 | # Transit links: external boundary + connectivity tag. |
| 236 | - 'Interface.Description contains "TRANSIT" && ClassifyConnectivity("transit") && ClassifyExternal()' |
| 237 | |
| 238 | # Peering and IX -- case-insensitive regex. |
| 239 | - 'Interface.Description matches "(?i)^(IX|peering)-.*" && ClassifyConnectivity("peering") && ClassifyExternal()' |
| 240 | |
| 241 | # Internal customer-facing access ports. |
| 242 | - 'Interface.Description startsWith "CUSTOMER-" && ClassifyConnectivity("customer") && ClassifyInternal()' |
| 243 | |
| 244 | ``` |
| 245 | </details> |
| 246 | |
| 247 | ###### Boundary inferred from interface speed |
| 248 | |
| 249 | A pragmatic shorthand when descriptions are unreliable but speed is |
| 250 | consistent. 100Gbps and faster interfaces are core, 10Gbps are |
| 251 | aggregation, 1Gbps and slower are access. `Interface.Speed` is in bits |
| 252 | per second -- numeric comparisons are safe. |
| 253 | |
| 254 | |
| 255 | <details open><summary>Config</summary> |
| 256 | |
| 257 | ```yaml |
| 258 | enrichment: |
| 259 | interface_classifiers: |
| 260 | - 'Interface.Speed >= 100000000000 && ClassifyConnectivity("core")' |
| 261 | - 'Interface.Speed >= 10000000000 && ClassifyConnectivity("aggregation")' |
| 262 | - 'Interface.Speed > 0 && ClassifyConnectivity("access")' |
| 263 | |
| 264 | ``` |
| 265 | </details> |
| 266 | |
| 267 | ###### Combining exporter context with interface rules |
| 268 | |
| 269 | Interface rules see the exporter's already-resolved classification |
| 270 | via `CurrentClassification.*`. Use it to scope interface rules to |
| 271 | specific tiers -- for example: every interface on an edge exporter |
| 272 | without a more-specific match falls back to "external". |
| 273 | |
| 274 | |
| 275 | <details open><summary>Config</summary> |
| 276 | |
| 277 | ```yaml |
| 278 | enrichment: |
| 279 | exporter_classifiers: |
| 280 | - 'Exporter.Name startsWith "edge-" && Classify("edge") && ClassifyRole("border")' |
| 281 | - 'Exporter.Name startsWith "core-" && Classify("core") && ClassifyRole("backbone")' |
| 282 | interface_classifiers: |
| 283 | # Specific provider rules first (most-specific to least-specific). |
| 284 | - 'Interface.Description startsWith "BACKBONE-LUMEN" && ClassifyProvider("Lumen")' |
| 285 | - 'Interface.Description startsWith "BACKBONE-COGENT" && ClassifyProvider("Cogent")' |
| 286 | |
| 287 | # Generic transit rule. |
| 288 | - 'Interface.Description contains "TRANSIT" && ClassifyConnectivity("transit") && ClassifyExternal()' |
| 289 | |
| 290 | # Fallback: any unclassified interface on an edge box is external. |
| 291 | - 'CurrentClassification.Role == "border" && CurrentClassification.Boundary == 0 && ClassifyExternal()' |
| 292 | |
| 293 | ``` |
| 294 | </details> |
| 295 | |
| 296 | ###### Building values with Format and human-readable names |
| 297 | |
| 298 | `Format(pattern, args...)` mimics Go's `fmt.Sprintf` for `%s`, `%v`, |
| 299 | `%d`, `%%`. `Classify*` |
| 300 | normalises (lowercase + strip non-alphanumeric); `SetName` and |
| 301 | `SetDescription` do not, so they preserve the case and spaces of the |
| 302 | computed value. |
| 303 | |
| 304 | |
| 305 | <details open><summary>Config</summary> |
| 306 | |
| 307 | ```yaml |
| 308 | enrichment: |
| 309 | exporter_classifiers: |
| 310 | # Tenant computed from name, normalised on write -> "tenant-edge01". |
| 311 | - 'ClassifyTenant(Format("tenant-%s", Exporter.Name))' |
| 312 | interface_classifiers: |
| 313 | # Human-readable name = "<exporter>:if<index>". Preserved verbatim. |
| 314 | - 'SetName(Format("%s:if%d", Exporter.Name, Interface.Index))' |
| 315 | |
| 316 | ``` |
| 317 | </details> |
| 318 | |
| 319 | ###### Tuning the cache for a large fleet |
| 320 | |
| 321 | The default 5-minute last-access TTL is right for steady-state. Raise |
| 322 | it when the (exporter, interface) population is large enough that |
| 323 | evicted entries are quickly re-queried. Lower it when |
| 324 | actively iterating on rule changes so misses pick up the new rules |
| 325 | quickly. |
| 326 | |
| 327 | |
| 328 | <details open><summary>Config</summary> |
| 329 | |
| 330 | ```yaml |
| 331 | enrichment: |
| 332 | classifier_cache_duration: 30m |
| 333 | exporter_classifiers: |
| 334 | - 'Exporter.Name startsWith "edge-" && Classify("edge")' |
| 335 | interface_classifiers: |
| 336 | - 'Interface.Speed >= 100000000000 && ClassifyConnectivity("core")' |
| 337 | |
| 338 | ``` |
| 339 | </details> |
| 340 | |
| 341 | |
| 342 | |
| 343 | ### Plugin fails to start with a parser error |
| 344 | |
| 345 | A rule failed to parse. The journal log includes the index in the list |
| 346 | and a parser context (`unsupported rule term`, `unsupported value |
| 347 | expression`, `Reject() does not accept arguments`, etc.). Common causes: |
| 348 | missing `&&` between condition and action; an action used in the wrong |
| 349 | list (`ClassifyExternal` in an exporter rule); strings written with |
| 350 | single quotes (only JSON-style double quotes are accepted); regex literals |
| 351 | that fail to compile. |
| 352 | |
| 353 | |
| 354 | ### Classifier rules never run for an exporter or interface |
| 355 | |
| 356 | Likely cause: `metadata_static` already set **any** classification field |
| 357 | on that target. By design, the matching list is suppressed entirely when |
| 358 | the classification is non-empty. Either remove the static-metadata entry for that target, or |
| 359 | keep static-metadata as the sole source for it. |
| 360 | |
| 361 | |
| 362 | ### A value appears differently in the dashboard than in the rule |
| 363 | |
| 364 | `Classify*` actions normalise output to `[a-z0-9.+-]` only -- so |
| 365 | `ClassifyRegion("EU West")` lands as `euwest`, and |
| 366 | `Classify("Edge_Tier_1")` lands as `edgetier1`. Use `SetName` / |
| 367 | `SetDescription` to preserve case and whitespace; those write the value |
| 368 | verbatim. |
| 369 | |
| 370 | |
| 371 | ### First rule always wins, later rules never fire for the same slot |
| 372 | |
| 373 | First-write-wins is by design and per slot. Order your |
| 374 | rules from most-specific to least-specific. If you want a tiered |
| 375 | fallback, use distinct slots (e.g. `Classify` for the broad group and |
| 376 | `ClassifyRole` for the tier within that group). |
| 377 | |
| 378 | |
| 379 | ### A working rule stops matching some time after startup |
| 380 | |
| 381 | Cached results expire after `classifier_cache_duration` (default 5 |
| 382 | minutes, last-access). When you change rules, restart the plugin so the |
| 383 | caches clear immediately -- otherwise stale cached classifications keep |
| 384 | returning until they idle out. |
| 385 | |
| 386 | |
| 387 | ### A rule with `>` or `<` aborts the rule list |
| 388 | |
| 389 | Comparing a string-typed identifier with `>` / `<` / `>=` / `<=` raises |
| 390 | a runtime error, and the loop breaks out for that record. Subsequent rules in |
| 391 | the list are skipped for that record. Use `matches`, `startsWith`, |
| 392 | `endsWith`, `contains`, or `==` / `!=` on string fields. Keep `>` / `<` |
| 393 | for `Interface.Index`, `Interface.Speed`, and `Interface.VLAN` (the |
| 394 | numeric identifiers). |
| 395 | |
| 396 | |
| 397 | ### ClassifyExternal fires only on one side |
| 398 | |
| 399 | Interface classifiers run twice per flow record -- once for the input |
| 400 | interface, once for the output. Both invocations see the same rule list. If your rule conditions on |
| 401 | `Interface.Index == 42` and that ifIndex appears in `IN_IF` of one flow |
| 402 | and `OUT_IF` of another, the rule fires correctly in both places. But |
| 403 | the `IN_IF_BOUNDARY` / `OUT_IF_BOUNDARY` columns are independent -- a |
| 404 | rule firing on the output side of a flow only sets the output side's |
| 405 | boundary, and vice versa. |
| 406 | |
| 407 | |
| 408 | ### Interface fields are empty in the rule even though SNMP is configured |
| 409 | |
| 410 | The plugin does not poll SNMP -- `Interface.Name`, `Description`, and |
| 411 | `Speed` come exclusively from `enrichment.metadata_static.exporters.<ip>.interfaces.<index>`. |
| 412 | If you populate them through an external SNMP discovery and write them |
| 413 | into `metadata_static`, the rules will see them. Otherwise those fields |
| 414 | resolve to empty strings / zero, and any rule that conditions on them |
| 415 | never matches. |
| 416 | |
| 417 | |
| 418 | ### Referencing Interface.* in an exporter rule silently does nothing |
| 419 | |
| 420 | Field resolution does not error when the wrong context is missing -- it |
| 421 | returns the type's zero value. So `Interface.Speed >= 1` written in an `exporter_classifiers` rule |
| 422 | resolves to `0 >= 1` (false) on every call. Use |
| 423 | `interface_classifiers` for any rule that needs an interface field. |
| 424 | |
| 425 | |
| 426 |