| 1 | # Query network-flow Functions via Netdata Cloud |
| 2 | |
| 3 | This guide is part of the [`query-netdata-cloud`](./SKILL.md) skill. |
| 4 | Read the [SKILL.md prerequisites](./SKILL.md#prerequisites) first. |
| 5 | For the generic Function transport, see |
| 6 | [query-functions.md](./query-functions.md). |
| 7 | |
| 8 | Flow Functions return network-flow records (NetFlow / sFlow / |
| 9 | IPFIX) ingested by the agent's flow collector. Their dataset is |
| 10 | table-shaped (one row per flow tuple) AND time-windowed AND |
| 11 | faceted, sitting between table snapshots (`processes`) and log |
| 12 | queries (`systemd-journal`). |
| 13 | |
| 14 | --- |
| 15 | |
| 16 | ## Function names registered today |
| 17 | |
| 18 | Verified live and in source: |
| 19 | |
| 20 | | Function | Source crate | Layer | What it returns | |
| 21 | |---|---|---|---| |
| 22 | | `flows:netflow` | `src/crates/netflow-plugin/` | L3 | Network flow records ingested via NetFlow v5/v9, IPFIX, sFlow | |
| 23 | |
| 24 | The `flows:` prefix is the canonical namespace; only `netflow` is |
| 25 | registered today. The Function name covers all three protocols |
| 26 | (the collector parses NetFlow, IPFIX, and sFlow into a single |
| 27 | record schema). |
| 28 | |
| 29 | --- |
| 30 | |
| 31 | ## Endpoint and request |
| 32 | |
| 33 | Standard Cloud Function-call endpoint: |
| 34 | |
| 35 | `POST /api/v2/nodes/{nodeId}/function?function=flows:netflow` |
| 36 | |
| 37 | ```bash |
| 38 | TOKEN="YOUR_API_TOKEN" |
| 39 | NODE="YOUR_NODE_UUID" |
| 40 | |
| 41 | read -r -d '' PAYLOAD <<'EOF' |
| 42 | { |
| 43 | "mode": "flows", |
| 44 | "view": "table-sankey", |
| 45 | "after": -3600, |
| 46 | "before": 0, |
| 47 | "group_by": ["SRC_AS_NAME", "PROTOCOL", "DST_AS_NAME"], |
| 48 | "sort_by": "bytes", |
| 49 | "top_n": 100 |
| 50 | } |
| 51 | EOF |
| 52 | |
| 53 | curl -sS -X POST \ |
| 54 | -H 'Content-Type: application/json' \ |
| 55 | -H "Authorization: Bearer $TOKEN" \ |
| 56 | "https://app.netdata.cloud/api/v2/nodes/$NODE/function?function=flows:netflow" \ |
| 57 | -d "$PAYLOAD" |
| 58 | ``` |
| 59 | |
| 60 | ### Modes |
| 61 | |
| 62 | The Function has three modes selected by the `mode` body field: |
| 63 | |
| 64 | | Mode | Purpose | |
| 65 | |---|---| |
| 66 | | `flows` (default) | Return flow records / aggregations / charts | |
| 67 | | `autocomplete` | Return values for a single facet field, given a search prefix | |
| 68 | |
| 69 | ### Body parameters |
| 70 | |
| 71 | Verified against `src/crates/netflow-plugin/src/api/flows/handler.rs`: |
| 72 | |
| 73 | | Parameter | Used in mode | Description | |
| 74 | |---|---|---| |
| 75 | | `mode` | both | `flows` or `autocomplete` | |
| 76 | | `view` | flows | One of: `table-sankey`, `timeseries`, `country-map`, `state-map`, `city-map` | |
| 77 | | `after` | flows | Unix seconds, lower bound. Negative = relative seconds from `before` | |
| 78 | | `before` | flows | Unix seconds, upper bound. `0` = now | |
| 79 | | `query` | flows | Free-text filter | |
| 80 | | `selections` | flows | Pre-applied facet filters as `{ "FIELD_NAME": ["val", "val2"] }`. Common fields: `SRC_ADDR`, `DST_ADDR`, `SRC_PORT`, `DST_PORT`, `PROTOCOL`, `SRC_AS_NAME`, `DST_AS_NAME`, `SRC_COUNTRY`, `DST_COUNTRY`, `INTERFACE`, ... | |
| 81 | | `facets` | flows | Array of facet field names whose value-distributions should appear in the response | |
| 82 | | `group_by` | flows | Up to 10 tuple-key field names (e.g. `["SRC_ADDR","DST_ADDR","PROTOCOL"]`) -- order defines the aggregation tuple | |
| 83 | | `sort_by` | flows | `bytes` or `packets` | |
| 84 | | `top_n` | flows | One of `25`, `50`, `100`, `200`, `500` | |
| 85 | | `field` | autocomplete | Facet field to autocomplete (`SRC_ADDR`, etc.) | |
| 86 | | `term` | autocomplete | Search prefix | |
| 87 | |
| 88 | --- |
| 89 | |
| 90 | ## Response envelope |
| 91 | |
| 92 | Flow Functions wrap their content in the standard Function |
| 93 | envelope (same shape as topology and logs): |
| 94 | |
| 95 | | Key | Description | |
| 96 | |---|---| |
| 97 | | `status` | HTTP-style status | |
| 98 | | `v` | Function schema version | |
| 99 | | `type` | **`flows`** -- the family discriminator | |
| 100 | | `help` / `accepted_params` / `required_params` / `has_history` / `update_every` | Discovery metadata | |
| 101 | | `data` | Mode-specific payload (object) | |
| 102 | |
| 103 | ### `data` object -- mode `flows`, view `table-sankey` |
| 104 | |
| 105 | | Key | Description | |
| 106 | |---|---| |
| 107 | | `schema_version` | `2.0` | |
| 108 | | `source` | `netflow` | |
| 109 | | `layer` | `3` | |
| 110 | | `agent_id` | Producing-agent identifier | |
| 111 | | `collected_at` | RFC3339 timestamp | |
| 112 | | `view` | Echo of requested view | |
| 113 | | `group_by` | Echo of requested group-by tuple | |
| 114 | | `columns` | Per-column display metadata | |
| 115 | | `flows[]` | Aggregated flow rows (one row per group_by tuple) | |
| 116 | | `stats` | Counters: `flows_total`, `packets_total`, `bytes_total`, etc. | |
| 117 | | `metrics` | Optional metric block | |
| 118 | | `warnings[]` | Optional non-fatal diagnostics | |
| 119 | | `facets` | When `facets` was requested in body, per-field value-counts plus `selections` echo | |
| 120 | |
| 121 | ### `data` object -- mode `flows`, view `timeseries` |
| 122 | |
| 123 | Replaces `flows[]` with `metric` (string) and `chart` (object); used |
| 124 | for line/area charts of bytes-per-second / packets-per-second |
| 125 | broken down by the group-by tuple. |
| 126 | |
| 127 | ### `data` object -- mode `flows`, geo views (`country-map`, `state-map`, `city-map`) |
| 128 | |
| 129 | Returns geo-keyed aggregations (per-country / per-state / per-city |
| 130 | totals) suitable for map rendering. |
| 131 | |
| 132 | ### `data` object -- mode `autocomplete` |
| 133 | |
| 134 | | Key | Description | |
| 135 | |---|---| |
| 136 | | `mode` | `autocomplete` | |
| 137 | | `field` | Echo of requested field | |
| 138 | | `term` | Echo of requested search term | |
| 139 | | `values[]` | Matching values for the field | |
| 140 | | `stats` / `warnings` | Same as flows mode | |
| 141 | |
| 142 | --- |
| 143 | |
| 144 | ## Examples |
| 145 | |
| 146 | ### Example 1: top-100 talker pairs by bytes, last hour |
| 147 | |
| 148 | ```bash |
| 149 | TOKEN="YOUR_API_TOKEN" |
| 150 | NODE="YOUR_NODE_UUID" |
| 151 | |
| 152 | read -r -d '' PAYLOAD <<'EOF' |
| 153 | { |
| 154 | "mode": "flows", |
| 155 | "view": "table-sankey", |
| 156 | "after": -3600, |
| 157 | "before": 0, |
| 158 | "group_by": ["SRC_ADDR", "DST_ADDR"], |
| 159 | "sort_by": "bytes", |
| 160 | "top_n": 100 |
| 161 | } |
| 162 | EOF |
| 163 | |
| 164 | curl -sS -X POST \ |
| 165 | -H 'Content-Type: application/json' \ |
| 166 | -H "Authorization: Bearer $TOKEN" \ |
| 167 | "https://app.netdata.cloud/api/v2/nodes/$NODE/function?function=flows:netflow" \ |
| 168 | -d "$PAYLOAD" \ |
| 169 | | jq '.data.flows[:5]' |
| 170 | ``` |
| 171 | |
| 172 | ### Example 2: breakdown of TCP traffic by AS name, with histogram |
| 173 | |
| 174 | ```bash |
| 175 | read -r -d '' PAYLOAD <<'EOF' |
| 176 | { |
| 177 | "mode": "flows", |
| 178 | "view": "timeseries", |
| 179 | "after": -86400, |
| 180 | "before": 0, |
| 181 | "selections": { "PROTOCOL": ["TCP"] }, |
| 182 | "group_by": ["DST_AS_NAME"], |
| 183 | "sort_by": "bytes", |
| 184 | "top_n": 25 |
| 185 | } |
| 186 | EOF |
| 187 | ``` |
| 188 | |
| 189 | ### Example 3: country-map of egress bytes |
| 190 | |
| 191 | ```bash |
| 192 | read -r -d '' PAYLOAD <<'EOF' |
| 193 | { |
| 194 | "mode": "flows", |
| 195 | "view": "country-map", |
| 196 | "after": -3600, |
| 197 | "before": 0, |
| 198 | "group_by": ["DST_COUNTRY"], |
| 199 | "sort_by": "bytes", |
| 200 | "top_n": 500 |
| 201 | } |
| 202 | EOF |
| 203 | ``` |
| 204 | |
| 205 | ### Example 4: autocomplete for a destination IP filter |
| 206 | |
| 207 | ```bash |
| 208 | read -r -d '' PAYLOAD <<'EOF' |
| 209 | { |
| 210 | "mode": "autocomplete", |
| 211 | "field": "DST_ADDR", |
| 212 | "term": "10.0.0." |
| 213 | } |
| 214 | EOF |
| 215 | ``` |
| 216 | |
| 217 | ### Example 5: discover the live parameter set first |
| 218 | |
| 219 | ```bash |
| 220 | read -r -d '' PAYLOAD <<'EOF' |
| 221 | { "info": true } |
| 222 | EOF |
| 223 | |
| 224 | curl -sS -X POST \ |
| 225 | -H 'Content-Type: application/json' \ |
| 226 | -H "Authorization: Bearer $TOKEN" \ |
| 227 | "https://app.netdata.cloud/api/v2/nodes/$NODE/function?function=flows:netflow" \ |
| 228 | -d "$PAYLOAD" \ |
| 229 | | jq '{accepted_params, required_params}' |
| 230 | ``` |
| 231 | |
| 232 | --- |
| 233 | |
| 234 | ## Limits and gotchas |
| 235 | |
| 236 | - **Cloud timeout default 120 s.** Wide-window queries |
| 237 | (`after: -86400`) over high-volume agents can hit it. Narrow |
| 238 | the time window or filter via `selections`. |
| 239 | - **`top_n` is enumerated, not free.** Allowed values are 25, 50, |
| 240 | 100, 200, 500. Other integers are rejected. |
| 241 | - **`group_by` accepts up to 10 fields.** The order matters -- |
| 242 | it's the tuple ordering for the aggregation key. |
| 243 | - **AS names depend on the configured GeoIP/AS database.** If the |
| 244 | collector has no AS database, `SRC_AS_NAME` / `DST_AS_NAME` |
| 245 | will be empty strings. Same for country/city fields. |
| 246 | - **Privacy**: flow records reveal who-talks-to-whom and how much. |
| 247 | Treat raw output as production-sensitive; never paste into |
| 248 | committed files. Direct working output to |
| 249 | `<repo>/.local/audits/...` (gitignored). |
| 250 | - **Sampled vs full flows**: NetFlow v5/v9 and sFlow are sampled |
| 251 | by source devices; reported byte/packet counts are scaled by |
| 252 | the sample rate. The collector reports raw counts -- consult |
| 253 | source-device sampling configuration when interpreting |
| 254 | absolute volumes. |
| 255 | - **Function is L3-only.** No L2 visibility (use |
| 256 | [topology Functions](./query-topology.md) for L2). No |
| 257 | application-layer dissection (use logs). |