| 1 | # Query agent metrics directly |
| 2 | |
| 3 | This guide is part of the [`query-netdata-agents`](./SKILL.md) skill. |
| 4 | Read [SKILL.md](./SKILL.md#prerequisites) first. |
| 5 | |
| 6 | For the full request body (scope / selectors / window / |
| 7 | aggregations / format / options), the response envelope (jsonwrap |
| 8 | with summary / view / result / db / timings), the time-aggregation |
| 9 | and dimension-aggregation rules, and worked examples, see |
| 10 | [../query-netdata-cloud/query-metrics.md](../query-netdata-cloud/query-metrics.md). |
| 11 | The Cloud `/api/v3/spaces/{sp}/rooms/{rm}/data` endpoint forwards |
| 12 | the same body to the agent's `/api/v3/data` endpoint. |
| 13 | |
| 14 | --- |
| 15 | |
| 16 | ## Endpoint (agent v3) |
| 17 | |
| 18 | `POST /api/v3/data` on the agent. The request body is identical to |
| 19 | the Cloud `/data` body except `scope.nodes` (cloud) is implicit |
| 20 | on the agent (you're already targeting one node). |
| 21 | |
| 22 | ## Use the wrapper |
| 23 | |
| 24 | ```bash |
| 25 | source "$(git rev-parse --show-toplevel)/.agents/skills/query-netdata-agents/scripts/_lib.sh" |
| 26 | agents_load_env |
| 27 | |
| 28 | read -r -d '' BODY <<'JSON' |
| 29 | { |
| 30 | "scope": {"contexts": ["system.cpu"]}, |
| 31 | "selectors": {"nodes": ["*"], "contexts": ["*"], "instances": ["*"], "dimensions": ["*"], "labels": ["*"], "alerts": ["*"]}, |
| 32 | "window": {"after": -600, "before": 0, "points": 5}, |
| 33 | "aggregations": { |
| 34 | "metrics": [{"group_by": ["dimension"], "aggregation": "sum"}], |
| 35 | "time": {"time_group": "average"} |
| 36 | }, |
| 37 | "format": "json2", |
| 38 | "options": ["jsonwrap", "minify", "unaligned"], |
| 39 | "timeout": 30000 |
| 40 | } |
| 41 | JSON |
| 42 | |
| 43 | agents_query_agent \ |
| 44 | --node "$NODE_UUID" \ |
| 45 | --host "$AGENT_HOST:19999" \ |
| 46 | --machine-guid "$AGENT_MG" \ |
| 47 | POST /api/v3/data "$BODY" \ |
| 48 | | jq '{view: .view.dimensions.names, points: (.result.data | length)}' |
| 49 | ``` |
| 50 | |
| 51 | ## Discover available contexts on the agent |
| 52 | |
| 53 | ```bash |
| 54 | agents_query_agent --node "$NODE_UUID" --host "$AGENT_HOST:19999" --machine-guid "$AGENT_MG" \ |
| 55 | GET '/api/v3/contexts' |
| 56 | ``` |
| 57 | |
| 58 | `/api/v3/contexts` returns the metric contexts the agent currently |
| 59 | collects (e.g. `system.cpu`, `disk.space`, `nginx.connections`). |
| 60 | Use these as `scope.contexts` values. |
| 61 | |
| 62 | ## Time resolution: `duration ÷ points = seconds per point` |
| 63 | |
| 64 | The number of `points` is NOT "give me per-second data". It is |
| 65 | "split the duration into N equal buckets". Actual time |
| 66 | resolution: |
| 67 | |
| 68 | ``` |
| 69 | seconds_per_point = abs(after) ÷ points (when before = 0) |
| 70 | seconds_per_point = abs(duration) ÷ points (when duration is set) |
| 71 | ``` |
| 72 | |
| 73 | **To get per-second data, set `points` equal to the duration in |
| 74 | seconds.** |
| 75 | |
| 76 | | You want | Set `after` | Set `points` | Result | |
| 77 | |---|---|---|---| |
| 78 | | Per-second resolution, last 2 minutes | `-120` | `120` | 1 second per point | |
| 79 | | Per-second resolution, last 5 minutes | `-300` | `300` | 1 second per point | |
| 80 | | 10-second buckets, last 10 minutes | `-600` | `60` | 10 seconds per point | |
| 81 | | Per-minute resolution, last hour | `-3600` | `60` | 60 seconds per point | |
| 82 | |
| 83 | **Common mistake**: `after: -600, points: 30` is NOT per-second |
| 84 | data over 10 minutes -- it is 20-seconds-per-point heavily |
| 85 | aggregated data. Per-second resolution over 10 minutes requires |
| 86 | `points: 600` (at the 500-point server cap; reduce duration or |
| 87 | accept coarser resolution). |
| 88 | |
| 89 | **Per-second data also requires dbengine tier 0** (per-second |
| 90 | storage) covers the requested time range. If tier 0 retention is |
| 91 | shorter than `abs(after)`, the engine auto-selects a coarser |
| 92 | tier silently. Force tier 0 with `"tier": 0` in the window to |
| 93 | fail loudly rather than silently downsample. |
| 94 | |
| 95 | **`points: 0` is NOT "per-second"** -- it means "all available |
| 96 | points within the 500 cap", which the engine still aggregates |
| 97 | when the duration exceeds 500 seconds. |
| 98 | |
| 99 | ## Limits and gotchas |
| 100 | |
| 101 | - **`scope.contexts` MUST be set.** Without it, the response |
| 102 | contains metadata for every context on the agent. |
| 103 | - **`unaligned`**: include in `options` for API queries to avoid |
| 104 | wall-clock alignment of the time window. |
| 105 | - **Max points ≈ 500** per query (server-side cap). |
| 106 | - **Single host.** For multi-node aggregation, use the Cloud |
| 107 | `/data` path documented in |
| 108 | [../query-netdata-cloud/query-metrics.md](../query-netdata-cloud/query-metrics.md). |
| 109 | |
| 110 | ## See also |
| 111 | |
| 112 | - [../query-netdata-cloud/query-metrics.md](../query-netdata-cloud/query-metrics.md) |
| 113 | -- full body / response / examples. |
| 114 | - [query-functions.md](./query-functions.md) -- generic Function |
| 115 | transport (Functions != metrics, but they share the same |
| 116 | Cloud-proxy and bearer semantics). |