@cryptotaxi247 / netdata-1 / commits / cc50307bc

Fix streaming topology graph output (#22432)

fix streaming topology graph output

Costa Tsaousis committed May 6, 2026 at 19:52 UTC cc50307bc6ca180285a0e0cda16d73ab7a42cd86
12 files changed +2223 -1097
.agents/sow/current/SOW-0012-20260505-streaming-topology-classification-bugs.md new
+544
@@ -0,0 +1,544 @@
1 +# SOW-0012 - Streaming topology: parent/child classification, missing-parent rendering, and "Since" column type
2 +
3 +## Status
4 +
5 +Status: in-progress
6 +
7 +Sub-state: triage complete; live cloud evidence captured; root causes located; reviewed by six external assistants in three iterations; user decisions recorded 2026-05-06; implementation in progress; backend topology filters removed from scope by project-owner decision.
8 +
9 +## Requirements
10 +
11 +### Purpose
12 +
13 +Make the `topology:streaming` Function (`function-streaming.c::function_streaming_topology`) produce correct, useful output on **both sides of a streaming relationship**: when invoked on a parent agent and when invoked on a child agent. The view from a child must (a) classify the child correctly, (b) render its upstream parent(s) as actors, and (c) render time fields in human form.
14 +
15 +This is in service of the broader topology PR (https://github.com/netdata/netdata/pull/22110, merged April 14 as commit `7da4565`) being usable by SREs without manual decoding of Unix epoch values or manual reasoning about misclassified node types.
16 +
17 +### User Request
18 +
19 +The reporter raised four concerns, summarised:
20 +
21 +1. "Child agent is identified as parent — bug related to vnodes?"
22 +2. "In the streaming topology, the 'Since' field shows number 1,777,992,161 — assume this is a timestamp not rendered properly."
23 +3. "I also hoped to have it show its parents."
24 +4. "The streaming topology function on the child seems broken. No facets, no filters, nothing."
25 +
26 +The reporter shared a screenshot of `topology:streaming on [child-1] ([CHILD_IP])` with `0 results` in the result counter, the child's modal popup showing `Type: parent`, `Children: 1`, and a streaming-path table where the `Since` column rendered raw integers (`1,777,992,161` and `1,777,992,171`).
27 +
28 +### Assistant Understanding
29 +
30 +Facts (verified via live Cloud queries on the reporter's space, 2026-05-05; raw responses preserved at `<repo>/.local/audits/streaming-topology/` which is gitignored):
31 +
32 +- Topology: `[parent-1]` is the parent; `[child-1]` is the only real streaming child. Six other actors (legacy stale streams) appear as `vnode` (marked virtual).
33 +- On the **parent** (`[parent-1]`) view: `actors_n=8`, `links_n=7`. **`[parent-1]` itself is classified `node_type:"child"`, `child_count:0`** — wrong; it has 1 active streaming child. `[child-1]` is correctly `child`. Link `[child-1] → [parent-1]` correctly emitted with `link_type:"streaming"`.
34 +- On the **child** (`[child-1]`) view: `actors_n=1`, `links_n=0`. The single actor is `[child-1]` itself, classified `node_type:"parent"` with `child_count:1`. `[parent-1]` (the actual parent) is **not present as an actor**. The actor's `streaming_path` field is `[[child-1], [parent-1]]` — so the path data is known to the function, but is not turned into a second actor.
35 +- The streaming-path table on each actor (emitted via `rrdhost_stream_path_to_json` at function-streaming.c:1327) carries `since` as raw Unix epoch seconds (`1777992161` for the child, `1777992171` for the parent).
36 +- Function name is `topology:streaming` (registered at functions.c:23). Cloud Function URL: `POST /api/v2/nodes/{nodeId}/function?function=topology:streaming`.
37 +
38 +Inferences:
39 +
40 +- The "no facets, no filters" symptom on the child is a consequence of the response being almost empty (1 actor, 0 links). Once the child's view contains a correctly classified child plus a synthetic parent actor plus the upstream link, the FE will have data to populate facets/filters from.
41 +- The misclassification on the parent and the misclassification on the child are produced by **different bugs that happen to land on the same `parent_child_count` dictionary** — see Analysis.
42 +
43 +Resolved (after iteration-2 review with cloud-frontend access):
44 +
45 +- The cloud-frontend's topology table renderer accepts `"timestamp"` columns in **either seconds or milliseconds**: at `cloud-frontend/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` it auto-detects via `ms > 1e12 ? ms : ms * 1000`. The agent emits `since` in seconds; the FE multiplies by 1000 → correct date rendering. Decision 4 changes accordingly (Option 2 chosen — see below).
46 +- Re-emitting `stream_path_send_to_parent(host)` after a parent-originated path update **does** create an oscillation risk. `rrdhost_stream_path_to_json` overlays a fresh `rrdhost_stream_path_self()` at every emit (stream-path.c:181-202), and that fresh self pulls live values (`first_time_t` from `rrdhost_retention()`, `start_time_ms`/`shutdown_time_ms` from `get_agent_event_time_median()`) that change continuously. The XXH128 hash is computed over stored content but the emit produces fresh content each time, so the receiving side's hash differs every round-trip → perpetual re-sends. This rules out a naive protocol fix and reinforces the SOW's original Option 3 Hybrid recommendation for Decision 1.
47 +
48 +### Acceptance Criteria
49 +
50 +- On the **parent view**: parent's actor has `node_type:"parent"` and `child_count >= 1`. Verification: `agents_call_function --via cloud --node $PARENT --function topology:streaming` returns `actors[0].attributes` with `node_type=="parent"` and `child_count > 0` for the localhost actor.
51 +- On the **child view**: child's actor has `node_type:"child"` and `child_count==0`. Verification: same Cloud call against the child's node id; `actors[0].attributes.node_type=="child"`.
52 +- On the **child view**: at least one synthetic actor for the parent is present, and a `link` from child to parent with `link_type=="streaming"`. Verification: `data.actors[]` contains an entry for the parent's machine GUID with `actor_type=="parent"` (the existing actor type — no new type introduced; see Decision 3); `data.links[]` contains an entry with `src_actor_id` = child and `dst_actor_id` = parent. For deeper chains, `data.links[]` contains links between every consecutive non-localhost path slot.
53 +- The `Since` column in the streaming-path table on each actor renders as a human date in the FE, not a raw integer. Verification: visual check in Netdata Cloud UI on both parent and child views.
54 +- The backend always exposes the complete topology graph. `topology:streaming` must not accept or apply backend graph-pruning filters such as `node_type`, `ingest_status`, or `stream_status`. Facets or filtering in the UI are client-side concerns over the complete response. Verification: `accepted_params` contains only non-pruning params, currently `info`, and actors/links are emitted without backend filter checks.
55 +- The two functions previously co-located in `src/web/api/functions/function-streaming.{c,h}` are split into one file each: `function-netdata-streaming.{c,h}` (registered as `netdata-streaming`) and `function-topology-streaming.{c,h}` (registered as `topology:streaming`). The simple-table function is renamed `function_streaming` → `function_netdata_streaming` for symbol/file consistency. Verification: the old files are removed; both new files compile and the registered Function names continue to work end-to-end via Cloud calls.
56 +- A maintenance reference document `src/streaming/STREAM_PATH.md` is added describing the streaming-path subsystem (storage rule, propagation rules, cycle terminator, update triggers, vnode special case, intended classification logic, mergeability requirements). Verification: the file exists, is **not referenced in `docs/.map/map.yaml`** (so not picked up by the learn-ingestion pipeline; co-location under `src/streaming/` is not by itself sufficient — `src/streaming/README.md` IS in the map and gets ingested), and covers the nine outline points in the Plan section below.
57 +- Tests covering the four root causes — see Validation Plan.
58 +
59 +## Analysis
60 +
61 +Sources checked:
62 +
63 +- `src/web/api/functions/function-streaming.c` (full file, 2459 lines)
64 +- `src/web/api/functions/function-streaming.h`
65 +- `src/web/api/functions/functions.c` (Function registration)
66 +- `src/streaming/stream-path.c` (full file)
67 +- `src/streaming/stream-path.h`
68 +- `src/streaming/stream-sender.c:147-161` (`stream_sender_on_ready_to_dispatch`, only caller of `stream_path_send_to_parent` from connection setup)
69 +- `src/streaming/stream-sender-execute.c:109-110` (sender-side receive of upstream STREAM_PATH)
70 +- `src/plugins.d/pluginsd_parser.c:1211` (parser-side receive of STREAM_PATH from a child)
71 +- `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` (FE timestamp handling — auto-detects seconds vs ms)
72 +- `<dashboard-repo>/src/domains/functions/topology/payload.js` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` (FE actor-id dedup, link-derived synthesis)
73 +- Live data: sanitized summaries derived from raw Function responses stored under `<repo>/.local/audits/streaming-topology/` (gitignored)
74 +- Reporter screenshot of the broken UI (saved locally; not committed)
75 +
76 +### Intended classification logic (the rule the function should implement)
77 +
78 +Confirmed with project owner 2026-05-06. The topology function is meant to classify each entry in `rrdhost_root_index` according to the following rule:
79 +
80 +1. **Source of truth = `rrdhost_root_index`.** This is the local agent's authoritative list of nodes it knows about: localhost, every real child currently or previously connected (including stale/archived ones), and every vnode registered locally.
81 +2. **For each non-vnode node, read its `stream.path.array`.** Slot 0 is the origin of that chain (the originating "child"). Slots 1+ are the upstream hops.
82 +3. **Tag entries by slot:**
83 + - Slot 0 → origin (a child role in this chain).
84 + - Slots 1+ → parent role in this chain.
85 +4. **Globally,** a node X is classified as `parent` if X appears at slot 1+ in **any** node's path. Otherwise X is `child`.
86 +5. **vnodes are special.** Their stored path is empty (vnodes do not stream themselves — their data is collected by the local agent). At JSON-emit time, slot 0 of a vnode's path is filled by the collecting agent (via emit-time self-append). vnodes are tagged `vnode` directly and skip the slot-1+ counter step.
87 +6. **Stale nodes are special.** A node whose ingest status is `RRDHOST_INGEST_STATUS_ARCHIVED` is tagged `stale` directly and rendered with a stale link back to the collecting agent (localhost).
88 +
89 +Both Bug A and Bug B sit on the same step (#3 — slot-1+ extraction feeding the global classification at #4) but fail for different reasons. Bug A: the data being read at slot 1+ is incomplete (stored paths haven't converged yet on the apex parent). Bug B: the slot-1+ extraction itself is contaminated (the helper appends localhost into a result that is supposed to contain only upstream entries). Together they cause the classification to be wrong on **both** sides of the same parent-child relationship — the parent shows as a child (Bug A), the child shows as a parent (Bug B).
90 +
91 +### Intended actor set per agent
92 +
93 +Confirmed with project owner 2026-05-06. From any single agent's perspective, the topology view emits the following actors. All edges/chains route through `self`; nothing branches around it.
94 +
95 +| Source | Actor type | Where the data comes from | Link to self |
96 +|---|---|---|---|
97 +| `self` (localhost) | `parent` (if it has any active streaming children) or `child` (otherwise) | `rrdhost_root_index` (localhost) | — (origin of self's chain) |
98 +| Active vnodes | `vnode` | `rrdhost_root_index` (entries with `RRDHOST_OPTION_VIRTUAL_HOST`) | `vnode → self`, link_type derives from how the agent collects the vnode |
99 +| Active children (real, currently streaming in) | `child` | `rrdhost_root_index` (entries with active receiver, not virtual, not archived) | `child → self`, link_type `streaming` |
100 +| Stale children (archived, last seen on this agent) | `stale` | `rrdhost_root_index` (entries with `RRDHOST_INGEST_STATUS_ARCHIVED`) | `stale → self`, link_type `stale` |
101 +| Upstream parents of self | `parent` (existing actor type, no new type introduced) | self's `stream.path.array` slots 1+ (synthesized actors — they are not in `rrdhost_root_index`) | `self → parent_1`, `parent_1 → parent_2`, ... — link_type `streaming`, one link per consecutive non-localhost path slot |
102 +
103 +Children's full chains, after path convergence, look like `[child, self, parent_1, parent_2, ...]`. The tail `[self, parent_1, ...]` is identical to self's own chain, so after dedup by `actor_id`, the children's chains add no new upstream actors — they share self's chain. Same for vnodes and stale children.
104 +
105 +Counting example: an agent with 100 vnodes + 100 active children + 20 stale children + 2 upstream parents emits **223 actors total** (1 self + 100 + 100 + 20 + 2) and **222 unique links** (100 + 100 + 20 + 1 + 1).
106 +
107 +Sizing note: this matches the original PR description's caveat that very large topologies produce large responses (carried over from the unfixed scope, not introduced by this SOW). The synthesized upstream `parent` actors are bounded by the chain depth (typically 1–3), so this SOW does not add a sizing risk.
108 +
109 +### Topology must be mergeable across agents
110 +
111 +Confirmed with project owner 2026-05-06. The Cloud frontend (or any consumer) must be able to merge the per-agent topology responses from N parents into a single unified view. The agent's response shape and per-actor / per-link attributes must support this.
112 +
113 +What each parent contributes to a merged view (where it does not overlap with what other parents already contributed):
114 +
115 +- **Retention** for each child the parent received data from — already emitted per-actor in the `retention` and (per-host) inbound/outbound tables.
116 +- **Its own vnodes** — vnodes are local to the collecting agent, so two parents typically do not overlap on vnodes (unless misconfigured).
117 +- **Its own stale children** — these can overlap (a child that streamed to A, then moved to B, will appear `stale` on A and `child` on B). The merge must pick the authoritative current state.
118 +- **Its own active children and upstream parents**, where they don't overlap.
119 +
120 +Mergeability requirements satisfied by the current response schema:
121 +
122 +1. **Stable, deterministic actor IDs.** Every actor's `actor_id` is `netdata-machine-guid:<guid>` produced by `streaming_topology_actor_id_from_guid` (`function-streaming.c:107-115`). The same guid produces the same id across agents. Merge dedups by id.
123 +2. **Stable link tuples.** Each link is keyed by (`src_actor_id`, `dst_actor_id`, `link_type`). Same src + dst + type across two parents = the same link.
124 +3. **Timestamps for tie-breaking.** Each link carries `discovered_at` (when this agent first saw the connection) and `last_seen` (the most recent observation). When the same actor appears as `child` on one parent and `stale` on another, the merge can compare timestamps and decide the authoritative current state (the `last_seen` of the streaming link should be more recent than the `last_seen` of the stale link). The Bug C synthesis must preserve these fields on synthesized actors and links — derive `since`/timestamps from the `STREAM_PATH` struct fields (`since`, `first_time_t`).
125 +4. **No agent claims authority over hosts it doesn't see directly.** Each parent reports only the children/vnodes that connect to it, plus its own upstream chain. A child of parent-A is not represented in parent-B's response unless that child has also (currently or historically) connected to B.
126 +
127 +Implication for Bug C synthesis: a synthesized `parent` actor on a child's view must use the same canonical id format (`netdata-machine-guid:<guid>`) so dedup by `actor_id` works across responses. Synthesized links must carry `discovered_at`/`last_seen` derived from the `STREAM_PATH` struct's `since` and `first_time_t` fields (NOT from `now`), so two parents reporting the same upstream produce comparable timestamps and the merge layer can resolve overlaps deterministically.
128 +
129 +The aggregator does not need a new "self" marker on the response. The existing top-level `data.agent_id` (function-streaming.c:840-841) carries the emitting agent's GUID; the aggregator can compute the corresponding self-actor_id as `netdata-machine-guid:<agent_id>` to find the localhost actor in `actors[]`. Authority resolution per actor: when actor X appears in multiple responses, the response whose `agent_id` matches X's GUID is authoritative for X's attributes (full retention, full inbound/outbound, etc.); other responses contribute confirmations and link records but do not override.
130 +
131 +Implication for the migration scenario (a child moved from A to B): both A's and B's responses are valid local truth. The merge layer holds both link records (`child→A stale` and `child→B streaming`), with `last_seen` distinguishing them; the actor's "current state" in the merged view derives from the most recent link.
132 +
133 +Current state:
134 +
135 +The function has four observable defects. Each is documented with file:line evidence and a concrete reproduction trace. Bugs A and B are **distinct root causes** that both feed the same `parent_child_count` dictionary; Bugs C and D are independent of A and B. Fixing any one bug does not fix the others.
136 +
137 +### Bug A — On a parent's view, the parent classifies itself as `child`
138 +
139 +Symptom: `actors[0]` in the parent's response: `actor_type:"child"`, `attributes.child_count:0`. The localhost is classified as a non-parent on its own view.
140 +
141 +Root cause (mapped to the intended logic above): step #3 (slot-1+ extraction) reads stored paths that haven't converged yet. The data feeding the classification is incomplete — the parent does not yet appear at slot 1+ in the child's stored path on the apex parent's side, because convergence is sparse-trigger-driven and hasn't fired since the child connected.
142 +
143 +Design context (verified by code review and confirmed with project owner):
144 +
145 +The streaming-path subsystem is designed for full multi-hop convergence. In a chain `child → proxy → grandparent → ...`, every agent in the chain eventually stores the full path for any host whose data flows through, by this mechanism:
146 +
147 +- **From-below (`from_parent=false`):** store, then propagate UP via `stream_path_send_to_parent(host)` (works on a proxy because each forwarded host has its own sender — `stream-receiver-connection.c:188` creates the child host record with a sender when the local agent has streaming forwarding configured) and DOWN via `stream_path_send_to_child(host)`. Bidirectional.
148 +- **From-above (`from_parent=true`):** store, propagate DOWN only. The `if(!from_parent)` guard at stream-path.c:423 is the cycle terminator.
149 +- **Self is appended at JSON-emit time** (`rrdhost_stream_path_to_json` at stream-path.c:198-202), not at storage time. This avoids storing stale local fields (timing medians, retention edges) that would diverge from "live" self.
150 +- **Update triggers** are sparse: connect, retention boundary movement (`first_time_s` change at `rrdcontext-worker.c:99-104`), node_id assignment, parent disconnect.
151 +
152 +Trace (verified) — why the bug appears on the **apex parent** in this specific topology:
153 +
154 +1. The child starts streaming to the parent. The child's stored `stream.path.array` is initially empty.
155 +2. On `stream_sender_on_ready_to_dispatch` (stream-sender.c:157), the child calls `stream_path_send_to_parent(localhost)`. The payload, built by `rrdhost_stream_path_to_json` (stream-path.c:177-209), contains only the appended `tmp = rrdhost_stream_path_self(localhost)` entry — `[{child, hops=0}]`.
156 +3. The parent receives at `pluginsd_parser.c:1211`: `stream_path_set_from_json(child, json, from_parent=false)`. The parent stores `child->stream.path.array = [{child, hops=0}]`.
157 +4. After the hash changes (stream-path.c:422-429), the parent calls `stream_path_send_to_parent(host=child_host_on_parent)` and `stream_path_send_to_child(host=child_host_on_parent)`:
158 + - `stream_path_send_to_parent` is a **no-op on the apex parent** in this topology because the apex has no upstream — `child_host_on_parent->sender == NULL` (the local agent does not forward to a grandparent). On a proxy this would propagate the path up. **The apex case is what breaks convergence in this specific scenario.**
159 + - `stream_path_send_to_child` does the JSON-emit-time self-append and sends the child the enriched view including the parent. The child stores `[{child, hops=0}, {parent, hops=1}]`.
160 +5. After this initial round-trip, the parent's stored copy of the child's path is `[{child}]` — the parent's own host_id is not yet stored. Convergence on the parent depends on the child re-emitting later.
161 +6. The child re-emits via `stream_path_send_to_parent(localhost)` only on sparse triggers — retention boundary movement (`first_time_s` change), node_id update, or reconnect. **In a freshly-connected child (e.g., after the reporter removed cloud.d on the child), none of these have fired yet.** The parent's storage stays at the initial `[{child}]`.
162 +7. Phase 1 of `function_streaming_topology` (function-streaming.c:780-799) iterates `rrdhost_root_index` and, for each host, calls `streaming_topology_get_path_ids(host, from=1, ...)`:
163 + - For `host=parent` (localhost on apex): `parent->stream.path.array` is empty → returns 0.
164 + - For `host=child`: stored `[{child, hops=0}]` only → `rrdhost_stream_path_get_host_ids(child, 1, ...)` returns 0 (nothing at position 1+) → the localhost-append at function-streaming.c:243-244 is gated by `n > 0` so it does not fire → returns 0.
165 + - For each vnode: empty stored path → returns 0.
166 +8. `parent_child_count[parent's UUID]` is therefore **never** incremented.
167 +9. Actor classification (function-streaming.c:858-866): `cc = NULL` → `node_type = "child"` → wrong.
168 +
169 +So the bug is **not** "the parent never knows about itself" — the design eventually converges. The bug is that the topology function reads stored paths for classification, and those stored paths can lag behind the converged state for an arbitrary period (until the next retention/node_id event on the originating child). In a "fresh" topology (recent reconnect, recent cloud.d removal, etc.) the lag is observable. The function returning a wrong answer because the storage hasn't caught up yet is incorrect behavior — the function should report current truth, not last-converged-state.
170 +
171 +### Bug B — On a child's view, the child classifies itself as `parent`
172 +
173 +Symptom: `actors[0]` in the child's response: `actor_type:"parent"`, `attributes.child_count:1`. The localhost is classified as a parent of itself.
174 +
175 +Root cause (mapped to the intended logic above): step #3 (slot-1+ extraction) is **contaminated** at the helper. `streaming_topology_get_path_ids` blindly appends `localhost->host_id` to the result for **any** value of `from`, including `from=1` which is used by Phase 1 parent-counting. The append is correct semantics for `from=0` (full-path rendering — slot 0 plus slots 1+) but wrong for `from=1` (only slots 1+, by definition the host itself must not be there). The helper sneaks the localhost into the slot-1+ result, polluting the count.
176 +
177 +File: `src/web/api/functions/function-streaming.c:225-247`
178 +
179 +```c
180 +static uint16_t streaming_topology_get_path_ids(RRDHOST *host, uint16_t from, ND_UUID *host_ids, uint16_t max) {
181 + uint16_t n = rrdhost_stream_path_get_host_ids(host, from, host_ids, max);
182 + uint16_t filtered_n = 0;
183 +
184 + bool found_localhost = false;
185 + for(uint16_t i = 0; i < n; i++) {
186 + if(UUIDiszero(host_ids[i]))
187 + continue;
188 + host_ids[filtered_n++] = host_ids[i];
189 + if(UUIDeq(host_ids[i], localhost->host_id)) {
190 + found_localhost = true;
191 + }
192 + }
193 + n = filtered_n;
194 +
195 + // append localhost if not found (same as rrdhost_stream_path_to_json)
196 + if(!found_localhost && n < max && n > 0)
197 + host_ids[n++] = localhost->host_id;
198 +
199 + return n;
200 +}
201 +```
202 +
203 +Trace (verified):
204 +
205 +1. The child's stored path on itself is `[{child, hops=0}, {parent, hops=1}]` (set after the child receives the parent's stream-path back; verified via `actors[0].tables.streaming_path` in the live response).
206 +2. Phase 1: `streaming_topology_get_path_ids(child, from=1, ...)`:
207 + - `rrdhost_stream_path_get_host_ids(child, 1, ...)` returns `[parent's host_id]` (n=1).
208 + - found_localhost: parent != child (localhost) → false.
209 + - Append condition `!found_localhost && n > 0` → **append child** → returns `[parent, child]` (n=2).
210 +3. Loop at function-streaming.c:786-798 increments both `parent_child_count[parent] = 1` (correct) and `parent_child_count[child] = 1` (incorrect).
211 +4. Actor classification for the child (function-streaming.c:864-865): `cc = 1` → `node_type = "parent"` → wrong.
212 +5. `attributes.child_count` (function-streaming.c:923) reads the same value, so the modal shows `Children: 1` as in the screenshot.
213 +
214 +This bug is independent of Bug A. Even if Bug A is fixed at the protocol layer, Bug B will still misclassify any child agent on its own view.
215 +
216 +Every caller of `streaming_topology_get_path_ids` (verified by grep — there are **6**, not 5):
217 +
218 +- `function-streaming.c:785` — Phase 1 parent counting. `from=1`. **Wants the append off.**
219 +- `function-streaming.c:801` — Phase 1 descendants computation. `from=0`. Wants the append on.
220 +- `function-streaming.c:971` — Per-actor `streaming_path` field for FE highlight_path. `from=0`. Wants the append on.
221 +- `function-streaming.c:1045` — Observer-mode inbound table source resolution. `from=0`. Wants the append on. *(Missed in the iteration-1 SOW; identified by iter-1 reviewers; verified.)*
222 +- `function-streaming.c:1269` — Outbound table (non-observer parent). `from=0`. Wants the append on.
223 +- `function-streaming.c:1407` — Phase 4 link emission (looks at `link_ids[1]` — the direct parent). `from=0`, `max=2`. Wants the append on (this is how the child→parent link is drawn even on the parent despite Bug A).
224 +
225 +So one caller wants the append off, five want it on. The narrowest fix is to gate the append on `from == 0` inside `streaming_topology_get_path_ids` itself. No call site changes needed.
226 +
227 +### Bug C — On a child's view, the parent agent is not rendered as an actor
228 +
229 +Symptom: child's response has `actors_n: 1`, `links_n: 0`. The parent is referenced by `actors[0].streaming_path[1]` and known to the function via the child's `stream.path.array[1]`, but no actor is emitted for the parent.
230 +
231 +Root cause: actors are emitted exclusively from `rrdhost_root_index` (function-streaming.c:847-848 in Phase 3, again at 1372-1375 in Phase 4 for links). On a child agent, that index contains only localhost. Path entries that point to nodes outside the local index are never synthesized.
232 +
233 +This is what the reporter asked for in their second message — *"I also hoped to have it show its parents"*. The information is local on the child (in `STREAM_PATH` array entries which carry `hostname`, `host_id`, `node_id`, `claim_id`, `hops`, `since`, `first_time_t`, `start_time_ms`, `shutdown_time_ms`, `capabilities`, `flags`); it is just not turned into actor records.
234 +
235 +The cloud-frontend can synthesize endpoint nodes from links if a link references a missing actor (`cloud-frontend/src/domains/functions/topology/payload.js:395-404`), but it does not synthesize actors from the per-actor `streamingPath` array. Since the child view has zero links, the FE fallback does not rescue this case.
236 +
237 +### Bug D — `Since` column declared as `"number"`, FE renders raw integer
238 +
239 +Symptom: streaming-path table shows `1,777,992,161` instead of a date.
240 +
241 +Root cause: `function-streaming.c:355` declares the column as `"number"`. The path's `since` field is emitted in seconds (stream-path.c:88, `buffer_json_member_add_uint64(wb, "since", p->since)`).
242 +
243 +After iteration-2 review with the cloud-frontend source available: the FE accepts `"timestamp"` columns in **either seconds or milliseconds** (`cloud-frontend/src/domains/functions/components/topology/actorModal/dataTable.js:52-58`):
244 +
245 +```javascript
246 +case "timestamp": {
247 + const ms = Number(val)
248 + if (isNaN(ms) || ms === 0) return "-"
249 + const ts = ms > 1e12 ? ms : ms * 1000
250 + // ... renders as date
251 +}
252 +```
253 +
254 +So changing the column type to `"timestamp"` is sufficient — no data emission change needed. (Compare to `db_from`/`db_to` at function-streaming.c:374-375 which use `"timestamp"` and emit ms via `* MSEC_PER_SEC` — a stylistic convention but not required by the FE.)
255 +
256 +### Bug E — "no facets, no filters" on the child view (consequence)
257 +
258 +Symptom: screenshot shows the right pane with the Function picker tree but no facet pills.
259 +
260 +Root cause: with one actor and zero links, the FE has no field distributions to populate facets from. Fixing A+B+C should populate the response with the child (correctly classified), the parent (synthetic actor for the upstream parent), and the link between them, which gives the FE enough material for facets.
261 +
262 +Risks:
263 +
264 +- The fix is contained inside `function-streaming.c`. The streaming-path subsystem stays as-is — no protocol change, no wire-format change, no change to `stream-path.c`. The cycle terminator at stream-path.c:423 (`if(!from_parent) stream_path_send_to_parent(host);`) is load-bearing and must not be touched: it's what stops a child→parent→grandparent chain from re-sending forever after each downstream echo.
265 +- (Bug B fix) flipping the localhost-append off for `from > 0` is a behavior change in a small public-ish helper. The six call sites must be re-validated; in particular the link-drawing path at function-streaming.c:1407 must continue to work for stale/disconnected children. (Verified: with `from=0`, the gate stays on.)
266 +- (Bug C fix) synthesizing actors from path entries reuses the existing `actor_type:"parent"`. The synthesized actors use the same canonical id format (`streaming_topology_actor_id_from_guid` at function-streaming.c:107-115) so the FE highlight_path lookup matches across the per-actor `streaming_path` field references and the new actor records. Sub-specs the implementation must pin down: (a) de-duplication against `rrdhost_root_index` so a path entry whose host_id matches a real local host does not produce a duplicate actor; (b) which sub-tables are populated — the synthetic actor's `tables` block is built directly from the `STREAM_PATH` struct fields so the streaming-path table renders; the inbound/outbound/retention tabs that depend on data the agent doesn't have for a remote parent will be empty (acceptable for a single-agent view; aggregated views in Cloud will fill them from the parent's own response); (c) link-emission pass for the synthesized chain — emit a `link_type:"streaming"` for **every consecutive non-localhost path slot** (`slot[i] → slot[i+1]` for `i >= 1`), not only `slot[0] → slot[1]`. The existing Phase 4 loop at function-streaming.c:1372-1467 only emits one link per `rrdhost_root_index` host with `max=2` at line 1407; the new synthesis pass emits the multi-hop links from the path; (d) link timestamps — synthesized `discovered_at` and `last_seen` derived from the `STREAM_PATH.since` and `STREAM_PATH.first_time_t` fields of each path entry, so cross-agent merge of the same upstream link produces comparable values.
267 +- (Bug D fix) changing only the column type (Option 2) is a pure metadata change; it does not touch the data emission or the inter-agent wire format. Risk: very low.
268 +- Same-failure search: `rrdhost_stream_path_total_reboot_time_ms` (stream-path.c:145-158) shares Bug A's blind assumption (that localhost is in localhost's own stored path). On a top parent it returns 0 silently. Add to the scan list during validation. The `topology:snmp` Function (separate) may have similar column-type issues; check `since`/timestamp columns there as part of validation, even if no fix is needed.
269 +
270 +## Pre-Implementation Gate
271 +
272 +Status: ready (user decisions recorded 2026-05-06: Decision 1 = Option 1, Decision 2 = Option 1, Decision 3 = Option 1 with no new actor type, Decision 4 = Option 2, hardening deferred to separate SOW)
273 +
274 +Problem / root-cause model:
275 +
276 +- See Bugs A–E in Analysis. Two distinct root causes (A: stale storage on parent; B: spurious localhost-append on `from > 0`) plus two missing-feature-style root causes (C: actors only iterate `rrdhost_root_index`; D: column type/unit). E is a downstream consequence of A+B+C.
277 +
278 +Evidence reviewed:
279 +
280 +- Code: `src/web/api/functions/function-streaming.c` (Phase 1: 759-829; Phase 3 actors: 845-1361; Phase 4 links: 1363-1505; helpers: 56-247, 313-360), `src/web/api/functions/functions.c:20-30`, `src/streaming/stream-path.c` (state machine: 79-96 emit, 98-143 self, 177-209 to_json, 219-247 send paths, 249-259 get_host_ids, 261-288 disconnect handlers, 368-435 set_from_json), `src/streaming/stream-sender.c:147-161`, `src/streaming/stream-sender-execute.c:109-110`, `src/plugins.d/pluginsd_parser.c:1211`.
281 +- Cloud-frontend (commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc`): `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` (timestamp auto-detection), `<dashboard-repo>/src/domains/functions/topology/payload.js:263-265, 350-383, 395-404` (actor-id dedup; link-derived synthesis fallback).
282 +- Live evidence: Cloud Function responses for the reporter's parent and child, captured 2026-05-05, stored under `<repo>/.local/audits/streaming-topology/` (gitignored). Sanitized facts above.
283 +- Reporter screenshot of the failing UI (viewed locally; not committed).
284 +- Independent read-only review batches completed in two iterations. Iteration outputs preserved under `<repo>/.local/audits/streaming-topology/` (gitignored).
285 +
286 +Affected contracts and surfaces:
287 +
288 +- The `topology:streaming` Function output (consumed by Netdata Cloud frontend topology renderer).
289 +- The presentation metadata under `presentation.actor_types.{parent,child,vnode,stale}` (function-streaming.c:561-615) — Decision 3 reuses the existing `parent` type for synthesized upstream actors; **no new actor type is introduced**.
290 +- Internal helper `streaming_topology_get_path_ids` semantics — Decision 2.
291 +- Streaming-path table column metadata at function-streaming.c:345-360 — Decision 4.
292 +- File layout: `src/web/api/functions/function-streaming.{c,h}` is split into `function-netdata-streaming.{c,h}` and `function-topology-streaming.{c,h}`; `function_streaming` is renamed to `function_netdata_streaming`. Plus a new maintenance reference at `src/streaming/STREAM_PATH.md`. See Plan items 0 and 6.
293 +- Decision 1 (localhost live-state classification) does NOT affect inter-agent streaming. The streaming protocol and `stream-path.c` are NOT touched by this SOW. (Defensive hardening for the streaming-path JSON parser — array length clamp + scalar range checks — is moved to a separate hardening SOW; out of scope here.)
294 +
295 +Existing patterns to reuse:
296 +
297 +- `streaming_topology_actor_id_from_guid` (function-streaming.c:107-115) for synthetic actor IDs.
298 +- `STREAM_PATH` struct fields (stream-path.h, stream-path.c:79-96) for synthetic-actor attributes.
299 +- The existing `parent` actor preset (function-streaming.c:563-575) is reused as-is for synthesized upstream actors. No new presentation block is needed.
300 +
301 +Risk and blast radius:
302 +
303 +- Decision 1 (localhost live-state classification) is contained in `function-streaming.c`. Implementation must populate **both** `parent_child_count[localhost]` AND `parent_descendants[localhost]` from the same root-index walk; otherwise the `received_nodes` block (function-streaming.c:982-1002) stays empty even when `child_count > 0`. **Zero changes to the streaming protocol or to `stream-path.c`.**
304 +- Decisions 2 and 3 are local to `function-streaming.c`.
305 +- Decision 4 Option 2 is pure metadata; trivial blast radius.
306 +
307 +Sensitive data handling plan:
308 +
309 +- The Cloud responses captured for evidence contain customer-identifying info (real hostnames, private IPs, agent UUIDs, claim IDs, machine GUIDs, host labels). Raw responses are stored under `<repo>/.local/audits/streaming-topology/` (gitignored).
310 +- This SOW has been sanitized: hostnames are `[parent-1]`, `[child-1]`, vnodes unnamed; private IPs `[CHILD_IP]`; UUIDs not written; reporter referred to as "the reporter"; the customer space referred to as the reporter's space; Slack identifiers not included.
311 +- Test fixtures derived from the Cloud responses must use sanitized hostnames/UUIDs (e.g., `parent-1`, `child-1`, `00000000-0000-0000-0000-000000000001`).
312 +- No tokens, bearers, or claim IDs appear in this SOW or in any planned test fixture.
313 +
314 +Implementation plan (deferred; depends on user decisions below):
315 +
316 +0. **File split (precondition).** Confirmed with project owner 2026-05-06 — both streaming functions move to their own files, with `function-streaming.{c,h}` deleted. Steps:
317 + - Create `src/web/api/functions/function-topology-streaming.{c,h}`. Move `function_streaming_topology` and every `streaming_topology_*` static helper, plus the `RRDFUNCTIONS_STREAMING_TOPOLOGY_HELP` macro, into the new file. New header exports `function_streaming_topology` and the help macro. The former `value_in_csv` helper is removed by Decision 5 because the topology function no longer applies backend graph filters.
318 + - Create `src/web/api/functions/function-netdata-streaming.{c,h}`. Move `function_streaming` (currently at function-streaming.c:1499) and the `RRDFUNCTIONS_STREAMING_HELP` macro into the new file. **Rename** the C symbol `function_streaming` → `function_netdata_streaming` to match the registered Function name (`netdata-streaming`) and the file name. Update the registration in `functions.c:8-18`.
319 + - Delete the old `src/web/api/functions/function-streaming.{c,h}` files.
320 + - Update the build manifest (CMake source list under `src/web/api/functions/` or its parent) to remove `function-streaming.c` and add the two new sources.
321 + - All four bug fixes below (steps 1–4) land in the new `function-topology-streaming.c` after the split commit. The split is a single mechanical commit with no behavior change.
322 +1. Bug B fix (Decision 2). Lowest risk; isolated to `streaming_topology_get_path_ids` (in the new `function-topology-streaming.c`). Ship after the split.
323 +2. Bug A fix (Decision 1). Risk depends on chosen option.
324 +3. Bug C fix (Decision 3). Adds synthetic upstream actors and corresponding link entries on the child view. Reuses existing `actor_type:"parent"` — no new actor type and no presentation block changes.
325 +4. Bug D fix (Decision 4). Smallest patch; metadata only.
326 +5. **(Removed.)** Defensive hardening for the streaming-path JSON parser (array length clamp + scalar range checks) is moved to a separate hardening SOW. This SOW does not touch `src/streaming/stream-path.c` at all.
327 +6. **Maintenance documentation.** Confirmed with project owner 2026-05-06 — write `src/streaming/STREAM_PATH.md` co-located with the streaming-path source. The file must **not be referenced in `docs/.map/map.yaml`** so the learn-ingestion pipeline ignores it. (Co-location under `src/streaming/` is not by itself sufficient — `src/streaming/README.md` IS in the map.) Covers:
328 + 1. What `stream.path.array` represents — chain of hops a host's data takes; per-host on every agent that knows about the host.
329 + 2. Slot semantics — slot 0 = origin; slots 1+ = upstream hops (parents/proxies).
330 + 3. Storage rule — `stream_path_set_from_json` stores received entries as-is; `rrdhost_stream_path_to_json` overlays/appends a fresh `rrdhost_stream_path_self()` at JSON-emit time only. Why the asymmetry exists (avoids storing volatile local fields like `first_time_t` and agent-event medians).
331 + 4. Propagation — bidirectional from-below (`from_parent=false`: up + down); down-only from-above (`from_parent=true`: down only). Multi-hop convergence works because each forwarded host on a proxy gets its own sender (`stream-receiver-connection.c:188`).
332 + 5. The cycle terminator — `if(!from_parent)` at `stream-path.c:423`. Why it must not be touched. What a future protocol redesign would need to replace it with first.
333 + 6. Update triggers — connect (`stream-sender.c:157`); retention boundary movement, specifically `first_time_s` change (`rrdcontext-worker.c:99-104`); node_id update (`sqlite_metadata.c:288`, `command-nodeid.c:163`); parent disconnect (`stream-sender.c:171`). Note: triggers are sparse — convergence has lag; consumers must not assume "stored = current truth".
334 + 7. Consumer guidance — for any feature that asks "is X a parent? what is X's chain?", read live state where possible; fall back to stored paths only as a hint. Lists known consumers: `function-topology-streaming.c` (post-split), `api_v2_contexts.c:425, 510`, `rrdhost_stream_path_total_reboot_time_ms` (`stream-path.c:145-158`).
335 + 8. vnode special case — vnodes don't stream; their stored path is empty; emit-time self-append fills slot 0 with the collecting agent.
336 + 9. Topology-function intended classification logic — short summary mirroring the SOW's "Intended classification logic" and "Intended actor set per agent" sections; cross-reference SOW-0012 in `.agents/sow/done/` for the bug history and fix design.
337 +7. Tests covering each bug at the function level (parent view, child view, mid-chain view). Real-use validation against the reporter's environment after each fix lands.
338 +8. Same-failure search across the codebase: any other column declared `"number"` that holds Unix epoch; any other consumer of `host->stream.path.array` that assumes self is stored — including `rrdhost_stream_path_total_reboot_time_ms` (`stream-path.c:145-158`) which shares the same blind assumption.
339 +
340 +Validation plan:
341 +
342 +- Unit-style: build a minimal test that constructs synthetic `STREAM_PATH` arrays for a 2-host parent/child scenario and asserts the classification, `child_count`, `actors[]`, `links[]`, and the streaming-path table types and values. The function emits text to a `BUFFER`, so the test runs the emission path and parses the JSON.
343 +- Integration-style: re-run the captured Cloud queries after the fix is deployed to a build of the parent and the child, and re-fetch the screenshot UI on Cloud.
344 +- Same-failure scan: `grep` for columns of type `"number"` that hold timestamps elsewhere in `function-streaming.c` and other Function emitters; check `rrdhost_stream_path_total_reboot_time_ms`.
345 +- Decision 1 specific: verify that `received_nodes` populates correctly on the parent view after the fix (i.e., that `parent_descendants[localhost]` was also populated by the live-state walk).
346 +
347 +Artifact impact plan:
348 +
349 +- `AGENTS.md`: not affected. No workflow change.
350 +- Runtime project skills: not affected directly.
351 +- Specs (`.agents/sow/specs/`): no spec change. The streaming protocol is untouched. The topology function's classification semantics are local to `function-streaming.c` and need no spec.
352 +- End-user/operator docs: Netdata Cloud frontend public docs do not document the streaming-topology Function's response shape at the field level; no docs update needed.
353 +- End-user/operator skills: `query-netdata-cloud` skill should add a how-to for "Calling `topology:streaming` and interpreting actor types" — mandated by the skill's own live-catalog rule.
354 +- SOW lifecycle: this SOW is opened in `pending/` and will move to `current/` only after Decisions 1–4 are answered.
355 +
356 +Open-source reference evidence:
357 +
358 +- No external reference required for these bugs — they are entirely internal to Netdata's streaming/topology code path.
359 +
360 +Open decisions:
361 +
362 +See "Implications And Decisions" below. Implementation cannot start until each is answered.
363 +
364 +## Implications And Decisions
365 +
366 +### Decision 1 — Fix scope for Bug A (parent classification)
367 +
368 +Background: the streaming-path subsystem is designed for full multi-hop convergence and **must not be touched**. Each forwarded host has its own sender on a proxy, and `stream_path_send_to_parent` propagates child paths up the chain through proxies. The cycle terminator at stream-path.c:423 is load-bearing. The bug is that on the **apex parent** in any topology, convergence depends on the child re-emitting after a sparse trigger (retention boundary movement, node_id update, reconnect). Until that fires, the apex's storage of the child's path is stuck at the initial `[{child}]` from connect-time, so Phase 1 sees nothing. The topology function reads stored paths and therefore returns a wrong answer until convergence completes.
369 +
370 +The fix is **entirely inside the topology function** (`function-streaming.c`). The streaming-path subsystem stays as-is.
371 +
372 +Options:
373 +
374 +1. **Localhost-only live-state fix.** For `host == localhost` only, replace Phase 1's path-based counting with a direct check on the local agent state via `rrdhost_status()` (the same helper the function already uses at lines 850, 1021, 1113, 1158, 1220, 1289 — do **not** access `host->receiver` directly). A host counts as an active streaming child iff `rrdhost_status(host, ...).ingest.type == RRDHOST_INGEST_TYPE_CHILD` and `s.ingest.status` is in `{RRDHOST_INGEST_STATUS_ONLINE, RRDHOST_INGEST_STATUS_REPLICATING}`. `child_count` becomes the count of such hosts. The implementation must populate **both** `parent_child_count[localhost]` AND `parent_descendants[localhost]` from the same root-index walk, so `received_nodes` (function-streaming.c:982-1002) stays consistent with the new count. The existing slot-0+ append at function-streaming.c:815-822 must skip localhost in this fix's path so we do not double-write `parent_descendants[localhost]`. For non-localhost hosts, keep the existing path-based logic unchanged.
375 + - Pros: contained in `function-streaming.c`. Zero streaming-protocol change. Trivially correct on the localhost side regardless of stored-path lag.
376 + - Cons: multi-hop / non-localhost parent classification (e.g., the topology view from a child showing whether its parent is itself a parent of further children) still depends on the stored streaming_path of children. Heals on the next sparse trigger.
377 + - Risk: low.
378 +
379 +2. **All-host live-state fix.** Same as Option 1 but extend the live-state check to non-localhost hosts too: walk `rrdhost_root_index` once and compute parent classification for every host based on observable state (receivers attached, sender attached, virtual flag, archive status), not stored paths.
380 + - Pros: classification is correct immediately for every host on every view, regardless of stored-path lag.
381 + - Cons: more code, more cases to enumerate. Non-localhost hosts cannot be checked via `rrdhost_status()` for receiver-presence in the same way as localhost — that information is only locally observable for hosts whose children are also locally registered (the proxy/apex case). For pure remote children that are not themselves proxies, there is nothing more to count anyway, so the additional code mostly degenerates back to the Option 1 case.
382 + - Risk: low-medium (more surface to test).
383 +
384 +**Recommendation: Option 1.** Smallest change that fixes the user-visible bug. Non-localhost classification heals on the next sparse trigger — acceptable degradation for this SOW. If, after this lands, the residual surfaces in real use, a follow-up SOW can extend the live-state check to all hosts (still no protocol change).
385 +
386 +**Residual note (line 1197):** Option 1 fixes the classification but leaves a related symptom — the observer-mode outbound table at `function-streaming.c:1196-1208` calls `rrdhost_stream_path_get_host_ids(host, 0, ...)` directly to find the streaming destination. On the apex parent the stored path for a child is `[{child}]` only at connect-time, so this lookup finds no destination and `dst_hostname` falls back to the peer IP at function-streaming.c:1209. The IP fallback already produces a usable label; this heals on the next sparse trigger like the classification does. **No protocol change; if needed later, fix in `function-streaming.c` by reading live state for the destination too.**
387 +
388 +### Decision 2 — Fix for Bug B (localhost-append for `from > 0`)
389 +
390 +Background: `streaming_topology_get_path_ids` blindly appends `localhost->host_id` for any value of `from`. Of the six call sites (verified by grep), only one (`from=1`, Phase 1 parent counting) wants the append off.
391 +
392 +Options:
393 +
394 +1. **Gate the append on `from == 0`.** Smallest change: at function-streaming.c:243-244, change the condition from `if(!found_localhost && n < max && n > 0)` to `if(from == 0 && !found_localhost && n < max && n > 0)`.
395 + - Pros: minimal diff; existing five "want append on" call sites unchanged.
396 + - Cons: ties the append semantic to a magic value (`from == 0` happens to mean "full path"). Caller still has to know.
397 + - Risk: very low.
398 +
399 +2. **Add an explicit boolean parameter.** `streaming_topology_get_path_ids(host, from, append_localhost, host_ids, max)`. Update all six call sites.
400 + - Pros: explicit semantics; future-proof.
401 + - Cons: more invasive diff; six call sites to touch.
402 + - Risk: low.
403 +
404 +3. **Split into two helpers.** `streaming_topology_get_full_path_ids` (with append) and `streaming_topology_get_upstream_path_ids` (without). Make the intent obvious from the function name.
405 + - Pros: most readable.
406 + - Cons: most invasive diff.
407 + - Risk: low.
408 +
409 +**Recommendation: Option 1.** Smallest, safest change. Pair with a comment at the helper explaining why the gate exists. If more flexibility is needed later, refactor in a follow-up.
410 +
411 +### Decision 3 — Fix scope for Bug C (rendering parents on child view)
412 +
413 +Background: actors are emitted only for hosts in `rrdhost_root_index`. The child has the parent's metadata in its `STREAM_PATH` array but no rrdhost record for it.
414 +
415 +Options:
416 +
417 +1. **Synthesize `parent` actors from `STREAM_PATH` entries that point outside `rrdhost_root_index`.** For each path entry whose host_id is not localhost and not already in `rrdhost_root_index`, emit an actor with the existing `actor_type:"parent"`. Use STREAM_PATH fields for attributes (hostname, host_id, node_id, claim_id, since, flags, capabilities, hops, first_time_t, start_time_ms, shutdown_time_ms). Add multi-hop links between every consecutive non-localhost path slot in a separate pass after the existing Phase 4 link loop.
418 + - Sub-specs the implementation must define before coding:
419 + - (a) De-duplication: skip emission if the path entry's host_id matches any host already iterated from `rrdhost_root_index`. (FE drops duplicate backend actor IDs at `<dashboard-repo>/src/domains/functions/topology/payload.js:263-265` — relying on FE dedup is not the agent's contract.)
420 + - (b) Sub-tables: synthetic actors populate only the `streaming_path` table from the STREAM_PATH struct fields. The `inbound`, `outbound`, `retention` tabs that depend on data the agent doesn't have for a remote upstream will render empty — acceptable for a single-agent view; aggregated views in Cloud will fill them from the parent's own response (whose `agent_id` matches that parent's actor_id, making it the authoritative source for that actor — see "Topology must be mergeable across agents" section).
421 + - (c) Link emission: a second loop after the Phase 4 link loop. For each synthesized chain, emit `link_type:"streaming"` for **every consecutive non-localhost path slot pair** (`slot[i] → slot[i+1]` for `i >= 1`), not only `slot[0] → slot[1]`. This means deep chains (e.g., `[child, parent_1, parent_2, parent_3]`) get all transitive links emitted: `parent_1 → parent_2`, `parent_2 → parent_3`. The existing Phase 4 already emits `child → parent_1` (`function-streaming.c:1407` with `max=2`).
422 + - (d) Link timestamps: synthesized links derive `discovered_at` and `last_seen` from the STREAM_PATH `since` and `first_time_t` fields of each path entry, NOT from `now`. This preserves cross-agent merge semantics.
423 + - FE coupling: with `actor_type:"parent"` reused, no change to `presentation.actor_types`, no change to `presentation.legend.actors`, no FE update required.
424 + - Pros: solves the reporter's third complaint. Uses data already local. No new actor type, no FE coupling, no presentation block additions.
425 + - Cons: synthesized parent's modal will show empty inbound/outbound/retention tabs in single-agent view. Aggregated Cloud views fill them.
426 + - Risk: low-medium. Touches actor emission loop and adds a small link-emission loop. Reuses existing infrastructure throughout.
427 +
428 +2. **Defer.** Track in a follow-up SOW.
429 + - Pros: smaller PR, easier to review.
430 + - Cons: child view stays useless until the follow-up lands. Two of the reporter's three complaints (#1 the misclassification, #3 the parents-not-shown) are about the child view; #1 is fixed by Bug B alone but #3 is not addressed.
431 +
432 +**Recommendation: Option 1.** The data is already local; the reporter asked explicitly; reusing the existing `parent` actor type avoids any FE coupling.
433 +
434 +### Decision 4 — Fix for Bug D (Since column type)
435 +
436 +Background: `since` is emitted as `uint64` Unix-epoch seconds at stream-path.c:88. Column type at function-streaming.c:355 is `"number"`. Frontend renders the raw integer (with locale-formatted separators).
437 +
438 +Iteration-2 review with cloud-frontend access confirmed:
439 +
440 +- The FE's `"timestamp"` case at `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` auto-detects ms vs seconds: `const ts = ms > 1e12 ? ms : ms * 1000`. A value of `1777992161` (< 1e12) is multiplied by 1000 → correct date. A value of `1777992161000` (> 1e12) is used directly → also correct.
441 +
442 +Options:
443 +
444 +1. **Change column type AND post-process to ms inside `function-streaming.c` only.** Introduce a local helper that emits the path with `since * MSEC_PER_SEC`. Matches the in-tree style used for `db_from`/`db_to`.
445 + - Pros: stylistic consistency with the rest of `function-streaming.c`.
446 + - Cons: requires a custom emit path duplicating `rrdhost_stream_path_to_json`.
447 + - Risk: low.
448 +
449 +2. **Change column type only.** Set type to `"timestamp"` at function-streaming.c:355; leave the data emission path unchanged (raw seconds). The FE auto-detects and converts.
450 + - Pros: minimal diff (one string change). No new helper.
451 + - Cons: depends on a FE auto-detection heuristic. If the FE ever tightens to require ms, this silently regresses. The threshold `1e12` (year 33658 in seconds, year 2001 in ms) is comfortably distant from realistic timestamps either way.
452 + - Risk: very low.
453 +
454 +3. **Use a different column type** (e.g., `"datetime"` or `"duration_ago"`).
455 + - Pros: matches FE's intended semantic if `"timestamp"` is not the right type.
456 + - Cons: requires reading FE topology-table renderer to confirm the supported types — the topology-modal renderer's switch shows only `"timestamp"`, `"duration"`, `"badge"`, `"actor_link"`, plus a default. So this is not strictly necessary.
457 +
458 +**Recommendation: Option 2.** Switched from Option 1 after iteration-2 FE review. The FE's auto-detection means there is no unit to align — the simplest fix is correct. The in-tree style argument for Option 1 is real but not load-bearing.
459 +
460 +### Decision 5 — Backend graph filters
461 +
462 +Background: the topology function currently advertises and applies optional backend graph-pruning filters (`node_type`, `ingest_status`, `stream_status`). The UI does not expose these filters, and graph pruning in the backend makes the topology response incomplete: actors can disappear, links can be suppressed, and synthetic parent emission has to reason about filtered-out local hosts. This conflicts with the product requirement that topology should expose the full graph.
463 +
464 +Options:
465 +
466 +1. **Remove backend graph filters.** Keep only `info` as an accepted non-pruning parameter. Always emit all actors and links known to the local agent plus synthetic upstream parents derived from `STREAM_PATH` when no local `RRDHOST` exists. UI facets may still filter or highlight client-side over the full response.
467 + - Pros: complete topology response; simpler actor/link flow; no filtered-out actor edge cases; matches the current UI behavior.
468 + - Cons: callers cannot request a smaller backend response by status/type.
469 + - Risk: low. These filters are not visible in the UI and are optional today.
470 +
471 +2. **Keep backend filters.** Preserve current accepted params and filtering branches.
472 + - Pros: existing API shape unchanged for any hidden caller.
473 + - Cons: incomplete graph responses; complex synth/dedup semantics; contradicts the project-owner requirement that all nodes must be exposed.
474 + - Risk: medium. Keeps the current class of graph correctness traps.
475 +
476 +**Decision: Option 1.** Remove `node_type`, `ingest_status`, and `stream_status` from `accepted_params` and delete backend actor/link/synthetic filter checks. `info` remains because it controls response metadata shape, not graph pruning.
477 +
478 +## Plan
479 +
480 +User decisions recorded 2026-05-06: Decision 1 = Option 1 (localhost-only live-state); Decision 2 = Option 1 (gate append on `from==0`); Decision 3 = Option 1 (synthesize `parent` actors, no new actor type); Decision 4 = Option 2 (column type only, FE auto-detects); Decision 5 = Option 1 (remove backend graph filters; keep only `info`). Defensive hardening for `stream-path.c` deferred to a separate SOW.
481 +
482 +PR commit order: split → B → A → D → C → doc → tests → validation.
483 +
484 +0. **File split** (precondition; mechanical, no behavior change). Create `function-topology-streaming.{c,h}` and `function-netdata-streaming.{c,h}`; move `function_streaming_topology` + helpers and `function_streaming` (renamed `function_netdata_streaming`) respectively; delete `function-streaming.{c,h}`; update `functions.c` includes/registrations and the build manifest. `STREAMING_FUNCTION_UPDATE_EVERY` is duplicated in each new file (per project owner's call — no shared header). All subsequent line-number references in this SOW are pre-split; the bug-fix commits land in the new `function-topology-streaming.c` and may be at different line numbers.
485 +1. **Bug B fix** (Decision 2 = Option 1): one-line change at the equivalent of `function-streaming.c:243-244` (now in `function-topology-streaming.c`) plus comment. Add unit-style test asserting Phase 1 parent-counting on a child agent yields `parent_child_count[localhost] == 0`.
486 +2. **Bug A fix** (Decision 1 = Option 1, localhost live-state): rewrite Phase 1's localhost write into both `parent_child_count` and `parent_descendants` from a `rrdhost_root_index` walk that uses `rrdhost_status()` (the abstraction the function already uses) and counts hosts where `s.ingest.type == RRDHOST_INGEST_TYPE_CHILD` and `s.ingest.status ∈ {ONLINE, REPLICATING}`. **No `host->receiver` access.** Skip the slot-0+ append for localhost in `parent_descendants` to avoid double-write. No protocol change.
487 +3. **Bug C fix** (Decision 3 = Option 1, no new actor type): add synthetic-actor emission loop after the existing `rrdhost_root_index` loop in Phase 3 (dedup against local `rrdhost_root_index` actors, streaming_path table built directly from STREAM_PATH struct fields). Synthesized actors use `actor_type:"parent"` — existing type, no presentation block changes. Add a second link-emission loop after the existing Phase 4 loop that emits `link_type:"streaming"` for every consecutive non-localhost path slot pair (`slot[i] → slot[i+1]` for `i >= 1`); link timestamps derived from STREAM_PATH `since`/`first_time_t`, not `now`.
488 +4. **Bug D fix** (Decision 4 = Option 2): change the `"since"` column type from `"number"` to `"timestamp"`. No data change.
489 +5. **Backend filter removal** (Decision 5 = Option 1): remove `node_type`, `ingest_status`, and `stream_status` parsing, accepted params, and actor/link/synthetic emission checks. Keep `info`.
490 +6. **Maintenance documentation**: write `src/streaming/STREAM_PATH.md` per the outline in the implementation plan above. Verify the file is **not referenced in `docs/.map/map.yaml`** (co-location under `src/streaming/` is not by itself sufficient).
491 +7. **Same-failure scan**: grep for other `"number"`-typed columns holding Unix-epoch values; grep for other consumers of `host->stream.path.array` that may share Bug A's assumption (specifically `rrdhost_stream_path_total_reboot_time_ms` at `stream-path.c:145-158`).
492 +8. **Tests**: see Validation Plan; minimum coverage = one test per bug, plus a test for the line 1197 outbound-table behavior (verify peer-IP fallback works for stale-storage cases).
493 +9. **Validation**: build & deploy to the reporter's parent and child; re-fetch Cloud responses; verify all acceptance criteria.
494 +
495 +## Execution Log
496 +
497 +### 2026-05-05
498 +
499 +- Read full Slack thread reporting the issue (11 messages including the screenshot the reporter shared).
500 +- Loaded `query-netdata-cloud` skill helpers from `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
501 +- Resolved the reporter's space, room, and node UUIDs.
502 +- Captured live Cloud Function responses for `topology:streaming` on both parent and child; saved under `<repo>/.local/audits/streaming-topology/`.
503 +- Confirmed Bugs A, B, C, D, E with file:line evidence.
504 +- Drafted SOW iteration-1.
505 +- Ran an independent read-only review batch for iteration-1; outputs preserved under `<repo>/.local/audits/streaming-topology/`.
506 +- Ran the same independent review scope for iteration-2 with the cloud-frontend source path included; outputs preserved under `<repo>/.local/audits/streaming-topology/`.
507 +- Iteration-2 corrections applied to this SOW: caller count 5 → 6 (added line 1045); Bug A trace step 4 reworded (the call exists but is a no-op due to `host->sender == NULL` on the parent side); Decision 4 recommendation switched to Option 2 (FE auto-detects seconds via `ms > 1e12 ? ms : ms * 1000`); Decision 1 sub-spec added (must populate both `parent_child_count[localhost]` and `parent_descendants[localhost]`); Decision 3 sub-specs added (filter, dedup, sub-tables, second link pass); same-failure scan target added (`rrdhost_stream_path_total_reboot_time_ms`); line 1197 residual noted; uint16_t truncation hardening item added; SOW sanitized (hostnames, UUIDs, IPs, names, customer-space identifiers replaced with placeholders).
508 +- 2026-05-06 — confirmed with project owner that the streaming-path subsystem must not be touched by this SOW. Decision 1 collapsed from three options (protocol fix / function-only / hybrid) to two options that are both contained in `function-streaming.c`: (1) localhost-only live-state classification — recommended; (2) all-host live-state classification — deferred to a follow-up if the residual surfaces. Removed all references to a protocol fix follow-up. The cycle terminator at stream-path.c:423 stays load-bearing and untouched.
509 +- 2026-05-06 — added the project owner's intended classification logic (six rules) and intended actor set (per-agent count formula) to the Analysis section. Added the cross-agent mergeability requirements section (stable actor IDs, stable link tuples, timestamps for tie-breaking, no overreach). Synthesized upstream `parent` actors must preserve the canonical actor-id format and derive link timestamps from the `STREAM_PATH` struct so the merge layer can reconcile views from multiple parents.
510 +- 2026-05-06 — added scope: split `function-streaming.{c,h}` into `function-netdata-streaming.{c,h}` and `function-topology-streaming.{c,h}`, with `function_streaming` renamed to `function_netdata_streaming`. The bug fixes land in the new topology file. Added scope: write `src/streaming/STREAM_PATH.md` as a co-located maintenance reference (not picked up by learn ingestion). Both additions confirmed with project owner.
511 +- 2026-05-06 — iteration-3 independent read-only review batch completed; outputs preserved under `<repo>/.local/audits/streaming-topology/`. Project owner's corrections applied: (1) **dropped `remote_parent` actor type** — synthesized upstream actors reuse the existing `actor_type:"parent"`, no new type, no presentation block changes, no FE coupling; (2) Decision 3 sub-spec adds multi-hop link emission for every consecutive non-localhost path slot, not only `slot[0] → slot[1]`; (3) Decision 3 sub-spec mandates synthesized link timestamps derive from STREAM_PATH `since`/`first_time_t`, not `now`; (4) Decision 1 wording corrected — use `rrdhost_status()` (`s.ingest.type == CHILD` and `s.ingest.status ∈ {ONLINE, REPLICATING}`), no `host->receiver` access; (5) Decision 1 sub-spec — skip slot-0+ append for localhost to avoid double-write to `parent_descendants[localhost]`; (6) **deferred all `stream-path.c` defensive hardening** (uint16_t array clamp + scalar range checks) to a separate hardening SOW — this SOW now touches zero shared subsystems beyond the topology function and a new doc; (7) replaced workstation paths (`~/src/...`) with repo-relative paths (`<repo>/...`) and dashboard-repo placeholder (`<dashboard-repo>/...`); (8) corrected STREAM_PATH.md framing — the protection from learn ingestion is "not referenced in `docs/.map/map.yaml`", not "not under `docs/`"; (9) pinned cloud-frontend evidence to commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` next to the `dataTable.js:52-58` citation. STREAMING_FUNCTION_UPDATE_EVERY shared macro: project owner's call — duplicate per file, no shared header. Aggregator self-marker proposal dropped — existing `data.agent_id` is sufficient for cross-agent merge.
512 +- 2026-05-06 — project-owner decision: remove backend topology graph filters because the UI does not expose them and the backend must return the full graph. Recorded as Decision 5. Implementation updated `function-topology-streaming.c` so `accepted_params` only advertises `info`; removed `node_type`, `ingest_status`, and `stream_status` parsing and all actor/link/synthetic filter checks.
513 +- 2026-05-06 — revisited topology flow after removing filters. Added a separate `local_actor_ids` set for all `RRDHOST`-backed actors, keeping `emitted_actors` as only the actors actually written to JSON. Synthetic upstream parents are now skipped when a local `RRDHOST` actor exists, not because a filtered actor happened to be present or absent. `parent_descendants[localhost]` is now written by one live-state pass: vnodes as `virtual`, active/repl children as `streaming`, and disconnected/history-only local hosts as `stale`; the path walk no longer has a competing localhost stale branch. Phase 4 now emits vnode links as `link_type:"virtual"` to localhost regardless of stored path length, registers each emitted link in `emitted_links`, and the synthetic link pass relies on that shared dedup instead of the previous `phase4_emitted` heuristic.
514 +- 2026-05-06 — validation so far: focused compiler syntax check of `src/web/api/functions/function-topology-streaming.c` passed using the existing `functions.c` compile flags from `compile_commands.json`. Full `cmake --build build-clion --target netdata -j4` did not reach code compilation because CMake reconfigured and failed while fetching the pre-existing Sentry/crashpad dependency (`mini_chromium` HTTP 400 / expected acknowledgments). This is an environment/dependency fetch failure, not a compiler error from the topology file.
515 +- 2026-05-06 — after project owner installed the build locally, queried the installed local agent through bearer-protected direct API without printing tokens or durable identifiers. Sanitized runtime summary: `status=200`, `accepted_params=["info"]`, `actors=22`, `links=22`, actor type counts `child=13`, `parent=2`, `stale=1`, `vnode=6`, link type counts `streaming=15`, `virtual=6`, `stale=1`, duplicate actor ids `0`, duplicate link tuples `0`, vnode stale links `0`.
516 +- 2026-05-06 — project owner decision: do not add tests before first PR publication. Open a draft PR now so online reviewers can inspect the code. Tests remain a known validation gap and this SOW remains in progress until follow-up validation is complete or the project owner explicitly accepts closure without tests.
517 +
518 +## Validation
519 +
520 +Partial, implementation still in progress.
521 +
522 +- Focused syntax validation passed for `src/web/api/functions/function-topology-streaming.c` using the existing `functions.c` compile command from `compile_commands.json`, replacing the source path and using `-fsyntax-only`.
523 +- Full local build attempted with `cmake --build build-clion --target netdata -j4`. It failed during CMake reconfigure while fetching the pre-existing Sentry/crashpad dependency (`external/crashpad/third_party/mini_chromium/mini_chromium`, HTTP 400 / expected acknowledgments) before compiling Netdata sources.
524 +- Same-failure search for removed backend filter identifiers in `function-topology-streaming.c` found no remaining `value_in_csv`, `filter_node_type`, `filter_ingest_status`, `filter_stream_status`, `node_type:`, `ingest_status:`, or `stream_status:` references.
525 +- Installed-agent runtime validation via direct local Function call: `status=200`; `accepted_params` contains only `info`; graph contains 22 actors and 22 links; actor/link duplicate checks are zero; vnode stale-link check is zero.
526 +- Tests: deferred by project-owner decision for the first draft PR. This is a known gap for reviewers, not a completed validation item.
527 +
528 +## Outcome
529 +
530 +Pending.
531 +
532 +## Lessons Extracted
533 +
534 +Pending.
535 +
536 +## Followup
537 +
538 +- If the multi-hop / non-localhost classification residual surfaces in real use after this SOW lands, open a follow-up SOW that extends the live-state classification (Decision 1 Option 1) to all hosts (Option 2 in this SOW). **No protocol change is contemplated** — both this SOW and any follow-up are entirely inside the topology function.
539 +- Add a how-to under `docs/netdata-ai/skills/query-netdata-cloud/how-tos/` describing how to call `topology:streaming` and how to interpret actor types and the streaming-path table.
540 +- **Open a separate SOW for streaming-path defensive hardening** in `src/streaming/stream-path.c`: (a) clamp incoming JSON array length to `UINT16_MAX` before `callocz` to prevent uint16_t truncation of `host->stream.path.size`/`used`; (b) add scalar range checks for `hops` (int16_t), `start_time_ms` (uint32_t), `shutdown_time_ms` (uint32_t) against negative inputs from a malformed peer at `stream-path.c:313-317`. Pre-existing defensive gap, not introduced by this work.
541 +
542 +## Regression Log
543 +
544 +None yet.
CMakeLists.txt
+4 -2
@@ -1429,8 +1429,10 @@ set(API_PLUGIN_FILES
1429 src/web/api/functions/functions.h
1430 src/web/api/functions/function-progress.c
1431 src/web/api/functions/function-progress.h
1432 - src/web/api/functions/function-streaming.c
1433 - src/web/api/functions/function-streaming.h
1432 + src/web/api/functions/function-netdata-streaming.c
1433 + src/web/api/functions/function-netdata-streaming.h
1434 + src/web/api/functions/function-topology-streaming.c
1435 + src/web/api/functions/function-topology-streaming.h
1436 src/web/api/queries/rrdr.c
1437 src/web/api/queries/rrdr.h
1438 src/web/api/queries/query.c
src/streaming/STREAM_PATH.md new
+193
@@ -0,0 +1,193 @@
1 +# Streaming-path subsystem — maintenance reference
2 +
3 +## 1. What `stream.path.array` represents
4 +
5 +For every `RRDHOST` known to the local agent — its own `localhost`, every
6 +real child currently or previously connected, and every vnode registered
7 +locally — the `host->stream.path` field stores a small ordered array of
8 +[`STREAM_PATH`](stream-path.c) entries. Each entry describes one hop in the
9 +chain that this host's data takes from its origin upstream toward whichever
10 +agent currently serves as the apex of the chain.
11 +
12 +The chain is per-host. On any given agent, two hosts can have different
13 +chains stored.
14 +
15 +## 2. Slot semantics
16 +
17 +Entries are sorted ascending by the `hops` field (`compare_by_hops`). Slot 0
18 +has the lowest hops and is the **origin** — the agent where the data was
19 +collected. Slots 1+ are the **upstream hops**: the agent at slot 1 is the
20 +direct parent of slot 0, the agent at slot 2 is the parent of slot 1, etc.
21 +
22 +Helper for read access by index: `rrdhost_stream_path_get_host_ids(host,
23 +from, host_ids, max)` (returns just the UUIDs starting at `from`).
24 +
25 +For full per-entry data with a callback: `rrdhost_stream_path_visit(host,
26 +from, cb, userdata)` (added by SOW-0012; mirrors the get_host_ids locking
27 +pattern but exposes hostname, hops, since, capabilities, flags, etc.).
28 +
29 +## 3. Storage rule (received vs emitted)
30 +
31 +This is the single most important invariant to understand:
32 +
33 +- **Storage**: `stream_path_set_from_json(host, json, from_parent)` clears
34 + the existing array and stores exactly the entries received in the JSON,
35 + then sorts by hops. **Self is not added at storage time.**
36 +- **Emission**: `rrdhost_stream_path_to_json(wb, host, key, add_version)`
37 + iterates the stored array. For any entry whose `host_id == localhost->
38 + host_id`, it overlays the entry with a fresh `rrdhost_stream_path_self()`
39 + to refresh the volatile fields (timing medians, retention boundaries). If
40 + no stored entry matches localhost, it appends a fresh self at the end.
41 +
42 +Why the asymmetry exists: `rrdhost_stream_path_self()` reads `first_time_t`
43 +from `rrdhost_retention()` and `start_time_ms` / `shutdown_time_ms` from
44 +`get_agent_event_time_median()`. These change continuously. Storing them
45 +would either require a watcher to keep storage current, or accept that
46 +storage always lags the live values. Appending self at emit time avoids
47 +both problems by always emitting current self while keeping the stored array
48 +constant for everyone else's view of the host.
49 +
50 +**Consequence**: any consumer that reads `host->stream.path.array` directly
51 +(or via `rrdhost_stream_path_get_host_ids` / the visitor) is reading
52 +storage, which may lag the live wire format. See §7.
53 +
54 +## 4. Propagation rules
55 +
56 +`stream_path_set_from_json(host, json, from_parent)` runs the
57 +[propagation switch](stream-path.c) at the bottom:
58 +
59 +```c
60 +if(!XXH128_isEqual(old_hash, new_hash)) {
61 + if(!from_parent)
62 + stream_path_send_to_parent(host);
63 + stream_path_send_to_child(host);
64 +}
65 +```
66 +
67 +| Trigger source | `from_parent` | Path emitted UP | Path emitted DOWN |
68 +|---|---|---|---|
69 +| Path message received from a **child** (via `pluginsd_parser`) | `false` | YES (only meaningful on a proxy where `host->sender` is set) | YES (echo back to the child with self appended) |
70 +| Path message received from our **upstream parent** (via `stream-sender-execute`) | `true` | NO (cycle terminator) | YES (forward downstream to our children, only meaningful on a proxy) |
71 +
72 +Multi-hop convergence works because each forwarded host on a proxy gets its
73 +own sender — see `stream-receiver-connection.c:188` where the receiver
74 +creates the new host with `config.send.enabled` propagated. The same host
75 +record on the proxy has both a receiver (from below) and a sender (to
76 +above), so a child-originated update flows all the way to the apex.
77 +
78 +## 5. The cycle terminator
79 +
80 +The `if(!from_parent)` guard at `stream-path.c:423` is **load-bearing**.
81 +Without it, every parent-to-child echo would generate a child-to-parent
82 +re-send, generating another echo, forever.
83 +
84 +Combined with the volatile self-overlay (see §3), even the hash check at
85 +`stream-path.c:422` cannot stop oscillation: the live timing fields change
86 +between emits, so the hash would differ on every round-trip even when the
87 +structural content is identical.
88 +
89 +If a future protocol redesign needs to remove this guard (e.g., to make
90 +storage authoritative on every node so consumers don't have to read stored
91 +data with a "may be stale" caveat), it must FIRST replace the cycle
92 +terminator with a different mechanism (e.g., a per-message generation
93 +counter that gets incremented only by the originating node) AND make the
94 +self-overlay deterministic for hash purposes (split volatile fields out of
95 +the hash input).
96 +
97 +## 6. Update triggers
98 +
99 +`stream_path_send_to_parent` and `stream_path_send_to_child` are called
100 +from a small, sparse set of triggers — they do NOT fire on every metric
101 +write:
102 +
103 +- **Connect** — `stream_sender_on_ready_to_dispatch` at
104 + `stream-sender.c:157` calls `stream_path_send_to_parent(localhost)` when
105 + the sender becomes ready.
106 +- **Retention boundary movement** — `rrdcontext_post_process_updates` at
107 + `rrdcontext-worker.c:99-104` calls `stream_path_retention_updated(host)`
108 + only when `host->retention.first_time_s` changes. This is sparse — once
109 + per DB rotation / archive event.
110 +- **node_id update** — `sqlite_metadata.c:288` (when persisted) and
111 + `command-nodeid.c:163` (when receiving a NODE_ID command from a child)
112 + call `stream_path_node_id_updated(host)`.
113 +- **Parent disconnect** — `stream-sender.c:171` calls
114 + `stream_path_parent_disconnected(s->host)` when our upstream connection
115 + drops. This truncates self's stored chain back to localhost.
116 +
117 +Storage convergence has lag. Right after a child connects, the parent's
118 +storage of the child's path is `[child]` only. After the next sparse trigger
119 +fires on the child (typically minutes-to-hours), the parent's storage
120 +catches up to the full chain.
121 +
122 +## 7. Consumer guidance
123 +
124 +For any feature that asks "is X a parent? what is X's chain?", read live
125 +state where possible (`rrdhost_status()` exposes `s.ingest.type` and
126 +`s.ingest.status` per host, no need to touch `host->receiver` or
127 +`host->sender` directly). Fall back to stored paths only as a hint, with
128 +the understanding that storage lags the live state.
129 +
130 +Known consumers of `host->stream.path.array` in this tree:
131 +
132 +- `src/web/api/functions/function-topology-streaming.c` — the
133 + `topology:streaming` Function. Uses both `streaming_topology_get_path_ids`
134 + (paths for actor/link emission) and `rrdhost_stream_path_visit` (Bug C
135 + synthesis from SOW-0012). Reads stored data, falls back to the
136 + rrdhost-derived live data via `rrdhost_status()` for the localhost
137 + classification (Bug A in SOW-0012).
138 +- `src/database/contexts/api_v2_contexts.c:425, :510` — emits
139 + `streaming_path` per host via `rrdhost_stream_path_to_json` (so the
140 + emit-time self-append fires). No direct array access.
141 +- `src/streaming/stream-path.c:145-158` — `rrdhost_stream_path_total_reboot_time_ms`
142 + walks the stored array looking for localhost's own entry. **This shares
143 + Bug A's blind assumption**: on the apex parent, the localhost entry is
144 + not in storage, so this function returns 0 silently. Tracked as a
145 + same-failure follow-up in SOW-0012.
146 +
147 +## 8. vnode special case
148 +
149 +Vnodes do not stream — they are virtual hosts collected directly by the
150 +local agent (e.g., SNMP devices). Their stored `stream.path.array` is empty.
151 +At JSON-emit time, `rrdhost_stream_path_to_json` appends a fresh self
152 +(the collecting agent's identity) so the wire format always shows a vnode
153 +hop into the collector at slot 0.
154 +
155 +For any consumer that synthesizes actors from path entries (e.g., the
156 +topology function's Bug C synthesis), vnodes contribute nothing — their
157 +visits via `rrdhost_stream_path_visit` return zero entries.
158 +
159 +## 9. Topology-function classification logic
160 +
161 +The `topology:streaming` Function classifies each entry in
162 +`rrdhost_root_index` as `parent`, `child`, `vnode`, or `stale`. The intended
163 +rule (project-owner confirmed in SOW-0012):
164 +
165 +1. **Source of truth = `rrdhost_root_index`.** Every actor on the
166 + topology graph corresponds to either an entry in this index OR a
167 + synthesized "remote parent" derived from path data (Bug C synthesis).
168 +2. **For each non-vnode node, slot 0 of its `stream.path.array` is the
169 + origin (a child role in this chain) and slots 1+ are upstream parents.**
170 +3. **A node X is classified `parent` if X appears at slot 1+ in any
171 + node's path.** Otherwise `child`.
172 +4. **vnodes** are tagged `vnode` directly — they have no upstream chain
173 + visible in their own stored data.
174 +5. **Stale** nodes (`s.ingest.status == RRDHOST_INGEST_STATUS_ARCHIVED`) are
175 + tagged `stale` directly and rendered as a stale link to localhost.
176 +
177 +For the localhost itself, classification reads live state via
178 +`rrdhost_status()` (count of non-virtual children with `s.ingest.type ==
179 +CHILD` and `s.ingest.status` ∈ `{ONLINE, REPLICATING}`). This was Bug A in
180 +SOW-0012 — the path-based check was unreliable on apex agents because
181 +storage hadn't converged yet.
182 +
183 +For the cross-agent merge (Cloud combines topology responses from multiple
184 +parents into a single graph), the agent's response carries a stable
185 +`agent_id` at the top level. The merge layer maps it to the canonical
186 +actor_id form (`netdata-machine-guid:<agent_id>`) and uses that to identify
187 +which actor in `actors[]` is the response's "self" — that response is the
188 +authoritative source for that actor's attributes.
189 +
190 +For the bug history, decisions, and design rationale, see
191 +[`SOW-0012`](../../.agents/sow/done/SOW-0012-20260505-streaming-topology-classification-bugs.md)
192 +once it lands in `done/`. While the SOW is in progress it lives at
193 +`.agents/sow/current/`.
src/streaming/stream-path.c
+24
@@ -258,6 +258,30 @@ uint16_t rrdhost_stream_path_get_host_ids(struct rrdhost *host, uint16_t from, N
258 return count;
259 }
260
261 +uint16_t rrdhost_stream_path_visit(struct rrdhost *host, uint16_t from,
262 + stream_path_visit_cb cb, void *userdata) {
263 + if(!host || !cb)
264 + return 0;
265 +
266 + uint16_t count = 0;
267 + rw_spinlock_read_lock(&host->stream.path.spinlock);
268 + for(uint16_t i = from; i < host->stream.path.used; i++) {
269 + STREAM_PATH *p = &host->stream.path.array[i];
270 + bool keep_going = cb(userdata, i,
271 + p->hostname,
272 + p->host_id, p->node_id, p->claim_id,
273 + p->hops,
274 + p->since, p->first_time_t,
275 + p->start_time_ms, p->shutdown_time_ms,
276 + p->capabilities,
277 + (uint32_t)p->flags);
278 + count++;
279 + if(!keep_going) break;
280 + }
281 + rw_spinlock_read_unlock(&host->stream.path.spinlock);
282 + return count;
283 +}
284 +
285 void stream_path_child_disconnected(RRDHOST *host) {
286 rrdhost_stream_path_clear(host, true);
287 }
src/streaming/stream-path.h
+20
@@ -42,4 +42,24 @@ bool rrdhost_is_host_in_stream_path_before_us(struct rrdhost *host, ND_UUID remo
42 // returns the number of entries copied (up to max)
43 uint16_t rrdhost_stream_path_get_host_ids(struct rrdhost *host, uint16_t from, ND_UUID *host_ids, uint16_t max);
44
45 +// Visitor callback. Called once per stream-path entry from index 'from' onward.
46 +// hostname is a borrowed reference; do NOT retain it after the callback returns.
47 +// flags is the STREAM_PATH_FLAGS bitmap as a uint32_t (treat as opaque).
48 +// Return false from the callback to stop iteration early.
49 +typedef bool (*stream_path_visit_cb)(
50 + void *userdata, uint16_t index,
51 + STRING *hostname,
52 + ND_UUID host_id, ND_UUID node_id, ND_UUID claim_id,
53 + int16_t hops,
54 + time_t since, time_t first_time_t,
55 + uint32_t start_time_ms, uint32_t shutdown_time_ms,
56 + STREAM_CAPABILITIES capabilities,
57 + uint32_t flags);
58 +
59 +// Visit each path entry from 'from' onward. Acquires the path read-spinlock
60 +// for the duration of the iteration; keep the callback fast. Returns the
61 +// number of entries visited.
62 +uint16_t rrdhost_stream_path_visit(struct rrdhost *host, uint16_t from,
63 + stream_path_visit_cb cb, void *userdata);
64 +
65 #endif //NETDATA_STREAM_PATH_H
src/web/api/functions/function-netdata-streaming.c new
+981
@@ -0,0 +1,981 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#include "function-netdata-streaming.h"
4 +
5 +#include <ctype.h>
6 +
7 +#define STREAMING_FUNCTION_UPDATE_EVERY 10
8 +
9 +#define GROUP_BY_COLUMN(name, descr) \
10 + buffer_json_member_add_object(wb, name);\
11 + {\
12 + buffer_json_member_add_string(wb, "name", descr);\
13 + buffer_json_member_add_array(wb, "columns");\
14 + {\
15 + buffer_json_add_array_item_string(wb, name);\
16 + }\
17 + buffer_json_array_close(wb);\
18 + }\
19 + buffer_json_object_close(wb);
20 +
21 +int function_netdata_streaming(BUFFER *wb, const char *function __maybe_unused, BUFFER *payload __maybe_unused, const char *source __maybe_unused) {
22 +
23 + time_t now = now_realtime_sec();
24 +
25 + buffer_flush(wb);
26 + wb->content_type = CT_APPLICATION_JSON;
27 + buffer_json_initialize(wb, "\"", "\"", 0, true, BUFFER_JSON_OPTIONS_DEFAULT);
28 +
29 + buffer_json_member_add_string(wb, "hostname", rrdhost_hostname(localhost));
30 + buffer_json_member_add_uint64(wb, "status", HTTP_RESP_OK);
31 + buffer_json_member_add_string(wb, "type", "table");
32 + buffer_json_member_add_time_t(wb, "update_every", STREAMING_FUNCTION_UPDATE_EVERY);
33 + buffer_json_member_add_boolean(wb, "has_history", false);
34 + buffer_json_member_add_string(wb, "help", RRDFUNCTIONS_STREAMING_HELP);
35 + buffer_json_member_add_array(wb, "data");
36 +
37 + size_t max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_MAX] = { 0 };
38 + size_t max_db_metrics = 0, max_db_instances = 0, max_db_contexts = 0;
39 + size_t max_collection_replication_instances = 0, max_streaming_replication_instances = 0;
40 + size_t max_ml_anomalous = 0, max_ml_normal = 0, max_ml_trained = 0, max_ml_pending = 0, max_ml_silenced = 0;
41 +
42 + time_t
43 + max_db_duration = 0,
44 + max_db_from = 0,
45 + max_db_to = 0,
46 + max_in_age = 0,
47 + max_out_age = 0,
48 + max_out_attempt_age = 0;
49 +
50 + uint64_t
51 + max_in_since = 0,
52 + max_out_since = 0,
53 + max_out_attempt_since = 0;
54 +
55 + int16_t
56 + max_in_hops = -1,
57 + max_out_hops = -1;
58 +
59 + int
60 + max_in_local_port = 0,
61 + max_in_remote_port = 0,
62 + max_out_local_port = 0,
63 + max_out_remote_port = 0;
64 +
65 + uint32_t
66 + max_in_connections = 0,
67 + max_out_connections = 0;
68 +
69 + {
70 + RRDHOST *host;
71 + dfe_start_read(rrdhost_root_index, host) {
72 + RRDHOST_STATUS s;
73 + rrdhost_status(host, now, &s, RRDHOST_STATUS_ALL);
74 + buffer_json_add_array_item_array(wb);
75 +
76 + if(s.db.metrics > max_db_metrics)
77 + max_db_metrics = s.db.metrics;
78 +
79 + if(s.db.instances > max_db_instances)
80 + max_db_instances = s.db.instances;
81 +
82 + if(s.db.contexts > max_db_contexts)
83 + max_db_contexts = s.db.contexts;
84 +
85 + if(s.ingest.replication.instances > max_collection_replication_instances)
86 + max_collection_replication_instances = s.ingest.replication.instances;
87 +
88 + if(s.stream.replication.instances > max_streaming_replication_instances)
89 + max_streaming_replication_instances = s.stream.replication.instances;
90 +
91 + for(int i = 0; i < STREAM_TRAFFIC_TYPE_MAX ;i++) {
92 + if (s.stream.sent_bytes_on_this_connection_per_type[i] >
93 + max_sent_bytes_on_this_connection_per_type[i])
94 + max_sent_bytes_on_this_connection_per_type[i] =
95 + s.stream.sent_bytes_on_this_connection_per_type[i];
96 + }
97 +
98 + // Node
99 + buffer_json_add_array_item_string(wb, rrdhost_hostname(s.host));
100 +
101 + // rowOptions
102 + buffer_json_add_array_item_object(wb);
103 + {
104 + const char *severity = NULL; // normal, debug, notice, warning, critical
105 + if(!rrdhost_option_check(host, RRDHOST_OPTION_EPHEMERAL_HOST)) {
106 + switch(s.ingest.status) {
107 + case RRDHOST_INGEST_STATUS_OFFLINE:
108 + case RRDHOST_INGEST_STATUS_ARCHIVED:
109 + severity = "critical";
110 + break;
111 +
112 + default:
113 + case RRDHOST_INGEST_STATUS_INITIALIZING:
114 + case RRDHOST_INGEST_STATUS_ONLINE:
115 + case RRDHOST_INGEST_STATUS_REPLICATING:
116 + break;
117 + }
118 +
119 + switch(s.stream.status) {
120 + case RRDHOST_STREAM_STATUS_OFFLINE:
121 + if(!severity && s.stream.reason != STREAM_HANDSHAKE_SP_NO_DESTINATION)
122 + severity = "warning";
123 + break;
124 +
125 + default:
126 + case RRDHOST_STREAM_STATUS_REPLICATING:
127 + case RRDHOST_STREAM_STATUS_ONLINE:
128 + break;
129 + }
130 + }
131 + buffer_json_member_add_string(wb, "severity", severity ? severity : "normal");
132 + }
133 + buffer_json_object_close(wb); // rowOptions
134 +
135 + // Ephemerality
136 + buffer_json_add_array_item_string(wb, rrdhost_option_check(s.host, RRDHOST_OPTION_EPHEMERAL_HOST) ? "ephemeral" : "permanent");
137 +
138 + // AgentName and AgentVersion
139 + buffer_json_add_array_item_string(wb, rrdhost_program_name(host));
140 + buffer_json_add_array_item_string(wb, rrdhost_program_version(host));
141 +
142 + // System Info
143 + rrdhost_system_info_to_streaming_function_array(wb, s.host->system_info);
144 +
145 + // retention
146 + buffer_json_add_array_item_uint64(wb, s.db.first_time_s * MSEC_PER_SEC); // dbFrom
147 + if(s.db.first_time_s > max_db_from) max_db_from = s.db.first_time_s;
148 +
149 + buffer_json_add_array_item_uint64(wb, s.db.last_time_s * MSEC_PER_SEC); // dbTo
150 + if(s.db.last_time_s > max_db_to) max_db_to = s.db.last_time_s;
151 +
152 + if(s.db.first_time_s && s.db.last_time_s && s.db.last_time_s > s.db.first_time_s) {
153 + time_t db_duration = s.db.last_time_s - s.db.first_time_s;
154 + buffer_json_add_array_item_uint64(wb, db_duration); // dbDuration
155 + if(db_duration > max_db_duration) max_db_duration = db_duration;
156 + }
157 + else
158 + buffer_json_add_array_item_string(wb, NULL); // dbDuration
159 +
160 + buffer_json_add_array_item_uint64(wb, s.db.metrics); // dbMetrics
161 + buffer_json_add_array_item_uint64(wb, s.db.instances); // dbInstances
162 + buffer_json_add_array_item_uint64(wb, s.db.contexts); // dbContexts
163 +
164 + // statuses
165 + buffer_json_add_array_item_string(wb, rrdhost_ingest_status_to_string(s.ingest.status)); // InStatus
166 + buffer_json_add_array_item_string(wb, rrdhost_streaming_status_to_string(s.stream.status)); // OutStatus
167 + buffer_json_add_array_item_string(wb, rrdhost_ml_status_to_string(s.ml.status)); // MLStatus
168 +
169 + // collection
170 +
171 + // InConnections
172 + buffer_json_add_array_item_uint64(wb, s.host->stream.rcv.status.connections);
173 + if(s.host->stream.rcv.status.connections > max_in_connections)
174 + max_in_connections = s.host->stream.rcv.status.connections;
175 +
176 + if(s.ingest.since) {
177 + uint64_t in_since = s.ingest.since * MSEC_PER_SEC;
178 + buffer_json_add_array_item_uint64(wb, in_since); // InSince
179 + if(in_since > max_in_since) max_in_since = in_since;
180 +
181 + time_t in_age = s.now - s.ingest.since;
182 + buffer_json_add_array_item_time_t(wb, in_age); // InAge
183 + if(in_age > max_in_age) max_in_age = in_age;
184 + }
185 + else {
186 + buffer_json_add_array_item_string(wb, NULL); // InSince
187 + buffer_json_add_array_item_string(wb, NULL); // InAge
188 + }
189 +
190 + // InReason
191 + if(s.ingest.type == RRDHOST_INGEST_TYPE_LOCALHOST)
192 + buffer_json_add_array_item_string(wb, "LOCALHOST");
193 + else if(s.ingest.type == RRDHOST_INGEST_TYPE_VIRTUAL)
194 + buffer_json_add_array_item_string(wb, "VIRTUAL NODE");
195 + else
196 + buffer_json_add_array_item_string(wb, stream_handshake_error_to_string(s.ingest.reason));
197 +
198 + buffer_json_add_array_item_int64(wb, s.ingest.hops); // InHops
199 + if(s.ingest.hops > max_in_hops) max_in_hops = s.ingest.hops;
200 +
201 + buffer_json_add_array_item_double(wb, s.ingest.replication.completion); // InReplCompletion
202 + buffer_json_add_array_item_uint64(wb, s.ingest.replication.instances); // InReplInstances
203 + buffer_json_add_array_item_string(wb, s.ingest.type == RRDHOST_INGEST_TYPE_LOCALHOST || s.ingest.type == RRDHOST_INGEST_TYPE_VIRTUAL ? "localhost" : s.ingest.peers.local.ip); // InLocalIP
204 +
205 + buffer_json_add_array_item_uint64(wb, s.ingest.peers.local.port); // InLocalPort
206 + if(s.ingest.peers.local.port > max_in_local_port) max_in_local_port = s.ingest.peers.local.port;
207 +
208 + buffer_json_add_array_item_string(wb, s.ingest.peers.peer.ip); // InRemoteIP
209 + buffer_json_add_array_item_uint64(wb, s.ingest.peers.peer.port); // InRemotePort
210 + if(s.ingest.peers.peer.port > max_in_remote_port) max_in_remote_port = s.ingest.peers.peer.port;
211 +
212 + buffer_json_add_array_item_string(wb, s.ingest.ssl ? "SSL" : "PLAIN"); // InSSL
213 + stream_capabilities_to_json_array(wb, s.ingest.capabilities, NULL); // InCapabilities
214 +
215 + buffer_json_add_array_item_uint64(wb, s.ingest.collected.metrics); // CollectedMetrics
216 + buffer_json_add_array_item_uint64(wb, s.ingest.collected.instances); // CollectedInstances
217 + buffer_json_add_array_item_uint64(wb, s.ingest.collected.contexts); // CollectedContexts
218 +
219 + // streaming
220 +
221 + // OutConnections
222 + buffer_json_add_array_item_uint64(wb, s.host->stream.snd.status.connections);
223 + if(s.host->stream.snd.status.connections > max_out_connections)
224 + max_out_connections = s.host->stream.snd.status.connections;
225 +
226 + if(s.stream.since) {
227 + uint64_t out_since = s.stream.since * MSEC_PER_SEC;
228 + buffer_json_add_array_item_uint64(wb, out_since); // OutSince
229 + if(out_since > max_out_since) max_out_since = out_since;
230 +
231 + time_t out_age = s.now - s.stream.since;
232 + buffer_json_add_array_item_time_t(wb, out_age); // OutAge
233 + if(out_age > max_out_age) max_out_age = out_age;
234 + }
235 + else {
236 + buffer_json_add_array_item_string(wb, NULL); // OutSince
237 + buffer_json_add_array_item_string(wb, NULL); // OutAge
238 + }
239 + buffer_json_add_array_item_string(wb, stream_handshake_error_to_string(s.stream.reason)); // OutReason
240 +
241 + buffer_json_add_array_item_int64(wb, s.stream.hops); // OutHops
242 + if(s.stream.hops > max_out_hops) max_out_hops = s.stream.hops;
243 +
244 + buffer_json_add_array_item_double(wb, s.stream.replication.completion); // OutReplCompletion
245 + buffer_json_add_array_item_uint64(wb, s.stream.replication.instances); // OutReplInstances
246 + buffer_json_add_array_item_string(wb, s.stream.peers.local.ip); // OutLocalIP
247 + buffer_json_add_array_item_uint64(wb, s.stream.peers.local.port); // OutLocalPort
248 + if(s.stream.peers.local.port > max_out_local_port) max_out_local_port = s.stream.peers.local.port;
249 +
250 + buffer_json_add_array_item_string(wb, s.stream.peers.peer.ip); // OutRemoteIP
251 + buffer_json_add_array_item_uint64(wb, s.stream.peers.peer.port); // OutRemotePort
252 + if(s.stream.peers.peer.port > max_out_remote_port) max_out_remote_port = s.stream.peers.peer.port;
253 +
254 + buffer_json_add_array_item_string(wb, s.stream.ssl ? "SSL" : "PLAIN"); // OutSSL
255 + buffer_json_add_array_item_string(wb, s.stream.compression ? "COMPRESSED" : "UNCOMPRESSED"); // OutCompression
256 + stream_capabilities_to_json_array(wb, s.stream.capabilities, NULL); // OutCapabilities
257 + buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_DATA]);
258 + buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_METADATA]);
259 + buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_REPLICATION]);
260 + buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_FUNCTIONS]);
261 +
262 + buffer_json_add_array_item_array(wb); // OutAttemptHandshake
263 + usec_t last_attempt = stream_parent_handshake_error_to_json(wb, host);
264 + buffer_json_array_close(wb); // // OutAttemptHandshake
265 +
266 + if(!last_attempt) {
267 + buffer_json_add_array_item_string(wb, NULL); // OutAttemptSince
268 + buffer_json_add_array_item_string(wb, NULL); // OutAttemptAge
269 + }
270 + else {
271 + uint64_t out_attempt_since = last_attempt / USEC_PER_MS;
272 + buffer_json_add_array_item_uint64(wb, out_attempt_since); // OutAttemptSince
273 + if(out_attempt_since > max_out_attempt_since) max_out_attempt_since = out_attempt_since;
274 +
275 + time_t out_attempt_age = s.now - (time_t)(last_attempt / USEC_PER_SEC);
276 + buffer_json_add_array_item_time_t(wb, out_attempt_age); // OutAttemptAge
277 + if(out_attempt_age > max_out_attempt_age) max_out_attempt_age = out_attempt_age;
278 + }
279 +
280 + // ML
281 + if(s.ml.status == RRDHOST_ML_STATUS_RUNNING) {
282 + buffer_json_add_array_item_uint64(wb, s.ml.metrics.anomalous); // MlAnomalous
283 + buffer_json_add_array_item_uint64(wb, s.ml.metrics.normal); // MlNormal
284 + buffer_json_add_array_item_uint64(wb, s.ml.metrics.trained); // MlTrained
285 + buffer_json_add_array_item_uint64(wb, s.ml.metrics.pending); // MlPending
286 + buffer_json_add_array_item_uint64(wb, s.ml.metrics.silenced); // MlSilenced
287 +
288 + if(s.ml.metrics.anomalous > max_ml_anomalous)
289 + max_ml_anomalous = s.ml.metrics.anomalous;
290 +
291 + if(s.ml.metrics.normal > max_ml_normal)
292 + max_ml_normal = s.ml.metrics.normal;
293 +
294 + if(s.ml.metrics.trained > max_ml_trained)
295 + max_ml_trained = s.ml.metrics.trained;
296 +
297 + if(s.ml.metrics.pending > max_ml_pending)
298 + max_ml_pending = s.ml.metrics.pending;
299 +
300 + if(s.ml.metrics.silenced > max_ml_silenced)
301 + max_ml_silenced = s.ml.metrics.silenced;
302 +
303 + }
304 + else {
305 + buffer_json_add_array_item_string(wb, NULL); // MlAnomalous
306 + buffer_json_add_array_item_string(wb, NULL); // MlNormal
307 + buffer_json_add_array_item_string(wb, NULL); // MlTrained
308 + buffer_json_add_array_item_string(wb, NULL); // MlPending
309 + buffer_json_add_array_item_string(wb, NULL); // MlSilenced
310 + }
311 +
312 + // close
313 + buffer_json_array_close(wb);
314 + }
315 + dfe_done(host);
316 + }
317 + buffer_json_array_close(wb); // data
318 + buffer_json_member_add_object(wb, "columns");
319 + {
320 + size_t field_id = 0;
321 +
322 + // Node
323 + buffer_rrdf_table_add_field(wb, field_id++, "Node", "Node's Hostname",
324 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
325 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
326 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
327 + RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_UNIQUE_KEY | RRDF_FIELD_OPTS_STICKY,
328 + NULL);
329 +
330 + buffer_rrdf_table_add_field(wb, field_id++, "rowOptions", "rowOptions",
331 + RRDF_FIELD_TYPE_NONE, RRDR_FIELD_VISUAL_ROW_OPTIONS, RRDF_FIELD_TRANSFORM_NONE,
332 + 0, NULL, NAN, RRDF_FIELD_SORT_FIXED, NULL,
333 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE, RRDF_FIELD_OPTS_DUMMY,
334 + NULL);
335 +
336 + buffer_rrdf_table_add_field(wb, field_id++, "Ephemerality", "The type of ephemerality for the node",
337 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
338 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
339 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
340 + RRDF_FIELD_OPTS_VISIBLE,
341 + NULL);
342 +
343 + buffer_rrdf_table_add_field(wb, field_id++, "AgentName", "The name of the Netdata agent",
344 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
345 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
346 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
347 + RRDF_FIELD_OPTS_NONE,
348 + NULL);
349 +
350 + buffer_rrdf_table_add_field(wb, field_id++, "AgentVersion", "The version of the Netdata agent",
351 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
352 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
353 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
354 + RRDF_FIELD_OPTS_NONE,
355 + NULL);
356 +
357 + buffer_rrdf_table_add_field(wb, field_id++, "OSName", "The name of the host's operating system",
358 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
359 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
360 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
361 + RRDF_FIELD_OPTS_NONE,
362 + NULL);
363 +
364 + buffer_rrdf_table_add_field(wb, field_id++, "OSId", "The identifier of the host's operating system",
365 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
366 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
367 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
368 + RRDF_FIELD_OPTS_NONE,
369 + NULL);
370 +
371 + buffer_rrdf_table_add_field(wb, field_id++, "OSIdLike", "The ID-like string for the host's OS",
372 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
373 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
374 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
375 + RRDF_FIELD_OPTS_NONE,
376 + NULL);
377 +
378 + buffer_rrdf_table_add_field(wb, field_id++, "OSVersion", "The version of the host's operating system",
379 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
380 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
381 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
382 + RRDF_FIELD_OPTS_NONE,
383 + NULL);
384 +
385 + buffer_rrdf_table_add_field(wb, field_id++, "OSVersionId", "The version identifier of the host's OS",
386 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
387 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
388 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
389 + RRDF_FIELD_OPTS_NONE,
390 + NULL);
391 +
392 + buffer_rrdf_table_add_field(wb, field_id++, "OSDetection", "Details about host OS detection",
393 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
394 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
395 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
396 + RRDF_FIELD_OPTS_NONE,
397 + NULL);
398 +
399 + buffer_rrdf_table_add_field(wb, field_id++, "CPUCores", "The number of CPU cores in the host",
400 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
401 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
402 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
403 + RRDF_FIELD_OPTS_NONE,
404 + NULL);
405 +
406 + buffer_rrdf_table_add_field(wb, field_id++, "DiskSpace", "The total disk space available on the host",
407 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
408 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
409 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
410 + RRDF_FIELD_OPTS_NONE,
411 + NULL);
412 +
413 + buffer_rrdf_table_add_field(wb, field_id++, "CPUFreq", "The CPU frequency of the host",
414 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
415 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
416 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
417 + RRDF_FIELD_OPTS_NONE,
418 + NULL);
419 +
420 + buffer_rrdf_table_add_field(wb, field_id++, "RAMTotal", "The total RAM available on the host",
421 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
422 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
423 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
424 + RRDF_FIELD_OPTS_NONE,
425 + NULL);
426 +
427 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSName", "The name of the container's operating system",
428 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
429 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
430 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
431 + RRDF_FIELD_OPTS_NONE,
432 + NULL);
433 +
434 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSId", "The identifier of the container's operating system",
435 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
436 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
437 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
438 + RRDF_FIELD_OPTS_NONE,
439 + NULL);
440 +
441 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSIdLike", "The ID-like string for the container's OS",
442 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
443 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
444 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
445 + RRDF_FIELD_OPTS_NONE,
446 + NULL);
447 +
448 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSVersion", "The version of the container's OS",
449 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
450 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
451 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
452 + RRDF_FIELD_OPTS_NONE,
453 + NULL);
454 +
455 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSVersionId", "The version identifier of the container's OS",
456 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
457 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
458 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
459 + RRDF_FIELD_OPTS_NONE,
460 + NULL);
461 +
462 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSDetection", "Details about container OS detection",
463 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
464 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
465 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
466 + RRDF_FIELD_OPTS_NONE,
467 + NULL);
468 +
469 + buffer_rrdf_table_add_field(wb, field_id++, "IsK8sNode", "Whether this node is part of a Kubernetes cluster",
470 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
471 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
472 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
473 + RRDF_FIELD_OPTS_NONE,
474 + NULL);
475 +
476 + buffer_rrdf_table_add_field(wb, field_id++, "KernelName", "The kernel name",
477 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
478 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
479 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
480 + RRDF_FIELD_OPTS_NONE,
481 + NULL);
482 +
483 + buffer_rrdf_table_add_field(wb, field_id++, "KernelVersion", "The kernel version",
484 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
485 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
486 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
487 + RRDF_FIELD_OPTS_NONE,
488 + NULL);
489 +
490 + buffer_rrdf_table_add_field(wb, field_id++, "Architecture", "The system architecture",
491 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
492 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
493 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
494 + RRDF_FIELD_OPTS_NONE,
495 + NULL);
496 +
497 + buffer_rrdf_table_add_field(wb, field_id++, "Virtualization", "The virtualization technology in use",
498 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
499 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
500 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
501 + RRDF_FIELD_OPTS_NONE,
502 + NULL);
503 +
504 + buffer_rrdf_table_add_field(wb, field_id++, "VirtDetection", "Details about virtualization detection",
505 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
506 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
507 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
508 + RRDF_FIELD_OPTS_NONE,
509 + NULL);
510 +
511 + buffer_rrdf_table_add_field(wb, field_id++, "Container", "Container type information",
512 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
513 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
514 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
515 + RRDF_FIELD_OPTS_NONE,
516 + NULL);
517 +
518 + buffer_rrdf_table_add_field(wb, field_id++, "ContainerDetection", "Details about container detection",
519 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
520 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
521 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
522 + RRDF_FIELD_OPTS_NONE,
523 + NULL);
524 +
525 + buffer_rrdf_table_add_field(wb, field_id++, "CloudProviderType", "The type of cloud provider",
526 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
527 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
528 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
529 + RRDF_FIELD_OPTS_NONE,
530 + NULL);
531 +
532 + buffer_rrdf_table_add_field(wb, field_id++, "CloudInstanceType", "The type of cloud instance",
533 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
534 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
535 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
536 + RRDF_FIELD_OPTS_NONE,
537 + NULL);
538 +
539 + buffer_rrdf_table_add_field(wb, field_id++, "CloudInstanceRegion", "The region of the cloud instance",
540 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
541 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
542 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
543 + RRDF_FIELD_OPTS_NONE,
544 + NULL);
545 +
546 + buffer_rrdf_table_add_field(wb, field_id++, "dbFrom", "DB Data Retention From",
547 + RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
548 + 0, NULL, (double)max_db_from * MSEC_PER_SEC, RRDF_FIELD_SORT_ASCENDING, NULL,
549 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_NONE,
550 + RRDF_FIELD_OPTS_NONE, NULL);
551 +
552 + buffer_rrdf_table_add_field(wb, field_id++, "dbTo", "DB Data Retention To",
553 + RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
554 + 0, NULL, (double)max_db_to * MSEC_PER_SEC, RRDF_FIELD_SORT_ASCENDING, NULL,
555 + RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
556 + RRDF_FIELD_OPTS_NONE, NULL);
557 +
558 + buffer_rrdf_table_add_field(wb, field_id++, "dbDuration", "DB Data Retention Duration",
559 + RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
560 + 0, NULL, (double)max_db_duration, RRDF_FIELD_SORT_ASCENDING, NULL,
561 + RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
562 + RRDF_FIELD_OPTS_VISIBLE, NULL);
563 +
564 + buffer_rrdf_table_add_field(wb, field_id++, "dbMetrics", "Time-series Metrics in the DB",
565 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
566 + 0, NULL, (double)max_db_metrics, RRDF_FIELD_SORT_DESCENDING, NULL,
567 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
568 + RRDF_FIELD_OPTS_VISIBLE, NULL);
569 +
570 + buffer_rrdf_table_add_field(wb, field_id++, "dbInstances", "Instances in the DB",
571 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
572 + 0, NULL, (double)max_db_instances, RRDF_FIELD_SORT_DESCENDING, NULL,
573 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
574 + RRDF_FIELD_OPTS_VISIBLE, NULL);
575 +
576 + buffer_rrdf_table_add_field(wb, field_id++, "dbContexts", "Contexts in the DB",
577 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
578 + 0, NULL, (double)max_db_contexts, RRDF_FIELD_SORT_DESCENDING, NULL,
579 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
580 + RRDF_FIELD_OPTS_VISIBLE, NULL);
581 +
582 + // --- statuses ---
583 +
584 + buffer_rrdf_table_add_field(wb, field_id++, "InStatus", "Data Collection Online Status",
585 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
586 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
587 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
588 + RRDF_FIELD_OPTS_VISIBLE, NULL);
589 +
590 +
591 + buffer_rrdf_table_add_field(wb, field_id++, "OutStatus", "Streaming Online Status",
592 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
593 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
594 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
595 + RRDF_FIELD_OPTS_VISIBLE, NULL);
596 +
597 + buffer_rrdf_table_add_field(wb, field_id++, "MlStatus", "ML Status",
598 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
599 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
600 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
601 + RRDF_FIELD_OPTS_VISIBLE, NULL);
602 +
603 + // --- collection ---
604 +
605 + buffer_rrdf_table_add_field(wb, field_id++, "InConnections", "Number of times this child connected",
606 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
607 + 0, NULL, (double)max_in_connections, RRDF_FIELD_SORT_DESCENDING, NULL,
608 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
609 + RRDF_FIELD_OPTS_NONE, NULL);
610 +
611 + buffer_rrdf_table_add_field(wb, field_id++, "InSince", "Last Data Collection Status Change",
612 + RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
613 + 0, NULL, (double)max_in_since, RRDF_FIELD_SORT_DESCENDING, NULL,
614 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_NONE,
615 + RRDF_FIELD_OPTS_NONE, NULL);
616 +
617 + buffer_rrdf_table_add_field(wb, field_id++, "InAge", "Last Data Collection Online Status Change Age",
618 + RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
619 + 0, NULL, (double)max_in_age, RRDF_FIELD_SORT_ASCENDING, NULL,
620 + RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_RANGE,
621 + RRDF_FIELD_OPTS_VISIBLE, NULL);
622 +
623 + buffer_rrdf_table_add_field(wb, field_id++, "InReason", "Data Collection Online Status Reason",
624 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
625 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
626 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
627 + RRDF_FIELD_OPTS_VISIBLE, NULL);
628 +
629 + buffer_rrdf_table_add_field(wb, field_id++, "InHops", "Data Collection Distance Hops from Origin Node",
630 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
631 + 0, NULL, (double)max_in_hops, RRDF_FIELD_SORT_ASCENDING, NULL,
632 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
633 + RRDF_FIELD_OPTS_VISIBLE, NULL);
634 +
635 + buffer_rrdf_table_add_field(wb, field_id++, "InReplCompletion", "Inbound Replication Completion",
636 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_BAR, RRDF_FIELD_TRANSFORM_NUMBER,
637 + 1, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL,
638 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
639 + RRDF_FIELD_OPTS_VISIBLE, NULL);
640 +
641 + buffer_rrdf_table_add_field(wb, field_id++, "InReplInstances", "Inbound Replicating Instances",
642 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
643 + 0, "instances", (double)max_collection_replication_instances, RRDF_FIELD_SORT_DESCENDING,
644 + NULL,
645 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
646 + RRDF_FIELD_OPTS_NONE, NULL);
647 +
648 + buffer_rrdf_table_add_field(wb, field_id++, "InLocalIP", "Inbound Local IP",
649 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
650 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
651 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
652 + RRDF_FIELD_OPTS_NONE, NULL);
653 +
654 + buffer_rrdf_table_add_field(wb, field_id++, "InLocalPort", "Inbound Local Port",
655 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
656 + 0, NULL, (double)max_in_local_port, RRDF_FIELD_SORT_ASCENDING, NULL,
657 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
658 + RRDF_FIELD_OPTS_NONE, NULL);
659 +
660 + buffer_rrdf_table_add_field(wb, field_id++, "InRemoteIP", "Inbound Remote IP",
661 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
662 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
663 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
664 + RRDF_FIELD_OPTS_NONE, NULL);
665 +
666 + buffer_rrdf_table_add_field(wb, field_id++, "InRemotePort", "Inbound Remote Port",
667 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
668 + 0, NULL, (double)max_in_remote_port, RRDF_FIELD_SORT_ASCENDING, NULL,
669 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
670 + RRDF_FIELD_OPTS_NONE, NULL);
671 +
672 + buffer_rrdf_table_add_field(wb, field_id++, "InSSL", "Inbound SSL Connection",
673 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
674 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
675 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
676 + RRDF_FIELD_OPTS_NONE, NULL);
677 +
678 + buffer_rrdf_table_add_field(wb, field_id++, "InCapabilities", "Inbound Connection Capabilities",
679 + RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
680 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
681 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
682 + RRDF_FIELD_OPTS_NONE, NULL);
683 +
684 + buffer_rrdf_table_add_field(wb, field_id++, "CollectedMetrics", "Time-series Metrics Currently Collected",
685 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
686 + 0, NULL, (double)max_db_metrics, RRDF_FIELD_SORT_DESCENDING, NULL,
687 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
688 + RRDF_FIELD_OPTS_VISIBLE, NULL);
689 +
690 + buffer_rrdf_table_add_field(wb, field_id++, "CollectedInstances", "Instances Currently Collected",
691 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
692 + 0, NULL, (double)max_db_instances, RRDF_FIELD_SORT_DESCENDING, NULL,
693 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
694 + RRDF_FIELD_OPTS_VISIBLE, NULL);
695 +
696 + buffer_rrdf_table_add_field(wb, field_id++, "CollectedContexts", "Contexts Currently Collected",
697 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
698 + 0, NULL, (double)max_db_contexts, RRDF_FIELD_SORT_DESCENDING, NULL,
699 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
700 + RRDF_FIELD_OPTS_VISIBLE, NULL);
701 +
702 + // --- streaming ---
703 +
704 + buffer_rrdf_table_add_field(wb, field_id++, "OutConnections", "Number of times connected to a parent",
705 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
706 + 0, NULL, (double)max_out_connections, RRDF_FIELD_SORT_DESCENDING, NULL,
707 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
708 + RRDF_FIELD_OPTS_NONE, NULL);
709 +
710 + buffer_rrdf_table_add_field(wb, field_id++, "OutSince", "Last Streaming Status Change",
711 + RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
712 + 0, NULL, (double)max_out_since, RRDF_FIELD_SORT_DESCENDING, NULL,
713 + RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
714 + RRDF_FIELD_OPTS_NONE, NULL);
715 +
716 + buffer_rrdf_table_add_field(wb, field_id++, "OutAge", "Last Streaming Status Change Age",
717 + RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
718 + 0, NULL, (double)max_out_age, RRDF_FIELD_SORT_ASCENDING, NULL,
719 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
720 + RRDF_FIELD_OPTS_VISIBLE, NULL);
721 +
722 + buffer_rrdf_table_add_field(wb, field_id++, "OutReason", "Streaming Status Reason",
723 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
724 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
725 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
726 + RRDF_FIELD_OPTS_VISIBLE, NULL);
727 +
728 + buffer_rrdf_table_add_field(wb, field_id++, "OutHops", "Streaming Distance Hops from Origin Node",
729 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
730 + 0, NULL, (double)max_out_hops, RRDF_FIELD_SORT_ASCENDING, NULL,
731 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
732 + RRDF_FIELD_OPTS_VISIBLE, NULL);
733 +
734 + buffer_rrdf_table_add_field(wb, field_id++, "OutReplCompletion", "Outbound Replication Completion",
735 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_BAR, RRDF_FIELD_TRANSFORM_NUMBER,
736 + 1, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL,
737 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
738 + RRDF_FIELD_OPTS_VISIBLE, NULL);
739 +
740 + buffer_rrdf_table_add_field(wb, field_id++, "OutReplInstances", "Outbound Replicating Instances",
741 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
742 + 0, "instances", (double)max_streaming_replication_instances, RRDF_FIELD_SORT_DESCENDING,
743 + NULL,
744 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
745 + RRDF_FIELD_OPTS_NONE, NULL);
746 +
747 + buffer_rrdf_table_add_field(wb, field_id++, "OutLocalIP", "Outbound Local IP",
748 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
749 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
750 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
751 + RRDF_FIELD_OPTS_NONE, NULL);
752 +
753 + buffer_rrdf_table_add_field(wb, field_id++, "OutLocalPort", "Outbound Local Port",
754 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
755 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
756 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
757 + RRDF_FIELD_OPTS_NONE, NULL);
758 +
759 + buffer_rrdf_table_add_field(wb, field_id++, "OutRemoteIP", "Outbound Remote IP",
760 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
761 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
762 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
763 + RRDF_FIELD_OPTS_NONE, NULL);
764 +
765 + buffer_rrdf_table_add_field(wb, field_id++, "OutRemotePort", "Outbound Remote Port",
766 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
767 + 0, NULL, (double)max_out_remote_port, RRDF_FIELD_SORT_ASCENDING, NULL,
768 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
769 + RRDF_FIELD_OPTS_NONE, NULL);
770 +
771 + buffer_rrdf_table_add_field(wb, field_id++, "OutSSL", "Outbound SSL Connection",
772 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
773 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
774 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
775 + RRDF_FIELD_OPTS_NONE, NULL);
776 +
777 + buffer_rrdf_table_add_field(wb, field_id++, "OutCompression", "Outbound Compressed Connection",
778 + RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
779 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
780 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
781 + RRDF_FIELD_OPTS_NONE, NULL);
782 +
783 + buffer_rrdf_table_add_field(wb, field_id++, "OutCapabilities", "Outbound Connection Capabilities",
784 + RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
785 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
786 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
787 + RRDF_FIELD_OPTS_NONE, NULL);
788 +
789 + buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficData", "Outbound Metric Data Traffic",
790 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
791 + 0, "bytes", (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_DATA],
792 + RRDF_FIELD_SORT_DESCENDING, NULL,
793 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
794 + RRDF_FIELD_OPTS_NONE, NULL);
795 +
796 + buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficMetadata", "Outbound Metric Metadata Traffic",
797 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
798 + 0, "bytes",
799 + (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_METADATA],
800 + RRDF_FIELD_SORT_DESCENDING, NULL,
801 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
802 + RRDF_FIELD_OPTS_NONE, NULL);
803 +
804 + buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficReplication", "Outbound Metric Replication Traffic",
805 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
806 + 0, "bytes",
807 + (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_REPLICATION],
808 + RRDF_FIELD_SORT_DESCENDING, NULL,
809 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
810 + RRDF_FIELD_OPTS_NONE, NULL);
811 +
812 + buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficFunctions", "Outbound Metric Functions Traffic",
813 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
814 + 0, "bytes",
815 + (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_FUNCTIONS],
816 + RRDF_FIELD_SORT_DESCENDING, NULL,
817 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
818 + RRDF_FIELD_OPTS_NONE, NULL);
819 +
820 + buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptHandshake",
821 + "Outbound Connection Attempt Handshake Status",
822 + RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
823 + 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
824 + RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
825 + RRDF_FIELD_OPTS_NONE, NULL);
826 +
827 + buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptSince",
828 + "Last Outbound Connection Attempt Status Change Time",
829 + RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
830 + 0, NULL, (double)max_out_attempt_since, RRDF_FIELD_SORT_DESCENDING, NULL,
831 + RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
832 + RRDF_FIELD_OPTS_NONE, NULL);
833 +
834 + buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptAge",
835 + "Last Outbound Connection Attempt Status Change Age",
836 + RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
837 + 0, NULL, (double)max_out_attempt_age, RRDF_FIELD_SORT_ASCENDING, NULL,
838 + RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
839 + RRDF_FIELD_OPTS_VISIBLE, NULL);
840 +
841 + // --- ML ---
842 +
843 + buffer_rrdf_table_add_field(wb, field_id++, "MlAnomalous", "Number of Anomalous Metrics",
844 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
845 + 0, "metrics",
846 + (double)max_ml_anomalous,
847 + RRDF_FIELD_SORT_DESCENDING, NULL,
848 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
849 + RRDF_FIELD_OPTS_NONE, NULL);
850 +
851 + buffer_rrdf_table_add_field(wb, field_id++, "MlNormal", "Number of Not Anomalous Metrics",
852 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
853 + 0, "metrics",
854 + (double)max_ml_normal,
855 + RRDF_FIELD_SORT_DESCENDING, NULL,
856 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
857 + RRDF_FIELD_OPTS_NONE, NULL);
858 +
859 + buffer_rrdf_table_add_field(wb, field_id++, "MlTrained", "Number of Trained Metrics",
860 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
861 + 0, "metrics",
862 + (double)max_ml_trained,
863 + RRDF_FIELD_SORT_DESCENDING, NULL,
864 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
865 + RRDF_FIELD_OPTS_NONE, NULL);
866 +
867 + buffer_rrdf_table_add_field(wb, field_id++, "MlPending", "Number of Pending Metrics",
868 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
869 + 0, "metrics",
870 + (double)max_ml_pending,
871 + RRDF_FIELD_SORT_DESCENDING, NULL,
872 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
873 + RRDF_FIELD_OPTS_NONE, NULL);
874 +
875 + buffer_rrdf_table_add_field(wb, field_id++, "MlSilenced", "Number of Silenced Metrics",
876 + RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
877 + 0, "metrics",
878 + (double)max_ml_silenced,
879 + RRDF_FIELD_SORT_DESCENDING, NULL,
880 + RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
881 + RRDF_FIELD_OPTS_NONE, NULL);
882 + }
883 + buffer_json_object_close(wb); // columns
884 + buffer_json_member_add_string(wb, "default_sort_column", "Node");
885 + buffer_json_member_add_object(wb, "charts");
886 + {
887 + // Data Collection Age chart
888 + buffer_json_member_add_object(wb, "InAge");
889 + {
890 + buffer_json_member_add_string(wb, "name", "Data Collection Age");
891 + buffer_json_member_add_string(wb, "type", "stacked-bar");
892 + buffer_json_member_add_array(wb, "columns");
893 + {
894 + buffer_json_add_array_item_string(wb, "InAge");
895 + }
896 + buffer_json_array_close(wb);
897 + }
898 + buffer_json_object_close(wb);
899 +
900 + // Streaming Age chart
901 + buffer_json_member_add_object(wb, "OutAge");
902 + {
903 + buffer_json_member_add_string(wb, "name", "Streaming Age");
904 + buffer_json_member_add_string(wb, "type", "stacked-bar");
905 + buffer_json_member_add_array(wb, "columns");
906 + {
907 + buffer_json_add_array_item_string(wb, "OutAge");
908 + }
909 + buffer_json_array_close(wb);
910 + }
911 + buffer_json_object_close(wb);
912 +
913 + // DB Duration
914 + buffer_json_member_add_object(wb, "dbDuration");
915 + {
916 + buffer_json_member_add_string(wb, "name", "Retention Duration");
917 + buffer_json_member_add_string(wb, "type", "stacked-bar");
918 + buffer_json_member_add_array(wb, "columns");
919 + {
920 + buffer_json_add_array_item_string(wb, "dbDuration");
921 + }
922 + buffer_json_array_close(wb);
923 + }
924 + buffer_json_object_close(wb);
925 + }
926 + buffer_json_object_close(wb); // charts
927 +
928 + buffer_json_member_add_array(wb, "default_charts");
929 + {
930 + buffer_json_add_array_item_array(wb);
931 + buffer_json_add_array_item_string(wb, "InAge");
932 + buffer_json_add_array_item_string(wb, "Node");
933 + buffer_json_array_close(wb);
934 +
935 + buffer_json_add_array_item_array(wb);
936 + buffer_json_add_array_item_string(wb, "OutAge");
937 + buffer_json_add_array_item_string(wb, "Node");
938 + buffer_json_array_close(wb);
939 + }
940 + buffer_json_array_close(wb);
941 +
942 + buffer_json_member_add_object(wb, "group_by");
943 + {
944 + GROUP_BY_COLUMN("OSName", "O/S Name");
945 + GROUP_BY_COLUMN("OSId", "O/S ID");
946 + GROUP_BY_COLUMN("OSIdLike", "O/S ID Like");
947 + GROUP_BY_COLUMN("OSVersion", "O/S Version");
948 + GROUP_BY_COLUMN("OSVersionId", "O/S Version ID");
949 + GROUP_BY_COLUMN("OSDetection", "O/S Detection");
950 + GROUP_BY_COLUMN("CPUCores", "CPU Cores");
951 + GROUP_BY_COLUMN("ContainerOSName", "Container O/S Name");
952 + GROUP_BY_COLUMN("ContainerOSId", "Container O/S ID");
953 + GROUP_BY_COLUMN("ContainerOSIdLike", "Container O/S ID Like");
954 + GROUP_BY_COLUMN("ContainerOSVersion", "Container O/S Version");
955 + GROUP_BY_COLUMN("ContainerOSVersionId", "Container O/S Version ID");
956 + GROUP_BY_COLUMN("ContainerOSDetection", "Container O/S Detection");
957 + GROUP_BY_COLUMN("IsK8sNode", "Kubernetes Nodes");
958 + GROUP_BY_COLUMN("KernelName", "Kernel Name");
959 + GROUP_BY_COLUMN("KernelVersion", "Kernel Version");
960 + GROUP_BY_COLUMN("Architecture", "Architecture");
961 + GROUP_BY_COLUMN("Virtualization", "Virtualization Technology");
962 + GROUP_BY_COLUMN("VirtDetection", "Virtualization Detection");
963 + GROUP_BY_COLUMN("Container", "Container");
964 + GROUP_BY_COLUMN("ContainerDetection", "Container Detection");
965 + GROUP_BY_COLUMN("CloudProviderType", "Cloud Provider Type");
966 + GROUP_BY_COLUMN("CloudInstanceType", "Cloud Instance Type");
967 + GROUP_BY_COLUMN("CloudInstanceRegion", "Cloud Instance Region");
968 +
969 + GROUP_BY_COLUMN("InStatus", "Collection Status");
970 + GROUP_BY_COLUMN("OutStatus", "Streaming Status");
971 + GROUP_BY_COLUMN("MlStatus", "ML Status");
972 + GROUP_BY_COLUMN("InRemoteIP", "Inbound IP");
973 + GROUP_BY_COLUMN("OutRemoteIP", "Outbound IP");
974 + }
975 + buffer_json_object_close(wb); // group_by
976 +
977 + buffer_json_member_add_time_t(wb, "expires", now_realtime_sec() + STREAMING_FUNCTION_UPDATE_EVERY);
978 + buffer_json_finalize(wb);
979 +
980 + return HTTP_RESP_OK;
981 +}
src/web/api/functions/function-netdata-streaming.h new
+12
@@ -0,0 +1,12 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#ifndef NETDATA_FUNCTION_NETDATA_STREAMING_H
4 +#define NETDATA_FUNCTION_NETDATA_STREAMING_H
5 +
6 +#include "database/rrd.h"
7 +
8 +#define RRDFUNCTIONS_STREAMING_HELP "Shows real-time streaming connections and replication status between parent and child nodes, including connection health, data flow metrics, and ML status."
9 +
10 +int function_netdata_streaming(BUFFER *wb, const char *function, BUFFER *payload, const char *source);
11 +
12 +#endif //NETDATA_FUNCTION_NETDATA_STREAMING_H
src/web/api/functions/function-streaming.h deleted
-14
@@ -1,14 +0,0 @@
1 -// SPDX-License-Identifier: GPL-3.0-or-later
2 -
3 -#ifndef NETDATA_FUNCTION_STREAMING_H
4 -#define NETDATA_FUNCTION_STREAMING_H
5 -
6 -#include "database/rrd.h"
7 -
8 -#define RRDFUNCTIONS_STREAMING_HELP "Shows real-time streaming connections and replication status between parent and child nodes, including connection health, data flow metrics, and ML status."
9 -#define RRDFUNCTIONS_STREAMING_TOPOLOGY_HELP "Shows streaming topology relations across Netdata agents, including directional parent/child links and transport metadata."
10 -
11 -int function_streaming(BUFFER *wb, const char *function, BUFFER *payload, const char *source);
12 -int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payload, const char *source);
13 -
14 -#endif //NETDATA_FUNCTION_STREAMING_H
src/web/api/functions/function-topology-streaming.c renamed
+430 -1079
@@ -1,22 +1,9 @@
1 // SPDX-License-Identifier: GPL-3.0-or-later
2
3 -#include "function-streaming.h"
4 -
5 -#include <ctype.h>
3 +#include "function-topology-streaming.h"
4
5 #define STREAMING_FUNCTION_UPDATE_EVERY 10
6
9 -#define GROUP_BY_COLUMN(name, descr) \
10 - buffer_json_member_add_object(wb, name);\
11 - {\
12 - buffer_json_member_add_string(wb, "name", descr);\
13 - buffer_json_member_add_array(wb, "columns");\
14 - {\
15 - buffer_json_add_array_item_string(wb, name);\
16 - }\
17 - buffer_json_array_close(wb);\
18 - }\
19 - buffer_json_object_close(wb);
7
8 static bool streaming_topology_host_guid(RRDHOST *host, char *dst, size_t dst_size);
9 static bool streaming_topology_uuid_guid(ND_UUID host_id, char *dst, size_t dst_size);
@@ -45,11 +32,8 @@ struct streaming_topology_descendant_list {
32 size_t size;
33 };
34
48 -struct streaming_topology_filters {
35 +struct streaming_topology_options {
36 bool info_only;
50 - const char *node_type;
51 - const char *ingest_status;
52 - const char *stream_status;
37 char *function_copy;
38 };
39
@@ -239,34 +223,31 @@ static uint16_t streaming_topology_get_path_ids(RRDHOST *host, uint16_t from, ND
223 }
224 n = filtered_n;
225
242 - // append localhost if not found (same as rrdhost_stream_path_to_json)
243 - if(!found_localhost && n < max && n > 0)
226 + // append localhost only when callers want the full path (from == 0). The
227 + // self-append mirrors rrdhost_stream_path_to_json's emit-time semantics.
228 + // For from > 0 (e.g. parent counting that asks for upstream-only entries)
229 + // appending self would falsely count the host as its own parent.
230 + if(from == 0 && !found_localhost && n < max && n > 0)
231 host_ids[n++] = localhost->host_id;
232
233 return n;
234 }
235
249 -static void streaming_topology_parse_filters(const char *function, struct streaming_topology_filters *filters) {
250 - if(!filters)
236 +static void streaming_topology_parse_options(const char *function, struct streaming_topology_options *options) {
237 + if(!options)
238 return;
239
253 - *filters = (struct streaming_topology_filters){ 0 };
240 + *options = (struct streaming_topology_options){ 0 };
241 if(!function || !*function)
242 return;
243
257 - filters->function_copy = strdupz(function);
244 + options->function_copy = strdupz(function);
245 char *words[1024];
259 - size_t num_words = quoted_strings_splitter_whitespace(filters->function_copy, words, 1024);
246 + size_t num_words = quoted_strings_splitter_whitespace(options->function_copy, words, 1024);
247 for(size_t i = 1; i < num_words; i++) {
248 char *param = get_word(words, num_words, i);
249 if(strcmp(param, "info") == 0)
263 - filters->info_only = true;
264 - else if(strncmp(param, "node_type:", 10) == 0)
265 - filters->node_type = param + 10;
266 - else if(strncmp(param, "ingest_status:", 14) == 0)
267 - filters->ingest_status = param + 14;
268 - else if(strncmp(param, "stream_status:", 14) == 0)
269 - filters->stream_status = param + 14;
250 + options->info_only = true;
251 }
252 }
253
@@ -287,29 +268,6 @@ static int streaming_topology_return_error(BUFFER *wb, char *function_copy, int
268 return status;
269 }
270
290 -// check if a value appears in a comma-separated list (NULL list matches everything)
291 -static bool value_in_csv(const char *csv, const char *value) {
292 - if(!csv || !*csv) return true;
293 - if(!value || !*value) return false;
294 - size_t vlen = strlen(value);
295 - const char *p = csv;
296 - while(*p) {
297 - const char *comma = strchr(p, ',');
298 - size_t len = comma ? (size_t)(comma - p) : strlen(p);
299 - while(len && isspace((unsigned char)*p)) {
300 - p++;
301 - len--;
302 - }
303 - while(len && isspace((unsigned char)p[len - 1]))
304 - len--;
305 - if(len == vlen && strncmp(p, value, len) == 0)
306 - return true;
307 - if(!comma) break;
308 - p = comma + 1;
309 - }
310 - return false;
311 -}
312 -
271 // helpers: emit one summary field and one table column
272 static void streaming_topology_emit_sf(BUFFER *wb, const char *key, const char *label, const char *source) {
273 buffer_json_add_array_item_object(wb);
@@ -352,7 +310,7 @@ static void streaming_topology_emit_streaming_path_table(BUFFER *wb, uint64_t or
310 {
311 streaming_topology_emit_col(wb, "hostname", "Agent", NULL);
312 streaming_topology_emit_col(wb, "hops", "Hops", "number");
355 - streaming_topology_emit_col(wb, "since", "Since", "number");
313 + streaming_topology_emit_col(wb, "since", "Since", "timestamp");
314 streaming_topology_emit_col(wb, "flags", "Flags", NULL);
315 }
316 buffer_json_array_close(wb);
@@ -523,17 +481,242 @@ static void streaming_topology_stale_presentation(BUFFER *wb) {
481 streaming_topology_info_tab(wb);
482 }
483
484 +// Bug C synthesis context — passed through rrdhost_stream_path_visit
485 +// to emit synthetic upstream actors and multi-hop links.
486 +struct streaming_topology_synth_ctx {
487 + BUFFER *wb;
488 + DICTIONARY *local_actor_ids; // actor_id -> sentinel; local RRDHOST-backed actors
489 + DICTIONARY *emitted_actors; // actor_id -> sentinel; emitted actor dedup
490 + DICTIONARY *emitted_links; // "src|dst" -> sentinel; emitted link dedup
491 + size_t *actors_total;
492 + size_t *links_total;
493 + // for the multi-hop link pass — previous slot info
494 + bool has_prev;
495 + char prev_actor_id[256];
496 + char prev_hostname[256];
497 + time_t prev_since;
498 + time_t prev_first_time_t;
499 +};
500 +
501 +// Visitor: emit a synthetic "parent" actor for each upstream path entry that
502 +// has no local RRDHOST-backed actor. Local actors are authoritative and are
503 +// always emitted by Phase 3.
504 +static bool streaming_topology_synth_actor_visitor(
505 + void *userdata, uint16_t index __maybe_unused,
506 + STRING *hostname,
507 + ND_UUID host_id, ND_UUID node_id, ND_UUID claim_id __maybe_unused,
508 + int16_t hops,
509 + time_t since, time_t first_time_t,
510 + uint32_t start_time_ms, uint32_t shutdown_time_ms,
511 + STREAM_CAPABILITIES capabilities,
512 + uint32_t flags __maybe_unused) {
513 +
514 + struct streaming_topology_synth_ctx *ctx = userdata;
515 +
516 + // Skip the visiting host's own self entry — Phase 3 emits it if needed.
517 + if(UUIDeq(host_id, localhost->host_id))
518 + return true;
519 +
520 + char guid[UUID_STR_LEN];
521 + if(!streaming_topology_uuid_guid(host_id, guid, sizeof(guid)))
522 + return true;
523 +
524 + char actor_id[256];
525 + streaming_topology_actor_id_from_guid(guid, actor_id, sizeof(actor_id));
526 +
527 + // Skip local RRDHOST-backed actors and already synthesized actors.
528 + if(dictionary_get(ctx->local_actor_ids, actor_id) || dictionary_get(ctx->emitted_actors, actor_id))
529 + return true;
530 +
531 + {
532 + uint8_t one = 1;
533 + dictionary_set(ctx->emitted_actors, actor_id, &one, sizeof(one));
534 + }
535 + (*ctx->actors_total)++;
536 +
537 + BUFFER *wb = ctx->wb;
538 + buffer_json_add_array_item_object(wb);
539 + {
540 + buffer_json_member_add_string(wb, "actor_id", actor_id);
541 + buffer_json_member_add_string(wb, "actor_type", "parent");
542 + buffer_json_member_add_string(wb, "layer", "infra");
543 + buffer_json_member_add_string(wb, "source", "streaming");
544 +
545 + // Match block — same key names as streaming_topology_add_host_match
546 + // (function-topology-streaming.c:45) so the cloud-frontend resolves
547 + // hostnames/GUIDs identically for synthetic and real actors.
548 + buffer_json_member_add_object(wb, "match");
549 + {
550 + buffer_json_member_add_array(wb, "hostnames");
551 + buffer_json_add_array_item_string(wb, string2str(hostname));
552 + buffer_json_array_close(wb);
553 +
554 + buffer_json_member_add_string(wb, "netdata_machine_guid", guid);
555 +
556 + if(!UUIDiszero(node_id))
557 + buffer_json_member_add_uuid(wb, "netdata_node_id", node_id.uuid);
558 + }
559 + buffer_json_object_close(wb); // match
560 +
561 + // Limited attributes — synthesized from path data only. No local
562 + // rrdhost record exists for this remote upstream, so the agent has
563 + // no direct view of the parent's child_count, retention, OS info,
564 + // alerts, etc. The merge layer in Cloud will fill these in from the
565 + // parent's own response (where this actor is the "self" — its
566 + // agent_id matches this actor_id).
567 + buffer_json_member_add_object(wb, "attributes");
568 + {
569 + buffer_json_member_add_string(wb, "display_name", string2str(hostname));
570 + buffer_json_member_add_string(wb, "node_type", "parent");
571 + buffer_json_member_add_string(wb, "severity", "normal");
572 + buffer_json_member_add_uint64(wb, "child_count", 0);
573 + buffer_json_member_add_string(wb, "ephemerality", "permanent");
574 + buffer_json_member_add_int64(wb, "hops", hops);
575 + buffer_json_member_add_uint64(wb, "since", (uint64_t)since);
576 + buffer_json_member_add_uint64(wb, "first_time_t", (uint64_t)first_time_t);
577 + buffer_json_member_add_uint64(wb, "start_time_ms", start_time_ms);
578 + buffer_json_member_add_uint64(wb, "shutdown_time_ms", shutdown_time_ms);
579 + stream_capabilities_to_json_array(wb, capabilities, "capabilities");
580 + }
581 + buffer_json_object_close(wb); // attributes
582 +
583 + // Labels block — mirror Phase 3's set so the FE facet renderer sees
584 + // the same field keys for synthetic and real actors. Status fields
585 + // use path-derived defaults because no local RRDHOST exists.
586 + buffer_json_member_add_object(wb, "labels");
587 + {
588 + buffer_json_member_add_string(wb, "hostname", string2str(hostname));
589 + buffer_json_member_add_string(wb, "node_type", "parent");
590 + buffer_json_member_add_string(wb, "severity", "normal");
591 + buffer_json_member_add_string(wb, "ephemerality", "permanent");
592 + buffer_json_member_add_string(wb, "ingest_status", "online");
593 + buffer_json_member_add_string(wb, "stream_status", "online");
594 + buffer_json_member_add_string(wb, "ml_status", "unknown");
595 + buffer_json_member_add_string(wb, "display_name", string2str(hostname));
596 + }
597 + buffer_json_object_close(wb); // labels
598 +
599 + // The streaming_path field (highlight_path) is left empty for
600 + // synthesized actors — we don't have the full chain visible from
601 + // this side. The FE highlight_path still works for cross-actor
602 + // references because every actor_id is the canonical
603 + // netdata-machine-guid:<guid> form.
604 + buffer_json_member_add_array(wb, "streaming_path");
605 + buffer_json_array_close(wb);
606 +
607 + buffer_json_member_add_object(wb, "tables");
608 + {
609 + buffer_json_member_add_array(wb, "streaming_path");
610 + {
611 + buffer_json_add_array_item_object(wb);
612 + {
613 + buffer_json_member_add_string(wb, "hostname", string2str(hostname));
614 + buffer_json_member_add_int64(wb, "hops", hops);
615 + buffer_json_member_add_uint64(wb, "since", (uint64_t)since);
616 + // flags omitted: STREAM_PATH_FLAGS bitmap is opaque to
617 + // synthesizing code; the table column renders blank for
618 + // synthesized rows. Real actors include flags via
619 + // rrdhost_stream_path_to_json.
620 + }
621 + buffer_json_object_close(wb);
622 + }
623 + buffer_json_array_close(wb);
624 + }
625 + buffer_json_object_close(wb); // tables
626 + }
627 + buffer_json_object_close(wb); // actor
628 +
629 + return true;
630 +}
631 +
632 +// Visitor: emit a streaming link between consecutive path slots. Phase 4
633 +// registers the direct local-host links first; this pass uses emitted_links
634 +// for uniform dedup and adds whatever remains (localhost's own upstream link
635 +// and deeper multi-hop links).
636 +static bool streaming_topology_synth_link_visitor(
637 + void *userdata, uint16_t index __maybe_unused,
638 + STRING *hostname,
639 + ND_UUID host_id, ND_UUID node_id __maybe_unused, ND_UUID claim_id __maybe_unused,
640 + int16_t hops __maybe_unused,
641 + time_t since, time_t first_time_t,
642 + uint32_t start_time_ms __maybe_unused, uint32_t shutdown_time_ms __maybe_unused,
643 + STREAM_CAPABILITIES capabilities __maybe_unused,
644 + uint32_t flags __maybe_unused) {
645 +
646 + struct streaming_topology_synth_ctx *ctx = userdata;
647 +
648 + char guid[UUID_STR_LEN];
649 + if(!streaming_topology_uuid_guid(host_id, guid, sizeof(guid))) {
650 + ctx->has_prev = false;
651 + return true;
652 + }
653 +
654 + char cur_actor_id[256];
655 + streaming_topology_actor_id_from_guid(guid, cur_actor_id, sizeof(cur_actor_id));
656 +
657 + if(ctx->has_prev) {
658 + if(dictionary_get(ctx->emitted_actors, ctx->prev_actor_id) &&
659 + dictionary_get(ctx->emitted_actors, cur_actor_id)) {
660 +
661 + char link_key[600];
662 + snprintfz(link_key, sizeof(link_key), "%s|%s", ctx->prev_actor_id, cur_actor_id);
663 +
664 + if(!dictionary_get(ctx->emitted_links, link_key)) {
665 + uint8_t one = 1;
666 + dictionary_set(ctx->emitted_links, link_key, &one, sizeof(one));
667 + (*ctx->links_total)++;
668 +
669 + BUFFER *wb = ctx->wb;
670 + buffer_json_add_array_item_object(wb);
671 + {
672 + buffer_json_member_add_string(wb, "layer", "infra");
673 + buffer_json_member_add_string(wb, "protocol", "streaming");
674 + buffer_json_member_add_string(wb, "link_type", "streaming");
675 + buffer_json_member_add_string(wb, "src_actor_id", ctx->prev_actor_id);
676 + buffer_json_member_add_string(wb, "dst_actor_id", cur_actor_id);
677 + buffer_json_member_add_string(wb, "state", "online");
678 + // discovered_at / last_seen derive from STREAM_PATH
679 + // timestamps so the merge layer can reconcile views from
680 + // multiple agents reporting the same upstream link.
681 + buffer_json_member_add_datetime_rfc3339(wb, "discovered_at",
682 + ((uint64_t)(ctx->prev_first_time_t ? ctx->prev_first_time_t
683 + : ctx->prev_since)) * USEC_PER_SEC, true);
684 + buffer_json_member_add_datetime_rfc3339(wb, "last_seen",
685 + ((uint64_t)(since ? since : ctx->prev_since)) * USEC_PER_SEC, true);
686 +
687 + // port_name mirrors Phase 4: it is the source hostname.
688 + // Here the source is the previous stream-path slot.
689 + buffer_json_member_add_object(wb, "dst");
690 + {
691 + buffer_json_member_add_object(wb, "attributes");
692 + {
693 + buffer_json_member_add_string(wb, "port_name", ctx->prev_hostname);
694 + }
695 + buffer_json_object_close(wb);
696 + }
697 + buffer_json_object_close(wb);
698 + }
699 + buffer_json_object_close(wb); // link
700 + }
701 + }
702 + }
703 +
704 + ctx->has_prev = true;
705 + snprintfz(ctx->prev_actor_id, sizeof(ctx->prev_actor_id), "%s", cur_actor_id);
706 + snprintfz(ctx->prev_hostname, sizeof(ctx->prev_hostname), "%s", string2str(hostname));
707 + ctx->prev_since = since;
708 + ctx->prev_first_time_t = first_time_t;
709 + return true;
710 +}
711 +
712 int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payload __maybe_unused, const char *source __maybe_unused) {
713 time_t now = now_realtime_sec();
714 usec_t now_ut = now_realtime_usec();
715
530 - struct streaming_topology_filters filters = { 0 };
531 - streaming_topology_parse_filters(function, &filters);
532 - bool info_only = filters.info_only;
533 - const char *filter_node_type = filters.node_type;
534 - const char *filter_ingest_status = filters.ingest_status;
535 - const char *filter_stream_status = filters.stream_status;
536 - char *function_copy = filters.function_copy;
716 + struct streaming_topology_options options = { 0 };
717 + streaming_topology_parse_options(function, &options);
718 + bool info_only = options.info_only;
719 + char *function_copy = options.function_copy;
720
721 buffer_flush(wb);
722 wb->content_type = CT_APPLICATION_JSON;
@@ -546,9 +729,6 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
729 buffer_json_member_add_string(wb, "help", RRDFUNCTIONS_STREAMING_TOPOLOGY_HELP);
730 buffer_json_member_add_array(wb, "accepted_params");
731 {
549 - buffer_json_add_array_item_string(wb, "node_type");
550 - buffer_json_add_array_item_string(wb, "ingest_status");
551 - buffer_json_add_array_item_string(wb, "stream_status");
732 buffer_json_add_array_item_string(wb, "info");
733 }
734 buffer_json_array_close(wb);
@@ -764,8 +944,29 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
944 DICTIONARY *parent_descendants = dictionary_create_advanced(
945 DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_FIXED_SIZE,
946 NULL, sizeof(struct streaming_topology_descendant_list));
767 -
768 - if(!parent_child_count || !parent_descendants) {
947 + // local_actor_ids: every actor backed by a local RRDHOST.
948 + // emitted_actors / emitted_links: dedup sets for actors/links actually
949 + // written to the response.
950 + // The dictionary stores a 1-byte sentinel so dictionary_get() returns
951 + // a non-NULL value for present keys. With value_len=0 the stored value
952 + // is NULL and dictionary_get() can't distinguish present vs absent.
953 + DICTIONARY *local_actor_ids = dictionary_create_advanced(
954 + DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_FIXED_SIZE,
955 + NULL, sizeof(uint8_t));
956 + DICTIONARY *emitted_actors = dictionary_create_advanced(
957 + DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_FIXED_SIZE,
958 + NULL, sizeof(uint8_t));
959 + DICTIONARY *emitted_links = dictionary_create_advanced(
960 + DICT_OPTION_SINGLE_THREADED | DICT_OPTION_DONT_OVERWRITE_VALUE | DICT_OPTION_FIXED_SIZE,
961 + NULL, sizeof(uint8_t));
962 +
963 + if(!parent_child_count || !parent_descendants || !local_actor_ids || !emitted_actors || !emitted_links) {
964 + if(emitted_links)
965 + dictionary_destroy(emitted_links);
966 + if(emitted_actors)
967 + dictionary_destroy(emitted_actors);
968 + if(local_actor_ids)
969 + dictionary_destroy(local_actor_ids);
970 if(parent_descendants)
971 dictionary_destroy(parent_descendants);
972 if(parent_child_count)
@@ -777,6 +978,17 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
978 }
979 else {
980
981 + {
982 + RRDHOST *host;
983 + dfe_start_read(rrdhost_root_index, host) {
984 + char host_actor_id[256];
985 + streaming_topology_actor_id_for_host(host, host_actor_id, sizeof(host_actor_id));
986 + uint8_t one = 1;
987 + dictionary_set(local_actor_ids, host_actor_id, &one, sizeof(one));
988 + }
989 + dfe_done(host);
990 + }
991 +
992 {
993 RRDHOST *host;
994 dfe_start_read(rrdhost_root_index, host) {
@@ -801,33 +1013,85 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1013 uint16_t full_path_n = streaming_topology_get_path_ids(host, 0, full_path_ids, 128);
1014 ND_UUID empty_uuid = {};
1015
804 - if(rrdhost_is_virtual(host)) {
805 - if(full_path_n > 0) {
806 - streaming_topology_descendants_append(parent_descendants,
807 - full_path_ids[0], host, STREAMING_TOPOLOGY_RECEIVED_VIRTUAL, true, empty_uuid);
1016 + if(rrdhost_is_virtual(host))
1017 + continue;
1018
809 - for(uint16_t i = 1; i < full_path_n; i++) {
810 - streaming_topology_descendants_append(parent_descendants,
811 - full_path_ids[i], host, STREAMING_TOPOLOGY_RECEIVED_STREAMING, false, full_path_ids[i - 1]);
812 - }
813 - }
814 - }
815 - else if(full_path_n > 0) {
1019 + if(full_path_n > 0) {
1020 for(uint16_t i = 0; i < full_path_n; i++) {
1021 + // Skip the localhost slot here — Bug A's
1022 + // live-state walk below populates
1023 + // parent_descendants[localhost] authoritatively
1024 + // from rrdhost_status(). If we appended here too
1025 + // we would either double-count or mis-tag offline
1026 + // children as STREAMING.
1027 + if(UUIDeq(full_path_ids[i], localhost->host_id))
1028 + continue;
1029 +
1030 bool source_local = (i == 0);
1031 ND_UUID source_uuid = source_local ? empty_uuid : full_path_ids[i - 1];
1032 streaming_topology_descendants_append(parent_descendants,
1033 full_path_ids[i], host, STREAMING_TOPOLOGY_RECEIVED_STREAMING, source_local, source_uuid);
1034 }
1035 }
823 - else if(host != localhost) {
824 - streaming_topology_descendants_append(parent_descendants,
825 - localhost->host_id, host, STREAMING_TOPOLOGY_RECEIVED_STALE, false, empty_uuid);
826 - }
1036 }
1037 dfe_done(host);
1038 }
1039
1040 + // Bug A fix: localhost live-state classification and descendants.
1041 + // On an apex agent, the path-based walk above can miss localhost
1042 + // as a parent because each child's stored path on us still only
1043 + // contains the child itself until a sparse trigger (retention
1044 + // boundary, node_id update) fires on that child. localhost is
1045 + // appended to the wire format only at JSON-emit time, not stored.
1046 + // We walk rrdhost_root_index once and authoritatively populate
1047 + // parent_child_count[localhost] and parent_descendants[localhost].
1048 + // The path walk above skips localhost so this is the single writer
1049 + // for localhost descendants.
1050 + {
1051 + char localhost_guid[UUID_STR_LEN];
1052 + if(streaming_topology_uuid_guid(localhost->host_id, localhost_guid, sizeof(localhost_guid))) {
1053 + uint32_t live_count = 0;
1054 + ND_UUID empty_uuid_for_live = {};
1055 + RRDHOST *h;
1056 + dfe_start_read(rrdhost_root_index, h) {
1057 + if(h == localhost)
1058 + continue;
1059 +
1060 + if(rrdhost_is_virtual(h)) {
1061 + streaming_topology_descendants_append(parent_descendants,
1062 + localhost->host_id, h,
1063 + STREAMING_TOPOLOGY_RECEIVED_VIRTUAL, true, empty_uuid_for_live);
1064 + continue;
1065 + }
1066 +
1067 + RRDHOST_STATUS hs;
1068 + rrdhost_status(h, now, &hs, RRDHOST_STATUS_ALL);
1069 +
1070 + if(hs.ingest.type == RRDHOST_INGEST_TYPE_CHILD &&
1071 + (hs.ingest.status == RRDHOST_INGEST_STATUS_ONLINE ||
1072 + hs.ingest.status == RRDHOST_INGEST_STATUS_REPLICATING)) {
1073 + live_count++;
1074 +
1075 + streaming_topology_descendants_append(parent_descendants,
1076 + localhost->host_id, h,
1077 + STREAMING_TOPOLOGY_RECEIVED_STREAMING, false, empty_uuid_for_live);
1078 + }
1079 + else {
1080 + streaming_topology_descendants_append(parent_descendants,
1081 + localhost->host_id, h,
1082 + STREAMING_TOPOLOGY_RECEIVED_STALE, false, empty_uuid_for_live);
1083 + }
1084 + }
1085 + dfe_done(h);
1086 +
1087 + uint32_t *existing = dictionary_get(parent_child_count, localhost_guid);
1088 + if(existing)
1089 + *existing = live_count;
1090 + else if(live_count > 0)
1091 + dictionary_set(parent_child_count, localhost_guid, &live_count, sizeof(live_count));
1092 + }
1093 + }
1094 +
1095 buffer_json_member_add_object(wb, "data");
1096 {
1097 size_t actors_total = 0;
@@ -865,17 +1129,6 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1129 node_type = (cc && *cc > 0) ? "parent" : "child";
1130 }
1131
868 - // apply filters
869 - const char *ingest_status_str = rrdhost_ingest_status_to_string(s.ingest.status);
870 - const char *stream_status_str = rrdhost_streaming_status_to_string(s.stream.status);
871 -
872 - if(!value_in_csv(filter_node_type, node_type))
873 - continue;
874 - if(!value_in_csv(filter_ingest_status, ingest_status_str))
875 - continue;
876 - if(!value_in_csv(filter_stream_status, stream_status_str))
877 - continue;
878 -
1132 uint32_t child_count = 0;
1133 {
1134 uint32_t *cc = streaming_topology_parent_child_count_get(parent_child_count, host);
@@ -906,6 +1159,10 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1159 }
1160
1161 actors_total++;
1162 + {
1163 + uint8_t one = 1;
1164 + dictionary_set(emitted_actors, host_actor_id, &one, sizeof(one));
1165 + }
1166 buffer_json_add_array_item_object(wb);
1167 {
1168 buffer_json_member_add_string(wb, "actor_id", host_actor_id);
@@ -945,7 +1202,7 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1202
1203 buffer_json_member_add_object(wb, "labels");
1204 {
948 - // topology-specific labels (used for filtering)
1205 + // topology-specific labels (used by facets and tables)
1206 buffer_json_member_add_string(wb, "hostname", hostname);
1207 buffer_json_member_add_string(wb, "node_type", node_type);
1208 buffer_json_member_add_string(wb, "severity", severity);
@@ -1358,6 +1615,40 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1615 }
1616 dfe_done(host);
1617 }
1618 +
1619 + // --- Bug C synthesis: emit "parent" actors for upstream entries
1620 + // not represented by a host in rrdhost_root_index. The agent may
1621 + // have STREAM_PATH metadata (hostname, hops, since, capabilities)
1622 + // about upstream parents that are not locally registered as
1623 + // RRDHOSTs (e.g., on a child viewing its own topology, the parent
1624 + // is in localhost->stream.path.array but not in rrdhost_root_index).
1625 + // This pass walks every host's stored path and emits a "parent"
1626 + // actor for each unique upstream entry, with attributes sourced
1627 + // directly from the path data. The merge layer fills missing
1628 + // attributes from the parent's own response.
1629 + {
1630 + struct streaming_topology_synth_ctx synth = {
1631 + .wb = wb,
1632 + .local_actor_ids = local_actor_ids,
1633 + .emitted_actors = emitted_actors,
1634 + .emitted_links = emitted_links,
1635 + .actors_total = &actors_total,
1636 + .links_total = NULL,
1637 + .has_prev = false,
1638 + };
1639 +
1640 + RRDHOST *sh;
1641 + dfe_start_read(rrdhost_root_index, sh) {
1642 + // Walk from slot 1: slot 0 of any host's stored path is
1643 + // the host itself (origin) and is already emitted by
1644 + // Phase 3 if it is in rrdhost_root_index. Synthesizing
1645 + // parent actors only makes sense for upstream entries
1646 + // (slot >= 1).
1647 + rrdhost_stream_path_visit(sh, 1, streaming_topology_synth_actor_visitor, &synth);
1648 + }
1649 + dfe_done(sh);
1650 + }
1651 +
1652 buffer_json_array_close(wb); // actors
1653
1654 // --- Phase 4: emit links from streaming_path ---
@@ -1377,23 +1668,6 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1668 RRDHOST_STATUS s;
1669 rrdhost_status(host, now, &s, RRDHOST_STATUS_ALL);
1670
1380 - // apply same filters as actors
1381 - const char *node_type;
1382 - if(rrdhost_is_virtual(host))
1383 - node_type = "vnode";
1384 - else if(s.ingest.status == RRDHOST_INGEST_STATUS_ARCHIVED)
1385 - node_type = "stale";
1386 - else {
1387 - uint32_t *cc = streaming_topology_parent_child_count_get(parent_child_count, host);
1388 - node_type = (cc && *cc > 0) ? "parent" : "child";
1389 - }
1390 - if(!value_in_csv(filter_node_type, node_type))
1391 - continue;
1392 - if(!value_in_csv(filter_ingest_status, rrdhost_ingest_status_to_string(s.ingest.status)))
1393 - continue;
1394 - if(!value_in_csv(filter_stream_status, rrdhost_streaming_status_to_string(s.stream.status)))
1395 - continue;
1396 -
1671 char host_actor_id[256];
1672 streaming_topology_actor_id_for_host(host, host_actor_id, sizeof(host_actor_id));
1673
@@ -1406,12 +1680,12 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1680 ND_UUID link_ids[2];
1681 uint16_t link_n = streaming_topology_get_path_ids(host, 0, link_ids, 2);
1682
1409 - if(is_vnode && link_n >= 2) {
1410 - // vnodes: path[0] is the originating agent
1411 - streaming_topology_actor_id_for_uuid(link_ids[0], target_actor_id, sizeof(target_actor_id));
1683 + if(is_vnode) {
1684 + // vnodes do not stream; they are collected by localhost.
1685 + snprintfz(target_actor_id, sizeof(target_actor_id), "%s", localhost_actor_id);
1686 link_type = "virtual";
1687 }
1414 - else if(!is_vnode && link_n >= 2) {
1688 + else if(link_n >= 2) {
1689 // children/parents: path[1] is the direct parent
1690 streaming_topology_actor_id_for_uuid(link_ids[1], target_actor_id, sizeof(target_actor_id));
1691 link_type = "streaming";
@@ -1424,6 +1698,16 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1698
1699 const char *hostname = rrdhost_hostname(host);
1700
1701 + char link_key[600];
1702 + snprintfz(link_key, sizeof(link_key), "%s|%s", host_actor_id, target_actor_id);
1703 + if(dictionary_get(emitted_links, link_key))
1704 + continue;
1705 +
1706 + {
1707 + uint8_t one = 1;
1708 + dictionary_set(emitted_links, link_key, &one, sizeof(one));
1709 + }
1710 +
1711 links_total++;
1712 buffer_json_add_array_item_object(wb);
1713 {
@@ -1465,6 +1749,33 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1749 }
1750 dfe_done(host);
1751 }
1752 +
1753 + // --- Bug C link synthesis: emit streaming links for consecutive
1754 + // path slots not already emitted by Phase 4. Phase 4 writes one
1755 + // direct link per local non-localhost actor and registers it in
1756 + // emitted_links; this pass adds localhost's own upstream link and
1757 + // any deeper multi-hop links.
1758 + // discovered_at / last_seen derive from STREAM_PATH timestamps so
1759 + // the merge layer can reconcile views from multiple agents.
1760 + {
1761 + struct streaming_topology_synth_ctx synth = {
1762 + .wb = wb,
1763 + .local_actor_ids = local_actor_ids,
1764 + .emitted_actors = emitted_actors,
1765 + .emitted_links = emitted_links,
1766 + .actors_total = NULL,
1767 + .links_total = &links_total,
1768 + .has_prev = false,
1769 + };
1770 +
1771 + RRDHOST *lh;
1772 + dfe_start_read(rrdhost_root_index, lh) {
1773 + synth.has_prev = false;
1774 + rrdhost_stream_path_visit(lh, 0, streaming_topology_synth_link_visitor, &synth);
1775 + }
1776 + dfe_done(lh);
1777 + }
1778 +
1779 buffer_json_array_close(wb); // links
1780
1781 buffer_json_member_add_object(wb, "stats");
@@ -1486,6 +1797,9 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1797 dfe_done(descendants);
1798 dictionary_destroy(parent_descendants);
1799 dictionary_destroy(parent_child_count);
1800 + dictionary_destroy(local_actor_ids);
1801 + dictionary_destroy(emitted_actors);
1802 + dictionary_destroy(emitted_links);
1803 }
1804 }
1805
@@ -1494,966 +1808,3 @@ int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payloa
1808 freez(function_copy);
1809 return HTTP_RESP_OK;
1810 }
1497 -
1498 -
1499 -int function_streaming(BUFFER *wb, const char *function __maybe_unused, BUFFER *payload __maybe_unused, const char *source __maybe_unused) {
1500 -
1501 - time_t now = now_realtime_sec();
1502 -
1503 - buffer_flush(wb);
1504 - wb->content_type = CT_APPLICATION_JSON;
1505 - buffer_json_initialize(wb, "\"", "\"", 0, true, BUFFER_JSON_OPTIONS_DEFAULT);
1506 -
1507 - buffer_json_member_add_string(wb, "hostname", rrdhost_hostname(localhost));
1508 - buffer_json_member_add_uint64(wb, "status", HTTP_RESP_OK);
1509 - buffer_json_member_add_string(wb, "type", "table");
1510 - buffer_json_member_add_time_t(wb, "update_every", STREAMING_FUNCTION_UPDATE_EVERY);
1511 - buffer_json_member_add_boolean(wb, "has_history", false);
1512 - buffer_json_member_add_string(wb, "help", RRDFUNCTIONS_STREAMING_HELP);
1513 - buffer_json_member_add_array(wb, "data");
1514 -
1515 - size_t max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_MAX] = { 0 };
1516 - size_t max_db_metrics = 0, max_db_instances = 0, max_db_contexts = 0;
1517 - size_t max_collection_replication_instances = 0, max_streaming_replication_instances = 0;
1518 - size_t max_ml_anomalous = 0, max_ml_normal = 0, max_ml_trained = 0, max_ml_pending = 0, max_ml_silenced = 0;
1519 -
1520 - time_t
1521 - max_db_duration = 0,
1522 - max_db_from = 0,
1523 - max_db_to = 0,
1524 - max_in_age = 0,
1525 - max_out_age = 0,
1526 - max_out_attempt_age = 0;
1527 -
1528 - uint64_t
1529 - max_in_since = 0,
1530 - max_out_since = 0,
1531 - max_out_attempt_since = 0;
1532 -
1533 - int16_t
1534 - max_in_hops = -1,
1535 - max_out_hops = -1;
1536 -
1537 - int
1538 - max_in_local_port = 0,
1539 - max_in_remote_port = 0,
1540 - max_out_local_port = 0,
1541 - max_out_remote_port = 0;
1542 -
1543 - uint32_t
1544 - max_in_connections = 0,
1545 - max_out_connections = 0;
1546 -
1547 - {
1548 - RRDHOST *host;
1549 - dfe_start_read(rrdhost_root_index, host) {
1550 - RRDHOST_STATUS s;
1551 - rrdhost_status(host, now, &s, RRDHOST_STATUS_ALL);
1552 - buffer_json_add_array_item_array(wb);
1553 -
1554 - if(s.db.metrics > max_db_metrics)
1555 - max_db_metrics = s.db.metrics;
1556 -
1557 - if(s.db.instances > max_db_instances)
1558 - max_db_instances = s.db.instances;
1559 -
1560 - if(s.db.contexts > max_db_contexts)
1561 - max_db_contexts = s.db.contexts;
1562 -
1563 - if(s.ingest.replication.instances > max_collection_replication_instances)
1564 - max_collection_replication_instances = s.ingest.replication.instances;
1565 -
1566 - if(s.stream.replication.instances > max_streaming_replication_instances)
1567 - max_streaming_replication_instances = s.stream.replication.instances;
1568 -
1569 - for(int i = 0; i < STREAM_TRAFFIC_TYPE_MAX ;i++) {
1570 - if (s.stream.sent_bytes_on_this_connection_per_type[i] >
1571 - max_sent_bytes_on_this_connection_per_type[i])
1572 - max_sent_bytes_on_this_connection_per_type[i] =
1573 - s.stream.sent_bytes_on_this_connection_per_type[i];
1574 - }
1575 -
1576 - // Node
1577 - buffer_json_add_array_item_string(wb, rrdhost_hostname(s.host));
1578 -
1579 - // rowOptions
1580 - buffer_json_add_array_item_object(wb);
1581 - {
1582 - const char *severity = NULL; // normal, debug, notice, warning, critical
1583 - if(!rrdhost_option_check(host, RRDHOST_OPTION_EPHEMERAL_HOST)) {
1584 - switch(s.ingest.status) {
1585 - case RRDHOST_INGEST_STATUS_OFFLINE:
1586 - case RRDHOST_INGEST_STATUS_ARCHIVED:
1587 - severity = "critical";
1588 - break;
1589 -
1590 - default:
1591 - case RRDHOST_INGEST_STATUS_INITIALIZING:
1592 - case RRDHOST_INGEST_STATUS_ONLINE:
1593 - case RRDHOST_INGEST_STATUS_REPLICATING:
1594 - break;
1595 - }
1596 -
1597 - switch(s.stream.status) {
1598 - case RRDHOST_STREAM_STATUS_OFFLINE:
1599 - if(!severity && s.stream.reason != STREAM_HANDSHAKE_SP_NO_DESTINATION)
1600 - severity = "warning";
1601 - break;
1602 -
1603 - default:
1604 - case RRDHOST_STREAM_STATUS_REPLICATING:
1605 - case RRDHOST_STREAM_STATUS_ONLINE:
1606 - break;
1607 - }
1608 - }
1609 - buffer_json_member_add_string(wb, "severity", severity ? severity : "normal");
1610 - }
1611 - buffer_json_object_close(wb); // rowOptions
1612 -
1613 - // Ephemerality
1614 - buffer_json_add_array_item_string(wb, rrdhost_option_check(s.host, RRDHOST_OPTION_EPHEMERAL_HOST) ? "ephemeral" : "permanent");
1615 -
1616 - // AgentName and AgentVersion
1617 - buffer_json_add_array_item_string(wb, rrdhost_program_name(host));
1618 - buffer_json_add_array_item_string(wb, rrdhost_program_version(host));
1619 -
1620 - // System Info
1621 - rrdhost_system_info_to_streaming_function_array(wb, s.host->system_info);
1622 -
1623 - // retention
1624 - buffer_json_add_array_item_uint64(wb, s.db.first_time_s * MSEC_PER_SEC); // dbFrom
1625 - if(s.db.first_time_s > max_db_from) max_db_from = s.db.first_time_s;
1626 -
1627 - buffer_json_add_array_item_uint64(wb, s.db.last_time_s * MSEC_PER_SEC); // dbTo
1628 - if(s.db.last_time_s > max_db_to) max_db_to = s.db.last_time_s;
1629 -
1630 - if(s.db.first_time_s && s.db.last_time_s && s.db.last_time_s > s.db.first_time_s) {
1631 - time_t db_duration = s.db.last_time_s - s.db.first_time_s;
1632 - buffer_json_add_array_item_uint64(wb, db_duration); // dbDuration
1633 - if(db_duration > max_db_duration) max_db_duration = db_duration;
1634 - }
1635 - else
1636 - buffer_json_add_array_item_string(wb, NULL); // dbDuration
1637 -
1638 - buffer_json_add_array_item_uint64(wb, s.db.metrics); // dbMetrics
1639 - buffer_json_add_array_item_uint64(wb, s.db.instances); // dbInstances
1640 - buffer_json_add_array_item_uint64(wb, s.db.contexts); // dbContexts
1641 -
1642 - // statuses
1643 - buffer_json_add_array_item_string(wb, rrdhost_ingest_status_to_string(s.ingest.status)); // InStatus
1644 - buffer_json_add_array_item_string(wb, rrdhost_streaming_status_to_string(s.stream.status)); // OutStatus
1645 - buffer_json_add_array_item_string(wb, rrdhost_ml_status_to_string(s.ml.status)); // MLStatus
1646 -
1647 - // collection
1648 -
1649 - // InConnections
1650 - buffer_json_add_array_item_uint64(wb, s.host->stream.rcv.status.connections);
1651 - if(s.host->stream.rcv.status.connections > max_in_connections)
1652 - max_in_connections = s.host->stream.rcv.status.connections;
1653 -
1654 - if(s.ingest.since) {
1655 - uint64_t in_since = s.ingest.since * MSEC_PER_SEC;
1656 - buffer_json_add_array_item_uint64(wb, in_since); // InSince
1657 - if(in_since > max_in_since) max_in_since = in_since;
1658 -
1659 - time_t in_age = s.now - s.ingest.since;
1660 - buffer_json_add_array_item_time_t(wb, in_age); // InAge
1661 - if(in_age > max_in_age) max_in_age = in_age;
1662 - }
1663 - else {
1664 - buffer_json_add_array_item_string(wb, NULL); // InSince
1665 - buffer_json_add_array_item_string(wb, NULL); // InAge
1666 - }
1667 -
1668 - // InReason
1669 - if(s.ingest.type == RRDHOST_INGEST_TYPE_LOCALHOST)
1670 - buffer_json_add_array_item_string(wb, "LOCALHOST");
1671 - else if(s.ingest.type == RRDHOST_INGEST_TYPE_VIRTUAL)
1672 - buffer_json_add_array_item_string(wb, "VIRTUAL NODE");
1673 - else
1674 - buffer_json_add_array_item_string(wb, stream_handshake_error_to_string(s.ingest.reason));
1675 -
1676 - buffer_json_add_array_item_int64(wb, s.ingest.hops); // InHops
1677 - if(s.ingest.hops > max_in_hops) max_in_hops = s.ingest.hops;
1678 -
1679 - buffer_json_add_array_item_double(wb, s.ingest.replication.completion); // InReplCompletion
1680 - buffer_json_add_array_item_uint64(wb, s.ingest.replication.instances); // InReplInstances
1681 - buffer_json_add_array_item_string(wb, s.ingest.type == RRDHOST_INGEST_TYPE_LOCALHOST || s.ingest.type == RRDHOST_INGEST_TYPE_VIRTUAL ? "localhost" : s.ingest.peers.local.ip); // InLocalIP
1682 -
1683 - buffer_json_add_array_item_uint64(wb, s.ingest.peers.local.port); // InLocalPort
1684 - if(s.ingest.peers.local.port > max_in_local_port) max_in_local_port = s.ingest.peers.local.port;
1685 -
1686 - buffer_json_add_array_item_string(wb, s.ingest.peers.peer.ip); // InRemoteIP
1687 - buffer_json_add_array_item_uint64(wb, s.ingest.peers.peer.port); // InRemotePort
1688 - if(s.ingest.peers.peer.port > max_in_remote_port) max_in_remote_port = s.ingest.peers.peer.port;
1689 -
1690 - buffer_json_add_array_item_string(wb, s.ingest.ssl ? "SSL" : "PLAIN"); // InSSL
1691 - stream_capabilities_to_json_array(wb, s.ingest.capabilities, NULL); // InCapabilities
1692 -
1693 - buffer_json_add_array_item_uint64(wb, s.ingest.collected.metrics); // CollectedMetrics
1694 - buffer_json_add_array_item_uint64(wb, s.ingest.collected.instances); // CollectedInstances
1695 - buffer_json_add_array_item_uint64(wb, s.ingest.collected.contexts); // CollectedContexts
1696 -
1697 - // streaming
1698 -
1699 - // OutConnections
1700 - buffer_json_add_array_item_uint64(wb, s.host->stream.snd.status.connections);
1701 - if(s.host->stream.snd.status.connections > max_out_connections)
1702 - max_out_connections = s.host->stream.snd.status.connections;
1703 -
1704 - if(s.stream.since) {
1705 - uint64_t out_since = s.stream.since * MSEC_PER_SEC;
1706 - buffer_json_add_array_item_uint64(wb, out_since); // OutSince
1707 - if(out_since > max_out_since) max_out_since = out_since;
1708 -
1709 - time_t out_age = s.now - s.stream.since;
1710 - buffer_json_add_array_item_time_t(wb, out_age); // OutAge
1711 - if(out_age > max_out_age) max_out_age = out_age;
1712 - }
1713 - else {
1714 - buffer_json_add_array_item_string(wb, NULL); // OutSince
1715 - buffer_json_add_array_item_string(wb, NULL); // OutAge
1716 - }
1717 - buffer_json_add_array_item_string(wb, stream_handshake_error_to_string(s.stream.reason)); // OutReason
1718 -
1719 - buffer_json_add_array_item_int64(wb, s.stream.hops); // OutHops
1720 - if(s.stream.hops > max_out_hops) max_out_hops = s.stream.hops;
1721 -
1722 - buffer_json_add_array_item_double(wb, s.stream.replication.completion); // OutReplCompletion
1723 - buffer_json_add_array_item_uint64(wb, s.stream.replication.instances); // OutReplInstances
1724 - buffer_json_add_array_item_string(wb, s.stream.peers.local.ip); // OutLocalIP
1725 - buffer_json_add_array_item_uint64(wb, s.stream.peers.local.port); // OutLocalPort
1726 - if(s.stream.peers.local.port > max_out_local_port) max_out_local_port = s.stream.peers.local.port;
1727 -
1728 - buffer_json_add_array_item_string(wb, s.stream.peers.peer.ip); // OutRemoteIP
1729 - buffer_json_add_array_item_uint64(wb, s.stream.peers.peer.port); // OutRemotePort
1730 - if(s.stream.peers.peer.port > max_out_remote_port) max_out_remote_port = s.stream.peers.peer.port;
1731 -
1732 - buffer_json_add_array_item_string(wb, s.stream.ssl ? "SSL" : "PLAIN"); // OutSSL
1733 - buffer_json_add_array_item_string(wb, s.stream.compression ? "COMPRESSED" : "UNCOMPRESSED"); // OutCompression
1734 - stream_capabilities_to_json_array(wb, s.stream.capabilities, NULL); // OutCapabilities
1735 - buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_DATA]);
1736 - buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_METADATA]);
1737 - buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_REPLICATION]);
1738 - buffer_json_add_array_item_uint64(wb, s.stream.sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_FUNCTIONS]);
1739 -
1740 - buffer_json_add_array_item_array(wb); // OutAttemptHandshake
1741 - usec_t last_attempt = stream_parent_handshake_error_to_json(wb, host);
1742 - buffer_json_array_close(wb); // // OutAttemptHandshake
1743 -
1744 - if(!last_attempt) {
1745 - buffer_json_add_array_item_string(wb, NULL); // OutAttemptSince
1746 - buffer_json_add_array_item_string(wb, NULL); // OutAttemptAge
1747 - }
1748 - else {
1749 - uint64_t out_attempt_since = last_attempt / USEC_PER_MS;
1750 - buffer_json_add_array_item_uint64(wb, out_attempt_since); // OutAttemptSince
1751 - if(out_attempt_since > max_out_attempt_since) max_out_attempt_since = out_attempt_since;
1752 -
1753 - time_t out_attempt_age = s.now - (time_t)(last_attempt / USEC_PER_SEC);
1754 - buffer_json_add_array_item_time_t(wb, out_attempt_age); // OutAttemptAge
1755 - if(out_attempt_age > max_out_attempt_age) max_out_attempt_age = out_attempt_age;
1756 - }
1757 -
1758 - // ML
1759 - if(s.ml.status == RRDHOST_ML_STATUS_RUNNING) {
1760 - buffer_json_add_array_item_uint64(wb, s.ml.metrics.anomalous); // MlAnomalous
1761 - buffer_json_add_array_item_uint64(wb, s.ml.metrics.normal); // MlNormal
1762 - buffer_json_add_array_item_uint64(wb, s.ml.metrics.trained); // MlTrained
1763 - buffer_json_add_array_item_uint64(wb, s.ml.metrics.pending); // MlPending
1764 - buffer_json_add_array_item_uint64(wb, s.ml.metrics.silenced); // MlSilenced
1765 -
1766 - if(s.ml.metrics.anomalous > max_ml_anomalous)
1767 - max_ml_anomalous = s.ml.metrics.anomalous;
1768 -
1769 - if(s.ml.metrics.normal > max_ml_normal)
1770 - max_ml_normal = s.ml.metrics.normal;
1771 -
1772 - if(s.ml.metrics.trained > max_ml_trained)
1773 - max_ml_trained = s.ml.metrics.trained;
1774 -
1775 - if(s.ml.metrics.pending > max_ml_pending)
1776 - max_ml_pending = s.ml.metrics.pending;
1777 -
1778 - if(s.ml.metrics.silenced > max_ml_silenced)
1779 - max_ml_silenced = s.ml.metrics.silenced;
1780 -
1781 - }
1782 - else {
1783 - buffer_json_add_array_item_string(wb, NULL); // MlAnomalous
1784 - buffer_json_add_array_item_string(wb, NULL); // MlNormal
1785 - buffer_json_add_array_item_string(wb, NULL); // MlTrained
1786 - buffer_json_add_array_item_string(wb, NULL); // MlPending
1787 - buffer_json_add_array_item_string(wb, NULL); // MlSilenced
1788 - }
1789 -
1790 - // close
1791 - buffer_json_array_close(wb);
1792 - }
1793 - dfe_done(host);
1794 - }
1795 - buffer_json_array_close(wb); // data
1796 - buffer_json_member_add_object(wb, "columns");
1797 - {
1798 - size_t field_id = 0;
1799 -
1800 - // Node
1801 - buffer_rrdf_table_add_field(wb, field_id++, "Node", "Node's Hostname",
1802 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1803 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1804 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1805 - RRDF_FIELD_OPTS_VISIBLE | RRDF_FIELD_OPTS_UNIQUE_KEY | RRDF_FIELD_OPTS_STICKY,
1806 - NULL);
1807 -
1808 - buffer_rrdf_table_add_field(wb, field_id++, "rowOptions", "rowOptions",
1809 - RRDF_FIELD_TYPE_NONE, RRDR_FIELD_VISUAL_ROW_OPTIONS, RRDF_FIELD_TRANSFORM_NONE,
1810 - 0, NULL, NAN, RRDF_FIELD_SORT_FIXED, NULL,
1811 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE, RRDF_FIELD_OPTS_DUMMY,
1812 - NULL);
1813 -
1814 - buffer_rrdf_table_add_field(wb, field_id++, "Ephemerality", "The type of ephemerality for the node",
1815 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1816 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1817 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1818 - RRDF_FIELD_OPTS_VISIBLE,
1819 - NULL);
1820 -
1821 - buffer_rrdf_table_add_field(wb, field_id++, "AgentName", "The name of the Netdata agent",
1822 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1823 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1824 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1825 - RRDF_FIELD_OPTS_NONE,
1826 - NULL);
1827 -
1828 - buffer_rrdf_table_add_field(wb, field_id++, "AgentVersion", "The version of the Netdata agent",
1829 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1830 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1831 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1832 - RRDF_FIELD_OPTS_NONE,
1833 - NULL);
1834 -
1835 - buffer_rrdf_table_add_field(wb, field_id++, "OSName", "The name of the host's operating system",
1836 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1837 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1838 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1839 - RRDF_FIELD_OPTS_NONE,
1840 - NULL);
1841 -
1842 - buffer_rrdf_table_add_field(wb, field_id++, "OSId", "The identifier of the host's operating system",
1843 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1844 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1845 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1846 - RRDF_FIELD_OPTS_NONE,
1847 - NULL);
1848 -
1849 - buffer_rrdf_table_add_field(wb, field_id++, "OSIdLike", "The ID-like string for the host's OS",
1850 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1851 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1852 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1853 - RRDF_FIELD_OPTS_NONE,
1854 - NULL);
1855 -
1856 - buffer_rrdf_table_add_field(wb, field_id++, "OSVersion", "The version of the host's operating system",
1857 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1858 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1859 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1860 - RRDF_FIELD_OPTS_NONE,
1861 - NULL);
1862 -
1863 - buffer_rrdf_table_add_field(wb, field_id++, "OSVersionId", "The version identifier of the host's OS",
1864 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1865 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1866 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1867 - RRDF_FIELD_OPTS_NONE,
1868 - NULL);
1869 -
1870 - buffer_rrdf_table_add_field(wb, field_id++, "OSDetection", "Details about host OS detection",
1871 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1872 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1873 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1874 - RRDF_FIELD_OPTS_NONE,
1875 - NULL);
1876 -
1877 - buffer_rrdf_table_add_field(wb, field_id++, "CPUCores", "The number of CPU cores in the host",
1878 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1879 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1880 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1881 - RRDF_FIELD_OPTS_NONE,
1882 - NULL);
1883 -
1884 - buffer_rrdf_table_add_field(wb, field_id++, "DiskSpace", "The total disk space available on the host",
1885 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1886 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1887 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
1888 - RRDF_FIELD_OPTS_NONE,
1889 - NULL);
1890 -
1891 - buffer_rrdf_table_add_field(wb, field_id++, "CPUFreq", "The CPU frequency of the host",
1892 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1893 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1894 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
1895 - RRDF_FIELD_OPTS_NONE,
1896 - NULL);
1897 -
1898 - buffer_rrdf_table_add_field(wb, field_id++, "RAMTotal", "The total RAM available on the host",
1899 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1900 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1901 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
1902 - RRDF_FIELD_OPTS_NONE,
1903 - NULL);
1904 -
1905 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSName", "The name of the container's operating system",
1906 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1907 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1908 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1909 - RRDF_FIELD_OPTS_NONE,
1910 - NULL);
1911 -
1912 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSId", "The identifier of the container's operating system",
1913 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1914 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1915 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1916 - RRDF_FIELD_OPTS_NONE,
1917 - NULL);
1918 -
1919 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSIdLike", "The ID-like string for the container's OS",
1920 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1921 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1922 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1923 - RRDF_FIELD_OPTS_NONE,
1924 - NULL);
1925 -
1926 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSVersion", "The version of the container's OS",
1927 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1928 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1929 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1930 - RRDF_FIELD_OPTS_NONE,
1931 - NULL);
1932 -
1933 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSVersionId", "The version identifier of the container's OS",
1934 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1935 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1936 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1937 - RRDF_FIELD_OPTS_NONE,
1938 - NULL);
1939 -
1940 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerOSDetection", "Details about container OS detection",
1941 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1942 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1943 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1944 - RRDF_FIELD_OPTS_NONE,
1945 - NULL);
1946 -
1947 - buffer_rrdf_table_add_field(wb, field_id++, "IsK8sNode", "Whether this node is part of a Kubernetes cluster",
1948 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1949 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1950 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1951 - RRDF_FIELD_OPTS_NONE,
1952 - NULL);
1953 -
1954 - buffer_rrdf_table_add_field(wb, field_id++, "KernelName", "The kernel name",
1955 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1956 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1957 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1958 - RRDF_FIELD_OPTS_NONE,
1959 - NULL);
1960 -
1961 - buffer_rrdf_table_add_field(wb, field_id++, "KernelVersion", "The kernel version",
1962 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1963 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1964 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1965 - RRDF_FIELD_OPTS_NONE,
1966 - NULL);
1967 -
1968 - buffer_rrdf_table_add_field(wb, field_id++, "Architecture", "The system architecture",
1969 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1970 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1971 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1972 - RRDF_FIELD_OPTS_NONE,
1973 - NULL);
1974 -
1975 - buffer_rrdf_table_add_field(wb, field_id++, "Virtualization", "The virtualization technology in use",
1976 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1977 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1978 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1979 - RRDF_FIELD_OPTS_NONE,
1980 - NULL);
1981 -
1982 - buffer_rrdf_table_add_field(wb, field_id++, "VirtDetection", "Details about virtualization detection",
1983 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1984 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1985 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1986 - RRDF_FIELD_OPTS_NONE,
1987 - NULL);
1988 -
1989 - buffer_rrdf_table_add_field(wb, field_id++, "Container", "Container type information",
1990 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1991 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1992 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
1993 - RRDF_FIELD_OPTS_NONE,
1994 - NULL);
1995 -
1996 - buffer_rrdf_table_add_field(wb, field_id++, "ContainerDetection", "Details about container detection",
1997 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
1998 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
1999 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2000 - RRDF_FIELD_OPTS_NONE,
2001 - NULL);
2002 -
2003 - buffer_rrdf_table_add_field(wb, field_id++, "CloudProviderType", "The type of cloud provider",
2004 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2005 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2006 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2007 - RRDF_FIELD_OPTS_NONE,
2008 - NULL);
2009 -
2010 - buffer_rrdf_table_add_field(wb, field_id++, "CloudInstanceType", "The type of cloud instance",
2011 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2012 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2013 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2014 - RRDF_FIELD_OPTS_NONE,
2015 - NULL);
2016 -
2017 - buffer_rrdf_table_add_field(wb, field_id++, "CloudInstanceRegion", "The region of the cloud instance",
2018 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2019 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2020 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2021 - RRDF_FIELD_OPTS_NONE,
2022 - NULL);
2023 -
2024 - buffer_rrdf_table_add_field(wb, field_id++, "dbFrom", "DB Data Retention From",
2025 - RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
2026 - 0, NULL, (double)max_db_from * MSEC_PER_SEC, RRDF_FIELD_SORT_ASCENDING, NULL,
2027 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_NONE,
2028 - RRDF_FIELD_OPTS_NONE, NULL);
2029 -
2030 - buffer_rrdf_table_add_field(wb, field_id++, "dbTo", "DB Data Retention To",
2031 - RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
2032 - 0, NULL, (double)max_db_to * MSEC_PER_SEC, RRDF_FIELD_SORT_ASCENDING, NULL,
2033 - RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
2034 - RRDF_FIELD_OPTS_NONE, NULL);
2035 -
2036 - buffer_rrdf_table_add_field(wb, field_id++, "dbDuration", "DB Data Retention Duration",
2037 - RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
2038 - 0, NULL, (double)max_db_duration, RRDF_FIELD_SORT_ASCENDING, NULL,
2039 - RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
2040 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2041 -
2042 - buffer_rrdf_table_add_field(wb, field_id++, "dbMetrics", "Time-series Metrics in the DB",
2043 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2044 - 0, NULL, (double)max_db_metrics, RRDF_FIELD_SORT_DESCENDING, NULL,
2045 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2046 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2047 -
2048 - buffer_rrdf_table_add_field(wb, field_id++, "dbInstances", "Instances in the DB",
2049 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2050 - 0, NULL, (double)max_db_instances, RRDF_FIELD_SORT_DESCENDING, NULL,
2051 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2052 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2053 -
2054 - buffer_rrdf_table_add_field(wb, field_id++, "dbContexts", "Contexts in the DB",
2055 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2056 - 0, NULL, (double)max_db_contexts, RRDF_FIELD_SORT_DESCENDING, NULL,
2057 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2058 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2059 -
2060 - // --- statuses ---
2061 -
2062 - buffer_rrdf_table_add_field(wb, field_id++, "InStatus", "Data Collection Online Status",
2063 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2064 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2065 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2066 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2067 -
2068 -
2069 - buffer_rrdf_table_add_field(wb, field_id++, "OutStatus", "Streaming Online Status",
2070 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2071 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2072 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2073 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2074 -
2075 - buffer_rrdf_table_add_field(wb, field_id++, "MlStatus", "ML Status",
2076 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2077 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2078 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2079 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2080 -
2081 - // --- collection ---
2082 -
2083 - buffer_rrdf_table_add_field(wb, field_id++, "InConnections", "Number of times this child connected",
2084 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2085 - 0, NULL, (double)max_in_connections, RRDF_FIELD_SORT_DESCENDING, NULL,
2086 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2087 - RRDF_FIELD_OPTS_NONE, NULL);
2088 -
2089 - buffer_rrdf_table_add_field(wb, field_id++, "InSince", "Last Data Collection Status Change",
2090 - RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
2091 - 0, NULL, (double)max_in_since, RRDF_FIELD_SORT_DESCENDING, NULL,
2092 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_NONE,
2093 - RRDF_FIELD_OPTS_NONE, NULL);
2094 -
2095 - buffer_rrdf_table_add_field(wb, field_id++, "InAge", "Last Data Collection Online Status Change Age",
2096 - RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
2097 - 0, NULL, (double)max_in_age, RRDF_FIELD_SORT_ASCENDING, NULL,
2098 - RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_RANGE,
2099 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2100 -
2101 - buffer_rrdf_table_add_field(wb, field_id++, "InReason", "Data Collection Online Status Reason",
2102 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2103 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2104 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2105 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2106 -
2107 - buffer_rrdf_table_add_field(wb, field_id++, "InHops", "Data Collection Distance Hops from Origin Node",
2108 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2109 - 0, NULL, (double)max_in_hops, RRDF_FIELD_SORT_ASCENDING, NULL,
2110 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2111 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2112 -
2113 - buffer_rrdf_table_add_field(wb, field_id++, "InReplCompletion", "Inbound Replication Completion",
2114 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_BAR, RRDF_FIELD_TRANSFORM_NUMBER,
2115 - 1, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL,
2116 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2117 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2118 -
2119 - buffer_rrdf_table_add_field(wb, field_id++, "InReplInstances", "Inbound Replicating Instances",
2120 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2121 - 0, "instances", (double)max_collection_replication_instances, RRDF_FIELD_SORT_DESCENDING,
2122 - NULL,
2123 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2124 - RRDF_FIELD_OPTS_NONE, NULL);
2125 -
2126 - buffer_rrdf_table_add_field(wb, field_id++, "InLocalIP", "Inbound Local IP",
2127 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2128 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2129 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2130 - RRDF_FIELD_OPTS_NONE, NULL);
2131 -
2132 - buffer_rrdf_table_add_field(wb, field_id++, "InLocalPort", "Inbound Local Port",
2133 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2134 - 0, NULL, (double)max_in_local_port, RRDF_FIELD_SORT_ASCENDING, NULL,
2135 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
2136 - RRDF_FIELD_OPTS_NONE, NULL);
2137 -
2138 - buffer_rrdf_table_add_field(wb, field_id++, "InRemoteIP", "Inbound Remote IP",
2139 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2140 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2141 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2142 - RRDF_FIELD_OPTS_NONE, NULL);
2143 -
2144 - buffer_rrdf_table_add_field(wb, field_id++, "InRemotePort", "Inbound Remote Port",
2145 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2146 - 0, NULL, (double)max_in_remote_port, RRDF_FIELD_SORT_ASCENDING, NULL,
2147 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
2148 - RRDF_FIELD_OPTS_NONE, NULL);
2149 -
2150 - buffer_rrdf_table_add_field(wb, field_id++, "InSSL", "Inbound SSL Connection",
2151 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2152 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2153 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2154 - RRDF_FIELD_OPTS_NONE, NULL);
2155 -
2156 - buffer_rrdf_table_add_field(wb, field_id++, "InCapabilities", "Inbound Connection Capabilities",
2157 - RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
2158 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2159 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2160 - RRDF_FIELD_OPTS_NONE, NULL);
2161 -
2162 - buffer_rrdf_table_add_field(wb, field_id++, "CollectedMetrics", "Time-series Metrics Currently Collected",
2163 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2164 - 0, NULL, (double)max_db_metrics, RRDF_FIELD_SORT_DESCENDING, NULL,
2165 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2166 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2167 -
2168 - buffer_rrdf_table_add_field(wb, field_id++, "CollectedInstances", "Instances Currently Collected",
2169 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2170 - 0, NULL, (double)max_db_instances, RRDF_FIELD_SORT_DESCENDING, NULL,
2171 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2172 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2173 -
2174 - buffer_rrdf_table_add_field(wb, field_id++, "CollectedContexts", "Contexts Currently Collected",
2175 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2176 - 0, NULL, (double)max_db_contexts, RRDF_FIELD_SORT_DESCENDING, NULL,
2177 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2178 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2179 -
2180 - // --- streaming ---
2181 -
2182 - buffer_rrdf_table_add_field(wb, field_id++, "OutConnections", "Number of times connected to a parent",
2183 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2184 - 0, NULL, (double)max_out_connections, RRDF_FIELD_SORT_DESCENDING, NULL,
2185 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2186 - RRDF_FIELD_OPTS_NONE, NULL);
2187 -
2188 - buffer_rrdf_table_add_field(wb, field_id++, "OutSince", "Last Streaming Status Change",
2189 - RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
2190 - 0, NULL, (double)max_out_since, RRDF_FIELD_SORT_DESCENDING, NULL,
2191 - RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
2192 - RRDF_FIELD_OPTS_NONE, NULL);
2193 -
2194 - buffer_rrdf_table_add_field(wb, field_id++, "OutAge", "Last Streaming Status Change Age",
2195 - RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
2196 - 0, NULL, (double)max_out_age, RRDF_FIELD_SORT_ASCENDING, NULL,
2197 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2198 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2199 -
2200 - buffer_rrdf_table_add_field(wb, field_id++, "OutReason", "Streaming Status Reason",
2201 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2202 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2203 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2204 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2205 -
2206 - buffer_rrdf_table_add_field(wb, field_id++, "OutHops", "Streaming Distance Hops from Origin Node",
2207 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2208 - 0, NULL, (double)max_out_hops, RRDF_FIELD_SORT_ASCENDING, NULL,
2209 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2210 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2211 -
2212 - buffer_rrdf_table_add_field(wb, field_id++, "OutReplCompletion", "Outbound Replication Completion",
2213 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_BAR, RRDF_FIELD_TRANSFORM_NUMBER,
2214 - 1, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL,
2215 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2216 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2217 -
2218 - buffer_rrdf_table_add_field(wb, field_id++, "OutReplInstances", "Outbound Replicating Instances",
2219 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2220 - 0, "instances", (double)max_streaming_replication_instances, RRDF_FIELD_SORT_DESCENDING,
2221 - NULL,
2222 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2223 - RRDF_FIELD_OPTS_NONE, NULL);
2224 -
2225 - buffer_rrdf_table_add_field(wb, field_id++, "OutLocalIP", "Outbound Local IP",
2226 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2227 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2228 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2229 - RRDF_FIELD_OPTS_NONE, NULL);
2230 -
2231 - buffer_rrdf_table_add_field(wb, field_id++, "OutLocalPort", "Outbound Local Port",
2232 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2233 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2234 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_NONE,
2235 - RRDF_FIELD_OPTS_NONE, NULL);
2236 -
2237 - buffer_rrdf_table_add_field(wb, field_id++, "OutRemoteIP", "Outbound Remote IP",
2238 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2239 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2240 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2241 - RRDF_FIELD_OPTS_NONE, NULL);
2242 -
2243 - buffer_rrdf_table_add_field(wb, field_id++, "OutRemotePort", "Outbound Remote Port",
2244 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2245 - 0, NULL, (double)max_out_remote_port, RRDF_FIELD_SORT_ASCENDING, NULL,
2246 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_RANGE,
2247 - RRDF_FIELD_OPTS_NONE, NULL);
2248 -
2249 - buffer_rrdf_table_add_field(wb, field_id++, "OutSSL", "Outbound SSL Connection",
2250 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2251 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2252 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2253 - RRDF_FIELD_OPTS_NONE, NULL);
2254 -
2255 - buffer_rrdf_table_add_field(wb, field_id++, "OutCompression", "Outbound Compressed Connection",
2256 - RRDF_FIELD_TYPE_STRING, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NONE,
2257 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2258 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2259 - RRDF_FIELD_OPTS_NONE, NULL);
2260 -
2261 - buffer_rrdf_table_add_field(wb, field_id++, "OutCapabilities", "Outbound Connection Capabilities",
2262 - RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
2263 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2264 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2265 - RRDF_FIELD_OPTS_NONE, NULL);
2266 -
2267 - buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficData", "Outbound Metric Data Traffic",
2268 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2269 - 0, "bytes", (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_DATA],
2270 - RRDF_FIELD_SORT_DESCENDING, NULL,
2271 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2272 - RRDF_FIELD_OPTS_NONE, NULL);
2273 -
2274 - buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficMetadata", "Outbound Metric Metadata Traffic",
2275 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2276 - 0, "bytes",
2277 - (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_METADATA],
2278 - RRDF_FIELD_SORT_DESCENDING, NULL,
2279 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2280 - RRDF_FIELD_OPTS_NONE, NULL);
2281 -
2282 - buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficReplication", "Outbound Metric Replication Traffic",
2283 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2284 - 0, "bytes",
2285 - (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_REPLICATION],
2286 - RRDF_FIELD_SORT_DESCENDING, NULL,
2287 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2288 - RRDF_FIELD_OPTS_NONE, NULL);
2289 -
2290 - buffer_rrdf_table_add_field(wb, field_id++, "OutTrafficFunctions", "Outbound Metric Functions Traffic",
2291 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2292 - 0, "bytes",
2293 - (double)max_sent_bytes_on_this_connection_per_type[STREAM_TRAFFIC_TYPE_FUNCTIONS],
2294 - RRDF_FIELD_SORT_DESCENDING, NULL,
2295 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2296 - RRDF_FIELD_OPTS_NONE, NULL);
2297 -
2298 - buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptHandshake",
2299 - "Outbound Connection Attempt Handshake Status",
2300 - RRDF_FIELD_TYPE_ARRAY, RRDF_FIELD_VISUAL_PILL, RRDF_FIELD_TRANSFORM_NONE,
2301 - 0, NULL, NAN, RRDF_FIELD_SORT_ASCENDING, NULL,
2302 - RRDF_FIELD_SUMMARY_COUNT, RRDF_FIELD_FILTER_MULTISELECT,
2303 - RRDF_FIELD_OPTS_NONE, NULL);
2304 -
2305 - buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptSince",
2306 - "Last Outbound Connection Attempt Status Change Time",
2307 - RRDF_FIELD_TYPE_TIMESTAMP, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DATETIME_MS,
2308 - 0, NULL, (double)max_out_attempt_since, RRDF_FIELD_SORT_DESCENDING, NULL,
2309 - RRDF_FIELD_SUMMARY_MAX, RRDF_FIELD_FILTER_NONE,
2310 - RRDF_FIELD_OPTS_NONE, NULL);
2311 -
2312 - buffer_rrdf_table_add_field(wb, field_id++, "OutAttemptAge",
2313 - "Last Outbound Connection Attempt Status Change Age",
2314 - RRDF_FIELD_TYPE_DURATION, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_DURATION_S,
2315 - 0, NULL, (double)max_out_attempt_age, RRDF_FIELD_SORT_ASCENDING, NULL,
2316 - RRDF_FIELD_SUMMARY_MIN, RRDF_FIELD_FILTER_RANGE,
2317 - RRDF_FIELD_OPTS_VISIBLE, NULL);
2318 -
2319 - // --- ML ---
2320 -
2321 - buffer_rrdf_table_add_field(wb, field_id++, "MlAnomalous", "Number of Anomalous Metrics",
2322 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2323 - 0, "metrics",
2324 - (double)max_ml_anomalous,
2325 - RRDF_FIELD_SORT_DESCENDING, NULL,
2326 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2327 - RRDF_FIELD_OPTS_NONE, NULL);
2328 -
2329 - buffer_rrdf_table_add_field(wb, field_id++, "MlNormal", "Number of Not Anomalous Metrics",
2330 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2331 - 0, "metrics",
2332 - (double)max_ml_normal,
2333 - RRDF_FIELD_SORT_DESCENDING, NULL,
2334 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2335 - RRDF_FIELD_OPTS_NONE, NULL);
2336 -
2337 - buffer_rrdf_table_add_field(wb, field_id++, "MlTrained", "Number of Trained Metrics",
2338 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2339 - 0, "metrics",
2340 - (double)max_ml_trained,
2341 - RRDF_FIELD_SORT_DESCENDING, NULL,
2342 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2343 - RRDF_FIELD_OPTS_NONE, NULL);
2344 -
2345 - buffer_rrdf_table_add_field(wb, field_id++, "MlPending", "Number of Pending Metrics",
2346 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2347 - 0, "metrics",
2348 - (double)max_ml_pending,
2349 - RRDF_FIELD_SORT_DESCENDING, NULL,
2350 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2351 - RRDF_FIELD_OPTS_NONE, NULL);
2352 -
2353 - buffer_rrdf_table_add_field(wb, field_id++, "MlSilenced", "Number of Silenced Metrics",
2354 - RRDF_FIELD_TYPE_INTEGER, RRDF_FIELD_VISUAL_VALUE, RRDF_FIELD_TRANSFORM_NUMBER,
2355 - 0, "metrics",
2356 - (double)max_ml_silenced,
2357 - RRDF_FIELD_SORT_DESCENDING, NULL,
2358 - RRDF_FIELD_SUMMARY_SUM, RRDF_FIELD_FILTER_RANGE,
2359 - RRDF_FIELD_OPTS_NONE, NULL);
2360 - }
2361 - buffer_json_object_close(wb); // columns
2362 - buffer_json_member_add_string(wb, "default_sort_column", "Node");
2363 - buffer_json_member_add_object(wb, "charts");
2364 - {
2365 - // Data Collection Age chart
2366 - buffer_json_member_add_object(wb, "InAge");
2367 - {
2368 - buffer_json_member_add_string(wb, "name", "Data Collection Age");
2369 - buffer_json_member_add_string(wb, "type", "stacked-bar");
2370 - buffer_json_member_add_array(wb, "columns");
2371 - {
2372 - buffer_json_add_array_item_string(wb, "InAge");
2373 - }
2374 - buffer_json_array_close(wb);
2375 - }
2376 - buffer_json_object_close(wb);
2377 -
2378 - // Streaming Age chart
2379 - buffer_json_member_add_object(wb, "OutAge");
2380 - {
2381 - buffer_json_member_add_string(wb, "name", "Streaming Age");
2382 - buffer_json_member_add_string(wb, "type", "stacked-bar");
2383 - buffer_json_member_add_array(wb, "columns");
2384 - {
2385 - buffer_json_add_array_item_string(wb, "OutAge");
2386 - }
2387 - buffer_json_array_close(wb);
2388 - }
2389 - buffer_json_object_close(wb);
2390 -
2391 - // DB Duration
2392 - buffer_json_member_add_object(wb, "dbDuration");
2393 - {
2394 - buffer_json_member_add_string(wb, "name", "Retention Duration");
2395 - buffer_json_member_add_string(wb, "type", "stacked-bar");
2396 - buffer_json_member_add_array(wb, "columns");
2397 - {
2398 - buffer_json_add_array_item_string(wb, "dbDuration");
2399 - }
2400 - buffer_json_array_close(wb);
2401 - }
2402 - buffer_json_object_close(wb);
2403 - }
2404 - buffer_json_object_close(wb); // charts
2405 -
2406 - buffer_json_member_add_array(wb, "default_charts");
2407 - {
2408 - buffer_json_add_array_item_array(wb);
2409 - buffer_json_add_array_item_string(wb, "InAge");
2410 - buffer_json_add_array_item_string(wb, "Node");
2411 - buffer_json_array_close(wb);
2412 -
2413 - buffer_json_add_array_item_array(wb);
2414 - buffer_json_add_array_item_string(wb, "OutAge");
2415 - buffer_json_add_array_item_string(wb, "Node");
2416 - buffer_json_array_close(wb);
2417 - }
2418 - buffer_json_array_close(wb);
2419 -
2420 - buffer_json_member_add_object(wb, "group_by");
2421 - {
2422 - GROUP_BY_COLUMN("OSName", "O/S Name");
2423 - GROUP_BY_COLUMN("OSId", "O/S ID");
2424 - GROUP_BY_COLUMN("OSIdLike", "O/S ID Like");
2425 - GROUP_BY_COLUMN("OSVersion", "O/S Version");
2426 - GROUP_BY_COLUMN("OSVersionId", "O/S Version ID");
2427 - GROUP_BY_COLUMN("OSDetection", "O/S Detection");
2428 - GROUP_BY_COLUMN("CPUCores", "CPU Cores");
2429 - GROUP_BY_COLUMN("ContainerOSName", "Container O/S Name");
2430 - GROUP_BY_COLUMN("ContainerOSId", "Container O/S ID");
2431 - GROUP_BY_COLUMN("ContainerOSIdLike", "Container O/S ID Like");
2432 - GROUP_BY_COLUMN("ContainerOSVersion", "Container O/S Version");
2433 - GROUP_BY_COLUMN("ContainerOSVersionId", "Container O/S Version ID");
2434 - GROUP_BY_COLUMN("ContainerOSDetection", "Container O/S Detection");
2435 - GROUP_BY_COLUMN("IsK8sNode", "Kubernetes Nodes");
2436 - GROUP_BY_COLUMN("KernelName", "Kernel Name");
2437 - GROUP_BY_COLUMN("KernelVersion", "Kernel Version");
2438 - GROUP_BY_COLUMN("Architecture", "Architecture");
2439 - GROUP_BY_COLUMN("Virtualization", "Virtualization Technology");
2440 - GROUP_BY_COLUMN("VirtDetection", "Virtualization Detection");
2441 - GROUP_BY_COLUMN("Container", "Container");
2442 - GROUP_BY_COLUMN("ContainerDetection", "Container Detection");
2443 - GROUP_BY_COLUMN("CloudProviderType", "Cloud Provider Type");
2444 - GROUP_BY_COLUMN("CloudInstanceType", "Cloud Instance Type");
2445 - GROUP_BY_COLUMN("CloudInstanceRegion", "Cloud Instance Region");
2446 -
2447 - GROUP_BY_COLUMN("InStatus", "Collection Status");
2448 - GROUP_BY_COLUMN("OutStatus", "Streaming Status");
2449 - GROUP_BY_COLUMN("MlStatus", "ML Status");
2450 - GROUP_BY_COLUMN("InRemoteIP", "Inbound IP");
2451 - GROUP_BY_COLUMN("OutRemoteIP", "Outbound IP");
2452 - }
2453 - buffer_json_object_close(wb); // group_by
2454 -
2455 - buffer_json_member_add_time_t(wb, "expires", now_realtime_sec() + STREAMING_FUNCTION_UPDATE_EVERY);
2456 - buffer_json_finalize(wb);
2457 -
2458 - return HTTP_RESP_OK;
2459 -}
src/web/api/functions/function-topology-streaming.h new
+12
@@ -0,0 +1,12 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#ifndef NETDATA_FUNCTION_TOPOLOGY_STREAMING_H
4 +#define NETDATA_FUNCTION_TOPOLOGY_STREAMING_H
5 +
6 +#include "database/rrd.h"
7 +
8 +#define RRDFUNCTIONS_STREAMING_TOPOLOGY_HELP "Shows streaming topology relations across Netdata agents, including directional parent/child links and transport metadata."
9 +
10 +int function_streaming_topology(BUFFER *wb, const char *function, BUFFER *payload, const char *source);
11 +
12 +#endif //NETDATA_FUNCTION_TOPOLOGY_STREAMING_H
src/web/api/functions/functions.c
+1 -1
@@ -15,7 +15,7 @@ void global_functions_add(void) {
15 RRDFUNCTIONS_STREAMING_HELP,
16 "top",
17 HTTP_ACCESS_SIGNED_ID | HTTP_ACCESS_SAME_SPACE | HTTP_ACCESS_SENSITIVE_DATA,
18 - function_streaming);
18 + function_netdata_streaming);
19
20 rrd_function_add_inline(
21 localhost,
src/web/api/functions/functions.h
+2 -1
@@ -6,7 +6,8 @@
6 #include "database/rrd.h"
7
8 #include "function-metrics-cardinality.h"
9 -#include "function-streaming.h"
9 +#include "function-netdata-streaming.h"
10 +#include "function-topology-streaming.h"
11 #include "function-progress.h"
12 #include "function-bearer_get_token.h"
13