| 1 | <!-- markdownlint-disable-file MD043 --> |
| 2 | |
| 3 | # Netdata Topology Function Schema |
| 4 | |
| 5 | This document defines the production topology payload contract for Netdata |
| 6 | Functions. It is the source of truth for new topology producers and for the |
| 7 | Cloud topology aggregator. |
| 8 | |
| 9 | The JSON Schema is [FUNCTION_TOPOLOGY_SCHEMA.json](/src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json). |
| 10 | |
| 11 | ## Purpose |
| 12 | |
| 13 | Topology payloads describe relationships between observed entities: |
| 14 | |
| 15 | - infrastructure nodes and virtual machines; |
| 16 | - containers, pods, namespaces, and workloads; |
| 17 | - processes and sockets; |
| 18 | - network devices, ports, VLANs, MACs, and IPs; |
| 19 | - streaming parents and children; |
| 20 | - storage, virtualization, and custom topology domains. |
| 21 | |
| 22 | The payload is optimized for two consumers: |
| 23 | |
| 24 | - the Cloud aggregator, which merges topology evidence across nodes and views; |
| 25 | - the UI, which renders graph views and drilldown tables. |
| 26 | |
| 27 | It is not optimized for reconstructing compatibility payloads. Parity with |
| 28 | compatibility consumers is a test concern, implemented by test-side projection |
| 29 | code. |
| 30 | |
| 31 | ## Non-Goals |
| 32 | |
| 33 | Do not put these in production topology payloads: |
| 34 | |
| 35 | - compatibility field names kept only for rollout adapters; |
| 36 | - instructions for reconstructing compatibility payloads; |
| 37 | - repeated per-row display strings or labels copied from actor/type metadata; |
| 38 | - duplicated actor modal rows that repeat relationship evidence; |
| 39 | - visual layout hints that the UI can own; |
| 40 | - raw CSS, raw SVG, component names, coordinates, viewport state, or force-layout |
| 41 | physics; |
| 42 | - raw secrets or customer-identifying sample data in fixtures. |
| 43 | |
| 44 | Type-level labels, UI-owned visual tokens, safe label policies, and small |
| 45 | column descriptions are acceptable when they help render generic topology UI. |
| 46 | Define them once per type or column, not once per row. |
| 47 | |
| 48 | ## Mental Model |
| 49 | |
| 50 | The schema has six planes: |
| 51 | |
| 52 | 1. **Actors** are entities: nodes, processes, containers, ports, devices, |
| 53 | vSphere objects, streaming agents, and custom objects. |
| 54 | 2. **Graph links** are the renderable relationship groups between actors. |
| 55 | 3. **Evidence** is the lossless relationship proof behind each graph link: |
| 56 | sockets, LLDP observations, streaming path hops, vSphere inventory edges, |
| 57 | or custom relationship facts. |
| 58 | 4. **Detail tables** are non-graph data shown in actor or relationship |
| 59 | drilldowns. They are explicitly typed as actor-owned or relationship-owned. |
| 60 | 5. **Presentation** is backend-selected composition of UI-owned tokens for |
| 61 | labels, colors, icons, legend, link styles, highlight behavior, and graph |
| 62 | port bullets. |
| 63 | 6. **Overlay refs** describe how to query refreshable metrics for links or |
| 64 | actors without recomputing topology. |
| 65 | |
| 66 | Graph links are intentionally smaller than evidence. A graph link may have |
| 67 | one evidence row or tens of thousands of evidence rows. |
| 68 | |
| 69 | ## Response Envelope |
| 70 | |
| 71 | Topology Functions return a normal Function envelope with `type: "topology"`: |
| 72 | |
| 73 | ```json |
| 74 | { |
| 75 | "v": 3, |
| 76 | "status": 200, |
| 77 | "type": "topology", |
| 78 | "has_history": false, |
| 79 | "update_every": 5, |
| 80 | "expires": 10, |
| 81 | "data": { |
| 82 | "schema_version": "netdata.topology.v1", |
| 83 | "producer": { |
| 84 | "source": "network-connections", |
| 85 | "instance": "local", |
| 86 | "node_id": "node-uuid", |
| 87 | "machine_guid": "machine-guid", |
| 88 | "plugin": "network-viewer.plugin" |
| 89 | }, |
| 90 | "collected_at": "2026-05-09T10:00:00Z", |
| 91 | "dictionaries": { |
| 92 | "strings": ["process", "endpoint", "tcp", "inbound", "established"] |
| 93 | }, |
| 94 | "types": { |
| 95 | "actor_types": {}, |
| 96 | "link_types": {} |
| 97 | }, |
| 98 | "presentation": { |
| 99 | "selection": { |
| 100 | "actor_click": {"mode": "highlight_connections"} |
| 101 | }, |
| 102 | "legend": { |
| 103 | "actors": [], |
| 104 | "links": [], |
| 105 | "ports": [] |
| 106 | } |
| 107 | }, |
| 108 | "actors": { |
| 109 | "rows": 0, |
| 110 | "columns": [], |
| 111 | "values": [] |
| 112 | }, |
| 113 | "links": { |
| 114 | "rows": 0, |
| 115 | "columns": [], |
| 116 | "values": [] |
| 117 | } |
| 118 | } |
| 119 | } |
| 120 | ``` |
| 121 | |
| 122 | `v` is the optional Function transport protocol version. It belongs to the |
| 123 | response envelope, not to `data`, and is commonly `3` for Functions that accept |
| 124 | POST JSON parameters. |
| 125 | |
| 126 | `schema_version` is the topology contract version. It is not the producer |
| 127 | version. Producers may expose their own version in `producer.agent_version`, |
| 128 | `producer.plugin`, or `producer.capabilities`. |
| 129 | |
| 130 | Function `info` responses are response metadata, not topology payloads. They |
| 131 | may advertise `accepted_params`, `required_params`, and help text without a |
| 132 | `data` object. Validate only full topology responses against |
| 133 | `FUNCTION_TOPOLOGY_SCHEMA.json`. |
| 134 | |
| 135 | ## Mode Requests |
| 136 | |
| 137 | Use the request parameter `__topology_mode` when a topology producer has a real |
| 138 | detailed vs aggregated output difference. Supported values are `detailed` and |
| 139 | `aggregated`. |
| 140 | |
| 141 | Do not expose a detailed/aggregated selector for mode-invariant topologies. |
| 142 | SNMP/L2 and streaming currently emit the same topology grain for both use cases |
| 143 | and should not advertise a mode option until a real difference exists. |
| 144 | |
| 145 | When a producer has a real mode split, set `data.view.supported_modes` to the |
| 146 | available modes. When the field is absent or contains a single value, consumers |
| 147 | must treat the topology as mode-invariant and avoid showing a mode toggle. |
| 148 | |
| 149 | The Cloud topology aggregator consumes detailed payloads when a producer |
| 150 | supports the mode, even when the user asked Cloud for aggregated output. The |
| 151 | aggregator correlates first, then aggregates the returned graph. Producers must |
| 152 | therefore keep detailed mode lossless enough for cross-node matching. |
| 153 | |
| 154 | ## Compact Tables |
| 155 | |
| 156 | All large arrays use the same compact table shape: |
| 157 | |
| 158 | ```json |
| 159 | { |
| 160 | "rows": 3, |
| 161 | "columns": [ |
| 162 | {"id": "type", "type": "string_ref", "dictionary": "strings"}, |
| 163 | {"id": "socket_count", "type": "uint", "aggregation": "sum"} |
| 164 | ], |
| 165 | "values": [ |
| 166 | {"codec": "dict", "values": [0, 1], "indexes": [0, 0, 1]}, |
| 167 | {"codec": "values", "values": [10, 4, 19]} |
| 168 | ] |
| 169 | } |
| 170 | ``` |
| 171 | |
| 172 | `columns` and `values` are parallel arrays. The Nth value encoding belongs to |
| 173 | the Nth column. Every decoded column must produce exactly `rows` values. |
| 174 | |
| 175 | Supported codecs: |
| 176 | |
| 177 | - `const`: one value repeated for all rows; |
| 178 | - `values`: one value per row; |
| 179 | - `dict`: a per-column value dictionary plus integer indexes. |
| 180 | |
| 181 | Column values may be plain scalars or references into a global dictionary. |
| 182 | For string-heavy columns, prefer `string_ref` with `dictionary: "strings"` and |
| 183 | integer values. This is how production socket evidence reached about 22 raw |
| 184 | bytes per socket evidence row in the measured corpus. |
| 185 | |
| 186 | Use column type `json` only for actor/custom detail cells that must preserve |
| 187 | nested producer-owned data, such as SNMP port `neighbors` or `vlans`. Do not |
| 188 | use `json` for high-cardinality relationship evidence when a typed scalar, |
| 189 | reference, or array column can carry the fact. |
| 190 | |
| 191 | Go topology producers should use `src/go/pkg/topology/v1` for the |
| 192 | `netdata.topology.v1` response model and compact-table constructors. The helper |
| 193 | validates row counts, parallel `columns`/`values` lengths, dictionary indexes, |
| 194 | and JSON round-trip behavior before a producer response reaches the Function |
| 195 | transport. |
| 196 | |
| 197 | ## Required Actor Semantics |
| 198 | |
| 199 | The `actors` table must carry enough canonical identity for aggregation. Actor |
| 200 | row indexes are used as link endpoints inside the same payload. |
| 201 | |
| 202 | Required actor columns: |
| 203 | |
| 204 | - a type column, normally `type`; |
| 205 | - a layer column, normally `layer`; |
| 206 | - every identity column declared by `types.actor_types.<type>.identity`. |
| 207 | |
| 208 | Common actor identity columns: |
| 209 | |
| 210 | - `node_id` |
| 211 | - `machine_guid` |
| 212 | - `hostname` |
| 213 | - `ip` |
| 214 | - `mac` |
| 215 | - `process_name` |
| 216 | - `pid` |
| 217 | - `container_id` |
| 218 | - `container_name` |
| 219 | - `pod` |
| 220 | - `namespace` |
| 221 | - `k8s_label:<name>` |
| 222 | - `vsphere_moid` |
| 223 | - `vsphere_inventory_path` |
| 224 | |
| 225 | Actor types define source-local identity and cross-source merge identity: |
| 226 | |
| 227 | ```json |
| 228 | { |
| 229 | "types": { |
| 230 | "actor_types": { |
| 231 | "process": { |
| 232 | "layer": "process", |
| 233 | "identity": ["node_id", "process_name"], |
| 234 | "merge_identity": ["node_id", "process_name"], |
| 235 | "parent_identity": ["node_id"], |
| 236 | "aggregation_scopes": ["node", "process_name", "container", "k8s_workload"], |
| 237 | "search": { |
| 238 | "enabled": true, |
| 239 | "columns": ["display_name", "process_name"], |
| 240 | "label_keys": ["cmdline", "username"] |
| 241 | }, |
| 242 | "presentation": { |
| 243 | "label": "Process", |
| 244 | "role": "actor", |
| 245 | "icon": "process", |
| 246 | "color_slot": "primary", |
| 247 | "border": {"enabled": true}, |
| 248 | "size": {"mode": "link_count", "scale": "normal"}, |
| 249 | "layout": {"repulsion": "normal"}, |
| 250 | "label_policy": { |
| 251 | "columns": ["display_name", "process_name"], |
| 252 | "fallback": "type_label", |
| 253 | "max_length": 80, |
| 254 | "array": "reject" |
| 255 | }, |
| 256 | "ports": { |
| 257 | "show_bullets": true, |
| 258 | "sources": [ |
| 259 | { |
| 260 | "source": "actor_table", |
| 261 | "table": "socket_ports", |
| 262 | "actor_column": "actor", |
| 263 | "name_column": "port", |
| 264 | "value_column": "socket_count", |
| 265 | "default_type": "topology" |
| 266 | } |
| 267 | ] |
| 268 | } |
| 269 | } |
| 270 | } |
| 271 | } |
| 272 | } |
| 273 | } |
| 274 | ``` |
| 275 | |
| 276 | `identity` is what makes an actor unique in the producer payload. |
| 277 | `merge_identity` is what the Cloud aggregator may use across payloads. |
| 278 | `parent_identity` expresses containment or "lives on" relationships. |
| 279 | |
| 280 | Do not use display names as identity. Identity is canonical matching data, not |
| 281 | UI text. If an identity column is not explicitly listed in the actor type |
| 282 | `presentation.label_policy.columns`, the UI must not use it as a label |
| 283 | fallback. |
| 284 | |
| 285 | Actor type `search` declares exactly what the graph search bar may index for |
| 286 | that actor type. `search.columns[]` references actor-table scalar columns. |
| 287 | `search.label_keys[]` references values in the actor label table, normally |
| 288 | `actor_labels.key`. Set `search.enabled: false` for helper actors that should |
| 289 | not appear in graph search, such as synthetic segment or grouping actors. The |
| 290 | UI must not traverse producer-specific `details`, `match`, `attributes`, or |
| 291 | labels paths when rendering a v1 payload. |
| 292 | |
| 293 | ## Required Link Semantics |
| 294 | |
| 295 | The `links` table is the renderable graph projection. It must contain: |
| 296 | |
| 297 | - `src_actor`: actor row index; |
| 298 | - `dst_actor`: actor row index; |
| 299 | - `type`: link type id; |
| 300 | - enough counters or summary metrics for a graph view, such as |
| 301 | `evidence_count` or `socket_count`. |
| 302 | |
| 303 | Link rows are not required to be one-to-one with raw observations. They should |
| 304 | usually be grouped by graph identity. Evidence rows preserve the details. |
| 305 | |
| 306 | Link types define direction and aggregation semantics: |
| 307 | |
| 308 | ```json |
| 309 | { |
| 310 | "types": { |
| 311 | "link_types": { |
| 312 | "socket": { |
| 313 | "orientation": "directed", |
| 314 | "direction_role": "flow", |
| 315 | "semantic_role": "traffic", |
| 316 | "aggregation": { |
| 317 | "direction": "preserve", |
| 318 | "evidence": "append", |
| 319 | "metrics": { |
| 320 | "socket_count": "sum", |
| 321 | "rtt_ms_max": "max" |
| 322 | } |
| 323 | }, |
| 324 | "evidence_types": ["socket"], |
| 325 | "presentation": { |
| 326 | "label": "Socket", |
| 327 | "color_slot": "primary", |
| 328 | "line_style": "solid", |
| 329 | "width": "normal", |
| 330 | "curve": "auto", |
| 331 | "arrow": "forward", |
| 332 | "layout": { |
| 333 | "strength": "normal", |
| 334 | "distance": "normal" |
| 335 | }, |
| 336 | "variable": { |
| 337 | "channel": "width", |
| 338 | "scale_key": "sockets", |
| 339 | "value_column": "socket_count", |
| 340 | "min": "normal", |
| 341 | "max": "emphasis" |
| 342 | } |
| 343 | } |
| 344 | }, |
| 345 | "l2_adjacency": { |
| 346 | "orientation": "undirected", |
| 347 | "direction_role": "observation", |
| 348 | "aggregation": { |
| 349 | "direction": "canonicalize_unordered", |
| 350 | "evidence": "append" |
| 351 | }, |
| 352 | "evidence_types": ["snmp_l2_observation"] |
| 353 | } |
| 354 | } |
| 355 | } |
| 356 | } |
| 357 | ``` |
| 358 | |
| 359 | Direction rules: |
| 360 | |
| 361 | - `directed` links preserve source and destination order; |
| 362 | - `undirected` links may be canonicalized by sorting endpoints; |
| 363 | - `hierarchical` links express ownership or containment; |
| 364 | - `observed_bidirectional` links preserve observation completeness without |
| 365 | implying traffic direction. |
| 366 | |
| 367 | `direction_role` tells the aggregator and UI what direction means: |
| 368 | |
| 369 | - `flow`: traffic or socket direction; |
| 370 | - `dependency`: logical dependency direction; |
| 371 | - `ownership`: parent to child or owner to owned; |
| 372 | - `observation`: discovery direction only; |
| 373 | - `none`: direction is not meaningful. |
| 374 | |
| 375 | When `types.link_types.<id>.presentation.arrow` is `auto` or omitted, the UI |
| 376 | derives arrows from `orientation` and `direction_role`: |
| 377 | |
| 378 | - `undirected` -> no arrow; |
| 379 | - `observed_bidirectional` -> no arrow; |
| 380 | - `direction_role: none` -> no arrow; |
| 381 | - `direction_role: observation` -> no arrow; |
| 382 | - `directed` with `flow` or `dependency` -> forward from `src_actor` to |
| 383 | `dst_actor`; |
| 384 | - `hierarchical` with `ownership` -> forward from `src_actor` to `dst_actor`; |
| 385 | - all other combinations -> no arrow and a diagnostic if the combination is |
| 386 | schema-valid but semantically unusual. |
| 387 | |
| 388 | `observed_bidirectional` does not mean draw arrows at both ends. Producers that |
| 389 | need reverse or both arrows must set `presentation.arrow` explicitly. |
| 390 | |
| 391 | `direction_role` is required. A link type with `orientation: "directed"` and no |
| 392 | `direction_role` is invalid v1 and must not get an inferred arrow from `auto`. |
| 393 | Declare `direction_role: "flow"` or `direction_role: "dependency"` for directed |
| 394 | links that should infer a forward arrow, or set `presentation.arrow` |
| 395 | explicitly. |
| 396 | |
| 397 | For valid values, the `auto` diagnostic boundary is: |
| 398 | |
| 399 | - normal: `directed+flow`, `directed+dependency`, |
| 400 | `hierarchical+ownership`, `undirected+none`, `undirected+observation`, |
| 401 | `observed_bidirectional+none`, `observed_bidirectional+observation`; |
| 402 | - diagnostic: `directed+none`, `directed+observation`, `directed+ownership`, |
| 403 | `hierarchical+none`, `hierarchical+flow`, `hierarchical+dependency`, |
| 404 | `hierarchical+observation`, `undirected+flow`, `undirected+dependency`, |
| 405 | `undirected+ownership`, `observed_bidirectional+flow`, |
| 406 | `observed_bidirectional+dependency`, `observed_bidirectional+ownership`. |
| 407 | |
| 408 | `semantic_role` is an optional topology-agnostic link classification used by |
| 409 | the UI and aggregator when behavior is not just visual: |
| 410 | |
| 411 | - `normal`: ordinary relationship; |
| 412 | - `discovery`: discovery-protocol relationship such as LLDP/CDP; |
| 413 | - `ownership`: graph-coherence or containment relationship; |
| 414 | - `traffic`: traffic, socket, or data-path relationship; |
| 415 | - `correlation`: visible correlation or partial-correlation relationship; |
| 416 | - `control`: control-plane relationship. |
| 417 | |
| 418 | Do not infer this role from link type names, protocols, or labels. If a link |
| 419 | needs discovery filtering, ownership treatment, traffic emphasis, or |
| 420 | correlation treatment, the producer must declare the semantic role explicitly. |
| 421 | |
| 422 | ## Presentation Plane |
| 423 | |
| 424 | Presentation is compact and backend-controlled, but it is not raw frontend |
| 425 | layout. The UI owns token meanings; producers only choose from documented |
| 426 | tokens. |
| 427 | |
| 428 | Type definitions carry their own presentation: |
| 429 | |
| 430 | - actor types choose label, role, icon token, fill color token, border, |
| 431 | annotation ring, size policy, safe label policy, and graph port-bullet policy; |
| 432 | - link types choose label, color token, line style, width token, curve token, |
| 433 | arrow token, tokenized layout strength/distance, and one optional variable |
| 434 | visual channel; |
| 435 | - port types choose label, color token, and opacity token. |
| 436 | |
| 437 | The graph-level `data.presentation` object carries definitions that are not |
| 438 | owned by one type: legend order, actor-click highlight behavior, port tooltip |
| 439 | field labels, and scale-key labels. |
| 440 | |
| 441 | Graph-level and type-level presentation are complementary. If both exist, there |
| 442 | is no precedence rule to apply: type-level presentation styles actor/link/port |
| 443 | types, while `data.presentation` describes cross-type behavior. |
| 444 | |
| 445 | Example: |
| 446 | |
| 447 | ```json |
| 448 | { |
| 449 | "types": { |
| 450 | "port_types": { |
| 451 | "topology": { |
| 452 | "presentation": { |
| 453 | "label": "Socket", |
| 454 | "color_slot": "primary", |
| 455 | "opacity": "normal" |
| 456 | } |
| 457 | } |
| 458 | } |
| 459 | }, |
| 460 | "presentation": { |
| 461 | "profile_version": "network-connections.v1", |
| 462 | "selection": { |
| 463 | "actor_click": { |
| 464 | "mode": "highlight_connections" |
| 465 | } |
| 466 | }, |
| 467 | "legend": { |
| 468 | "actors": [ |
| 469 | {"type": "process", "label": "Process"} |
| 470 | ], |
| 471 | "links": [ |
| 472 | {"type": "socket", "label": "Socket"} |
| 473 | ], |
| 474 | "ports": [ |
| 475 | {"type": "topology", "label": "Socket"} |
| 476 | ] |
| 477 | }, |
| 478 | "port_fields": [ |
| 479 | {"key": "type", "label": "Type"} |
| 480 | ], |
| 481 | "scale_keys": { |
| 482 | "sockets": { |
| 483 | "label": "Sockets", |
| 484 | "unit": "count" |
| 485 | } |
| 486 | } |
| 487 | } |
| 488 | } |
| 489 | ``` |
| 490 | |
| 491 | Safe label policy is mandatory for polished aggregated graphs. It prevents |
| 492 | canonical identity arrays such as many MAC addresses from becoming actor names. |
| 493 | Use only human-safe scalar columns in `label_policy.columns`; set |
| 494 | `fallback: "type_label"` unless showing row numbers is explicitly useful. |
| 495 | |
| 496 | Link variable scaling is intentionally raw. The producer gives one numeric |
| 497 | `value_column` and one `scale_key`; Cloud or the UI scales visible links that |
| 498 | share the same key. `min` and `max` are visual tokens for the selected channel: |
| 499 | width scaling uses width tokens and opacity scaling uses opacity tokens. Do not |
| 500 | pre-scale to pixels or opacity in the producer. |
| 501 | |
| 502 | Link layout is also tokenized. Producers may set |
| 503 | `types.link_types.<id>.presentation.layout.strength` and `.distance` for any |
| 504 | link type. These are relative hints for the UI force layout, not raw physics |
| 505 | values. The allowed strength tokens are `weakest`, `weaker`, `normal`, |
| 506 | `stronger`, and `strongest`. The allowed distance tokens are `closest`, |
| 507 | `closer`, `normal`, `farther`, and `farthest`. |
| 508 | |
| 509 | Current Netdata topology tuning keeps `strength` at `normal` and varies only |
| 510 | `distance` where a topology needs semantic separation. Do not use non-normal |
| 511 | `strength` tokens for graph polish unless a later product decision explicitly |
| 512 | re-enables force-strength tuning. |
| 513 | |
| 514 | Actor layout is tokenized separately from link layout. Producers may set |
| 515 | `types.actor_types.<id>.presentation.layout.repulsion` to `weakest`, `weaker`, |
| 516 | `normal`, `stronger`, or `strongest`. This controls the relative separation of |
| 517 | actors of that type in the UI force graph. It is not interchangeable with link |
| 518 | `strength`: actor repulsion pushes nodes apart, while link strength pulls link |
| 519 | endpoints together. Producers must not emit raw charge or force numbers. |
| 520 | Initial UI-owned mappings are `weakest=-200`, `weaker=-300`, `normal=-450`, |
| 521 | `stronger=-700`, and `strongest=-1000`; these numbers are not schema and may be |
| 522 | tuned after visual QA. |
| 523 | |
| 524 | Actor size supports: |
| 525 | |
| 526 | - `fixed`: no data-driven size changes; |
| 527 | - `link_count`: size may reflect graph degree; |
| 528 | - `metric`: size comes from a numeric actor row column named by |
| 529 | `metric_column`. |
| 530 | |
| 531 | Actor size may also set `scale: "compact"`, `"normal"`, or `"emphasized"` for |
| 532 | type-level fixed visual emphasis. Use this for deliberate distinctions such as |
| 533 | self/current node emphasis. Do not force the UI to infer emphasis from actor |
| 534 | type names or labels. Initial UI-owned mappings are `compact=0.85`, |
| 535 | `normal=1.0`, and `emphasized=1.18`; these numbers are not schema and may be |
| 536 | tuned after visual QA. `size.scale` composes with `size.mode`; it does not |
| 537 | override data-driven sizing. |
| 538 | |
| 539 | When `selection.actor_click.mode` is `highlight_path`, the payload must also |
| 540 | set `path_table`, `path_actor_column`, and `path_order_column`. The path table |
| 541 | is an actor detail table. The actor column must contain path-member `actor_ref` |
| 542 | values and the order column must be numeric so the UI can render the path |
| 543 | deterministically. If the table carries paths for multiple clicked actors, also |
| 544 | set `path_owner_column` to an `actor_ref` column that identifies the actor whose |
| 545 | click should use that row. |
| 546 | |
| 547 | Port bullets are graph presentation, not modal/table composition. Producers |
| 548 | must define `ports.sources[]` for actor types that set `show_bullets: true`. |
| 549 | Each source says where bullets come from: |
| 550 | |
| 551 | - `links`: read from the graph links table; |
| 552 | - `evidence`: read from a named evidence type or evidence section; |
| 553 | - `actor_table`: read from a named actor detail table when present. |
| 554 | |
| 555 | Each source must define `actor_column` and `name_column`. Optional |
| 556 | `value_column`, `type_column`, `status_column`, `mode_column`, `role_column`, |
| 557 | and `sources_column` enrich bullet multiplicity, color, and tooltip fields. |
| 558 | `value_column` must reference a numeric source-table column; the UI sums it per |
| 559 | bullet key and uses the sum for visible bullet count, overflow, and sizing. Use |
| 560 | it when an aggregated row represents several observations, such as several |
| 561 | sockets on the same process port. `default_type` must reference |
| 562 | `types.port_types` and is used when `type_column` is absent or empty. |
| 563 | `name_column` must reference a scalar display column, not raw actor/link/ |
| 564 | evidence references, arrays, or JSON cells. For evidence sources, `evidence` |
| 565 | names an evidence type id. |
| 566 | |
| 567 | `hover.fields` is intentionally lightweight graph hover metadata. Full modal |
| 568 | and table composition lives in `types.actor_types.<id>.presentation.modal`, |
| 569 | `types.link_types.<id>.presentation.modal`, and optional |
| 570 | `types.table_types.<id>.presentation`. |
| 571 | |
| 572 | `annotation` is a small actor marker such as a ring or dot. It must use closed |
| 573 | color slots and style tokens; it is not a free-form badge or CSS hook. |
| 574 | |
| 575 | Do not emit raw SVG icons. Use the closed UI icon token vocabulary and request |
| 576 | a new UI token when a producer needs a new visual concept. |
| 577 | |
| 578 | ### Closed Token Vocabulary |
| 579 | |
| 580 | Color slots: |
| 581 | |
| 582 | `primary`, `secondary`, `accent`, `self`, `neutral`, `muted`, `dim`, `derived`, |
| 583 | `info`, `structural`, `warning`, `success`, `danger`, `blue`, `green`, |
| 584 | `orange`, `purple`, `cyan`, `yellow`, `teal`, `gray`. |
| 585 | |
| 586 | Prefer semantic slots such as `primary`, `warning`, `derived`, or `structural` |
| 587 | when they fit. Use hue slots only when a topology needs distinct categories that |
| 588 | do not map to the semantic slots, such as the vSphere migration. |
| 589 | |
| 590 | Opacity tokens: |
| 591 | |
| 592 | `normal`, `muted`, `faded`. |
| 593 | |
| 594 | Width tokens: |
| 595 | |
| 596 | `thin`, `normal`, `thick`, `emphasis`. |
| 597 | |
| 598 | Icon tokens: |
| 599 | |
| 600 | `router`, `switch`, `firewall`, `access_point`, `server`, `storage`, |
| 601 | `load_balancer`, `printer`, `phone`, `ups`, `camera`, `process`, `agent`, |
| 602 | `netdata-agent`, `parent`, `remote-endpoint`, `local-endpoint`, `segment`, |
| 603 | `self`, `ip`, `cloud`, `container`, `vm`, `database`, `service`, `datacenter`, |
| 604 | `cluster`, `host`, `network`, `datastore`, `datastore_cluster`, |
| 605 | `resource_pool`, `device`, `endpoint`, `correlation`, `interface`, `group`, |
| 606 | `unknown`. |
| 607 | |
| 608 | Producers must not emit tokens outside the schema. The UI should render |
| 609 | unsupported tokens with a safe fallback and record diagnostics for version skew. |
| 610 | Producers must not emit raw SVG icons or ask the UI to infer icons from |
| 611 | capability strings. If a topology needs a new visual concept, add a closed icon |
| 612 | token to the schema and UI token map first. |
| 613 | |
| 614 | ## Evidence Plane |
| 615 | |
| 616 | Evidence sections are keyed by evidence type. Each evidence row must reference |
| 617 | the graph link it supports using a `link_ref` column. |
| 618 | |
| 619 | Example socket evidence type: |
| 620 | |
| 621 | ```json |
| 622 | { |
| 623 | "types": { |
| 624 | "evidence_types": { |
| 625 | "socket": { |
| 626 | "link_type": "socket", |
| 627 | "role": "relationship_evidence", |
| 628 | "match_columns": [ |
| 629 | "client_ip", |
| 630 | "client_port", |
| 631 | "server_ip", |
| 632 | "server_port", |
| 633 | "protocol" |
| 634 | ], |
| 635 | "columns": [ |
| 636 | {"id": "link", "type": "link_ref", "role": "reference"}, |
| 637 | {"id": "src_actor", "type": "actor_ref", "role": "reference"}, |
| 638 | {"id": "dst_actor", "type": "actor_ref", "role": "reference"}, |
| 639 | { |
| 640 | "id": "client_ip", |
| 641 | "type": "ip_ref", |
| 642 | "dictionary": "strings", |
| 643 | "role": "group_key" |
| 644 | }, |
| 645 | {"id": "client_port", "type": "uint", "role": "group_key"}, |
| 646 | { |
| 647 | "id": "server_ip", |
| 648 | "type": "ip_ref", |
| 649 | "dictionary": "strings", |
| 650 | "role": "group_key" |
| 651 | }, |
| 652 | {"id": "server_port", "type": "uint", "role": "group_key"}, |
| 653 | { |
| 654 | "id": "protocol", |
| 655 | "type": "string_ref", |
| 656 | "dictionary": "strings", |
| 657 | "role": "group_key" |
| 658 | }, |
| 659 | {"id": "namespace", "type": "string_ref", "dictionary": "strings"}, |
| 660 | {"id": "rtt_ms_max", "type": "float", "unit": "ms", "aggregation": "max"} |
| 661 | ] |
| 662 | } |
| 663 | } |
| 664 | } |
| 665 | } |
| 666 | ``` |
| 667 | |
| 668 | Evidence rows are the lossless plane for topology relationships. If Cloud must |
| 669 | cross-match sockets across nodes, every socket evidence row must remain |
| 670 | available. The graph link can be highly aggregated while the evidence rows stay |
| 671 | one-per-observation. |
| 672 | |
| 673 | ## Detail Tables |
| 674 | |
| 675 | Detail tables support actor modals and drilldowns. |
| 676 | |
| 677 | There are four roles: |
| 678 | |
| 679 | - `relationship_evidence`: exact relationship rows, usually backed by an |
| 680 | evidence section; |
| 681 | - `relationship_summary`: aggregated relationship rows derived from evidence; |
| 682 | - `actor_detail`: actor-owned custom data that is not generally aggregatable; |
| 683 | - `actor_inventory`: actor-owned inventory data that may be appended or set |
| 684 | merged. |
| 685 | |
| 686 | Streaming `stream_path` is actor detail. It must not be treated as a link |
| 687 | evidence table unless each row is also a relationship proof. |
| 688 | |
| 689 | Some actor-detail data is intentionally producer-specific and not generally |
| 690 | aggregatable. Use table role `actor_detail` with aggregation `append` or |
| 691 | `none`; use column type `json` only for cells that need nested objects or |
| 692 | arrays that cannot be represented as scalar columns. |
| 693 | |
| 694 | Example actor custom table type: |
| 695 | |
| 696 | ```json |
| 697 | { |
| 698 | "types": { |
| 699 | "table_types": { |
| 700 | "stream_path": { |
| 701 | "role": "actor_detail", |
| 702 | "owner": "actor", |
| 703 | "aggregation": "append", |
| 704 | "columns": [ |
| 705 | {"id": "actor", "type": "actor_ref", "role": "reference"}, |
| 706 | {"id": "hop", "type": "uint"}, |
| 707 | {"id": "node_id", "type": "string_ref", "dictionary": "strings"}, |
| 708 | {"id": "since", "type": "timestamp"} |
| 709 | ] |
| 710 | } |
| 711 | } |
| 712 | } |
| 713 | } |
| 714 | ``` |
| 715 | |
| 716 | Do not duplicate evidence in actor-owned tables. If a modal needs a socket list |
| 717 | for an actor, derive it from socket evidence by filtering evidence rows whose |
| 718 | link touches that actor. |
| 719 | |
| 720 | ## Actor Labels And Modal Composition |
| 721 | |
| 722 | Actor modals are composed from existing topology facts. They do not get a |
| 723 | second copy of actor attributes, socket rows, SNMP endpoint objects, or |
| 724 | relationship evidence only for display. |
| 725 | |
| 726 | Every actor modal has four top-level entities: |
| 727 | |
| 728 | - actor name from the actor row through `presentation.label_policy`; |
| 729 | - actor labels from an actor-owned `actor_labels` table when labels exist; |
| 730 | - a depth-1 topology miniature built from existing incident links and opposite |
| 731 | actors; |
| 732 | - one or more table sections built from actors, links, evidence, or detail |
| 733 | tables. |
| 734 | |
| 735 | Use a compact actor-owned label table for display labels and metadata: |
| 736 | |
| 737 | ```json |
| 738 | { |
| 739 | "types": { |
| 740 | "table_types": { |
| 741 | "actor_labels": { |
| 742 | "role": "actor_inventory", |
| 743 | "owner": "actor", |
| 744 | "aggregation": "set", |
| 745 | "columns": [ |
| 746 | {"id": "actor", "type": "actor_ref", "role": "reference"}, |
| 747 | {"id": "key", "type": "string_ref", "dictionary": "strings"}, |
| 748 | {"id": "value", "type": "string_ref", "dictionary": "strings"}, |
| 749 | { |
| 750 | "id": "source", |
| 751 | "type": "string_ref", |
| 752 | "dictionary": "strings", |
| 753 | "nullable": true |
| 754 | }, |
| 755 | { |
| 756 | "id": "kind", |
| 757 | "type": "string_ref", |
| 758 | "dictionary": "strings", |
| 759 | "nullable": true |
| 760 | }, |
| 761 | {"id": "value_index", "type": "uint", "nullable": true} |
| 762 | ] |
| 763 | } |
| 764 | } |
| 765 | }, |
| 766 | "tables": { |
| 767 | "actor": { |
| 768 | "actor_labels": { |
| 769 | "type": "actor_labels", |
| 770 | "table": {"rows": 0, "columns": [], "values": []} |
| 771 | } |
| 772 | } |
| 773 | } |
| 774 | } |
| 775 | ``` |
| 776 | |
| 777 | The `key`, `value`, `source`, and `kind` columns may use either `string` or |
| 778 | `string_ref` encoding. Producers should prefer `string_ref` when a local |
| 779 | dictionary is already used, but aggregators and UI adapters must treat both |
| 780 | encodings as the same logical label fields. |
| 781 | |
| 782 | Host/node actors should expose the complete host label set when available. |
| 783 | Non-node actors should expose all useful producer-known labels and metadata, |
| 784 | such as process command line, user, group, namespace, interface role, or |
| 785 | virtualization object properties. If a fact is needed for identity, |
| 786 | correlation, grouping, sorting, filtering, or aggregation, keep it as a typed |
| 787 | canonical actor/evidence/detail column too; `actor_labels` is not a replacement |
| 788 | for canonical data. |
| 789 | |
| 790 | Repeated label values are repeated rows with the same `actor` and `key`, |
| 791 | ordered by `value_index`. Do not encode repeated labels as raw JSON arrays for |
| 792 | normal modal display. |
| 793 | |
| 794 | `actor_labels` inherits the topology Function sensitive-data classification. |
| 795 | Labels may include command lines, users, host labels, system contact/location |
| 796 | fields, or other operator-controlled metadata. Cloud services, aggregators, and |
| 797 | UI adapters that consume topology payloads must apply the same access-control |
| 798 | assumptions as the source Function. |
| 799 | |
| 800 | Use `modal.labels.identification.fields[]` to select the small ordered subset |
| 801 | of label keys that belongs in the actor modal identification/header area. This |
| 802 | selection references the existing `actor_labels` table; it must not duplicate |
| 803 | values into a separate modal-only table. Missing selected keys are skipped, and |
| 804 | the full Labels tab remains complete. |
| 805 | |
| 806 | Modal sections are recipes over existing tables: |
| 807 | |
| 808 | ```json |
| 809 | { |
| 810 | "types": { |
| 811 | "actor_types": { |
| 812 | "process": { |
| 813 | "layer": "process", |
| 814 | "identity": ["node_id", "process_name"], |
| 815 | "presentation": { |
| 816 | "label": "Process", |
| 817 | "label_policy": { |
| 818 | "columns": ["display_name", "process_name"], |
| 819 | "fallback": "type_label", |
| 820 | "array": "reject" |
| 821 | }, |
| 822 | "modal": { |
| 823 | "labels": { |
| 824 | "enabled": true, |
| 825 | "table": "actor_labels", |
| 826 | "identification": { |
| 827 | "fields": [ |
| 828 | {"key": "process", "label": "Process", "max_values": 1}, |
| 829 | {"key": "username", "label": "User", "max_values": 1} |
| 830 | ] |
| 831 | } |
| 832 | }, |
| 833 | "mini_topology": { |
| 834 | "enabled": true, |
| 835 | "depth": 1, |
| 836 | "exclude_link_types": ["ownership"] |
| 837 | }, |
| 838 | "sections": [ |
| 839 | { |
| 840 | "id": "connections", |
| 841 | "label": "Connections", |
| 842 | "source": {"kind": "links"}, |
| 843 | "owner_filter": { |
| 844 | "mode": "incident_link", |
| 845 | "src_actor_column": "src_actor", |
| 846 | "dst_actor_column": "dst_actor" |
| 847 | }, |
| 848 | "row_filters": [ |
| 849 | {"column": "type", "op": "not_in", "values": ["ownership"]} |
| 850 | ], |
| 851 | "columns": [ |
| 852 | { |
| 853 | "id": "remote", |
| 854 | "label": "Remote", |
| 855 | "projection": { |
| 856 | "kind": "opposite_actor", |
| 857 | "src_actor_column": "src_actor", |
| 858 | "dst_actor_column": "dst_actor" |
| 859 | }, |
| 860 | "cell": "actor_link" |
| 861 | }, |
| 862 | { |
| 863 | "id": "protocol", |
| 864 | "label": "Protocol", |
| 865 | "projection": {"kind": "direct", "column": "protocol"}, |
| 866 | "cell": "badge" |
| 867 | }, |
| 868 | { |
| 869 | "id": "sockets", |
| 870 | "label": "Sockets", |
| 871 | "projection": {"kind": "direct", "column": "socket_count"}, |
| 872 | "cell": "number" |
| 873 | } |
| 874 | ] |
| 875 | } |
| 876 | ] |
| 877 | } |
| 878 | } |
| 879 | } |
| 880 | } |
| 881 | } |
| 882 | } |
| 883 | ``` |
| 884 | |
| 885 | Table recipes must support these source kinds: |
| 886 | |
| 887 | - `actors`; |
| 888 | - `links`; |
| 889 | - `evidence` with an `evidence` id; |
| 890 | - `actor_table` with a `table` id; |
| 891 | - `relationship_table` with a `table` id. |
| 892 | |
| 893 | Use `owner_filter` to bind rows to the selected actor or link. Common filters |
| 894 | are `actor_column`, `link_column`, `incident_link`, `incident_evidence`, and |
| 895 | `selected_link`. |
| 896 | |
| 897 | Use projections instead of duplicated display fields: |
| 898 | |
| 899 | - `direct`: read a column from the source row; |
| 900 | - `actor_ref_label`: render an actor-ref column through actor label policy; |
| 901 | - `opposite_actor`: render the opposite endpoint of a link row; |
| 902 | - `formatted_endpoint`: combine IP, port, and optional protocol columns; |
| 903 | - `selected_side_endpoint`: choose local or remote endpoint fields based on |
| 904 | whether the selected actor matches the source or destination actor columns; |
| 905 | producers must provide `src_actor_column` and `dst_actor_column` plus at |
| 906 | least one local endpoint column and one remote endpoint column; |
| 907 | - `label_lookup`: read a value from `actor_labels` by `label_key`; omit |
| 908 | `actor_column` to look up labels for the selected modal actor, or provide an |
| 909 | actor-ref source column when the lookup belongs to another actor in the row; |
| 910 | - `coalesce`: choose the first non-empty column; |
| 911 | - `json_path`: extract a declared scalar `path` from a declared JSON `column`; |
| 912 | - `const`: emit a fixed value. |
| 913 | |
| 914 | Use cell types to keep rendering generic and polished: |
| 915 | |
| 916 | - `text`; |
| 917 | - `number`; |
| 918 | - `badge`; |
| 919 | - `actor_link`; |
| 920 | - `timestamp`; |
| 921 | - `duration`; |
| 922 | - `endpoint`; |
| 923 | - `array_count`; |
| 924 | - `debug_json`. |
| 925 | |
| 926 | Use visibility annotations instead of duplicating rows: |
| 927 | |
| 928 | - `table`: shown in the normal table; |
| 929 | - `expanded`: hidden from the main grid but shown when the row expands; |
| 930 | - `hidden`: available for joins, sorting, or future use, not displayed; |
| 931 | - `debug`: diagnostic-only. Raw JSON belongs here unless a curated scalar |
| 932 | projection exists. |
| 933 | |
| 934 | `json` columns are allowed only when they preserve nested producer-owned facts |
| 935 | that the UI or aggregator understands through declared projections, or when |
| 936 | they are explicitly marked as `debug_json`. They are not acceptable as the |
| 937 | normal user-facing rendering for labels, endpoint objects, neighbor arrays, or |
| 938 | actor attributes. |
| 939 | |
| 940 | Use `empty_label` for the section-level empty-state label. Use column |
| 941 | `badge_map` only for explicit value-to-token mapping, and keep `align` and |
| 942 | `sortable` as presentation hints over already-projected columns. These fields |
| 943 | must not add new data or embed UI components. |
| 944 | |
| 945 | The Cloud aggregator should preserve and merge modal/table definitions by the |
| 946 | same namespace/deduplicate rules used for type presentation. It should not |
| 947 | materialize modal rows during aggregation unless it is already merging the |
| 948 | underlying canonical table. |
| 949 | |
| 950 | ## Telemetry Overlays |
| 951 | |
| 952 | Topology should be refreshable without recomputing topology. Overlay templates |
| 953 | define how the UI or Cloud can query metrics for an actor or link. |
| 954 | |
| 955 | Templates live once in the type registry. Overlay refs carry only template ids, |
| 956 | one owner reference, and selector parameters. |
| 957 | |
| 958 | For `provider: "netdata.metrics"`, selector params are interpreted by the |
| 959 | consumer: |
| 960 | |
| 961 | - `node_id` scopes the metric query to a node; |
| 962 | - `collect_job` maps to the chart label `_collect_job`; |
| 963 | - other params map to same-named chart labels. |
| 964 | |
| 965 | Example: |
| 966 | |
| 967 | ```json |
| 968 | { |
| 969 | "types": { |
| 970 | "overlay_templates": { |
| 971 | "snmp_interface_traffic": { |
| 972 | "provider": "netdata.metrics", |
| 973 | "contexts": ["snmp.interface_traffic"], |
| 974 | "dimensions": ["received", "sent"], |
| 975 | "selector_params": ["node_id", "if_name"], |
| 976 | "merge": { |
| 977 | "refs": "set", |
| 978 | "values": "sum" |
| 979 | } |
| 980 | } |
| 981 | } |
| 982 | }, |
| 983 | "overlays": { |
| 984 | "refs": { |
| 985 | "rows": 1, |
| 986 | "columns": [ |
| 987 | {"id": "template", "type": "string_ref", "dictionary": "strings"}, |
| 988 | {"id": "link", "type": "link_ref", "role": "reference"}, |
| 989 | {"id": "node_id", "type": "string_ref", "dictionary": "strings"}, |
| 990 | {"id": "if_name", "type": "string_ref", "dictionary": "strings"} |
| 991 | ], |
| 992 | "values": [ |
| 993 | {"codec": "const", "value": 0}, |
| 994 | {"codec": "const", "value": 0}, |
| 995 | {"codec": "const", "value": 1}, |
| 996 | {"codec": "const", "value": 2} |
| 997 | ] |
| 998 | } |
| 999 | } |
| 1000 | } |
| 1001 | ``` |
| 1002 | |
| 1003 | Overlay refs use schema ids for column names, so selector params and ref column |
| 1004 | ids must start with a letter and contain only letters, digits, `_`, `.`, `:`, |
| 1005 | or `-`. The `template` column identifies the template by name. Exactly one |
| 1006 | convention owner column must be present: `actor` with type `actor_ref` or `link` |
| 1007 | with type `link_ref`. No other `actor_ref` or `link_ref` columns are valid in |
| 1008 | overlay refs. Every row must have a non-null value in the owner column. Each |
| 1009 | selector param used by a referenced template must have a same-named refs-table |
| 1010 | column. |
| 1011 | Selector params must not use the reserved refs-table convention column names |
| 1012 | `template`, `actor`, or `link`. |
| 1013 | |
| 1014 | The `template` column and selector-param columns required by a row's resolved |
| 1015 | template must be `string` or `string_ref`. Required selector-param values must |
| 1016 | resolve to non-empty strings. Selector-param columns used by other templates may |
| 1017 | be nullable and null on rows whose template does not require them. |
| 1018 | |
| 1019 | Overlay refs are optional. Do not fabricate per-link bandwidth if the producer |
| 1020 | does not have it. For network sockets, current evidence may include snapshot |
| 1021 | metrics such as RTT or retransmissions, but that is not a time-series overlay |
| 1022 | unless a query provider exists. |
| 1023 | |
| 1024 | ## Aggregation Rules |
| 1025 | |
| 1026 | Aggregation must be schema-driven: |
| 1027 | |
| 1028 | - actor type identity defines what actors can merge; |
| 1029 | - link type direction policy defines whether edge direction is preserved; |
| 1030 | - evidence type match columns define which details must remain exact; |
| 1031 | - column aggregation rules define how summaries are produced. |
| 1032 | |
| 1033 | Common column rules: |
| 1034 | |
| 1035 | - `set`: preserve unique values; |
| 1036 | - `sum`: add counters; |
| 1037 | - `min` / `max`: preserve bounds; |
| 1038 | - `last` / `first`: pick by observation order; |
| 1039 | - `count`: count rows; |
| 1040 | - `none`: not aggregatable. |
| 1041 | |
| 1042 | If a table or column has no valid aggregation, mark it `none` and keep rows |
| 1043 | attached to their owner. Do not silently drop rows. |
| 1044 | |
| 1045 | ## Correlation Plane |
| 1046 | |
| 1047 | Correlation is producer-visible graph semantics, not aggregator state. Producers |
| 1048 | declare how independently produced topology maps can be correlated by keys, but |
| 1049 | they do not expose internal aggregator states such as candidate, absorbed, |
| 1050 | rewrite plan, or equivalence class in final payloads. |
| 1051 | |
| 1052 | Correlation can resolve several shapes: |
| 1053 | |
| 1054 | - loose relationship sides, where one side of a detailed row has endpoint facts |
| 1055 | but no known actor; |
| 1056 | - visible correlation actors, where the input graph intentionally materializes |
| 1057 | unresolved peers; |
| 1058 | - weaker placeholder actors that should be replaced by stronger managed actors; |
| 1059 | - equivalent actors that should be merged and enriched with facts from multiple |
| 1060 | payloads. |
| 1061 | |
| 1062 | Use `data.correlation` when a topology can resolve correlation actors across |
| 1063 | payloads: |
| 1064 | |
| 1065 | - `rules` defines named correlation rules, priority, key space, key template, |
| 1066 | rule class, action, point actor types when visible points exist, optional |
| 1067 | claim actor types, correlation link types, and the final output link type; |
| 1068 | - `points` is a compact table of visible correlation actors and their keys; |
| 1069 | - `claims` is a compact table of real actors and the keys they satisfy. |
| 1070 | |
| 1071 | Correlation keys are declarative. A key is built from table columns and string |
| 1072 | literals. The aggregator normalizes values by column type and concatenates the |
| 1073 | parts; it does not execute code and does not need to know what IP, port, MAC, |
| 1074 | chassis id, vSphere MOID, or another domain value means. |
| 1075 | |
| 1076 | Supported rule classes: |
| 1077 | |
| 1078 | - `resolve_loose_side`: resolve a loose relationship side or visible |
| 1079 | correlation actor to a claim actor when the key is unambiguous; |
| 1080 | - `replace_actor`: remove weaker placeholder actors and rewire incident |
| 1081 | relationships to stronger managed actors; |
| 1082 | - `merge_enrich_actor`: merge actors with the same declared identity and merge |
| 1083 | their labels, attributes, evidence, and detail tables by table policy. |
| 1084 | |
| 1085 | Supported rule actions remain the visible output behavior: |
| 1086 | |
| 1087 | - `absorb`: on an exact unambiguous match, matching correlation actors are |
| 1088 | removed from the aggregated output, or loose-side placeholders are consumed, |
| 1089 | and incident correlation relationships are rewired to the matched real actor |
| 1090 | using `output_link_type`; |
| 1091 | - `link`: on an unambiguous broader/partial match, the correlation actor remains |
| 1092 | visible, or a materialized partial actor remains visible, and a weak |
| 1093 | correlation link to the matched actor is emitted using `output_link_type`. |
| 1094 | |
| 1095 | No match keeps the correlation actor visible. Ambiguous matches must stay |
| 1096 | unresolved and produce diagnostics in the aggregator; producers must not encode |
| 1097 | guessing policy in the schema. |
| 1098 | |
| 1099 | NAT or other alias evidence can be represented by adding more point or claim |
| 1100 | rows for the same actor and rule. The original key remains intact; aliases are |
| 1101 | additional facts, not mutations of the source observation. |
| 1102 | |
| 1103 | Links between real actors and visible correlation actors must use semantic |
| 1104 | correlation link types even in a single-node topology. The UI and aggregator |
| 1105 | must not infer correlation behavior from actor type names or topology kind. The |
| 1106 | legend should include visible correlation actor and link types so users can |
| 1107 | distinguish unresolved, partial, inferred, and resolved relationships. |
| 1108 | |
| 1109 | ## Network Connections Shape |
| 1110 | |
| 1111 | For network-connections, graph links must be split into semantic families: |
| 1112 | |
| 1113 | - node-to-process ownership links that keep the graph together; |
| 1114 | - resolved process-to-process links where both process endpoints are already |
| 1115 | known; |
| 1116 | - process-to-correlation-endpoint links for unresolved or cross-node socket |
| 1117 | endpoints. |
| 1118 | |
| 1119 | Socket evidence preserves exact tuples. |
| 1120 | |
| 1121 | Canonical socket evidence columns: |
| 1122 | |
| 1123 | - graph link reference; |
| 1124 | - source actor reference; |
| 1125 | - destination actor reference; |
| 1126 | - client IP; |
| 1127 | - client port; |
| 1128 | - server IP; |
| 1129 | - server port; |
| 1130 | - protocol; |
| 1131 | - TCP state when available; |
| 1132 | - network namespace or address-space tags; |
| 1133 | - optional snapshot metrics such as RTT, receive RTT, retransmissions, and |
| 1134 | socket count. |
| 1135 | |
| 1136 | Do not emit these per socket: |
| 1137 | |
| 1138 | - repeated display strings or labels; |
| 1139 | - `port_name` if it can be derived from port; |
| 1140 | - duplicated endpoint objects; |
| 1141 | - actor labels repeated as evidence labels; |
| 1142 | - actor modal socket rows. |
| 1143 | |
| 1144 | In the measured Cloud corpus, this production-only socket evidence shape was |
| 1145 | about 7.25 MB raw for 323,077 socket evidence rows, or about 11.25 MB raw when |
| 1146 | including current RTT/retransmission metrics. |
| 1147 | |
| 1148 | Network-connections graph direction is dependency direction: link types use |
| 1149 | `direction_role: "dependency"`, the client actor is always `src_actor`, and the |
| 1150 | server actor is always `dst_actor`. The topology payload must not expose a |
| 1151 | separate `local` direction. Local host or same-node sockets still resolve to |
| 1152 | either inbound or outbound dependency direction, and same-node process pairs |
| 1153 | should emit resolved process-to-process links when both actors are known. |
| 1154 | |
| 1155 | For outbound sockets, the process claims the client `protocol + client_ip + |
| 1156 | client_port` tuple and the correlation endpoint points at the server `protocol + |
| 1157 | server_ip + server_port` tuple. For inbound sockets, the process claims the |
| 1158 | server tuple and the correlation endpoint points at the client tuple. Listening |
| 1159 | sockets have no remote correlation point. |
| 1160 | The current `socket_exact` rule key is `protocol + address_space + ip + port`; |
| 1161 | `address_space` prevents private or otherwise scoped endpoint identities from |
| 1162 | matching across unrelated address domains. |
| 1163 | |
| 1164 | Network-connections uses four link types with different graph semantics: |
| 1165 | |
| 1166 | - `endpoint_socket`: process-to-correlation-endpoint links. These are the |
| 1167 | primary unresolved network dependencies in a single-node view. They should be |
| 1168 | solid, colored, thin, normal-strength, and normal-distance so unresolved |
| 1169 | endpoints do not force the graph to zoom out. |
| 1170 | - `correlated_socket`: aggregator output after exact endpoint absorption. These |
| 1171 | are cross-payload process-to-process dependencies and should be |
| 1172 | normal-strength and farthest so independent topology clusters do not blend |
| 1173 | into one dense layout. |
| 1174 | - `socket`: resolved local process-to-process socket links. These are gray, |
| 1175 | thin, normal-distance links whose width can vary by `socket_count`. |
| 1176 | - `ownership`: node-to-process containment links. These are graph-coherence |
| 1177 | links, not network traffic. They should be dotted, faded/dim, thin, normal |
| 1178 | strength, and normal distance. |
| 1179 | |
| 1180 | Aggregated network-connections payloads still need process port bullets without |
| 1181 | shipping detailed socket evidence. Emit an `actor_inventory` table such as |
| 1182 | `socket_ports` with `actor`, `port`, and numeric `socket_count`, then point the |
| 1183 | process actor type's `presentation.ports.sources[]` at that actor table with |
| 1184 | `value_column: "socket_count"`. Size process actors with |
| 1185 | `presentation.size: {"mode": "metric", "metric_column": "socket_count"}`. |
| 1186 | |
| 1187 | Network-connections actor modals should expose dependency semantics, not every |
| 1188 | internal table as a peer tab: |
| 1189 | |
| 1190 | - self/node actors: `Processes` over `links`, filtered to `type == ownership`; |
| 1191 | - non-node actors in aggregated mode: `Dependencies` and `Dependants` over |
| 1192 | `tables.relationship.connections`; |
| 1193 | - non-node actors in detailed mode: `Dependencies` and `Dependants` over |
| 1194 | `evidence.socket`; |
| 1195 | - `Dependencies` filters rows where the selected actor is `src_actor`; |
| 1196 | - `Dependants` filters rows where the selected actor is `dst_actor`. |
| 1197 | |
| 1198 | Do not show `socket_ports` as a normal network-connections modal section. It is |
| 1199 | only the graph port-bullet inventory. Put less common per-connection fields such |
| 1200 | as retransmissions or receiver RTT behind `visibility: "expanded"` instead of |
| 1201 | creating another duplicate tab. |
| 1202 | |
| 1203 | ## Streaming Shape |
| 1204 | |
| 1205 | For streaming: |
| 1206 | |
| 1207 | - actors are Netdata agents or streaming endpoints; |
| 1208 | - links are parent/child or replication relationships; |
| 1209 | - direction is meaningful as hierarchy/ownership, not packet flow; |
| 1210 | - `stream_path` is an actor-detail table, not relationship evidence unless a |
| 1211 | row proves a specific relationship. |
| 1212 | |
| 1213 | Streaming custom tables are allowed because they carry actor-owned state that |
| 1214 | cannot be derived from links. |
| 1215 | |
| 1216 | Streaming parent actor size should use the actor row `retained_node_count` |
| 1217 | metric: `presentation.size: |
| 1218 | {"mode":"metric","metric_column":"retained_node_count"}`. Do not use generic |
| 1219 | graph degree or direct child count for parent sizing; the operational question |
| 1220 | is how many nodes have data retained by the parent, including self, virtual |
| 1221 | nodes, stale nodes, and transit descendants when they have DB retention state. |
| 1222 | Parent graph bullets should be derived from incoming streaming links by using a |
| 1223 | `ports.sources[]` entry over `links` with `actor_column: "dst_actor"` and a |
| 1224 | scalar child/node display column such as `port_name`. |
| 1225 | |
| 1226 | Streaming modal identification should be role-specific. Host-like actors |
| 1227 | (`parent`, `child`, and `stale`) should use compact status plus OS/hardware/ |
| 1228 | platform labels from `actor_labels`: hostname, node type, health, stream, |
| 1229 | ingest, OS, OS version, kernel, architecture, CPU, cores, RAM, virtualization, |
| 1230 | container, cloud placement, and Agent version. Parents also include retained |
| 1231 | nodes and direct children. Vnode actors should use inventory labels such as |
| 1232 | vnode type, vendor, model, address, location, sys object id, LLDP name, and |
| 1233 | status. Keep long identifiers such as machine GUID and node id available in the |
| 1234 | full Labels tab instead of promoting them into the header. |
| 1235 | |
| 1236 | Streaming detail tables can contain stable node or Cloud identifiers. These |
| 1237 | fields must not be used as graph labels and must not be copied into logs, |
| 1238 | diagnostics, docs, SOWs, or durable review artifacts without redaction. |
| 1239 | |
| 1240 | ## SNMP/L2 Shape |
| 1241 | |
| 1242 | For SNMP/L2: |
| 1243 | |
| 1244 | - actors are devices, interfaces, VLANs, bridge domains, and endpoints; |
| 1245 | - links are L2 adjacencies, containment, forwarding evidence, and ownership; |
| 1246 | - adjacency links are usually `undirected` or `observed_bidirectional`; |
| 1247 | - discovery direction is observation metadata and should not prevent |
| 1248 | aggregation of the same physical adjacency; |
| 1249 | - FDB/ARP/LLDP/CDP rows are evidence sections or actor inventory tables, |
| 1250 | depending on whether each row proves a relationship. |
| 1251 | |
| 1252 | Managed SNMP device actor modals are port-centric: |
| 1253 | |
| 1254 | - use `modal.labels.identification.fields[]` to show key device labels such as |
| 1255 | device name, management IP, vendor, model, port counts, and LLDP/CDP counts |
| 1256 | in the modal identification area; |
| 1257 | - use `actor_ports` as the primary `Ports` section; |
| 1258 | - expose real port identity columns, including SNMP `if_index` as the visible |
| 1259 | numeric port ID when known, source `port_id`, display `name`, `if_name`, |
| 1260 | `if_descr`, `if_alias`, MAC, speed, status, mode, role, VLAN, FDB, link, and |
| 1261 | neighbor counts; |
| 1262 | - never invent numeric port IDs. Do not use row order or any generated sequence; |
| 1263 | `if_index` must come from the device/SNMP facts; |
| 1264 | - include compact expanded-row neighbor columns such as nullable |
| 1265 | `neighbor_actor` and `neighbor_port_name` when graph-link facts can align the |
| 1266 | port to a remote actor; |
| 1267 | - use `actor_port_links` as the `Port Neighbors` section when device modal |
| 1268 | rows need remote actor/port/evidence details; |
| 1269 | - keep endpoint, segment, and custom actors on a generic graph-link `Links` |
| 1270 | section when they do not own port inventory. |
| 1271 | |
| 1272 | `actor_port_links` is an actor-owned modal index over existing graph links and |
| 1273 | evidence. It may duplicate compact references and side-specific port facts so |
| 1274 | the UI can show a device's local port next to its remote actor, but it must not |
| 1275 | duplicate raw LLDP/CDP/FDB/ARP/STP evidence JSON. Every row should include the |
| 1276 | selected actor, link reference, remote actor, local `if_index`, local port name, |
| 1277 | remote port facts, protocol, link type, state, evidence count, confidence, |
| 1278 | inference, attachment mode, and timestamps when known. |
| 1279 | |
| 1280 | SNMP endpoint port names must come only from real port fields: `port_name`, |
| 1281 | `if_name`, `if_descr`, or source `port_id`. Do not fall back to actor labels |
| 1282 | such as `display_name` or `sys_name`; those belong in actor labels, not in |
| 1283 | local/remote port cells. |
| 1284 | |
| 1285 | SNMP interface traffic, packets, errors, and state should use overlay |
| 1286 | templates with compact refs to node/context/label selectors. |
| 1287 | |
| 1288 | ## vSphere Shape |
| 1289 | |
| 1290 | For vSphere: |
| 1291 | |
| 1292 | - actors should use stable vSphere managed object ids when available; |
| 1293 | - inventory path can be an additional merge identity but should not be the |
| 1294 | only identity when a stable object id exists; |
| 1295 | - containment links such as datacenter -> cluster -> host -> VM are |
| 1296 | `hierarchical` with `direction_role: "ownership"`; |
| 1297 | - VM-to-host relationships are topology facts, not metric overlays; |
| 1298 | - metrics for datastore, host, or VM utilization should be overlay templates |
| 1299 | when the UI needs refreshable values. |
| 1300 | |
| 1301 | The vSphere producer in the separate worktree must be updated in place only |
| 1302 | after coordination with the user because another agent may be editing it. |
| 1303 | |
| 1304 | ## Test Reconstruction |
| 1305 | |
| 1306 | Compatibility tests may project the new canonical payload into rollout adapter |
| 1307 | shapes to prove that no information needed by compatibility consumers was lost. |
| 1308 | |
| 1309 | That projection code must live only in tests or local schema-lab tools. It may |
| 1310 | hardcode adapter presentation fields, display strings, modal table shapes, and |
| 1311 | compatibility object layouts. None of that belongs in production payloads. |
| 1312 | |
| 1313 | The acceptance check is: |
| 1314 | |
| 1315 | 1. decode the new canonical payload; |
| 1316 | 2. derive the compatibility shape in test code; |
| 1317 | 3. compare against sanitized fixtures or expected compatibility behavior. |
| 1318 | |
| 1319 | Do not add fields to production payloads solely to make this projection easier. |
| 1320 | |
| 1321 | ## Producer Checklist |
| 1322 | |
| 1323 | Before shipping a topology producer: |
| 1324 | |
| 1325 | - define actor, link, evidence, table, and overlay types; |
| 1326 | - choose actor identities that survive aggregation; |
| 1327 | - separate graph links from evidence rows; |
| 1328 | - mark direction semantics explicitly in link types; |
| 1329 | - classify custom tables as actor detail, actor inventory, |
| 1330 | relationship evidence, or relationship summary; |
| 1331 | - expose host/node labels in an actor-owned `actor_labels` table when |
| 1332 | available; |
| 1333 | - expose useful non-node actor labels and metadata in `actor_labels`, while |
| 1334 | keeping identity, correlation, grouping, sorting, filtering, and aggregation |
| 1335 | facts as typed canonical columns; |
| 1336 | - define actor/link modal sections as recipes over existing actors, links, |
| 1337 | evidence, and detail tables; |
| 1338 | - migrate old `show_port_bullets` to |
| 1339 | `types.actor_types.<id>.presentation.ports.show_bullets` and define |
| 1340 | `ports.sources[]` so bullet data is not implicit; |
| 1341 | - use compact tables for high-cardinality sections; |
| 1342 | - use `src/go/pkg/topology/v1` helpers for Go producers; |
| 1343 | - avoid per-row display strings and duplicate labels; |
| 1344 | - keep raw JSON out of normal user-facing modal tables unless a section marks |
| 1345 | it as explicit debug output; |
| 1346 | - include only canonical facts and optional metrics the producer really has; |
| 1347 | - validate with [FUNCTION_TOPOLOGY_SCHEMA.json](/src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json); |
| 1348 | - run payload-size measurements on realistic or captured fixtures; |
| 1349 | - keep raw real-environment payloads under `.local/` only. |