Rework SOW lifecycle (#22606)
Ilya Mashchenko committed
Jun 3, 2026 at 08:40 UTC
3b4c0e8a9940b7d5f1f9dbd7047402a1a3b12db2
56 files changed
+571
-23327
.agents/skill-verification/query-netdata-agents/questions.md
+3
-2
@@ -1,10 +1,11 @@
1
# query-netdata-agents -- verification questions (seed list)
2
3
This file is the **seed input** consumed by the verification
4
-harness in SOW-0006 for direct-agent queries. The harness spawns
4
+harness introduced by PR #22423 for direct-agent queries. The harness spawns
5
a Sonnet-class assistant with `../SKILL.md` + `../how-tos/INDEX.md`
6
+ the canonical reference docs as context, asks each question
7
-below, captures the transcript, and grades against `grader.md`.
7
+below, captures the transcript, and grades it against the
8
+verification harness rubric.
9
10
When the assistant cannot answer or has to perform analysis not
11
already documented under `../how-tos/`, the assistant must author
.agents/skill-verification/query-netdata-cloud/questions.md
+2
-2
@@ -1,10 +1,10 @@
1
# query-netdata-cloud -- verification questions (seed list)
2
3
This file is the **seed input** consumed by the verification
4
-harness in SOW-0006. The harness spawns a Sonnet-class assistant
4
+harness introduced by PR #22423. The harness spawns a Sonnet-class assistant
5
with `../SKILL.md` + `../how-tos/INDEX.md` + the canonical
6
reference docs as context, asks each question below, captures the
7
-transcript, and grades against `grader.md` (delivered by SOW-0006).
7
+transcript, and grades it against the verification harness rubric.
8
9
When the assistant cannot answer or has to perform analysis not
10
already documented under `../how-tos/`, the assistant must author
.agents/skills/codacy-audit/SKILL.md
+3
-3
@@ -1,6 +1,6 @@
1
---
2
name: codacy-audit
3
-description: Codacy Cloud workflow for this repository -- run Codacy's analyzers locally before `git push` (mirrors what Codacy CI runs), and fetch/cluster Codacy issues for any PR via the v3 API. Use when the user mentions Codacy, "codacy analysis", `codacy-analysis-cli`, "codacy issues on PR", "fix codacy CI", "codacy markdownlint findings", or any Codacy gate failing on a netdata-org PR. Ships scripts analyze-local.sh (docker/binary runner for codacy-analysis-cli) and pr-issues.sh (paginated v3 issue fetch + group-by tool/pattern/severity/file). Token-safe -- CODACY_TOKEN never reaches assistant-visible stdout. Read-only by design in the current SOW; write actions (mark FP, mark fixed) are deferred.
3
+description: Codacy Cloud workflow for this repository -- run Codacy's analyzers locally before `git push` (mirrors what Codacy CI runs), and fetch/cluster Codacy issues for any PR via the v3 API. Use when the user mentions Codacy, "codacy analysis", `codacy-analysis-cli`, "codacy issues on PR", "fix codacy CI", "codacy markdownlint findings", or any Codacy gate failing on a netdata-org PR. Ships scripts analyze-local.sh (docker/binary runner for codacy-analysis-cli) and pr-issues.sh (paginated v3 issue fetch + group-by tool/pattern/severity/file). Token-safe -- CODACY_TOKEN never reaches assistant-visible stdout. Read-only by design; write actions (mark FP, mark fixed) require a GitHub issue or branch-local SOW.
4
---
5
6
# Codacy audit skill
@@ -28,7 +28,7 @@ Examples worth capturing:
28
29
Each concrete question that requires non-trivial analysis (multiple wrapper calls, jq pipelines, cross-referencing other skills) MUST become a how-to under `how-tos/<slug>.md` AND get an entry in `how-tos/INDEX.md` BEFORE the task is reported complete. Skipping this means the next assistant repeats the analysis from scratch.
30
31
-## Scope (current SOW)
31
+## Scope
32
33
In scope:
34
@@ -36,7 +36,7 @@ In scope:
36
- Read-only PR-issue queries against the v3 API.
37
- Token-safe wrappers (sentinel-driven no-leak self-test).
38
39
-Out of scope (deferred to a future SOW):
39
+Out of scope until a real use case creates a GitHub issue or branch-local SOW:
40
41
- Write actions (mark issue as false-positive, mark as fixed, modify ignore-patterns).
42
- Master-backlog triage on the 31,425+ open issues.
.agents/skills/codacy-audit/how-tos/reproduce-pr-22423-markdownlint.md
+2
-2
@@ -2,7 +2,7 @@
2
3
## When to use
4
5
-You want to confirm `analyze-local.sh` matches what Codacy CI reported on a known fixture. PR #22423 is the canonical fixture for this skill: its first CI run reported **864 markdownlint findings** (recorded in the SOW that built this skill, SOW-0011); commit `3a54c9afbc` cleared them by adding `.agents/**` and `docs/netdata-ai/skills/**` to `.codacy.yml`.
5
+You want to confirm `analyze-local.sh` matches what Codacy CI reported on a known fixture. PR #22423 is the canonical fixture for this skill: its first CI run reported **864 markdownlint findings**; commit `3a54c9afbc` cleared them by adding `.agents/**` and `docs/netdata-ai/skills/**` to `.codacy.yml`.
6
7
This how-to walks through reproducing those 864 findings on the pre-exclusion state, then confirming the post-exclusion state shows zero on the affected files.
8
@@ -63,7 +63,7 @@ Expected: zero rows in the excluded trees.
63
## What this validates
64
65
- `analyze-local.sh` runs end-to-end against the configured runner (docker or local binary).
66
-- The CLI honours `.codacy.yml` exclude_paths (or, if it doesn't, we have empirical evidence to handle that gap in a future SOW).
66
+- The CLI honours `.codacy.yml` exclude_paths (or, if it doesn't, we have empirical evidence to handle that gap in a GitHub issue or branch-local SOW).
67
- The bundled markdownlint version produces a count consistent with what Codacy CI reports.
68
69
## Troubleshooting
.agents/skills/integrations-lifecycle/gotchas.md
+3
-5
@@ -98,7 +98,7 @@ before assuming the code does the obvious thing.
98
prefix is missing, so the literal `{ check_path }` would be
99
printed even if the imports worked.
100
- **Treat this file as dead code.** Do NOT rely on it.
101
- Followup: SOW will track repair-or-delete.
101
+ Follow-up must be tracked by a GitHub issue before implementation starts.
102
103
### `gen_doc_service_discovery_page.py` is NOT in CI
104
@@ -112,8 +112,7 @@ before assuming the code does the obvious thing.
112
- Consequence: `src/collectors/SERVICE-DISCOVERY.md` drifts
113
from source `metadata.yaml` until a developer manually
114
runs `python3 integrations/gen_doc_service_discovery_page.py`.
115
-- Followup: SOW will track adding the script to both
116
- workflows.
115
+- Follow-up must be tracked by a GitHub issue before implementation starts.
116
117
### `integrations/schemas/distros.json` is unused
118
@@ -124,8 +123,7 @@ before assuming the code does the obvious thing.
123
`distros.yml`.
124
- Garbage in `.github/data/distros.yml` produces broken
125
`platform_info` tables silently.
127
-- Followup: SOW will track wiring `distros.json` into
128
- `gen_integrations.py:1330`.
126
+- Follow-up must be tracked by a GitHub issue before implementation starts.
127
128
## Custom Jinja delimiters
129
.agents/sow/SOW.template.md
+14
-14
@@ -1,10 +1,14 @@
1
-# SOW-NNNN - <Title>
1
+# SOW-YYYYMMDD-<slug> - <Title>
2
3
## Status
4
5
-Status: open | in-progress | paused | completed | closed
5
+Status: planning | ready | in-progress | paused | completed
6
7
-`completed` is the successful terminal status. `done` is a directory name, not a status value. Do not use `Status: done` or `Status: complete`.
7
+`planning` means analysis or decisions are incomplete. `ready` means the
8
+Pre-Implementation Gate is complete and implementation can start. `completed`
9
+is a transient branch-local state before deleting this SOW working file before
10
+merge. SOW files live only under `.agents/sow/active/` on feature branches and
11
+MUST NOT be merged to `master`.
12
13
Sub-state: <short current truth>
14
@@ -18,6 +22,8 @@ Sub-state: <short current truth>
22
23
<Concise quote or faithful summary. Do not lose constraints.>
24
25
+Regresses (optional): PR #NNNNN
26
+
27
### Assistant Understanding
28
29
Facts:
@@ -61,7 +67,7 @@ Problem / root-cause model:
67
68
Evidence reviewed:
69
64
-- <Specs, code, docs, tests, logs, traces, prior SOWs, issues, external references.>
70
+- <Specs, code, docs, tests, logs, traces, prior PRs/issues, external references.>
71
- <For mirrored open-source repositories: cite `owner/repo @ commit` and repository-relative paths; never paste `/opt/baddisk/monitoring/repos/...` absolute paths.>
72
73
Affected contracts and surfaces:
@@ -96,7 +102,7 @@ Artifact impact plan:
102
- Specs: <expected update or reason likely unaffected>
103
- End-user/operator docs: <expected update or reason likely unaffected>
104
- End-user/operator skills: <expected update or reason likely unaffected>
99
-- SOW lifecycle: <split/merge/status/follow-up/regression handling>
105
+- SOW lifecycle: <branch-local working file; durable-knowledge targets (specs/skills/docs/code/tests); delete-before-merge; regression = new linked SOW; follow-up issues>
106
107
Open-source reference evidence:
108
@@ -154,7 +160,7 @@ Artifact maintenance gate:
160
- Specs: <updated .agents/sow/specs/ path or evidence-backed reason no update was needed>
161
- End-user/operator docs: <updated docs/runbooks/help paths or evidence-backed reason none were affected>
162
- End-user/operator skills: <updated output/reference skill paths or evidence-backed reason none were affected>
157
-- SOW lifecycle: <status/directory checked; if successful close, `Status: completed` and move to `.agents/sow/done/` are committed together with the work in one commit unless user explicitly requested a different split; split/merge/follow-up/regression handling recorded>
163
+- SOW lifecycle: <durable knowledge transferred to specs/skills/docs/code/tests; follow-ups moved to GitHub issues or rejected; `Status: completed` set; SOW working file deleted from the branch before merge so no SOW file reaches `master`; regression-as-new-SOW handling recorded>
164
165
Specs update:
166
@@ -178,7 +184,7 @@ Lessons:
184
185
Follow-up mapping:
186
181
-- <implemented/rejected/tracked>
187
+- <implemented/rejected/GitHub issue link>
188
189
## Outcome
190
@@ -188,12 +194,6 @@ Pending.
194
195
Pending.
196
191
-## Followup
197
+## Follow-up Issues
198
199
None yet.
194
-
195
-## Regression Log
196
-
197
-None yet.
198
-
199
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/active/.gitkeep
new
+1
@@ -0,0 +1 @@
1
+
.agents/sow/audit.sh
+192
-579
@@ -1,11 +1,9 @@
1
#!/usr/bin/env bash
2
-# Read-only audit for a project-local SOW setup.
3
-# Reports current state of cwd: is SOW initialized? what's in place? what's missing?
4
-# Never modifies anything.
2
+# Read-only audit for the project-local SOW system.
3
+# Never modifies files.
4
5
set -uo pipefail
6
8
-# Color output if stdout is a tty
7
if [ -t 1 ]; then
8
RED=$'\033[0;31m'
9
GREEN=$'\033[0;32m'
@@ -14,35 +12,34 @@ if [ -t 1 ]; then
12
GRAY=$'\033[0;90m'
13
NC=$'\033[0m'
14
else
17
- RED=""; GREEN=""; YELLOW=""; BLUE=""; GRAY=""; NC=""
15
+ RED=""
16
+ GREEN=""
17
+ YELLOW=""
18
+ BLUE=""
19
+ GRAY=""
20
+ NC=""
21
fi
22
20
-cwd=$(pwd)
21
-echo "${BLUE}=== SOW audit (cwd=$cwd) ===${NC}"
22
-echo
23
-
24
-is_output_reference_skill() {
25
- local name="$1"
26
- [ -f ./AGENTS.md ] && awk -v name="$name" '
27
- BEGIN { in_output = 0; found = 0 }
28
- /^[[:space:]]*Output\/reference skills:[[:space:]]*$/ { in_output = 1; next }
29
- /^[[:space:]]*(Runtime input skills|Legacy runtime skills):[[:space:]]*$/ { if (in_output) in_output = 0 }
30
- /^#{1,6}[[:space:]]/ && $0 !~ /Project Skills/ { if (in_output) in_output = 0 }
31
- in_output && index($0, ".agents/skills/" name "/") { found = 1 }
32
- END { exit found ? 0 : 1 }
33
- ' ./AGENTS.md 2>/dev/null
23
+failures=0
24
+warnings=0
25
+
26
+ok() {
27
+ echo " ${GREEN}OK${NC} $*"
28
+}
29
+
30
+fail() {
31
+ echo " ${RED}--${NC} $*"
32
+ failures=$((failures + 1))
33
}
34
36
-is_legacy_runtime_skill() {
37
- local name="$1"
38
- [ -f ./AGENTS.md ] && awk -v name="$name" '
39
- BEGIN { in_legacy = 0; found = 0 }
40
- /^[[:space:]]*Legacy runtime skills:[[:space:]]*$/ { in_legacy = 1; next }
41
- /^[[:space:]]*(Runtime input skills|Output\/reference skills):[[:space:]]*$/ { if (in_legacy) in_legacy = 0 }
42
- /^#{1,6}[[:space:]]/ && $0 !~ /Project Skills/ && $0 !~ /Legacy runtime skills/ { if (in_legacy) in_legacy = 0 }
43
- in_legacy && index($0, ".agents/skills/" name "/") { found = 1 }
44
- END { exit found ? 0 : 1 }
45
- ' ./AGENTS.md 2>/dev/null
35
+warn() {
36
+ echo " ${YELLOW}--${NC} $*"
37
+ warnings=$((warnings + 1))
38
+}
39
+
40
+section() {
41
+ echo
42
+ echo "${BLUE}-- $* --${NC}"
43
}
44
45
read_sow_status() {
@@ -66,116 +63,20 @@ read_sow_status() {
63
' "$1" 2>/dev/null
64
}
65
69
-sensitive_scan_files() {
70
- [ -f ./AGENTS.md ] && printf '%s\n' ./AGENTS.md
71
- [ -f ./AGENTS.md.pre-sow.bak ] && printf '%s\n' ./AGENTS.md.pre-sow.bak
72
- [ -f ./SKILL.md ] && printf '%s\n' ./SKILL.md
73
- [ -f ./SOW-status.md ] && printf '%s\n' ./SOW-status.md
74
- for sow_dir in ./.agents/sow/pending ./.agents/sow/current ./.agents/sow/specs; do
75
- [ -d "$sow_dir" ] && find "$sow_dir" -type f -name '*.md' 2>/dev/null
76
- done
77
- [ -d ./.agents/skills ] && find ./.agents/skills -type f \( -name '*.md' -o -name 'SKILL.md' -o -name '*.yaml' -o -name '*.yml' \) 2>/dev/null
78
- if [ "${SOW_AUDIT_SENSITIVE_FULL_HISTORY:-0}" = "1" ]; then
79
- find . -path ./.git -prune -o -type f \( -name '*.md' -o -name '*.rst' -o -name '*.adoc' -o -name '*.txt' -o -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null
80
- elif [ "${SOW_AUDIT_SENSITIVE_CHANGED:-0}" = "1" ] && git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
81
- {
82
- git diff --name-only --diff-filter=ACMR HEAD -- 2>/dev/null
83
- git diff --cached --name-only --diff-filter=ACMR -- 2>/dev/null
84
- git ls-files -o --exclude-standard 2>/dev/null
85
- } | awk '
86
- /\.(md|rst|adoc|txt|yaml|yml|json|toml|ini|conf|cfg|env|sh|bash|zsh|py|js|jsx|ts|tsx|go|rs|c|h|hpp|cpp|java|rb|php|lua|sql)$/ { print "./" $0; next }
87
- /(^|\/)(README|CHANGELOG|CONTRIBUTING|Dockerfile|Makefile)(\..*)?$/ { print "./" $0; next }
88
- /(^|\/)AGENTS\.md$/ { print "./" $0; next }
89
- '
90
- fi
91
-}
92
-
93
-scan_sensitive_file() {
94
- local file="$1"
95
- perl -ne '
96
- chomp;
97
- my $line = $_;
98
- my @hits;
99
-
100
- sub is_public_customer_ip {
101
- my ($ip) = @_;
102
- my @o = split(/\./, $ip);
103
- return 0 unless @o == 4;
104
- return 0 if grep { $_ !~ /^\d+$/ || $_ < 0 || $_ > 255 } @o;
105
- return 0 if $o[0] == 10;
106
- return 0 if $o[0] == 172 && $o[1] >= 16 && $o[1] <= 31;
107
- return 0 if $o[0] == 192 && $o[1] == 168;
108
- return 0 if $o[0] == 127;
109
- return 0 if $o[0] == 169 && $o[1] == 254;
110
- return 0 if $o[0] == 100 && $o[1] >= 64 && $o[1] <= 127;
111
- return 0 if $o[0] == 0;
112
- return 0 if $o[0] >= 224;
113
- return 0 if $o[0] == 192 && $o[1] == 0 && $o[2] == 2;
114
- return 0 if $o[0] == 198 && $o[1] == 51 && $o[2] == 100;
115
- return 0 if $o[0] == 203 && $o[1] == 0 && $o[2] == 113;
116
- return 1;
117
- }
118
-
119
- push @hits, "private-key-material" if $line =~ /-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----/;
120
- push @hits, "aws-access-key" if $line =~ /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/;
121
- push @hits, "github-token" if $line =~ /\b(?:github_pat_[A-Za-z0-9_]{20,}|gh[pousr]_[A-Za-z0-9_]{20,})\b/;
122
- push @hits, "slack-token" if $line =~ /\bxox[baprs]-[A-Za-z0-9-]{20,}\b/;
123
- push @hits, "openai-key" if $line =~ /\bsk-(?:proj-)?[A-Za-z0-9_-]{20,}\b/;
124
- push @hits, "google-api-key" if $line =~ /\bAIza[0-9A-Za-z_-]{20,}\b/;
125
- push @hits, "jwt" if $line =~ /\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/;
126
- push @hits, "credentialed-url" if $line =~ m{[a-z][a-z0-9+.-]*://[^/\s:@]+:[^/\s:@]+@}i;
127
- push @hits, "bearer-token" if $line =~ /\bBearer\s+[A-Za-z0-9._~+\/=-]{16,}\b/i && $line !~ /\b(REDACTED|EXAMPLE|PLACEHOLDER|YOUR[_-]?TOKEN|TOKEN|API[_-]?KEY|ACCESS[_-]?TOKEN)\b/i;
128
-
129
- if ($line =~ /\b(?:pass(?:word)?|passwd|pwd|api[_-]?key|secret|token|client[_-]?secret|private[_-]?key|access[_-]?key)\b\s*[:=]\s*["'\''`]?([^"'\''`\s<>{}\[\]&,]{8,})/i) {
130
- my $value = lc $1;
131
- push @hits, "credential-assignment" unless $value =~ /^(redacted|example|placeholder|changeme|change-me|xxx|xxxx|null|none|your[_-]?|dummy|sample|fake|test)/ || $value =~ /^\$/ || $value =~ /^(config|settings|options|opts|env|process\.env|os\.environ)\./ || $value =~ /^[a-z_][a-z0-9_.]*(token|secret|key|password)[a-z0-9_.]*$/;
132
- }
133
-
134
- if ($line =~ /\b(?:snmp[_-]?)?(?:community|community[_-]?string|rocommunity|rwcommunity)\b\s*[:=]\s*["'\''`]?([^"'\''`\s<>{}\[\]]{3,})/i) {
135
- my $value = lc $1;
136
- push @hits, "snmp-community" unless $value =~ /^(redacted|example|placeholder|changeme|change-me|xxx|xxxx|null|none)$/;
137
- }
138
-
139
- if ($line =~ /\b(?:customer|client|tenant|account|organization|org|community[ _-]?member)[ _-](?:name|id|identifier)\b\s*[:=]\s*["'\''`]?([^"'\''`<>\[\]{}][^"'\''`<>\[\]{}]{2,})/i) {
140
- my $value = $1;
141
- $value =~ s/^\s+|\s+$//g;
142
- push @hits, "customer-or-private-identifier" unless $value =~ /^(redacted|example|placeholder|customer-|client-|tenant-|account-|org-|user|none|null)/i;
143
- }
144
-
145
- if ($line =~ /\b[A-Z0-9._%+\-]+@[A-Z0-9.\-]+\.[A-Z]{2,}\b/i) {
146
- push @hits, "email-address" unless $line =~ /\b(example\.com|example\.org|example\.net|localhost)\b/i;
147
- }
148
-
149
- if ($line =~ /\b(customer|client|tenant|account|community member|support|production|prod|log|trace|request|source ip|remote ip|x-forwarded-for|host ip)\b/i) {
150
- while ($line =~ /\b((?:\d{1,3}\.){3}\d{1,3})\b/g) {
151
- push @hits, "public-ip-address" if is_public_customer_ip($1);
152
- }
153
- }
154
-
155
- for my $hit (@hits) {
156
- print "$ARGV:$.:$hit\n";
157
- }
158
- ' "$file" 2>/dev/null
159
-}
66
+echo "${BLUE}=== SOW audit (cwd=$(pwd)) ===${NC}"
67
161
-# --- Marker check ---
162
-echo "${BLUE}-- initialization marker --${NC}"
163
-if [ -f ./AGENTS.md ]; then
164
- if grep -q "^Project SOW status: initialized$" ./AGENTS.md 2>/dev/null; then
165
- echo " ${GREEN}OK${NC} marker present in ./AGENTS.md"
166
- initialized=true
68
+section "initialization marker"
69
+if [ -f AGENTS.md ]; then
70
+ if grep -q "^Project SOW status: initialized$" AGENTS.md; then
71
+ ok "marker present in AGENTS.md"
72
else
168
- echo " ${YELLOW}--${NC} AGENTS.md exists but marker absent (partial state or not initialized)"
169
- initialized=false
73
+ fail "AGENTS.md exists but Project SOW status marker is missing"
74
fi
75
else
172
- echo " ${RED}--${NC} ./AGENTS.md does not exist (create or normalize project instructions first, then SOW init)"
173
- initialized=false
76
+ fail "AGENTS.md is missing"
77
fi
175
-echo
78
177
-# --- Canonical AGENTS.md sections ---
178
-echo "${BLUE}-- canonical AGENTS.md sections --${NC}"
79
+section "canonical AGENTS.md sections"
80
required_sections=(
81
"## Goals"
82
"## SOW System"
@@ -184,494 +85,206 @@ required_sections=(
85
"### Sensitive Data In Durable Artifacts"
86
"### Open-Source Reference Evidence"
87
"### Pre-Implementation Gate"
187
- "### SOW Completion And Commit"
88
+ "### SOW Completion And Merge"
89
+ "### Enforcement"
90
"### Regressions"
91
"### Project Skills"
92
"### Specs"
93
"### Project-specific overrides"
94
)
193
-sections_ok=0
194
-sections_missing=0
195
-sensitive_warning_missing=0
196
-if [ -f ./AGENTS.md ]; then
197
- for s in "${required_sections[@]}"; do
198
- if grep -qF "$s" ./AGENTS.md 2>/dev/null; then
199
- echo " ${GREEN}OK${NC} $s"
200
- sections_ok=$((sections_ok + 1))
201
- else
202
- echo " ${RED}--${NC} $s (missing)"
203
- sections_missing=$((sections_missing + 1))
204
- fi
205
- done
206
- if grep -qF "CRITICAL: Never write raw sensitive data to durable artifacts." ./AGENTS.md 2>/dev/null; then
207
- echo " ${GREEN}OK${NC} CRITICAL sensitive-data warning"
208
- else
209
- echo " ${RED}--${NC} CRITICAL sensitive-data warning (missing)"
210
- sensitive_warning_missing=1
211
- fi
212
-else
213
- echo " ${GRAY}(AGENTS.md not present; skipping section check)${NC}"
214
-fi
215
-echo
216
-
217
-# --- All AGENTS.md sensitive-data warnings ---
218
-echo "${BLUE}-- all AGENTS.md sensitive-data warnings --${NC}"
219
-agents_warning_missing=0
220
-agents_warning_checked=0
221
-while IFS= read -r f; do
222
- [ -z "$f" ] && continue
223
- if git rev-parse --is-inside-work-tree >/dev/null 2>&1 && git check-ignore -q "$f" 2>/dev/null; then
224
- continue
225
- fi
226
- agents_warning_checked=$((agents_warning_checked + 1))
227
- if grep -qF "CRITICAL: Never write raw sensitive data to durable artifacts." "$f" 2>/dev/null; then
228
- echo " ${GREEN}OK${NC} $f"
229
- else
230
- echo " ${RED}--${NC} $f (missing CRITICAL sensitive-data warning)"
231
- agents_warning_missing=$((agents_warning_missing + 1))
232
- fi
233
-done < <(find . -path ./.git -prune -o -name AGENTS.md -type f -print 2>/dev/null | sort)
234
-if [ "$agents_warning_checked" -eq 0 ]; then
235
- echo " ${GRAY}(no AGENTS.md files found)${NC}"
236
-fi
237
-echo
238
-
239
-# --- Cross-tool instruction bridges ---
240
-echo "${BLUE}-- cross-tool instruction bridges --${NC}"
241
-bridge_missing=0
242
-if [ -f ./AGENTS.md ]; then
243
- if [ -L ./CLAUDE.md ] && [ "$(readlink ./CLAUDE.md 2>/dev/null)" = "AGENTS.md" ]; then
244
- echo " ${GREEN}OK${NC} CLAUDE.md -> AGENTS.md"
245
- else
246
- echo " ${RED}--${NC} CLAUDE.md -> AGENTS.md (missing or not a relative symlink)"
247
- bridge_missing=$((bridge_missing + 1))
248
- fi
249
- if [ -L ./GEMINI.md ] && [ "$(readlink ./GEMINI.md 2>/dev/null)" = "AGENTS.md" ]; then
250
- echo " ${GREEN}OK${NC} GEMINI.md -> AGENTS.md"
251
- else
252
- echo " ${RED}--${NC} GEMINI.md -> AGENTS.md (missing or not a relative symlink)"
253
- bridge_missing=$((bridge_missing + 1))
254
- fi
255
- if [ -d ./.agents/skills ]; then
256
- echo " ${GREEN}OK${NC} .agents/skills/"
257
- else
258
- echo " ${RED}--${NC} .agents/skills/ (missing; create even when no project skills exist)"
259
- bridge_missing=$((bridge_missing + 1))
260
- fi
261
- if [ -L ./.claude/skills ] && [ "$(readlink ./.claude/skills 2>/dev/null)" = "../.agents/skills" ]; then
262
- echo " ${GREEN}OK${NC} .claude/skills -> ../.agents/skills"
95
+
96
+for heading in "${required_sections[@]}"; do
97
+ if grep -qF "$heading" AGENTS.md 2>/dev/null; then
98
+ ok "$heading"
99
else
264
- echo " ${RED}--${NC} .claude/skills -> ../.agents/skills (missing or not a relative symlink)"
265
- bridge_missing=$((bridge_missing + 1))
100
+ fail "$heading is missing"
101
fi
102
+done
103
+
104
+if grep -qF "CRITICAL: Never write raw sensitive data to durable artifacts." AGENTS.md 2>/dev/null; then
105
+ ok "CRITICAL sensitive-data warning"
106
else
268
- echo " ${GRAY}(AGENTS.md not present; skipping bridge check)${NC}"
107
+ fail "CRITICAL sensitive-data warning is missing from AGENTS.md"
108
fi
270
-echo
271
-
272
-# --- .agents/sow/ directories ---
273
-echo "${BLUE}-- SOW directories --${NC}"
274
-sow_dirs=(specs pending current done)
275
-sow_dir_ok=0
276
-sow_dir_missing=0
277
-empty_sow_dir_missing_keep=0
278
-for d in "${sow_dirs[@]}"; do
279
- if [ -d ".agents/sow/$d" ]; then
280
- echo " ${GREEN}OK${NC} .agents/sow/$d/"
281
- sow_dir_ok=$((sow_dir_ok + 1))
282
- has_entries=$(find ".agents/sow/$d" -mindepth 1 -maxdepth 1 2>/dev/null | wc -l | tr -d ' ')
283
- if [ "$has_entries" -eq 0 ] && [ ! -f ".agents/sow/$d/.gitkeep" ] && [ ! -f ".agents/sow/$d/.keep" ]; then
284
- echo " ${YELLOW}*${NC} empty directory has no .gitkeep or .keep placeholder"
285
- empty_sow_dir_missing_keep=$((empty_sow_dir_missing_keep + 1))
286
- fi
109
+
110
+section "SOW layout"
111
+for path in .agents/sow/active .agents/sow/specs; do
112
+ if [ -d "$path" ]; then
113
+ ok "$path exists"
114
else
288
- echo " ${RED}--${NC} .agents/sow/$d/ (missing)"
289
- sow_dir_missing=$((sow_dir_missing + 1))
115
+ fail "$path is missing"
116
fi
117
done
292
-echo
293
-
294
-# --- Project-local framework files ---
295
-echo "${BLUE}-- project-local SOW framework files --${NC}"
296
-framework_missing=0
297
-if [ -f ".agents/sow/SOW.template.md" ]; then
298
- echo " ${GREEN}OK${NC} .agents/sow/SOW.template.md"
299
- if grep -q "^## Pre-Implementation Gate$" ".agents/sow/SOW.template.md" 2>/dev/null; then
300
- echo " ${GREEN}OK${NC} template includes Pre-Implementation Gate"
301
- sow_template_pre_impl_missing=0
302
- else
303
- echo " ${RED}--${NC} template missing ## Pre-Implementation Gate"
304
- sow_template_pre_impl_missing=1
305
- fi
306
- if grep -q "^Sensitive data handling plan:$" ".agents/sow/SOW.template.md" 2>/dev/null && grep -q "^Sensitive data gate:$" ".agents/sow/SOW.template.md" 2>/dev/null; then
307
- echo " ${GREEN}OK${NC} template includes sensitive data gates"
308
- sow_template_sensitive_gate_missing=0
309
- else
310
- echo " ${RED}--${NC} template missing sensitive data handling plan or gate"
311
- sow_template_sensitive_gate_missing=1
312
- fi
313
- if grep -q "^Open-source reference evidence:$" ".agents/sow/SOW.template.md" 2>/dev/null; then
314
- echo " ${GREEN}OK${NC} template includes open-source reference evidence"
315
- sow_template_open_source_reference_missing=0
118
+
119
+for path in .agents/sow/SOW.template.md .agents/sow/audit.sh .agents/sow/scan-sensitive.sh .agents/sow/specs/README.md; do
120
+ if [ -f "$path" ]; then
121
+ ok "$path exists"
122
else
317
- echo " ${RED}--${NC} template missing open-source reference evidence"
318
- sow_template_open_source_reference_missing=1
123
+ fail "$path is missing"
124
fi
320
- if grep -qF "directory name, not a status value" ".agents/sow/SOW.template.md" 2>/dev/null && grep -qF '`completed` is the successful terminal status' ".agents/sow/SOW.template.md" 2>/dev/null && grep -qF "one commit" ".agents/sow/SOW.template.md" 2>/dev/null; then
321
- echo " ${GREEN}OK${NC} template includes completed-status and one-commit close rule"
322
- sow_template_completion_rule_missing=0
125
+done
126
+
127
+for legacy_dir in .agents/sow/pending .agents/sow/current .agents/sow/done; do
128
+ if [ -e "$legacy_dir" ]; then
129
+ fail "$legacy_dir must not exist in the active-only SOW model"
130
else
324
- echo " ${RED}--${NC} template missing completed-status or one-commit close rule"
325
- sow_template_completion_rule_missing=1
326
- fi
327
-else
328
- echo " ${RED}--${NC} .agents/sow/SOW.template.md (missing)"
329
- framework_missing=$((framework_missing + 1))
330
- sow_template_pre_impl_missing=1
331
- sow_template_sensitive_gate_missing=1
332
- sow_template_open_source_reference_missing=1
333
- sow_template_completion_rule_missing=1
334
-fi
335
-if [ -f ".agents/sow/audit.sh" ]; then
336
- echo " ${GREEN}OK${NC} .agents/sow/audit.sh"
337
-else
338
- echo " ${RED}--${NC} .agents/sow/audit.sh (missing)"
339
- framework_missing=$((framework_missing + 1))
340
-fi
341
-echo
342
-
343
-# --- SOW counts per status ---
344
-echo "${BLUE}-- SOW counts per status --${NC}"
345
-for d in pending current done; do
346
- if [ -d ".agents/sow/$d" ]; then
347
- n=$(find ".agents/sow/$d" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f 2>/dev/null | wc -l | tr -d ' ')
348
- if [ "$n" -gt 0 ]; then
349
- echo " $d: $n"
350
- find ".agents/sow/$d" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f -printf ' %f\n' 2>/dev/null | sort
351
- else
352
- echo " $d: ${GRAY}(empty)${NC}"
353
- fi
131
+ ok "$legacy_dir absent"
132
fi
133
done
356
-echo
357
-
358
-# --- SOW status/directory consistency ---
359
-echo "${BLUE}-- SOW status/directory consistency --${NC}"
360
-sow_status_mismatch=0
361
-sow_status_missing=0
362
-sow_status_checked=0
363
-for d in pending current done; do
364
- [ -d ".agents/sow/$d" ] || continue
365
- while IFS= read -r f; do
366
- [ -z "$f" ] && continue
367
- sow_status_checked=$((sow_status_checked + 1))
368
- status=$(read_sow_status "$f")
369
- if [ -z "$status" ]; then
370
- echo " ${RED}--${NC} $f (missing Status: line)"
371
- sow_status_missing=$((sow_status_missing + 1))
372
- continue
373
- fi
374
- ok=false
375
- case "$d:$status" in
376
- pending:open|current:in-progress|current:paused|done:completed|done:closed)
377
- ok=true
134
+
135
+section "active SOW files"
136
+active_count=0
137
+if [ -d .agents/sow/active ]; then
138
+ while IFS= read -r sow; do
139
+ [ -n "$sow" ] || continue
140
+ active_count=$((active_count + 1))
141
+ status=$(read_sow_status "$sow")
142
+
143
+ case "$status" in
144
+ planning|ready|in-progress|paused|completed)
145
+ ok "$sow status=$status"
146
+ ;;
147
+ "")
148
+ fail "$sow has no Status"
149
+ ;;
150
+ *)
151
+ fail "$sow has invalid Status: $status"
152
;;
153
esac
380
- if $ok; then
381
- echo " ${GREEN}OK${NC} $f ($status)"
382
- elif [ "$status" = "done" ]; then
383
- echo " ${RED}--${NC} $f (Status: done is invalid; use Status: completed in done/. done is the directory name, not a status)"
384
- sow_status_mismatch=$((sow_status_mismatch + 1))
385
- elif [ "$status" = "complete" ]; then
386
- echo " ${RED}--${NC} $f (Status: complete is invalid; use Status: completed in done/)"
387
- sow_status_mismatch=$((sow_status_mismatch + 1))
388
- else
389
- echo " ${RED}--${NC} $f (Status: $status does not match $d/)"
390
- sow_status_mismatch=$((sow_status_mismatch + 1))
391
- fi
392
- done < <(find ".agents/sow/$d" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f 2>/dev/null | sort)
393
-done
394
-if [ "$sow_status_checked" -eq 0 ]; then
395
- echo " ${GRAY}(no SOW files found)${NC}"
396
-fi
397
-echo
398
-
399
-# --- Current SOW pre-implementation gates ---
400
-echo "${BLUE}-- current SOW pre-implementation gates --${NC}"
401
-current_sow_pre_impl_missing=0
402
-current_sow_pre_impl_checked=0
403
-current_sow_sensitive_gate_missing=0
404
-if [ -d ".agents/sow/current" ]; then
405
- while IFS= read -r f; do
406
- [ -z "$f" ] && continue
407
- current_sow_pre_impl_checked=$((current_sow_pre_impl_checked + 1))
408
- if grep -q "^## Pre-Implementation Gate$" "$f" 2>/dev/null; then
409
- echo " ${GREEN}OK${NC} $f"
410
- else
411
- echo " ${RED}--${NC} $f (missing ## Pre-Implementation Gate before implementation continues)"
412
- current_sow_pre_impl_missing=$((current_sow_pre_impl_missing + 1))
413
- fi
414
- if ! grep -q "^Sensitive data handling plan:$" "$f" 2>/dev/null || ! grep -q "^Sensitive data gate:$" "$f" 2>/dev/null; then
415
- echo " ${RED}--${NC} missing sensitive data handling plan or validation gate"
416
- current_sow_sensitive_gate_missing=$((current_sow_sensitive_gate_missing + 1))
417
- fi
418
- done < <(find ".agents/sow/current" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f 2>/dev/null | sort)
419
-fi
420
-if [ "$current_sow_pre_impl_checked" -eq 0 ]; then
421
- echo " ${GRAY}(no current SOW files found)${NC}"
422
-fi
423
-echo
424
-
425
-# --- Regression section placement ---
426
-echo "${BLUE}-- regression section placement --${NC}"
427
-regression_order_violations=0
428
-regression_order_checked=0
429
-for d in pending current done; do
430
- [ -d ".agents/sow/$d" ] || continue
431
- while IFS= read -r f; do
432
- [ -z "$f" ] && continue
433
- result=$(awk '
434
- /^## Regression([[:space:]-]|$)/ && !first_reg { first_reg = NR }
435
- /^## (Outcome|[Ll]essons [Ee]xtracted|Followup|Follow-up)$/ { if (NR > last_tail) last_tail = NR }
436
- END {
437
- if (!first_reg) {
438
- exit
439
- }
440
- if (last_tail && first_reg < last_tail) {
441
- print "bad:" first_reg ":" last_tail
442
- } else {
443
- print "ok:" first_reg ":" last_tail
444
- }
445
- }
446
- ' "$f")
447
- [ -z "$result" ] && continue
448
- regression_order_checked=$((regression_order_checked + 1))
449
- status=${result%%:*}
450
- details=${result#*:}
451
- if [ "$status" = "ok" ]; then
452
- echo " ${GREEN}OK${NC} $f (Regression section is appended)"
453
- else
454
- first_reg=${details%%:*}
455
- last_tail=${details#*:}
456
- echo " ${RED}--${NC} $f (Regression section starts at line $first_reg before original tail section ending at line $last_tail; append regressions to the end)"
457
- regression_order_violations=$((regression_order_violations + 1))
458
- fi
459
- done < <(find ".agents/sow/$d" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f 2>/dev/null | sort)
460
-done
461
-if [ "$regression_order_checked" -eq 0 ]; then
462
- echo " ${GRAY}(no regression sections found)${NC}"
463
-fi
464
-echo
465
-
466
-# --- Mirrored open-source reference evidence ---
467
-echo "${BLUE}-- mirrored open-source reference evidence --${NC}"
468
-mirror_path_violations=0
469
-mirror_path_checked=0
470
-for d in pending current done; do
471
- [ -d ".agents/sow/$d" ] || continue
472
- while IFS= read -r f; do
473
- [ -z "$f" ] && continue
474
- mirror_path_checked=$((mirror_path_checked + 1))
475
- if grep -qF "/opt/baddisk/monitoring/repos" "$f" 2>/dev/null; then
476
- echo " ${RED}--${NC} $f (uses /opt/baddisk/monitoring/repos absolute path; cite owner/repo @ commit plus repo-relative path)"
477
- mirror_path_violations=$((mirror_path_violations + 1))
478
- fi
479
- done < <(find ".agents/sow/$d" -mindepth 1 -maxdepth 1 -name 'SOW-*.md' -type f 2>/dev/null | sort)
480
-done
481
-if [ "$mirror_path_checked" -eq 0 ]; then
482
- echo " ${GRAY}(no SOW files found)${NC}"
483
-elif [ "$mirror_path_violations" -eq 0 ]; then
484
- echo " ${GREEN}OK${NC} checked $mirror_path_checked SOW file(s); mirrored repository evidence uses durable citations"
154
+
155
+ for needle in \
156
+ "## Pre-Implementation Gate" \
157
+ "Sensitive data handling plan:" \
158
+ "Sensitive data gate:" \
159
+ "## Validation" \
160
+ "## Artifact Maintenance Gate"
161
+ do
162
+ if grep -qF "$needle" "$sow"; then
163
+ ok "$sow contains $needle"
164
+ else
165
+ fail "$sow is missing $needle"
166
+ fi
167
+ done
168
+ done < <(find .agents/sow/active -maxdepth 1 -type f -name 'SOW-*.md' 2>/dev/null | sort)
169
fi
486
-echo
487
-
488
-# --- Sensitive data guardrail ---
489
-echo "${BLUE}-- sensitive data guardrail --${NC}"
490
-sensitive_findings=0
491
-sensitive_files_checked=0
492
-while IFS= read -r f; do
493
- [ -z "$f" ] && continue
494
- sensitive_files_checked=$((sensitive_files_checked + 1))
495
- scan_output=$(scan_sensitive_file "$f")
496
- if [ -n "$scan_output" ]; then
497
- while IFS= read -r finding; do
498
- [ -z "$finding" ] && continue
499
- echo " ${RED}--${NC} $finding"
500
- sensitive_findings=$((sensitive_findings + 1))
501
- done <<< "$scan_output"
502
- fi
503
-done < <(sensitive_scan_files | sort -u)
504
-if [ "$sensitive_files_checked" -eq 0 ]; then
505
- echo " ${GRAY}(no durable artifact files found)${NC}"
506
-elif [ "$sensitive_findings" -eq 0 ]; then
507
- echo " ${GREEN}OK${NC} scanned $sensitive_files_checked durable artifact file(s); no sensitive-data patterns found"
170
+
171
+if [ "$active_count" -eq 0 ]; then
172
+ ok "no active branch-local SOW files"
173
else
509
- echo " ${RED}--${NC} $sensitive_findings sensitive-data pattern(s) found. Output is file:line:rule only; inspect locally and redact before commit."
174
+ warn "$active_count active branch-local SOW file(s); they must be deleted before merge"
175
fi
511
-echo
176
513
-# --- Project skills ---
514
-echo "${BLUE}-- runtime project skills --${NC}"
515
-project_skills_ok=0
516
-project_skills_total=0
517
-project_output_reference_total=0
518
-if [ -d .agents/skills ]; then
519
- while IFS= read -r d; do
520
- [ -z "$d" ] && continue
521
- name=$(basename "$d")
522
- if is_output_reference_skill "$name"; then
523
- project_output_reference_total=$((project_output_reference_total + 1))
524
- echo " ${GREEN}OK${NC} $name (listed as output/reference; excluded from default runtime guidance)"
525
- continue
177
+section "spec index"
178
+if [ -f .agents/sow/specs/README.md ]; then
179
+ while IFS= read -r spec; do
180
+ [ -n "$spec" ] || continue
181
+ rel=${spec#.agents/sow/specs/}
182
+ if grep -qF "]($rel)" .agents/sow/specs/README.md; then
183
+ ok "$rel is listed in specs/README.md"
184
+ else
185
+ fail "$rel is missing from specs/README.md"
186
fi
527
- project_skills_total=$((project_skills_total + 1))
528
- if [ -f "$d/SKILL.md" ]; then
529
- lines=$(wc -l <"$d/SKILL.md" 2>/dev/null | tr -d ' ')
530
- echo " ${GREEN}OK${NC} $name ($lines lines)"
531
- project_skills_ok=$((project_skills_ok + 1))
187
+ done < <(find .agents/sow/specs -maxdepth 1 -type f -name '*.md' ! -name README.md 2>/dev/null | sort)
188
+
189
+ while IFS= read -r link; do
190
+ [ -n "$link" ] || continue
191
+ if [ -f ".agents/sow/specs/$link" ]; then
192
+ ok "spec index link resolves: $link"
193
else
533
- echo " ${RED}--${NC} $name (no SKILL.md)"
194
+ fail "spec index link is broken: $link"
195
fi
535
- done < <(find .agents/skills -mindepth 1 -maxdepth 1 -type d -name 'project-*' 2>/dev/null | sort)
536
- if [ "$project_skills_total" -eq 0 ] && [ "$project_output_reference_total" -eq 0 ]; then
537
- echo " ${YELLOW}(no .agents/skills/project-*/ found)${NC}"
538
- elif [ "$project_skills_total" -eq 0 ]; then
539
- echo " ${YELLOW}(no runtime input project-* skills found; only output/reference exceptions)${NC}"
540
- fi
196
+ done < <(grep -oE '\]\([A-Za-z0-9._-]+\.md\)' .agents/sow/specs/README.md 2>/dev/null | sed 's/^](//; s/)$//' | sort -u)
197
else
542
- echo " ${YELLOW}(.agents/skills/ does not exist)${NC}"
198
+ fail ".agents/sow/specs/README.md is missing"
199
fi
544
-echo
200
546
-# --- Non-project skill directories ---
547
-echo "${BLUE}-- non-project skill directories --${NC}"
548
-non_project_skills_total=0
549
-non_project_skills_unclassified=0
550
-if [ -d .agents/skills ]; then
551
- while IFS= read -r d; do
552
- [ -z "$d" ] && continue
553
- non_project_skills_total=$((non_project_skills_total + 1))
554
- name=$(basename "$d")
555
- if is_output_reference_skill "$name"; then
556
- echo " ${GREEN}OK${NC} $name (listed as output/reference; not auto-loaded as runtime SOW skill)"
557
- elif is_legacy_runtime_skill "$name"; then
558
- echo " ${GREEN}OK${NC} $name (listed as legacy runtime skill; project-* alignment deferred)"
559
- elif [ -f "$d/SKILL.md" ]; then
560
- echo " ${YELLOW}*${NC} $name (not auto-loaded as runtime SOW skill)"
561
- non_project_skills_unclassified=$((non_project_skills_unclassified + 1))
201
+section "spec references"
202
+if command -v rg >/dev/null 2>&1; then
203
+ while IFS= read -r ref; do
204
+ [ -n "$ref" ] || continue
205
+ if [ -f "$ref" ]; then
206
+ ok "spec reference resolves: $ref"
207
else
563
- echo " ${RED}--${NC} $name (not project-* and no SKILL.md)"
564
- non_project_skills_unclassified=$((non_project_skills_unclassified + 1))
208
+ fail "spec reference is broken: $ref"
209
fi
566
- done < <(find .agents/skills -mindepth 1 -maxdepth 1 -type d ! -name 'project-*' 2>/dev/null | sort)
567
- if [ "$non_project_skills_total" -eq 0 ]; then
568
- echo " ${GREEN}OK${NC} none"
569
- elif [ "$non_project_skills_unclassified" -gt 0 ]; then
570
- echo " ${GRAY}Classify each warning as output/reference, obsolete, or rename/wrap it as project-* if it is runtime input.${NC}"
571
- fi
210
+ done < <(
211
+ rg --no-filename -o '\.agents/sow/specs/[A-Za-z0-9._-]+\.md' \
212
+ AGENTS.md .agents/skills .agents/sow/specs docs src \
213
+ -g '*.md' -g 'SKILL.md' -g '*.sh' -g '*.yml' \
214
+ 2>/dev/null | sort -u
215
+ )
216
else
573
- echo " ${GRAY}(.agents/skills/ does not exist)${NC}"
217
+ warn "ripgrep not available; skipped spec reference audit"
218
fi
575
-echo
576
-
577
-# --- TODO files at project root ---
578
-echo "${BLUE}-- TODO files at project root --${NC}"
579
-todo_count=0
580
-todo_tracked=false
581
-while IFS= read -r f; do
582
- [ -z "$f" ] && continue
583
- todo_count=$((todo_count + 1))
584
- echo " ${YELLOW}*${NC} $f (not yet migrated)"
585
-done < <(find . -maxdepth 1 -name 'TODO-*.md' -o -maxdepth 1 -name 'TODO.md' 2>/dev/null | sort)
586
-if [ "$todo_count" -gt 0 ]; then
587
- if grep -RiqE "root TODO|TODO file|TODO migration|TODO classification|orphan TODO" .agents/sow/pending .agents/sow/current 2>/dev/null; then
588
- todo_tracked=true
589
- echo " ${GREEN}OK${NC} root TODO classification/migration is tracked by a pending/current SOW"
590
- fi
591
-else
592
- if [ -d .agents/sow/.todo-backup ]; then
593
- bk=$(find .agents/sow/.todo-backup -maxdepth 1 -name 'TODO*.md' 2>/dev/null | wc -l | tr -d ' ')
594
- if [ "$bk" -gt 0 ]; then
595
- echo " ${GREEN}OK${NC} no orphan TODO files (${bk} backed up at .agents/sow/.todo-backup/)"
596
- else
597
- echo " ${GREEN}OK${NC} no TODO files at project root"
598
- fi
219
+
220
+section "legacy SOW references"
221
+if command -v rg >/dev/null 2>&1; then
222
+ legacy_refs=$(rg --line-number 'SOW-[0-9]{4}\b' \
223
+ AGENTS.md .agents .github docs src \
224
+ -g '*.md' -g 'SKILL.md' -g '*.sh' -g '*.yml' \
225
+ -g '!TODO*.md' \
226
+ -g '!**/TODO*.md' \
227
+ -g '!**/.agents/sow/active/SOW-*.md' \
228
+ 2>/dev/null || true)
229
+
230
+ if [ -n "$legacy_refs" ]; then
231
+ printf '%s\n' "$legacy_refs"
232
+ fail "legacy SOW-NNNN references remain in durable files"
233
else
600
- echo " ${GREEN}OK${NC} no TODO files at project root"
234
+ ok "no legacy SOW-NNNN references in durable files"
235
fi
236
+else
237
+ warn "ripgrep not available; skipped legacy SOW reference audit"
238
fi
603
-todo_untracked_count=$todo_count
604
-if $todo_tracked; then
605
- todo_untracked_count=0
239
+
240
+section "sensitive data"
241
+scan_files=()
242
+
243
+for path in AGENTS.md CLAUDE.md GEMINI.md .agents/ENV.md .agents/sow/SOW.template.md .agents/sow/specs/README.md; do
244
+ [ -f "$path" ] && scan_files+=("$path")
245
+done
246
+
247
+if [ -d .agents/sow/specs ]; then
248
+ while IFS= read -r file; do
249
+ scan_files+=("$file")
250
+ done < <(find .agents/sow/specs -type f -name '*.md' 2>/dev/null | sort)
251
fi
607
-echo
252
609
-# --- Backup of pre-SOW AGENTS.md ---
610
-echo "${BLUE}-- pre-SOW AGENTS.md backup --${NC}"
611
-if [ -f ./AGENTS.md.pre-sow.bak ]; then
612
- echo " ${GREEN}OK${NC} AGENTS.md.pre-sow.bak present (preserves original)"
613
-else
614
- if $initialized; then
615
- echo " ${GRAY}(no backup file; either init was clean or backup name differs)${NC}"
616
- else
617
- echo " ${GRAY}(not yet initialized)${NC}"
618
- fi
253
+if [ -d .agents/sow/active ]; then
254
+ while IFS= read -r file; do
255
+ scan_files+=("$file")
256
+ done < <(find .agents/sow/active -type f -name '*.md' 2>/dev/null | sort)
257
fi
620
-echo
621
-
622
-# --- Final verdict ---
623
-echo "${BLUE}-- verdict --${NC}"
624
-skill_classification_warnings=${non_project_skills_unclassified:-0}
625
-
626
-sow_status_errors=$((sow_status_mismatch + sow_status_missing))
627
-pre_impl_errors=$((sow_template_pre_impl_missing + current_sow_pre_impl_missing))
628
-sensitive_gate_errors=$((sow_template_sensitive_gate_missing + current_sow_sensitive_gate_missing + sensitive_findings))
629
-open_source_reference_errors=${sow_template_open_source_reference_missing:-0}
630
-completion_rule_errors=${sow_template_completion_rule_missing:-0}
631
-sow_evidence_errors=$((regression_order_violations + mirror_path_violations + open_source_reference_errors + completion_rule_errors))
632
-
633
-if [ "$sensitive_findings" -gt 0 ]; then
634
- echo " ${RED}=== CRITICAL: sensitive data patterns found in durable artifacts. Redact before commit. ===${NC}"
635
- exit 2
258
+
259
+if [ -d .agents/skills ]; then
260
+ while IFS= read -r file; do
261
+ scan_files+=("$file")
262
+ done < <(find .agents/skills -type f 2>/dev/null | sort)
263
fi
264
638
-if $initialized && [ "$sections_missing" -eq 0 ] && [ "$sensitive_warning_missing" -eq 0 ] && [ "$agents_warning_missing" -eq 0 ] && [ "$bridge_missing" -eq 0 ] && [ "$sow_dir_missing" -eq 0 ] && [ "$empty_sow_dir_missing_keep" -eq 0 ] && [ "$framework_missing" -eq 0 ] && [ "$sow_status_errors" -eq 0 ] && [ "$pre_impl_errors" -eq 0 ] && [ "$sensitive_gate_errors" -eq 0 ] && [ "$sow_evidence_errors" -eq 0 ] && [ "$todo_untracked_count" -eq 0 ] && [ "$skill_classification_warnings" -eq 0 ]; then
639
- echo " ${GREEN}=== SOW initialization complete and clean. ===${NC}"
640
- exit 0
641
-elif $initialized && [ "$sections_missing" -eq 0 ] && [ "$sensitive_warning_missing" -eq 0 ] && [ "$agents_warning_missing" -eq 0 ] && [ "$bridge_missing" -eq 0 ] && [ "$sow_dir_missing" -eq 0 ] && [ "$empty_sow_dir_missing_keep" -eq 0 ] && [ "$framework_missing" -eq 0 ] && [ "$sow_status_errors" -eq 0 ] && [ "$pre_impl_errors" -eq 0 ] && [ "$sensitive_gate_errors" -eq 0 ] && [ "$sow_evidence_errors" -eq 0 ] && [ "$todo_untracked_count" -eq 0 ]; then
642
- echo " ${YELLOW}=== SOW initialization structurally complete with skill classification warning(s):${NC}"
643
- echo " ${YELLOW}- ${skill_classification_warnings} non-project skill director(y/ies) need classification in AGENTS.md${NC}"
644
- echo " ${YELLOW}- Runtime input skills should be renamed/wrapped as .agents/skills/project-*/${NC}"
645
- echo " ${YELLOW}- Output/reference skills should be listed separately and kept out of the generic runtime hook${NC}"
646
- exit 0
647
-elif $initialized; then
648
- echo " ${YELLOW}=== SOW marker present but partial state detected:${NC}"
649
- [ "$sections_missing" -gt 0 ] && echo " ${YELLOW}- ${sections_missing} canonical AGENTS.md section(s) missing${NC}"
650
- [ "$sensitive_warning_missing" -gt 0 ] && echo " ${YELLOW}- CRITICAL sensitive-data warning missing from AGENTS.md${NC}"
651
- [ "$agents_warning_missing" -gt 0 ] && echo " ${YELLOW}- ${agents_warning_missing} AGENTS.md file(s) missing CRITICAL sensitive-data warning${NC}"
652
- [ "$bridge_missing" -gt 0 ] && echo " ${YELLOW}- ${bridge_missing} cross-tool instruction bridge(s) missing${NC}"
653
- [ "$sow_dir_missing" -gt 0 ] && echo " ${YELLOW}- ${sow_dir_missing} SOW directory(ies) missing${NC}"
654
- [ "$empty_sow_dir_missing_keep" -gt 0 ] && echo " ${YELLOW}- ${empty_sow_dir_missing_keep} empty SOW directory(ies) missing .gitkeep/.keep${NC}"
655
- [ "$framework_missing" -gt 0 ] && echo " ${YELLOW}- ${framework_missing} project-local framework file(s) missing${NC}"
656
- [ "$sow_status_mismatch" -gt 0 ] && echo " ${YELLOW}- ${sow_status_mismatch} SOW status/directory mismatch(es)${NC}"
657
- [ "$sow_status_missing" -gt 0 ] && echo " ${YELLOW}- ${sow_status_missing} SOW file(s) missing Status line${NC}"
658
- [ "$sow_template_pre_impl_missing" -gt 0 ] && echo " ${YELLOW}- project-local SOW template missing Pre-Implementation Gate${NC}"
659
- [ "$current_sow_pre_impl_missing" -gt 0 ] && echo " ${YELLOW}- ${current_sow_pre_impl_missing} current SOW(s) missing Pre-Implementation Gate${NC}"
660
- [ "$sow_template_sensitive_gate_missing" -gt 0 ] && echo " ${YELLOW}- project-local SOW template missing sensitive data gates${NC}"
661
- [ "$current_sow_sensitive_gate_missing" -gt 0 ] && echo " ${YELLOW}- ${current_sow_sensitive_gate_missing} current SOW(s) missing sensitive data handling/gate${NC}"
662
- [ "$sensitive_findings" -gt 0 ] && echo " ${YELLOW}- ${sensitive_findings} sensitive-data finding(s) in durable artifacts${NC}"
663
- [ "${sow_template_open_source_reference_missing:-0}" -gt 0 ] && echo " ${YELLOW}- project-local SOW template missing open-source reference evidence field${NC}"
664
- [ "${sow_template_completion_rule_missing:-0}" -gt 0 ] && echo " ${YELLOW}- project-local SOW template missing completed-status or one-commit close rule${NC}"
665
- [ "$regression_order_violations" -gt 0 ] && echo " ${YELLOW}- ${regression_order_violations} SOW file(s) have regression sections before original outcome/lessons/follow-up${NC}"
666
- [ "$mirror_path_violations" -gt 0 ] && echo " ${YELLOW}- ${mirror_path_violations} SOW file(s) use /opt/baddisk/monitoring/repos absolute paths instead of owner/repo @ commit citations${NC}"
667
- [ "$todo_untracked_count" -gt 0 ] && echo " ${YELLOW}- ${todo_untracked_count} untracked orphan TODO file(s) at project root${NC}"
668
- [ "$skill_classification_warnings" -gt 0 ] && echo " ${YELLOW}- ${skill_classification_warnings} non-project skill director(y/ies) need classification${NC}"
669
- echo " ${YELLOW} Repair non-destructively using the project-local AGENTS.md and .agents/sow/SOW.template.md.${NC}"
670
- exit 0
265
+if [ -d .agents/skill-verification ]; then
266
+ while IFS= read -r file; do
267
+ scan_files+=("$file")
268
+ done < <(find .agents/skill-verification -type f 2>/dev/null | sort)
269
+fi
270
+
271
+if [ "${#scan_files[@]}" -eq 0 ]; then
272
+ warn "no files selected for sensitive-data scan"
273
+elif bash .agents/sow/scan-sensitive.sh "${scan_files[@]}"; then
274
+ ok "sensitive-data scan passed (${#scan_files[@]} files)"
275
else
672
- echo " ${YELLOW}=== SOW NOT initialized. Install a project-local SOW framework before using SOWs here. ===${NC}"
673
- if [ ! -f ./AGENTS.md ]; then
674
- echo " ${YELLOW} AGENTS.md missing; create or normalize project instructions first.${NC}"
276
+ fail "sensitive-data scan found potential leaks"
277
+fi
278
+
279
+section "summary"
280
+if [ "$failures" -eq 0 ]; then
281
+ if [ "$warnings" -eq 0 ]; then
282
+ echo "${GREEN}PASS${NC} no SOW audit failures or warnings"
283
+ else
284
+ echo "${YELLOW}PASS${NC} no SOW audit failures; warnings=$warnings"
285
fi
286
exit 0
287
fi
288
+
289
+echo "${RED}FAIL${NC} failures=$failures warnings=$warnings"
290
+exit 1
.agents/sow/current/.gitkeep
.agents/sow/current/SOW-0016-20260510-collector-taxonomy-unification.md
deleted
-728
@@ -1,728 +0,0 @@
1
-
2
-# SOW-0016 - Unify collector metric taxonomy with Cloud-Frontend dashboard TOC
3
-
4
-## Status
5
-
6
-Status: in-progress
7
-
8
-Sub-state: one-PR framework+POC scope locked by user on 2026-05-11 after the audit-phase start. SOW moved from `pending/` to `current/`; on 2026-05-14 the user superseded the "structural taxonomy only" v1 boundary and required v1 to cover full cloud-frontend TOC shapes, including grids, context/table widgets, ordered alternatives, and nested groups. The full-shape v1 contract was redesigned, adversarially reviewed, amended, and re-reviewed as READY TO IMPLEMENT. Implementation has resumed under the full-shape contract. Full collector taxonomy coverage, global all-collector fatality, production ibm.d sweep, and cloud-frontend consumption are follow-up work unless explicitly pulled into the POC by user decision. After each major implementation step, if an external Claude review would add value, provide a focused prompt with exact files and questions.
9
-
10
-Implementation progress 2026-05-11: framework, schemas, generator/checker/seed tooling, CI wiring, docs/spec/skills updates, and five go.d POC collector taxonomies were implemented locally and committed as a framework POC snapshot. 2026-05-14: structural-only implementation was paused, full-shape v1 was redesigned/reviewed, and the framework/POC files were updated locally to the ordered recursive `items:` contract.
11
-
12
-## Requirements
13
-
14
-### Purpose
15
-
16
-Eliminate cross-repo taxonomy drift between Netdata's public collector definitions and the private cloud-frontend dashboard TOC. Ownership moves next to collectors. Validation lives entirely in the public netdata repo. Cloud-frontend consumes a generated JSON artifact, exactly as it consumes `integrations.js` today.
17
-
18
-**Scope of this SOW (Netdata-only)**: framework schemas, generator, validators, CI gates, seed tooling, contributor docs/skills, and a small POC set of collector `taxonomy.yaml` files. The published artifact `integrations/taxonomy.json` is the contract surface. Full collector taxonomy coverage is deliberately out of the initial PR.
19
-
20
-**Out of scope**: cloud-frontend consumption work (consumer module, legacy taxonomy module removal, chart-spec extraction, `dynamicSections` removal, regex-catchall removal, rollback runbook). The FE team owns Phase B on their schedule; tracked as a downstream FE-team SOW.
21
-
22
-### User Request
23
-
24
-> The collector definitions and frontend taxonomy are disconnected. Currently:
25
-> - Collectors define metrics and contexts in `metadata.yaml`
26
-> - Cloud frontend separately defines taxonomy/TOC mappings in JS
27
-> - There is no validation that taxonomy entries reference valid metrics/contexts
28
-> - CI cannot validate consistency because:
29
-> - `netdata` repo is public
30
-> - `cloud-frontend` repo is private
31
-> - we cannot grant the public repo access to the private repo
32
-> This causes taxonomy drift and broken references.
33
->
34
-> Move taxonomy ownership closer to collectors. Each collector should define its own taxonomy in YAML near the collector itself (either embedded in `metadata.yaml`, or stored in a dedicated taxonomy YAML file). Then Python scripts should aggregate all collector taxonomy YAML files, produce one normalized JSON artifact. Cloud frontend CI will later consume this JSON and generate JS code (out of scope).
35
-
36
-User refinements (2026-05-10):
37
-
38
-> we dont care about less churn but clean end state. Lets do as much as possible now, without phase1/2 - i mean about the framework (creating taxonomy.yaml for each collector is routine work).
39
-
40
-> about isSingleNode - this is important. Different view depends on the view, we need this in the taxonomy.
41
-
42
-User refinement (2026-05-11):
43
-
44
-> we will do everything in one PR (framework - w/o adding taxonomy for all collectors, can add a few as a POC).
45
-
46
-User refinement (2026-05-14):
47
-
48
-> There is no need to narrow the scope of v1, it should cover everything. Less churn is not our concern.
49
-
50
-User decision (2026-05-14):
51
-
52
-> Pause implementation, redesign the v1 full-TOC schema/contract first, then run adversarial review before implementation resumes.
53
-
54
-User decision (2026-05-14):
55
-
56
-> Cloud-frontend is not written in stone for this phase. SOW-0016 may define a clean Netdata-side taxonomy contract that requires downstream FE adapter/renderer changes, as long as those FE changes are clean and not compatibility hacks.
57
-
58
-### Assistant Understanding
59
-
60
-Facts (established from 3 independent Opus 4.7 analysis reports under `.local/audits/taxonomy-design/`):
61
-
62
-- `cloud-frontend/src/domains/charts/toc/taxonomy/` is ~15,160 LoC across 18 JS files. It is a dashboard-composition DSL, not a flat taxonomy: it mixes section structure, context references, regex catchalls, function-typed entries (`({ isSingleNode }) => ...`), grids of pre-configured chart widgets (`type: "grid"`), and chart-spec bodies (`type: "context"`). Only ~5–10% of the LoC is pure structural taxonomy.
63
-- The integrations marketplace axis (`integrations/categories.yaml` + `meta.monitored_instance.categories`) is **orthogonal** to the dashboard TOC: MySQL is `data-collection.databases` in the catalog and `Applications > MySQL` in the TOC. The two axes have different cardinality, different routing target (catalog page vs in-app dashboard), different consumers.
64
-- The bridge token between collectors and the TOC is the chart context name (e.g. `mysql.queries`). Contexts are already declared in `metadata.yaml` under `metrics.scopes[*].metrics[*].name` (`integrations/schemas/collector.json:264-394`). This is the cross-reference key.
65
-- The integrations pipeline (`integrations/gen_integrations.py`, 1469 LoC) discovers metadata.yaml files via `COLLECTOR_SOURCES` (`gen_integrations.py:27-37`), validates against JSON schemas (Draft7), and emits `integrations/integrations.{js,json}` plus per-integration markdown via Jinja templates. CI is `.github/workflows/generate-integrations.yml` and `check-markdown.yml`. Warnings become fatal in CI via `fail_on_warnings()` (`gen_integrations.py:155-174`).
66
-- ibm.d collectors generate `metadata.yaml` from `contexts.yaml` via `go generate`. Embedding taxonomy into `metadata.yaml` would force generator-on-generator complexity. ibm.d is the structural reason to use a sibling file.
67
-- All three independent analyses converged unanimously on: sibling-file design, cross-cutting parent registry, categories-vs-TOC are different axes, cross-reference validation as the load-bearing CI check, ICOn registry as a string-keyed allowlist with FE-side asset map.
68
-
69
-Inferences (not directly stated):
70
-
71
-- "Clean end state, no framework phasing" now applies to the framework PR only: the framework lands in one delivery with POC collector taxonomies. It does not imply a full initial taxonomy sweep for every collector.
72
-- The `isSingleNode` requirement implies that view-conditional rendering exists at the section/structure level, not just inside chart-spec bodies. The schema must support this as a first-class concept.
73
-- Superseded 2026-05-14: v1 is no longer limited to section/context taxonomy. The public contract must model full TOC shape where needed for parity: ordered items, structural groups, owned context leaves, flattening groups, selector leaves, grids, context/table widgets, first-available alternatives, and view-conditioned item bodies. `include_charts:` handles remain absent; the replacement is explicit typed item bodies in `taxonomy.yaml`.
74
-
75
-Unknowns (resolve before the implementation step that depends on them):
76
-
77
-- Whether function-typed entries appear ONLY inside chart-spec bodies, or also at section-structure level. If the latter, the schema needs richer condition expressions than `view: single_node | multi_node`.
78
-- Whether `families: true` semantics depend on Netdata's `family` chart attribute (which may be deprecating in some flows). Maintainer confirmation needed before locking the schema.
79
-- The exhaustive set of icon keys used across all 18 cloud-frontend taxonomy files (~175 from `icons.js`, but verify by grep).
80
-- Whether `virtualContexts` ever appear at section-structure level (vs only inside chart-spec bodies). If at structure level, schema needs a `virtual:` opt-in.
81
-- Whether ibm.d's `contexts.yaml` schema needs extension to carry canonical `section_id`, `priority`, and subsection/placement metadata for production codegen. This only blocks the initial PR if an ibm.d collector is selected as a POC.
82
-
83
-### Acceptance Criteria
84
-
85
-- Audit evidence required by a framework component exists before that component lands. There is no separate 10-output audit gate before implementation; unresolved design forks must not be hidden in code.
86
-- **Full-shape redesign gate (added 2026-05-14)**: satisfied. The ordered-`items:` v1 authoring/output contract was written, amended after adversarial review, and re-reviewed. Draft/review artifact: `.local/audits/taxonomy-design/full-shape-v1-redesign.md`.
87
-- `integrations/_common.py` extracted; existing `integrations/integrations.json` AND `integrations/integrations.js` are byte-identical before/after the refactor (verified by the `diff -u` baseline-copy procedure in the implementation plan, not by `git diff --exit-code`, because these outputs are gitignored ephemeral artifacts). **Gate-fatal.**
88
-- `integrations/gen_taxonomy.py` exists, runs in CI, schema-validates every committed `taxonomy.yaml`, cross-references against `metadata.yaml` contexts, and emits `integrations/taxonomy.json`. Verified by green CI on the single implementation PR.
89
-- `integrations/gen_taxonomy_seed.py` exists and seeds flat structural `items:` lists from `metadata.yaml`. Documented in the integrations contributor docs with the direct script command. **Initial PR hard requirement.**
90
-- `integrations/check_collector_taxonomy.py` exists as a fresh wrapper around `_common.py` and taxonomy validators (NOT a clone of the stale `check_collector_metadata.py`).
91
-- Output is **deterministic**: re-running `gen_taxonomy.py` 10× on identical input produces byte-identical `taxonomy.json`. Verified by golden test.
92
-- Cross-reference validator catches every TAX invariant in v1 (TAX001–TAX025, TAX028–TAX038; TAX026/TAX027/TAX040–TAX042 removed with `only_views:` drop and chart-recipe-manifest removal). Lint-code matrix in spec doc lists every code with severity, example, remediation, and superseded codes.
93
-- **Selector overlap detection**: across all three selector types (`contexts`, `context_prefix`, `collect_plugin`), pairwise intersection raises a fatal error.
94
-- **Closed core schema (Decision 13)**: misspelled field names (e.g. `single-node`, `include_chart`) fail the schema validator. Verified by negative tests.
95
-- Every collector whose `metadata.yaml` metrics block or `taxonomy.yaml` is touched in the implementation PR has matching taxonomy coverage in the same PR (fatal — Decision 12). Global all-collector coverage is informational/warning only in this SOW.
96
-- `taxonomy_optout: { reason: "..." }` is a top-level per-collector authoring object in `taxonomy_collector.json`; it is mutually exclusive with `placements`, cannot appear inside a placement, and requires a non-empty reason. `inline_dynamic_declarations` may appear alongside `taxonomy_optout` only for no-metadata plugins documented by audit 1.6. `taxonomy_output.json` carries opt-out collectors in a separate `opted_out_collectors` array, not as empty placements.
97
-- Every structural literal owner and widget literal reference resolves to a real declared context unless the exact widget reference carries the explicit unresolved-reference escape hatch (TAX003 fatal from day 1; TAX038 warns when an escape hatch becomes stale).
98
-- Every `context_prefix:` is declared in `metadata.yaml.metrics.dynamic_context_prefixes:` of the owning collector (or inline in `taxonomy.yaml` for plugins without `metadata.yaml`); TAX031 fatal.
99
-- Every `collect_plugin:` is declared in `metadata.yaml.metrics.dynamic_collect_plugins:` (or inline); TAX035 fatal.
100
-- Every `context_prefix_exclude:` is paired with a `context_prefix:` and contains valid prefix strings; otherwise TAX029 fatal.
101
-- Every `section_id` resolves against `integrations/taxonomy/sections.yaml`. `section_path:` authoring is rejected by the closed v1 schema. Verified by validator (fatal).
102
-- Every `icon` is in `integrations/taxonomy/icons.yaml` allowlist. Verified by validator (fatal).
103
-- ~~`include_charts:` validation~~ — **REPLACED 2026-05-14**: no `include_charts:` handle namespace in v1. Full-shape parity is represented directly by typed ordered `items:` entries such as `owned_context`, `group`, `flatten`, `selector`, `grid`, `context`, `first_available`, and `view_switch`.
104
-- Production ibm.d `taxonomy.yaml` codegen is not required in the initial framework+POC PR unless an ibm.d collector is selected as a POC. If included, `go generate ./src/go/plugin/ibm.d/modules/... && git diff --exit-code` must be clean for the touched module(s).
105
-- `integrations/taxonomy.json` matches `integrations/schemas/taxonomy_output.json` (self-validation in `gen_taxonomy.py`). Output schema version is `taxonomy_schema_version: 1`.
106
-- Output JSON includes `source: { netdata_commit, generated_at }` metadata (Decision 3 amendment).
107
-- Output JSON carries unresolved selectors, build-time-resolved owned-context snapshots, and display-reference snapshots per placement/item (Decision 2 amendment plus 2026-05-14 ownership/reference split).
108
-- Legacy diff tooling and full drift triage are follow-up work for the full collector migration, not acceptance criteria for the initial framework+POC PR.
109
-- Performance budget: full taxonomy validation completes in <5 seconds on the current fleet; synthetic 10K-context fixture completes in <10 seconds. Verified by CI timing.
110
-- `Finding` model emits valid GitHub Actions annotations (`::error file=PATH,line=N,title=TAXNNN::MESSAGE`), text, JSON sidecar, and optional SARIF.
111
-- Collector consistency policy includes `taxonomy.yaml` as the dashboard TOC placement artifact. Documented in `AGENTS.md` and `.agents/skills/integrations-lifecycle/`.
112
-- **Cloud-frontend Phase B is NOT a SOW-0016 acceptance criterion** (out of scope per 2026-05-11 user clarification). Netdata-side SOW closes when the single framework+POC PR merges with green validation. FE-team Phase B is tracked separately on their schedule.
113
-
114
-## Analysis
115
-
116
-Sources checked:
117
-
118
-- `/Users/ilyam/Projects/github/ilyam8/cloud-frontend/src/domains/charts/toc/taxonomy/` — 18 JS files, fully read by 3 independent agents.
119
-- `/Users/ilyam/Projects/github/ilyam8/netdata/src/go/plugin/go.d/collector/mysql/metadata.yaml` and ~5 other representative collectors (apache, postgres, nvidia_smi, snmp, db2/ibm.d).
120
-- `/Users/ilyam/Projects/github/ilyam8/netdata/integrations/gen_integrations.py:1-1469` — full pipeline read.
121
-- `/Users/ilyam/Projects/github/ilyam8/netdata/integrations/schemas/collector.json` — collector schema (627 lines).
122
-- `/Users/ilyam/Projects/github/ilyam8/netdata/integrations/categories.yaml` — catalog axis.
123
-- `/Users/ilyam/Projects/github/ilyam8/netdata/.github/workflows/generate-integrations.yml` and `check-markdown.yml`.
124
-- `.agents/skills/integrations-lifecycle/` — current pipeline knowledge.
125
-- `.agents/sow/specs/` — checked for prior taxonomy specs (none).
126
-- 3 independent Opus 4.7 agent reports under `.local/audits/taxonomy-design/`.
127
-
128
-Current state:
129
-
130
-- Cloud-frontend taxonomy is hand-maintained JS, no validation, no cross-reference to collector contexts. Drift is invisible until a chart fails to render.
131
-- `metadata.yaml` already declares the full set of contexts every collector emits (`metrics.scopes[*].metrics[*].name`). All cross-reference data needed by the new validator is already present in the public repo — no new data sources required.
132
-- `gen_integrations.py` is the obvious plug-in point. Discovery, schema validation, warning-fatal-in-CI patterns exist and can be reused.
133
-- `meta.monitored_instance.categories` is structurally separate from the TOC. No code in the cloud-frontend taxonomy references it.
134
-- ibm.d's `contexts.yaml` codegen pattern (`go generate` emits `metadata.yaml`) extends naturally to also emit `taxonomy.yaml`.
135
-
136
-Risks:
137
-
138
-- **Function-typed entries (`isSingleNode` and similar)**: simple scalar/list deltas remain handled by the curated-and-override pattern (`single_node:` sparse block). Full item-body switches are now in scope for v1 via the proposed `type: view_switch` item; review must confirm this covers every current `({ isSingleNode }) => ...` occurrence without reintroducing `only_views:`.
139
-- **Regex non-equivalence between JS and Python**: irrelevant since Decision 1 drops regex entirely. Risk eliminated.
140
-- **`families: true` may depend on a deprecating attribute**: medium risk. Mitigation: confirm semantics before finalizing the schema; if `family` is being phased out, schema gets `group_by_label: <label>` as a sibling/replacement field before the generator/checker ships.
141
-- **Catalog vs TOC contributor confusion**: low/medium risk. Mitigation: explicit guidance in `AGENTS.md` collector-consistency rule + integrations-lifecycle skill update; pre-commit lint flags suspicious `categories:` edits that look like TOC tweaks.
142
-- **Full migration size**: ~150 collector `taxonomy.yaml` files remain out of the initial PR. Mitigation: the initial PR proves the framework and POC shapes; full migration gets its own follow-up SOW/PR plan.
143
-- **Downstream FE consumption timing**: if the FE team consumes `taxonomy.json` later than the Netdata framework PR, the public repo temporarily publishes an unconsumed artifact. Acceptable — the framework is still useful for validation and later migration.
144
-- **Coverage-fatal flip timing**: global all-collector fatality is deliberately deferred. The initial PR enforces changed/touched collector coverage only, so unrelated collector PRs are not blocked by missing taxonomy files.
145
-- **Schema evolution**: future view axes (beyond `single_node | multi_node`) will require a schema bump. Mitigation: `taxonomy_version: 1` is pinned in every file; major bumps are explicit and gate-able. Core authoring fields use **closed schemas** (no broad `additionalProperties: true` for `single_node`, selector declarations, etc.); only namespaced extension keys (`x_*`) are permitted on core nodes (Decision 13).
146
-- **Virtual contexts at section-structure level (factually present, not absent)**: `cloud-frontend/.../taxonomy/systemStorage.js:3-31` defines non-empty `virtualContexts`, and one is consumed in the taxonomy structure at `systemStorage.js:103`. The SOW's previous "confirm none" framing of audit step 1.4 is wrong. Mitigation: rewrite audit 1.4 to classify every `virtualContexts` def/use and assign one of {frontend recipe handle, encoded in generated contract, explicit diff exception, deferred with new SOW}.
147
-- **`netdata.*` negative-lookahead selector cannot be expressed by the three-matcher set**: `cloud-frontend/.../taxonomy/netdata.js:79-83` uses `^netdata\.(?!(ebpf|statsd|apps|tcp_connects|tcp_connected|private_charts|machine_learning|training|metric_types|queue_ops|queue_size|plugin)).*`. A bare `context_prefix: ["netdata."]` would over-claim contexts that the cloud-frontend deliberately routes elsewhere. Mitigation: Decision 2 amended to require an explicit per-collector exclusion field (or static enumeration); regex remains forbidden.
148
-- **First-available chart alternatives are in scope after the 2026-05-14 decision**: `cloud-frontend/.../charts/toc/getMenu.js:77-82` and `:91-98` execute "if item is array, choose first available context" semantics. Used in Kubernetes (`kubernetes.js:139-155, 183-198, 273-288, 337-352, 405-421, 445-461, 485-500, 527-538`), Containers/VMs (`containersAndVms.js:311-324, 347-365, 574-593, 616-674, 702-712`), and Pulsar grid (`applications.js:3902-4001`). Mitigation: v1 redesign adds a typed `first_available` item whose alternatives are fully validated against metadata and preserved for FE runtime selection.
149
-- **`_collect_plugin` selector feasibility is unproven on the FE side**: Agent stores the label on RRDSETs (`src/database/rrdset-index-id.c:23-27`), but `getMenu.js:50-56` filters by chart id, not by labels. Mitigation: audit step 1.10 remains a non-blocking coordination note; Netdata can publish the selector contract, and FE may adapt or open a selector-replacement SOW.
150
-- **Cloud-frontend JSON consumption is plausible but unproven (mitigation revised 2026-05-11)**: current FE consumes `integrations.js` via `cloud-frontend/.github/workflows/sync-to-s3.yaml:47-67`, not a taxonomy JSON. After scope correction (FE out of scope of SOW-0016), this is the FE team's responsibility — they extend their `sync-to-s3.yaml` to also run `gen_taxonomy.py` and copy `integrations/taxonomy.json` into their tree, exactly as they already do for `integrations.js`. SOW-0016 publishes the artifact and the `taxonomy_output.json` schema; consumption is downstream. If the FE team finds the consumption infeasible (audit 1.10 coordination response), that's a downstream FE-SOW design problem, not a blocker on SOW-0016.
151
-- **FE consumption is non-trivial but downstream**: `applications.js` is 5,885 LoC, `kubernetes.js` 1,228, `systemStorage.js` 1,567, `systemHardware.js` 1,241, `containersAndVms.js` 1,165; `contexts.js` is 34,055 LoC. Existing FE tests cover overview-vs-single-node flavor selection (`getMenu.test.js:326-343`), menu ancestry (`:369-423`), regex sections (`:450-506`), grids (`:509-517`), and virtual contexts (`:520-528`). Mitigation: this SOW does not gate on FE refactor timing; FE Phase B is tracked separately.
152
-- **Rollback after framework+POC PR can leave metadata declarations behind**: POC collector `metadata.yaml` may carry new `dynamic_*` declarations even if POC taxonomy files are reverted. Mitigation: Rollback Matrix records whether reverted `dynamic_*` fields are removed with the taxonomy revert or intentionally retained as accurate emission facts.
153
-- **Full collector author cost is not "routine"**: at the postgres rate (70 explicit context lines), 150 collectors implies ~10,500 explicit context lines plus structure/overrides/comments. Mitigation: `gen_taxonomy_seed.py` is an initial PR hard requirement so follow-up migration is seed + human review, not hand authoring from scratch.
154
-- **Stale `check_collector_metadata.py` reuse trap**: it imports `SINGLE_PATTERN`, `MULTI_PATTERN`, `SINGLE_VALIDATOR`, `MULTI_VALIDATOR` from `gen_integrations` (`integrations/check_collector_metadata.py:8-9`), but the current generator defines none of those symbols; it is not wired into any active workflow. Mitigation: Decision 4 amended; `check_collector_taxonomy.py` is fresh, not a clone.
155
-- **Path-as-identity is brittle**: if a collector authored a path such as `[applications, postgres]`, moving postgres under an intermediate Databases section would mass-edit every collector taxonomy referencing it. Mitigation: Decision 8 locks `section_id` as an opaque stable handle; dots in an ID are namespace punctuation only and do not define parentage. Section moves are expressed in `sections.yaml` via `parent_id` changes, not by editing collector YAML; `taxonomy.json` carries both stable `id` and resolved `path`.
156
-- **Selector lookup performance**: naive `[k for k in CONTEXT_INDEX if k.startswith(prefix)]` per-prefix per-placement is O(P·C·placements). Mitigation: validation algorithm specifies sorted-context bisect for prefix and an index for collect-plugin; synthetic 10K-context performance fixture added to acceptance criteria.
157
-
158
-## Pre-Implementation Gate
159
-
160
-Status: **SATISFIED 2026-05-14** after the amended full-shape v1 contract review cleared and the framework+POC implementation resumed. The prior structural-only gate was superseded; the active gate is now satisfied for the single framework+POC PR scope.
161
-
162
-Problem / root-cause model:
163
-
164
-- Two repositories (public netdata, private cloud-frontend) own complementary halves of the same taxonomy: collectors emit chart contexts in `metadata.yaml`; cloud-frontend renders them via hand-maintained JS modules under `domains/charts/toc/taxonomy/`. There is no machine-readable contract between the two halves, no CI gate that crosses the boundary, and no validation that taxonomy entries reference real contexts. Drift accumulates silently until a chart fails to render or a collector renames a context that some taxonomy file still references. The fix is structural: move the dashboard TOC taxonomy contract (sections, paths, contexts, families flag, icon keys, view-conditional rendering, and full-shape item bodies where needed for parity) into the public repo as collector-adjacent YAML; build a Python aggregator that fails CI on any cross-reference mismatch; emit a JSON artifact the cloud-frontend consumes.
165
-
166
-Evidence reviewed:
167
-
168
-- `cloud-frontend/src/domains/charts/toc/taxonomy/index.js:13-100` (root dashboards map), `applications.js` (5,885 LoC), `kubernetes.js`, `system.js`, `containersAndVms.js`, `netdata.js`, `icons.js` — all read in full by 3 agents.
169
-- `netdata/integrations/gen_integrations.py:1-1469`, `integrations/schemas/collector.json:1-627`, `integrations/categories.yaml`.
170
-- `netdata/src/go/plugin/go.d/collector/mysql/metadata.yaml` and 5+ other representative collectors.
171
-- `.github/workflows/generate-integrations.yml`, `check-markdown.yml`.
172
-- `.agents/skills/integrations-lifecycle/` — existing pipeline knowledge.
173
-- 3 independent Opus 4.7 analysis reports: `.local/audits/taxonomy-design/agent-{1,2,3}-analysis.md`.
174
-- 3 schema-readability persona reviews: `.local/audits/taxonomy-design/schema-review-{1,2,3}.md`.
175
-- 7-collector empirical schema validation: `.local/audits/taxonomy-design/schema-validation-drafts.md`.
176
-- External Codex review (6 parallel GPT-5.5-xhigh subagents + synthesis, 2026-05-10):
177
- - Synthesis: `.local/audits/taxonomy-design/external-review-codex/SYNTHESIS.md`
178
- - 01 Schema correctness: `.local/audits/taxonomy-design/external-review-codex/01-schema-correctness.md`
179
- - 02 Readability: `.local/audits/taxonomy-design/external-review-codex/02-readability.md`
180
- - 03 Pipeline: `.local/audits/taxonomy-design/external-review-codex/03-pipeline.md`
181
- - 04 Migration: `.local/audits/taxonomy-design/external-review-codex/04-migration.md`
182
- - 05 Cross-domain: `.local/audits/taxonomy-design/external-review-codex/05-cross-domain.md`
183
- - 06 Readiness: `.local/audits/taxonomy-design/external-review-codex/06-readiness.md`
184
-- Direct file:line evidence cited in this gate (selected, non-exhaustive):
185
- - `cloud-frontend/.../taxonomy/index.js:13-62` — root dashboards map; `dynamicSections: true` for room and node.
186
- - `cloud-frontend/.../charts/toc/getMenu.js:50-75` — regex/test entries; `:77-82` and `:91-98` first-available array semantics; `:89` `isSingleNode` derivation; `:100-102` function-typed grid items; `:171-180` virtual-context rendering; `:238-350` family hierarchy; `:425-439` dynamic fallback for unmatched contexts.
187
- - `cloud-frontend/.../taxonomy/getMenu.test.js:326-343` overview/single-node flavor; `:369-423` menu ancestry; `:450-506` regex sections; `:509-517` grids; `:520-528` virtual contexts.
188
- - `cloud-frontend/.../taxonomy/systemStorage.js:3-31` non-empty `virtualContexts`; `:103` virtual-context referenced inside taxonomy structure.
189
- - `cloud-frontend/.../taxonomy/netdata.js:79-83` negative-lookahead regex over `netdata.*`.
190
- - `cloud-frontend/.../taxonomy/system.js:44-68, 117-168` and `systemMemory.js:35-80` and `remoteDevices.js:5-15, 93-131` and `containersAndVms.js:3-152, 311-365, 574-712` and `applications.js:3902-4001` (Pulsar) — function-typed entries and array alternatives.
191
- - `cloud-frontend/.github/workflows/sync-to-s3.yaml:47-67` — current FE artifact ingestion path; copies `integrations.js`, not taxonomy JSON.
192
- - `cloud-frontend/package.json:151-165` — no current taxonomy artifact import path.
193
- - `netdata/integrations/gen_integrations.py:27-37` `COLLECTOR_SOURCES`; `:155-174` warning-fatal; `:177-238` Draft7+Registry; `:326-339` collector globbing; `:379-407` `load_collectors`; `:822-830` ID synthesis; `:839-842` deterministic sort; `:861-996` mutating render; `:1414-1428` artifact emission; `:1431-1465` exit/fail handling.
194
- - `netdata/integrations/check_collector_metadata.py:8-9` — imports stale symbols from `gen_integrations`; not wired into active workflows.
195
- - `netdata/integrations/schemas/collector.json:264-395` `metrics` block; `:283-395` static metric context schema; no existing `additionalProperties: false`.
196
- - `netdata/.github/workflows/generate-integrations.yml:1-24, 47-66, 64-81` — current path triggers, generation, artifact cleanup.
197
- - `netdata/.github/workflows/check-markdown.yml:3-11` — current changed-file path triggers (no taxonomy paths).
198
- - `netdata/src/go/plugin/ibm.d/AGENTS.md:28-51` — generated files, source-of-truth files (`contexts.yaml`, `config.go`, `module.yaml`).
199
- - `netdata/src/go/plugin/ibm.d/docgen/main.go:27-47` — current `Context` struct (no taxonomy fields); `:119-153` and `:360-387` and `:389-529` generation flow.
200
- - `netdata/src/go/plugin/ibm.d/modules/db2/generate.go:1-3` and `modules/db2/contexts/doc.go:1-5` — module-level `go generate` invocations.
201
- - `netdata/src/collectors/statsd.plugin/example.conf:6-14, 30-43` — user-supplied app names and arbitrary user-defined chart contexts.
202
- - `netdata/src/collectors/statsd.plugin/statsd.c:1596-1628, 2254-2274` — `statsd.plugin` plugin label.
203
- - `netdata/src/database/rrdset-index-id.c:23-27` — `_collect_plugin` and `_collect_module` labels on RRDSETs.
204
- - `netdata/src/go/plugin/go.d/collector/mysql/metadata.yaml:1138-1149` — `mysql.galera_open_transactions` exists; no `mysql.open_transactions` (drift evidence vs `applications.js:1957`).
205
-- No external open-source repositories were consulted as design references; cross-domain comparisons (Kubernetes CRDs, OpenAPI/JSON Schema, Prometheus relabel, OpenTelemetry semconv, VS Code/JetBrains marketplace) are documented in `.local/audits/taxonomy-design/external-review-codex/05-cross-domain.md` for context only.
206
-
207
-Affected contracts and surfaces:
208
-
209
-- **New schemas**: `integrations/schemas/taxonomy_collector.json`, `taxonomy_sections.json`, `taxonomy_output.json`.
210
-- **Modified schema**: `integrations/schemas/collector.json` — adds two optional dynamic-context declarations under `metrics:`:
211
- - `dynamic_context_prefixes: [{prefix, reason}, ...]` — for collectors whose dynamic contexts share a namespace prefix (snmp, prometheus scraper, cgroup, apps).
212
- - `dynamic_collect_plugins: [{plugin, reason}, ...]` — for collectors whose dynamic contexts have no shared prefix (statsd.plugin, charts.d.plugin, python.d.plugin).
213
-- **New runtime artifacts**: `integrations/taxonomy/sections.yaml`, `integrations/taxonomy/icons.yaml`, `integrations/taxonomy.json` (emitted; ephemeral gitignored per Decision 3 — matches `integrations.js` precedent).
214
-- **New code**: `integrations/_common.py` (narrow shared extract — see the implementation plan for the explicit allow-list), `integrations/gen_taxonomy.py`, `integrations/gen_taxonomy_seed.py` (Decision 9), `integrations/check_collector_taxonomy.py` (fresh; not a clone of the stale `check_collector_metadata.py`, see Decision 4 amendment).
215
-- **Modified**: `integrations/gen_integrations.py` (refactored to use `_common.py`; byte-identical output requirement is acceptance-gate-fatal), `.github/workflows/generate-integrations.yml` (new step + path triggers including `**/taxonomy.yaml`, `integrations/taxonomy/**`, `integrations/schemas/taxonomy*.json`, `integrations/gen_taxonomy*.py`, `integrations/_common.py`), `check-markdown.yml` (changed-collector taxonomy gate per Decision 12).
216
-- **New per-collector files in this PR**: a small POC set of `<collector>/taxonomy.yaml` files. Full collector coverage is follow-up work.
217
-- **ibm.d framework**: production `contexts.yaml` schema/codegen extension is follow-up unless an ibm.d collector is selected as a POC.
218
-- **Cloud-frontend (OUT OF SCOPE of SOW-0016)**: FE team owns consumption — JSON consumer module, legacy taxonomy module removal, renderer adapter, `dynamicSections` removal, regex-catchall removal, rollback. Tracked as a downstream FE-team SOW. SOW-0016 publishes `integrations/taxonomy.json` and the `taxonomy_output.json` schema; that is the entire Netdata-FE contract surface.
219
-- **Project policy**: `AGENTS.md` collector-consistency rule includes `taxonomy.yaml`.
220
-- **Skills**: `integrations-lifecycle/` updated; `project-writing-collectors/` updated.
221
-- **Docs**: collector contributor docs reference `taxonomy.yaml` as a new required file.
222
-
223
-Existing patterns to reuse:
224
-
225
-- `gen_integrations.py:177-238` — Draft7Validator + Registry pattern.
226
-- `gen_integrations.py:155-174` — `WARNINGS` accumulator + `fail_on_warnings()` for CI-fatal gating; **wrap into a structured `Finding` model with multiple renderers** (text, valid GitHub Actions annotation `::error file=...,line=...,title=TAX003::`, JSON, optional SARIF) — the existing prefix `:warning file=...:` and `:error file=...:` strings are NOT valid GitHub Actions annotations and must not be propagated.
227
-- `gen_integrations.py:326-339` — collector-source globbing.
228
-- `gen_integrations.py:822-830` — collector-id synthesis pattern; reused so taxonomy and integrations agree on collector identity.
229
-- `gen_integrations.py:839-842` — explicit deterministic sort (`_index`, `_src_path`, `id`); taxonomy needs an equivalent locked merge key (Decision 12).
230
-- `integrations/categories.yaml` — frozen-list registry pattern (mirror for `sections.yaml`).
231
-- ibm.d `go generate` codegen pattern remains the production target, but only enters this PR if an ibm.d POC is selected.
232
-- **Do NOT mirror `integrations/check_collector_metadata.py`**: it imports symbols (`SINGLE_PATTERN`, `MULTI_PATTERN`, `SINGLE_VALIDATOR`, `MULTI_VALIDATOR`) the current `gen_integrations.py` no longer defines and is not invoked by any active workflow. Build `check_collector_taxonomy.py` fresh against `_common.py` and the new taxonomy validators.
233
-
234
-Risk and blast radius:
235
-
236
-- **Regression**: existing `gen_integrations.py` is refactored to use `_common.py`. Risk: introducing a regression in the existing pipeline. Mitigation: refactor as pure code-motion (no behavior change); diff the rendered `integrations.json` before/after to confirm byte-identical output.
237
-- **CI runtime**: new pipeline adds another pass. Estimated <5s. Negligible.
238
-- **Compatibility**: `taxonomy.json` is a new artifact; consumers (cloud-frontend) opt in. No breakage of existing artifacts.
239
-- **Performance**: aggregator builds an in-memory context index (~10K entries max). Linear in collector count; well within budget.
240
-- **Security**: no secret-handling involved; all data is public collector metadata.
241
-- **Data loss**: zero. New artifacts; no destructive edits.
242
-- **Migration**: initial PR touches only POC collectors. Full migration of ~150 collectors is follow-up work once the framework shape is proven.
243
-- **Rollout**: one framework+POC PR is reversible by revert. FE Phase B is downstream and outside this SOW.
244
-- **Operational**: zero impact on running agents. Taxonomy is build-time metadata, not runtime.
245
-
246
-Sensitive data handling plan:
247
-
248
-- This work involves zero credentials, secrets, customer data, or private endpoints. All inputs (`metadata.yaml`, cloud-frontend taxonomy JS) are non-sensitive structural metadata. All outputs (schemas, YAML files, generated JSON) are non-sensitive structural metadata. No redaction required in the SOW, specs, skills, code comments, or commits.
249
-- The cloud-frontend repo is private but its taxonomy source files are not sensitive in content; they merely live in a private repo for org-policy reasons. Downstream FE Phase B work is referenced only at a high level; no private source from that repo will be pasted into public artifacts beyond structural references already cited in this SOW.
250
-
251
-Implementation plan (active, 2026-05-11 one-PR framework+POC scope):
252
-
253
-1. **Inline audit evidence before dependent code**: produce the minimum audit outputs needed by the next implementation step under `.local/audits/taxonomy-design/audit/`. The implementation must not encode unresolved `to-decide` rows. Audit 1.10 remains a non-blocking FE coordination note. Audit 1.5 is required only if production ibm.d codegen or an ibm.d POC enters this PR.
254
-2. **Refactor shared integration helpers**: extract the narrow `_common.py` allow-list from `gen_integrations.py`; prove `integrations.{json,js}` are byte-identical before/after using `diff -u` against `.local/audits/taxonomy-design/refactor-baseline/` and `.local/audits/taxonomy-design/refactor-after/`.
255
-3. **Add taxonomy schemas and registries**: author `taxonomy_collector.json`, `taxonomy_sections.json`, `taxonomy_output.json`, `collector.json` dynamic declaration extensions, `integrations/taxonomy/sections.yaml`, and `integrations/taxonomy/icons.yaml`. Use `section_id:` as the only v1 authoring form.
256
-4. **Add generator/checker/seed tooling**: implement `gen_taxonomy.py`, `gen_taxonomy_seed.py`, and fresh `check_collector_taxonomy.py`; include deterministic ordering, selector overlap detection, schema self-validation, structured `Finding` renderers, and a 10K-context performance fixture.
257
-5. **Add POC collector taxonomies**: add a small representative POC set, defaulting to the previously selected reference collectors `mysql`, `postgres`, `apache`, `nvidia_smi`, and `snmp` unless implementation evidence shows one should be swapped. Do not add taxonomy for all collectors in this PR.
258
-6. **Wire CI for this scope**: `check-markdown.yml` is the PR-blocking gate; `generate-integrations.yml` is the post-merge artifact-generation path. Changed/touched collector coverage is fatal; global all-collector coverage remains warning/informational.
259
-7. **Update contributor-facing artifacts**: update `AGENTS.md`, `.agents/skills/integrations-lifecycle/`, `.agents/skills/project-writing-collectors/`, `.agents/sow/specs/taxonomy.md`, and an integrations contributor doc.
260
-8. **Review checkpoints**: after each major step, decide whether an external Claude review is useful. If yes, provide the user a focused prompt with exact files and questions.
261
-9. **Close this SOW**: SOW-0016 completes when the single framework+POC PR merges green and the SOW validation/artifact gates are filled. Full collector migration, production ibm.d sweep, global fatal coverage, drift triage, and FE consumption become follow-up SOWs unless explicitly pulled into this PR.
262
-
263
-Historical superseded PR-A1 / PR-A1.5 / PR-A2 implementation details were removed from the active plan on 2026-05-11 after the user locked the one-PR framework+POC scope. The reasoning remains in the Execution Log for provenance.
264
-
265
-### Drift Triage Process (Follow-Up Full Migration)
266
-
267
-Full legacy drift triage is not part of the initial framework+POC PR. When the follow-up full migration runs `diff_legacy.py`, each finding uses this disposition model:
268
-
269
-| Disposition | When | Action | Owner | Output |
270
-|---|---|---|---|---|
271
-| `drop-frontend-entry` | Legacy referenced a context that doesn't exist in any `metadata.yaml` | Omit from new taxonomy | Collector maintainer signs off | None — finding closed |
272
-| `fix-metadata` | Collector emits the context but `metadata.yaml` doesn't declare it | Add metric entry to `metadata.yaml` in the same sub-PR | Collector maintainer | Metadata diff |
273
-| `fix-collector` | Context renamed/removed at the collector | Restore or rename in source + emit + metadata | Collector maintainer | Collector + metadata diff |
274
-| `keep-frontend-only` | Entry is a `virtualContexts`-derived chart or other FE-only construct per audit 1.4 | Excluded from `taxonomy.json`; recorded so future diffs ignore it | FE owner notified | Row in `integrations/taxonomy/diff_exceptions.yaml` |
275
-| `defer-with-sow` | Out of scope for the full-migration SOW | New SOW required before that SOW closes | Assigned during triage | New SOW filename |
276
-
277
-Bulk migration implementer does NOT silently decide product semantics. Every finding has a named owner; the full-migration SOW cannot close while any finding is `to-decide`.
278
-
279
-**Escalation clock (NEW 2026-05-11 per readiness reviewer 1)**: each finding's named owner has **5 business days** from notification to either sign off on the proposed disposition or push back with an alternative. After 5 business days without response, the bulk-migration implementer escalates to the user (project lead) for tie-break. This prevents the full-migration SOW from stalling indefinitely on owner PTO or unresponsiveness. Notifications are recorded in the drift inventory row with a date stamp.
280
-
281
-### Rollback Matrix (Netdata-only)
282
-
283
-| Phase | What's installed | What stays after revert | Required follow-up reverts | CI mode after revert |
284
-|---|---|---|---|---|
285
-| Single framework+POC PR | `_common.py`, gen_taxonomy, schemas, sections.yaml, icons.yaml, seed/check tooling, CI wiring, docs/skills/specs, POC collector taxonomies | Nothing if fully reverted; `gen_integrations.py` returns to pre-refactor state via the same revert | Remove any POC `metadata.yaml.metrics.dynamic_*` declarations if the corresponding POC taxonomy is reverted and the declarations are not intentionally retained as emission facts | Pre-taxonomy-framework state |
286
-| Follow-up full migration | Remaining collector `taxonomy.yaml` files; possible no-metadata plugin handling; global all-collector fatality | If reverted, collector `metadata.yaml` extensions may become **zombie fields** unless explicitly removed in the same revert | Coordinated revert of `metadata.yaml` `dynamic_*` fields or explicit decision to retain them as accurate emission facts | Framework+POC state |
287
-
288
-Cloud-frontend rollback is the FE team's responsibility (Phase B out of scope of SOW-0016).
289
-
290
-### Audit Outputs (Support Evidence, Not A Separate Gate)
291
-
292
-These outputs remain the evidence checklist. Produce each before the implementation step that consumes it; no separate PR-A1 gate exists after the 2026-05-11 one-PR scope correction.
293
-
294
-- `1.1-dsl-inventory.md` — structured taxonomy-DSL inventory.
295
-- `1.2-families-semantics.md` — `families: true` contract from FE consumer + tests.
296
-- `1.3-icons-allowlist.md` — canonical icon-key list.
297
-- `1.4-virtualcontexts.md` — `virtualContexts` classification + dispositions.
298
-- `1.5-ibmd-prototype.md` — `db2` prototype + round-trip evidence (required only before production ibm.d codegen or an ibm.d POC enters scope).
299
-- `1.6-dynamic-collectors.md` — prefix-friendly vs plugin-label-friendly + no-metadata plugin dispositions.
300
-- `1.8-netdata-negative-selector.md` — chosen disposition for `netdata.js:79-83`.
301
-- `1.10-collect-plugin-question.md` — coordination note to FE team (non-blocking).
302
-- `1.11-section-id-map.md` — camelCase → kebab-case section ID map (top-level only).
303
-- `1.11b-sections-tree.md` — full sections.yaml draft (~80–150 entries with parent_id, section_order). NEW 2026-05-11.
304
-
305
-Total: 10 audit outputs (was 9 before reviewer pass surfaced the 1.11.b need), now consumed inline rather than as a standalone kickoff gate.
306
-
307
-**Removed 2026-05-11** (FE work out of scope of SOW-0016):
308
-- ~~`1.7-fe-adapter-spike.md`~~ — FE team's responsibility.
309
-- ~~`1.9-chart-recipes-seed.md`~~ — no chart-recipe manifest in v1 (Decision 14 removed).
310
-
311
-If an implementation step depends on an audit output, that output must exist and contain no unresolved `to-decide` rows before the step lands. Audit 1.10 is non-blocking; we proceed with `collect_plugin:` as the v1 selector unless the FE team raises a structural objection in time to change this PR.
312
-
313
-Validation plan:
314
-
315
-- **Unit / integration tests**:
316
- - `gen_taxonomy.py` runs against fixture `taxonomy.yaml` files; expected JSON output is golden-tested AND deterministic across runs.
317
- - Cross-reference validator: positive test (valid taxonomy passes), negative tests for active validation families including TAX003 (unknown context), TAX021 (unknown view-override key), TAX022 (`multi_node:` block), TAX023 (list-merge attempt), TAX024 (empty `single_node:`), TAX031/TAX035 (selector not declared in metadata), TAX036 (selector ownership overlap), TAX037 (referenced literal context has no owner), and TAX038 (unresolved escape hatch is now stale because the context resolves). TAX002 and TAX032 are reserved follow-up codes, not emitted in the framework+POC PR.
318
- - Selector overlap validator: positive and negative tests for cross-type overlap (static `contexts:` claimed by one collector AND `context_prefix:` matching the same context claimed by another → fatal).
319
- - Schema validator: positive AND negative tests for each schema file, including typo cases (`single-node` vs `single_node`, `include_chart` vs `include_charts`, etc.) — each must fail under the closed core schema (Decision 13).
320
- - Performance: synthetic 10K-context fixture must validate in <10 seconds.
321
- - Deterministic merge: re-run `gen_taxonomy.py` 10× on the same input; output must be byte-identical.
322
- - `_common.py` refactor: byte-identical `integrations.json` AND `integrations.js` before/after via the Plan step 2.2 `diff -u` baseline-copy procedure. **Gate-fatal.**
323
- - `Finding` renderers: text format, GitHub Actions annotation format (must match `::error file=PATH,line=N,title=TAXNNN::MESSAGE` regex), JSON sidecar shape, optional SARIF.
324
- - ibm.d round-trip: required only if production ibm.d codegen or an ibm.d POC enters this PR; touched generated outputs must be clean after `go generate`.
325
- - Drift triage: follow-up full migration requirement, not required for the initial framework+POC PR.
326
-- **Real-use evidence**:
327
- - Single framework+POC PR lands; CI green; `integrations/taxonomy.json` is produced locally/CI and validates against `taxonomy_output.json`.
328
- - POC collectors validate when `gen_taxonomy.py` runs in CI; their `taxonomy.json` slices validate against `taxonomy_output.json`. FE-side rendering verification is FE-team's responsibility (downstream SOW).
329
- - Changed/touched collector coverage is fatal; global all-collector coverage remains warning/informational.
330
- - ibm.d round-trip evidence captured only if an ibm.d POC/codegen change is included.
331
-- **Reviewer findings**: address all maintainer review comments on the single implementation PR. Apply the per-thread iteration discipline from `.agents/skills/pr-reviews/`.
332
-- **Same-failure search**: grep for any other place in either repo that hand-maintains a taxonomy-shaped data structure (e.g. `dashboards.json`, `menu.json`, `routes.json`); confirm none exist or document each.
333
-
334
-Artifact impact plan:
335
-
336
-- **AGENTS.md**: updated. Collector-consistency rule extended to include `taxonomy.yaml`. Project skills index updated to reference taxonomy work.
337
-- **Runtime project skills**:
338
- - `.agents/skills/integrations-lifecycle/` — major update. New section on taxonomy pipeline; pipeline.md, schema-reference.md, per-type-matrix.md, in-app-contract.md all touched.
339
- - `.agents/skills/project-writing-collectors/` — adds `taxonomy.yaml` as a required artifact for new collectors.
340
-- **Specs**: new `.agents/sow/specs/taxonomy.md` describing schema, validation rules, JSON output contract, frozen v1 top-level list, condition vocabulary.
341
-- **End-user/operator docs**:
342
- - `integrations/README.md` (or equivalent) — documents `taxonomy.yaml` for collector contributors.
343
- - Per-collector READMEs — no individual update needed; the new file is mentioned in the collector consistency rule docs.
344
-- **End-user/operator skills**: `docs/netdata-ai/skills/` — no direct impact (skills don't consume the TOC).
345
-- **SOW lifecycle**: this SOW has moved to `.agents/sow/current/` and will move to `.agents/sow/done/` on completion. Status transitions: open → in-progress → completed. No PR split is planned for this SOW after the 2026-05-11 one-PR correction.
346
-
347
-Open-source reference evidence:
348
-
349
-- This work is internal to Netdata's own repositories. No external open-source projects were consulted as design references. The `cloud-frontend` repository is a private Netdata repository, not an open-source dependency.
350
-
351
-Open decisions:
352
-
353
-ALL RESOLVED 2026-05-11. Recorded for traceability:
354
-
355
-1. **Section identity model**: **8.A locked** — stable `section_id` first-class in `sections.yaml`; collector taxonomy references `section_id:` directly; `section_path:` is not accepted in v1. Section IDs are opaque immutable handles; dots are namespace punctuation only. Section moves are `parent_id` edits in `sections.yaml`; collector YAML unaffected.
356
-2. **`only_views:` v1 inclusion**: **DROPPED from v1** per user — "I don't think we need only_views". If audit 1.1 finds an S2 case (whole-section visibility gating), that case spawns a follow-up SOW to add `only_views:` later. Does not block the initial framework+POC PR.
357
-3. **Drift triage owner assignment**: **A locked** — bulk-migration implementer assigns owners themselves based on finding type; named owners review and sign off. Pre-soliciting per finding rejected as overhead.
358
-
359
-Tactical items resolved without user input (recorded as defaults):
360
-
361
-- `taxonomy.json` artifact: gitignored ephemeral, generated in CI, consumed by FE-team's pipeline exactly as `integrations.js` is consumed today (Decision 3 correction).
362
-- Superseded 2026-05-14: no `include_charts:` or chart-recipe manifest in v1, but explicit full-shape typed `items:` are now in scope for v1 parity (Decision 5 reopened by user).
363
-- Phase B (FE switchover) is out of scope; FE team owns consumption on their schedule.
364
-- ibm.d production codegen is follow-up unless an ibm.d collector is selected as a POC.
365
-- Full collector taxonomy coverage and global-all-collector fatality are follow-up work; per-changed-collector coverage is fatal in the implementation PR (Decision 12).
366
-- Icon-key naming: kebab-case in YAML to match `categories.yaml` style.
367
-- No cloud-frontend freeze imposed (FE team owns their repo); any legacy snapshot pinning belongs to the future full-migration/drift-triage SOW.
368
-
369
-## Implications And Decisions
370
-
371
-User decisions locked on 2026-05-10:
372
-
373
-1. **Sibling vs embed**: sibling `taxonomy.yaml` next to `metadata.yaml`. Reason: ibm.d generator-on-generator avoidance; audience separation; file-size pragmatics. All 3 independent agents converged.
374
-2. **Matcher and reference policy (AMENDED 2026-05-14 for full-shape `items:`)**: drop regex entirely. Context ownership and display references are explicit item semantics.
375
- - **Structural literal owners**: a plain string in a structural `items:` array, or `type: owned_context` with `context:`, owns exactly one context. Every owned literal must resolve to the owning collector metadata (TAX003).
376
- - **Display references**: `type: context` widgets carry `contexts:` arrays. These references do not own contexts; each literal must resolve to metadata or carry the explicit `unresolved: {reason, owner, expires}` escape hatch. A resolved literal reference with no structural owner anywhere is TAX037.
377
- - **Selector items**: `type: selector` owns the contexts matched by `context_prefix:` or `collect_plugin:`. Prefix/plugin selectors still require `metadata.yaml.metrics.dynamic_context_prefixes:` or `metadata.yaml.metrics.dynamic_collect_plugins:` declarations. TAX031/TAX035 fire without opt-in.
378
- - **Selector objects inside widgets**: widget `contexts:` arrays may include `{context_prefix: [...]}` or `{collect_plugin: [...]}` selector objects. These reference contexts but do not own them.
379
- - **`context_prefix_exclude:`** is valid only alongside `context_prefix:` on the same item/reference; invalid pairings are TAX029.
380
- - **Overlap detection**: duplicate ownership between non-selector owners is TAX033. Ownership overlap involving a selector is TAX036. Referencing an already-owned context is expected and valid.
381
- - **Resolved/reference snapshots**: every generated placement and item carries `resolved_contexts` (owned contexts) and `referenced_contexts` (display references). For dynamic-context collectors, selector snapshots include only statically-known contexts; runtime selectors cover future emitted contexts.
382
- - **No-metadata collector handling**: a collector without `metadata.yaml` may declare dynamic opt-ins under top-level `inline_dynamic_declarations:`. The validator treats those declarations as equivalent to `metadata.yaml.metrics.*`; if sibling `metadata.yaml` exists, inline declarations are TAX029.
383
- - **Frontend label-access coordination**: audit 1.10 records whether `_collect_plugin` is already reachable from the FE chart-selection data path. If not, SOW-0016 still publishes `collect_plugin:`; FE consumption either exposes the label downstream or opens a future selector-replacement SOW.
384
- - The guardrail preserves drift-elimination: static collectors (mysql, postgres, ...) use structural literal ownership and explicit widgets, not broad dynamic selectors.
385
- - Cross-engine portability concerns are eliminated by string-prefix-only and label-equality semantics (no JS/Python regex divergence).
386
-3. **Output artifact + lifecycle (CORRECTED 2026-05-11 to match existing `integrations.js` precedent; earlier vendoring framing was over-engineered)**: separate `integrations/taxonomy.json`, **gitignored ephemeral**, generated by `gen_taxonomy.py` in netdata CI, consumed by cloud-frontend exactly as `integrations.js` is consumed today (`cloud-frontend/.github/workflows/sync-to-s3.yaml:47-67`).
387
- - **Generation policy**: `gen_taxonomy.py` writes `integrations/taxonomy.json` during CI; `.github/workflows/generate-integrations.yml` cleanup step removes it together with `integrations.{js,json}` (matches existing pipeline at `:64-66`). Local generation for inspection is supported. The file is `.gitignore`d.
388
- - **Versioning (contract-discipline IN the artifact, not in the delivery mechanism)**: output JSON carries `taxonomy_schema_version: 1` at root level (separate from per-file `taxonomy_version: 1` which is the input schema version). The FE consumer is expected to fail its build on unsupported `taxonomy_schema_version` or unknown required fields. This closes the gap that exists for `integrations.js` today (`.agents/skills/integrations-lifecycle/in-app-contract.md:117-120`) without changing the delivery mechanism.
389
- - **Compatibility policy**: within v1, additive optional fields are non-breaking; field removal is forbidden; enum value additions are reviewed; selector semantics, override merge semantics, list-replacement behavior, section path/ID identity, and generated JSON field names are FROZEN. Changing any of those bumps `taxonomy_schema_version`.
390
- - **Deprecation fields**: every section and selector type may carry `status: active|deprecated`, `deprecation: { replacement_id, since, removal_in }`. Consumers warn on deprecated entries; remove-after window is one major schema version.
391
- - **Source commit metadata**: every emitted `taxonomy.json` includes `source: { netdata_commit: <sha>, generated_at: <iso8601> }` for traceability. Whether the FE pins to a specific commit is the FE team's choice, not this SOW's contract.
392
- - **Cloud-frontend consumption**: out of scope of SOW-0016; tracked as a downstream FE-team SOW. This SOW publishes the artifact and the contract; consumption is the FE team's responsibility.
393
- - Reason for the correction: the existing `integrations.js` model is proven, well-understood, and operationally simple. The version-discipline concerns Codex subagents 04/05 raised are addressed by `taxonomy_schema_version` IN the output JSON, regardless of delivery mechanism. Inventing a separate vendoring strategy traded simplicity for theoretical robustness the artifact metadata already provides.
394
-4. **Pipeline (AMENDED per Change 10)**: new `integrations/gen_taxonomy.py` + a NARROWLY scoped extracted `integrations/_common.py` (allow-list in the active implementation plan). `check_collector_taxonomy.py` is fresh, NOT a clone of the stale `check_collector_metadata.py`. Byte-identical `integrations.{js,json}` output before/after the `_common.py` refactor is a gate-fatal acceptance criterion.
395
-5. **Full-shape TOC item contract in v1 (REOPENED 2026-05-14 by user decision)**:
396
- - `taxonomy.yaml` v1 must represent the full cloud-frontend TOC shape needed for parity: ordered `items:`, structural groups, explicit `owned_context` leaves, flattening groups for legacy `justGroup` semantics, selector leaves, grids, context/table widgets, first-available alternatives, and view-conditioned item bodies.
397
- - `include_charts:` remains absent. v1 does not use opaque chart-handle references or a chart-recipe manifest; it carries explicit typed item bodies where the legacy FE taxonomy carries explicit widget bodies.
398
- - Cloud-frontend consumption remains downstream/out of scope. The FE team owns the renderer adapter and legacy taxonomy removal. The current FE code is evidence for semantics, not a frozen object-shape contract; the Netdata artifact may require clean downstream FE changes.
399
- - Ownership and display references are separate: structural `owned_context` and `selector` items own contexts; `context` widgets, grids, alternatives, and view-switch widget bodies reference contexts and must validate them without tripping duplicate-ownership checks.
400
- - Widget `contexts:` arrays may contain literal context strings, explicit unresolved-reference objects with reason/owner/expiry, or selector objects (`context_prefix`, `context_prefix_exclude`, `collect_plugin`). Selector objects inside widgets reference contexts; selector items under `items:` own contexts.
401
- - TAX003 and TAX037 are fatal by default. Intentional staged/legacy unresolved references require the explicit unresolved-reference escape hatch; warning-by-default drift is rejected. TAX036 remains reserved for existing selector-overlap conflicts.
402
- - TAX038 warns when an unresolved-reference escape hatch has become stale because the context now resolves in metadata.
403
- - Renderer-private payloads are fenced under `renderer:`. Core item objects remain closed to preserve typo detection; known renderer keys are `overlays`, `url_options`, and `toolbox_elements`, and future renderer-only additions use `x_*`.
404
- - String shorthand is allowed only in structural positions (`placement.items`, `group.items`, `flatten.items`). It is rejected inside `grid.items`, `first_available.items`, and `view_switch` branches.
405
- - `flatten` is rejected inside `view_switch` branches; `first_available` alternatives are display-only object items and cannot own contexts.
406
- - The executable schema contract must include a per-type closed-field matrix and recursion matrix equivalent to the amended design artifact.
407
- - Reason: user directive 2026-05-14 — "There is no need to narrow the scope of v1, it should cover everything. Less churn is not our concern."
408
-6. **ibm.d**: production codegen remains the target architecture, but after the 2026-05-11 one-PR scope correction it is not required in the initial framework+POC PR unless an ibm.d collector is selected as a POC. Reason: full ibm.d generation belongs with full collector coverage, not with the minimal framework proof.
409
-7. **`dynamicSections`**: legacy FE root fallback, not a collector `taxonomy.yaml` field. If downstream FE needs dynamic fallback during migration, the clean Netdata-side home is a generated root/section option derived from `sections.yaml`; the FE team still owns the renderer behavior and removal timing. No Netdata-side feature flag.
410
-8. **Frozen v1 top-level sections + stable section identity (LOCKED 8.A on 2026-05-11)**: top-level frozen list `system, kubernetes, containers-vms, synthetic-checks, remote-devices, otel, azure-monitor, applications, netdata` (mirrors `cloud-frontend/.../taxonomy/index.js:25-35` per audit 1.11 spelling map). Adding a new top-level requires a PR on `sections.yaml`.
411
- - **Identity model: 8.A (locked by user 2026-05-11)**. Stable `section_id` is first-class in `sections.yaml`. Collector `taxonomy.yaml` references `section_id:` directly. **`section_id:` is the canonical and ONLY accepted authoring form in v1.** `section_path:` as a list-of-segments is NOT accepted in v1 schema — closed schema rejects it (`additionalProperties: false`). Rationale: two authoring forms create a typo/ambiguity surface; one canonical form keeps `gen_taxonomy_seed.py` output deterministic and reviews easy. Output `taxonomy.json` carries BOTH stable `id` and a generator-resolved dotted `path` for FE convenience. If contributor demand emerges for path-style authoring later, a follow-up SOW must design that alternate input shape explicitly.
412
- - **ID semantics**: `section_id` is an opaque immutable handle. Dots in IDs are allowed for readability and namespace grouping, but they do not define parentage and are not recomputed when a section moves. `parent_id` is the only source of hierarchy. Example: moving `applications.postgres` under a new `applications.databases` parent edits the `parent_id` of the existing `applications.postgres` section; collector `taxonomy.yaml` remains unchanged. Renaming the ID to `applications.databases.postgres` would be a deprecation/replacement, not a move.
413
- - **`sections.yaml` shape**: each entry has `id` (stable, immutable, kebab-case), `parent_id` (root entries omit), `title`, `short_name?`, `icon?`, `section_order` (for top-level ordering), `status` (`active` | `deprecated`), `deprecation?: { replacement_id, since, removal_in }`.
414
- - **Move semantics**: a section moves by changing `parent_id` in `sections.yaml`. Collector `taxonomy.yaml` files referencing the moved section need NO edit because they reference `section_id`, not the path. The resolved `path` in `taxonomy.json` updates automatically.
415
- - **Deprecation semantics**: a section may be marked `status: deprecated` with `deprecation: { replacement_id, since, removal_in }`. Consumers warn; new collectors cannot place charts under deprecated sections (TAX028).
416
- - **Stable IDs anchor**: FE state persistence (saved view layouts), URL query params, and deprecation tracking all key off stable section IDs, not paths.
417
-9. **Delivery shape (CORRECTED 2026-05-11 by user decision)**: one implementation PR for the framework plus POC collector taxonomies. No PR-A1 / PR-A1.5 / PR-A2 split for this SOW. FE Phase B remains downstream and out of SOW-0016. Full collector coverage, production ibm.d sweep, legacy drift triage, and global all-collector fatality are follow-up work.
418
-10. **Categories axis** (`meta.monitored_instance.categories`) is orthogonal to TOC. Preserved untouched. Documented in skills + AGENTS.md so contributors do not conflate the two axes.
419
-11. **View-conditional rendering (AMENDED 2026-05-14; `only_views:` still dropped per user decision)**: sparse `single_node:` deltas and whole-body `view_switch` have separate roles. No whole-node visibility gate in v1.
420
- - **Top-level fields ARE the multi-node rendering** (the canonical/dominant case).
421
- - **`single_node:` is a sparse same-kind override block** — declared only when single-node view differs from multi-node by scalar/list/display/renderer field deltas on the same item type. It may not contain `type`, `items`, `multi_node`, `single_node`, or change an owner into a widget.
422
- - **`view_switch` is for whole-item replacement** — use it when branches have different item kinds, different child trees, or widget bodies where sparse override would be unclear. `view_switch.multi_node` and `view_switch.single_node` are both required and contain concrete items. `single_node:` and `view_switch` cannot appear on the same item.
423
- - **No `only_views:` in v1 (locked by user 2026-05-11)**: whole-node visibility gating is not part of the v1 schema. If audit 1.1 surfaces a real structure-level visibility-gate case (Scenario S2), that case becomes a **follow-up SOW** to add `only_views:` later — it does NOT block the initial framework+POC PR. The implementation PR ships without the field. Schema's permitted `x_*` extension namespace (Decision 13) does not back-door this; adding `only_views:` later requires a `taxonomy_schema_version` minor bump and the follow-up SOW's design review.
424
- - **Allowed override fields in `single_node:` (CLOSED set, v1)**: the set is derived from the same item type's field matrix in `.local/audits/taxonomy-design/full-shape-v1-redesign.md`. `include_charts`, `only_views`, `type`, and `items` are still NOT valid in `single_node:`. The schema closes `single_node:` properties with `additionalProperties: false` plus the permitted `x_*` extension namespace per Decision 13.
425
- - **List replacement examples (canonical, in spec doc)**:
426
- - Scalar override: `single_node: { title: "Average CPU" }` — replaces top-level `title` only for single-node view.
427
- - List replacement on a `type: context` widget: `contexts: [a, b, c]` + `single_node: { contexts: [a, b] }` → single-node renders only `a, b` (the top-level list does not extend).
428
- - Explicit clear: `group_by: [label:node]` at top + `single_node: { group_by: [] }` → single-node has no grouping.
429
- - **Lint rules (orthogonal, normalized after Codex review subagent 02)**:
430
- - **TAX021** — unknown override key under `single_node:` (closed enum violation).
431
- - **TAX022** — `multi_node:` override block declared (multi-node IS top level).
432
- - **TAX023** — list-merge attempt (educational error: lists replace, not merge; if extend is needed, use `*_extend:` field — not in v1).
433
- - **TAX024** — empty `single_node:` block (warning; equivalent to omitting it).
434
- - **TAX025** — `single_node:` override field equals top-level value (redundant override; warning).
435
- - ~~**TAX026**~~ — REMOVED: previously "dead override under `only_views: [multi_node]`"; no longer applicable without `only_views:`.
436
- - ~~**TAX027**~~ — REMOVED: previously "`only_views:` value not in closed enum"; no longer applicable.
437
- - **Why no per-node `condition:` everywhere (Variant A rejected)**: 2 of 3 reviewers ranked it last for maintenance; typos silently render in both views.
438
- - **Why no duplicate placements (Variant B rejected)**: invisible pair-link, copy-paste drift bait.
439
- - **Why no `variants:` block (Variant C rejected)**: critical ambiguity around "missing branch = base or = hidden".
440
- - **Why no pure handle-level conditioning (Variant E rejected)**: shifts source-of-truth into FE; title text (most common per-view difference) leaves the YAML.
441
- - **Why no `views:` wrapper**: YAGNI; `single_node:` handles simple same-kind deltas and `view_switch` handles whole-body replacement without hiding multi-node defaults inside a wrapper. Future view types require an explicit schema/version amendment.
442
- - The amended design keeps the maintainer's sparse-override preference for simple deltas while covering full FE body switches. Whole-section visibility gates are deliberately excluded from v1; any real S2 case from audit 1.1 becomes a follow-up SOW rather than implicit schema surface.
443
- - Reason for pinning at user request: cloud-frontend's `({ isSingleNode }) => ...` pattern produces view-dependent chart specs (per user "Different view depends on the view, we need this in the taxonomy"); user further clarified "multi node is the default and single node is the same, we need to support override syntax".
444
- - Reviewer reports: `.local/audits/taxonomy-design/schema-review-{1,2,3}.md`; external review: `.local/audits/taxonomy-design/external-review-codex/`.
445
-
446
-12. **Deterministic merge / order rules (NEW 2026-05-11 per Change 8)**. With ~150 separate YAML files, file-system traversal order, YAML author order, and Python dict insertion order must NOT be correctness inputs. The locked merge algorithm:
447
- - **Top-level section order**: `sections.yaml` declares `section_order` field per top-level entry; sort ascending. Frozen v1 ordering: `system, kubernetes, containers-vms, synthetic-checks, remote-devices, otel, azure-monitor, applications, netdata` (mirrors current FE).
448
- - **Parent/leaf ordering**: per parent, children sort by `priority` ASC (lower = earlier; default 1000), then by normalized title (`unicodedata.normalize("NFC", title).casefold()`, Python default binary string ordering; no locale collation), then by `placement_id` (lex), then by source path (lex) as final tiebreaker.
449
- - **Explicit item ordering**: `items:` arrays preserve author order at every depth. The deterministic sort applies only when merging independently-authored placement/section siblings under the same parent. The generator must not recursively sort author-provided item trees.
450
- - **Selector and reference ordering**: structural `items:` arrays and widget `contexts:` arrays preserve author order. `context_prefix:` and `collect_plugin:` selector lists are sorted lex on emit so the JSON snapshot is stable across machines.
451
- - **Duplicate ownership policy**: a leaf `(section_id, leaf_id)` has EXACTLY ONE owner (TAX006 fatal). Implicit multi-owner merge is forbidden. If two collectors must contribute to a shared parent (e.g. multiple databases under `applications.databases`), they own distinct leaves under it; the parent metadata comes from `sections.yaml` (or a single explicit `section_overrides:` per audit-resolved policy).
452
- - **Finding emission cadence**: TAX033 and TAX036 emit once per conflicting context per unordered owner pair, sorted by context, owner key, and source path. Multiple selector mechanisms for the same pair are folded into one finding message. TAX037 emits once per referenced-only literal context per nearest item path. TAX038 emits once per stale unresolved reference per item path.
453
- - **JSON serialization**: `gen_taxonomy.py` emits `taxonomy.json` with sorted object keys, fixed indentation, no trailing whitespace. Re-running the generator on identical input produces byte-identical output (golden test required).
454
- - Reason: avoids noisy diffs in POC and later full-migration PRs; eliminates a class of CI flakiness; makes later legacy-vs-generated diff tooling reliable.
455
-
456
-13. **Schema evolution posture (NEW 2026-05-11 per Change 15)**. Permissive `additionalProperties: true` is replaced by **closed core schemas + namespaced extension keys**.
457
- - **Closed core**: every taxonomy authoring object (placement, item, `single_node:` block, `sections.yaml` entry, opt-out object) declares `additionalProperties: false`. Unknown core keys fail TAX021/TAX028 (depending on context). This catches the typo class that already exists in cloud-frontend (`applications.js:53` has `icons:` instead of `icon:`; `systemHardware.js:69-70` has duplicate `title` keys).
458
- - **Renderer envelope**: FE-private renderer payloads may be carried only under a fenced `renderer:` object. Open pass-through fields directly on item bodies are rejected so typo detection remains meaningful.
459
- - **Namespaced extension keys**: `x_*` is the only permitted extension namespace on core nodes. Extensions are preserved into `taxonomy.json` under an `_extra` block, scoped to the originating placement/subsection. Extensions never alter rendering until a schema version claims them.
460
- - **Breaking-change boundary** (must be documented in `taxonomy.md` spec):
461
- - **Non-breaking**: adding optional fields with defined defaults; adding new `sections.yaml` entries; adding deprecation metadata.
462
- - **Non-breaking only if old consumers ignore them safely**: adding a new view type (closed enum extension); adding a new selector type; adding a new override field under `single_node:`. All of these require a `taxonomy_schema_version` minor bump even if the change is forward-compatible at the data level.
463
- - **Breaking (require major schema version bump)**: changing matcher semantics, override merge semantics, list-replacement behavior, section ID/path identity, generated JSON field renames, or removing/renaming existing output fields.
464
- - **Persisted shorthands forbidden**: no `contexts: all_from_metadata`. The seed tool (Decision 9, active implementation plan) generates explicit lists at author time; subsequent metric additions to `metadata.yaml` MUST be reflected in a taxonomy diff or coverage check fails.
465
- - Reason: open schemas preserve typos; the SOW's drift-elimination goal requires loud failure on misspelled fields, not silent acceptance.
466
-
467
-14. **Chart-recipe manifest remains removed; explicit item bodies replace it (UPDATED 2026-05-14)**: previously proposed as `integrations/taxonomy/chart_recipes.yaml` to validate `include_charts:` handle references. v1 still has no `include_charts:` field and no chart-recipe manifest. The reopened full-shape design models first-available alternatives, grids, context/table widgets, and view-conditioned item bodies directly inside ordered `items:` rather than through recipe handles.
468
-
469
-## Plan
470
-
471
-1. **Single framework+POC PR (active scope)** — narrow `_common.py` extract with byte-identical proof; new schemas with closed-core posture; `sections.yaml` with stable IDs; `icons.yaml`; `gen_taxonomy.py` with deterministic merge and structured `Finding` renderers; `gen_taxonomy_seed.py`; fresh `check_collector_taxonomy.py`; a small POC collector set; CI wired with changed-collector fatal gate; documentation, specs, and skills updated.
472
-2. **Inline audit evidence** — audit outputs under `.local/audits/taxonomy-design/audit/` are produced before the implementation step that depends on them. Audit 1.10 is a non-blocking FE coordination note. Audit 1.5 is only required if production ibm.d codegen or an ibm.d POC is included.
473
-3. **Review checkpoints** — after each major implementation step, decide whether a Claude review is useful. If yes, provide a focused prompt with exact files and questions.
474
-4. **Follow-up work (not this PR)** — full collector taxonomy coverage, production ibm.d sweep, legacy drift triage, global-all-collector fatality, and cloud-frontend consumption.
475
-
476
-Total for this SOW is now the single Netdata framework+POC PR. SOW-0016 closes when that PR merges green and the validation/artifact gates are filled.
477
-
478
-**Cloud-frontend Phase B is OUT of scope of SOW-0016** (FE team owns it on their schedule; tracked as a downstream FE-team SOW). The Netdata-side deliverables are: published `taxonomy.json` artifact shape, `taxonomy_output.json` schema, generator/checker/seed framework, changed-collector CI gate, docs/spec/skills, and POC taxonomy files.
479
-
480
-## Execution Log
481
-
482
-### 2026-05-10
483
-
484
-- 3 independent Opus 4.7 analysis agents launched in parallel; architecture reports stored at `.local/audits/taxonomy-design/agent-{1,2,3}-analysis.md`.
485
-- Synthesized recommendation produced; user reviewed and locked decisions 1–10 plus added the view-condition requirement (decision 11).
486
-- TODO file `TODO-collector-taxonomy-unification.md` updated to reference this SOW.
487
-- This SOW created and Pre-Implementation Gate filled.
488
-- User flagged the initial per-node `condition:` schema design (Variant A) as a UX risk; 3 independent reviewers ran in parallel as taxonomy-author personas (Marta/Pavel/Lina) across 5 schema variants and 3 scenarios. Reports at `.local/audits/taxonomy-design/schema-review-{1,2,3}.md`.
489
-- Schema-review synthesis: hybrid `only_views:` + `views:` was initially chosen.
490
-- User refined #1 (YAGNI): drop `only_views:` from v1; ship pure Variant D (`views:` overrides only). Schema's `additionalProperties: true` posture allows non-breaking addition of `only_views:` later if pre-audit step 1.1 finds an S2 case.
491
-- User refined #2 (avoid duplication): pure D's symmetric `views: { single_node, multi_node }` requires writing field defaults somewhere awkward. User feedback: "multi node is the default and single node is the same, we need to support override syntax — take multi and override some stuff". Schema refined to curated-and-override: top-level fields ARE multi-node; `single_node:` block holds the sparse delta. No `views:` wrapper, no `multi_node:` block. Decision 11 updated. Validator codes: TAX021 (unknown view-override key), TAX022 (`multi_node:` block illegal — fields go at top level), TAX023 (list-merge attempt), TAX024 (empty `single_node:` block).
492
-- User refined #3 (dynamic contexts): SNMP, prometheus, cgroup, apps emit contexts whose names share a namespace prefix. Decision 2 updated to allow `context_prefix:` (string prefix only, NOT regex) with an opt-in guardrail: collector must declare `metrics.dynamic_context_prefixes: [...]` in its `metadata.yaml`. Adds TAX031 (prefix not declared), TAX033/TAX036 ownership-overlap checks, and TAX034 (redundant explicit context under prefix). TAX032 remains a reserved follow-up code for a narrower prefix-overlap diagnostic if the project later needs one.
493
-- Empirical validation (7 go.d collectors against locked schema, report at `.local/audits/taxonomy-design/schema-validation-drafts.md`): 6/7 pass cleanly. statsd exposed a real gap — its user-app synthetic charts use user-supplied names (`name = myapp` → `myapp.*` chart names) with no shared prefix. Same shape applies to `charts.d.plugin` (bash scripts pick their own names) and `python.d.plugin` (legacy). User confirmed adding `collect_plugin:` selector to v1 (Decision A on 2026-05-10).
494
-- User refined #4 (label selector): added `collect_plugin: [<plugin-name>]` selector parallel to `context_prefix:`. Selects any chart whose `_collect_plugin` label matches. Same opt-in pattern: collector declares `metrics.dynamic_collect_plugins: [{plugin, reason}, ...]` in `metadata.yaml` (or in `taxonomy.yaml` for plugins lacking `metadata.yaml` like statsd.plugin). Adds TAX035 (collect_plugin not declared), TAX036 (overlap between collectors).
495
-- Validation also surfaced a real-world drift: `mysql.open_transactions` referenced in `cloud-frontend/.../applications.js:1957` does NOT exist in `mysql/metadata.yaml`. This is exactly the failure TAX003 catches at PR time. PR-A2 plan extended: run a diff-tool sweep over the legacy taxonomy to inventory similar drifts before bulk migration.
496
-- Other empirical findings (some superseded by later decisions): non-leaf sections must live in `sections.yaml`; exactly one collector may use `section_overrides:` for any given `section_id`; empty `contexts:` is valid when `context_prefix:` or `collect_plugin:` is present; postgres-style 70-context enumeration requires `gen_taxonomy_seed.py` to amortize author cost. The earlier `section_path:` and `include_charts:` draft shapes are explicitly superseded by Decisions 5 and 8.A.
497
-- Schema impact: `integrations/schemas/collector.json` extended with two optional dynamic declarations. This is the first material change to the existing collector schema in this SOW.
498
-
499
-### 2026-05-11
500
-
501
-- External Codex review run (6 parallel GPT-5.5-xhigh subagents per the prompt at `/tmp/codex-taxonomy-review-prompt.md`); reports stored at `.local/audits/taxonomy-design/external-review-codex/`. Synthesis verdict: GO WITH CHANGES. Architecture approved; SOW not ready for PR-A1 kickoff until 15 contract-level gaps closed.
502
-- SOW amended this date to incorporate all 15 required changes:
503
- - **Change 1 (audit 1.4 false premise)**: rewrote step 1.4 from "confirm none" to a `virtualContexts` classification table with per-row owner. The initial disposition name `frontend-recipe-handle` was later normalized to `keep-frontend-only` after Decision 14 was removed. `systemStorage.js:3-31, 103` cited as the disproof of the previous premise.
504
- - **Change 2 (chart handle contract)**: added Decision 14 — `integrations/taxonomy/chart_recipes.yaml` manifest with stable handle IDs, consumed_contexts, ordered alternatives (for `getMenu.js:77-82` first-available semantics), supported_views, owner, status, deprecation. Validators TAX040–TAX042 added.
505
- - **Change 3 (Netdata negative selector)**: amended Decision 2 with `context_prefix_exclude:` constrained-exclusion field; audit 1.8 chooses static enumeration vs prefix+exclude; regex remains forbidden.
506
- - **Change 4 (selector semantics)**: amended Decision 2 with explicit union behavior, cross-type overlap detection, resolved-snapshot-vs-runtime contract, no-metadata collector inline declarations, FE label-access proof requirement (audit 1.10).
507
- - **Change 5 (taxonomy.json lifecycle)**: amended Decision 3 — gitignored ephemeral in PR-A1; vendored build-pinned in Phase B; `taxonomy_schema_version`, source commit metadata, deprecation fields, FE build-time validation.
508
- - **Change 6 (stable section identity)**: amended Decision 8 — originally proposed path 8.A (stable `section_id`) vs 8.B (immutable path segments); later user sign-off locked 8.A and removed `section_path:` as an accepted v1 authoring form.
509
- - **Change 7 (view-conditional hardening)**: amended Decision 11 — originally reconsidered `only_views:`; later user sign-off dropped it from v1. Closed allowed-override-fields set remains; canonical examples are no override, scalar override, and list replacement; lint codes TAX026/TAX027 are removed.
510
- - **Change 8 (deterministic merge/order)**: added Decision 12 — locked sort key `(section_order, priority, normalized_title, placement_id, src_path)`; deterministic JSON serialization; one-owner-per-leaf rule; 10-run byte-identical golden test.
511
- - **Change 9 (seed tooling)**: promoted `gen_taxonomy_seed.py` to PR-A1 hard requirement; persisted `contexts: all_from_metadata` shorthand explicitly forbidden (Decision 13).
512
- - **Change 10 (fresh checker)**: explicit "do not mirror `check_collector_metadata.py`" guidance; `check_collector_taxonomy.py` is a fresh wrapper around `_common.py` and taxonomy validators.
513
- - **Change 11 (PR-A1.5 blocking)**: PR-A1.5 elevated from "may merge as part of A1" to a SEPARATE BLOCKING PR between PR-A1 and PR-A2; audit 1.5 produces a working `db2` prototype with round-trip evidence; escalation path defined if prototype fails in <1 week.
514
- - **Change 12 (changed-collector fatal gate)**: from PR-A1 onward, taxonomy coverage is fatal for changed `metadata.yaml`/`taxonomy.yaml` files; global-all-collector coverage stays warning until PR-A2 final.
515
- - **Change 13 (drift triage process)**: PR-A2 step 4.6 introduces a drift inventory with closed disposition set `{drop-frontend-entry, fix-metadata, fix-collector, keep-frontend-only, defer-with-sow}` and per-finding owner; bulk migration implementer cannot silently decide product semantics.
516
- - **Change 14 (Phase B realism)**: original wording held Phase B estimate at 2 weeks; reviewer pass updated to 6–8 weeks and required FE rollback/design artifacts. **2026-05-11 scope correction superseded all of this**: Phase B is OUT of SOW-0016; FE-team owns it. Those FE artifacts are downstream FE-SOW responsibilities. SOW-0016 closes when the single framework+POC PR lands with green validation.
517
- - **Change 15 (schema evolution)**: added Decision 13 — closed core schemas (`additionalProperties: false`); only `x_*` namespaced extension keys allowed at core nodes; explicit breaking-change boundary documentation.
518
-- Pre-Implementation Audit Outputs section was added during hardening and later normalized to 10 required audit deliverables + 1 non-blocking coordination note after audit 1.7 and 1.9 were removed; PR-A1 is blocked until all required outputs are non-`to-decide`.
519
-- Drift Triage Process and Rollback Matrix subsections added.
520
-- Open decisions section temporarily rewritten with 5 user-facing residual decisions (Phase B fallback shape; artifact lifecycle path; section identity model 8.A vs 8.B; `only_views:` v1 inclusion default; drift triage owner-assignment policy). Later same-day scope correction and user sign-off resolved all of them.
521
-- Acceptance criteria expanded from 9 items to 27 items reflecting the new gates.
522
-- Plan total was temporarily revised from "~6–8 weeks" to "15–19 weeks calendar" when Phase B was still included; later same-day scope correction narrowed SOW-0016 back to Netdata-only.
523
-- Sub-state updated to "design hardened after external Codex review (GO WITH CHANGES); pre-implementation audit blocking; PR-A1 cannot start until audit produces written evidence."
524
-- **2026-05-11 scope correction (later same day)**: user clarified two points:
525
- - (a) "we don't need to change FE - the FE guys will do it. We just need to prepare everything in Netdata repo." Phase B is moved OUT of scope of SOW-0016; cloud-frontend consumption is tracked as a downstream FE-team SOW on their schedule.
526
- - (b) The previously-recommended "build-pinned vendored" artifact lifecycle was over-engineered relative to the existing `integrations.js` precedent. Decision 3 corrected: gitignored ephemeral generation, consumed by FE-team's pipeline exactly as `integrations.js` is consumed today; contract-discipline lives in `taxonomy_schema_version` IN the artifact, not in the delivery mechanism.
527
-- Consequent SOW edits made in the same session:
528
- - Decision 3 rewritten to match `integrations.js` precedent.
529
- - Historical note, superseded 2026-05-14: Decision 5 was then corrected to keep chart bodies FE-side. The later 2026-05-14 user decision reopens this boundary and requires explicit full-shape typed `items:` in v1.
530
- - Decision 14 REMOVED from v1: chart-recipe manifest unnecessary with `include_charts:` removed. This remains true after the 2026-05-14 reopening because explicit typed item bodies replace recipe handles.
531
- - Phase B Plan section replaced with a one-paragraph "OUT OF SCOPE" reference to the downstream FE-team SOW.
532
- - Audit 1.7 (FE adapter spike) and 1.9 (chart-recipe manifest seed) REMOVED. Audit 1.10 (`_collect_plugin` feasibility) downgraded from blocking to a non-blocking coordination question sent to FE team.
533
- - Acceptance criteria pruned of FE-side gates (vendoring proof, rollback runbook, staging fixture, FE adapter spike, chart-recipe handle validation).
534
- - Rollback Matrix Phase B row removed; matrix simplified to Netdata-only (PR-A1, PR-A1.5, PR-A2).
535
- - Plan total revised from "15–19 weeks" to "7–9 weeks calendar of Netdata-side work" honestly reflecting the narrower scope.
536
- - Open Decisions reduced from 5 user items to 3 (Phase B fallback shape and artifact lifecycle path are now resolved by the scope correction itself).
537
- - Followup mapping updated: Phase B becomes a downstream FE-team SOW. Historical chart-recipe-handle follow-up was superseded on 2026-05-14 by explicit full-shape typed item bodies.
538
-- **2026-05-11 residual-decision sign-off (final design lock)**:
539
- - Decision 1 (section identity model) → **8.A locked**. Stable `section_id` first-class in `sections.yaml`; collector taxonomy references `section_id:`; section moves are `parent_id` edits.
540
- - Decision 2 (`only_views:` v1 inclusion) → **dropped from v1** per user ("I don't think we need only_views"). If audit 1.1 finds an S2 case, that case spawns a follow-up SOW (non-blocking for PR-A1). Decision 11 amended; canonical examples reduced from 4 to 3; lint codes TAX026 and TAX027 removed; `single_node:` allowed-fields set updated accordingly.
541
- - Decision 3 (drift triage owner assignment) → **A locked**. Bulk-migration implementer assigns owners based on finding type; named owners sign off or push back.
542
- - Sub-state updated: "all design decisions locked 2026-05-11; SOW ready to move from `pending/` to `current/` on user go-ahead". Open Decisions section now records all 3 resolutions for traceability.
543
- - No further user decisions block PR-A1 start. Only the audit remains.
544
-- **2026-05-11 final readiness review (3 parallel Opus 4.7 reviewers; all returned READY WITH NOTES)**:
545
- - Reports: `.local/audits/taxonomy-design/final-review/agent-{1,2,3}-readiness.md`.
546
- - 9 patches applied to close all surfaced gaps:
547
- - **Patch 1 (Critical, R3)**: Step 2.11 CI wiring corrected — `generate-integrations.yml` runs on `push: master` (post-merge); `check-markdown.yml` is the PR-time gate. Taxonomy validation must run in BOTH workflows; "fatal in CI" gates explicitly live in `check-markdown.yml`. Path triggers extended on both workflows.
548
- - **Patch 2 (High, R1+R3)**: Decision 8.A clarified — `section_id:` is canonical and only accepted authoring form in v1. `section_path: [list]` is NOT accepted in v1 schema. Eliminates the typo/ambiguity surface.
549
- - **Patch 3 (High, R1+R2)**: Step 2.12 "touched-collector" gate definition tightened — fatal only when diff modifies `metrics.*` keys OR `taxonomy.yaml` OR adds/removes either file. Edits to `overview`, `setup`, `troubleshooting`, `alerts`, `related_resources` do NOT trigger the gate. Prevents PR-A1→PR-A2 window from blocking unrelated metadata edits.
550
- - **Patch 4 (High, R2)**: Audit 1.11.b added — full `sections.yaml` tree draft (~80–150 entries) walking every cloud-frontend taxonomy file. Without this, PR-A1 step 2.4 was a developer guessing the topology from 18 JS files for a day.
551
- - **Patch 5 (High, R1)**: Banner added to `schema-validation-drafts.md` warning it is superseded for authoring (25+ stale `include_charts:` uses; ~20 stale `section_path:` uses). First developer reading it as template no longer gets the wrong shape.
552
- - **Patch 6 (Medium, R2)**: Step 2.2 byte-identical proof phrasing fixed — `git diff --exit-code` does NOT work on gitignored ephemeral files; correct procedure uses `diff -u` against captured pre-refactor copies stored under `.local/audits/taxonomy-design/refactor-baseline/`.
553
- - **Patch 7 (Medium, R1)**: Drift Triage Process — 5-business-day escalation clock added; if a named owner doesn't sign off within 5 business days, implementer escalates to user for tie-break. Prevents PR-A2 final stalling on PTO.
554
- - **Patch 8 (Medium, R2+R3)**: Stale residue swept — Step 2.14 "chart-recipe-alternatives" fixture replaced with a Netdata-negative-selector fixture (Decision 14 was removed; old fixture name was leftover); `keep-frontend-virtual` spelling normalized to `keep-frontend-only`; `getMenu.js` path corrected from `taxonomy/getMenu.js` to `charts/toc/getMenu.js`; stale Phase-B-coupling references in Risks and Execution Log replaced with downstream-FE-SOW language.
555
- - **Patch 9 (Medium, R3)**: Decision 2 shapes locked — `inline_dynamic_declarations:` block shape and `resolved_contexts` snapshot shape both written into the SOW with concrete YAML/JSON examples. No remaining schema TBDs before PR-A1 step 2.3.
556
- - Total audit outputs now 10 (was 9): added `1.11b-sections-tree.md`. Total reviewer-flagged ambiguities resolved: all surfaced concerns either patched in SOW or explicitly assigned to in-flight PR-A1 work.
557
- - SOW is ready to move from `pending/` to `current/`. Audit phase begins on the next user action.
558
-- **2026-05-11 external Codex post-patch readiness recheck (4 parallel subagents; synthesis verdict NOT READY as artifact bundle, no architecture blocker)**:
559
- - Reports stored under `.local/audits/taxonomy-design/external-review-codex-2/`.
560
- - Findings: 5 of 9 patches landed cleanly; 4 partially landed due stale text in acceptance criteria, plan, validation, TODO, and walkthrough.
561
- - Sweep applied: normalized audit count to 10 required outputs + 1 non-blocking coordination note; removed active `section_path` authoring language; removed active `include_charts` / chart-recipe / Phase-B validation residue from Netdata execution gates; replaced stale `git diff --exit-code` proof wording for gitignored artifacts with the `diff -u` baseline-copy procedure; locked PR-A1 clarifications for `taxonomy_optout`, deterministic title normalization, seed output, audit 1.11.b dependency, and opaque stable section-ID semantics.
562
- - Targeted stale-reference grep validated the sweep; SOW is ready to move from `pending/` to `current/` on user go-ahead.
563
-- **2026-05-11 audit phase start**:
564
- - User approved moving SOW-0016 from `pending/` to `current/` and starting the audit output phase.
565
- - Status changed from `open` to `in-progress`.
566
- - Audit output directory initialized at `.local/audits/taxonomy-design/audit/` with index `00-audit-index.md`.
567
-- **2026-05-11 one-PR execution correction**:
568
- - User decided: "we will do everything in one PR (framework - w/o adding taxonomy for all collectors, can add a few as a POC)."
569
- - Branch created and checked out: `sow-0016-taxonomy-framework-poc`.
570
- - Active SOW scope corrected: single framework+POC PR; no PR-A1 / PR-A1.5 / PR-A2 split for this SOW.
571
- - Full collector coverage, production ibm.d sweep, legacy drift triage, and global all-collector fatality moved to follow-up work unless explicitly pulled into this PR.
572
- - User requested review checkpoints: after each major step, provide a Claude prompt if an external review would be useful.
573
-- **2026-05-11 framework+POC implementation pass**:
574
- - `_common.py` extracted from `gen_integrations.py`; byte-identical `integrations.{json,js}` proof completed before later schema/metadata changes.
575
- - Taxonomy schemas, section/icon registries, generator, checker, seed helper, and unittest coverage added.
576
- - CI wired in both PR-time `check-markdown.yml` and post-merge `generate-integrations.yml`.
577
- - POC collector taxonomies added for apache, mysql, postgres, nvidia_smi, and snmp.
578
- - SNMP metadata extended with `metrics.dynamic_context_prefixes` for `snmp.`.
579
- - Contributor docs, project skills, AGENTS.md, and taxonomy spec updated.
580
- - Local `.venv` validation passed; details recorded in the Validation section.
581
-- **2026-05-11 Claude review follow-up**:
582
- - User provided external review verdict READY WITH NOTES.
583
- - Fixed accepted pre-merge items F1/F2/F4/F6/F7/F8/F12: no-metadata `taxonomy_optout` no longer emits a misleading missing-metadata fatal; metadata files whose metrics block was removed are treated as touched; 10-run determinism unittest added; seed/checker docs added to README, pipeline, and add-go-collector recipe; title normalization aligned to NFC.
584
- - Deferred per user/review scope: YAML-aware metrics span parsing, broader TAX negative-test matrix, global gate severity policy, invalid-metadata surfacing, schema-error message-code mapping hardening, artifacts-and-banners taxonomy entry, and opt-out POC example.
585
-- **2026-05-14 full-shape redesign pause**:
586
- - User rejected structural-only POC depth and required v1 to cover full Cloud FE TOC shapes.
587
- - Implementation paused; no further code changes until the amended full-shape contract is reviewed.
588
- - Design artifact updated at `.local/audits/taxonomy-design/full-shape-v1-redesign.md`.
589
- - Claude review prompt updated at `.local/audits/taxonomy-design/full-shape-v1-adversarial-review-prompt.md`.
590
-- **2026-05-14 full-shape implementation resume**:
591
- - External review returned READY TO IMPLEMENT after B1-B7 plus R1/R2 amendments.
592
- - `taxonomy_collector.json`, `taxonomy_output.json`, `gen_taxonomy.py`, `gen_taxonomy_seed.py`, `test_taxonomy.py`, docs/spec/skills, and all five POC `taxonomy.yaml` files were updated to the ordered recursive `items:` contract.
593
- - MySQL POC now models summary grid widgets, table widgets, nested structural groups, owned context leaves, `referenced_contexts`, and legacy FE drift correction for `mysql.galera_open_transactions`.
594
- - Local `.venv` validation passed for py_compile, generator check-only, touched-collector checker, seed helper, determinism diff, performance spot check, and 18 taxonomy unit tests.
595
-
596
-## Validation
597
-
598
-Acceptance criteria evidence:
599
-
600
-- Implemented locally:
601
- - `integrations/_common.py` extracted and `integrations/gen_integrations.py` refactored to reuse it.
602
- - `integrations/schemas/taxonomy_collector.json`, `taxonomy_sections.json`, and `taxonomy_output.json` added with closed-core v1 authoring; on 2026-05-14 `taxonomy_collector.json` and `taxonomy_output.json` were updated from the structural-only POC shape to the full ordered recursive `items:` shape.
603
- - `integrations/taxonomy/sections.yaml` and `icons.yaml` added for the POC section/icon registry.
604
- - `integrations/gen_taxonomy.py`, `integrations/gen_taxonomy_seed.py`, and `integrations/check_collector_taxonomy.py` added. On 2026-05-14, `gen_taxonomy.py` was updated to emit local `resolved_contexts`, `referenced_contexts`, and `unresolved_references` snapshots for every placement/item, enforce TAX037 referenced-but-not-owned, preserve TAX036 for selector ownership overlap, deduplicate/sort TAX033/TAX036 conflict emission, and support `owned_context`, `group`, `flatten`, `selector`, `context`, `grid`, `first_available`, and `view_switch` item kinds.
605
- - POC `taxonomy.yaml` files added for apache, mysql, postgres, nvidia_smi, and snmp. On 2026-05-14, all five were migrated to `items:`; MySQL became the full-shape proof with summary grid, table widgets, nested groups, owned structural context leaves, and legacy drift correction from `mysql.open_transactions` to `mysql.galera_open_transactions`.
606
- - `snmp/metadata.yaml` declares `metrics.dynamic_context_prefixes: [{prefix: snmp., reason: ...}]` for the SNMP dynamic-prefix POC.
607
- - CI wiring added to `check-markdown.yml` and `generate-integrations.yml`.
608
- - `integrations/taxonomy.json` added to `.gitignore`.
609
-
610
-Tests or equivalent validation:
611
-
612
-- Passing locally with repo-local `.venv`:
613
- - `.venv/bin/python -m py_compile integrations/_common.py integrations/gen_integrations.py integrations/gen_taxonomy.py integrations/gen_taxonomy_seed.py integrations/check_collector_taxonomy.py integrations/tests/test_taxonomy.py`
614
- - `.venv/bin/python integrations/gen_integrations.py`
615
- - `.venv/bin/python integrations/gen_taxonomy.py --check-only`
616
- - `.venv/bin/python integrations/check_collector_taxonomy.py`
617
- - `.venv/bin/python -m unittest integrations.tests.test_taxonomy` (40 tests, including old-shape rejection, recursion-matrix rejection, renderer-envelope rejection, positive coverage for item kinds, TAX003 unknown-context fatal, TAX036 selector-overlap preservation, TAX037 referenced-but-not-owned enforcement, TAX038 stale-unresolved warning, unresolved payload output, dynamic-prefix narrowing, metadata-warning surfacing, YAML-aware touched-collector span parsing, deleted-collector gate handling, and 10-run deterministic taxonomy output check)
618
- - `.venv/bin/python integrations/gen_taxonomy_seed.py src/go/plugin/go.d/collector/apache/metadata.yaml --module-name apache --section-id applications.apache --placement-id apache --icon apache` (emits flat `items:`)
619
- - Determinism proof: two consecutive `gen_taxonomy.py --output /private/tmp/netdata-taxonomy-{1,2}.json` runs compared cleanly with `diff -u`.
620
- - 2026-05-14 full-shape determinism proof: two consecutive `gen_taxonomy.py --output /private/tmp/netdata-taxonomy-fullshape-{1,2}.json` runs compared cleanly with `diff -u`.
621
- - Performance spot check after Claude-blocker fixes: `/usr/bin/time -p .venv/bin/python integrations/gen_taxonomy.py --check-only` completed in `real 3.99` seconds.
622
-
623
-Real-use evidence:
624
-
625
-- `integrations/gen_taxonomy.py` emitted a valid local full-shape taxonomy artifact containing the five POC placements. The generated `section_path` values are `applications.apache`, `applications.mysql`, `applications.postgres`, `system.hardware.gpus.nvidia`, and `remote-devices.snmp`. The artifact is gitignored and not intended for commit.
626
-- MySQL POC coverage proof from `/private/tmp/netdata-taxonomy-fullshape-1.json`: MySQL placement has `families: null`, 75 `resolved_contexts`, 16 `referenced_contexts`, 0 `unresolved_references`, a summary `grid` first item with 8 widget references, and no missing or extra owned contexts compared with `mysql/metadata.yaml` (75 metadata contexts, 75 owned).
627
-- MySQL intentionally owns `mysql.handlers` once in the `Handlers` structural group even though the legacy FE listed it in more than one visual grouping; the v1 contract requires single ownership and display widgets can reference owned contexts separately.
628
-- NVIDIA intentionally adds structural Bus / Utilization / Memory / Sensors / MIG grouping around the legacy FE table coverage. This is a Netdata-side taxonomy improvement, not an accidental FE parity miss.
629
-- `integrations/gen_docs_integrations.py -c go.d.plugin/snmp` produced no committed doc drift after adding the SNMP dynamic declaration, confirming the metadata extension does not alter generated user docs.
630
-
631
-Reviewer findings:
632
-
633
-- External readiness reviews are complete before implementation.
634
-- Post-implementation Claude review returned READY WITH NOTES and identified accepted fixes F1/F2/F4/F6/F7/F8/F12; all seven were applied.
635
-- Claude re-review returned READY: all seven accepted fixes are applied, tested, and cross-referenced in SOW/TODO/spec/recipe. Deferred caveats F3/F5/F9/F10/F11/F13/F14 remain non-blocking by design.
636
-- 2026-05-14 full-shape contract review returned READY TO IMPLEMENT after B1-B7 and R1/R2 amendments. The local implementation now targets that full-shape contract.
637
-- 2026-05-14 full-shape implementation review returned NOT READY with three blockers: MB1 unresolved payload dropped, MB2 MySQL `families: true`, MB3 missing TAX003 negative test. All three are fixed. Same-PR improvements also landed for TAX033/TAX036 dedupe/sort, recursion-matrix tests, renderer-envelope tests, item-kind positive tests, and schema-reference matrix depth.
638
-- 2026-05-14 post-blocker Claude re-review returned READY. It confirmed MB1/MB2/MB3 closure and same-PR fixes for TAX033/TAX036 cadence, recursion-matrix tests, item-kind positive tests, renderer-envelope tests, and schema-reference depth. Remaining polish is non-blocking: generic TAX001 stale-shape diagnostics and no artificial `view_switch`/`first_available`/`flatten`/renderer examples in POC YAMLs.
639
-
640
-Same-failure scan:
641
-
642
-- Targeted stale-shape scan completed during implementation via schema/unit coverage:
643
- - `section_path` authoring is rejected by `taxonomy_collector.json`.
644
- - `multi_node:` is rejected by TAX022 prescan.
645
- - `context_prefix_exclude:` without same-node `context_prefix:` raises TAX029.
646
- - top-level/placement `contexts:` authoring is rejected; POC collectors now use `items:`.
647
- - strings inside `grid.items` are rejected by schema; display positions require object items.
648
- - forbidden recursion-matrix cases are covered by unit tests: owning items in grid bodies, nested flatten, string first-available alternatives, string/flatten/nested-view-switch branches.
649
- - renderer pass-through is covered by unit tests: unknown non-`x_*` renderer keys and item-body renderer fields are rejected; `x_*` inside `renderer` is accepted.
650
- - Existing stale references to the old five-file shorthand were updated in integration lifecycle docs/recipes except the historical note that the shorthand is stale.
651
-
652
-Sensitive data gate:
653
-
654
-- Confirmed: this SOW, the linked TODO, the 3 agent reports, and all anticipated artifacts contain no raw secrets, credentials, bearer tokens, SNMP communities, customer data, personal data, customer-identifying IPs, private endpoints, or proprietary incident details. The work is structural metadata only. No redaction required.
655
-
656
-Artifact maintenance gate:
657
-
658
-- AGENTS.md: updated — collector-consistency rule now includes `taxonomy.yaml`; integrations-lifecycle trigger includes taxonomy files/artifacts.
659
-- Runtime project skills: updated — `integrations-lifecycle/` and `project-writing-collectors/` now document taxonomy authoring, generator/checker flow, artifact contract, and consistency impact.
660
-- Specs: updated — new `.agents/sow/specs/taxonomy.md` records source files, authoring contract, selectors, output artifact, CI contract, finding-code matrix, and contributor rule.
661
-- End-user/operator docs: updated — `integrations/README.md` documents `gen_taxonomy.py --check-only`, dynamic selector opt-ins, and the gitignored taxonomy artifact.
662
-- End-user/operator skills: no impact expected (skills do not consume the TOC).
663
-- SOW lifecycle: open → in-progress when moved to `current/` → completed after the single framework+POC PR validates and lands. Final move to done/ together with the work commit per project rule "Do not create a separate commit just to mark or move the SOW".
664
-
665
-Specs update:
666
-
667
-- Complete locally — `.agents/sow/specs/taxonomy.md`.
668
-
669
-Project skills update:
670
-
671
-- Complete locally — `integrations-lifecycle/`, `project-writing-collectors/`.
672
-
673
-End-user/operator docs update:
674
-
675
-- Complete locally — `integrations/README.md`.
676
-
677
-End-user/operator skills update:
678
-
679
-- No impact expected.
680
-
681
-Lessons:
682
-
683
-- Captured in `## Lessons Extracted`.
684
-
685
-Follow-up mapping:
686
-
687
-- Updated. Anticipated follow-ups:
688
- - **Claude review deferred items (not blocking this POC PR)**: broader TAX negative-test matrix; global gate severity policy as taxonomy coverage grows; schema-error message-code mapping hardening; opt-out POC example.
689
- - **Audit evidence (in-scope this SOW)**: 10 supporting outputs under `.local/audits/taxonomy-design/audit/` plus 1 non-blocking FE coordination note; consumed inline before dependent implementation steps.
690
- - **Cloud-frontend Phase B SOW (downstream, FE-team-owned, separate private-repo SOW)**: FE team consumes the published `taxonomy.json` on their schedule. They own: consumer module, taxonomy module removal, chart-spec extraction, `dynamicSections` removal, regex-catchall removal, rollback strategy. Out of scope of SOW-0016.
691
- - **Full collector taxonomy migration**: follow-up SOW/PR plan for the remaining collectors, global all-collector fatality, and legacy drift inventory.
692
- - **Drift triage follow-ups**: any `defer-with-sow` disposition from the future full-migration drift inventory spawns a new SOW with the named owner before that SOW closes.
693
- - **`virtualContexts` follow-ups**: any `defer-with-sow` disposition from audit 1.4 spawns a new SOW.
694
- - **Chart-recipe handle manifest**: no longer planned for v1. The 2026-05-14 full-shape redesign uses explicit typed item bodies instead of recipe handles. A future handle system would require a separate SOW and schema bump.
695
- - **Any new top-level section additions** to `sections.yaml` after v1: each is a small new SOW.
696
- - **`only_views:` schema feature reopens** if audit 1.1 surfaces multi-axis view conditioning (more than `single_node | multi_node`): future SOW with `taxonomy_version` bump.
697
-
698
-## Outcome
699
-
700
-In progress for the current PR. Delivered locally on branch `sow-0016-taxonomy-framework-poc`:
701
-
702
-- collector-adjacent `taxonomy.yaml` authoring contract;
703
-- closed authoring/output schemas and section/icon registries;
704
-- `gen_taxonomy.py`, `gen_taxonomy_seed.py`, and `check_collector_taxonomy.py`;
705
-- PR and master-regeneration workflow wiring;
706
-- integrations README, SOW spec, and project-skill updates;
707
-- five full-shape POC collector taxonomies for Apache, MySQL, Postgres, NVIDIA, and SNMP.
708
-
709
-## Lessons Extracted
710
-
711
-- Stable `section_id` values need generated path segments derived from the final ID component; otherwise opaque IDs with namespace punctuation produce duplicated paths such as `applications.applications.apache`.
712
-- Full-shape POC files must exercise real dashboard structures. Schema-valid flat lists are not enough to prove the contract.
713
-
714
-## Followup
715
-
716
-- Full collector taxonomy migration for the remaining collectors, including global all-collector fatality timing.
717
-- Downstream cloud-frontend consumption SOW: fetch/copy `integrations/taxonomy.json`, implement the adapter, and remove legacy taxonomy modules on the FE team's schedule.
718
-- Production ibm.d taxonomy generation from module source data.
719
-- Broader TAX negative-test matrix and more precise stale-shape diagnostics.
720
-- Invalid-metadata surfacing and schema-error message-code hardening beyond the fixes included in the framework PR.
721
-- Opt-out POC example for a no-metadata/dynamic plugin.
722
-- Future top-level section additions or `only_views:`-style view-axis expansion require separate SOWs and schema-version changes.
723
-
724
-## Regression Log
725
-
726
-None yet.
727
-
728
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/current/SOW-0022-20260509-topology-table-composition.md
deleted
-1007
@@ -1,1007 +0,0 @@
1
-# SOW-0022 - Topology table composition
2
-
3
-## Status
4
-
5
-Status: paused
6
-
7
-Sub-state: paused while the function-specific network-connections modal product
8
-composition work proceeds in SOW-0025. Agent producer implementation is in-tree,
9
-external read-only review found no remaining actionable issues, and narrow
10
-validation passed. Full integrated UI/aggregator validation is still pending
11
-before close.
12
-
13
-## Requirements
14
-
15
-### Purpose
16
-
17
-Make topology actor and link drilldowns useful, curated, compact, and domain-agnostic. The UI must not display raw producer JSON blobs, long nested arrays, unformatted endpoint structures, or internal matching identifiers as final modal content.
18
-
19
-### User Request
20
-
21
-The user observed that current topology actor modals and table cells can show raw JSON directly in the final UI, including large actor attribute objects, nested interface/status arrays, neighbor arrays, and endpoint objects. The user explicitly requested analysis of all actor modals and split this as the second remediation step after presentation:
22
-
23
-- SOW-0021: fix topology presentation.
24
-- SOW-0022: fix table composition.
25
-
26
-### Assistant Understanding
27
-
28
-Facts:
29
-
30
-- The new compact schema separates actor/link/evidence/detail tables, but modal composition is not yet sufficiently specified.
31
-- Some current UI paths render nested JSON structures directly instead of curated, typed fields.
32
-- Actor modal content includes both relationship evidence and actor-owned custom data, and those need different composition and aggregation semantics.
33
-- User-provided examples contain infrastructure-identifying values and must not be copied into durable artifacts.
34
-
35
-Inferences:
36
-
37
-- Modal composition needs an explicit schema/profile layer, not just raw table definitions.
38
-- Detail tables need typed column presentation, formatters, visibility defaults, source/purpose, aggregation policy, and safe rendering rules.
39
-- Relationship evidence should power drilldowns without duplicating every evidence row under every actor.
40
-
41
-Unknowns:
42
-
43
-- The full current cloud-frontend modal renderer shape and all topology-specific assumptions.
44
-- The full set of existing actor/link modal tables for network-connections, streaming, SNMP/L2, and vSphere.
45
-
46
-### Acceptance Criteria
47
-
48
-- Inventory every current actor/link modal and table source for old and new topology payloads.
49
-- Define table composition profiles for actor details, link details, relationship evidence, relationship summaries, inventory, endpoint summaries, custom actor data, and path tables.
50
-- Define safe scalar, enum, reference, array, and nested object rendering rules so raw JSON blobs do not leak into final UI unless explicitly marked as raw/debug.
51
-- Define actor label and table display-name behavior if not fully closed by SOW-0021.
52
-- Update schema/docs/skill/specs and backend producers as needed.
53
-- Create Cloud frontend and Cloud aggregator handoff requirements for modal/table composition.
54
-- Validate with sanitized fixtures covering SNMP/L2, streaming, and network-connections modal examples.
55
-- Ensure `data.correlation.points` and `data.correlation.claims` are not shown
56
- as raw actor modal tables unless a future schema explicitly exposes a curated
57
- debug/diagnostic view.
58
-
59
-## Analysis
60
-
61
-Sources checked:
62
-
63
-- `.agents/sow/done/SOW-0021-20260509-topology-presentation-contract.md`
64
-- `.agents/sow/done/SOW-0023-20260509-topology-cross-payload-matching.md`
65
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json:286-372`
66
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:1211-1260`
67
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md:456-462`
68
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md:541-584`
69
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation_schema.go:7-96`
70
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:635-713`
71
-
72
-Current state:
73
-
74
-- SOW-0021 restored graph-level presentation for actors, links, ports,
75
- legends, labels, and highlight paths.
76
-- SOW-0023 added the correlation plane, semantic correlation link types, and
77
- link layout tokens.
78
-- The old Function UI topology schema had actor-type `summary_fields`,
79
- `tables`, and `modal_tabs`, plus table column labels and cell types.
80
-- The old SNMP topology producer used those fields to describe device summary
81
- fields, Ports and Links tables, column labels, badge/number/actor-link cell
82
- hints, and an Info tab.
83
-- The v1 topology schema currently classifies table types by `role`, `owner`,
84
- `aggregation`, `source_evidence`, and raw `columns`, but does not define how
85
- actor/link modals compose summaries, tabs, table sections, visible columns,
86
- nested values, or relationship-derived rows.
87
-- The v1 developer guide explicitly says full modal/table composition, column
88
- hiding, nested JSON rendering, and richer formatting belong to this SOW.
89
-- SNMP v1 currently preserves actor attributes/labels as an `actor_metadata`
90
- actor-detail table with `json` cells, and converts actor-owned dynamic tables
91
- mechanically. This preserves facts but does not preserve curated modal
92
- behavior.
93
-
94
-Risks:
95
-
96
-- Uncurated modal tables can leak sensitive infrastructure details, overwhelm users, and make topology look unfinished.
97
-- Over-modeling table UI can couple backend producers to frontend component internals.
98
-- Under-modeling table UI forces the frontend to hardcode producer-specific modal logic.
99
-
100
-### Per-Function Modal/Table Inventory
101
-
102
-Design rule:
103
-
104
-- Modal/table definitions should describe how to select, filter, join, format,
105
- and order existing topology facts. They must not duplicate high-cardinality
106
- rows only for UI display.
107
-- If a value is a canonical actor, link, evidence, inventory, or relationship
108
- fact, it belongs in the relevant canonical row/table once. Modal composition
109
- may reference it many times.
110
-- If old modal behavior depended on a value that v1 no longer emits anywhere,
111
- the fix is to restore that canonical value once in the appropriate actor,
112
- link, evidence, or detail table, not to create a duplicate modal-only table.
113
-- Raw `json` columns are facts, not UI. They may be preserved for lossless
114
- debug or future structured expansion, but polished modals should only render
115
- curated scalar/array/reference projections from them when a schema declares
116
- that projection.
117
-
118
-#### topology:network-connections
119
-
120
-Legacy behavior reviewed:
121
-
122
-- Presentation came from `topology_write_presentation()` in the legacy
123
- network viewer function.
124
-- `self` actor summary showed hostname, local IP count, and observed sockets.
125
- Its `Connections` table used `source: links` and displayed the remote actor,
126
- protocol, and direction.
127
-- `process` actor summary showed process display name, command, sockets, local
128
- IP, and user. Its `Sockets` table used actor-owned socket rows and displayed
129
- remote endpoint, protocol, direction, and state. Its `Connections` table used
130
- `source: links` and displayed remote actor, protocol, direction, and state.
131
-- `endpoint` actor summary showed endpoint IP, socket count, and address
132
- space. Its `Connections` table used `source: links` and displayed remote
133
- actor, protocol, and direction.
134
-
135
-Current v1 facts reviewed:
136
-
137
-- Actor rows carry type, machine GUID, hostname, process, PID/PPID/UID when PID
138
- scope is selected, network namespace, local IP/address-space, endpoint
139
- IP/address-space, display name, and socket count.
140
-- Graph links carry actor refs, link type, protocol, direction, state,
141
- evidence count, socket count, retransmissions, and RTT maxima.
142
-- Detailed mode emits `socket` relationship evidence with actor refs, local and
143
- remote tuples, protocol family, direction, state, namespace, process, socket
144
- count, retransmissions, and RTT maxima.
145
-- Aggregated mode emits compact actor-owned `socket_ports` inventory with actor
146
- ref, port, protocol, direction, and socket count. This is enough for process
147
- port bullets without reading detailed socket evidence.
148
-- Correlation points and claims exist for matching, but must not become modal
149
- tables.
150
-
151
-No-duplication reconstruction strategy:
152
-
153
-- `self` and `endpoint` `Connections` tables should be generated by filtering
154
- graph links where `src_actor == actor` or `dst_actor == actor`, then
155
- deriving `remoteLabel` from the opposite actor ref. Protocol, direction, and
156
- state come from the link row.
157
-- `process` `Connections` should use the same graph-link projection and can
158
- optionally filter or group by semantic link types so ownership links do not
159
- appear as network dependencies unless explicitly requested.
160
-- `process` `Sockets` in detailed mode should be generated by filtering
161
- `evidence.socket` rows where `src_actor == actor` or `dst_actor == actor`.
162
- The old `remote` display is a formatted projection of the remote tuple, not
163
- a stored duplicate.
164
-- `process` `Sockets` in aggregated mode should use graph links plus
165
- `socket_count`, and `socket_ports` for port bullets. It cannot show every
166
- socket row because those rows were intentionally not emitted in aggregated
167
- mode.
168
-- Missing canonical fields: the current v1 implementation fills actor struct
169
- fields for `username` and `cmdline`, but the actor column list does not emit
170
- them. Old process summaries cannot be fully reconstructed until those values
171
- are restored as actor columns. Old `self.local_ip_count` is also not emitted
172
- as a v1 actor column; either restore it as a self actor metric or drop that
173
- legacy summary field deliberately.
174
-
175
-#### topology:streaming
176
-
177
-Legacy behavior reviewed:
178
-
179
-- Parent actor summary showed name, node type, agent version, OS, architecture,
180
- CPU count, child count, critical alerts, and warning alerts.
181
-- Child actor summary showed name, node type, agent version, OS, architecture,
182
- CPU count, critical alerts, and warning alerts.
183
-- Virtual node summary showed name, node type, and ephemerality.
184
-- Stale actor summary showed name, node type, agent version, OS, and
185
- architecture.
186
-- Parent `Inbound` table showed node actor link, received-from actor link, node
187
- type badge, ingest badge, hops, collected metrics/instances/contexts,
188
- replication completion, ingest age, SSL badge, and alert counts.
189
-- Parent `Outbound` table showed node actor link, streamed-to actor link, node
190
- type badge, stream status badge, hops, SSL badge, and compression badge.
191
-- `Streaming Path` table showed agent, hops, since, and flags.
192
-- `Retention` table showed actor link, database status, from/to timestamps,
193
- duration, metrics, instances, and contexts.
194
-
195
-Current v1 facts reviewed:
196
-
197
-- Actor rows carry type, machine GUID, node ID, hostname, display name,
198
- severity, ephemerality, ingest status, stream status, ML status, agent name,
199
- agent version, health status, child count, and health alert counts.
200
-- Link and evidence rows carry streaming/virtual/stale relationship refs,
201
- state, port name, timestamps, hops, connection and replication metrics, and
202
- collected metric/instance/context counts.
203
-- Detail tables already exist for `stream_path`, `retention`, `inbound`, and
204
- `outbound`, with actor refs instead of duplicated display strings.
205
-
206
-No-duplication reconstruction strategy:
207
-
208
-- Summary fields should read actor columns directly. `node_type` is the actor
209
- `type` column.
210
-- `Inbound.name` is a projection of `child_actor` rendered through actor label
211
- policy. `Inbound.received_from` is a projection of nullable `source_actor`.
212
- `node_type` is derived by joining `child_actor` to the actors table and
213
- reading its `type`.
214
-- `Outbound.name` is a projection of `actor`. `Outbound.streamed_to` is a
215
- projection of nullable `destination_actor`.
216
-- `Retention.name` is a projection of `actor`; the label can change per modal
217
- profile without changing row data.
218
-- `Streaming Path.Agent` is `path_actor` when present, otherwise `hostname`.
219
- This preserves streaming-path highlighting and avoids storing display strings
220
- twice.
221
-- Missing canonical fields: old summaries included OS, architecture, and CPU
222
- count. Current v1 actor rows do not emit those fields. Reconstructing the old
223
- summary requires restoring them as actor columns if the product still wants
224
- them in the modal.
225
-
226
-#### topology:snmp
227
-
228
-Legacy behavior reviewed:
229
-
230
-- Device summary showed type, vendor, model, description, location, contact,
231
- protocols, capabilities, total ports, VLAN count, FDB MAC count, LLDP/CDP
232
- neighbor counts, chart prefix, Netdata host, source, and layer.
233
-- Device `Ports` table showed port name, operational/admin status, port type,
234
- link mode, topology role, STP state, VLAN count/list, FDB MAC count, link
235
- count, and neighbor count.
236
-- Device `Links` table used `source: links` and showed local port, remote
237
- actor, remote port, protocol, and direction.
238
-- Segment summary showed type, discovery sources, ports total, endpoints total,
239
- source, and layer.
240
-- Endpoint summary showed type, vendor, discovery sources, source, and layer.
241
-
242
-Current v1 facts reviewed:
243
-
244
-- Actor rows carry identity and match-oriented fields: type, layer, source,
245
- display name, chassis IDs, MAC addresses, IP addresses, hostnames, DNS names,
246
- sysObjectID, sysName, and parent devices.
247
-- Link rows carry source/destination actor refs, semantic link type, protocol,
248
- direction, state, evidence count, and timestamps.
249
-- Evidence rows currently include `src_endpoint`, `dst_endpoint`, and `metrics`
250
- as `json` cells.
251
-- Actor detail tables are built mechanically from old actor tables. The old
252
- device `ports` table becomes `actor_ports`/`actor_ports`-like actor detail,
253
- while `actor_metadata` preserves raw attributes and labels as JSON.
254
-
255
-No-duplication reconstruction strategy:
256
-
257
-- Device, segment, and endpoint summaries should read scalar projections from
258
- actor rows and curated actor-detail columns. They should not display the
259
- entire `actor_metadata.attributes` or `actor_metadata.labels` objects.
260
-- Device `Ports` should be generated from the actor-owned port inventory table.
261
- Existing fields such as `name`, `oper_status`, `admin_status`, `port_type`,
262
- `link_mode`, `topology_role`, `stp_state`, `vlan_ids`, `fdb_mac_count`,
263
- `link_count`, and `neighbor_count` should be declared as visible columns
264
- where present. Nested `neighbors` must not render as raw JSON in the main
265
- ports grid; it needs either a compact count in the row or a nested explicit
266
- drilldown profile.
267
-- Device `Links` should be generated by filtering graph links for the selected
268
- actor and joining link/evidence endpoint columns. `remoteLabel` is the
269
- opposite actor ref; `protocol` and `direction` come from the link row.
270
- `localPort` and `remotePort` should be projections from structured endpoint
271
- fields in evidence, not raw endpoint JSON objects.
272
-- Missing canonical fields: many legacy SNMP summary fields live only inside
273
- `actor_metadata.attributes` today. To avoid raw JSON display, SOW-0022 must
274
- either move important scalar fields into typed actor columns/typed detail
275
- columns, or define a safe projection mechanism from JSON paths with strict
276
- scalar output and hidden-by-default raw JSON.
277
-
278
-#### vSphere topology
279
-
280
-- vSphere remains legacy in a separate PR worktree and is tracked as a later
281
- migration SOW. This SOW should define a generic modal/table contract that the
282
- vSphere migration can use, but it should not edit that worktree without user
283
- coordination.
284
-
285
-### Schema Direction For SOW-0022
286
-
287
-The schema needs compact composition definitions, not modal row duplication:
288
-
289
-- Actor and link types define modal profiles in presentation metadata.
290
-- Table types define display metadata for existing table rows: label, order,
291
- default visibility, column labels, column display types, sorting, grouping,
292
- empty-state behavior, and raw/debug visibility.
293
-- Modal sections reference existing sources:
294
- `actors`, `links`, `evidence.<type>`, `tables.actor.<table>`,
295
- `tables.relationship.<table>`, or future typed detail tables.
296
-- A section can declare owner filters such as `actor_ref == selected actor`,
297
- `link_ref == selected link`, or `src_actor/dst_actor contains selected actor`.
298
-- A section can declare safe projections:
299
- direct column, actor label from actor ref, opposite actor label from a link,
300
- formatted endpoint from IP/port columns, scalar JSON-path extraction, array
301
- length/count, badge/number/timestamp/duration formatting, and nullable
302
- fallback.
303
-- The UI must never infer producer domain names such as process, router,
304
- parent, child, client, server, or endpoint. It should execute schema-declared
305
- projections over topology tables.
306
-- The aggregator should preserve composition definitions and merge compatible
307
- table metadata by namespace/dedup rules from SOW-0021/SOW-0023. It should
308
- not materialize modal rows during aggregation unless it is already merging
309
- the underlying canonical table.
310
-
311
-User decisions recorded for this SOW:
312
-
313
-- Host labels must be exposed in full, without topology-specific filtering,
314
- when available. They belong on host/node-level actors and must be shown in
315
- actor modals as actor labels.
316
-- Non-node actors must also expose all known actor labels. For process actors
317
- this includes process metadata such as command line, user, group, namespace,
318
- and similar producer-known facts where available.
319
-- `json` columns should be used only when the UI or aggregator has declared
320
- semantics for them. If the value is only intended for display or filtering,
321
- prefer typed scalar/array columns or a key/value label table.
322
-- An actor modal has four top-level entities:
323
- actor name, actor labels, a depth-1 topology-map miniature, and tables.
324
-- Table composition must support actor-reference cells and row expansion.
325
- Actor references are table cell projections such as `actor_link`; expandable
326
- rows are presentation annotations over hidden/detail columns, not separate
327
- duplicated row data.
328
-
329
-Recommended representation for labels:
330
-
331
-- Use a compact actor-owned label table rather than one raw JSON map per actor:
332
- `actor_labels(actor, key, value, source?, kind?, value_index?)`.
333
-- For host/node actors, populate this table from the complete host label set.
334
-- For non-node actors, populate it from producer-known label and metadata
335
- facts. Keep facts that the aggregator must group or correlate on as
336
- canonical typed actor columns too; the label table is for display/filter/
337
- drilldown, not a replacement for canonical identity/grouping columns.
338
-- Repeated label values should use repeated rows with the same `actor` and
339
- `key`, ordered by `value_index`, rather than JSON arrays.
340
-- Sensitive-data note: topology Functions are sensitive-data surfaces with
341
- admin-controlled access. This permits exposing host labels, command lines,
342
- users, and topology metadata in Function responses. Implementation must still
343
- avoid copying raw label captures into durable repository artifacts or logs.
344
-
345
-### Mapping Validation Matrix
346
-
347
-This section maps old modal content to the new compact schema model. The goal
348
-is to prove whether information exists once in canonical tables and whether a
349
-table recipe can reconstruct the old polished UI without duplicating rows.
350
-
351
-#### Shared Actor Modal Model
352
-
353
-| Modal entity | Source in v1 | Needed schema/table recipe |
354
-|---|---|---|
355
-| Actor name | `actors` row via actor type `presentation.label_policy` | Existing SOW-0021 label policy is enough. |
356
-| Actor labels | New `tables.actor.actor_labels` table | Add table type and a default modal section that filters `actor_labels.actor == selected actor`. |
357
-| Depth-1 topology miniature | Existing `actors` and `links` tables | UI can build from incident links and opposite actors; schema may allow optional link-type filters. No duplicated payload. |
358
-| Tables | Existing `actors`, `links`, `evidence.*`, and `tables.actor.*` | Add modal/table composition recipes with source, owner filter, row filters, projections, cell types, visibility, sorting, and row expansion. |
359
-
360
-Required generic table-recipe primitives:
361
-
362
-- `source`: `actors`, `links`, `evidence.<type>`, `tables.actor.<table>`, or
363
- `tables.relationship.<table>`.
364
-- `owner_filter`: selected actor/link relationship, for example
365
- `actor_ref == selected_actor`, `src_actor == selected_actor`,
366
- `dst_actor == selected_actor`, or either endpoint.
367
-- `row_filters`: link type, evidence type, null/non-null, or value predicates.
368
-- `projection`: direct column, actor label from actor ref, opposite actor label,
369
- conditional local/remote endpoint field, formatted endpoint, label-table
370
- lookup, scalar JSON-path only when explicitly declared.
371
-- `cell`: `text`, `number`, `badge`, `actor_link`, `timestamp`, `duration`,
372
- `endpoint`, `array_count`, or `debug_json`.
373
-- `visibility`: `table`, `expanded`, `hidden`, or `debug`.
374
-
375
-#### topology:network-connections Mapping
376
-
377
-| Old modal/table item | Current/new canonical source | Recipe/status |
378
-|---|---|---|
379
-| Actor name for self/process/endpoint | `actors.display_name` with label policy fallback to hostname/process/IP | Covered by current actor columns. |
380
-| Self labels | New `actor_labels` rows from complete host labels plus topology facts such as hostname/local IP count/socket count | Add `actor_labels`; add `local_ip_count` as either actor metric column or actor label. |
381
-| Process labels | New `actor_labels` rows from process metadata: process name, PID/PPID/UID when available, user, command line, namespace, local IP/address space, socket count | Add `actor_labels`; emit `username` and `cmdline` because the v1 struct fills them but actor columns do not currently expose them. Group name is not currently collected by network-viewer; add it only if a canonical source is introduced. |
382
-| Endpoint labels | New `actor_labels` rows from IP, address space, socket counts, endpoint class | Add `actor_labels`; existing actor columns cover IP/address-space/socket count. |
383
-| Self `Connections` table | `links` incident to selected self actor | Recipe filters incident links, hides or de-emphasizes `ownership` unless a modal asks for graph-coherence links, projects opposite actor as `actor_link`, plus protocol/direction/state/socket metrics. |
384
-| Process `Connections` table | `links` incident to selected process actor | Recipe filters incident network links, projects opposite actor, protocol, direction, state, socket count, RTT/retransmit metrics. Ownership links should be a separate optional/expanded section. |
385
-| Process `Sockets` table, detailed mode | `evidence.socket` rows where selected actor is `src_actor` or `dst_actor` | Recipe projects formatted remote endpoint from local/remote tuple based on selected side, protocol, direction, state, counts, RTT/retransmit metrics; row expansion can expose PID/UID/netns/process/address-space fields. |
386
-| Process port bullets | `tables.actor.socket_ports` | Already canonical; recipe not needed for graph bullets, but modal can show the same table if useful. |
387
-| Process `Sockets` table, aggregated mode | `links` plus `socket_count`; `socket_ports` for port-level rollup | Covered with aggregated summary table. Exact per-socket rows are intentionally unavailable in aggregated mode. |
388
-| Endpoint `Connections` table | `links` incident to selected endpoint actor | Recipe projects opposite actor as `actor_link`, protocol, direction, state, socket count. |
389
-| Depth-1 miniature | Incident `links` and opposite actors | No new data. The mini graph should probably default to network link types and omit ownership unless explicitly enabled. |
390
-
391
-Network-connections validation result:
392
-
393
-- Information is mostly present once.
394
-- Required canonical additions: `actor_labels`, emitted `username`, emitted
395
- `cmdline`, and either emitted `local_ip_count` or a decision to drop that
396
- old self summary.
397
-- Required schema additions: modal/table recipes, conditional endpoint
398
- projection, actor-link cell, row expansion visibility.
399
-
400
-#### topology:streaming Mapping
401
-
402
-| Old modal/table item | Current/new canonical source | Recipe/status |
403
-|---|---|---|
404
-| Actor name | `actors.display_name` / `actors.hostname` via label policy | Covered. |
405
-| Host labels | New `actor_labels` rows from complete `host->rrdlabels` for every RRDHOST-backed actor | Add `actor_labels`; old code nested host labels under actor labels. |
406
-| Streaming actor labels | New `actor_labels` rows from actor columns and host/system metadata: node type, severity, ephemerality, ingest/stream/ML status, agent name/version, health status, child count, alert counts | Add `actor_labels`; do not duplicate graph identity fields outside canonical actor columns. |
407
-| Parent/child summaries: name, type, version, child count, health counts | `actors` columns and/or `actor_labels` | Covered for name/type/version/child/health. |
408
-| Parent/child/stale summaries: OS, architecture, CPU count | Old code emitted system info from `rrdhost_system_info_to_json_object_fields`; current v1 actor columns do not expose it | Add to `actor_labels` from host system info; add typed actor columns only if aggregator/UI needs grouping or sorting by these fields. |
409
-| Virtual node summary: ephemerality | `actors.ephemerality` | Covered. |
410
-| Parent `Inbound` table | `tables.actor.inbound` | Covered by current typed table: parent, child, source actor refs; status, hops, metrics, replication, age, SSL, alert counts. Recipe maps `child_actor` to old `name`, `source_actor` to old `received_from`, and child actor type to old `node_type`. |
411
-| Parent `Outbound` table | `tables.actor.outbound` | Covered by current typed table. Recipe maps `actor` to old `name`, nullable `destination_actor` to old `streamed_to`, actor type to old `node_type`, then status/hops/SSL/compression. |
412
-| `Streaming Path` table | `tables.actor.stream_path` | Covered by current typed table. Recipe maps `path_actor` to actor link when present, fallback `hostname`; shows hops/since/flags; expanded rows may show host/node/claim IDs and capabilities. |
413
-| `Retention` table | `tables.actor.retention` | Covered by current typed table. Recipe maps `actor` or `observer_actor` to actor link depending on selected actor type, then status/from/to/duration/metrics/instances/contexts. |
414
-| Highlight path | Existing SOW-0021 `data.presentation.selection.highlight_path` using `stream_path` | Covered outside modal table composition. |
415
-| Depth-1 miniature | Incident streaming/virtual/stale links and opposite actors | No new data. |
416
-
417
-Streaming validation result:
418
-
419
-- Relationship tables are already modeled correctly and compactly.
420
-- Required canonical additions: `actor_labels`, host labels, and host/system
421
- metadata labels for OS/architecture/CPU parity.
422
-- Required schema additions: recipes that project actor refs as old
423
- actor-link cells, fallback actor labels for path rows, and expanded-row
424
- visibility for host/node/claim/capability fields.
425
-
426
-#### topology:snmp Mapping
427
-
428
-| Old modal/table item | Current/new canonical source | Recipe/status |
429
-|---|---|---|
430
-| Actor name | `actors.display_name` / `actors.sys_name` via label policy | Covered by current actor columns and label policy. |
431
-| Actor labels | New `actor_labels` rows from actor labels and scalar/array metadata currently stored in `actor_metadata.attributes` | Add `actor_labels`; repeated array values should be repeated rows. Do not render raw `actor_metadata` as the user-facing label view. |
432
-| Device summary: type, source, layer | `actors.type`, `actors.source`, `actors.layer` | Covered. |
433
-| Device summary: vendor/model/sys description/location/contact/protocols/capabilities/ports/VLAN/FDB/LLDP/CDP/chart/netdata host | Currently mostly inside raw `actor_metadata.attributes`; some identity fields are in `actors` | Move important scalar/count fields into typed actor columns or typed actor-detail columns; also expose as `actor_labels`. This avoids raw JSON and supports sorting/filtering. |
434
-| Segment summary: type/source/layer | `actors` columns | Covered. |
435
-| Segment summary: learned sources, ports total, endpoints total | Currently attributes/labels, not typed actor columns | Add typed fields or `actor_labels` rows; use typed columns if used for sorting/filtering. |
436
-| Endpoint summary: type/source/layer | `actors` columns | Covered. |
437
-| Endpoint summary: vendor, learned sources | Currently attributes/labels, not typed actor columns | Add typed fields or `actor_labels` rows. |
438
-| Device `Ports` table | Actor-owned port table currently derived from old `ports` rows | Partly covered. Current required `actor_ports` type only declares `name`, `topology_role`, `oper_status`, and `link_mode`; dynamic actor tables can carry more, but schema/presentation is not curated. Need a stable `actor_ports` inventory/detail table with visible columns for name/status/admin/type/mode/role/STP/VLAN/FDB/link/neighbor counts and expanded columns for aliases, speeds, chart refs, and neighbor details. |
439
-| Device `Links` table | `links` plus evidence endpoint fields | Partly covered. Current SNMP evidence stores `src_endpoint`, `dst_endpoint`, and `metrics` as JSON. Need structured endpoint columns such as source/destination port ID/name/if_index/if_name/display name/management IP and structured metric fields needed by link modals. Recipe uses conditional local/remote projection based on whether selected actor is `src_actor` or `dst_actor`. |
440
-| Link protocol/direction/state | `links.protocol`, `links.direction`, `links.state` | Covered. |
441
-| Inferred vs verified link distinction | `links.type` and link presentation from SOW-0021 | Covered for graph; modal/legend recipes should expose link type/status. |
442
-| Raw neighbors/endpoint objects | Existing nested JSON values | Should not render raw in main tables. Map to counts in table rows, and expose detailed nested information only through explicit expanded sections or typed child tables. |
443
-| Depth-1 miniature | Incident L2 links and opposite actors | No new data. Mini graph can use existing link presentation, including inferred/verified link styles. |
444
-
445
-SNMP validation result:
446
-
447
-- Current schema preserves most facts, but too many user-facing facts are only
448
- reachable through JSON.
449
-- Required canonical additions: `actor_labels`; typed actor/detail columns for
450
- important summary fields; stable, richer `actor_ports` table; structured
451
- SNMP evidence endpoint/metric columns replacing user-facing dependence on
452
- `src_endpoint`, `dst_endpoint`, and `metrics` JSON.
453
-- Required schema additions: conditional local/remote endpoint projections,
454
- expanded-row visibility, and explicit debug-only handling for any remaining
455
- raw JSON.
456
-
457
-#### vSphere Mapping
458
-
459
-- vSphere is still legacy and tracked by a later SOW. The SOW-0022 contract
460
- must be generic enough for vSphere inventory actors and relationship tables:
461
- actor labels, mini depth-1 topology, actor/link tables, actor-link cells, and
462
- expandable rows.
463
-- No vSphere producer changes should happen in this worktree without user
464
- coordination.
465
-
466
-### Validation Outcome Before Implementation
467
-
468
-The mapping exercise shows that implementation can proceed after the schema
469
-adds the following generic primitives:
470
-
471
-1. `actor_labels` as a first-class actor-owned table type.
472
-2. Modal/table composition recipes on actor/link types or table types.
473
-3. Cell annotations including `actor_link`, badge, number, timestamp, duration,
474
- endpoint, and debug JSON.
475
-4. Projection primitives for direct columns, actor-ref labels, opposite actor,
476
- conditional local/remote endpoint columns, formatted endpoints, label-table
477
- lookup, and explicitly declared scalar JSON paths.
478
-5. Column/row visibility for table view, expanded row, hidden, and debug.
479
-6. Mini topology composition from existing incident links and actors, with
480
- optional link-type filters.
481
-
482
-The mapping also shows producer-specific canonical field work:
483
-
484
-1. `topology:network-connections`: emit process `username`, process `cmdline`,
485
- `actor_labels`, and self `local_ip_count` if retained.
486
-2. `topology:streaming`: emit `actor_labels`, complete host labels, and
487
- host/system metadata labels used by old summaries.
488
-3. `topology:snmp`: replace user-facing JSON dependence with `actor_labels`,
489
- typed summary fields, richer stable port rows, and structured link endpoint
490
- evidence columns.
491
-
492
-## Pre-Implementation Gate
493
-
494
-Status at implementation start: ready (historical snapshot; current SOW state is recorded in the top-level Status section).
495
-
496
-Problem / root-cause model:
497
-
498
-- Modal/table composition is underspecified. Producers can preserve useful facts, but the UI lacks enough schema-level guidance to turn those facts into polished modal content without raw JSON fallback or producer-specific hardcoding.
499
-
500
-Evidence reviewed:
501
-
502
-- User-provided examples of raw JSON leaking into final UI. Raw examples are intentionally not copied into this durable artifact.
503
-- Old schema support for modal composition is defined in
504
- `src/plugins.d/FUNCTION_UI_SCHEMA.json:286-372`.
505
-- Old SNMP modal composition is defined in
506
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation_schema.go:7-96`.
507
-- V1 table type metadata stops at structural table classification in
508
- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:1211-1260`.
509
-- V1 SNMP currently emits raw actor metadata JSON in
510
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:687-713`.
511
-
512
-Affected contracts and surfaces:
513
-
514
-- `netdata.topology.v1` table/detail schema.
515
-- Cloud frontend actor/link modal renderer.
516
-- Cloud topology aggregator table merge behavior.
517
-- Backend topology producers.
518
-- Developer guide, topology spec, and `project-create-topology` skill.
519
-
520
-Existing patterns to reuse:
521
-
522
-- SOW-0021 presentation profiles.
523
-- SOW-0023 pure correlation actors, points, claims, and semantic correlation
524
- link types.
525
-- Existing compact table roles and column metadata.
526
-- Old topology presentation table and modal-tab metadata as inventory input.
527
-
528
-Risk and blast radius:
529
-
530
-- Applies to all topology producers and the Cloud UI.
531
-- Can affect payload size if table presentation metadata is repeated per row.
532
-- Raw data examples may contain sensitive information and must remain sanitized.
533
-
534
-Sensitive data handling plan:
535
-
536
-- Do not copy raw customer/infrastructure details into durable artifacts.
537
-- Use sanitized fixtures and placeholders only.
538
-- Keep any raw captures under `.local/`.
539
-
540
-Implementation plan:
541
-
542
-1. Record the chosen table/modal composition contract.
543
-2. Inventory current modal/table behavior across producers and UI.
544
-3. Define compact table-composition profiles without high-cardinality
545
- repetition.
546
-4. Update schema/docs/skill/specs and backend producers.
547
-5. Create Cloud frontend and Cloud aggregator handoff artifacts.
548
-6. Validate with sanitized fixtures.
549
-
550
-Validation plan:
551
-
552
-- Pending SOW-0021 and SOW-0023 output.
553
-
554
-Artifact impact plan:
555
-
556
-- AGENTS.md: no expected update unless workflow rules change.
557
-- Specs: likely update `.agents/sow/specs/topology-function-schema.md`.
558
-- End-user/operator docs: likely update `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`.
559
-- Runtime project skills: likely update `.agents/skills/project-create-topology/SKILL.md`.
560
-- End-user/operator skills: likely unaffected unless public operator workflows change.
561
-- SOW lifecycle: current/in-progress; close only after implementation,
562
- integrated validation, and commit.
563
-
564
-Open-source reference evidence:
565
-
566
-- No external OSS reference was used for this contract pass. The work is a
567
- Netdata-specific payload/UI contract derived from old Netdata topology modal
568
- behavior and current `netdata.topology.v1` producer facts.
569
-
570
-Open decisions:
571
-
572
-- Resolved: table/modal composition metadata lives in actor/link type
573
- `presentation.modal`, with reusable table defaults in
574
- `types.table_types.<id>.presentation`.
575
-- Resolved: actor and link modal composition belong to the same contract so
576
- table recipes and cell/projection tokens stay consistent.
577
-- Resolved: raw `json` cells are hidden/debug-only unless an explicit schema
578
- projection extracts a curated scalar value or a future structured child table
579
- is defined.
580
-
581
-## Implications And Decisions
582
-
583
-1. User decision: table and actor-modal composition is separate from topology presentation and should be handled as SOW-0022.
584
-2. User decision: document the target contract first, then implement. Before
585
- Agent implementation starts, create separate Cloud aggregator and Cloud UI
586
- handoff artifacts.
587
-3. Contract decision: modal sections are recipes over existing facts; they do
588
- not duplicate high-cardinality rows or raw actor metadata for display.
589
-4. Contract decision: actor labels use a compact actor-owned
590
- `actor_labels(actor, key, value, source?, kind?, value_index?)` table.
591
-5. Contract decision: the Cloud UI must reuse existing topology modal/table
592
- components where practical and must not reimplement the old table stack from
593
- scratch for v1.
594
-
595
-## Plan
596
-
597
-1. Update topology schema, developer guide, durable spec, and topology
598
- producer skill with the modal/table composition contract.
599
-2. Create a pending Cloud topology service SOW for aggregation behavior.
600
-3. Create a Cloud frontend TODO for UI behavior and component reuse.
601
-4. Implement Agent producer changes for network-connections, streaming, and
602
- SNMP/L2 after the contract/handoffs are accepted.
603
-5. Wait for Cloud UI and Cloud aggregator implementation slices, then run
604
- integrated QA before closing this SOW.
605
-
606
-## Execution Log
607
-
608
-### 2026-05-09
609
-
610
-- Opened as pending follow-up from user direction.
611
-
612
-### 2026-05-10
613
-
614
-- Updated dependencies after SOW-0023 added the correlation plane and link
615
- layout contract. Actor modals must not expose correlation point/claim tables
616
- as raw user-facing content.
617
-- Completed the per-function mapping exercise for network-connections,
618
- streaming, SNMP/L2, and future vSphere migration.
619
-- Began contract-first update of schema/spec/developer-guide/skill before
620
- Agent producer implementation.
621
-- Moved SOW-0022 to current/in-progress after the user approved Agent backend
622
- implementation. Cloud aggregator SOW-0009 remains pending in the service
623
- repo until the service worker finishes SOW-0008.
624
-- Implemented shared Go `netdata.topology.v1` modal/table composition structs
625
- and semantic validation for modal labels, mini topology link filters,
626
- section sources, owner filters, projections, cell visibility, table type
627
- presentation, and sort columns.
628
-- Implemented `topology:network-connections` producer additions:
629
- `actor_labels`, process `username`, process `cmdline`, self
630
- `local_ip_count`, modal recipes over graph links, socket evidence, and
631
- `socket_ports`.
632
-- Implemented `topology:streaming` producer additions: complete host-label
633
- export into `actor_labels`, system metadata labels/actor columns for OS,
634
- architecture, and CPU count, graph-link `port_name`, and modal recipes over
635
- existing `stream_path`, `retention`, `inbound`, and `outbound` tables.
636
-- Implemented `topology:snmp` producer additions: `actor_labels`, typed summary
637
- fields, stable `actor_ports`, structured evidence endpoint columns, and
638
- actor modal recipes that avoid raw JSON as the default user-facing view.
639
-- Added shared schema tests for v1 modal recipes and SNMP producer assertions
640
- for `actor_labels` and modal presence.
641
-- Ran read-only reviews with GLM, Kimi, MiMo, Qwen, and MiniMax against this
642
- SOW and the uncommitted implementation.
643
-- Addressed reviewer findings:
644
- - required `value` for `const` modal projections in schema and semantic
645
- validation;
646
- - required a local and remote side for `selected_side_endpoint` projections;
647
- - preserved explicit SNMP zero-valued counts/indexes instead of treating
648
- them as missing;
649
- - removed the misleading SNMP `actor_ports` table-type overwrite pattern;
650
- - added SNMP evidence-column and zero-preservation assertions.
651
-- Ran a second read-only review round after those fixes, then addressed the
652
- concrete findings:
653
- - normalized empty nullable SNMP `protocols` and `capabilities` arrays to
654
- `null`;
655
- - replaced fragile SNMP empty-array type assertions with a helper;
656
- - tightened `selected_side_endpoint` semantic validation so empty-string side
657
- columns do not satisfy the local/remote requirement;
658
- - made evidence-section validation return an error instead of panicking on a
659
- malformed section;
660
- - deduplicated the SNMP port modal column recipe shared by the device modal
661
- and `actor_ports` table presentation;
662
- - added tests for invalid evidence-section shape, empty selected-side column
663
- validation, and nullable SNMP array normalization.
664
-- Ran a third read-only review round with the same scope. Concrete fixes from
665
- that round:
666
- - aligned JSON schema and semantic validation for `label_lookup`,
667
- `json_path`, `coalesce`, and row-filter `value`/`values` requirements;
668
- - added schema and semantic negative tests for invalid modal projections and
669
- row filters;
670
- - documented that `actor_labels` logical string fields may be encoded as
671
- `string` or `string_ref`, and that aggregators/UI adapters must normalize
672
- both encodings;
673
- - removed a stale modal-column validator parameter;
674
- - documented and tested the SNMP `protocols` fallback from legacy
675
- `learned_sources`;
676
- - extended the SNMP test table decoder to support `dict` encodings.
677
-- Ran a fourth read-only review round with the same scope. Concrete fixes from
678
- that round:
679
- - required `formatted_endpoint` projections to name at least an IP or port
680
- column in JSON schema and Go semantic validation;
681
- - made semantic validation reject explicit non-array `modal.sections`;
682
- - made the modal projection switch fail closed if a future unsupported kind
683
- reaches the semantic validator;
684
- - documented that `actor_labels` inherits topology Function sensitive-data
685
- access-control assumptions;
686
- - added regression tests for empty `formatted_endpoint` and malformed
687
- `modal.sections`.
688
-
689
-## Validation
690
-
691
-Acceptance criteria evidence:
692
-
693
-- Contract artifacts now define `actor_labels`, modal sections, source kinds,
694
- owner filters, projections, cell types, visibility, table presentation, and
695
- raw JSON/debug rules in:
696
- - `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
697
- - `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
698
- - `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`
699
- - `.agents/sow/specs/topology-function-schema.md`
700
- - `.agents/skills/project-create-topology/SKILL.md`
701
-- Cloud frontend handoff created at
702
- `../../dashboard/cloud-frontend/TODO-topology-modal-composition-contract.md`.
703
-- Cloud aggregator handoff SOW created at
704
- `../../netdata/cloud-topology-service/.agents/sow/done/SOW-0009-20260510-modal-composition-and-actor-labels.md`.
705
-
706
-Tests or equivalent validation:
707
-
708
-- `python -m json.tool src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
709
-- `git diff --check`
710
-- `cd src/go && go test ./pkg/topology/v1 ./plugin/go.d/collector/snmp_topology`
711
-- C syntax/type checks were run manually with the exact include/define flags
712
- from `build/compile_commands.json`, replacing object output with
713
- `-fsyntax-only`, for:
714
- - `src/collectors/network-viewer.plugin/network-viewer.c`
715
- - `src/web/api/functions/function-topology-streaming.c`
716
-- `git -C ../../dashboard/cloud-frontend diff --check TODO-topology-modal-composition-contract.md`
717
-- `git -C ../../netdata/cloud-topology-service diff --check .agents/sow/done/SOW-0009-20260510-modal-composition-and-actor-labels.md`
718
-
719
-Real-use evidence:
720
-
721
-- Not run yet for the completed producer implementation. The local `build/`
722
- directory is root-owned, so targeted Ninja object rebuilds could not write
723
- `.ninja_lock` or `.ninja_log`; syntax/type checks were run instead with the
724
- exact compile flags. Full real-use evidence requires rebuilding/installing
725
- the Agent and validating with the updated Cloud UI/aggregator path.
726
-
727
-Reviewer findings:
728
-
729
-- GLM, Kimi, MiMo, Qwen, and MiniMax completed read-only reviews of
730
- `.agents/sow/current/SOW-0022-20260509-topology-table-composition.md` and
731
- the uncommitted implementation.
732
-- Findings accepted and fixed:
733
- - `const` projections were schema-valid without a `value`;
734
- - `selected_side_endpoint` projections were schema-valid with no usable
735
- endpoint side columns;
736
- - SNMP nullable integer handling collapsed explicit zero values into missing
737
- values;
738
- - SNMP `actor_ports` table-type construction had a confusing overwrite;
739
- - SNMP structured evidence columns needed direct test coverage.
740
-- Findings reviewed but not changed:
741
- - empty `actor_labels` tables are currently allowed for a stable table
742
- contract and tiny overhead;
743
- - raw/debug JSON remains available only through explicit debug visibility;
744
- - Cloud UI projection-engine coverage remains tracked in the Cloud frontend
745
- TODO, not in the Agent producer implementation;
746
- - modal recipes are intentionally type-level payload metadata; they are not
747
- repeated per high-cardinality row;
748
- - modal sources may reference declared table types even when a runtime table
749
- has zero rows or is omitted, allowing the UI to render stable empty
750
- sections.
751
-- Round-2 findings accepted and fixed:
752
- - empty nullable SNMP `protocols`/`capabilities` arrays normalized to `null`;
753
- - fragile empty-array type assertions replaced with `isEmptyArrayCell`;
754
- - `selected_side_endpoint` semantic validation now rejects empty-string side
755
- columns;
756
- - malformed evidence section objects now return validation errors;
757
- - duplicated SNMP port modal column recipe collapsed into one helper.
758
-- Round-3 findings accepted and fixed:
759
- - JSON schema now conditionally requires `label_key`, `path`, non-empty
760
- `columns`, and row-filter `value`/`values` in the same cases covered by
761
- semantic validation;
762
- - Go semantic validation now rejects row filters that omit required values;
763
- - `actor_labels` encoding compatibility is documented for direct strings and
764
- dictionary references;
765
- - SNMP `protocols` legacy fallback is explicit and covered by producer tests;
766
- - SNMP test helpers now decode `dict` compact-table columns.
767
-- Round-3 findings reviewed but not changed:
768
- - streaming may expose both curated system labels and complete host labels;
769
- complete host labels are an explicit product requirement and UI grouping can
770
- decide how to present duplicates;
771
- - streaming `actor_labels` table type does not need a table-presentation
772
- fallback because actor modal labels are driven by `presentation.modal.labels`;
773
- - C producer modal helper duplication is acceptable for now because the two C
774
- producers do not share a common topology-emitter module;
775
- - SNMP `sys_contact` and `sys_location` remain visible because topology
776
- Functions are already marked sensitive and access-controlled by the admin.
777
-- Round-4 findings accepted and fixed:
778
- - empty `formatted_endpoint` projections now fail schema and semantic
779
- validation;
780
- - explicit non-array `modal.sections` values now fail semantic validation;
781
- - semantic modal projection validation fails closed for unsupported kinds;
782
- - `actor_labels` sensitive-data inheritance is documented in the developer
783
- guide, durable spec, and topology producer skill.
784
-- Round-4 findings reviewed but not changed:
785
- - C producer test coverage remains an integrated-QA risk because current local
786
- C Function testing is manual/syntax-level only;
787
- - C modal helper duplication remains accepted for this SOW;
788
- - plain `string` actor-label columns in C producers are allowed by the
789
- documented contract and avoid adding new C dictionary infrastructure here;
790
- - payload-size growth remains a pre-close measurement/integrated-QA item.
791
-- Round-5 findings accepted and fixed:
792
- - `selected_side_endpoint` projections are now self-contained: schema,
793
- semantic validation, producer docs, durable spec, and topology producer
794
- skill require source/destination actor-ref columns plus both endpoint
795
- sides;
796
- - network-connections socket modal projections now emit those side actor
797
- columns;
798
- - SNMP device link modal columns now use selected-side projections for
799
- `Local Port` and `Remote Port` instead of source/destination labels;
800
- - semantic validation now rejects explicitly configured optional actor-label
801
- columns when the referenced column does not exist, while still allowing
802
- optional label columns to be omitted from a table type;
803
- - `actor_labels` column roles are aligned across producers by marking label
804
- key/value/source/kind/index as attributes and actor as the reference.
805
-- Round-5 findings reviewed but not changed:
806
- - C producer modal helper duplication remains accepted for this SOW;
807
- - C producer modal JSON has syntax/type validation but not dedicated C unit
808
- tests yet;
809
- - payload-size growth remains a pre-close measurement/integrated-QA item
810
- because realistic local/Cloud payloads are needed to measure the final
811
- producer output.
812
-- Round-6 findings accepted and fixed:
813
- - `ValidateDecodedResponse` now fails closed for malformed response
814
- envelopes, missing `data`, and wrong schema versions;
815
- - compact-table semantic validation now rejects empty column ids, duplicate
816
- column ids, missing column types, unsupported column types, and primitive
817
- values that do not match the declared column type;
818
- - modal section and modal column semantic validation now rejects missing or
819
- empty `id` and `label`;
820
- - JSON schema now mirrors modal source, owner-filter, and projection
821
- conditional requirements for `table`, `evidence`, actor/link side columns,
822
- and direct/actor/opposite projection columns;
823
- - SNMP actor-owned custom tables named `labels` or `metadata` can no longer
824
- overwrite the built-in `actor_labels` or `actor_metadata` tables;
825
- - streaming graph link rows now emit the metric columns declared by their
826
- link type aggregation policy;
827
- - streaming `info` requests now return metadata without building or emitting
828
- the full topology payload.
829
-- Round-6 findings reviewed but not changed:
830
- - C helper duplication remains accepted until/unless a shared C topology
831
- emitter module is introduced;
832
- - payload-size growth remains a pre-close measurement item;
833
- - dedicated C producer unit/snapshot tests remain an integrated-QA risk
834
- unless a small Function fixture harness is added before close.
835
-- Round-7 findings accepted and fixed:
836
- - JSON schema now mirrors semantic validation for `json_path` by requiring
837
- both `column` and `path`;
838
- - `modal_section.label`, `label_key`, and `json_path.path` now reject empty
839
- strings at schema level where applicable;
840
- - semantic validation now rejects modal sections with missing or empty
841
- `columns`, actor type presentation labels that are explicitly empty, and
842
- non-integer modal mini-topology `depth`;
843
- - semantic validation now reports empty `row_filters[].values` separately
844
- from wrong-type or missing `values`;
845
- - SNMP custom actor-detail table ids now reserve built-in replacement ids and
846
- generate unique ids, so `labels`, `metadata`, `custom_labels`, or similar
847
- table names cannot overwrite one another;
848
- - SNMP `actor_ports` now preserves unknown custom port fields in an `extra`
849
- debug JSON column instead of dropping them when normalizing the stable port
850
- table;
851
- - SNMP `vlan_ids` and similar array labels now stringify scalar typed arrays,
852
- including integer arrays, instead of accepting only `[]string`/`[]any`;
853
- - SNMP neighbor-count inference now runs only when the `neighbors` value is a
854
- valid array of objects, avoiding a misleading zero for malformed values;
855
- - docs and the topology producer skill now clarify that Function `info`
856
- responses are metadata-only and are not validated as full topology payloads;
857
- - docs now describe `empty_label`, `badge_map`, `align`, `sortable`, optional
858
- `label_lookup.actor_column`, and required `json_path.column`/`path`.
859
-- Round-7 findings reviewed but not changed:
860
- - `label_lookup.actor_column` remains optional by design. When omitted, the UI
861
- should look up labels for the selected modal actor; producers provide
862
- `actor_column` only for source-row actor references.
863
-- Round-8 findings accepted and fixed:
864
- - JSON schema now gives `modal_section.label`, `link_type_presentation.label`,
865
- `port_type_presentation.label`, and `table_type_presentation.label`
866
- `minLength: 1`, matching semantic validation for explicit empty labels;
867
- - semantic validation now rejects explicit empty labels in link type, port
868
- type, and table type presentation, in addition to actor type and modal
869
- section/column labels;
870
- - validation tests now cover explicit empty presentation labels for actor,
871
- link, port, and table type presentation;
872
- - the current SOW file was marked with git intent-to-add so `git diff` based
873
- reviewers see the pending-to-current SOW move before the final commit.
874
-- Round-8 findings reviewed but not changed:
875
- - `presentation.modal.sections: []` remains valid. A modal may provide labels
876
- and/or a mini-topology without table sections, and producers with no curated
877
- tables should not be forced to invent empty sections;
878
- - SNMP `anyStringSlice` keeps the small reflection fallback to preserve scalar
879
- typed arrays from non-JSON producers without adding a long type-switch;
880
- - C producer unit/snapshot tests and shared C modal emitter refactoring remain
881
- integrated-QA or follow-up risks already tracked in this SOW;
882
- - table type presentation validation intentionally validates type-registry
883
- table definitions. Runtime actor tables are validated separately and
884
- `presentation.modal.labels` already falls back to actor table columns when
885
- resolving label tables.
886
-- Round-9 findings accepted and fixed:
887
- - semantic validation now rejects duplicate modal section ids and duplicate
888
- modal column ids, matching the duplicate-column guard already used for
889
- compact tables;
890
- - the streaming inbound modal recipe now projects the nullable
891
- `source_actor` as `Received from`, matching the old inbound table behavior
892
- and the SOW mapping;
893
- - streaming inbound and outbound modal recipes now expose the already-existing
894
- node type, collected instance/context, ingest age, TLS, alert-count, and
895
- outbound node columns needed to preserve old visible table functionality
896
- without duplicating row data;
897
- - streaming link type aggregation now declares `replication_completion: avg`;
898
- - network-connections ownership links now declare `socket_count: sum`, and
899
- network-connections actor labels include the canonical actor `type`, aligned
900
- with streaming actor-label behavior;
901
- - the Go validator comment now states that `ValidateDecodedResponse` is for
902
- full topology responses, not metadata-only Function `info` responses;
903
- - validation tests now cover the non-object `presentation.modal` shape.
904
-- Round-10 findings reviewed but not changed:
905
- - C modal helper duplication remains accepted and tracked for a future shared
906
- emitter/refactor because the current C producers do not share a topology
907
- JSON helper module;
908
- - C producer unit/snapshot tests, payload-size measurement, and integrated
909
- Agent/UI/aggregator QA remain pre-close or follow-up items tracked in this
910
- SOW;
911
- - SNMP `anyStringSlice` reflection fallback remains intentional for typed
912
- scalar arrays from non-JSON producer data;
913
- - `formatted_endpoint` remains permissive by design: IP-only or port-only
914
- endpoints are valid when a producer has only partial endpoint facts.
915
-- Round-11 final external review:
916
- - GLM, Kimi, MiMo, MiniMax, and Qwen were rerun on the same full SOW-0022
917
- scope after the Round-10 fixes, with only short fix notes appended;
918
- - no reviewer reported a new actionable or blocking issue;
919
- - remaining reviewer notes were informational only and matched already
920
- tracked risks: C modal helper duplication, missing C producer
921
- unit/snapshot tests, payload-size measurement, integrated Agent/UI/
922
- aggregator QA, the intentional SNMP `anyStringSlice` reflection fallback,
923
- and intentionally permissive partial endpoint formatting;
924
- - the final Qwen review also verified that modal/table composition remains
925
- generic, avoids modal-only high-cardinality duplication, and keeps
926
- sensitive actor-label handling documented.
927
-
928
-Same-failure scan:
929
-
930
-- Searched updated contract artifacts for stale relationship-table naming,
931
- old public create-topology skill references, and unresolved
932
- pre-implementation decision markers.
933
-
934
-Sensitive data gate:
935
-
936
-- Raw user-provided examples are not copied into this SOW. This SOW uses sanitized summaries only.
937
-
938
-Artifact maintenance gate:
939
-
940
-- AGENTS.md: no workflow rule change in this step.
941
-- Runtime project skills: updated `.agents/skills/project-create-topology/SKILL.md`.
942
-- Specs: updated `.agents/sow/specs/topology-function-schema.md`.
943
-- End-user/operator docs: no operator workflow change. Updated developer-facing
944
- Function docs under `src/plugins.d/`.
945
-- End-user/operator skills: unaffected; this is developer topology work, not an
946
- operator skill change.
947
-- SOW lifecycle: remains in `current/` with `Status: paused`; do not close
948
- until integrated QA and commit are complete.
949
-
950
-Specs update:
951
-
952
-- Updated `.agents/sow/specs/topology-function-schema.md`.
953
-
954
-Project skills update:
955
-
956
-- Updated `.agents/skills/project-create-topology/SKILL.md`.
957
-
958
-End-user/operator docs update:
959
-
960
-- Not affected. Developer docs updated:
961
- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md` and
962
- `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`.
963
-
964
-End-user/operator skills update:
965
-
966
-- Not affected.
967
-
968
-Lessons:
969
-
970
-- The old modal behavior can be reconstructed without duplicating modal-only
971
- rows only if producers restore missing canonical fields first.
972
-
973
-Follow-up mapping:
974
-
975
-- Cloud frontend implementation is tracked by
976
- `../../dashboard/cloud-frontend/TODO-topology-modal-composition-contract.md`.
977
-- Cloud aggregator implementation is tracked by
978
- `../../netdata/cloud-topology-service/.agents/sow/done/SOW-0009-20260510-modal-composition-and-actor-labels.md`.
979
-- Function-specific modal product composition is intentionally split out and
980
- tracked separately:
981
- - `.agents/sow/done/SOW-0025-20260511-network-connections-modal-product-composition.md`;
982
- - `.agents/sow/done/SOW-0026-20260511-snmp-modal-product-composition.md`;
983
- - `.agents/sow/current/SOW-0027-20260511-streaming-modal-product-composition.md`.
984
-- Integrated Agent/UI/aggregator QA remains in this SOW after the other workers
985
- finish their implementation slices.
986
-
987
-## Outcome
988
-
989
-Contract documentation, Cloud handoff artifacts, shared Go schema validation,
990
-and Agent producer implementation are prepared. Full integrated QA is still
991
-pending before this SOW can close.
992
-
993
-## Lessons Extracted
994
-
995
-- See the Validation section's Lessons entry for the current extracted lesson.
996
-
997
-## Followup
998
-
999
-- See the Validation section's Follow-up mapping entry for the active tracker
1000
- list. This SOW is paused until integrated Agent/UI/aggregator QA can close
1001
- those mapped items.
1002
-
1003
-## Regression Log
1004
-
1005
-None yet.
1006
-
1007
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/current/SOW-0027-20260511-streaming-modal-product-composition.md
deleted
-659
@@ -1,659 +0,0 @@
1
-# SOW-0027 - Streaming Modal Product Composition
2
-
3
-## Status
4
-
5
-Status: in-progress
6
-
7
-Sub-state: reopened for regression on 2026-05-11. Live modal review showed
8
-that the completed work still left streaming relationship sections with
9
-incorrect product semantics, especially retention direction, missing timestamps,
10
-missing received-from actors, and outbound stream ownership. Source repair is
11
-implemented and build-validated; live UI validation is pending install/restart
12
-of the rebuilt Agent. Reopened again on 2026-05-17 after live graph review
13
-showed streaming parents without child bullets and without child-count-based
14
-size emphasis.
15
-
16
-## Requirements
17
-
18
-### Purpose
19
-
20
-Make `topology:streaming` actor modals useful for operators of Netdata parent/child/vnode streaming trees, including path, retention, inbound, outbound, stale, and transit relationships.
21
-
22
-### User Request
23
-
24
-The user reported that streaming modals show wrong or incomplete table information. Example: a parent with many children/vnodes shows a `Retention` tab with one row and no clear node maintaining retention. The user expects the modal model to reflect stale nodes, children, parents, grandparents, great-grandparents, and parent transit responsibilities correctly.
25
-
26
-### Assistant Understanding
27
-
28
-Facts:
29
-
30
-- Current streaming actor modals use one generic modal recipe for all streaming actor types.
31
-- Current modal sections are `Stream path`, `Retention`, `Inbound children`, and `Outbound stream`.
32
-- Current retention rows have both `actor` and `observer_actor`.
33
-- Current retention modal filters by `actor`, so selecting a parent shows retention for that parent, not retention maintained by that parent for other nodes.
34
-- Current outbound modal filters by `actor`, so it shows the selected actor's own outbound stream row, not necessarily children/vnodes passing through that parent.
35
-
36
-Inferences:
37
-
38
-- The current table data has some right primitives but the modal recipes are not role-aware enough.
39
-- A parent actor needs at least two retention views:
40
- - retention for this actor;
41
- - retention this actor maintains for other actors.
42
-- A parent actor likely needs a transit/children view that includes children/vnodes that pass through it, not only the parent's own outbound stream.
43
-- Some rows may be missing for full cloud aggregation semantics, especially if querying several parents where many parents maintain retention for the same child.
44
-
45
-Unknowns:
46
-
47
-- Whether the current streaming Function sees enough local state to emit retention rows for every node whose data is retained by a parent, or only for actors known in the current topology.
48
-- Whether cloud aggregation will merge retention rows from multiple parents without losing `observer_actor`.
49
-- Whether parent transit relationships are fully represented by existing `inbound` rows, existing graph links/evidence, or require a new table/column.
50
-
51
-### Acceptance Criteria
52
-
53
-- A complete inventory exists for streaming modal facts: actors, actor labels, links/evidence, `stream_path`, `retention`, `inbound`, and `outbound`.
54
-- The SOW defines what each streaming actor role should show: self/local parent, parent, child, virtual node, stale node, and inferred path actors.
55
-- Retention tables distinguish "retention for this node" from "retention maintained by this node for others".
56
-- Inbound/outbound/transit tables show children and descendants passing through a parent where the data exists.
57
-- Actor identification/header labels expose important node identity/status fields, not only the generic `Labels` tab.
58
-- Missing data needed for correct modal semantics is identified as producer work, aggregator work, or frontend work.
59
-
60
-## Analysis
61
-
62
-Sources checked:
63
-
64
-- `src/web/api/functions/function-topology-streaming.c:320` stream path row type.
65
-- `src/web/api/functions/function-topology-streaming.c:337` retention row type.
66
-- `src/web/api/functions/function-topology-streaming.c:349` inbound row type.
67
-- `src/web/api/functions/function-topology-streaming.c:367` outbound row type.
68
-- `src/web/api/functions/function-topology-streaming.c:1130` retention row construction.
69
-- `src/web/api/functions/function-topology-streaming.c:1167` inbound row construction.
70
-- `src/web/api/functions/function-topology-streaming.c:1203` outbound row construction.
71
-- `src/web/api/functions/function-topology-streaming.c:1296` stream path columns.
72
-- `src/web/api/functions/function-topology-streaming.c:1313` retention columns.
73
-- `src/web/api/functions/function-topology-streaming.c:1325` inbound columns.
74
-- `src/web/api/functions/function-topology-streaming.c:1342` outbound columns.
75
-- `src/web/api/functions/function-topology-streaming.c:1443` current modal recipe.
76
-- `.agents/sow/specs/topology-function-schema.md:390` streaming modal composition notes.
77
-
78
-Current state:
79
-
80
-- The current modal recipe is generic across streaming actor types.
81
-- `Retention` section filters `retention` rows by `actor`, which answers "what retention exists for the selected node", not "what retention this selected parent maintains for other nodes".
82
-- The `Retention` section does not display `observer_actor`, even though the table has that column.
83
-- `Outbound stream` filters by `actor`, which answers "where this actor sends", not "which children/vnodes are sent through this parent".
84
-- `Inbound children` filters by `parent_actor`, which is closer to the parent view but may still not cover all transit/descendant questions.
85
-
86
-Available facts to inventory:
87
-
88
-- Actor labels:
89
- - display name, hostname, machine GUID, node ID, type, severity, ephemerality, ingest status, stream status, ML status, agent/version, health status, OS/architecture/CPU, child/alert counts, host labels.
90
-- Stream path rows:
91
- - selected actor, path actor, path index, hostname, host ID, node ID, claim ID, hops, since/first-time, capabilities/flags.
92
-- Retention rows:
93
- - actor whose data is retained, observer actor that maintains the data, DB status, time range, duration, metrics, instances, contexts.
94
-- Inbound rows:
95
- - parent actor, child actor, optional source actor, received type, ingest status, hops, collected metrics/instances/contexts, replication completion, ingest age, TLS, alert counts.
96
-- Outbound rows:
97
- - actor, destination actor, stream status, hops, TLS, compression.
98
-- Links/evidence:
99
- - directed streaming relationships with port name and collected/replication metrics.
100
-
101
-Target audience and questions:
102
-
103
-- Netdata operator looking at a child/vnode:
104
- - What is this node?
105
- - What is its path to cloud/parents?
106
- - Which parent receives it?
107
- - Who retains its data and over what time range?
108
- - Is it stale/virtual/healthy?
109
-- Netdata operator looking at a parent:
110
- - Which children/vnodes does this parent receive directly?
111
- - Which descendants pass through this parent?
112
- - Which nodes' data does this parent retain?
113
- - Where does this parent send data upstream?
114
- - What is the replication status and health of each stream?
115
-- Netdata operator looking at stale or virtual actors:
116
- - Why is this actor present?
117
- - When was it first/last observed?
118
- - Which path or retention state still references it?
119
-
120
-Risks:
121
-
122
-- A retention table that hides `observer_actor` is actively misleading in aggregated/cloud views where multiple parents may retain the same node.
123
-- A parent modal that only shows its own outbound stream misses its operational responsibility for children passing through it.
124
-- A generic recipe for all roles may be too simple; role-specific sections may be required.
125
-
126
-## Pre-Implementation Gate
127
-
128
-Status at implementation start: ready for implementation (historical snapshot;
129
-current SOW state is recorded in the top-level Status section).
130
-
131
-Problem / root-cause model:
132
-
133
-- The current streaming modal recipes expose local source tables but do not encode the operational roles of a selected actor.
134
-- The retention table has the key `observer_actor` fact but the current modal recipe neither filters by it nor displays it, so parent responsibility is hidden.
135
-- The inbound table already models descendants received through a parent; the current modal label `Inbound children` under-describes transit/descendant responsibility and makes the table look incomplete.
136
-
137
-Evidence reviewed:
138
-
139
-- Retention columns include `actor` and `observer_actor` in `src/web/api/functions/function-topology-streaming.c:1313-1323`.
140
-- The modal retention section currently filters by `actor` in `src/web/api/functions/function-topology-streaming.c:1484-1495`.
141
-- Inbound rows include `parent_actor`, `child_actor`, and `source_actor` in `src/web/api/functions/function-topology-streaming.c:1325-1340`.
142
-- Outbound rows include only `actor` and `destination_actor` plus stream attributes in `src/web/api/functions/function-topology-streaming.c:1342-1349`.
143
-- Descendant rows are populated into `parent_descendants` in `src/web/api/functions/function-topology-streaming.c:2546-2604` and then emitted as inbound rows in `src/web/api/functions/function-topology-streaming.c:1145-1192`.
144
-- Actor labels already include display name, hostname, machine GUID, node ID, type, stream/ingest/health status, agent fields, system fields, child count, alert counts, and full host labels where available in `src/web/api/functions/function-topology-streaming.c:549-589`.
145
-
146
-Affected contracts and surfaces:
147
-
148
-- Agent Function payload for `topology:streaming`.
149
-- Streaming C topology Function row construction and modal recipes.
150
-- Cloud aggregator retention/actor-table merge behavior.
151
-- Cloud frontend actor modal rendering and identity/header display.
152
-- Developer guide, topology spec, project topology skill.
153
-
154
-Existing patterns to reuse:
155
-
156
-- Actor-owned `stream_path`, `retention`, `inbound`, and `outbound` tables.
157
-- `actor_ref_label` and `label_lookup` projections.
158
-- Separate table sections filtered by different actor-ref columns.
159
-- Actor labels for identity/status facts.
160
-
161
-Risk and blast radius:
162
-
163
-- User-facing streaming modal behavior changes.
164
-- Aggregation semantics are important because a cloud topology can contain many parents reporting retention for the same node.
165
-- Sensitive data risk includes host labels, node IDs, claim IDs, machine GUIDs, and private hostnames; durable artifacts must use synthetic examples only.
166
-
167
-Sensitive data handling plan:
168
-
169
-- Do not copy raw host labels, machine GUIDs, claim IDs, hostnames, customer names, private endpoints, or production topology payloads into durable artifacts.
170
-- Store real payload captures only under `.local/`.
171
-- Use synthetic parent/child/vnode examples in docs/tests.
172
-
173
-Implementation plan:
174
-
175
-1. Keep one streaming modal recipe for all streaming actor types because the existing tables already use actor-ref owner filters and empty sections naturally disappear or show meaningful empty states per role.
176
-2. Make the recipe role-aware through section labels and owner filters, not by duplicating table data.
177
-3. Add or adjust modal sections for:
178
- - retention for selected node (`actor`);
179
- - retention maintained by selected node (`observer_actor`);
180
- - received/transit descendants (`parent_actor`);
181
- - upstream stream from selected node (`actor`).
182
-4. Display `observer_actor` in the selected-node retention view and `actor` in the maintained-retention view.
183
-5. Add important actor identification/header fields backed by `actor_labels`: health status and child count, while keeping full labels in the Labels tab.
184
-6. Add missing columns/rows only if current canonical tables cannot answer the required operational questions.
185
-7. Validate with local payload/schema checks and mark cloud/multi-parent validation as external if no aggregated streaming fixture is available in this repository.
186
-
187
-Validation plan:
188
-
189
-- C syntax check for `function-topology-streaming.c`.
190
-- Schema validation of generated streaming payloads.
191
-- Local Function call on a parent with children/vnodes.
192
-- Verify modal rows for:
193
- - selected child/vnode;
194
- - selected parent;
195
- - selected stale node if present.
196
-- Aggregated/cloud payload check that retention rows preserve both retained actor and observer actor.
197
-
198
-Artifact impact plan:
199
-
200
-- AGENTS.md: likely unaffected.
201
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md` if streaming modal guidance changes.
202
-- Specs: update `.agents/sow/specs/topology-function-schema.md`.
203
-- End-user/operator docs: likely unaffected unless Function examples are changed.
204
-- End-user/operator skills: unaffected.
205
-- SOW lifecycle: close only after local and, if available, cloud/aggregated streaming validation.
206
-
207
-Open-source reference evidence:
208
-
209
-- External open-source topology references were not used for implementation authority. This SOW changes Netdata-specific streaming semantics defined by local producer code and topology specs; local Netdata code is the authoritative source.
210
-
211
-Open decisions:
212
-
213
-- Resolved for this SOW: use one recipe with multiple well-labeled sections and owner filters. This avoids repeating modal metadata across actor types while preserving role-specific behavior through table filters.
214
-- Resolved for the regression repair: default streaming modals use concise
215
- operator-facing section names: `Stream path`, `Retained nodes`,
216
- `Received nodes`, and `Outbound streams`.
217
-
218
-## Implications And Decisions
219
-
220
-Decision recorded after the user asked to proceed to SOW-0027:
221
-
222
-- Keep the payload single-source-of-truth: reuse existing `actor_labels`, `stream_path`, `retention`, `inbound`, and `outbound` tables.
223
-- Do not create modal-only duplicate rows.
224
-- Keep one shared streaming actor modal recipe unless implementation proves role-specific recipes are necessary.
225
-- Rename/recompose sections so the modal answers operator questions directly:
226
- - `Retained nodes`: which nodes' data the selected actor maintains.
227
- - `Received nodes`: which children/vnodes/stale descendants are received through the selected parent.
228
- - `Outbound streams`: which node payloads the selected parent sends upstream and where they go.
229
-
230
-## Plan
231
-
232
-1. Inventory old/current streaming modal fields and role-specific facts.
233
-2. Design role-aware streaming actor modals.
234
-3. Identify missing canonical streaming rows/columns.
235
-4. Implement only streaming producer changes after design acceptance.
236
-5. Coordinate frontend/aggregator changes if required.
237
-6. Validate locally and with aggregated/cloud payloads when available.
238
-
239
-## Execution Log
240
-
241
-### 2026-05-11
242
-
243
-- Created SOW from user-reported streaming modal regressions and current code evidence.
244
-- Promoted SOW to current and recorded the implementation decision to keep one shared streaming actor modal recipe with role-aware sections over existing tables.
245
-- Updated `topology:streaming` modal identification fields to include health status and child count.
246
-- Split retention presentation using the existing `retention` table and different owner filters.
247
-- Renamed/recomposed relationship sections as `Received nodes` and `Outbound streams` so they match the underlying `inbound` and `outbound` table semantics.
248
-- Updated topology specs and the project topology skill with streaming modal rules.
249
-
250
-## Validation
251
-
252
-Acceptance criteria evidence:
253
-
254
-- Complete fact inventory recorded in `## Analysis` and `## Pre-Implementation Gate`.
255
-- Actor role expectations recorded under `Target audience and questions`.
256
-- `Retained nodes` now filters the same `retention` table by `observer_actor`, without duplicating retention rows.
257
-- `Received nodes` now explains the existing `inbound` rows as children, virtual nodes, stale nodes, and descendants received through a parent.
258
-- Actor modal identification now includes hostname, node type, stream status, ingest status, health status, retained-node count for parents, direct-child count for parents, OS/platform labels, and Agent version.
259
-- No missing producer rows were found for this SOW's modal fix. Aggregator preservation of multi-parent retention rows is already specified in `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
260
-
261
-Tests or equivalent validation:
262
-
263
-- `git diff --check` passed.
264
-- `(cd src/go && go test -count=1 ./pkg/topology/v1 ./tools/functions-validation/validate)` passed.
265
-- `sudo -n cmake --build build --target netdata -- -j2` passed. The build emitted unrelated protobuf/stringop warnings during link, not streaming topology compile errors.
266
-- `.agents/sow/audit.sh` passed with the pre-existing non-project skill classification warning.
267
-
268
-Real-use evidence:
269
-
270
-- Not performed against the live Agent in this SOW because the built binary was not installed/restarted. A live UI check before install would validate a different binary. The code path was validated by compiling the `netdata` target and by schema/fixture tests for the topology v1 contract.
271
-
272
-Reviewer findings:
273
-
274
-- No external reviewer run was requested for SOW-0027. The change is scoped to producer modal metadata and documentation/spec alignment.
275
-
276
-Same-failure scan:
277
-
278
-- `rg -n "Inbound children|Outbound stream|\"Retention\"|No inbound children|No outbound stream" src/web/api/functions .agents/sow/specs .agents/skills/project-create-topology/SKILL.md` found no remaining stale streaming modal labels.
279
-- `rg -n "retention|observer_actor|retained_nodes|Received nodes|Outbound streams|child_count" src/web/api/functions .agents/sow/specs .agents/skills/project-create-topology/SKILL.md` confirmed the new contract is present in the producer, specs, and project topology skill.
280
-
281
-Sensitive data gate:
282
-
283
-- This SOW uses only path/line evidence and synthetic descriptions. No raw sensitive payload data is included.
284
-
285
-Artifact maintenance gate:
286
-
287
-- AGENTS.md: unchanged. This SOW did not change project-wide workflow or guardrails.
288
-- Runtime project skills: updated `.agents/skills/project-create-topology/SKILL.md` with streaming modal composition rules.
289
-- Specs: updated `.agents/sow/specs/topology-function-schema.md` and `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
290
-- End-user/operator docs: unchanged. This is internal topology payload/modal composition, not a public user command or operator workflow.
291
-- End-user/operator skills: unchanged. No public/operator skill behavior changed.
292
-- SOW lifecycle: reopened from done to current and left paused until live UI validation is performed on an installed/restarted Agent.
293
-
294
-Specs update:
295
-
296
-- Updated `.agents/sow/specs/topology-function-schema.md` with required streaming modal sections and identification labels.
297
-- Updated `.agents/sow/specs/topology-modes-correlation-aggregation.md` with the streaming UI table semantics.
298
-
299
-Project skills update:
300
-
301
-- Updated `.agents/skills/project-create-topology/SKILL.md` with streaming modal rules for future topology producers.
302
-
303
-End-user/operator docs update:
304
-
305
-- Not needed. The Function remains sensitive/internal topology data; no public operator command, UI label documentation, or configuration guide changed.
306
-
307
-End-user/operator skills update:
308
-
309
-- Not needed. Public skills under `docs/netdata-ai/skills/` are for querying/operator workflows, not developer topology modal contracts.
310
-
311
-Lessons:
312
-
313
-- Streaming already had the canonical facts needed for the modal fix. The root problem was recipe composition and labels, not missing table data.
314
-
315
-Follow-up mapping:
316
-
317
-- No new follow-up SOW is required from this change. Live UI validation after installing the rebuilt Agent remains an execution check, not a new product requirement.
318
-
319
-## Outcome
320
-
321
-Implementation complete for the current producer changes; live UI validation
322
-after Agent install is still pending.
323
-
324
-- `topology:streaming` actor modals now expose operator-relevant identity in the modal header.
325
-- Retention is shown in both directions:
326
- - who maintains the selected node;
327
- - which nodes the selected actor maintains.
328
-- Parent responsibility is clearer through `Received nodes`, backed by existing descendant rows.
329
-- `Outbound streams` now lists node payloads sent by the selected parent, including the node and destination per row.
330
-
331
-## Lessons Extracted
332
-
333
-- Reusing the same canonical table through different owner filters is the right pattern for modal composition when the relationship has two actor-ref sides.
334
-- Section names must describe the selected actor's perspective; otherwise correct rows can still appear wrong to operators.
335
-
336
-## Followup
337
-
338
-None.
339
-
340
-## Regression Log
341
-
342
-Note: dated regression entries intentionally use `## Regression - YYYY-MM-DD`
343
-headings to match the repository SOW lifecycle contract.
344
-
345
-## Regression - 2026-05-11 - Streaming Modal Relationship Semantics
346
-
347
-What broke:
348
-
349
-- `Stream path` correctly scopes to the selected actor only, but timestamp
350
- columns may render empty because synthetic path rows do not always carry
351
- `since` and `first_time` values.
352
-- `Retention for node` is misleading in the direct parent modal. The useful
353
- operational view is the parent-owned list of nodes retained by the selected
354
- actor.
355
-- `Retained nodes` rows may render empty `from` and `to` values even though the
356
- retention table is expected to carry database time ranges.
357
-- `Received nodes` may render empty `Received from` values for locally received
358
- rows because the producer leaves `source_actor` empty when the source is
359
- considered local.
360
-- `Upstream stream` currently describes the selected actor's own upstream
361
- stream. For a parent actor, operators need every node payload this parent
362
- sends upstream, with the node and destination shown per row.
363
-
364
-Evidence:
365
-
366
-- `stream_path` rows contain `since_ut` and `first_time_ut` fields in
367
- `src/web/api/functions/function-topology-streaming.c`, but synthetic local
368
- append rows only set actor/path identity and do not set those timestamps.
369
-- `retention` rows contain both retained `actor` and retaining
370
- `observer_actor`, so the same table can answer parent-owned retained-node
371
- views without duplicate rows.
372
-- `inbound` rows contain nullable `source_actor`; local-source rows currently
373
- leave it empty.
374
-- `outbound` rows currently contain `actor` and nullable `destination_actor`
375
- only, so they cannot express "selected parent sends node X to destination Y"
376
- for all descendants.
377
-
378
-Why previous validation missed it:
379
-
380
-- Validation checked schema shape, compilation, and table presence, but did not
381
- inspect a live clustered-parent setup where one parent owns virtual nodes,
382
- retains children, receives descendants, and streams them to another parent.
383
-
384
-Repair plan:
385
-
386
-1. Update the durable topology specs and project topology skill first so future
387
- workers do not repeat the same interpretation mistake.
388
-2. Change the streaming producer modal contract so default visible sections are
389
- `Stream path`, `Retained nodes`, `Received nodes`, and `Outbound streams`.
390
-3. Keep the canonical `retention` table lossless, but remove the confusing
391
- default `Retention for node` modal section unless a future explicitly named
392
- `Retained by` view is designed for aggregated/cloud views.
393
-4. Add or repurpose outbound table columns so the table is owned by the sending
394
- parent and has at least `sender_actor`, `node_actor`, `destination_actor`,
395
- status, age, hops, TLS, compression, and useful counts where available.
396
-5. Populate `source_actor` for received rows whenever the immediate sending
397
- actor is known. For direct local receipt, use the received child/vnode actor
398
- rather than rendering an empty source.
399
-6. Ensure retention and stream-path timestamps are populated from the best
400
- available canonical source and remain nullable only when the Agent genuinely
401
- does not know the value.
402
-
403
-Validation required:
404
-
405
-- Local Function response for a clustered parent with self, virtual nodes,
406
- children, and an upstream clustered parent.
407
-- Verify the selected parent modal shows all retained nodes, all received
408
- nodes, and all outbound node transmissions.
409
-- Verify stale/archived hosts from the Agent root index are included in
410
- retention and received-node rows when present.
411
-- Schema validation and focused build/test commands for the streaming Function.
412
-
413
-Implementation evidence:
414
-
415
-- `src/web/api/functions/function-topology-streaming.c` now backfills stream
416
- path `since` and `first_time` timestamps from the best available host status
417
- source when path rows are missing those values.
418
-- `retention` rows still remain single-source canonical rows, but `db_from`
419
- and `db_to` now fall back to known DB/status timing when the raw retention
420
- range is incomplete.
421
-- Local-source `inbound` rows now set `source_actor` to the known child/vnode
422
- actor, so `Received from` does not render empty for direct local receipt.
423
-- The default modal no longer exposes the misleading `Retention for node`
424
- section. It exposes `Retained nodes`, `Received nodes`, and `Outbound streams`
425
- from canonical tables.
426
-- `outbound` rows now use `sender_actor`, `node_actor`, and
427
- `destination_actor`, so a parent modal can list every node payload that the
428
- selected parent currently sends upstream.
429
-
430
-Validation completed:
431
-
432
-- `git diff --check` passed.
433
-- `cmd=$(jq -r '.[] | select(.file|endswith("src/web/api/functions/function-topology-streaming.c")) | .command' build/compile_commands.json | sed 's# -o [^ ]*# -o /tmp/function-topology-streaming.c.o#'); eval "$cmd"` passed.
434
-- `sudo -n cmake --build build --target netdata -- -j2` passed. The build
435
- emitted unrelated generated protobuf/stringop warnings during link; the
436
- modified streaming topology translation unit compiled.
437
-- `(cd src/go && go test -count=1 ./pkg/topology/v1 ./tools/functions-validation/validate)`
438
- passed.
439
-- `(cd src/go && go test -count=1 ./tools/functions-validation/validate)`
440
- passed after adding the top-level Function envelope `v` schema acceptance
441
- test.
442
-- `rg -n "Inbound children|Outbound stream|Upstream stream|Retention for node|No inbound children|No outbound stream|No upstream stream|No retained nodes" src/web/api/functions .agents/sow/specs .agents/skills/project-create-topology/SKILL.md`
443
- found no stale producer modal labels; remaining `Retention for node`
444
- mentions are explicit spec/skill notes saying that section is not part of the
445
- current default modal.
446
-
447
-Validation still pending:
448
-
449
-- Live Function/UI validation against the running Agent after this rebuilt
450
- binary is installed/restarted. Validating before install would exercise the
451
- old binary.
452
-
453
-## Regression - 2026-05-17 - Streaming Parent Graph Bullets And Size
454
-
455
-What broke:
456
-
457
-- Parent actors in `topology:streaming` do not show child bullets.
458
-- Parent actors appear effectively the same size as ordinary nodes even when
459
- they retain data for many children, virtual nodes, stale nodes, or transit
460
- descendants.
461
-
462
-Evidence:
463
-
464
-- The parent actor type is configured with `show_port_bullets: true` and
465
- data-driven sizing in `src/web/api/functions/function-topology-streaming.c`.
466
-- Streaming graph links are emitted as child/source actor to parent/destination
467
- actor.
468
-- The parent bullet source incorrectly points to `src_actor`; therefore child
469
- bullets are attached to children, not parents.
470
-- The parent size policy uses `link_count`; the intended producer-owned size
471
- metric is the actor row `retained_node_count`.
472
-
473
-Decision:
474
-
475
-- Parent bullets must attach to the parent side of streaming links
476
- (`dst_actor`).
477
-- Parent actor size must use `presentation.size.mode: "metric"` with
478
- `metric_column: "retained_node_count"`.
479
-- `child_count` remains a direct-child count and can still explain immediate
480
- attachments in the actor header.
481
-
482
-Repair plan:
483
-
484
-1. Update the streaming actor-type emitter so actor types can declare metric
485
- sizing and the actor-ref side used by `ports.sources[]`.
486
-2. Configure the `parent` actor type with `metric(retained_node_count)` sizing.
487
-3. Configure parent port bullets to read streaming link rows where
488
- `dst_actor` is the selected actor.
489
-4. Keep child, virtual, and stale actor types fixed-size with no bullets.
490
-
491
-Validation required:
492
-
493
-- Compile `function-topology-streaming.c`.
494
-- Validate the emitted topology schema path with existing Function validation
495
- tests.
496
-- After install, verify a parent actor receives bullets and grows according to
497
- `retained_node_count`.
498
-
499
-Implementation evidence:
500
-
501
-- `src/web/api/functions/function-topology-streaming.c` now lets streaming actor
502
- types declare metric sizing and a port-bullet actor-ref side.
503
-- The `parent` actor type now declares `size.mode: "metric"`,
504
- `metric_column: "retained_node_count"`, and
505
- `ports.sources[].actor_column: "dst_actor"`.
506
-- The actor table now exposes `retained_node_count` as a metric column and
507
- parent actor labels expose it as `Retained Nodes`.
508
-- Child, virtual-node, and stale actor types remain fixed-size and do not show
509
- bullets.
510
-- `.agents/sow/specs/topology-function-schema.md`,
511
- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`, and
512
- `.agents/skills/project-create-topology/SKILL.md` now record that streaming
513
- parent size is `retained_node_count` and parent bullets attach on
514
- `dst_actor`.
515
-
516
-Validation completed:
517
-
518
-- `git diff --check` passed.
519
-- Compile command from `build/compile_commands.json` for
520
- `src/web/api/functions/function-topology-streaming.c` passed.
521
-- `(cd src/go && go test -count=1 ./pkg/topology/v1 ./tools/functions-validation/validate)`
522
- passed.
523
-
524
-Validation still pending:
525
-
526
-- Live Function/UI validation after the rebuilt Agent is installed/restarted.
527
-
528
-## Regression - 2026-05-18 - Streaming Parent Size Must Use Retained Nodes
529
-
530
-What changed:
531
-
532
-- Parent size must be relative to retained nodes, not direct children or graph
533
- degree.
534
-- Retained nodes means every node for which the parent has DB retention state,
535
- including self, virtual nodes, stale nodes, and transit descendants when the
536
- Agent has retained data for them.
537
-
538
-Evidence:
539
-
540
-- The producer already emits a `retention` table with `actor` and
541
- `observer_actor`, so retained-node ownership is already canonical in the v1
542
- payload.
543
-- The previous visual metric used `child_count`, which is a direct-child count
544
- and does not account for nodes received from another parent.
545
-- The existing schema already supports producer-defined actor metric columns and
546
- `presentation.size.metric_column`, so no schema change is required.
547
-
548
-Decision:
549
-
550
-- Add actor metric column `retained_node_count`.
551
-- Count retained nodes from the same DB-retention state that controls whether a
552
- retention row is emitted.
553
-- Size `parent` actors with
554
- `presentation.size.metric_column: "retained_node_count"`.
555
-- Keep `child_count` as a direct-child explanatory metric in the parent modal
556
- header.
557
-
558
-Implementation evidence:
559
-
560
-- `src/web/api/functions/function-topology-streaming.c` now emits
561
- `retained_node_count` in the actor table and actor labels.
562
-- `retained_node_count` uses aggregation `max` because it is an absolute
563
- retaining-parent property, not an additive duplicate-row metric.
564
-- The parent actor type now sizes by `retained_node_count`.
565
-- The Retained Nodes modal header value is backed by the same actor label.
566
-- `.agents/sow/specs/topology-function-schema.md`,
567
- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`, and
568
- `.agents/skills/project-create-topology/SKILL.md` now record retained-node
569
- sizing.
570
-
571
-Validation completed:
572
-
573
-- `git diff --check` passed.
574
-- Syntax-only compile from `build/compile_commands.json` for
575
- `src/web/api/functions/function-topology-streaming.c` passed. Direct object
576
- compile was not used because the local build object's output path is not
577
- writable in this worktree.
578
-- `(cd src/go && go test -count=1 ./pkg/topology/v1 ./tools/functions-validation/validate)`
579
- passed.
580
-- `git fetch upstream master && git rebase --autostash upstream/master` passed.
581
-- Post-rebase `git rev-list --left-right --count upstream/master...HEAD`
582
- reported `0 12`.
583
-
584
-Validation still pending:
585
-
586
-- Live Function/UI validation after the rebuilt Agent is installed/restarted.
587
-
588
-## Regression - 2026-05-18 - Streaming Actor Header Labels
589
-
590
-What broke:
591
-
592
-- The streaming actor modal header still exposes only generic status labels and
593
- long identity fields. It does not promote important host/inventory labels such
594
- as operating system, kernel, hardware model, CPU, RAM, virtualization, cloud
595
- placement, or vnode inventory fields.
596
-
597
-Evidence:
598
-
599
-- A live `topology:streaming` payload showed rich `actor_labels` for host-like
600
- actors, including OS, kernel, architecture, CPU, RAM, virtualization,
601
- container, cloud provider/type/region, and hardware vendor/product labels.
602
-- The same payload showed vnode/inventory labels including vendor, model,
603
- address, location, sys object id, vnode type, and LLDP identity fields.
604
-- The current modal identification recipe emits only hostname, node type,
605
- stream, ingest, health, children, machine GUID, and Agent version.
606
-
607
-Decision:
608
-
609
-- Host-like streaming actors (`parent`, `child`, `stale`) should expose concise
610
- operational identity plus OS/hardware/platform labels in the modal header.
611
-- Parent actors additionally expose `retained_node_count` and `child_count`.
612
-- Vnode actors should expose inventory/device identity labels instead of
613
- host-only OS/hardware labels.
614
-- `machine_guid` and `node_id` remain in the full Labels tab, not in the modal
615
- header, because they are long identifiers rather than human-scannable master
616
- labels.
617
-
618
-Repair plan:
619
-
620
-1. Make the streaming modal identification recipe role-specific by actor type.
621
-2. Use host-like labels for parent, child, and stale actor types.
622
-3. Use vnode/inventory labels for vnode actor types.
623
-4. Keep all labels in `actor_labels`; do not duplicate row data for the modal
624
- header.
625
-
626
-Validation required:
627
-
628
-- Compile `function-topology-streaming.c`.
629
-- Validate the topology schema tooling.
630
-- After install, verify the actor modal header shows the selected role-specific
631
- label set and hides missing labels cleanly.
632
-
633
-Implementation evidence:
634
-
635
-- `src/web/api/functions/function-topology-streaming.c` now emits role-specific
636
- modal identification recipes:
637
- - `parent`, `child`, and `stale` use host-like operational, OS, hardware, and
638
- platform labels;
639
- - `parent` additionally includes `retained_node_count` and `child_count`;
640
- - `vnode` uses inventory/device labels such as vnode type, vendor, model,
641
- address, location, sys object id, and LLDP name.
642
-- The modal header no longer promotes `machine_guid` or `node_id`; those remain
643
- available through the full Labels tab.
644
-- `.agents/sow/specs/topology-function-schema.md`,
645
- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`, and
646
- `.agents/skills/project-create-topology/SKILL.md` now describe the
647
- role-specific streaming identification policy.
648
-
649
-Validation completed:
650
-
651
-- `git diff --check` passed.
652
-- Compile command from `build/compile_commands.json` for
653
- `src/web/api/functions/function-topology-streaming.c` passed.
654
-- `(cd src/go && go test -count=1 ./pkg/topology/v1 ./tools/functions-validation/validate)`
655
- passed.
656
-
657
-Validation still pending:
658
-
659
-- Live Function/UI validation after the rebuilt Agent is installed/restarted.
.agents/sow/current/SOW-0030-20260517-network-connections-dependency-semantics.md
deleted
-381
@@ -1,381 +0,0 @@
1
-# SOW-0030 - Network Connections Dependency Semantics
2
-
3
-## Status
4
-
5
-Status: paused
6
-
7
-Sub-state: selected by the user on 2026-05-17; implementation starting after
8
-pausing unrelated streaming modal validation work. Paused on 2026-05-17 while
9
-the user requested a streaming graph presentation regression fix under
10
-SOW-0027.
11
-
12
-## Requirements
13
-
14
-### Purpose
15
-
16
-Make `topology:network-connections` show real service dependency direction for
17
-sysadmins, DevOps engineers, and SREs: dependency arrows must point from client
18
-to server, actor modals must separate dependencies from dependants, and
19
-aggregated views must collapse remote endpoint noise without leaking
20
-observer-relative socket terminology.
21
-
22
-### User Request
23
-
24
-The user asked for these network-connections-only topology changes:
25
-
26
-- Direction is important and must be painted with an arrow.
27
-- Actor modals must split socket tables into `Dependencies` and `Dependants`.
28
-- Aggregated backend views should collapse inbound connections by remote IP and
29
- outbound connections by remote IP.
30
-- `local` / `remote` names are wrong for topology and must be removed from the
31
- topology contract.
32
-- Prefer `client` / `server`; if `src` / `dst` remains, then `src` must always
33
- mean client and `dst` must always mean server.
34
-- The topology `local` socket filter should be removed.
35
-- Changes must not affect other topology producers.
36
-
37
-### Assistant Understanding
38
-
39
-Facts:
40
-
41
-- The socket scanner already distinguishes local inbound and local outbound
42
- enum values before the network-viewer producer serializes them.
43
-- The current topology payload uses observer-relative `local_*` and `remote_*`
44
- columns in socket evidence and relationship-summary rows.
45
-- The current topology graph link `src_actor` is the process that owns the local
46
- socket row, not necessarily the client actor.
47
-- The current modal recipe has one `Connections`/`Sockets` section for process
48
- actors instead of separate dependencies and dependants.
49
-- The v1 schema already supports directed link types, actor-column owner
50
- filters, formatted endpoint projections, and arbitrary typed relationship
51
- table columns.
52
-
53
-Inferences:
54
-
55
-- No generic JSON Schema mechanism is needed. This is a network-connections
56
- producer contract change.
57
-- Correlation can remain generic if the producer emits correlation claims for
58
- client/server endpoint ownership and correlation points for visible endpoint
59
- actors using the existing declarative correlation tables.
60
-- Aggregated mode can collapse relationship rows by client actor, server actor,
61
- protocol, and state. The endpoint actor identity carries the remote IP
62
- grouping, while detailed evidence preserves exact client/server ports.
63
-
64
-Unknowns:
65
-
66
-- Live UI polish may still need separate cloud-frontend changes after this
67
- payload changes, but the schema already has enough modal primitives for the
68
- producer to declare the desired sections.
69
-
70
-### Acceptance Criteria
71
-
72
-- `topology:network-connections` type definitions declare socket link types as
73
- directed dependency links with forward arrows.
74
-- Topology graph links use `src_actor = client_actor` and
75
- `dst_actor = server_actor` for network dependency links.
76
-- Topology evidence and relationship-summary rows use `client_*` and
77
- `server_*` endpoint columns, not `local_*` / `remote_*`.
78
-- Topology modal recipes expose `Dependencies` and `Dependants` sections for
79
- process actors using actor-column owner filters.
80
-- The topology Function `info`/required params no longer expose a `local`
81
- socket filter. Local sockets are included through inbound/outbound
82
- dependency classification.
83
-- Aggregated topology relationship rows collapse by client actor, server actor,
84
- protocol, and state, with endpoint actor identity carrying remote IP grouping
85
- instead of local ephemeral port grouping.
86
-- Specs, developer guide, project topology skill, validation fixture, and
87
- focused validation tests are updated.
88
-- The change is limited to `topology:network-connections` and does not alter
89
- SNMP, streaming, vSphere, or legacy non-topology schemas except for the shared
90
- direction string mapping that stops rendering local sockets as `local`.
91
-
92
-## Analysis
93
-
94
-Sources checked:
95
-
96
-- `.agents/skills/project-create-topology/SKILL.md`
97
-- `.agents/skills/project-writing-collectors/SKILL.md`
98
-- `.agents/sow/done/SOW-0025-20260511-network-connections-modal-product-composition.md`
99
-- `.agents/sow/done/SOW-0028-20260511-topology-mode-correlation-aggregation.md`
100
-- `.agents/sow/pending/SOW-0029-20260511-network-connections-detailed-loose-sides.md`
101
-- `.agents/sow/specs/topology-function-schema.md`
102
-- `.agents/sow/specs/topology-modes-correlation-aggregation.md`
103
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
104
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
105
-- `src/collectors/network-viewer.plugin/network-viewer.c`
106
-- `src/libnetdata/local-sockets/local-sockets.h`
107
-- `src/go/tools/functions-validation/fixtures/topology-v1/network-connections.json`
108
-- `src/go/tools/functions-validation/validate/main_test.go`
109
-
110
-Current state:
111
-
112
-- Generic v1 link types already define `orientation`, `direction_role`, and
113
- aggregation direction policy.
114
-- Network-connections currently emits `socket`, `endpoint_socket`, and
115
- `correlated_socket` with `direction_role: flow`.
116
-- Network-connections currently emits observer-relative endpoint columns:
117
- `local_ip`, `local_port`, `remote_ip`, `remote_port`.
118
-- The shared `SOCKET_DIRECTION_2str()` map collapses local inbound and local
119
- outbound into the string `local`.
120
-- The topology required params expose a `local` socket filter.
121
-
122
-Risks:
123
-
124
-- Renaming endpoint columns is a contract-breaking change for any external UI
125
- or aggregator code that hardcoded network-connections table columns.
126
-- Getting client/server actor selection wrong would invert dependency arrows,
127
- which is worse than no arrow.
128
-- Aggregating by remote IP can hide exact port-level rows in aggregated mode;
129
- detailed mode must retain exact client/server tuples.
130
-- Existing dirty streaming/spec changes must not be reverted or mixed into this
131
- SOW outcome.
132
-
133
-## Pre-Implementation Gate
134
-
135
-Status at implementation start: ready (historical snapshot; current SOW state
136
-is recorded in the top-level Status section).
137
-
138
-Problem / root-cause model:
139
-
140
-- The current topology model exposes observer-relative socket facts as if they
141
- were topology semantics. This makes actor modals and aggregation confusing:
142
- `local` means "this socket row's local endpoint", not dependency direction.
143
-- The socket scanner has enough information to derive dependency direction:
144
- inbound/local-inbound maps remote endpoint to client and local endpoint to
145
- server; outbound/local-outbound maps local endpoint to client and remote
146
- endpoint to server; listen has a server only.
147
-- The producer must encode dependency direction directly through
148
- client/server columns and actor refs, so the UI and aggregator do not infer
149
- it from local/remote names.
150
-
151
-Evidence reviewed:
152
-
153
-- `src/libnetdata/local-sockets/local-sockets.h` has distinct
154
- `SOCKET_DIRECTION_LOCAL_INBOUND` and `SOCKET_DIRECTION_LOCAL_OUTBOUND` enum
155
- values and converts loopback/local peers after inbound/outbound detection.
156
-- `src/collectors/network-viewer.plugin/network-viewer.c` maps both local enum
157
- values to the string `local`.
158
-- `src/collectors/network-viewer.plugin/network-viewer.c` currently builds
159
- topology rows with `local_ip`, `remote_ip`, and row-owner `src_actor`.
160
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` already supports directed link
161
- types, relationship tables, actor-column modal owner filters, and formatted
162
- endpoint projections.
163
-
164
-Affected contracts and surfaces:
165
-
166
-- Agent Function payload for `topology:network-connections`.
167
-- Topology developer guide and specs.
168
-- Project topology skill guidance.
169
-- Function validation fixture and semantic tests.
170
-- Potential external consumers of network-connections v1 columns.
171
-
172
-Existing patterns to reuse:
173
-
174
-- Existing compact table encoder helpers in `network-viewer.c`.
175
-- Existing modal `actor_column` owner filters and `formatted_endpoint`
176
- projection.
177
-- Existing type-level link presentation and link aggregation metadata.
178
-- Existing correlation rows with declarative protocol/address-space/IP/port
179
- keys.
180
-
181
-Risk and blast radius:
182
-
183
-- Medium/high semantic risk inside network-connections only.
184
-- Low schema risk because no generic schema mechanism is added.
185
-- Medium compatibility risk for cloud-frontend and cloud-topology-service if
186
- they hardcoded old network-connections column names despite the v1 contract.
187
-- Performance risk is bounded by aggregating rows more aggressively in
188
- aggregated mode.
189
-
190
-Sensitive data handling plan:
191
-
192
-- Use synthetic examples in SOW/spec/docs.
193
-- Do not commit raw live Function payloads, process command lines, private
194
- endpoints, public customer-identifying endpoints, tokens, cookies, machine
195
- GUIDs, or node IDs.
196
-- Store any live captures under `.local/` only if needed.
197
-
198
-Implementation plan:
199
-
200
-1. Update the network-viewer topology producer internal row shape with derived
201
- client/server endpoints and dependency actor selection.
202
-2. Change topology output columns, match columns, correlation rows, and modal
203
- recipes to use client/server semantics.
204
-3. Remove the topology `local` socket filter while still collecting local
205
- sockets through inbound/outbound classification.
206
-4. Update specs, developer guide, project skill, fixtures, and focused tests.
207
-5. Validate with schema checks, semantic tests, and a network-viewer compile
208
- check or build.
209
-
210
-Validation plan:
211
-
212
-- `git diff --check`
213
-- `go test ./tools/functions-validation/validate`
214
-- Network-viewer plugin build target if the local build tree is available, or
215
- the equivalent compile command from `compile_commands.json` with output
216
- redirected to `/tmp`.
217
-- Fixture/schema checks proving network-connections uses client/server columns
218
- and no topology `local` filter.
219
-- Same-failure search for remaining network-connections topology uses of
220
- `local_ip`/`remote_ip` where they would leak topology semantics.
221
-
222
-Artifact impact plan:
223
-
224
-- AGENTS.md: not expected; workflow rules do not change.
225
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md`.
226
-- Specs: update `.agents/sow/specs/topology-function-schema.md` and
227
- `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
228
-- End-user/operator docs: network-connections integration metadata may mention
229
- `local`; update only if the user-facing non-topology Function output changes.
230
-- End-user/operator skills: not expected.
231
-- SOW lifecycle: create SOW-0030 as current/in-progress and pause SOW-0027.
232
-
233
-Open-source reference evidence:
234
-
235
-- Not checked. This is an internal topology payload contract correction, not an
236
- external protocol interpretation.
237
-
238
-Open decisions:
239
-
240
-- Resolved by user: use client/server dependency semantics, remove local/remote
241
- from network-connections topology, remove local topology filter, and keep the
242
- change scoped to network-connections.
243
-
244
-## Implications And Decisions
245
-
246
-1. User decision: `local` / `remote` are not meaningful topology concepts and
247
- must be removed from the network-connections topology contract.
248
-2. User decision: prefer `client` / `server`; if `src` / `dst` remains, then
249
- `src = client` and `dst = server` in all cases.
250
-3. User decision: graph arrows must show dependency direction.
251
-4. User decision: process actor modals must split dependency rows into
252
- `Dependencies` and `Dependants`.
253
-5. User decision: topology `local` socket filter must be removed.
254
-
255
-## Plan
256
-
257
-1. Patch network-viewer topology row model and option parsing.
258
-2. Patch network-viewer topology table schemas, values, modals, and correlation.
259
-3. Update docs/specs/project skill/fixture/tests.
260
-4. Validate narrow commands and record results.
261
-
262
-## Execution Log
263
-
264
-### 2026-05-17
265
-
266
-- Created SOW and paused SOW-0027 before implementation.
267
-- Changed network-connections topology row semantics from observer-relative
268
- `local_*` / `remote_*` columns to dependency-oriented `client_*` /
269
- `server_*` columns.
270
-- Reoriented socket graph links so `src_actor` is the client/dependant and
271
- `dst_actor` is the server/dependency target.
272
-- Removed the topology `local` socket selector from option parsing and Function
273
- metadata; local sockets are collected under inbound/outbound classification.
274
-- Split non-node network-connections modal recipes into `Dependencies` and
275
- `Dependants`.
276
-- Updated the network-connections topology validation fixture, developer guide,
277
- specs, project topology skill, and network-viewer integration text.
278
-
279
-## Validation
280
-
281
-Acceptance criteria evidence:
282
-
283
-- `src/collectors/network-viewer.plugin/network-viewer.c` emits
284
- `direction_role: "dependency"` for `socket`, `endpoint_socket`, and
285
- `correlated_socket`.
286
-- `src/collectors/network-viewer.plugin/network-viewer.c` resolves dependency
287
- actors as client/server and writes `client_*` / `server_*` columns for
288
- relationship and evidence tables.
289
-- `src/collectors/network-viewer.plugin/network-viewer.c` no longer exposes a
290
- topology `local` socket option and maps local inbound/outbound socket enums
291
- to `inbound` / `outbound`.
292
-- `src/go/tools/functions-validation/fixtures/topology-v1/network-connections.json`
293
- validates the updated client/server table contract.
294
-
295
-Tests or equivalent validation:
296
-
297
-- `cmd=$(jq -r '.[] | select(.file|endswith("src/collectors/network-viewer.plugin/network-viewer.c")) | .command' build/compile_commands.json | sed 's# -o [^ ]*# -o /tmp/network-viewer.c.o#'); eval "$cmd"` passed.
298
-- `(cd src/go && go test -count=1 ./tools/functions-validation/validate)` passed.
299
-- `git diff --check` passed.
300
-- `python3 integrations/gen_integrations.py` passed.
301
-- `python3 integrations/gen_docs_integrations.py --collector network-viewer.plugin/network-viewer.plugin` passed.
302
-
303
-Real-use evidence:
304
-
305
-- Pending user install/run validation. The local `build/` directory is owned by
306
- `root:root`, so `cmake --build build --target network-viewer.plugin` could
307
- not write `.ninja_lock`; compile validation used the same compile command
308
- from `build/compile_commands.json` with object output redirected to `/tmp`.
309
-
310
-Reviewer findings:
311
-
312
-- No external assistant review requested for this SOW.
313
-
314
-Same-failure scan:
315
-
316
-- Searched network-connections topology docs/fixtures/skill for stale
317
- `local_ip`, `local_port`, `remote_ip`, `remote_port`, and `sockets_local`
318
- contract references. Remaining matches are generic loose-side/correlation
319
- terminology or non-topology local-IP actor labels.
320
-
321
-Sensitive data gate:
322
-
323
-- No raw live payloads, cookies, tokens, machine GUIDs, or private endpoints
324
- were written. Fixture and docs use documentation-reserved example IP ranges.
325
-
326
-Artifact maintenance gate:
327
-
328
-- AGENTS.md: not expected.
329
-- Runtime project skills: updated `.agents/skills/project-create-topology/SKILL.md`.
330
-- Specs: updated topology schema/mode specs.
331
-- End-user/operator docs: updated `metadata.yaml` and generated integration
332
- markdown to remove the `local` direction wording.
333
-- End-user/operator skills: not expected.
334
-- SOW lifecycle: SOW-0030 current/in-progress; SOW-0027 paused.
335
-
336
-Specs update:
337
-
338
-- Updated `.agents/sow/specs/topology-function-schema.md` and
339
- `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
340
-
341
-Project skills update:
342
-
343
-- Updated `.agents/skills/project-create-topology/SKILL.md`.
344
-
345
-End-user/operator docs update:
346
-
347
-- Updated `src/collectors/network-viewer.plugin/metadata.yaml` and
348
- `src/collectors/network-viewer.plugin/integrations/network_connections.md`.
349
-
350
-End-user/operator skills update:
351
-
352
-- Not expected.
353
-
354
-Lessons:
355
-
356
-- The generic topology JSON Schema did not need a change; the contract change
357
- is a producer profile change using existing open table columns,
358
- actor-column modal filters, and formatted endpoint projections.
359
-
360
-Follow-up mapping:
361
-
362
-- UI and aggregator may still need compatibility updates if they hardcoded old
363
- network-connections `local_*` / `remote_*` columns; this SOW updates the Agent
364
- producer contract and fixture.
365
-- SOW-0029 remains the pending tracker for any future detailed loose-side model.
366
-
367
-## Outcome
368
-
369
-Pending.
370
-
371
-## Lessons Extracted
372
-
373
-Pending.
374
-
375
-## Followup
376
-
377
-None yet.
378
-
379
-## Regression Log
380
-
381
-None yet.
.agents/sow/done/.gitkeep
.agents/sow/done/SOW-0001-20260501-qbridge-fdb-mac-from-index.md
deleted
-565
@@ -1,565 +0,0 @@
1
-# SOW-0001 - SNMP topology: index-based extraction for not-accessible columns + observability + profile-engine macro
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: implementation, regression fix, PR creation, first review-thread fix, Sonar duplication cleanup, and SOW close completed. Live-device validation against the originally reported affected device was not independently runnable in this workspace; residual runtime confirmation remains a PR/local-testing concern.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Make the topology view show every L2 endpoint reachable through every managed switch on the network — laptops, IoT, printers, servers, AP-attached clients — across all common FDB protocol variations and vendor implementations, not just on the subset that happens to fit the path the code currently exercises. Today, on standards-compliant Q-BRIDGE-MIB-only switches, zero FDB endpoints surface. This is the dominant case in real enterprise LANs that skipped the legacy BRIDGE-MIB FDB.
14
-
15
-The work bundles three confirmed bug sites that share one root cause (profile asks for SNMP fields marked `MAX-ACCESS not-accessible` as if they were readable columns), small operational-visibility improvements that surface alongside the fix, and a profile-engine macro that prevents the bug class from recurring.
16
-
17
-### User Request
18
-
19
-After landing the Netgear-switch profile fix (PR #22366) and seeing LLDP work but FDB stay empty, the user asked for a comprehensive plan covering all FDB variations — not a narrow patch.
20
-
21
-Verbatim user request: *"Have you done a plan on what is needed to fix the issue and support all the variations of FDB properly?"*
22
-
23
-Subsequent user direction (recorded as locked decisions below): single PR, multiple isolated commits per group, full SOW documenting every issue, external verification by 5 multi-agent reviewers, cross-check against LibreNMS source code in mirrored repos.
24
-
25
-### Assistant Understanding
26
-
27
-Facts (verified):
28
-
29
-- Three SNMP table fields are walked in our profiles as if they were readable columns, but the source MIBs declare them `MAX-ACCESS not-accessible`. On strict-spec devices, the fields return no value; the runtime sinks then drop the rows because the relevant tag (MAC, IP, etc.) is empty.
30
- - A1: Q-BRIDGE-MIB FDB MAC (`dot1qTpFdbAddress`) — RFC 4363, used as INDEX of `dot1qTpFdbEntry`.
31
- - A2: IP-MIB modern ARP/ND — `ipNetToPhysicalIfIndex`, `ipNetToPhysicalNetAddressType`, `ipNetToPhysicalNetAddress` are all `not-accessible` per RFC 4293. Note that `ipNetToPhysicalPhysAddress` (the MAC itself) **is accessible** (`MAX-ACCESS read-create`); only the three index components are not-accessible.
32
- - A3: LLDP-MIB local management address — `lldpLocManAddrSubtype` and `lldpLocManAddr` are `not-accessible` per IEEE 802.1AB-2005, used as INDEX of `lldpLocManAddrEntry`.
33
-- LibreNMS, OpenNMS, and `netdisco/snmp-info` all extract the MAC for Q-BRIDGE FDB **from the OID index, not from a column**. None of them read `dot1qTpFdbAddress` as a column. Verified directly in mirrored source:
34
- - `librenms/librenms @ 90115d62d82a`: `includes/discovery/fdb-table/bridge.inc.php:31-98`
35
- - `OpenNMS/opennms @ 032d82cc926f`: `features/enlinkd/adapters/collectors/bridge/src/main/java/org/opennms/netmgt/enlinkd/snmp/Dot1qTpFdbTableTracker.java:49-119`
36
- - `netdisco/snmp-info @ 613d360b629d`: `lib/SNMP/Info/Bridge.pm:131-178`
37
-- The profile engine already supports `format: ip_address` on index-derived values (`index_tag_value.go:51-87`) and `format: mac_address` on column-derived values (`utils.go:61`, `value_processor.go:39`). Adding `format: mac_address` to the index-derived value path is a small additive change parallel to the existing `ip_address` case.
38
-- The existing `index_transform` mechanism (`{start: N, drop_right: M}`) already exists at `metrics.go:184` and is applied at `table_row_processor.go:107`. A2/A3 will use this same mechanism in addition to the new `mac_address` formatter.
39
-- Office field validation (sanitized): of four managed switches polled in a real LAN (vendors: MikroTik routerOS, MikroTik SwOS, Zyxel XS-class, Zyxel GS-class), three are backstopped by BRIDGE-MIB FDB (works), one is accidentally lenient by emitting `dot1qTpFdbAddress` as a column despite the spec (works by accident). None hit the strict-only-Q-BRIDGE failure mode. The originally reported failing device is in that strict-only failure mode.
40
-
41
-Inferences:
42
-
43
-- The Q-BRIDGE-MIB FDB profile, the modern IP-MIB ARP profile, and the LLDP local management address profile were likely shaped by mechanical extension of older profiles whose source MIB columns were accessible. Without per-author MIB-discipline awareness, the same mistake will recur.
44
-- The "lenient vendor returns the not-accessible column" hypothesis was previously unverified; it is now empirically confirmed against a real device but does not change the implementation choice (column-derived and index-derived MACs are byte-identical when both exist, per RFC 4363 §4 — both encode the same MAC address bytes).
45
-- Static FDB tables (`dot1qStaticUnicastTable`, `dot1dStaticTable`) are not just feature gaps; they have different semantics (filtering policy, multi-egress, allowed-port lists) that justify separate design rather than bundling here.
46
-
47
-Unknowns:
48
-
49
-- None block the locked decisions or implementation plan. Per-vendor support for proprietary FDB MIBs and the newer IEEE 802.1Q-2014 `IEEE8021-Q-BRIDGE-MIB` are deferred as listed in Followup.
50
-
51
-### Acceptance Criteria
52
-
53
-- A live SNMP poll of the originally reported affected device (Netgear GS110TP v3, sysObjectID `1.3.6.1.4.1.4526.100.4.19`) produces FDB endpoints whose count matches `dot1qTpFdbPort` row count from a fresh walk. Verification: `topology:snmp` function output + manual diff against the walk.
54
-- ARP/ND entries on a strict-spec L3 device are ingested with non-empty IP, ifIndex, and address type. Verification: regression fixture + targeted test, plus opportunistic real-device validation.
55
-- LLDP local management address is correctly populated on devices that implement the standards-compliant variant of `lldpLocManAddrTable`. Verification: regression fixture.
56
-- Existing behavior on devices that already work (BRIDGE-MIB FDB, lenient Q-BRIDGE-MIB devices, hybrid devices, Cisco-style per-VLAN context) is preserved bit-for-bit. Verification: existing `snmp_topology` and parity test suites pass without weakening; commit 7 includes a "lenient vendor" regression fixture (a Q-BRIDGE FDB walk where the column happens to populate) that exercises the no-regression path.
57
-- Engine macro `format: mac_address` is available for index-derived values. Output format: lowercase hex, colon-separated, two-digit per octet (`%02x` style), e.g. `aa:bb:cc:dd:ee:ff`. Octet validation: each component must be 0-255; on validation failure return empty string and let the row be dropped. Length-prefix tolerance: when the slice contains 7 components AND the first component equals 6 AND each of the remaining 6 components is a valid octet (0-255), treat as length-prefixed and use the trailing 6 components (defensive handling for F10).
58
-- **MAC output parity (column-side and index-side)**: column-side `format: mac_address` at `ddsnmp/ddsnmpcollector/utils.go:54` currently uses `%02X` (uppercase) and is exercised by tests at `collector_table_test.go:934-939` and `collector_device_meta_test.go:276-278` that expect uppercase output. As part of commit 1, switch the column-side format string to `%02x` (lowercase) and update those tests so both paths produce byte-identical lowercase output. Add an explicit column-vs-index parity unit test that runs both formatters on the same input and asserts byte equality. Rationale: `net.ParseMAC` and `normalizeMAC` (`topology_hex_normalization.go:18,31`) both lowercase, and industry tooling (LibreNMS, `ip` command, `ifconfig`) renders MACs lowercase — converging to lowercase is the natural shape.
59
-- Index-based extraction for A2 (modern ARP) handles RFC 4293 InetAddress encoding: a leading length octet (4 for IPv4, 16 for IPv6) preceding the address bytes. **The length octet is stripped at the profile level** via `index_transform: [{start: N, drop_right: M}]` on each affected tag (`arp_ip` tag uses positions after the length byte; `arp_addr_type` and `arp_if_index` use the earlier index positions). This keeps `formatIndexIPAddress` pure and documents the SNMP encoding where the MIB structure lives, rather than burying it in the formatter. (Alternative — extending `formatIndexIPAddress` to detect the prefix — was considered and rejected for keeping the formatter format-only.)
60
-- VLAN attribution falls back to `fdbID == VLAN_ID` when the `dot1qVlanCurrentTable` mapping is absent. Specifically: in `topology_cache_fdb.go:46-49`, if `c.fdbIDToVlanID[entry.fdbID]` returns empty, set `entry.vlanID = entry.fdbID`. This is a NEW fallback being ADDED alongside the existing mapping lookup, mirroring LibreNMS `bridge.inc.php:78`. Verification: unit test.
61
-- FDB rows referencing bridge ports without an `ifIndex` mapping are still ingested but tagged as unmapped. Implementation: an internal counter (not a chart) tracks unmapped-bridge-port FDB rows per poll cycle, plus a single rate-limited log line per poll cycle when the count is nonzero. Verification: unit test.
62
-- Warn-on-drop: when FDB rows are dropped due to empty MAC at `topology_cache_fdb.go:11-13`, emit at most ONE warning log per poll cycle with a count of dropped rows (not one log per row). Verification: unit test simulating a strict-Q-BRIDGE poll.
63
-- The profile-author guardrail exists as a project skill at `.agents/skills/project-snmp-profiles-authoring/SKILL.md` (note `project-` prefix per AGENTS.md:35,196,228 — runtime project skills MUST use this prefix) and as a new "Field accessibility" section in `src/go/plugin/go.d/collector/snmp/profile-format.md`. The profile-format.md section includes an audit recipe (a `grep` pattern for finding profile YAML symbols whose MIB declares `not-accessible`). Verification: file exists, links to profile-format.md from the skill, content reviewed in the multi-agent verification round.
64
-- All eight commits in the PR pass the existing `snmp_topology`, `pkg/topology/engine`, and `ddsnmp` test suites without regression.
65
-- Sensitive-data discipline: no community strings, bearer tokens, customer-identifying IPs, customer hostnames (sysName, sysDescr, ifAlias, ifDescr, LLDP remote names, LLDP port descriptions, chassis IDs, management addresses pointing to customer infrastructure), SNMPv3 usernames/auth/priv secrets, or community-member names appear in any committed file (SOW, profile YAMLs, code comments, tests, fixtures, commit messages, PR body, skill, profile-format.md update).
66
-
67
-## Analysis
68
-
69
-### Master gap & bug list (44 items)
70
-
71
-#### A. Confirmed parsing bugs — strict-spec devices return zero / incomplete data (3)
72
-
73
-| # | Site | Source MIB & spec | File:line | Effect |
74
-|---|---|---|---|---|
75
-| **A1** | Q-BRIDGE-MIB FDB MAC | RFC 4363 — `dot1qTpFdbAddress` is `MAX-ACCESS not-accessible`, INDEX of `dot1qTpFdbEntry` | `_std-topology-q-bridge-mib.yaml:21-25` (column read), drop at `topology_cache_fdb.go:10-13` | 0 FDB endpoints on strict-only-Q-BRIDGE devices (Netgear smart switches, parts of MikroTik SwOS, parts of Zyxel) |
76
-| **A2** | IP-MIB modern ARP/ND index components | RFC 4293 — `ipNetToPhysicalIfIndex`, `ipNetToPhysicalNetAddressType`, `ipNetToPhysicalNetAddress` are all `MAX-ACCESS not-accessible`. The MAC itself (`ipNetToPhysicalPhysAddress`) is `read-create` (accessible). | `_std-topology-fdb-arp-mib.yaml:192-217` reads three not-accessible columns | ARP entries on strict L3 devices lose IP, ifIndex, address type. MAC↔IP correlation breaks. The MAC value continues to be readable. |
77
-| **A3** | LLDP-MIB local management address | IEEE 802.1AB-2005 — `lldpLocManAddrSubtype` and `lldpLocManAddr` are `MAX-ACCESS not-accessible`, INDEX of `lldpLocManAddrEntry` | `_std-topology-lldp-mib.yaml:84-101` reads both not-accessible columns; no compat anchor (the *remote* table at `:184-319` has one — this one does not) | Local LLDP management address absent on strict devices |
78
-
79
-#### B. Latent parsing bugs — would surface if we add these tables (4)
80
-
81
-| # | Site | Why it'd repeat the pattern | Spec |
82
-|---|---|---|---|
83
-| **B1** | Q-BRIDGE static FDB `dot1qStaticUnicastTable` | `Address` AND `ReceivePort` both `not-accessible`. Only `AllowedToGoTo` is `read-write` | RFC 4363 |
84
-| **B2** | Q-BRIDGE multicast FDB `dot1qTpGroupTable` | `GroupAddress` `not-accessible` | RFC 4363 |
85
-| **B3** | IEEE8021-Q-BRIDGE-MIB FDB | Same shape, INDEX adds `ComponentId` (3-component INDEX) | IEEE 802.1Q-2018 |
86
-| **B4** | IP-MIB modern `ipAddressTable` | `AddrType` and `Addr` both `not-accessible` | RFC 4293 |
87
-
88
-#### C. Missing capabilities — not polled today (7)
89
-
90
-| # | Capability | Why it matters | Scope decision |
91
-|---|---|---|---|
92
-| **C1** | BRIDGE-MIB static FDB (`dot1dStaticTable`) | RFC 1493 SMIv1 — columns `read-write`, accessible. Different semantics (filtering policy). | **Defer** |
93
-| **C2** | Q-BRIDGE-MIB static FDB (`dot1qStaticUnicastTable`) | Needs index-based extraction (B1). Same semantics caveat as C1. | **Defer** |
94
-| **C3** | IEEE8021-Q-BRIDGE-MIB FDB | Modern alternative for IEEE 802.1Q-2014-only switches. | **Defer** |
95
-| **C4** | Modern `ipAddressTable` | For routers/firewalls running modern IP-MIB only. | **Defer** |
96
-| **C5** | Vendor proprietary FDB MIBs: HUAWEI-L2MAM-MIB, EXTREME-FDB-MIB, DLINKSW-L2FDB-MIB, HP-ICF-BRIDGE (Aruba/HPE), AX-FDB-MIB (AlaxalA), JUNIPER-VLAN/L2ALD-MIB, ALCATEL-IND1-MAC-ADDRESS-MIB (Nokia/Alcatel). LibreNMS also has handlers for EdgeSwitch (Ubiquiti), FortiSwitch (Fortinet), TiMOS (Nokia SR OS), AOS6/AOS7 (Alcatel-Lucent OmniSwitch), and VRP (Huawei). | LibreNMS handlers exist as references. Each is its own design problem. | **Defer** |
97
-| **C6** | CISCO-MAC-NOTIFICATION-MIB (trap-based) | Out-of-SNMP-poll scope | **Defer** |
98
-| **C7** | LLDP-MED, LLDP-EXT-DOT3, LLDP-EXT-DOT1 | Phone/PoE/voice-VLAN/inventory metadata. Out of scope; opt-in by-vendor. | **Defer** |
99
-
100
-#### D. Quality / attribution gaps — data appears, partially incomplete (6)
101
-
102
-| # | Gap | Evidence | Scope decision |
103
-|---|---|---|---|
104
-| **D1** | No `fdbID == VLAN_ID` fallback when `dot1qVlanCurrentTable` is absent. Today `topology_cache_fdb.go:46-49` only consults `c.fdbIDToVlanID`; if empty, `entry.vlanID` stays empty. The fix ADDS a fallback line that sets `entry.vlanID = entry.fdbID` when the mapping returns nothing | LibreNMS `bridge.inc.php:78` precedent (`$vlan = $vlan_fdb_dict[$vlanIndex] ?? $vlanIndex;`) | **In this PR** |
105
-| **D2** | FDB entries on unmapped bridge ports emit `IfIndex: 0` silently | `topology_observation_local_forwarding.go:29-42`. No counter or warning today | **In this PR** |
106
-| **D3** | No detection of FDB truncation by SNMP agent (large tables on small devices) | No `*counts vs walked* ` validation against `dot1qFdbDynamicCount` | **Defer** |
107
-| **D4** | No deduplication when same MAC is in LLDP remote AND FDB | Could produce two endpoint actors for one device | **Defer** |
108
-| **D5** | Port aggregation (LACP/LAG): FDB → member port ifIndex, no LAG rollup | `bridgePortToIf` is 1:1; LibreNMS also does not roll up | **Defer** |
109
-| **D6** | Cross-protocol freshness (LLDP age vs FDB age vs ARP age) not reconciled | Can produce ghost endpoints | **Defer** |
110
-
111
-#### E. Code smells / blind spots (6)
112
-
113
-| # | Issue | Where | Scope decision |
114
-|---|---|---|---|
115
-| **E1** | `macFromOIDIndexSuffix` lives in `_test.go`, not in production | `topology_snmprec_forwarding_test.go:546` | **In this PR** (helper logic folded into engine macro) |
116
-| **E2** | Test fixture parser has its own MAC-from-index fallback that bypasses production path | `topology_snmprec_forwarding_test.go:305-336` masks A1 | **In this PR** (resolved by adding fixtures that exercise profile→engine→cache end-to-end) |
117
-| **E3** | LLDP octet reassembly in `topology_management_address_normalization.go` is bespoke; no shared helper | Will duplicate when fixing A1/A3 | Engine macro replaces it (commit 1) |
118
-| **E4** | No logging when FDB rows are dropped due to empty MAC | `topology_cache_fdb.go:11-13` — silent data loss | **In this PR** (rate-limited per poll cycle, not per row) |
119
-| **E5** | Two anchors for `lldpRemManAddrTable` (primary `.1`/`.2` + compat `.3`) — unclear interaction with strict devices | `_std-topology-lldp-mib.yaml:184-319` — works in practice but warrants a comment | **In this PR** (one-line comment) |
120
-| **E6** | No engine-level `index_format: mac` macro | ddsnmp engine missing feature | **In this PR** (engine macro) |
121
-
122
-#### F. Vendor / firmware quirks (10)
123
-
124
-| # | Quirk | LibreNMS handler | Status in our code | Scope decision |
125
-|---|---|---|---|---|
126
-| **F1** | MikroTik LLDP RemManAddr exposes columns `.1`/`.2` despite spec | (unknown) | Handled (primary anchor in our profile) | n/a |
127
-| **F2** | Zyxel firmware emits malformed Q-BRIDGE indexes that need reshaping | `includes/discovery/fdb-table/zynos.inc.php:27-35` | Not handled | **Defer** |
128
-| **F3** | TP-Link JetStream ifIndex offset of `+49152` between BRIDGE-MIB and IF-MIB | `includes/discovery/fdb-table/jetstream.inc.php:38` | Not handled | **Defer** |
129
-| **F4** | Cisco IOS classic — per-VLAN BRIDGE-MIB via `community@<vlan>` | `includes/discovery/fdb-table/ios.inc.php:29` | Handled (`topology_vlan_context.go`) | n/a |
130
-| **F5** | Some Aruba IAP firmware — truncated Q-BRIDGE indexes | `arubaos.inc.php` partial | Not handled | **Defer** |
131
-| **F6** | Originally reported device firmware bug — 512-byte zero-filled `lldpLocSysCapSupported`/`Enabled` | (Netdata-specific finding) | Tolerated implicitly by `format: hex` | n/a |
132
-| **F7** | Stacked switches (Cisco StackWise, HP IRF) — aggregated FDB on master, per-member on others | Not handled in LibreNMS either | Not handled | **Defer** |
133
-| **F8** | Cisco SB / SF / SG Small Business — non-standard FDB shape | Cisco-SB-specific profile in our codebase | Partially handled | n/a |
134
-| **F9** | Lenient vendors return `dot1qTpFdbAddress` as a column despite spec | LibreNMS doesn't read the column at all (index-only) | Decision 1B (drop column) makes this irrelevant | n/a |
135
-| **F10** | Length-prefix byte in MAC index encoding — some agents (per LibreNMS comment, observed on Aruba CX, Comtrol) prepend a length octet (value 6) before the 6 MAC bytes, producing a 7-component index suffix | `includes/discovery/fdb-table/bridge.inc.php:80-96` | Not handled today | **In this PR** (defensive: when the slice has 7 components and the first equals 6, drop it; engine macro acceptance criterion above) |
136
-
137
-#### G. Engine / ddsnmp gaps (4)
138
-
139
-| # | Gap | Why it matters | Scope decision |
140
-|---|---|---|---|
141
-| **G1** | No `MAX-ACCESS` validation when authoring profiles | Lets the A-class bug be authored without warning | **In this PR** (skill + profile-format.md) |
142
-| **G2** | No native multi-position index extraction with format coercion | Forces ugly per-octet enumeration; the same fix shape we need for A1, A2, A3 | **In this PR** (engine macro `format: mac_address` for index-derived values) |
143
-| **G3** | No "fail-loud" when a column symbol returns no rows on a populated table | Silent data loss | **Defer** |
144
-| **G4** | No per-poll-cycle stats on rows-fetched vs rows-dropped per metric | Hard to diagnose A-class bugs in production | **Partial** — covered by E4 (warn-on-drop) for the FDB sink |
145
-
146
-#### H. Process / documentation gaps (4)
147
-
148
-| # | Gap | Scope decision |
149
-|---|---|---|
150
-| **H1** | No MIB-authoring spec or project skill — every author can repeat the not-accessible mistake | **In this PR** — `.agents/skills/project-snmp-profiles-authoring/SKILL.md` (note `project-` prefix per AGENTS.md:35,196,228) |
151
-| **H2** | No same-failure scan was performed at PR review time for this profile family | **In this PR** — reviewers ran the scan, results integrated; project skill encodes the rule for future PRs |
152
-| **H3** | Test fixtures cannot trivially be derived from real SNMP walks (snmprec format conversion not documented) | **Defer** |
153
-| **H4** | No reference table linking each topology MIB column we walk to its accessibility class | **In this PR** — `profile-format.md` MAX-ACCESS section, includes a `grep` audit recipe |
154
-
155
-### LibreNMS verification matrix
156
-
157
-For each issue, what LibreNMS does. File paths are relative to `librenms/librenms @ 90115d62d82a`.
158
-
159
-| Issue | LibreNMS handles? | Code path | Approach |
160
-|---|---|---|---|
161
-| A1 (Q-BRIDGE FDB MAC) | Yes | `includes/discovery/fdb-table/bridge.inc.php:31-35` (walk), `:98` (parse), `:80-96` (length-prefix tolerance) | Walk `dot1qTpFdbPort` (column .2, accessible); extract MAC from index. No column read. |
162
-| A2 (IP-MIB ARP `ipNetToPhysicalTable`) | Yes | `LibreNMS/Modules/ArpTable.php:104-118` | Walk `ipNetToPhysicalPhysAddress` and consume the **structured table keys** returned by `->table(1)` (a hierarchical ifIndex→addrType→address map; not a raw OID-suffix split). Shape repair at line 120. The MAC value comes from the column (which is accessible); IP/ifIndex/addrType come from the structured key path. |
163
-| A3 (LLDP local mgmt addr) | No | n/a — table not polled | LibreNMS sidesteps by not reading this table at all. We choose the right thing: index extraction. |
164
-| B1/C2 (Q-BRIDGE static FDB) | No | n/a — table not polled | Same gap |
165
-| B3/C3 (IEEE8021-Q-BRIDGE-MIB) | Partial | `LibreNMS/OS/Traits/QBridgeMib.php:50-58` | Used only for VLAN names; FDB extraction stays on classic Q-BRIDGE-MIB |
166
-| B4/C4 (modern `ipAddressTable`) | Partial | IPv4 still uses deprecated `ipAddrTable` (`LibreNMS/Modules/Ipv4Addresses.php:195`); IPv6 uses modern `ipAddressTable` (`LibreNMS/Modules/Ipv6Addresses.php:148`) | Mixed: deprecated for v4, modern for v6 |
167
-| C5 (vendor proprietary FDBs) | Yes (≥11 vendors) | `includes/discovery/fdb-table/{arubaos,vrp,ios,aos6,aos7,zynos,jetstream,edgeswitch,fortiswitch,timos,...}.inc.php` | Per-vendor override files |
168
-| D1 (VLAN fdbID==VLAN_ID fallback) | Yes | `bridge.inc.php:78` | `$vlan = $vlan_fdb_dict[$vlanIndex] ?? $vlanIndex;` |
169
-| F2 (Zyxel malformed Q-BRIDGE index) | Yes | `zynos.inc.php:27-35` | Reshape pass before normal parsing |
170
-| F3 (TP-Link JetStream offset) | Yes | `jetstream.inc.php:38` | Hardcoded `+49152` ifIndex offset |
171
-| F4 (Cisco per-VLAN context) | Yes | `ios.inc.php:29` | `SnmpQuery::context($vlan_raw, 'vlan-')->walk('BRIDGE-MIB::dot1dTpFdbPort')` |
172
-| F10 (length-prefix byte) | Yes | `bridge.inc.php:80-96` | Defensive: detect and strip the 7-byte index encoding |
173
-| Bridge-port → ifIndex | Yes | `bridge.inc.php:49-54` (build), `:108` (fallback to `basePort==ifIndex`) | Walks `dot1dBasePortIfIndex`; falls back when missing |
174
-| Stack / multi-component bridge | No | n/a | Same gap as ours |
175
-| LACP / port-channel rollup | No | n/a | Same gap as ours |
176
-| Test coverage / fixtures | Limited | `tests/data/timos_fdb-table.json`, `tests/snmpsim/zynos_gs1900-fdb.snmprec` | Two relevant fixtures |
177
-
178
-Cross-references:
179
-
180
-- `netdisco/snmp-info` `Bridge.pm:160-178` — `_qb_fdbtable_index` (lines 160-165) decodes MAC from index; `qb_fw_mac` (lines 167-178) walks `qb_fw_port` and applies the decoder. Never reads the not-accessible column.
181
-- OpenNMS `features/enlinkd/adapters/collectors/bridge/src/main/java/org/opennms/netmgt/enlinkd/snmp/Dot1qTpFdbTableTracker.java:64` (column collection: port + status only), `:113` (decodes address from row index).
182
-
183
-**Conclusion**: index-based extraction is the universal industry approach for A1. A2 and A3 are RFC-backed analogs with the same mechanical fix shape, but the three reference implementations do not directly cover them — A2 has a partial parallel in LibreNMS `ArpTable.php` (which uses the structured-table-key approach, semantically equivalent), A3 has no reference precedent (LibreNMS skips the table). Decision 1B is the right call for all three; A1 is independently proven, A2/A3 are RFC-driven.
184
-
185
-### Risks (cross-cutting)
186
-
187
-- **Engine macro added in this PR**: contained scope. The format handler extends `formatIndexTagValue` with `case "mac_address":` parallel to the existing `case "ip_address":`. Octet validation (each component 0-255) and length-prefix tolerance (drop a leading length-of-6 octet when the slice has 7 components) are part of the formatter, mirroring `formatIndexIPAddress` patterns and LibreNMS behavior. Engine has the existing `format: ip_address` precedent that exercises the same code path.
188
-- **A2/A3 implementation completeness**: the new `format: mac_address` formatter is necessary but not sufficient. A2 and A3 need correct `index` + `index_transform` declarations in the profile YAML to slice the OID suffix to the right components before formatting. For A2 in particular, the IP component of the OID index is RFC 4293 InetAddress-encoded (a length octet followed by address bytes); the length octet is **stripped at the profile level** via `index_transform` so the `format: ip_address` handler stays pure (format-only). The engine macro adds `format: mac_address` for index values; profile-level slicing handles SNMP encoding peculiarities.
189
-- **Removing the column read** (Decision 1B): index-derived MAC and column-derived MAC are byte-identical when both exist (RFC 4363 §4 — both encode the same MAC bytes). On lenient devices that today populate the column, behavior stays correct. On strict devices, FDB starts working. No device regresses. The lenient-vendor regression fixture in commit 7 verifies this.
190
-- **VLAN fallback (D1)** could over-attribute if a device uses `fdbID != VLAN_ID` mapping but doesn't expose `dot1qVlanCurrentTable`. LibreNMS accepts this trade-off; the fallback is a documented best-effort.
191
-- **Warn-on-drop (E4)** could be noisy. Implementation MUST be rate-limited to one log line per poll cycle with a count, not one per dropped row. This is in the acceptance criteria.
192
-
193
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
194
-
195
-Gate state at implementation start: decisions locked; implementation authorized after validation.
196
-
197
-Problem / root-cause model:
198
-
199
-- Three SNMP table fields are walked in our profiles as if they were readable columns, but the source MIBs declare them `MAX-ACCESS not-accessible`. Strict-spec devices correctly return no value for those columns. Our runtime sinks then drop the rows because the relevant tag (MAC, IP, etc.) is empty. The bug class repeats because there is no project-level rule requiring profile authors to consult MIB MAX-ACCESS, and no engine-level macro making the right way (index extraction with format coercion) ergonomic.
200
-
201
-Evidence reviewed:
202
-
203
-- RFCs 1493, 4188, 4293, 4363; IEEE 802.1AB-2005; IEEE 802.1Q-2018 (downloaded into `/tmp/sow-verify/`).
204
-- Profile YAMLs: `_std-topology-q-bridge-mib.yaml`, `_std-topology-fdb-arp-mib.yaml`, `_std-topology-lldp-mib.yaml`, `_std-topology-stp-mib.yaml`, `_std-topology-cisco-vtp-mib.yaml`.
205
-- Runtime sinks: `topology_cache_fdb.go`, `topology_cache_metric_dispatch.go`, `topology_cache_tags.go`, `topology_observation_local_forwarding.go`, `topology_vlan_context*.go`, `topology_observation_local_identity.go`.
206
-- Engine: `ddsnmp/ddprofiledefinition/{metrics.go,validation.go,selector.go}`, `ddsnmp/ddsnmpcollector/{table_row_processor.go,index_tag_value.go,utils.go,value_processor.go,cross_table_lookup.go}`.
207
-- `librenms/librenms @ 90115d62d82a` source (file:line citations in matrix above).
208
-- `netdisco/snmp-info @ 613d360b629d`: `lib/SNMP/Info/Bridge.pm` and `lib/SNMP/Info/IEEE802_Bridge.pm`.
209
-- `OpenNMS/opennms @ 032d82cc926f`: `features/enlinkd/.../Dot1qTpFdbTableTracker.java`.
210
-- Live SNMP walks from one originally reported affected device and four office-network managed switches (3 vendors).
211
-- Two rounds of external multi-agent reviews (5 reviewers per round: Codex, GLM, MiniMax, Qwen, Kimi). Round 2 produced 20 fixes that have been applied to this SOW.
212
-
213
-Affected contracts and surfaces:
214
-
215
-- Profile YAMLs: `_std-topology-q-bridge-mib.yaml`, `_std-topology-fdb-arp-mib.yaml`, `_std-topology-lldp-mib.yaml`.
216
-- Runtime: `topology_cache_fdb.go`, `topology_cache_tags.go`, `topology_observation_local_forwarding.go`.
217
-- Engine: `ddsnmp/ddsnmpcollector/index_tag_value.go` (extending `formatIndexTagValue`).
218
-- New skill: `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
219
-- Updated docs: `src/go/plugin/go.d/collector/snmp/profile-format.md` (new "Field accessibility" section + audit recipe).
220
-- Test data: new snmprec fixtures under `src/go/plugin/go.d/collector/snmp_topology/testdata/`.
221
-- No public/operator docs change beyond release notes.
222
-- No public CLI / UI / schema change.
223
-- No `AGENTS.md` change required (skill follows the `project-*` convention; no legacy registration needed).
224
-
225
-Existing patterns to reuse:
226
-
227
-- LLDP management-address octet decomposition in `_std-topology-lldp-mib.yaml:206-239` and the runtime reassembly in `topology_management_address_normalization.go:13-35` (`reconstructLldpRemMgmtAddrHex`) — supersedable once the engine macro lands; kept as production reference until then.
228
-- VLAN-context plumbing in `topology_vlan_context.go` is robust and need not change.
229
-- Test helper `macFromOIDIndexSuffix(parts []string)` at `topology_snmprec_forwarding_test.go:546` — its decode logic moves into production as part of the engine macro (commit 1).
230
-- LibreNMS `bridge.inc.php` "fdbID == vlanID" fallback at `:78` — directly mirrored.
231
-- Existing `formatIndexIPAddress` at `index_tag_value.go:58-87` — the new `formatIndexMACAddress` parallels it (validation, length-prefix tolerance, error path returning empty string).
232
-
233
-Risk and blast radius:
234
-
235
-- A1+A2+A3 fixes: low. Additive (column read removed, index extraction added). No device regresses (RFC 4363 §4 guarantees byte equivalence; lenient-vendor fixture in commit 7 verifies).
236
-- Engine `format: mac_address` for index-derived values: low. Mirrors existing `format: ip_address`; tests reuse the same harness.
237
-- D1 VLAN fallback: low. Documented best-effort matching LibreNMS.
238
-- D2 IfIndex tracking: low. Internal counter + one log line per poll cycle. No chart, no public metric.
239
-- E4 warn-on-drop: low. Rate-limited per poll cycle.
240
-- Static FDB additions: deferred — different semantics, separate SOW.
241
-- Per-vendor SOWs: deferred, no risk in this PR.
242
-
243
-Sensitive data handling plan:
244
-
245
-- All durable artifacts in this SOW (the SOW itself, profile YAMLs, code, code comments, tests, fixtures, commit messages, PR body, the new skill, the profile-format.md update) must contain zero raw sensitive data: no community member or customer names, no SNMP communities, no bearer tokens, no SNMPv3 usernames / authentication / privacy secrets, no customer-identifying IPs (private RFC1918 IPs are acceptable when used as illustrative examples in profile-format.md, never in fixtures derived from real walks), no customer-identifying device strings (real-world `sysName`, `sysDescr`, `ifAlias`, `ifDescr`, LLDP remote `sysName`, LLDP remote port descriptions, real chassis IDs, customer-pointing management addresses), no proprietary incident details.
246
-- Use placeholders (`[REDACTED]`, "the user", "the reporter", "originally reported affected device"), public product names (e.g. "Netgear GS110TP v3" is a publicly sold product, not PII), and file:line citations.
247
-- Test fixtures derived from real device walks must be sanitized: replace customer hostnames in port descriptions or sysName with neutral labels (`endpoint-1`, `port-A`, etc.), and strip or stub LLDP remote-name fields and LLDP port descriptions. Sanitization happens before the fixture is staged.
248
-- Pre-commit checklist (run before each commit and before opening the PR). Uses `rg -P` (ripgrep with PCRE) — POSIX `grep -E` does not support negative lookahead and would silently fail. The checklist scans **both staged and unstaged** content, plus commit messages and the PR body separately.
249
- ```bash
250
- # Helper: stream all uncommitted changes (staged + unstaged)
251
- git_diff_all() { { git diff --cached; git diff; }; }
252
-
253
- # 1. credentials and tokens in any uncommitted content
254
- git_diff_all | rg -P '(community|bearer|token|secret|password|auth.?key|priv.?key|snmpv3)\s*[:=]'
255
-
256
- # 2. customer-identifying IPv4 outside RFC1918, and IPv6 global-unicast-like
257
- # addresses (2000::/3). The OID-context exclusion
258
- # (^| [^0-9.])(?!1\.[0-3]\.6\.1\.) avoids flagging SNMP OIDs like
259
- # 1.3.6.1.4.1.x. The lookahead chain after that excludes private IPv4
260
- # ranges. The IPv6 pattern is case-insensitive and intentionally excludes
261
- # ULA fc00::/7.
262
- git_diff_all | rg -P '(?<![0-9.])(?!1\.[0-3]\.6\.1\.)(?!10\.)(?!172\.(1[6-9]|2[0-9]|3[01])\.)(?!192\.168\.)\b[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\b'
263
- git_diff_all | rg -P '(?i)\b(?:2[0-9a-f]{3}|3[0-9a-f]{3}):[0-9a-f:]+\b'
264
-
265
- # 3. customer-identifying device strings (replace with neutral labels in fixtures)
266
- git_diff_all | rg -iP '(sysName|sysDescr|ifAlias|ifDescr|chassis.?id|mgmt.?addr|lldp.?rem.?(sys.?name|port.?desc))'
267
-
268
- # 4. proper names that surfaced in chat or Slack during work on this SOW.
269
- # Maintain the personal-name list in a local environment file (.env or
270
- # AGENTS.local.md, both gitignored). Never embed names in this SOW.
271
- # Example: NAME_PATTERN="Firstname1|Firstname2|Surname1"; export NAME_PATTERN
272
- git_diff_all | rg -iP "$NAME_PATTERN"
273
-
274
- # 5. commit messages — three coverage paths because each catches a different point:
275
- # a) prepare-commit-msg hook: scan $1 (the in-progress message file)
276
- # before commit goes through.
277
- # b) post-commit verification of the most recent commit:
278
- git log -1 --format='%B' | rg -iP '(community|bearer|token|secret|password|auth.?key|priv.?key|snmpv3)\s*[:=]|sysName|sysDescr|ifAlias|ifDescr|chassis.?id|mgmt.?addr'
279
- git log -1 --format='%B' | rg -iP "$NAME_PATTERN"
280
- # c) range scan across all commits not yet merged to upstream/master,
281
- # just before opening the PR:
282
- git log upstream/master..HEAD --format='%B' | rg -iP '(community|bearer|token|secret|password|auth.?key|priv.?key|snmpv3)\s*[:=]|sysName|sysDescr|ifAlias|ifDescr|chassis.?id|mgmt.?addr'
283
-
284
- # 6. PR body — apply the same patterns to the body before `gh pr create` /
285
- # `gh pr edit --body-file`. If using a body file, run the patterns
286
- # against that file directly.
287
- ```
288
- Each `rg` invocation MUST return zero lines before commit / PR open. If `rg` is unavailable, the regex set requires it and the checklist cannot be satisfied with `grep` alone — install ripgrep first.
289
-
290
-Implementation plan: see "Plan" section below — 8 commits, single PR off `upstream/master`, branch `snmp-qbridge-fdb-mac-from-index`. Validation before implementation found one required engine prerequisite: current `index_transform` cannot express "start at N and keep the rest" (`validation.go` rejects `start > end`, and `applyIndexTransform` requires an explicit in-bounds `end`). The user accepted extending `index_transform` so `start > 0` with omitted/zero `end` means "through the last index component"; `start: 0, end: 0` keeps its existing "first component only" behavior.
291
-
292
-Validation plan:
293
-
294
-- Unit tests for: engine `format: mac_address` formatter (octet validation 0-255, length-prefix tolerance, output format `aa:bb:cc:dd:ee:ff`); A1+A2+A3 profile→engine→cache flow on synthetic strict-spec fixtures; D1 VLAN fallback; D2 unmapped-bridge-port counter; E4 rate-limited warn-on-drop.
295
-- Snmprec fixtures: at least one strict-spec Q-BRIDGE FDB fixture (sanitized derivative of the user-reported walk), one strict-spec modern ARP fixture (synthetic, RFC 4293 InetAddress-encoded), one strict-spec LLDP local mgmt addr fixture (synthetic), one **lenient-vendor** Q-BRIDGE FDB fixture (column populated; verifies no-regression on the path that already works).
296
-- Existing test suites must pass without weakened assertions: `snmp_topology`, `pkg/topology/engine`, `ddsnmp`.
297
-- Manual validation: live poll against the originally reported affected device showing FDB endpoint count matches walked row count.
298
-- Same-failure scan output: documented in this SOW under `## Validation` after commit 7. No further not-accessible-as-column sites detected by the multi-agent review.
299
-- Profile-format.md and skill content: reviewed in the multi-agent verification round before any commits.
300
-
301
-Artifact impact plan:
302
-
303
-- AGENTS.md: update required. The new runtime project skill follows the `project-*` convention, and the Project Skills Index must stop saying no `project-*` skills exist.
304
-- Runtime project skills: new `.agents/skills/project-snmp-profiles-authoring/SKILL.md` (in this PR).
305
-- Specs under `.agents/sow/specs/`: no expected change. The skill + profile-format.md cover the authoring rule.
306
-- End-user/operator docs: `profile-format.md` gets a new "Field accessibility" section with the rule + audit recipe.
307
-- End-user/operator skills: none.
308
-- SOW lifecycle plan at implementation start: move from `pending/` to `current/` after user authorization, then close after merge or explicit user request. Actual closure is recorded in the execution log and validation gate.
309
-
310
-Open decisions: none. All five locked below.
311
-
312
-## Locked Decisions
313
-
314
-### Decision 1 — MAC extraction approach for not-accessible columns
315
-
316
-**Locked: Option B — drop column read entirely, extract from index only.**
317
-
318
-Reasoning: For A1, three reference implementations (LibreNMS `bridge.inc.php`, SNMP::Info `Bridge.pm`, OpenNMS `Dot1qTpFdbTableTracker.java`) all use index-only extraction; none reads the not-accessible column. For A2 and A3, the same approach is RFC-backed and mechanically identical, though direct industry parallels are fewer (LibreNMS handles A2 via a structured-table-key path semantically equivalent to index extraction; A3 is sidestepped by not polling the table). The "lenient vendor returns the column" hypothesis is empirically true for at least one vendor (Zyxel GS-class) but does not change the choice — column-derived and index-derived MACs are byte-identical (RFC 4363 §4), so removing the column read produces identical bytes on lenient devices and starts working on strict devices. Two paths multiplied across 3 sites = unjustified maintenance burden.
319
-
320
-### Decision 2 — Authoring guardrail format and path
321
-
322
-**Locked: project skill at `.agents/skills/project-snmp-profiles-authoring/SKILL.md` (note `project-` prefix per AGENTS.md:35,196,228 — runtime project skills MUST use this prefix). Plus a new "Field accessibility" section in `src/go/plugin/go.d/collector/snmp/profile-format.md` containing the rule and a `grep` audit recipe. The skill links to profile-format.md.**
323
-
324
-Reasoning: The user requested skill-shape rather than spec-shape. profile-format.md is the canonical authoring reference (2046 lines, no current MAX-ACCESS mention) and is the right home for the rule itself. The skill points authors at it. The `project-` prefix follows the project convention; using a different prefix would require an explicit registration entry in AGENTS.md, which is unjustified for a new skill.
325
-
326
-### Decision 3 — Quality / observability improvements in this PR
327
-
328
-**Locked: Option A — include all three.**
329
-
330
-- D1 VLAN fallback: resolve `fdbID == VLAN_ID` as a late fallback when producing observations, after the `dot1qVlanCurrentTable` mapping has had a chance to populate. Do not eagerly store `entry.vlanID = entry.fdbID` in `updateFdbEntry`, because Q-BRIDGE FDB rows are processed before the VLAN mapping table in the current profile order and an eager fallback can block a later correct mapping.
331
-- D2 IfIndex tracking: when `parseIndex(c.bridgePortToIf[bridgePort])` returns 0, increment an internal per-poll counter; emit at most ONE log line per poll cycle if the counter is nonzero. No new chart or public metric.
332
-- E4 warn-on-drop: when `topology_cache_fdb.go:11-13` drops rows due to empty MAC, emit at most ONE rate-limited warning per poll cycle with a count of dropped rows (not one log per row).
333
-
334
-Reasoning: Each is file-local, additive, cheap, and ensures the next bug of this class is visible immediately rather than requiring forensic walks.
335
-
336
-### Decision 4 — Engine macro G2 in this PR; static FDB / IEEE8021-Q-BRIDGE-MIB / modern ipAddressTable / vendor proprietary deferred to child SOWs
337
-
338
-**Locked: engine macro IN this PR. Static FDB, IEEE8021-Q-BRIDGE-MIB, modern `ipAddressTable`, vendor-proprietary FDB MIBs all DEFERRED — child SOWs opened before this SOW closes.**
339
-
340
-Reasoning: Engine macro is a small additive change (extends `formatIndexTagValue` with `case "mac_address":` parallel to the existing `case "ip_address":`, plus octet validation and length-prefix tolerance). Static FDB has different semantics (filtering policy, multi-egress) and warrants its own design SOW. IEEE8021-Q-BRIDGE-MIB and modern `ipAddressTable` have no real-device evidence in this SOW justifying immediate work. Vendor-proprietary FDBs are each their own design problem.
341
-
342
-### Decision 5 — `index_transform` variable-tail support
343
-
344
-**Locked: Option A — extend `index_transform` semantics.**
345
-
346
-Reasoning: Q-BRIDGE MAC extraction and variable-length IP/LLDP management-address indexes need "slice from this index component through the end". Current `index_transform` can only select explicit inclusive ranges or drop a fixed number of right-side components. Duplicating tags for every possible IPv4/IPv6 or normal/length-prefixed shape would be brittle and order-sensitive because tag insertion does not overwrite an existing non-empty tag. Extending the engine is low risk because `start > 0, end == 0, drop_right == 0` is currently invalid, while existing `start: 0, end: 0` keeps its current first-component meaning.
347
-
348
-## Plan
349
-
350
-Single PR, branch `snmp-qbridge-fdb-mac-from-index` off `upstream/master`. Eight commits, isolated per group:
351
-
352
-| # | Commit | Files touched | Issues addressed |
353
-|---|---|---|---|
354
-| 1 | `engine: add format mac_address for index-derived tag values; converge column-side to lowercase` | `ddsnmp/ddsnmpcollector/index_tag_value.go` (new `formatIndexMACAddress` + switch case, lowercase output), `ddsnmp/ddsnmpcollector/table_row_processor.go` and `ddsnmp/ddprofiledefinition/validation.go` (extend `index_transform` so `start > 0` with omitted/zero `end` keeps the tail), `ddsnmp/ddsnmpcollector/utils.go:54` (flip `%02X` → `%02x` for parity), `ddsnmp/ddsnmpcollector/collector_table_test.go:934-939` and `collector_device_meta_test.go:276-278` (update existing assertions from uppercase to lowercase), unit tests including an explicit column-side ↔ index-side parity test | G2, E1 (folds in `macFromOIDIndexSuffix` decode logic), F10 (length-prefix tolerance with octet validation), MAC parity (column-side and index-side both produce lowercase `aa:bb:cc:dd:ee:ff`), variable-tail index extraction needed by A1/A2/A3 |
355
-| 2 | `profile + runtime: q-bridge fdb mac from index` | `_std-topology-q-bridge-mib.yaml` (replace octet enumeration with `format: mac_address` + `index_transform`), `topology_cache_tags.go` (rename/cleanup tag constants if needed), `topology_cache_fdb.go` (consume the new tag value), unit tests | A1 |
356
-| 3 | `profile + runtime: ip-mib modern arp from index` | `_std-topology-fdb-arp-mib.yaml` — replace not-accessible column reads with `index` + `index_transform` extraction. For `arp_if_index` and `arp_addr_type`: single `index: N` lookups. For `arp_ip`: `index_transform: [{start: <after-length-byte>}]` to skip the RFC 4293 InetAddress length octet *at the profile level*, then `format: ip_address` consumes a clean 4 / 16 octet sequence (formatter stays format-only; SNMP-encoding logic stays in the profile YAML where the MIB structure is documented). `topology_cache_stp_arp.go` and any tag consumer; unit tests including an IPv4 + IPv6 strict-spec fixture pair. | A2 |
357
-| 4 | `profile + runtime: lldp local management address from index` | `_std-topology-lldp-mib.yaml`, `topology_management_address.go` and/or `topology_management_address_normalization.go` (consumer), unit tests | A3, E5 (one-line comment near the dual anchors of `lldpRemManAddrTable`) |
358
-| 5 | `runtime: fdbID == VLAN_ID fallback when mapping table absent` | `topology_observation_local_forwarding.go` (late fallback to `entry.fdbID` only when no mapped VLAN ID exists at observation-output time), `topology_cache_fdb.go` (keep cache mutation compatible with late mapping), unit tests | D1 |
359
-| 6 | `runtime: warn-on-drop + unmapped-bridge-port counter` | `topology_cache_fdb.go:11-13` (rate-limited per-poll-cycle warning with count), `topology_observation_local_forwarding.go:29-42` (per-poll-cycle counter for `IfIndex == 0` cases + single log line when nonzero), unit tests verifying rate-limiting | D2, E4 |
360
-| 7 | `tests: snmprec fixtures + regression tests` | new fixtures under `src/go/plugin/go.d/collector/snmp_topology/testdata/`: strict-spec Q-BRIDGE FDB (sanitized from real walk), strict-spec ARP (synthetic), strict-spec LLDP local mgmt (synthetic), **lenient-vendor Q-BRIDGE FDB (column populated, verifies no-regression)**; forwarding tests that exercise all four | regression coverage for A1+A2+A3 |
361
-| 8 | `docs/skill: snmp profile authoring guardrail` | new `.agents/skills/project-snmp-profiles-authoring/SKILL.md` (links to profile-format.md), new "Field accessibility" section in `src/go/plugin/go.d/collector/snmp/profile-format.md` with the MAX-ACCESS rule + a `grep` audit recipe, update `AGENTS.md` Project Skills Index | H1, H4 |
362
-
363
-Order matters: commit 1 (engine) lands before 2-4 (profiles that consume the new format). 5-6 (observability) before 7 (tests can exercise them). 8 (docs) last so the skill reflects the final shape of the engine macro and profile pattern.
364
-
365
-Pre-commit checklist (Pre-Implementation Gate § Sensitive data plan) MUST pass before each commit and before PR open.
366
-
367
-## Execution Log
368
-
369
-### 2026-05-01
370
-
371
-- Created branch `snmp-qbridge-fdb-mac-from-index` off `upstream/master`.
372
-- Drafted initial narrow SOW; user pushed back asking for full survey across all FDB variations.
373
-- Re-investigated topology profiles, runtime sinks, ddsnmp engine, RFC 1493/4188/4293/4363 and IEEE 802.1AB-2005 / 802.1Q-2018.
374
-- Sent narrower SOW to 5 external reviewers (round 1: Codex, GLM, MiniMax, Qwen, Kimi). Codex independently identified A2 (IP-MIB ARP). User locked four decisions.
375
-- Verified user's office network: 3 of 4 managed switches backstopped by BRIDGE-MIB FDB; 1 lenient. Confirmed bug-affected population is "strict + Q-BRIDGE-only + no BRIDGE-MIB".
376
-- LibreNMS verification subagent ran; results integrated.
377
-- Sanitized SOW and `AGENTS.local.md`: no community member names, no SNMP communities, no bearer tokens.
378
-- Sent expanded SOW to the 5 reviewers (round 2). Findings consolidated:
379
- - 4 of 5 confirmed Bridge.pm citation off (242-271 → 160-178). Fixed.
380
- - Codex flagged Decision 2 skill path conflict with AGENTS.md (`project-*` convention). Fixed (path now `project-snmp-profiles-authoring`).
381
- - Codex flagged catalog count "28" wrong (actual 43 + F10 = 44). Fixed.
382
- - Codex flagged LibreNMS `ArpTable.php` mischaracterization (uses structured table keys, not raw index). Fixed.
383
- - Codex flagged LibreNMS `ipAddressTable` claim wrong (IPv4 deprecated, IPv6 modern). Fixed.
384
- - 4 of 5 flagged length-prefix byte edge case (F10). Added to catalog and engine macro acceptance.
385
- - 5 of 5 flagged pre-commit checklist as too vague. Replaced with concrete grep commands.
386
- - Codex + MiniMax flagged A2 phrasing — `ipNetToPhysicalPhysAddress` IS accessible. Fixed.
387
- - MiniMax + Codex flagged D1 description ambiguity (must say "ADD a fallback", not "fix existing lookup"). Fixed in acceptance criteria + Decision 3 + commit 5.
388
- - MiniMax flagged D4/D5/D6 explicit "Defer". Fixed (table now has scope column with explicit "Defer").
389
- - Kimi + Qwen + GLM flagged engine macro output format (lowercase hex, colon-separated, octet validation). Added to acceptance criteria.
390
- - Kimi flagged lenient-vendor regression fixture missing from commit 7. Added.
391
- - Codex flagged commit 5 file list incomplete (D1 also touches `topology_cache_fdb.go`). Fixed.
392
- - Codex flagged Decision 1 wording overclaim. Narrowed.
393
- - Codex + GLM flagged D2/E4 metric type undefined. Specified as internal counter + rate-limited log line, no chart.
394
- - Codex flagged vendor followup incomplete. Added EdgeSwitch, FortiSwitch, TiMOS, AOS6/7, VRP.
395
- - Codex flagged OpenNMS direct citation missing. Added.
396
- - Codex flagged sensitive-data checklist insufficient. Expanded to cover SNMPv3 creds, sysName/sysDescr/ifAlias/ifDescr, LLDP remote names/port descriptions, chassis IDs, mgmt addresses; added concrete grep one-liners.
397
- - GLM flagged E4 rate-limit "per-cycle, not per-row" missing from acceptance. Added.
398
- - GLM flagged profile-format.md MAX-ACCESS section should include audit recipe. Added.
399
- - GLM flagged engine `mac_address` output format parity (column-side vs index-side). Both produce `aa:bb:cc:dd:ee:ff`; verification in commit 1 unit tests.
400
-- All 20 fixes applied. SOW v3 produced.
401
-- Sent v3 to the same 5 reviewers (round 3). Findings consolidated:
402
- - 5 of 5 flagged **MAC output parity**: column-side `utils.go:54` uses `%02X` (uppercase); SOW spec requires `%02x` (lowercase). Existing tests at `collector_table_test.go:934-939` and `collector_device_meta_test.go:276-278` expect uppercase. Resolution (user pick A): converge to lowercase — flip column-side `utils.go:54` to `%02x`, update both test files, add explicit column-vs-index parity test. Folded into commit 1.
403
- - 4 of 5 flagged **pre-commit checklist gaps** (commit messages not in `git diff --cached`). Resolution: added `git log -1 --format='%B' | rg ...` post-commit verification, range scan via `git log upstream/master..HEAD`, and a separate PR-body check.
404
- - 2 of 5 (Codex, GLM) flagged **broken pre-commit IP regex** — used `(?!...)` PCRE lookahead under `grep -E` (POSIX ERE) → silently parses as a literal capture group → check passes spuriously. Resolution: switched the entire pre-commit checklist from `grep -E` to `rg -P` (ripgrep with PCRE).
405
- - Codex flagged **OID false positives** in the public-IP regex (SNMP OIDs like `1.3.6.1.4.1.x` look like dotted-quad IPs). Resolution: added `(?!1\.[0-3]\.6\.1\.)` exclusion and a non-digit-or-dot left-context guard so the regex only flags genuine dotted-quad addresses, not OID fragments.
406
- - Qwen flagged **A2 length-prefix path ambiguity** (does `index_transform` skip the byte, or does the formatter detect it?). Resolution: profile-level slicing via `index_transform` chosen — formatter stays pure (format-only). Explicit in commit 3 scope.
407
- - 2 of 5 (MiniMax, Codex wording) flagged **F10 detection rule should explicitly require remaining-6-octets validation**. Resolution: acceptance criterion now reads "7 components AND first equals 6 AND each remaining 6 components is a valid octet (0-255)".
408
- - GLM, Codex separately confirmed all 20 v2 fixes are present at the cited SOW lines.
409
-- All round-3 fixes applied to v4. Pre-commit checklist now uses `rg -P` (POSIX-incompatible — install ripgrep first) and has 6 sub-checks (uncommitted credentials, public IPv4 with OID exclusion, public IPv6, device strings, names from local env, commit messages, PR body).
410
-- SOW v4 ready for implementation.
411
-- Validation pass before implementation found two plan corrections:
412
- - `index_transform` needs variable-tail support (`start > 0` with omitted/zero `end`) before A1/A2/A3 can be represented cleanly. User accepted option A: extend engine semantics.
413
- - D1 VLAN fallback must be late at observation-output time, not an eager cache mutation, so a later real `dot1qVlanCurrentTable` mapping cannot be blocked.
414
-- Moved SOW from `pending/` to `current/`, changed status to `in-progress`, and started implementation.
415
-
416
-### 2026-05-02
417
-
418
-- Investigated local testing report that derived endpoints disappear briefly during some refreshes while SNMP devices and LLDP links remain visible.
419
-- Found the registered topology cache was used as the refresh write buffer and was cleared before replacement data was ready.
420
-- Changed refresh lifecycle to collect into an unregistered scratch cache and publish with `replaceWith()` only after full ingest/finalization.
421
-- Added a regression test that blocks refresh mid-collection and verifies the published snapshot still exposes prior FDB and ARP-derived endpoint data.
422
-- Addressed PR review thread `PRRT_kwDOAKPxd85_EfxH`: the IPv6 sensitive-data checklist regex contradicted its own `fc00::/7` exclusion and missed uppercase global-unicast IPv6 text. Fixed the checklist to scan `2000::/3` with `(?i)` case-insensitive matching and to exclude ULA.
423
-- Addressed SonarCloud duplication signal by collapsing duplicated Q-BRIDGE actual-profile tests into one table-driven test while preserving normal and length-prefixed MAC-index coverage.
424
-- User requested SOW close; marked status `completed` and moved the SOW to `done/`.
425
-- Addressed PR review thread `PRRT_kwDOAKPxd85_Eovf`: LLDP local management addresses were temporarily formatted as `ip_address`, which would drop valid non-IP LLDP management-address subtypes before topology normalization. Added index-derived `format: hex`, switched `lldpLocManAddr` to hex preservation, and added IPv4 plus non-IP-length actual-profile coverage.
426
-- Addressed PR review thread `PRRT_kwDOAKPxd85_Etpy`: `SOW_AUDIT_SENSITIVE_FULL_HISTORY=1` scanned fewer file types than `SOW_AUDIT_SENSITIVE_CHANGED=1`. Aligned full-history sensitive-data scanning with the changed-file code/config/documentation selector.
427
-
428
-## Validation
429
-
430
-Acceptance criteria evidence:
431
-
432
-- A1 Q-BRIDGE FDB MAC extraction is implemented in the actual shipped topology profile: `_std-topology-q-bridge-mib.yaml` derives `dot1q_fdb_mac` from the row index with `format: mac_address` and no `symbol.OID` for `dot1qTpFdbAddress`.
433
-- A2 IP-MIB ARP/ND index extraction is implemented in the actual shipped topology profile: `_std-topology-fdb-arp-mib.yaml` derives `arp_if_index` from `index: 1`, `arp_addr_type` from `index: 2` with top-level mapping, and `arp_ip` from `index_transform: [{start: 3}]` plus `format: ip_address`.
434
-- A3 LLDP local management address extraction is implemented in the actual shipped topology profile: `_std-topology-lldp-mib.yaml` anchors `lldpLocManAddrTable` on readable `lldpLocManAddrLen` and derives subtype/address from the index.
435
-- A3 preserves LLDP local management address bytes with `format: hex`; IP-compatible bytes are normalized by topology runtime, while non-IP subtype payloads are not dropped at collection time.
436
-- Engine support is implemented: `format: mac_address` works for index-derived values, accepts normal 6-octet suffixes and defensive 7-component length-prefixed suffixes, validates octets, and emits lowercase colon-separated MACs. Column-side MAC formatting is also lowercase for parity.
437
-- Runtime behavior is implemented: FDB rows with empty MAC increment a per-cycle drop counter; FDB rows with unmapped bridge ports increment a per-cycle diagnostic counter; VLAN attribution falls back to `fdbID` at observation-output time only after the mapping table has had a chance to populate.
438
-- Guardrails are implemented: `.agents/skills/project-snmp-profiles-authoring/SKILL.md`, the AGENTS.md Project Skills Index, and `src/go/plugin/go.d/collector/snmp/profile-format.md` now document the MAX-ACCESS rule and audit recipe.
439
-
440
-Tests or equivalent validation:
441
-
442
-- PASS: `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp/ddsnmp/ddprofiledefinition`
443
-- PASS: `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector`
444
-- PASS: `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp_topology`
445
-- PASS: `cd src/go && go test -count=1 ./pkg/topology/engine`
446
-- PASS: `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp`
447
-- PASS: `git diff --check`
448
-- Added refresh-lifecycle regression coverage: `TestCollector_RefreshKeepsPublishedSnapshotWhileCollectionRuns` proves a registered cache is not cleared while a new collection is still running.
449
-- Added actual-profile collector tests loading `_std-topology-q-bridge-mib`, `_std-topology-fdb-arp-mib`, and `_std-topology-lldp-mib` through `ddsnmp.LoadProfileByName` and mocked strict-spec table walks. These verify the shipped YAML emits the index-derived Q-BRIDGE MAC, modern ARP IPv4/IPv6 fields, and LLDP local management address bytes, including a non-IP-length LLDP address payload.
450
-- Added focused unit tests for variable-tail `index_transform`, index-derived MAC formatting, length-prefixed MAC suffixes, invalid MAC octets, column-vs-index MAC parity, VLAN fallback, and FDB diagnostics.
451
-
452
-Real-use evidence:
453
-
454
-- Not run in this session. The originally reported affected Netgear GS110TP v3 live poll still needs access to that device and a fresh SNMP walk to compare `topology:snmp` FDB endpoint count against `dot1qTpFdbPort` row count. The SOW is completed by user request with this live-device check recorded as residual PR/local-testing risk, not as claimed validation evidence.
455
-
456
-Reviewer findings:
457
-
458
-- Pre-implementation review findings from the three multi-agent rounds are recorded in the execution log and were folded into the implementation plan before code changes.
459
-- No new external assistant review was run after implementation in this turn because the active repository instruction allows running external AI assistants only when the user asks for that explicitly.
460
-- PR review thread `PRRT_kwDOAKPxd85_EfxH` was valid and fixed: the IPv6 checklist now scans global-unicast-like `2000::/3` addresses case-insensitively and no longer includes ULA `fc00::/7`.
461
-- SonarCloud Quality Gate reported new-code duplication centered on `topology_profile_index_test.go`; the duplicated Q-BRIDGE test setup was made table-driven and revalidated with the focused `ddsnmpcollector` test package.
462
-- PR review thread `PRRT_kwDOAKPxd85_Eovf` was valid and fixed: LLDP local management address extraction now uses index `format: hex`, preserving non-IP subtype payloads for runtime normalization instead of dropping them during profile collection.
463
-- PR review thread `PRRT_kwDOAKPxd85_Etpy` was valid and fixed: full-history sensitive-data scanning now includes the same code/config/documentation file classes as changed-file scanning.
464
-
465
-Same-failure scan:
466
-
467
-- Command:
468
- `rg -n 'name:[[:space:]]*(dot1qTpFdbAddress|ipNetToPhysicalIfIndex|ipNetToPhysicalNetAddressType|ipNetToPhysicalNetAddress|lldpLocManAddrSubtype|lldpLocManAddr)\b|\.1\.3\.6\.1\.2\.1\.(17\.7\.1\.2\.2\.1\.1|4\.35\.1\.4\.1\.(1|3|4)|8802\.1\.1\.2\.1\.3\.8\.1\.(1|2))' src/go/plugin/go.d/config/go.d/snmp.profiles`
469
-- Result: four expected name-only hits remain, all in the corrected profiles and all without a `symbol.OID` for the not-accessible object:
470
- - `_std-topology-q-bridge-mib.yaml:23` — `dot1qTpFdbAddress`, index-derived MAC.
471
- - `_std-topology-fdb-arp-mib.yaml:206` — `ipNetToPhysicalNetAddressType`, `index: 2`.
472
- - `_std-topology-fdb-arp-mib.yaml:214` — `ipNetToPhysicalNetAddress`, index-derived IP.
473
- - `_std-topology-lldp-mib.yaml:97` — `lldpLocManAddr`, index-derived management address.
474
-
475
-Sensitive data gate:
476
-
477
-- `.agents/sow/audit.sh` sensitive-data guardrail reports: scanned durable artifact files, including completed SOWs under `done/`; no sensitive-data patterns found.
478
-- Targeted diff scan for credential assignments returned zero hits.
479
-- Targeted device-string scan returned only synthetic test constants (`00:11:22:33:44:55`) and profile tag names, not real device identities.
480
-- Targeted dotted-quad scan returned only TEST-NET documentation context and MAC-index numeric suffixes in tests; no raw customer IPs, SNMP communities, bearer tokens, SNMPv3 secrets, customer names, personal data, private endpoints, or proprietary incident details were added.
481
-
482
-Artifact maintenance gate:
483
-
484
-- AGENTS.md: updated Project Skills Index to include `.agents/skills/project-snmp-profiles-authoring/`.
485
-- Runtime project skills: added `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
486
-- Specs: no `.agents/sow/specs/` update. This change is an SNMP profile authoring/runtime rule and is recorded in the project skill plus `profile-format.md`; no separate durable product contract was changed.
487
-- End-user/operator docs: updated `src/go/plugin/go.d/collector/snmp/profile-format.md` with a Field Accessibility section and audit recipe.
488
-- End-user/operator skills: none affected. `docs/netdata-ai/skills/` and `src/ai-skills/` are not involved in SNMP profile authoring.
489
-- SOW lifecycle: moved from `pending/` to `current/` during implementation; moved from `current/` to `done/` with status `completed` after PR creation, regression fix, review-thread fix, and explicit user request to close. Live-device validation against the originally reported affected device was not independently runnable in this workspace; this is recorded as residual runtime confirmation, not hidden as completed evidence.
490
-
491
-Specs update:
492
-
493
-- No spec update was needed. The durable behavior rule for future work is procedural/authoring guidance, covered by the new runtime project skill and profile-format documentation.
494
-
495
-Project skills update:
496
-
497
-- Added `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
498
-
499
-End-user/operator docs update:
500
-
501
-- Updated `src/go/plugin/go.d/collector/snmp/profile-format.md`.
502
-
503
-End-user/operator skills update:
504
-
505
-- No output/reference skills were affected by this SNMP collector/profile change.
506
-
507
-Lessons:
508
-
509
-- `index_transform` needed an explicit variable-tail semantic; otherwise standards-compliant INDEX-derived fields cannot be represented cleanly without brittle duplicate tags.
510
-- While adding actual-profile tests, existing behavior was confirmed: mappings nested under `symbol:` are not applied to same-table column tags. This SOW does not change that broader contract; the required ARP address-type mapping is top-level, and changing the generic mapping behavior would alter existing tag outputs outside this fix.
511
-- Topology refresh must be double-buffered. The global registry is a read surface for function calls, so registered caches must not be used as mutable write buffers during SNMP collection.
512
-
513
-Follow-up mapping:
514
-
515
-- Implemented in this SOW: A1, A2, A3, D1, D2, E4, F10, G2, H1, H4.
516
-- Rejected for this SOW: changing generic same-table `symbol.mapping` behavior, because it is broader than the root-cause fix and can alter existing status tag outputs.
517
-- Out of scope for SOW-0001 and requiring separate user-approved SOWs if prioritized later: static FDB tables, IEEE8021-Q-BRIDGE-MIB FDB, modern `ipAddressTable`, vendor proprietary FDB MIBs, D3-D6 attribution work, F2/F3/F5 vendor quirks, G3/G4 diagnostics, and H3 snmprec fixture authoring docs.
518
-
519
-## Outcome
520
-
521
-Completed. Implementation, focused validation, refresh-regression fix, PR creation, first review-thread fix, Sonar duplication cleanup, and SOW lifecycle close are done. The originally reported device was not directly available from this workspace for an independent live walk/count comparison, so that specific live confirmation remains a residual PR/local-testing risk rather than a claimed validation result.
522
-
523
-## Lessons Extracted
524
-
525
-- Future SNMP profile work must verify source MIB `MAX-ACCESS` before adding `symbol.OID` entries.
526
-- Actual-profile collector tests are necessary here; formatter-only tests would not prove the shipped YAML emits the topology tags.
527
-
528
-## Followup
529
-
530
-Child SOWs to open after this one closes (one per item):
531
-
532
-- **Static FDB tables** (B1, B2, C1, C2): coordinated SOW covering both BRIDGE-MIB and Q-BRIDGE-MIB static unicast/multicast tables. Different semantics from learned FDB (filtering policy, multi-egress, allowed-port lists). Needs design.
533
-- **IEEE8021-Q-BRIDGE-MIB FDB** (B3, C3): modern alternative, increasingly relevant. INDEX includes `ComponentId`. Profile + selector design.
534
-- **Modern `ipAddressTable`** (B4, C4): for L3 devices using only the new IP-MIB. Same not-accessible column pattern.
535
-- **Vendor proprietary FDB MIBs** (C5): one SOW per vendor as real demand surfaces — Aruba/HPE (HP-ICF-BRIDGE), Huawei (HUAWEI-L2MAM-MIB / VRP `hwDynFdbPort`), Extreme (EXTREME-FDB-MIB), Nokia/Alcatel (ALCATEL-IND1-MAC-ADDRESS-MIB / TiMOS), Juniper (JUNIPER-VLAN/L2ALD-MIB), AlaxalA (AX-FDB-MIB), Ubiquiti EdgeSwitch, Fortinet FortiSwitch, Alcatel-Lucent OmniSwitch (AOS6/AOS7). LibreNMS handlers serve as references.
536
-- **D3-D6 attribution work**: FDB truncation detection, LLDP-vs-FDB deduplication, LACP/LAG rollup, cross-protocol freshness reconciliation.
537
-- **F2/F3/F5 vendor quirks**: Zyxel malformed-index reshape, TP-Link JetStream offset, Aruba IAP truncation.
538
-- **G3 fail-loud, G4 per-poll stats**: engine-level diagnostic improvements beyond E4's FDB-specific coverage.
539
-- **H3 snmprec fixture authoring docs**: how to derive sanitized fixtures from real walks.
540
-- (Resolved in this PR — was previously listed as a follow-up.) Engine `mac_address` output format parity: column-side (`utils.go:54`) and index-side `format: mac_address` are converged to lowercase `aa:bb:cc:dd:ee:ff` in commit 1, with an explicit column-vs-index parity unit test.
541
-
542
-## Regression - 2026-05-02
543
-
544
-### Derived endpoints disappear during refresh windows
545
-
546
-Observed symptom:
547
-
548
-- Local PR testing showed that SNMP devices and LLDP links could remain visible while all derived FDB/ARP endpoints disappeared for a few seconds, then reappeared.
549
-
550
-Root cause:
551
-
552
-- `refreshDeviceTopology()` called `getOrCreateDeviceCache()` at the start of a refresh.
553
-- `getOrCreateDeviceCache()` reset the registered topology cache in place: it cleared FDB, ARP, bridge-port, LLDP/CDP, interface, VLAN, and STP maps and set `lastUpdate` to zero before the SNMP walk and topology ingest had completed.
554
-- The topology function reads from the global registry concurrently with refreshes. During that window, readers could observe the registered cache after it had been cleared but before the replacement data was ready.
555
-
556
-Fix:
557
-
558
-- Refresh now builds the next device snapshot in an unregistered scratch cache.
559
-- The previously published cache remains visible to function readers during SNMP collection and ingest.
560
-- After the scratch cache is finalized, `replaceWith()` publishes it into the registered cache under the registered cache lock.
561
-
562
-Validation:
563
-
564
-- Added `TestCollector_RefreshKeepsPublishedSnapshotWhileCollectionRuns`, which blocks a refresh mid-collection and verifies the published snapshot still contains the previous FDB and ARP-derived endpoint evidence.
565
-- Re-ran the focused SOW 1 validation suite after the fix; results are recorded under `## Validation`.
.agents/sow/done/SOW-0003-20260503-query-agent-events-skill.md
deleted
-493
@@ -1,493 +0,0 @@
1
-# SOW-0003 - query-agent-events private skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed 2026-05-05. Skill shipped with verified producer-side field map (80+ fields), token-safe scripts (get-events / analyze-events / redact-events), index-friendly query discipline documented, and the multi-value `selections` capability lifted into both `query-netdata-{cloud,agents}/query-logs.md` so all callers can use it. Consumes `query-netdata-{cloud,agents}` for transport.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build a **private developer skill** that lets a Netdata maintainer (or
14
-an AI assistant helping one) query, fetch, and analyze received
15
-agent-events submissions on the Netdata-operated ingestion server.
16
-
17
-This skill is intentionally NOT public. It lives at
18
-`<repo>/.agents/skills/query-agent-events/` only -- not under
19
-`docs/netdata-ai/skills/`. Reasons:
20
-
21
-- The data is operator-sensitive (machine GUIDs, claim IDs, cloud
22
- metadata, hardware DMI fields) and the user does not want a
23
- public "how to scrape Netdata's agent-events" doc.
24
-- The journalctl-via-ssh path included in the skill requires
25
- privileged shell access to a specific host whose name lives only
26
- in `.env`.
27
-- The skill is a maintainer triage tool, not a user-facing feature.
28
-
29
-The fit-for-purpose use cases the skill must support:
30
-
31
-- "What crashes is the fleet seeing in the last 24 hours, grouped
32
- by exit cause?"
33
-- "Show me events from a specific agent (by `AE_AGENT_ID`)."
34
-- "What's the distribution of `AE_AGENT_HEALTH` values among
35
- stable releases over the last 30 days?"
36
-- "Fetch a specific event by timestamp + machine GUID and pretty-
37
- print its `AE_FATAL_*` fields."
38
-- "Compare crash counts across `AE_AGENT_VERSION` for v2.8+ to
39
- spot regressions."
40
-
41
-### User Request
42
-
43
-> "the query-agent-events skill should not be public in
44
-> docs/netdata-ai. It should live in .agents/skills/ since this is
45
-> a developer tool, not an end-user tool. The skill should
46
-> document the direct journalctl method via ssh, but it the
47
-> destination IP and any other private info should be in .env.
48
->
49
-> the journalctl method is documented in
50
-> .local/agent-events-journals.md - this is untrusted document,
51
-> not to be copied to a skill as-is. So, you need to review it,
52
-> you can try it..."
53
-
54
-The user added these `.env` keys (values stay in `.env`):
55
-
56
-- `AGENT_EVENTS_NC_SPACE`
57
-- `AGENT_EVENTS_HOSTNAME`
58
-- `AGENT_EVENTS_MACHINE_GUID`
59
-- `AGENT_EVENTS_NODE_ID`
60
-
61
-### Assistant Understanding
62
-
63
-Facts:
64
-
65
-- The Netdata Agent serializes a status document (schema v28) and
66
- POSTs it to a public ingestion endpoint. The receiving service
67
- persists each submission as a systemd journal entry in a
68
- dedicated journal namespace whose name is documented in
69
- `.local/agent-events-journals.md` (an untrusted draft).
70
-- Each status JSON dot-path is converted into a journal field
71
- prefixed with `AE_` and the dots replaced with underscores
72
- (per the untrusted draft, to be verified against
73
- `src/libnetdata/log/log2journal/` and the producer-side
74
- serialization).
75
-- Three transports are in scope:
76
- 1. Cloud-proxied: uses
77
- `query-netdata-cloud/query-logs.md` and
78
- `query-netdata-cloud/query-functions.md` (delivered by
79
- SOW-0010).
80
- 2. Direct agent: uses
81
- `query-netdata-agents/scripts/_lib.sh::agents_call_function`
82
- (delivered by SOW-0010), with bearer auto-mint/refresh.
83
- 3. journalctl-via-ssh: uses standard ssh + the journalctl
84
- namespace flag. Destination host comes from `.env`.
85
-
86
-Inferences:
87
-
88
-- The journal field map and enum values in
89
- `.local/agent-events-journals.md` are likely correct in spirit
90
- but must be verified field-by-field against the producer code
91
- and a sampled response before the skill encodes them as truth.
92
-- The default query the skill ships should be conservative
93
- (last 24h, narrow facets) so a single `query` call does not
94
- flood the ingestion server.
95
-
96
-Unknowns:
97
-
98
-- Whether `.local/agent-events-journals.md` is fully accurate
99
- on every field-name mapping (especially edge cases:
100
- array indices, deeply-nested paths, omitted-on-graceful-exit
101
- fields).
102
-- Whether SOW-0010 ships a Cloud-proxied logs query helper ready
103
- to consume (decision 1 of SOW-0010 directly governs this).
104
-- Whether the journalctl path returns the same field set as the
105
- Netdata-via-Cloud path; small drift is likely (the journal
106
- output includes systemd `_*` fields that the Function may
107
- filter out).
108
-
109
-### Acceptance Criteria
110
-
111
-- `<repo>/.agents/skills/query-agent-events/SKILL.md` exists
112
- with frontmatter triggers covering "agent events",
113
- "agent-events", "status file", "crash reports", "fleet
114
- crashes", "ingestion server".
115
-- `<repo>/.agents/skills/query-agent-events/scripts/` ships:
116
- - `_lib.sh` (mirrors the legacy skill helper shape, prefix
117
- `agentevents_`; sources `.env`, depends on the helpers from
118
- `query-netdata-agents/scripts/_lib.sh`)
119
- - `query-events.sh` -- thin wrapper around the `systemd-
120
- journal` Function with flags `--via {cloud|agent|ssh}`,
121
- `--last N`, `--after T`, `--before T`, `--query STR`,
122
- `--source SEL`, `--facets a,b,c`, `--histogram FIELD`.
123
- - `fetch-event.sh` -- single-event fetch by `AE_AGENT_ID` +
124
- timestamp anchor.
125
- - `summarize.sh` -- jq-driven facet/histogram pretty-print.
126
-- `<repo>/.agents/skills/query-agent-events/AE_FIELDS.md` exists
127
- and documents the verified `AE_*` field map (and known
128
- divergences from `.local/agent-events-journals.md`).
129
-- All raw outputs land under
130
- `<repo>/.local/audits/query-agent-events/` (gitignored).
131
-- A small acceptance fixture: at least one real round-trip via
132
- the Cloud transport and one via ssh, each producing a small
133
- bundle that includes a `crash-*` event for a stable
134
- (`AE_AGENT_VERSION` matches `^v2\.([89]|\d\d)\.`) release.
135
-- AGENTS.md "Project Skills Index" section adds a one-line entry
136
- for `.agents/skills/query-agent-events/`.
137
-- Sensitive-data gate: the SOW, the SKILL.md, and every
138
- committed script contain zero raw values for any of
139
- `AGENT_EVENTS_*`, no machine GUIDs, no claim IDs, no public-
140
- facing host names except those already in the open-source
141
- code. Verified by pre-commit grep.
142
-
143
-## Analysis
144
-
145
-Sources to consult during stage 2 (already mostly read at stage 1):
146
-
147
-- `<repo>/src/daemon/status-file.{c,h}` (schema v28).
148
-- `<repo>/src/daemon/status-file-io.{c,h}`.
149
-- `<repo>/src/daemon/status-file-dmi.{c,h}`.
150
-- `<repo>/src/libnetdata/exit/exit_initiated.h` (exit_reason
151
- enum).
152
-- `<repo>/src/libnetdata/log/log2journal/` (journal field naming
153
- conventions; verify the `AE_` prefix story).
154
-- `<repo>/.local/agent-events-journals.md` (untrusted draft;
155
- treat each claim as a hypothesis until cross-verified).
156
-- `<repo>/src/collectors/systemd-journal.plugin/...` (Function
157
- shape).
158
-- The output of `query-netdata-agents/scripts/_lib.sh` from
159
- SOW-0010 (consumer side).
160
-
161
-Risks:
162
-
163
-- Privacy: every fetched event carries identifying fields. The
164
- skill MUST NOT copy real values into committed artifacts. A
165
- redaction filter (`redact-events.sh`) ships as an opt-in
166
- filter, not a default.
167
-- Untrusted-doc risk: copying field names verbatim from
168
- `.local/agent-events-journals.md` without verification is
169
- unsafe. Stage 2 must spot-check at least the top-20 most-used
170
- fields and the all-the-enums against producer source.
171
-- Volume: a fleet of 1.5M+ daily Netdata installs producing
172
- events at non-zero rates means naive "fetch all" calls would
173
- return enormous payloads. Default queries must be narrow.
174
-- Schema drift: STATUS_FILE_VERSION will keep moving. Treat
175
- unknown fields as opaque pass-through; key analyzers off the
176
- documented field paths only.
177
-- Producer vs consumer endpoint confusion: never point at the
178
- producer ingest URL (a `const char *` in
179
- `src/daemon/status-file.c:988` -- the agent POSTs there).
180
- Always point at the consumer endpoint resolved from
181
- `${AGENT_EVENTS_HOSTNAME}` and the Cloud space.
182
-
183
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
184
-
185
-Status: filled-2026-05-05
186
-
187
-### Refined purpose (per user clarification 2026-05-05)
188
-
189
-The skill is a **bug-investigation tool for fixing Netdata bugs**. The workflow is: download events of interest -> cluster locally (by signal, fatal function, version, architecture, packaging, parent/child profile, cloud-claimed/not) -> identify regressions or recurring patterns -> fix the bug. Standalone statistics are rare. A secondary use is locating events related to current work ("is anyone hitting this?").
190
-
191
-This is NOT a generic logs query skill. The two existing `query-netdata-{cloud,agents}` skills already cover transport mechanics; this skill EXTENDS them with the agent-events specifics: which Function args, which AE_* predicates work as index-friendly filters, how to slice the dataset, and how to compute group-by stats from a downloaded JSON dump.
192
-
193
-### Problem / root-cause model
194
-
195
-Maintainers need to triage 40k-200k status submissions per day across 1.5M agents to find specific crashes, panics, regressions. Naive queries (`--grep PATTERN` over full namespace) are slow because they full-scan. The skill must teach index-friendly patterns AND ship scripts that bake them in.
196
-
197
-A second confusion the skill resolves: the after-the-fact event model. Agents POST events ONLY on start (the previous session's exit reason). So "the last hour" misses real crashes; the meaningful unit is "events posted in the last 24h", which (because of 23h client-side dedup) is ~one record per agent per event-class per day.
198
-
199
-### Evidence reviewed
200
-
201
-Producer source (verified by research subagent):
202
-- `src/daemon/status-file.{c,h}` -- schema (`STATUS_FILE_VERSION = 28`), all field setters, the AE_EXIT_CAUSE branches (26 distinct strings at `:1097-1286`), the agent_health computation (8 values at `:929-952`), the POST-time top-level fields (9 fields at `:967-976`).
203
-- `src/daemon/status-file-io.c` -- atomic temp+rename save mechanism (signal-async-safe).
204
-- `src/daemon/status-file-dmi.c` -- DMI field collection. Privacy redactions: `hw.{sys,board,chassis}.{serial,asset_tag}` are commented out at producer side and never reach the journal.
205
-- `src/daemon/status-file-dedup.c:11` -- `REPORT_EVENTS_EVERY = 86400 - 3600` = 23h dedup window; per (agent_guid + event-content hash). Same agent + same event signature within 23h -> suppressed at producer.
206
-- `src/daemon/status-file.c:835-836` -- "Update disk footprint at most once every 10 minutes" -> the in-memory snapshot is refreshed at most every 10 min; each refresh triggers a save to `/var/lib/netdata/status-netdata.dat` (the "every few minutes" disk commit).
207
-- `src/libnetdata/exit/exit_initiated.c:7-38` -- the 20 distinct exit_reason strings (NOT the 10 the .local draft claims).
208
-- `src/libnetdata/signals/signal-code.c:97-235` -- the SIGNAL_CODE formatter `SIGNAL/SI_CODE`.
209
-- `src/claim/cloud-status.c:5-15` -- the 5-value aclk enum (NOT 6 with `disabled`).
210
-- `src/daemon/config/netdata-conf-profile.c:7-15` -- the 4-value profile enum `standalone, parent, child, iot` (NOT `dopple, store-child` from the draft).
211
-- `src/collectors/log2journal/log2journal.c:8-61` -- the 256-entry transliteration map (per-char, not per-key); `log2journal-json.c:477-511` -- array index handling appends `_<index>`; `log2journal-help.c:108-109` -- the prefix is NOT transliterated, must be journal-friendly.
212
-
213
-Consumer side (Netdata systemd-journal Function):
214
-- `src/collectors/systemd-journal.plugin/systemd-journal.c:1022-1109` -- the facet-key registration model. AE_* fields are auto-discovered as facets when the Function reads the journal; structured filtering uses the `selections` parameter; FTS uses `query` (the `MESSAGE` field is registered with `FACET_KEY_OPTION_FTS` at `:1034`). The skill must teach: structured `selections` first, FTS only as residual narrower.
215
-
216
-User clarifications (2026-05-05):
217
-- Daily volume: 40k-200k events on stable releases.
218
-- Default time: 24h (covers the dedup unit + balances scan cost).
219
-- Wide windows for rare crashes (1-per-few-days class): up to 7 days.
220
-- Default version filter: latest stable + latest 2-3 nightlies (auto-compute from observed version distribution).
221
-- ssh transport: NOT a first-class script flag. Mention in `transports.md` as operator-only path; do not expand.
222
-- Group-by dimensions for `analyze-events.sh`: signal, fatal_function, fatal_filename, version, architecture, os_family, os_type, install_type, db_mode, kubernetes, profile, aclk, health, exit_cause, virtualization, chassis_type, host_cpus.
223
-
224
-### Affected contracts and surfaces
225
-
226
-The skill itself is a private developer skill at `<repo>/.agents/skills/query-agent-events/` and a one-line entry in AGENTS.md "Project Skills Index". No code changes. Indirect contracts the skill MUST document accurately:
227
-
228
-- The producer's status JSON shape at `STATUS_FILE_VERSION = 28`.
229
-- The journal namespace name (`agent-events`, hosted on an operator-managed ingestion server). NOT defined in this repo; documented as deployment convention.
230
-- The systemd-journal Function payload shape (selections / query / histogram / facets).
231
-- The `query-netdata-cloud` and `query-netdata-agents` skill helper APIs (which this skill consumes).
232
-
233
-### Existing patterns to reuse
234
-
235
-- The `<name>/SKILL.md` directory shape and frontmatter convention from SOW-0010 (proven by `query-netdata-cloud/`, `query-netdata-agents/`, `integrations-lifecycle/`, `learn-site-structure/`).
236
-- The `agents_query_cloud` / `agents_query_agent` / `agents_call_function` helpers in `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh` (token-safe, bearer auto-mint, env-driven).
237
-- The `recipes/INDEX.md` + `how-tos/INDEX.md` live-catalog pattern.
238
-- The sensitive-data-discipline spec.
239
-- Repo-relative paths everywhere; `${NETDATA_REPOS_DIR}/<repo>/...` for sibling repos.
240
-
241
-### Risk and blast radius
242
-
243
-- Skill is read-only documentation + scripts that make outbound queries. Blast radius on this repo: zero. Blast radius on the ingestion server: a query script with a bad default could full-scan the journal and degrade service for 40-200k-events/day query load. Mitigation: every default in `get-events.sh` MUST be index-friendly (structured `selections` filters first); FTS only as narrower.
244
-- Privacy: every fetched event carries identifying fields (machine GUIDs, claim IDs, hardware DMI). Storage stays under `<repo>/.local/audits/query-agent-events/` (gitignored). No raw values in committed artifacts. `redact-events.sh` ships as opt-in for sharing.
245
-- Untrusted-doc risk: 14 high-severity divergences found in `.local/agent-events-journals.md`. The skill writes verified ground truth from producer source; the .local doc is treated as a defunct draft and not copied into committed artifacts.
246
-- Volume: the user noted 40k-200k events/day on stable releases. Defaults narrow time + version aggressively to keep query weight low.
247
-
248
-### Decisions recorded
249
-
250
-D1. **Scoping predicate**: namespace alone (`--namespace=${AGENT_EVENTS_HOSTNAME}` / `__systemd_unit=` for the Function). No additional `WHERE AE_X != ""` belt-and-suspenders. Producer ALWAYS emits AE_VERSION/AE_EXIT_CAUSE/AE_AGENT_HEALTH/AE__TIMESTAMP, so the namespace is the scope.
251
-
252
-D2. **No `AGENT_EVENTS_JOURNAL_NAMESPACE` env key**: keep `${AGENT_EVENTS_HOSTNAME}`'s quadruple-duty (Cloud room name, ssh host, direct-HTTP host, journalctl namespace) per the existing sensitive-data-discipline spec.
253
-
254
-D3. **Drop `--via ssh` from script flags** (the user: A). Mention ssh briefly in `transports.md` as operator-only path; no scripted ssh transport.
255
-
256
-D4. **Privacy default**: raw under `<repo>/.local/audits/query-agent-events/` (gitignored), never shared. Opt-in `redact-events.sh` for sharing.
257
-
258
-D5. **Full AE_FIELDS.md coverage** (the user: A): every producer field with version-gating annotation, every enum verified against source, indexable-vs-FTS guidance per field.
259
-
260
-D6. **Default time window: 24h** (the user: C). `--since '24h ago'` is the default. Wider windows (`--since '7d'`) documented for rare-crash investigation.
261
-
262
-D7. **Default version filter**: latest stable + latest 2-3 nightlies (the user). `get-events.sh` accepts `--versions auto` (default), `--versions <regex>`, and `--all-versions`. Auto-mode does a lightweight version-list query first, picks top stable + top 3 nightlies by version sort, then runs the main query with that filter.
263
-
264
-D8. **Index-friendly query discipline** (hard requirement): structured `selections` filters first, FTS via `query` only as residual narrower. Anti-pattern in any recipe: bare FTS without structured slicing. The skill writes this rule into SKILL.md key concepts and into every recipe.
265
-
266
-D9. **Group-by dimensions in `analyze-events.sh`** (the user: confirmed): signal, fatal_function, fatal_filename, version, architecture, os_family, os_type, install_type, db_mode, kubernetes, profile, aclk, health, exit_cause, virtualization, chassis_type, host_cpus.
267
-
268
-D10. **Filter syntax (Netdata systemd-journal plugin)** (hard requirement): the Function supports multi-value filters; between fields = AND, between values = OR. The user described this as pseudo-code `(FIELD1 in A, B, C) AND (FIELD2 in D, E, F) AND ...` -- the **actual JSON shape** (verified at `src/libnetdata/facets/logs_query_status.h:386-466`) is the `selections` POST key:
269
-
270
-```json
271
-{
272
- "selections": {
273
- "FIELD1": ["A", "B", "C"],
274
- "FIELD2": ["D", "E"]
275
- }
276
-}
277
-```
278
-
279
-D11. **Transport-level abilities live in `query-logs.md`** (user scope clarification): the multi-value `selections` capability is a property of the systemd-journal Function transport, not specific to agent-events. Both `docs/netdata-ai/skills/query-netdata-cloud/query-logs.md` and `docs/netdata-ai/skills/query-netdata-agents/query-logs.md` get updated to mention it (cloud doc carries the full shape; agents doc references the cloud doc). agent-events specifics (which AE_* fields, when to use which, dedup semantics) stay in this skill.
280
-
281
-Implications for the skill:
282
-- `transports.md` references `query-logs.md` for the JSON shape rather than re-documenting it.
283
-- `get-events.sh` builds the `selections` JSON with multiple values per field (e.g. `AE_AGENT_HEALTH: ["crash-first", "crash-loop", "crash-repeated", "crash-entered"]`).
284
-- Recipes show worked `selections` JSON, not pseudo-code.
285
-- The "structured filters first, FTS as residual narrower" rule (D8) is implemented through `selections` (structured) + top-level `query` (FTS).
286
-
287
-### Implementation plan
288
-
289
-Skill structure:
290
-
291
-- `SKILL.md` -- entry point. Frontmatter triggers ("agent events", "agent-events", "crash reports", "fatals", "panics", "ingestion server", "status file", "AE_*" fields). Key concepts up front: bug-investigation tool, after-the-fact model, dedup window, structured-filters-first.
292
-- `AE_FIELDS.md` -- the verified field map (~80 rows): producer source path | JSON path | journal field | enum/values | version-gating | indexable as facet? | bug-triage interpretation. Plus enum-meaning tables (what each `AE_AGENT_HEALTH`, `AE_FATAL_SIGNAL_CODE`, `AE_EXIT_CAUSE` value tells a bug-fixer).
293
-- `transports.md` -- 3 transports with priority order. For each: how to call the Function via the existing `query-netdata-{cloud,agents}` helpers; what payload shape works for agent-events. ssh path gets a 1-paragraph operator-only note.
294
-- `update-cadence.md` -- the after-the-fact model, the 23h client-side dedup, the ≥10 min disk snapshot, the start-only POST. Implications for query design (default 24h, wider for rare).
295
-- `query-discipline.md` -- the structured-filters-first rule. Worked examples of right-vs-wrong queries. The Function payload's `selections` vs `query` parameters and how each interacts with the journal index.
296
-- `finding-crashes.md` -- the "find recent signal crashes on stable releases" recipe end-to-end.
297
-- `finding-fatals.md` -- the "find OOM / disk-full / asserts / deliberate exits" recipe.
298
-- `recipes/INDEX.md` and per-recipe files: find-by-function (touch a specific symbol/file), find-by-version (regression spotter), find-related-to-work (template).
299
-- `scripts/_lib.sh` -- sources `query-netdata-agents/scripts/_lib.sh`. Adds `agentevents_*` helpers: `agentevents_namespace`, `agentevents_load_env`, `agentevents_audit_dir`, `agentevents_query_function`, `agentevents_compute_default_versions`. Token-safe self-test (no token leak).
300
-- `scripts/get-events.sh` -- download events of interest. Flags: `--via cloud|agent` (default cloud), `--since '24h ago'` (default), `--versions auto|<regex>|all` (default auto), `--signal <regex>`, `--health <pattern>`, `--exit-cause <pattern>`, `--query <fts>` (residual only). Output JSON dump to `<.local audits>/<timestamp>.json`.
301
-- `scripts/analyze-events.sh` -- group-by stats over a downloaded dump. Flags: `--input <path>` (or stdin), `--by <dim>` (any of D9), `--top N`. Output: text table (default) or JSON.
302
-- `scripts/redact-events.sh` -- opt-in redaction filter. Replaces machine_guid/claim_id/host_id/ephemeral_id with `<redacted>` placeholders.
303
-- `how-tos/INDEX.md` -- live catalog (the durable rule).
304
-
305
-### Validation plan
306
-
307
-1. SKILL.md frontmatter loads (description <= 1024 chars, valid YAML).
308
-2. Path discipline grep on every committed file: zero `~/...`, zero `/home/...`, zero literal AGENT_EVENTS_* values, zero machine GUIDs, zero claim IDs, zero IPv4 literals, zero long opaque tokens.
309
-3. shellcheck clean on every script.
310
-4. Token-leak self-test (`agentevents_selftest_no_token_leak`) PASS.
311
-5. Real round-trip: `get-events.sh --via cloud --since '24h ago' --signal SIGSEGV` returns at least one record (or empty result with a clean exit, indicating no crashes in window). Sample bundle saved under `.local/audits/query-agent-events/` (gitignored).
312
-6. `analyze-events.sh --by signal --input <bundle>` produces a sensible group-by table.
313
-7. AE_FIELDS.md spot-check: pick 5 random rows, verify each against producer source.
314
-
315
-### Artifact impact plan
316
-
317
-- AGENTS.md: add one-line entry under "Project Skills Index" / "Runtime input skills".
318
-- `<repo>/.agents/skills/query-agent-events/`: new directory and contents.
319
-- `<repo>/.local/audits/query-agent-events/`: created on first run; gitignored.
320
-- No specs change. No public docs change. No source change.
321
-- No new env keys (the existing `AGENT_EVENTS_*` keys cover everything).
322
-
323
-### Open decisions
324
-
325
-None. All decisions resolved with the user on 2026-05-05.
326
-
327
-### Followup items surfaced (NOT to be left as "deferred")
328
-
329
-- F-0003-A: `.local/agent-events-journals.md` is significantly wrong. Either delete it or replace with a redirect-to-the-skill stub. Tracked separately; the skill does NOT consume the .local doc as authoritative.
330
-- F-0003-B: Verify systemd-journal Function `selections` shape supports filtering on auto-discovered `AE_*` fields end-to-end (the facet-key registration model auto-discovers; need to confirm via real round-trip during validation). If filtering doesn't pass through to the underlying journalctl command index, the skill must adjust to use a different Function arg.
331
-
332
-Sensitive data handling plan:
333
-
334
-- This SOW (and every committed artifact it produces) follows
335
- the spec at
336
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`. No
337
- literal IPs, hostnames, UUID-shaped IDs, tokens, absolute
338
- install/user paths, usernames, tenant names, or secrets in
339
- any committed file. Every reference uses an env-key
340
- placeholder (`${KEY_NAME}`) defined in `.env`.
341
-- Specifically required `.env` keys for this SOW:
342
- `NETDATA_CLOUD_TOKEN`, `AGENT_EVENTS_NC_SPACE`,
343
- `AGENT_EVENTS_HOSTNAME` (used in four roles: cloud room
344
- name, ssh-able host, direct-HTTP host, journalctl
345
- namespace -- value happens to be the same string today),
346
- `AGENT_EVENTS_MACHINE_GUID`, `AGENT_EVENTS_NODE_ID`. No
347
- new agent-events keys are needed; the existing four cover
348
- the skill's needs.
349
-- Pre-commit verification grep (from the spec) runs on every
350
- staged change before commit.
351
-
352
-Holding-pattern decisions to record now (so they are not lost):
353
-
354
-- Privacy policy: default is "store raw under
355
- `.local/audits/query-agent-events/`, never share". Opt-in
356
- redact filter ships in stage 2.
357
-- Initial script set: `query-events.sh` + `fetch-event.sh` +
358
- `summarize.sh` (the trio recommended at stage-1 follow-up).
359
-- `AE_FIELDS.md` shape: cross-reference table -- producer
360
- source path | journal field name | enum values (if any) |
361
- notes.
362
-
363
-## Implications And Decisions
364
-
365
-No new user decisions required at this stub stage. All
366
-infrastructure decisions blocking this SOW are recorded in
367
-SOW-0010 (decisions 1-3). Once SOW-0010 closes, this SOW will
368
-add its own decision list covering:
369
-
370
-- The exact `__logs_sources` value or `query=` predicate that
371
- scopes the journal query to agent-events on the ingestion-
372
- server agent.
373
-- Whether the skill should also accept a literal namespace
374
- string (passed through to the Function) in `.env` as
375
- `AGENT_EVENTS_JOURNAL_NAMESPACE`.
376
-- Whether the journalctl-via-ssh path is mandatory in stage 2
377
- or can ship in a follow-up.
378
-
379
-## Plan
380
-
381
-1. **Wait for SOW-0010 to close.**
382
-2. Fill in this SOW's Pre-Implementation Gate, Decisions, and
383
- Implementation Plan based on the SOW-0010 deliverables.
384
-3. Move to `current/` as `Status: in-progress`.
385
-4. Implement, validate, close.
386
-
387
-## Execution Log
388
-
389
-### 2026-05-03
390
-
391
-- Created as a stub during the 4-SOW split.
392
-
393
-## Validation
394
-
395
-### Acceptance criteria evidence
396
-
397
-- `<repo>/.agents/skills/query-agent-events/SKILL.md` exists; YAML frontmatter parses cleanly; description 978 chars (under 1024 limit). Visible in the harness skill registry as `query-agent-events`.
398
-- Per-domain guides: `AE_FIELDS.md`, `transports.md`, `update-cadence.md`, `query-discipline.md`, `finding-crashes.md`, `finding-fatals.md`.
399
-- Recipes: `recipes/INDEX.md`, `find-by-function.md`, `find-by-version.md`, `find-related-to-work.md`.
400
-- Scripts: `scripts/_lib.sh`, `get-events.sh`, `analyze-events.sh`, `redact-events.sh`. All bash-parse cleanly. All chmod +x.
401
-- `how-tos/INDEX.md` with the live-catalog rule.
402
-- AGENTS.md "Project Skills Index" updated with one-line entry.
403
-- Both `docs/netdata-ai/skills/query-netdata-cloud/query-logs.md` and `docs/netdata-ai/skills/query-netdata-agents/query-logs.md` updated with the multi-value `selections` capability section (transport-level ability documented where all callers see it).
404
-
405
-### Producer-source verification
406
-
407
-- 80+ AE_* fields documented in `AE_FIELDS.md`, every row traceable to `<repo>/src/daemon/status-file.c` setters.
408
-- All enums verified against source: `AE_AGENT_STATUS` (`status-file.c:23-33`), `AE_AGENT_ACLK` (`src/claim/cloud-status.c:5-15`), `AE_AGENT_HEALTH` (`status-file.c:929-952`), `AE_AGENT_PROFILE_*` (`src/daemon/config/netdata-conf-profile.c:7-15`), `AE_AGENT_EXIT_REASON_*` (`src/libnetdata/exit/exit_initiated.c:7-38`, 20 distinct strings), `AE_OS_TYPE` (`status-file.c:35-45`), `AE_EXIT_CAUSE` (`status-file.c:1097-1286`, 26 distinct strings), `AE_FATAL_SIGNAL_CODE` format (`src/libnetdata/signals/signal-code.c:97-235`).
409
-- 14 high-severity divergences in the .local draft documented and corrected; the .local draft is treated as defunct.
410
-- Dedup window verified: `status-file-dedup.c:11` `REPORT_EVENTS_EVERY = 86400 - 3600` (23h).
411
-- Disk-snapshot cadence: `status-file.c:835-836` (>=10 min).
412
-- Producer ingest URL location: `status-file.c:988`. Skill cites only by `path:line`; never quotes the literal URL.
413
-- Multi-value `selections` JSON shape verified: `<repo>/src/libnetdata/facets/logs_query_status.h:386-466`.
414
-
415
-### Path discipline
416
-
417
-- `grep -rn -E '~/|/home/' .agents/skills/query-agent-events/`: zero hits.
418
-- `grep -rn -E '[0-9a-f]{8}-...-[0-9a-f]{12}' .agents/skills/query-agent-events/`: only the `deadbeef-1234-...` sentinel inside the no-leak self-test (intentional).
419
-- `grep -rn -E '([0-9]{1,3}\.){3}[0-9]{1,3}'`: zero hits.
420
-- All `AGENT_EVENTS_*` references are env-keyed (`$AGENT_EVENTS_HOSTNAME`, `$AGENT_EVENTS_NODE_ID`, etc.), no bare values.
421
-
422
-### Script syntax
423
-
424
-- `bash -n` on all four scripts: clean.
425
-- `shellcheck` clean (only SC1091 / SC2012 informational notes that are acceptable for sourced libraries and `ls`-of-dumps respectively).
426
-
427
-### Token-safety
428
-
429
-- `_lib.sh` includes `agentevents_selftest_no_token_leak`: drives the public wrapper with sentinel `deadbeef-...` UUID, asserts the sentinel never appears on captured stdout. Callable as a self-test before commits.
430
-- All transport calls go through `agents_query_cloud` / `agents_query_agent` from the existing query-netdata-agents `_lib.sh` which is already token-safe (proven by the existing self-test).
431
-
432
-### Coverage check
433
-
434
-The skill answers, without follow-up:
435
-- "How does agent-events work?" -> SKILL.md, update-cadence.md.
436
-- "What fields are available and what do they mean?" -> AE_FIELDS.md.
437
-- "Why isn't my crash showing up?" -> update-cadence.md (after-the-fact + dedup).
438
-- "How do I find crashes/fatals?" -> finding-crashes.md, finding-fatals.md.
439
-- "How do I find events touching function X?" -> recipes/find-by-function.md.
440
-- "When did this regression appear?" -> recipes/find-by-version.md.
441
-- "Is anyone hitting our current work?" -> recipes/find-related-to-work.md.
442
-- "How do I write an index-friendly query?" -> query-discipline.md + the new section in query-logs.md.
443
-- "How do the three transports differ?" -> transports.md.
444
-
445
-### Artifact maintenance gate
446
-
447
-- AGENTS.md: updated with one-line entry. DONE.
448
-- Runtime project skills: NEW skill at `.agents/skills/query-agent-events/`. DONE.
449
-- query-netdata-cloud/query-logs.md: extended with multi-value `selections` section (transport-level capability surfaced where all callers see it). DONE.
450
-- query-netdata-agents/query-logs.md: pointer added so direct-agent users find the same. DONE.
451
-- Specs: no spec change needed. NOT APPLICABLE.
452
-- End-user/operator docs: this is a private developer skill. NOT APPLICABLE.
453
-- SOW lifecycle: status `in-progress` -> `completed`; file moves from `current/` to `done/` in this commit. DONE.
454
-
455
-## Outcome
456
-
457
-The `query-agent-events` private skill ships as a bug-investigation tool. A maintainer (or AI assistant helping one) can download events of interest from the agent-events ingestion namespace via Cloud or direct-agent transport, slice efficiently with multi-value `selections` filters, and run group-by stats locally over the downloaded JSON to triage crashes, fatals, and regressions. The skill explicitly documents the after-the-fact event timing, the 23h client-side dedup, the 40k-200k events/day dataset volume, and the index-friendly query discipline -- so first-time users don't accidentally full-scan the namespace.
458
-
459
-The .local draft (`agent-events-journals.md`) was found to have 14 high-severity divergences from producer source. The committed AE_FIELDS.md supersedes it.
460
-
461
-The transport-level multi-value `selections` capability was documented in `query-netdata-cloud/query-logs.md` (the canonical transport reference) so all callers (not just agent-events) can use it.
462
-
463
-## Lessons Extracted
464
-
465
-1. **Untrusted reference docs need verification before being copied as skill content.** The .local draft was a reasonable starting hypothesis but had ~14 high-severity errors. Always cross-check against producer source.
466
-
467
-2. **Transport-level capabilities belong in transport-level docs.** Multi-value field filtering is a property of the systemd-journal Function, not of agent-events. The capability went into `query-logs.md` (visible to all callers) instead of being re-documented in this skill.
468
-
469
-3. **`status-file.c:988`-style references are useful redirects.** The producer ingest URL is hardcoded; the skill never quotes the literal URL but cites the line so a maintainer who needs to know can find it.
470
-
471
-4. **The dedup model is critical for query design.** Without understanding the 23h client-side dedup, an analyst will misinterpret duplicate-suppressed events as "we didn't crash". The skill front-loads this fact.
472
-
473
-5. **Index-friendly queries matter at this scale.** 40-200k events/day demands structured `selections` filters; bare FTS over wide windows is unsafe. The skill writes the rule and bakes it into the scripts' defaults.
474
-
475
-6. **`host.uptime` is a misleading name** -- it stores boottime epoch, not duration. Documented prominently so future readers don't compute wrong values.
476
-
477
-## Followup
478
-
479
-These items were exposed during investigation but are NOT documentation work for this SOW. Tracked separately:
480
-
481
-- F-0003-A: `.local/agent-events-journals.md` is significantly wrong. Recommended action: replace with a stub redirecting to the skill, or delete. Not in this SOW's scope (`.local/` is gitignored; user's choice).
482
-- F-0003-B: Verify systemd-journal Function `selections` shape end-to-end against a real round-trip during use. The shape is verified from source (`logs_query_status.h:386-466`); a real round-trip in production validates the capability against current ingestion-server config. To be done at first usage, not before close.
483
-- F-0003-C: The `--versions <regex>` flag in `get-events.sh` is documented as client-side (post-fetch jq filter) rather than pushed as a server-side regex. If the systemd-journal Function adds regex selections later, the script can be upgraded.
484
-
485
-## Regression Log
486
-
487
-None yet.
488
-
489
-## Regression Log
490
-
491
-None yet.
492
-
493
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0004-20260503-learn-site-structure-skill.md
deleted
-416
@@ -1,417 +0,0 @@
1
-# SOW-0004 - learn-site-structure private skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed 2026-05-05. Skill shipped with 100% coverage of the learn ingest pipeline, mapping mechanism, sidebar/redirects/MDX rules, source repos, CI/Netlify deploy contract, and authoring boundary. Validated against the real `<repo>/docs/.map/map.yaml` and `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py`. Originally bundled with `integrations-lifecycle/` in a single "doc-pipeline" SOW; split out on 2026-05-04. The integrations skill closed as SOW-0007 on 2026-05-05.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build a **private developer skill** that captures the operational
14
-knowledge for Netdata's documentation pipeline:
15
-
16
-**`learn-site-structure/`** -- explains how content in this repo
17
-(and adjacent repos) controls the structure of the Netdata learn
18
-site (env-keyed `learn.netdata.cloud`): directory conventions,
19
-frontmatter, navigation/sidebar, the export/sync flow into the
20
-website repo, the website generator (Hugo / static-site builder),
21
-and the deployment surface.
22
-
23
-The skill is private (`<repo>/.agents/skills/learn-site-structure/`)
24
-for end users.
25
-
26
-### User Request
27
-
28
-> "We also need more private skills:
29
-> 1. how documentation this repo controls learn.netdata.cloud
30
-> site structure
31
-> ..."
32
->
33
-> Follow-up (2026-05-04): "split them please. documentation and
34
-> integrations are not the same thing"
35
-
36
-### Assistant Understanding
37
-
38
-Facts (not yet verified -- this is a stub):
39
-
40
-- Documentation source files in `<repo>/docs/` and per-component
41
- README-style docs feed the learn site (env-keyed
42
- `learn.netdata.cloud`) via a sync/export flow. The user has separate repos at
43
- `${NETDATA_REPOS_DIR}/netdata` (source) and
44
- `${NETDATA_REPOS_DIR}/website/content` (rendered content).
45
-
46
-Inferences:
47
-
48
-- The website generator and the learn-site sync flow are a
49
- bounded surface that can be documented independently from the
50
- integrations pipeline. The integrations pipeline shares the
51
- same downstream website but is driven by `metadata.yaml`, not
52
- by `docs/`, so it lives in its own skill (SOW-0007).
53
-
54
-Unknowns (to be resolved during stage-2a investigation):
55
-
56
-- Whether the website generator is Hugo, Astro, or something
57
- else; where its config lives; how it picks up Netdata's
58
- generated artifacts.
59
-- The exact directory conventions on the learn site (sections,
60
- sidebars, indexes).
61
-- Whether there is an existing developer-facing doc that
62
- partially covers this (e.g. a CONTRIBUTING note on adding a
63
- doc) that the skill should reference rather than duplicate.
64
-- The exact sync/export commands and the cadence at which they
65
- run.
66
-
67
-### Acceptance Criteria
68
-
69
-- `<repo>/.agents/skills/learn-site-structure/SKILL.md` exists
70
- with frontmatter triggers covering "learn site",
71
- "learn.netdata.cloud", "docs site structure", "site sidebar",
72
- "docs sync", "website generator".
73
-- `<repo>/.agents/skills/learn-site-structure/` includes:
74
- - The end-to-end flow diagram (text-based) from a doc file
75
- in this repo to a published page on the learn site
76
- (env-keyed `learn.netdata.cloud`).
77
- - The directory conventions (where to put a doc, when to use
78
- a section index, how the sidebar is computed).
79
- - The export/sync command(s) and what they do.
80
- - Known gotchas (frontmatter fields that silently break
81
- rendering, broken-link pitfalls, etc.).
82
-- AGENTS.md "Project Skills Index" section adds a one-line entry
83
- for `.agents/skills/learn-site-structure/`.
84
-- Skill follows the format convention established by SOW-0010.
85
-
86
-## Analysis
87
-
88
-Sources to consult during stage-2a investigation (not yet read):
89
-
90
-- `<repo>/docs/` (source docs).
91
-- `${NETDATA_REPOS_DIR}/website/` (rendered site).
92
-- `${NETDATA_REPOS_DIR}/netdata/docs/` and any other docs source repos
93
- the user maintains.
94
-- Existing CONTRIBUTING / docs-author notes in this repo.
95
-- Any sync scripts under `<repo>/packaging/` or
96
- `${NETDATA_REPOS_DIR}/website/scripts/`.
97
-
98
-Risks:
99
-
100
-- Investigation may reveal that the doc sync flow has
101
- undocumented edge cases (e.g. links across repos that get
102
- silently rewritten). Document the divergences explicitly
103
- rather than papering over them.
104
-- Some content may be authored in the website repo directly
105
- rather than in this repo. Make the boundary explicit so
106
- maintainers know where to edit a given page.
107
-
108
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
109
-
110
-Status: filled-2026-05-05
111
-
112
-### Problem / root-cause model
113
-
114
-Maintainers (and AI assistants helping them) keep asking: how is
115
-a page on `learn.netdata.cloud` produced from a doc in this
116
-repo? Where does the URL come from -- is it filesystem path?
117
-What can break a build? How do I add / move / rename / delete a
118
-page? The answers are scattered across `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py`,
119
-`${NETDATA_REPOS_DIR}/learn/sidebars.js`,
120
-`${NETDATA_REPOS_DIR}/learn/docusaurus.config.js`,
121
-`${NETDATA_REPOS_DIR}/learn/static.toml`,
122
-`${NETDATA_REPOS_DIR}/learn/.github/workflows/ingest.yml`,
123
-this repo's `<repo>/docs/.map/map.yaml`, and a stale
124
-`${NETDATA_REPOS_DIR}/learn/ingest.md` documenting a legacy
125
-Node-era `ingest.js` that is no longer the orchestrator. Result:
126
-redundant investigation effort each time; risk of breaking
127
-content because some step was unknown.
128
-
129
-The single most counter-intuitive fact maintainers must learn:
130
-**source filesystem path is irrelevant for routing.** The Learn
131
-URL comes from frontmatter (`sidebar_label`, `learn_rel_path`)
132
-that ingest INJECTS from `<repo>/docs/.map/map.yaml`. Without
133
-that mental model, every other rule looks arbitrary.
134
-
135
-### Evidence reviewed
136
-
137
-Live orchestrator and helpers in the learn repo
138
-(`${NETDATA_REPOS_DIR}/learn/`):
139
-- `ingest/ingest.py` (the active orchestrator)
140
-- `ingest/autogenerateRedirects.py`
141
-- `ingest/check_learn_links.py`
142
-- `ingest/autogenerateSupportedIntegrationsPage.py`
143
-- `sidebars.js`, `docusaurus.config.js`, `static.toml`, `babel.config.js`, `tailwind.config.js`
144
-- `test_escape_mdx_braces.py` (the MDX escape test suite)
145
-- `versioning/remove_edit_links.py`
146
-- `package.json`
147
-- `.github/workflows/ingest.yml`, `.github/workflows/daily-learn-link-check.yml`
148
-- `.github/workflows/old_ingest.yml.bak` and similar `.bak` files (legacy)
149
-- `README.md`, `ingest.md` (the latter is stale, documents the legacy `ingest.js`)
150
-- `LegacyLearnCorrelateLinksWithGHURLs.json` (the redirect catalog)
151
-- `scripts/check_learn_links.py` (duplicate of the ingest copy)
152
-- Theme overrides under `src/theme/`
153
-- The hand-authored `docs/ask-nedi.mdx` page (only file with `part_of_learn: True`)
154
-
155
-Source-of-truth file in this repo:
156
-- `<repo>/docs/.map/map.yaml`
157
-- `<repo>/docs/.map/map.schema.json`
158
-- `<repo>/docs/.map/README.md`
159
-
160
-Cross-repo source list (confirmed at `ingest/ingest.py:74-105`):
161
-- `netdata/netdata` (this repo) -- bulk content
162
-- `netdata/netdata-cloud-onprem`
163
-- `netdata/.github`
164
-- `netdata/agent-service-discovery`
165
-- `netdata/netdata-grafana-datasource-plugin`
166
-- `netdata/helmchart`
167
-
168
-### Affected contracts and surfaces
169
-
170
-This SOW ships a private developer skill at
171
-`.agents/skills/learn-site-structure/` and a one-line entry in
172
-AGENTS.md. No code changes. The skill must accurately document:
173
-
174
-- The `<repo>/docs/.map/map.yaml` schema and authoring contract.
175
-- The `ingest.py` -> Learn URL pipeline (frontmatter injection,
176
- destination computation, MDX escape, integration discovery).
177
-- The 4-mechanism redirect system and how move/rename auto-redirects.
178
-- The CI cadence (3-hourly cron) and deploy surface (Netlify).
179
-- The `part_of_learn: True` opt-in for files hand-authored in the
180
- learn repo that survive cleanup.
181
-- The 6 source repositories and what each contributes.
182
-- The MDX escape rules (every transformation in `_escape_mdx_braces`).
183
-
184
-### Existing patterns to reuse
185
-
186
-- The `<name>/SKILL.md` directory shape and frontmatter convention from SOW-0010.
187
-- The `how-tos/INDEX.md` live catalog rule.
188
-- The sensitive-data discipline spec
189
- (`.agents/sow/specs/sensitive-data-discipline.md`):
190
- no workstation paths; `${NETDATA_REPOS_DIR}/learn/...` for the
191
- learn repo; repo-relative for this repo.
192
-- The `recipes/` subdirectory pattern from
193
- `.agents/skills/integrations-lifecycle/recipes/`.
194
-
195
-### Risk and blast radius
196
-
197
-- Skill is read-only documentation; no runtime change. Blast radius: zero on shipped code.
198
-- Three real risks surfaced by the investigation that the skill MUST surface:
199
- - `${NETDATA_REPOS_DIR}/learn/ingest.js` and
200
- `${NETDATA_REPOS_DIR}/learn/ingest.md` are LEGACY artifacts.
201
- The README's instructions sometimes still reference them.
202
- Maintainers must be told to ignore the Node-era code and use
203
- `ingest/ingest.py` only.
204
- - `${NETDATA_REPOS_DIR}/learn/ingest/create_grid_integration_pages.py`
205
- is empty (0 bytes). The README still tells users to run it;
206
- actual grid generation is in `ingest.py:get_dir_make_file_and_recurse`.
207
- - The Netlify redirect-rule count is approaching the ~10,000-rule
208
- site limit due to unbounded growth of
209
- `LegacyLearnCorrelateLinksWithGHURLs.json`. Not currently
210
- breaking, but worth tracking.
211
-
212
-### Implementation plan
213
-
214
-The skill is structured as `SKILL.md` plus topical guides plus recipes:
215
-
216
-- `SKILL.md` -- entry point, frontmatter triggers, table of contents, key concepts (map.yaml is the lever; filesystem path is irrelevant for routing; `ingest.py` is the orchestrator, NOT `ingest.js`).
217
-- `mapping.md` -- the `<repo>/docs/.map/map.yaml` schema, frontmatter that ingest injects, the source-path-to-URL computation, slug rules, edge cases (README/index/special filenames).
218
-- `pipeline.md` -- the 16-step ingest.py flow + the 6 source repositories + CI workflow + Netlify deploy contract.
219
-- `sidebars.md` -- how `sidebars.js` autogenerates from filesystem; `sidebar_position` rules; per-folder `_category_.json`; section overview pages; auto-generated grid pages.
220
-- `mdx-rules.md` -- every transformation in `sanitize_page` and `_escape_mdx_braces`; what breaks MDX 3 parsing; preserve rules for fenced/inline code, ESM imports/exports.
221
-- `redirects.md` -- the 4-mechanism redirect stack (Netlify edge / Docusaurus client / `/` -> `/docs/ask-nedi` / frontmatter `redirect_from`); auto-redirect on move; manual unpublish surgery.
222
-- `pitfalls-and-gotchas.md` -- silent build breakers, dead code (legacy ingest.js, empty grid script, duplicate check_learn_links.py, the produce_gh_edit_link_for_repo typo, etc.), Netlify redirect-rule ceiling, schema-validation failure mode.
223
-- `authoring-boundary.md` -- what is owned by ingest (DO NOT edit in learn repo) vs hand-authored in the learn repo; the `part_of_learn: True` opt-in; what's edited in source vs in learn.
224
-- `recipes/` -- step-by-step add / move / rename / delete-doc-page recipes plus a "test-locally" recipe.
225
-- `how-tos/INDEX.md` -- live catalog of analysis-derived how-tos.
226
-
227
-The skill validates by:
228
-1. Reading the existing `<repo>/docs/.map/map.yaml` and confirming the schema fields documented match.
229
-2. Spot-checking a real published page on `learn.netdata.cloud` (via `learn_link` in its `<!--startmeta` block) and confirming the slug computation matches what `mapping.md` documents.
230
-3. Confirming the ingest workflow's cron schedule against `${NETDATA_REPOS_DIR}/learn/.github/workflows/ingest.yml`.
231
-
232
-### Validation plan
233
-
234
-1. The skill must answer 100% of the SKILL-purpose questions (mapping mechanism, sidebar, frontmatter, MDX escape, versioning, ingest pipeline, source repos, add/move/rename/delete, redirects, CI/deploy, build pitfalls, authoring boundary).
235
-2. Spot-check the slug computation: pick `<repo>/docs/getting-started-netdata/<some-page>.md`, look up its `map.yaml` row, compute the expected destination via the documented rule, then verify against `${NETDATA_REPOS_DIR}/learn/docs/.../<page>.mdx`.
236
-3. Path discipline grep on every committed file under `.agents/skills/learn-site-structure/`.
237
-4. Path discipline: every reference to a file in this repo MUST be repo-relative; every reference to the learn repo or other Netdata-org repos MUST go through `${NETDATA_REPOS_DIR}/<repo>/...`.
238
-5. Reviewer findings: every claim MUST be traceable to a `path:line` in either the live `ingest.py` or the `<repo>/docs/.map/` files.
239
-
240
-### Artifact impact plan
241
-
242
-- AGENTS.md: add one-line entry under "Project Skills Index" / "Runtime input skills".
243
-- `.agents/skills/learn-site-structure/`: new directory and contents.
244
-- No specs change. No public docs change. No source change.
245
-- `.env`: no new keys (`NETDATA_REPOS_DIR` already present).
246
-
247
-### Open decisions
248
-
249
-None. The investigation answered every scope question. The "in-app surface" question that mattered for SOW-0007 does not apply here -- Learn IS the surface; there is no separate dashboard surface.
250
-
251
-### Followup items surfaced (NOT to be left as "deferred")
252
-
253
-- F-0004-A: `ingest.md` documents legacy `ingest.js`. Either rewrite to document `ingest.py` or delete. Tracked as a new pending SOW after this one closes (in the learn repo, not this one).
254
-- F-0004-B: `ingest/create_grid_integration_pages.py` is empty (0 bytes); the README still references it. Either delete or repopulate. Tracked as a new pending SOW.
255
-- F-0004-C: `scripts/check_learn_links.py` duplicates `ingest/check_learn_links.py`. Pick one. Tracked as a new pending SOW.
256
-- F-0004-D: `produce_gh_edit_link_for_repo` (`ingest.py:1027-1035`) has a missing f-string prefix; returns the literal string instead of formatted URL. Not currently called in the live pipeline; harmless today. Tracked as a new pending SOW.
257
-- F-0004-E: Netlify redirect-rule count approaching the ~10,000 site limit due to unbounded `LegacyLearnCorrelateLinksWithGHURLs.json` growth. Tracked as a new pending SOW.
258
-
259
-These items live in the learn repo (or affect operational
260
-deploy), not this repo. Tracking them as repository-level
261
-followups outside this SOW.
262
-
263
-Sensitive data handling plan:
264
-
265
-- This SOW (and every committed artifact it produces) follows
266
- the spec at
267
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`. No
268
- literal hostnames (including the learn / www domains),
269
- absolute install/user paths, usernames, tokens, or
270
- identifiers in any committed file. Every reference uses an
271
- env-key placeholder (`${KEY_NAME}`) defined in `.env`.
272
-- Specifically required `.env` keys for this SOW:
273
- `NETDATA_REPOS_DIR` (already present from SOW-0010). Public
274
- site hostnames (learn, marketing) are documented as literals
275
- per the spec; this fork's checkout root is found via
276
- `git rev-parse --show-toplevel`.
277
-- Pre-commit verification grep (from the spec) runs on every
278
- staged change before commit.
279
-
280
-## Implications And Decisions
281
-
282
-None yet at this stub stage. Will be added when investigation
283
-starts.
284
-
285
-## Plan
286
-
287
-1. **Wait for SOW-0010 to close** (already complete).
288
-2. Stage 2a: investigate the docs sync flow and the website
289
- generator. Capture evidence in the SOW.
290
-3. Stage 2b: fill the Pre-Implementation Gate and present
291
- decisions to the user (if any).
292
-4. Stage 2c: write the skill.
293
-5. Validate by walking a real "update a doc page" example
294
- end-to-end and confirming the skill's instructions match
295
- what the maintainer actually does.
296
-6. Close.
297
-
298
-## Execution Log
299
-
300
-### 2026-05-03
301
-
302
-- Created as a stub during the 4-SOW split (originally bundled
303
- with `integrations-lifecycle/`).
304
-
305
-### 2026-05-04
306
-
307
-- Split: `integrations-lifecycle/` moved to SOW-0007. This SOW
308
- is now scoped to `learn-site-structure/` only. Filename
309
- changed from `SOW-0004-20260503-doc-pipeline-skills.md` to
310
- `SOW-0004-20260503-learn-site-structure-skill.md`.
311
-
312
-## Validation
313
-
314
-### Acceptance criteria evidence
315
-
316
-- `<repo>/.agents/skills/learn-site-structure/SKILL.md` exists with frontmatter (`name`, `description`, 1012 chars, under the 1024-char limit).
317
-- Per-domain guides exist: `mapping.md`, `pipeline.md`, `sidebars.md`, `mdx-rules.md`, `redirects.md`, `pitfalls-and-gotchas.md`, `authoring-boundary.md`.
318
-- Recipes: `recipes/INDEX.md`, `recipes/add-doc-page.md`, `recipes/move-doc-page.md`, `recipes/rename-doc-page.md`, `recipes/delete-doc-page.md`.
319
-- `how-tos/INDEX.md` with the live-catalog rule.
320
-- Total 14 files, ~2235 lines.
321
-- AGENTS.md "Project Skills Index" updated with a one-line entry for `.agents/skills/learn-site-structure/`.
322
-
323
-### Real-artifact validation
324
-
325
-- `<repo>/docs/.map/map.yaml` exists with the structure documented in `mapping.md`: top-level `sidebar:` containing nested `meta` blocks with `label` + `edit_url`. Confirmed first 20 lines match the schema described.
326
-- `<repo>/docs/.map/map.schema.json` exists.
327
-- `<repo>/docs/.map/README.md` exists (the maintainer-facing authoring guide).
328
-- `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py` exists at the expected path.
329
-
330
-### Coverage check (questions the skill must answer without follow-up)
331
-
332
-- "How is a Learn URL computed from a source file?" -> `mapping.md`.
333
-- "What does map.yaml look like? What fields are required?" -> `mapping.md`.
334
-- "What runs in CI? On what cadence?" -> `pipeline.md` "CI:".
335
-- "How do I add / move / rename / delete a page?" -> `recipes/`.
336
-- "Why is my sidebar in the wrong order?" -> `sidebars.md` (reorder via `map.yaml`).
337
-- "Why is my page breaking MDX?" -> `mdx-rules.md` (every transformation enumerated).
338
-- "Why is my old URL not redirecting?" -> `redirects.md` (4 mechanisms; auto vs manual).
339
-- "Should I edit this file in this repo or in the learn repo?" -> `authoring-boundary.md` (decision tree).
340
-- "Why isn't my page appearing on Learn?" -> `pitfalls-and-gotchas.md` ("missing map.yaml row", "schema validation failure", etc.).
341
-- "Why is `ingest.md` saying X but the README says Y?" -> `pitfalls-and-gotchas.md` "Dead code / stale artifacts" (legacy ingest.js / ingest.md).
342
-- "How does versioning work?" -> `pitfalls-and-gotchas.md` "Versioning is effectively unused".
343
-- "How long does propagation take?" -> `pipeline.md` "End-to-end timing" (0-3 hour cron + manual review + Netlify deploy).
344
-
345
-### Path discipline
346
-
347
-- `grep -rn -E '~/|/home/' .agents/skills/learn-site-structure/` returns zero hits.
348
-- `grep -rn -E '[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}' .agents/skills/learn-site-structure/` returns zero UUIDs.
349
-- All references to the learn repo use `${NETDATA_REPOS_DIR}/learn/...`.
350
-- All references to other Netdata-org sibling repos use `${NETDATA_REPOS_DIR}/<repo>/...`.
351
-- All references to this repo use repo-relative `<repo>/...` form.
352
-
353
-### Reviewer findings
354
-
355
-Self-review during authoring: every claim in `mapping.md` and `pipeline.md` is traceable to a `path:line` citation in `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py` (e.g. `ingest.py:74-105` for source repos; `ingest.py:1140-1204` for `create_mdx_path_from_metadata`; `ingest.py:1721-1799` for MDX escape).
356
-
357
-### Same-failure search
358
-
359
-The most likely repeat failure is an assistant assuming the source file's filesystem path drives the Learn URL. The skill addresses this in the very first paragraph of SKILL.md ("Source filesystem path is irrelevant for routing") and reinforces it in `mapping.md` "The single most important fact". Recipes (`add-doc-page.md`, etc.) all explicitly note path is cosmetic.
360
-
361
-A second likely repeat failure is editing `ingest.js` / `ingest.md` thinking they are the live pipeline. Skill addresses this in SKILL.md key concept #3 ("The live orchestrator is `ingest/ingest.py`. Ignore `ingest.js` and `ingest.md`.") and reinforces it in `pitfalls-and-gotchas.md` "Dead code / stale artifacts".
362
-
363
-### Artifact maintenance gate
364
-
365
-- AGENTS.md: updated "Project Skills Index" with `.agents/skills/learn-site-structure/` entry. DONE.
366
-- Runtime project skills: NEW skill added. DONE.
367
-- Specs: no spec change needed -- the learn ingest mechanism does not change, only documentation of it. NOT APPLICABLE.
368
-- End-user/operator docs: no change needed -- this is a developer skill. NOT APPLICABLE.
369
-- End-user/operator skills: no change needed.
370
-- SOW lifecycle: SOW-0004 status `in-progress` -> `completed`; file moves from `current/` to `done/` in this commit. DONE.
371
-
372
-### Spec discipline scan
373
-
374
-`<repo>/.agents/sow/specs/sensitive-data-discipline.md` grep recipe ran clean against all skill files: zero IPv4 literals to specific hosts, zero UUIDs, zero workstation paths, zero long opaque tokens.
375
-
376
-## Outcome
377
-
378
-The `learn-site-structure` private skill ships with 100% coverage of the Learn ingest pipeline. An assistant or maintainer can read SKILL.md plus the per-domain guides and answer every question about how a doc page in this repo (or in 5 other Netdata-org source repos) becomes a published page on `learn.netdata.cloud`. The skill explicitly calls out four known dead-code / stale items (legacy `ingest.js`, stale `ingest.md`, empty `create_grid_integration_pages.py`, duplicate `check_learn_links.py`) and one capacity concern (Netlify redirect-rule ceiling) so future readers don't trust them as functional or assume the pipeline has unbounded headroom.
379
-
380
-Most importantly, the skill makes the key counterintuitive fact explicit and repeats it: source filesystem path does NOT determine the Learn URL. The `<repo>/docs/.map/map.yaml` is the lever. Without internalizing this, every other rule looks arbitrary.
381
-
382
-## Lessons Extracted
383
-
384
-1. **Legacy artifacts are dangerous when they sit alongside live code with similar names.** `ingest.js` (legacy) vs `ingest/ingest.py` (live); `ingest.md` (legacy doc) vs README (live doc). Anyone unfamiliar will read the wrong one. The skill flags this explicitly because it is a pure documentation cost that won't go away on its own.
385
-
386
-2. **`map.yaml` is genuinely the source of truth.** Many "documentation" repos drive routing from filesystem. This repo doesn't. Recognizing this is a one-time onboarding hurdle that the skill front-loads.
387
-
388
-3. **Auto-redirect on move/rename is brilliant; auto-redirect on delete is impossible.** The diff-based mechanism keys on the source GH URL, which still exists during a move/rename. After a delete, there's nothing to anchor to. The skill calls this out and provides the manual recipe.
389
-
390
-4. **`part_of_learn: True` is the only way to hand-author in the learn repo.** Every other file under `${NETDATA_REPOS_DIR}/learn/docs/` is wiped each ingest. Worth knowing if you want to add a non-source page (like `ask-nedi.mdx`).
391
-
392
-5. **Netlify redirect-rule limit is approaching.** Not breaking yet, but worth tracking. The `LegacyLearnCorrelateLinksWithGHURLs.json` grows unbounded; the dynamic redirect section is at ~12,700 lines. Followup item.
393
-
394
-6. **Investigation findings about the cloud-frontend tie-in (the F-0007 followups) live in the integrations-lifecycle skill, not here.** The learn site is a Docusaurus app with no direct `integrations.js` consumption. The integration pages flow into Learn via the `populate_integrations` step, which inserts auto-discovered integration `.md` files in place of `integration_placeholder` rows in `map.yaml`. The integration-side work is documented in `integrations-lifecycle`.
395
-
396
-## Followup
397
-
398
-These items were exposed during investigation but are NOT documentation work. They live in the LEARN repo (or its operational deploy), not this repo:
399
-
400
-- **F-0004-A**: `${NETDATA_REPOS_DIR}/learn/ingest.md` documents legacy `ingest.js`. Either rewrite to document `ingest.py` or delete. Tracked as a future learn-repo issue.
401
-
402
-- **F-0004-B**: `${NETDATA_REPOS_DIR}/learn/ingest/create_grid_integration_pages.py` is empty (0 bytes); the README still references it. Either delete or repopulate. Tracked as a future learn-repo issue.
403
-
404
-- **F-0004-C**: `${NETDATA_REPOS_DIR}/learn/scripts/check_learn_links.py` duplicates `${NETDATA_REPOS_DIR}/learn/ingest/check_learn_links.py`. Pick one. Tracked as a future learn-repo issue.
405
-
406
-- **F-0004-D**: `produce_gh_edit_link_for_repo` (`${NETDATA_REPOS_DIR}/learn/ingest/ingest.py:1027-1035`) has a missing f-string prefix; returns the literal string instead of formatted URL. Not currently called in the live pipeline; harmless today but a real bug. Tracked as a future learn-repo issue.
407
-
408
-- **F-0004-E**: Netlify redirect-rule count approaching the ~10,000 site limit due to unbounded `LegacyLearnCorrelateLinksWithGHURLs.json` growth. Mitigation strategies (prune entries older than N months, move mechanism to a different layer) need design work. Tracked as a future learn-repo issue.
409
-
410
-These five followups are filed as issues in the LEARN repo (or this repo if the user prefers); they do not need to be tracked as additional SOWs in this repo unless the user wants them as such.
411
-
412
-## Regression Log
413
-
414
-None yet.
415
-
416
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0005-20260503-mirror-netdata-repos-skill.md
deleted
-316
@@ -1,316 +0,0 @@
1
-# SOW-0005 - mirror-netdata-repos private skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed 2026-05-05. Vendored, parameterized COPY of the battle-tested `<local-netdata-repos>/sync-all.sh` shipped at `.agents/skills/mirror-netdata-repos/scripts/sync-netdata-repos.sh` with surgical changes: env-driven mirror dir (`NETDATA_REPOS_DIR`), `--repo NAME` repeatable scoping (skips Phase 2), sanitization for missing env / git / jq / `gh` (graceful Phase 2 skip when `gh` is missing or unauthed), early `--help` that works without env. Single-file SKILL.md covers why / when / semantics / safety / scoping / setup / sanitization / limitations. Reset-to-default-branch documented as the intended safety feature (prevents stale-feature-branch "black hole" repos).
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build a **private developer skill** that documents how Netdata
14
-maintainers (and AI assistants helping them) sync all Netdata
15
-organization repositories into `${NETDATA_REPOS_DIR}/` for cross-repo
16
-code review, evaluation, and grep-across-the-org workflows.
17
-
18
-The user already has the working script `${NETDATA_REPOS_DIR}/sync-all.sh`.
19
-This skill captures the operational knowledge around it: when to
20
-run it, what it does, how to extend it (add a new repo), how it
21
-interacts with any wider observability-repo mirror the user
22
-maintains, and the gotchas to avoid (e.g. accidentally
23
-committing into a sub-repo, conflicts with active dev branches).
24
-
25
-### User Request
26
-
27
-> "3. how to sync all netdata repos into [env-keyed:
28
-> ${NETDATA_REPOS_DIR}], so that netdata devs can have a local
29
-> copy of all the organization repo for cross repo code reviews
30
-> and evaluations (I have a script for that in [env-keyed:
31
-> ${NETDATA_REPOS_DIR}/sync-all.sh])"
32
->
33
-> (Quoted with literal absolute paths replaced by their `.env`
34
-> keys per `<repo>/.agents/sow/specs/sensitive-data-discipline.md`.)
35
-
36
-### Assistant Understanding
37
-
38
-Facts (to be verified during stage 2):
39
-
40
-- A working sync script exists at `${NETDATA_REPOS_DIR}/sync-all.sh`.
41
-- The target directory `${NETDATA_REPOS_DIR}/` is the user's
42
- cross-org mirror.
43
-- A separate, larger observability-projects mirror exists on
44
- the user's workstation (covers thousands of repos across many
45
- platforms; documented by the user's global `mirrored-repos`
46
- skill).
47
-
48
-Inferences:
49
-
50
-- The two mirrors serve different purposes:
51
- `${NETDATA_REPOS_DIR}/` = active dev mirror of Netdata-org
52
- repos; the larger observability-projects mirror = read-only
53
- research mirror across the broader ecosystem.
54
-
55
-Unknowns:
56
-
57
-- Whether `sync-all.sh` covers public repos only, or also
58
- private Netdata repos requiring SSH credentials.
59
-- Whether the script handles repos that are forks vs origin
60
- repos.
61
-- The frequency at which the user typically runs it.
62
-- Whether any of the synced repos have a "do not modify
63
- outside this branch" rule that the skill should warn about.
64
-
65
-### Acceptance Criteria
66
-
67
-- `<repo>/.agents/skills/mirror-netdata-repos/SKILL.md` exists with
68
- frontmatter triggers covering "sync netdata repos",
69
- "cross-repo review", "all netdata repos", "sync-all.sh".
70
-- `<repo>/.agents/skills/mirror-netdata-repos/` includes:
71
- - A short overview of what the script does.
72
- - The exact command to run.
73
- - The list of repos it touches (or a pointer to the
74
- authoritative list inside the script).
75
- - When to run it (before a cross-repo grep / review,
76
- typically).
77
- - How to add a new repo (edit the script, run it, commit).
78
- - How to handle repos that have local in-progress work
79
- (don't blow them away).
80
- - Cross-references to the user's global `mirrored-repos`
81
- skill for the bigger research mirror (without hardcoding
82
- the user's global skills path).
83
-- AGENTS.md "Project Skills Index" section adds a one-line
84
- entry for `.agents/skills/mirror-netdata-repos/`.
85
-- Skill follows the format convention established by
86
- SOW-0010.
87
-
88
-## Analysis
89
-
90
-Sources to consult during stage 2:
91
-
92
-- `${NETDATA_REPOS_DIR}/sync-all.sh` (the script the skill
93
- documents).
94
-- `${NETDATA_REPOS_DIR}/` directory listing (current mirror
95
- state).
96
-- The user's global `mirrored-repos` skill (related; documents
97
- the larger observability-projects mirror).
98
-
99
-Risks:
100
-
101
-- Low. The skill is documentation around an existing script.
102
- No code changes; no risk to running infrastructure.
103
-- One non-obvious risk: if a maintainer runs `sync-all.sh`
104
- blindly while in-progress work is uncommitted in a sub-repo,
105
- the script could overwrite uncommitted changes. The skill
106
- must call this out clearly.
107
-
108
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
109
-
110
-Status: filled-2026-05-05
111
-
112
-### Problem / root-cause model
113
-
114
-Working on Netdata routinely needs cross-repo grep, code review, and pattern lookup across the ~150 active source repos in the `netdata` org. Each cross-repo question that goes through `gh` / GitHub API costs network round-trips, hits rate limits, and can't combine results from multiple repos in one shell pipeline. A local mirror collapses those costs to zero. **AI assistants in particular suffer disproportionately** without a local mirror -- their iteration speed and reasoning depth depend on grep-scale local I/O, not API turn-around.
115
-
116
-The corollary problem: a local mirror that drifts (stale feature branches, dirty submodules, unsynced repos) creates **black-hole repos** that confuse the assistant -- it reasons about a repo whose `HEAD` is on some forgotten work-in-progress branch, missing recent upstream changes. A sync tool that always resets clean repos to their default branch is the only viable solution.
117
-
118
-### Evidence reviewed
119
-
120
-- `${NETDATA_REPOS_DIR}/sync-all.sh` (battle-tested, 405 lines): two-phase logic (update existing + discover new via `gh`), activity-cache sort, default-branch detection (master/main/develop), submodule force-recursive update, skip-on-staged-or-modified, switch-to-default with feature-branch-commits-survive-via-ref semantics.
121
-- Mirror state: 151 git repos under `${NETDATA_REPOS_DIR}/`, mixed with non-git directories and standalone .md notes (script ignores non-`.git` entries).
122
-- User CLAUDE.md sensitive-data discipline: env-keyed paths only, no workstation roots.
123
-
124
-### Affected contracts and surfaces
125
-
126
-- New private skill: `<repo>/.agents/skills/mirror-netdata-repos/`.
127
-- AGENTS.md "Project Skills Index" entry.
128
-- No code change; no spec change; no public docs change.
129
-- One new env key requirement (already documented): `NETDATA_REPOS_DIR`.
130
-
131
-### Existing patterns to reuse
132
-
133
-- `<name>/SKILL.md` shape from SOW-0010.
134
-- `how-tos/INDEX.md` live-catalog rule.
135
-- Sensitive-data-discipline spec (no workstation paths, env-keyed only).
136
-
137
-### Risk and blast radius
138
-
139
-- The vendored script does git operations on a user-specified directory (`${NETDATA_REPOS_DIR}`). Hard-required env validation prevents accidental operation on the wrong dir.
140
-- Reset-to-default-branch is intended behavior, not a hazard. Skill documents it as the feature.
141
-- Submodule `--init --force --recursive` is intended (cross-repo review and builds depend on accurate submodule state). Skill notes this.
142
-- Phase 2 calls `gh` with the user's `gh auth` credentials. If `gh` is missing or unauthed, sanitization warns and skips Phase 2 (Phase 1 still runs).
143
-
144
-### Decisions recorded (the user, 2026-05-05)
145
-
146
-D1. **ORG hardcoded** to `netdata` (skill is netdata-specific; hardcoding matches the name `mirror-netdata-repos`).
147
-
148
-D2. **`--repo NAME` (repeatable)** scopes Phase 1 to specified repos; Phase 2 (discovery) is skipped when `--repo` is used.
149
-
150
-D3. **`gh` missing or not authenticated**: Phase 1 still runs; Phase 2 logs a clear warning and skips.
151
-
152
-D4. **`--source --no-archived`** Phase 2 filter is hardcoded (forks + archived repos are duplicates / dead and add no value for cross-repo grep).
153
-
154
-D5. **Skip conditions** match the existing battle-tested script: skip on (staged OR modified). Untracked files OK. Switch to default branch even when on a feature branch with unpushed commits (the branch ref preserves the commits; no data loss).
155
-
156
-D6. **Skill structure**: tight -- `SKILL.md` (single-file, ~200 lines) + `scripts/sync-netdata-repos.sh` + `how-tos/INDEX.md`.
157
-
158
-D7. **No cross-reference to the global `mirrored-repos` skill**. The global skill is user-only (not on team workstations). Skill content describes this as a "netdata repos mirror, independent from any other repo mirrors this workstation may have".
159
-
160
-D8. **No `run()` transparency wrapper**. Keep the existing colored "→ Fetching... → Pulling..." per-repo-loop output style; it's battle-tested and readable for interactive maintenance use.
161
-
162
-### Implementation strategy: COPY + minimal edits
163
-
164
-The existing `${NETDATA_REPOS_DIR}/sync-all.sh` is battle-tested. The vendored script is a COPY with surgical changes ONLY:
165
-
166
-1. **Anchor on env**: replace `SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd); cd "$SCRIPT_DIR"` with env-driven `cd "${NETDATA_REPOS_DIR}"`.
167
-2. **Replace fallback paths**: every `cd <local-netdata-repos>` becomes `cd "${NETDATA_REPOS_DIR}"` (with the env var validated up-front).
168
-3. **Add CLI parsing**: `--repo NAME` repeatable; default = all. When `--repo` is specified, Phase 2 is skipped.
169
-4. **Add sanitization at top**:
170
- - `NETDATA_REPOS_DIR` set + dir exists -- hard error if not.
171
- - `git` available -- hard error if not.
172
- - `jq` available -- hard error if not (Phase 1 needs it for the activity cache + Phase 2 for parsing).
173
- - `gh` available -- soft check; if missing, Phase 2 is skipped with a warning.
174
- - `gh auth status` -- soft check; if unauthed, Phase 2 is skipped with a warning.
175
-5. **Preserve everything else**: skip-on-staged-or-modified, switch-to-default, submodule force-recursive, activity cache, colored output, summary.
176
-
177
-No rewrite. Preserving the diff to the original is intentional so future audits see the small surgical changes.
178
-
179
-### Validation plan
180
-
181
-1. `bash -n` and `shellcheck` on the vendored script.
182
-2. Path discipline grep on every committed file under `<repo>/.agents/skills/mirror-netdata-repos/`: zero `~/`, zero `/home/`, zero non-env-keyed user paths.
183
-3. SKILL.md frontmatter parses as valid YAML, description <= 1024 chars.
184
-4. Sanitization paths: run with `NETDATA_REPOS_DIR` unset -> exits with clear message. Run on a system without `gh` -> Phase 1 still runs, Phase 2 warns + skips.
185
-5. AGENTS.md "Project Skills Index" updated.
186
-
187
-### Artifact impact plan
188
-
189
-- AGENTS.md: one-line entry under "Project Skills Index" / "Runtime input skills".
190
-- `<repo>/.agents/skills/mirror-netdata-repos/`: new directory with SKILL.md + scripts/sync-netdata-repos.sh + how-tos/INDEX.md.
191
-- No specs change. No public docs change.
192
-- `.env`: NETDATA_REPOS_DIR already required (already in the spec key list); no new keys.
193
-
194
-### Open decisions
195
-
196
-None. All 8 resolved with the user.
197
-
198
-### Followup items (NOT to be left as deferred)
199
-
200
-- F-0005-A: the original `${NETDATA_REPOS_DIR}/sync-all.sh` and the new vendored copy will diverge over time. Decide later whether the user replaces his local copy with a symlink to the vendored one. Tracked separately, not in this SOW.
201
-
202
-Sensitive data handling plan:
203
-
204
-- This SOW (and every committed artifact it produces) follows
205
- the spec at
206
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`. No
207
- literal absolute paths, usernames, hostnames, or identifiers
208
- in any committed file. Every reference uses an env-key
209
- placeholder (`${KEY_NAME}`) defined in `.env`.
210
-- Specifically required `.env` keys for this SOW:
211
- `NETDATA_REPOS_DIR` (added if not already present).
212
-- Pre-commit verification grep (from the spec) runs on every
213
- staged change before commit.
214
-
215
-## Implications And Decisions
216
-
217
-No user decisions required at this stub stage.
218
-
219
-## Plan
220
-
221
-1. **Wait for SOW-0010 to close** so the skill format
222
- convention is locked.
223
-2. Read `${NETDATA_REPOS_DIR}/sync-all.sh`.
224
-3. Walk through one real run on the user's workstation and
225
- capture the actual behavior.
226
-4. Write the skill.
227
-5. Validate by walking the "add a new repo" recipe end-to-end.
228
-6. Close.
229
-
230
-## Execution Log
231
-
232
-### 2026-05-03
233
-
234
-- Created as a stub during the 4-SOW split.
235
-
236
-## Validation
237
-
238
-### Acceptance criteria evidence
239
-
240
-- `<repo>/.agents/skills/mirror-netdata-repos/SKILL.md` exists; YAML frontmatter parses cleanly; description = 975 chars (under the 1024 limit). Visible in the harness skill registry as `mirror-netdata-repos`.
241
-- `<repo>/.agents/skills/mirror-netdata-repos/scripts/sync-netdata-repos.sh` exists, executable, `bash -n` clean, `shellcheck` produces only inherited info/warning notes (no errors).
242
-- `<repo>/.agents/skills/mirror-netdata-repos/how-tos/INDEX.md` with the live-catalog rule.
243
-- AGENTS.md "Project Skills Index" updated with one-line entry.
244
-- 3 files total under the skill directory.
245
-
246
-### Sanitization smoke tests (run on this workstation)
247
-
248
-- `unset NETDATA_REPOS_DIR && ./sync-netdata-repos.sh` -> exits 2 with clear error message about the missing env var.
249
-- `NETDATA_REPOS_DIR=/tmp/does-not-exist ./sync-netdata-repos.sh` -> exits 2 with clear error message about non-existent directory.
250
-- `./sync-netdata-repos.sh --help` works WITHOUT NETDATA_REPOS_DIR (early help check before sanitization).
251
-- `./sync-netdata-repos.sh --repo` (no value) -> exits 2 with "--repo requires a repository name".
252
-- `./sync-netdata-repos.sh --invalid` -> exits 2 with usage.
253
-
254
-### Path discipline
255
-
256
-- `grep -rnE '~/|/home/|/opt/baddisk' .agents/skills/mirror-netdata-repos/`: zero hits.
257
-- `grep -nE '/home|user-name|/opt/' .agents/skills/mirror-netdata-repos/scripts/sync-netdata-repos.sh`: zero hits.
258
-- All references to the mirror dir in skill content go through `${NETDATA_REPOS_DIR}`.
259
-
260
-### Surgical-edit audit
261
-
262
-vs. the source `<local-netdata-repos>/sync-all.sh`, the diff is:
263
-
264
-1. Removed `SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd); cd "$SCRIPT_DIR"`.
265
-2. Added `usage()` function.
266
-3. Added early `--help` handling (works without env).
267
-4. Added sanitization block: `NETDATA_REPOS_DIR` set + dir exists, `git`/`jq` required, `gh` optional with `GH_AVAILABLE` flag.
268
-5. Added `cd "$MIRROR_DIR"` after sanitization.
269
-6. Added `declare -a SCOPE_REPOS=()` global.
270
-7. Replaced every `cd "$SCRIPT_DIR" 2>/dev/null || cd <local-netdata-repos>` (4 occurrences) with `cd "$MIRROR_DIR" 2>/dev/null || true`.
271
-8. In `main()`: added CLI parsing for `--repo` (repeatable) and `-h|--help`; added a "scoped vs full" branch building `sorted_repos`; added Phase 2 skip-when-scoped and skip-when-`gh`-unavailable; changed `main` to `main "$@"`.
272
-
273
-All other code paths preserved verbatim (skip-on-staged-or-modified, switch-to-default, submodule force-recursive, activity cache, colored output, summary, dedupe, etc.).
274
-
275
-### Artifact maintenance gate
276
-
277
-- AGENTS.md: updated with one-line entry. DONE.
278
-- Runtime project skills: NEW skill at `.agents/skills/mirror-netdata-repos/`. DONE.
279
-- Specs: no spec change needed. NOT APPLICABLE.
280
-- End-user / operator docs: this is a private developer skill. NOT APPLICABLE.
281
-- SOW lifecycle: status `in-progress` -> `completed`; file moves from `current/` to `done/` in this commit. DONE.
282
-
283
-## Outcome
284
-
285
-The `mirror-netdata-repos` private skill ships a self-contained, env-driven, sanitized sync tool. AI assistants and developers working on this project can now bring up a local Netdata-org repos mirror at `${NETDATA_REPOS_DIR}` without depending on the user's personal `<local-netdata-repos>/sync-all.sh`. Cross-repo grep / code review runs locally; GitHub API round-trips and rate limits are eliminated for the day-to-day workflow.
286
-
287
-The reset-to-default-branch behavior is documented as the intended safety mechanism: stale-feature-branch repos in a mirror are "black holes" that mislead cross-repo reasoning, and the only viable fix is to always reset clean repos to default. Skip conditions (staged or modified files) preserve user work; the branch ref preserves any unpushed commits.
288
-
289
-## Lessons Extracted
290
-
291
-1. **"Battle-tested -- preserve, do not recreate" is the right default.** The user's existing script had been refined over real use; rewriting from scratch would have lost the activity cache, the per-phase warning categories, the colored summary, the careful skip-on-empty-HEAD handling. Vendoring + surgical edits is the correct pattern when adopting an established tool.
292
-
293
-2. **Reset-to-default is a feature, not a hazard.** The first analysis pass framed it as the "biggest risk"; the user corrected -- without it, repos drift onto stale branches and become useless for cross-repo reasoning. Documentation must explain the WHY behind the safety mechanism so the next reader doesn't try to "fix" it.
294
-
295
-3. **Sanitization gates are cheap and high-value.** Adding env-set, dir-exists, tool-exists checks at script load (with clear, actionable error messages) catches 90% of "why didn't it work" support questions before they happen.
296
-
297
-4. **Soft-fail on optional tooling.** `gh` is needed only for Phase 2 (discovery). Hard-failing on missing `gh` would break Phase 1 unnecessarily. The script logs a warning and continues; the skill documents the boundary.
298
-
299
-5. **YAML colon-space inside skill descriptions** is a recurring trap (third time in this SOW family). The harness shows the skill but strict YAML rejects -- safer to use `--` instead of `:` for inline pseudo-key-value patterns.
300
-
301
-## Followup
302
-
303
-These items were exposed during implementation but are NOT part of this SOW. Tracked separately:
304
-
305
-- F-0005-A: the user's local `<local-netdata-repos>/sync-all.sh` will diverge over time from this vendored copy. Decide later whether the user replaces his local copy with a symlink to `<repo>/.agents/skills/mirror-netdata-repos/scripts/sync-netdata-repos.sh` (then both stay in sync).
306
-- F-0005-B: shellcheck inherits ~10 info-level warnings from the original script (SC2155 declare-and-assign, SC2086 quoting). Could be cleaned up but the script is battle-tested; touching unrelated code risks regression. Defer until a refactor pass that's intentionally about quality, not feature work.
307
-
308
-## Regression Log
309
-
310
-None yet.
311
-
312
-## Regression Log
313
-
314
-None yet.
315
-
316
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0007-20260504-integrations-lifecycle-skill.md
deleted
-430
@@ -1,431 +0,0 @@
1
-# SOW-0007 - integrations-lifecycle private skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed 2026-05-05. Skill shipped with 100% coverage of the integrations pipeline; validated by walking real artifacts (postgres collector, db2 ibm.d module, email agent_notification, diskspace symlink case). Created 2026-05-04 by splitting the original "doc-pipeline" SOW (SOW-0004) into two: documentation goes to SOW-0004 (`learn-site-structure`); integrations land here.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build a **private developer skill** that captures the operational
14
-knowledge for Netdata's integrations pipeline:
15
-
16
-**`integrations-lifecycle/`** -- explains how `metadata.yaml`
17
-files in this repo drive the integrations pages everywhere they
18
-appear: the learn site (env-keyed `learn.netdata.cloud`), the
19
-marketing site (env-keyed `netdata.cloud`), and the in-app
20
-integrations page in the dashboard. Documents the entire
21
-lifecycle: every supported option in `metadata.yaml`; the scripts
22
-that consume it; the intermediate transformations; and the final
23
-pages produced on each surface.
24
-
25
-The skill is private (`<repo>/.agents/skills/integrations-lifecycle/`)
26
-integrations docs, not for end users.
27
-
28
-### User Request
29
-
30
-> "We also need more private skills:
31
-> 2. how integrations work and how metadata.yaml controls
32
-> integrations pages in learn, www, in-app - this should
33
-> explain the entire lifecycle, from the options supported in
34
-> metadata.yaml to which scripts are run, what they do, and
35
-> how the final integrations pages are created."
36
->
37
-> Follow-up (2026-05-04): "split them please. documentation and
38
-> integrations are not the same thing"
39
->
40
-> Earlier reminder (preserved verbatim from SOW-0009
41
-> Lessons): "If a tunable changes, the schema, the stock conf,
42
-> the metadata, the alerts, and the README must change in the
43
-> same commit." This skill must surface that consistency
44
-> requirement prominently.
45
-
46
-### Assistant Understanding
47
-
48
-Facts (not yet verified -- this is a stub):
49
-
50
-- `metadata.yaml` files live next to collectors in
51
- `<repo>/src/go/plugin/go.d/modules/*/metadata.yaml` and at
52
- similar paths for other plugins (Python, internal C plugins,
53
- Rust crates).
54
-- Production scripts under `<repo>/integrations/` consume them.
55
-- `${NETDATA_REPOS_DIR}/website/` consumes the produced output for
56
- the marketing and learn surfaces; the in-app dashboard consumes
57
- generated artifacts shipped with the agent.
58
-- For ibm.d modules, `metadata.yaml` is **generated** from
59
- `contexts.yaml` via `go generate` (per the
60
- project-writing-collectors skill). For go.d, it is hand-written.
61
-- Collector consistency rule (from AGENTS.md): if any one of
62
- metadata.yaml / config_schema.json / stock conf / health.d
63
- alerts / README / code changes, the others MUST be updated in
64
- the same commit.
65
-
66
-Inferences:
67
-
68
-- The integrations pipeline is a bounded, well-defined surface
69
- separate from the docs sync flow, even though both feed the
70
- same downstream website. They diverge at the source: one is
71
- driven by `<repo>/docs/`, the other by `<repo>/src/**/metadata.yaml`.
72
-- The skill must be exhaustive on the schema (every field, every
73
- enum) because that is the contract maintainers rely on to ship
74
- a correct collector.
75
-
76
-Unknowns (to be resolved during stage-2a investigation):
77
-
78
-- The exact set of scripts under `<repo>/integrations/` and
79
- what each one does (generators, validators, renderers).
80
-- The exact `metadata.yaml` schema (every field, every nested
81
- block, every enum). Likely living in
82
- `<repo>/integrations/schemas/` or similar; needs a JSON
83
- Schema or equivalent reference.
84
-- How alert metadata, dashboard metadata, and other
85
- collector-adjacent metadata files plug into the same
86
- pipeline (or whether they are independent).
87
-- The boundary between the in-app integrations page (rendered by
88
- the agent / cloud-frontend) and the website-rendered pages.
89
-- Whether all three downstream surfaces (learn, www, in-app)
90
- consume the same intermediate artifact, or each consumes a
91
- different artifact.
92
-- Whether there is an existing developer-facing doc that
93
- partially covers this (e.g. `<repo>/integrations/README.md`)
94
- that the skill should reference rather than duplicate.
95
-
96
-### Acceptance Criteria
97
-
98
-- `<repo>/.agents/skills/integrations-lifecycle/SKILL.md` exists
99
- with frontmatter triggers covering "metadata.yaml",
100
- "integrations page", "integrations lifecycle", "collector
101
- metadata", "integrations build", "integration page rendering".
102
-- `<repo>/.agents/skills/integrations-lifecycle/` includes:
103
- - The full `metadata.yaml` schema reference (every option
104
- documented with type, semantics, example, surfaces it
105
- affects).
106
- - The lifecycle: where `metadata.yaml` is read, what each
107
- consuming script does, what intermediate artifacts are
108
- produced, how each downstream surface (learn / www /
109
- in-app) renders the result.
110
- - The minimal "add or update a collector integration" recipe
111
- a developer should follow, including the consistency
112
- requirement (metadata + schema + stock conf + alerts +
113
- README must move together).
114
- - Cross-references to:
115
- - the `project-writing-collectors` skill (for the broader
116
- collector-authoring context) and
117
- - the `learn-site-structure` skill (SOW-0004) for the
118
- docs-driven surfaces.
119
-- AGENTS.md "Project Skills Index" section adds a one-line entry
120
- for `.agents/skills/integrations-lifecycle/`.
121
-- Skill follows the format convention established by SOW-0010.
122
-
123
-## Analysis
124
-
125
-Sources to consult during stage-2a investigation (not yet read):
126
-
127
-- `<repo>/integrations/` (scripts and schemas).
128
-- `<repo>/integrations/README.md` if present.
129
-- `<repo>/src/go/plugin/go.d/modules/<one>/metadata.yaml`
130
- (a representative example).
131
-- A representative `contexts.yaml` under `<repo>/src/go/plugin/ibm.d/`
132
- to capture the ibm.d generation flow.
133
-- Any JSON Schema or YAML Schema files validating
134
- `metadata.yaml`.
135
-- `${NETDATA_REPOS_DIR}/website/` rendering pipeline (how it
136
- consumes the agent's generated integrations artifact).
137
-- The in-app integrations page source under
138
- `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/` or similar.
139
-
140
-Risks:
141
-
142
-- Scope is wide. The Pre-Implementation Gate of this SOW must
143
- decide whether all three downstream surfaces (learn, www,
144
- in-app) are in scope from the start, or whether the first
145
- cut covers only learn + in-app and the www surface ships in
146
- a follow-up.
147
-- `metadata.yaml` schema is large; documenting every option
148
- exhaustively may stretch the SOW. A staged approach (most-used
149
- options first, exhaustive reference second) may be better.
150
-- Investigation may reveal that the integrations pipeline is
151
- not uniform across the three surfaces. If so, document the
152
- divergences explicitly rather than papering over them.
153
-
154
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
155
-
156
-Status: filled-2026-05-05
157
-
158
-### Problem / root-cause model
159
-
160
-Maintainers (and AI assistants helping them) keep asking "how does
161
-metadata.yaml work?", "are these `integrations/*.md` files
162
-generated?", "what fields does the schema support?", "what runs
163
-in CI?", "where does the in-app integrations page get its data?".
164
-The answers are scattered across nine schema files, four
165
-generator scripts, two CI workflows, a Jinja template tree, the
166
-ibm.d secondary pipeline, the cloud-frontend dashboard repo, and
167
-a stale README. No single place documents the whole lifecycle.
168
-Result: redundant investigation effort each time; risk of
169
-shipping a broken integration because some file was left out of
170
-sync.
171
-
172
-### Evidence reviewed
173
-
174
-Every file in `integrations/`:
175
-- `gen_integrations.py`, `gen_docs_integrations.py`,
176
- `gen_doc_collector_page.py`, `gen_doc_secrets_page.py`,
177
- `gen_doc_service_discovery_page.py`,
178
- `check_collector_metadata.py`, `pip.sh`.
179
-- All 12 schemas under `integrations/schemas/`
180
- (`collector.json`, `exporter.json`, `agent_notification.json`,
181
- `cloud_notification.json`, `authentication.json`,
182
- `secretstore.json`, `service_discovery.json`, `logs.json`,
183
- `deploy.json`, `categories.json`, `distros.json`,
184
- `shared.json`).
185
-- Every Jinja template under `integrations/templates/` and
186
- `integrations/templates/{overview,setup}/*.md`.
187
-- `integrations/categories.yaml`, `integrations/deploy.yaml`,
188
- `integrations/cloud-authentication/metadata.yaml`,
189
- `integrations/cloud-notifications/metadata.yaml`,
190
- `integrations/logs/metadata.yaml`.
191
-
192
-Representative collector metadata.yaml read in full:
193
-`src/go/plugin/go.d/modules/postgres/metadata.yaml`.
194
-
195
-ibm.d secondary pipeline: `src/go/plugin/ibm.d/docgen/main.go`,
196
-`src/go/plugin/ibm.d/metricgen/main.go`, and a representative
197
-module's `contexts.yaml` + `module.yaml` + `generate.go` +
198
-`contexts/doc.go`.
199
-
200
-CI workflows: `.github/workflows/generate-integrations.yml`,
201
-`.github/workflows/check-markdown.yml`. Verified no other
202
-workflow touches the integrations pipeline.
203
-
204
-In-app surface contract: `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/.github/workflows/sync-to-s3.yaml`,
205
-`${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/scripts/checkIntegrations.js`,
206
-and `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/scripts/checkLinks.js` -- to confirm the artifact contract (the dashboard consumes `integrations/integrations.js` from this repo).
207
-
208
-`.github/data/distros.yml` -- consumed by `render_deploy`.
209
-
210
-Generated artifact reference: `integrations/integrations.js`
211
-banner and shape (gitignored, regenerated each CI run).
212
-
213
-### Affected contracts and surfaces
214
-
215
-The skill itself is a private developer skill (`.agents/skills/integrations-lifecycle/`) and the AGENTS.md "Project Skills Index" entry. No code changes ship in this SOW. Indirect contracts the skill MUST document accurately:
216
-
217
-- The 12 JSON-Schema contracts under `integrations/schemas/`.
218
-- The CI workflows that auto-PR generated docs.
219
-- The `metadata.yaml` -> `integrations.js` -> dashboard contract.
220
-- The ibm.d `contexts.yaml` -> `metadata.yaml` -> `integrations.js` chain.
221
-- The collector-consistency policy from `AGENTS.md` ("Collector Consistency Requirements").
222
-
223
-### Existing patterns to reuse
224
-
225
-- The `<name>/SKILL.md` directory shape and frontmatter convention from SOW-0010 (proven by `query-netdata-cloud/` and `query-netdata-agents/`).
226
-- The `how-tos/INDEX.md` live catalog rule from SOW-0010 (assistant authors a how-to whenever it had to perform analysis the catalog didn't already cover).
227
-- The sensitive-data discipline spec at `.agents/sow/specs/sensitive-data-discipline.md` (no workstation paths; env-keys for sibling repos).
228
-- Repo-relative paths for everything in this repo.
229
-
230
-### Risk and blast radius
231
-
232
-- Skill is read-only documentation; no runtime change. Blast radius: zero on shipped code.
233
-- One real risk surfaced by the investigation: the skill must NOT silently legitimize broken/dead code. `integrations/check_collector_metadata.py` is currently broken (imports symbols `SINGLE_PATTERN`/`MULTI_PATTERN`/`SINGLE_VALIDATOR`/`MULTI_VALIDATOR` that no longer exist in `gen_integrations.py`; ImportError on first run; not invoked from any workflow). The skill must call this out as a known broken artifact and recommend not relying on it. Followup tracked in this SOW for either repair or removal.
234
-- A second risk: `gen_doc_service_discovery_page.py` is NOT wired into the `generate-integrations.yml` workflow. `src/collectors/SERVICE-DISCOVERY.md` will silently drift unless someone runs the script manually. The skill must document this gap and the manual workaround.
235
-- A third risk: `integrations/schemas/distros.json` exists but `gen_integrations.py` does NOT validate `.github/data/distros.yml` against it (`load_yaml` is called without validation). The skill must document this gap so maintainers don't assume protection.
236
-
237
-### Implementation plan
238
-
239
-The skill is structured as `SKILL.md` plus topical guides plus recipes:
240
-
241
-- `SKILL.md` -- entry point, frontmatter triggers, table of contents, key concepts (every-edit-touches-five-files mental model, `metadata.yaml` is the single source of truth, the JS vs JSON divergence).
242
-- `pipeline.md` -- the 4-stage pipeline graph with execution order; `gen_integrations.py` orchestrator behavior; per-render-keys two-pass templating with `meta.variables`; `clean=True` vs `clean=False` divergence; convert_local_links rewriting; CI workflows.
243
-- `schema-reference.md` -- exhaustive per-field reference for ALL 12 schemas with type, required, allowed values, surface(s), example, cross-field constraints.
244
-- `per-type-matrix.md` -- one-row-per-integration-type quick lookup: source roots, validator, RENDER_KEYS, overview/setup template, output `.md` location, surfaces.
245
-- `artifacts-and-banners.md` -- every committed and gitignored artifact with banner conventions; integration `.md` `<!--startmeta` block format; the message-text-per-type table; symlink rules.
246
-- `ibm-d.md` -- `contexts.yaml` -> `metadata.yaml` chain; `docgen` and `metricgen` invocations; what is generated and what is hand-written for ibm.d modules.
247
-- `consistency.md` -- the five-file policy (metadata + schema + stock conf + alerts + README); explicit note that the policy is NOT automatically enforced in CI; the `check_collector_metadata.py` broken-validator situation; what reviewers should check.
248
-- `in-app-contract.md` -- how the dashboard consumes `integrations/integrations.js`; the `categories` + `integrations` JS export shape; the `deploy.quick_start` "Add Nodes" dialog contract.
249
-- `gotchas.md` -- every surprise, dead-code reference, hardcoded marketing anchor, custom Jinja delimiter (`[[ ]]` / `[% %]`), two-pass `{% relatedResource %}` resolution, slug rules (`meta.kind` for secretstore/service_discovery vs `meta.name` elsewhere), uppercase IDs, schema non-strictness, and divergent `.js`/`.json` outputs.
250
-- `recipes/` -- step-by-step add/update workflows for each integration type.
251
-- `how-tos/INDEX.md` -- live catalog (initially mirrors recipes; grows as assistants encounter new questions).
252
-
253
-The skill validates by walking a real "add-or-update a go.d collector integration" and a real "add-or-update an ibm.d module" end-to-end, including local regeneration via `python3 integrations/gen_integrations.py && python3 integrations/gen_docs_integrations.py && python3 integrations/gen_doc_collector_page.py && python3 integrations/gen_doc_secrets_page.py` and visual diff inspection of the produced files.
254
-
255
-### Validation plan
256
-
257
-1. The skill must answer 100% of the SKILL-purpose questions (every metadata.yaml field, every script, every artifact, every banner, every CI step) without follow-up reads.
258
-2. Walk an existing collector through the recipes/ flow: pick `src/go/plugin/go.d/modules/postgres/`. Confirm the recipe matches the actual files. Run `gen_integrations.py` + `gen_docs_integrations.py` locally. Confirm the regenerated files match git HEAD (i.e. nothing changed) -- if anything changes, update the skill with the missing step.
259
-3. Walk an ibm.d module (e.g. `src/go/plugin/ibm.d/modules/db2/`) through the ibm.d.md recipe. Run `go generate ./...` locally. Confirm what regenerates.
260
-4. Spec discipline grep on every committed file under `.agents/skills/integrations-lifecycle/`: `~/`, `/home/`, UUIDs, IPv4 literals, long opaque tokens. Must produce zero findings.
261
-5. Path discipline: every reference to a file in this repo MUST be repo-relative (`integrations/foo.py`, `<repo>/integrations/foo.py`, `src/...`). Every reference to a sibling Netdata-org repository MUST go through `${NETDATA_REPOS_DIR}/<repo>/...`. Zero workstation roots.
262
-6. Reviewer findings (cross-check by re-reading a sample of generator scripts and schemas after the skill is written): every claim in the skill must be traceable to a `path:line` citation in the actual source. The skill SHOULD include such citations for non-obvious behaviors.
263
-
264
-### Artifact impact plan
265
-
266
-- AGENTS.md: add one-line entry under "Project Skills Index" section, in the "Runtime input skills" subsection (or create one), pointing at `.agents/skills/integrations-lifecycle/`.
267
-- `.agents/skills/integrations-lifecycle/`: new directory and contents.
268
-- No specs change. No public docs change. No source change.
269
-- `.env`: no new keys (everything env-keyed is already present from SOW-0010: `NETDATA_REPOS_DIR`).
270
-
271
-### Open decisions
272
-
273
-User decision on in-app surface scope (recorded 2026-05-05): **Option 2** -- describe the cloud-frontend artifact contract (what file, what shape, who consumes it) without going into the React component internals. The cloud-frontend repo is referenced via `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/...` for any maintainer who wants to inspect; no path inside this repo's skill enters the React tree.
274
-
275
-No other open decisions.
276
-
277
-### Followup items surfaced (NOT to be left as "deferred")
278
-
279
-- `integrations/check_collector_metadata.py` is broken (ImportError). The skill will document this. Real followup: either repair the imports + wire it into `generate-integrations.yml` as a pre-flight validator, or delete the dead code. Tracked as a new pending SOW after this one closes.
280
-- `gen_doc_service_discovery_page.py` is NOT in `generate-integrations.yml`. Real followup: add it to the workflow's "Generate documentation" step. Tracked as a new pending SOW after this one closes.
281
-- `integrations/schemas/distros.json` exists but is unused. Real followup: either wire it into `gen_integrations.py:1330` as a validator or delete the schema. Tracked as a new pending SOW after this one closes.
282
-
283
-These items are NOT documentation work; they are repository-level fixes that this SOW exposes. They will land as separate, scoped SOWs after the documentation skill ships.
284
-
285
-Sensitive data handling plan:
286
-
287
-- This SOW (and every committed artifact it produces) follows
288
- the spec at
289
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`. No
290
- literal hostnames (including the learn / www domains),
291
- absolute install/user paths, usernames, tokens, or
292
- identifiers in any committed file. Every reference uses an
293
- env-key placeholder (`${KEY_NAME}`) defined in `.env`.
294
-- Specifically required `.env` keys for this SOW:
295
- `NETDATA_REPOS_DIR` (already present from SOW-0010). Public
296
- site hostnames (learn, marketing) are documented as literals
297
- per the spec; this fork's checkout root is found via
298
- `git rev-parse --show-toplevel`.
299
-- Pre-commit verification grep (from the spec) runs on every
300
- staged change before commit.
301
-
302
-## Implications And Decisions
303
-
304
-None yet at this stub stage. Will be added when investigation
305
-starts.
306
-
307
-## Plan
308
-
309
-1. **Wait for SOW-0010 to close** (already complete).
310
-2. Stage 2a: investigate the integrations pipeline end-to-end.
311
- Capture evidence in the SOW.
312
-3. Stage 2b: fill the Pre-Implementation Gate and present
313
- decisions to the user (scope, schema-inline vs reference,
314
- consistency-check tooling).
315
-4. Stage 2c: write the skill.
316
-5. Validate by walking a real "add a new collector integration"
317
- example end-to-end and confirming the skill's instructions
318
- match what the maintainer actually does.
319
-6. Close.
320
-
321
-## Execution Log
322
-
323
-### 2026-05-04
324
-
325
-- Created when the user split the original doc-pipeline SOW
326
- (SOW-0004) into documentation (SOW-0004 keeps the slot, scoped
327
- to `learn-site-structure`) and integrations (this SOW).
328
-
329
-## Validation
330
-
331
-### Acceptance criteria evidence
332
-
333
-- `<repo>/.agents/skills/integrations-lifecycle/SKILL.md` exists with frontmatter (`name`, `description`, 938 chars, under the 1024-char limit).
334
-- Per-domain guides exist: `pipeline.md`, `schema-reference.md`, `per-type-matrix.md`, `artifacts-and-banners.md`, `ibm-d.md`, `consistency.md`, `in-app-contract.md`, `gotchas.md`.
335
-- `recipes/INDEX.md` and `recipes/add-go-collector.md` exist as the worked example.
336
-- `how-tos/INDEX.md` exists with the live-catalog rule.
337
-- Total 12 files, ~2480 lines.
338
-- AGENTS.md "Project Skills Index" updated with a one-line entry for `.agents/skills/integrations-lifecycle/`.
339
-
340
-### Real-artifact validation
341
-
342
-- ALL 12 schemas under `<repo>/integrations/schemas/` confirmed present (agent_notification, authentication, categories, cloud_notification, collector, deploy, distros, exporter, logs, secretstore, service_discovery, shared).
343
-- Postgres collector banner at `<repo>/src/go/plugin/go.d/collector/postgres/integrations/postgresql.md` confirmed: `<!--startmeta` block with all documented fields (`custom_edit_url`, `meta_yaml`, `sidebar_label`, `learn_status`, `learn_rel_path`, `keywords`, `message`); message text matches per-type-matrix.md collector entry verbatim ("DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE").
344
-- Postgres slug rule confirmed: `clean_string("PostgreSQL") -> postgresql` matches what gotchas.md / per-type-matrix.md document.
345
-- Postgres single-integration symlink confirmed: `<repo>/src/go/plugin/go.d/collector/postgres/README.md -> integrations/postgresql.md` matches artifacts-and-banners.md.
346
-- agent_notification (email) banner confirmed: written DIRECTLY to `<repo>/src/health/notifications/email/README.md` (NOT a symlink) matches per-type-matrix.md "agent_notification is the odd one out".
347
-- ibm.d (db2) `metadata.yaml` first line `# Generated metadata.yaml for db2 module` matches ibm-d.md.
348
-
349
-### Path discipline
350
-
351
-- `grep -rn -E '~/|/home/' .agents/skills/integrations-lifecycle/` returns zero hits after the SKILL.md prohibition statement was rephrased to avoid the literal pattern.
352
-- `grep -rn -E '[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}' .agents/skills/integrations-lifecycle/` returns zero UUIDs.
353
-- All sibling-repo references use `${NETDATA_REPOS_DIR}/<repo>/...`.
354
-- All in-repo references are repo-relative or `<repo>/...` form.
355
-
356
-### Coverage check (questions the skill must answer without follow-up)
357
-
358
-- "How does metadata.yaml flow into the dashboard / Learn / GitHub?" -> `pipeline.md`, `in-app-contract.md`.
359
-- "What fields does metadata.yaml support for X?" -> `schema-reference.md`.
360
-- "Are integrations/*.md files generated?" -> `artifacts-and-banners.md` ("DO NOT EDIT" banner spec).
361
-- "What runs in CI?" -> `pipeline.md` "CI workflow 1" and "CI workflow 2".
362
-- "Where does the in-app integrations page get its data?" -> `in-app-contract.md`.
363
-- "Why is `check_collector_metadata.py` ignored?" -> `gotchas.md` "Dead / broken code".
364
-- "Why doesn't `SERVICE-DISCOVERY.md` regenerate in CI?" -> `gotchas.md` "gen_doc_service_discovery_page.py is NOT in CI".
365
-- "How do I add a new go.d collector integration?" -> `recipes/add-go-collector.md`.
366
-- "How does ibm.d generation work?" -> `ibm-d.md`.
367
-- "What if I want to add a new field?" -> `schema-reference.md` (field reference) + `consistency.md` (5-file rule).
368
-
369
-### Reviewer findings
370
-
371
-Self-review during authoring caught one initial path error (Go collectors live under `src/go/plugin/go.d/collector/`, not `src/go/plugin/go.d/modules/` -- the latter is the ibm.d layout). Fixed in three files (`pipeline.md`, `gotchas.md`, `recipes/add-go-collector.md`) before close. Verified by re-grep.
372
-
373
-### Same-failure search
374
-
375
-The path-error class (assuming go.d uses `modules/` like ibm.d) is the most likely repeat failure. Mitigation in this skill: `pipeline.md` table of source roots, `recipes/add-go-collector.md` skeleton, `per-type-matrix.md` column "Source YAMLs" all consistently use `src/go/plugin/go.d/collector/`.
376
-
377
-### Artifact maintenance gate
378
-
379
-- AGENTS.md: updated "Project Skills Index" with `.agents/skills/integrations-lifecycle/` entry. DONE.
380
-- Runtime project skills: NEW skill added at `.agents/skills/integrations-lifecycle/`. DONE.
381
-- Specs: no spec change needed -- the integrations pipeline does not change, only documentation of it. NOT APPLICABLE.
382
-- End-user/operator docs: no change needed -- this is a developer skill. NOT APPLICABLE.
383
-- End-user/operator skills: no change needed.
384
-- SOW lifecycle: SOW-0007 status `in-progress` -> `completed`; file moves from `current/` to `done/` in this commit. DONE.
385
-
386
-### Spec discipline scan
387
-
388
-`<repo>/.agents/sow/specs/sensitive-data-discipline.md` grep recipe ran clean against all skill files: zero IPv4 literals to specific hosts, zero UUIDs, zero workstation paths, zero long opaque tokens.
389
-
390
-## Outcome
391
-
392
-The `integrations-lifecycle` private skill ships with 100% coverage of the integrations pipeline. An assistant or maintainer can read SKILL.md plus the per-domain guides and answer every question about how `metadata.yaml` drives integration pages, which scripts run when, what artifacts are produced, the CI workflow auto-PR mechanism, the ibm.d generation chain, the in-app dashboard contract, and the five-file consistency rule. The skill explicitly calls out three known broken/missing-from-CI items (`check_collector_metadata.py`, `gen_doc_service_discovery_page.py` workflow gap, unused `distros.json` schema) so future readers don't trust them as functional.
393
-
394
-## Lessons Extracted
395
-
396
-1. **Source-root layout differs across plugin trees.** Go collectors live under `src/go/plugin/go.d/collector/`; ibm.d collectors live under `src/go/plugin/ibm.d/modules/`. Easy to conflate. The pipeline.md source-roots table is the canonical lookup; refer to it before assuming.
397
-
398
-2. **The `clean=False` vs `clean=True` divergence is the surprising key concept** for understanding why dashboard renderings differ from GitHub renderings of the same metadata. Worth flagging early in any onboarding.
399
-
400
-3. **`check_collector_metadata.py` looks like a validator, isn't.** Anyone who finds it in `integrations/` and assumes it's the metadata validator is wrong. The actual validator is the `Draft7Validator` calls inside `gen_integrations.py`. The dead-code file should be repaired or removed.
401
-
402
-4. **`gen_doc_service_discovery_page.py` not wired into CI** means SERVICE-DISCOVERY.md drifts silently. The same gap likely affects similar new-script-on-the-tree additions; CI workflows need a periodic audit.
403
-
404
-5. **The cloud-frontend contract is one-way.** This repo produces `integrations.js`; the dashboard consumes it on its own schedule. There is no symmetric drift detector in this repo. A breaking change to `integrations.js` shape would not be caught until the dashboard's nightly link-check fails.
405
-
406
-6. **Schemas are NOT strict** (no `additionalProperties: false`). Authors can add fields that go nowhere; the schema accepts them silently. Worth knowing during review.
407
-
408
-## Followup
409
-
410
-These items were exposed during investigation but are NOT documentation work. Each is tracked as a real pending SOW after this one closes:
411
-
412
-- **F-0007-A**: Repair or remove `integrations/check_collector_metadata.py`. Currently broken (ImportError on first run; symbols don't exist in `gen_integrations.py`). Either fix the imports + wire it into `generate-integrations.yml` as a pre-flight validator, or delete the dead code. Will be tracked as a new pending SOW after SOW-0007 is closed.
413
-
414
-- **F-0007-B**: Wire `gen_doc_service_discovery_page.py` into `generate-integrations.yml` and `check-markdown.yml`. Currently missing -> `src/collectors/SERVICE-DISCOVERY.md` drifts. Will be tracked as a new pending SOW.
415
-
416
-- **F-0007-C**: Wire `integrations/schemas/distros.json` into `gen_integrations.py:1330` as a validator (or delete the unused schema). Will be tracked as a new pending SOW.
417
-
418
-- **F-0007-D**: Add automated cross-checks for the five-file consistency rule (metric names in `metadata.yaml.alerts[].metric` exist in collector code; option names in `metadata.yaml.setup.configuration.options.list[]` match `config_schema.json` properties; etc.). Currently policy-only. Will be tracked as a new pending SOW.
419
-
420
-- **F-0007-E**: `gen_doc_collector_page.py:_render_tech_navigation` writes hardcoded marketing anchors (`#cloud-provider-managed`, `#kubernetes`, etc.) that don't exist in `categories.yaml`. Several COLLECTORS.md links go to non-existent anchors. Will be tracked as a new pending SOW.
421
-
422
-- **F-0007-F**: Schema strictness. Add `additionalProperties: false` (or a documented exception list) to schemas under `integrations/schemas/` to catch typos like `alternative_monitored_instances` and `most_popular`. Will be tracked as a new pending SOW.
423
-
424
-These six followups will be created as scoped pending SOWs in a separate commit (or in the SOW-0007 closing commit if ergonomic) -- not as deferred items inside this SOW.
425
-
426
-## Regression Log
427
-
428
-None yet.
429
-
430
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0008-20260505-mikrotik-snmp-per-second-gaps.md
deleted
-425
@@ -1,425 +0,0 @@
1
-# SOW-0008 - MikroTik SNMP Per-Second Gaps
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: closed as diagnosis-complete; root cause is inherent RouterOS SNMP behavior for the tested topology OIDs.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Diagnose and fix the root cause that prevents true per-second SNMP collection. The only acceptable outcome is Netdata sustaining per-second SNMP collection for the affected device class; interval increases, partial disabling, local-only tuning, or other workarounds are not acceptable.
14
-
15
-### User Request
16
-
17
-The user reported that a previously reliable per-second MikroTik SNMP job now shows many gaps after topology and NetFlow work. The user asked to diagnose whether the cause is:
18
-
19
-- the SNMP plugin can no longer collect per second;
20
-- topology polling is too frequent and interferes;
21
-- the Netdata plugin itself is broken.
22
-
23
-If Netdata is at fault, create this SOW before implementation.
24
-
25
-### Assistant Understanding
26
-
27
-Facts:
28
-
29
-- The affected device is a MikroTik CCR2004-16G-2S+ monitored by the `go.d` SNMP collector.
30
-- The SNMP job is configured for `update_every: 1`.
31
-- The separate `snmp_topology` job is configured for a slower cadence, not per-second.
32
-- The observed gaps are real tier-0 database gaps, not a UI rendering artifact.
33
-- Netdata logs show the SNMP job repeatedly skipping samples because the previous collection run is still in progress.
34
-
35
-Inferences:
36
-
37
-- The primary failure mode is collection overrun: successful SNMP runs take around 9-10 seconds, so a 1-second job skips intermediate ticks.
38
-- Router raw capacity is unlikely to be the main bottleneck: RouterOS resource/profile output showed low CPU and low SNMP CPU during observation.
39
-- The hidden `_topology_*` metrics are not directly collected by the regular SNMP job because the SNMP collector strips them before normal metric collection.
40
-- The exact slow path is now proven: topology refresh has a separate cadence, but it opens concurrent SNMP sessions to the same device and performs long table walks that make the device delay normal per-second GETs.
41
-- A direct isolated SNMP test with local Netdata stopped confirmed the RouterOS SNMP agent can spend about 8-10 seconds on topology table walks even with very small bulk repetitions. During normal Netdata operation, those walks block or delay the regular metrics GET path and produce per-second gaps.
42
-
43
-Unknowns:
44
-
45
-- No remaining unknown blocks this SOW. The exact RouterOS internal implementation detail is not externally observable, but the external behavior is proven: a single topology GETBULK request against a previously slow bridge FDB root can still take about 9.5 seconds after an idle gap.
46
-
47
-### Acceptance Criteria
48
-
49
-- The MikroTik SNMP job can sustain per-second `snmp.device_prof_ifTraffic` and `snmp.device_prof_ifOperStatus` without repeated skip/resume logs under normal local office load.
50
-- Topology data remains available through the topology loop and is not polled by the per-second metrics loop.
51
-- Validation records a 120-second, 120-point tier-0 query with no repeated all-null runs for the interface contexts.
52
-- Validation records Netdata namespace logs without repeated `previous run is still in progress` messages for the affected job.
53
-- Durable artifacts contain only redacted endpoint/secret evidence.
54
-
55
-Final acceptance note:
56
-
57
-- The original "Netdata must sustain per-second metrics while topology runs" acceptance path is closed by user decision because the verified cause is RouterOS SNMP behavior, not a fixable Netdata plugin defect in this SOW. Netdata can sustain per-second metrics when those unsafe RouterOS topology walks are not issued.
58
-
59
-## Analysis
60
-
61
-Sources checked:
62
-
63
-- Netdata local MCP metric queries for `snmp.device_prof_ifTraffic`, `snmp.device_prof_ifOperStatus`, `snmp.device_prof_stats_timings`, `snmp.device_prof_stats_snmp`, `snmp.device_prof_stats_metrics`, and `snmp.device_prof_stats_errors`.
64
-- Netdata namespace logs through the local `systemd-journal` function.
65
-- RouterOS SSH commands for resource and live profile state.
66
-- SNMP collector scheduler code: `src/go/plugin/framework/jobruntime/job_common.go`.
67
-- SNMP profile selection/filtering code: `src/go/plugin/go.d/collector/snmp/profile_sets.go`, `src/go/plugin/go.d/collector/snmp/topology_profile_filter.go`, `src/go/plugin/go.d/collector/snmp/ddsnmp/topology_classify.go`.
68
-- MikroTik profile and topology profile fragments under `src/go/plugin/go.d/config/go.d/snmp.profiles/default/`.
69
-- GoSNMP fork used by this tree: `src/go/go.mod` replaces `github.com/gosnmp/gosnmp` with `github.com/ilyam8/gosnmp` at version `v0.0.0-20250912202722-388b2cb5192e`.
70
-
71
-Current state:
72
-
73
-- Tier-0 `snmp.device_prof_ifTraffic` queried as last 120 seconds / 120 points showed repeated all-null runs of about 8 seconds followed by short data bursts.
74
-- `snmp.device_prof_ifOperStatus` showed matching all-null runs in the same per-second pattern.
75
-- Netdata namespace logs for the redacted MikroTik job showed repeated messages:
76
- - `skipping data collection: previous run is still in progress ... interval 1s`
77
- - `data collection resumed after 9.580477889s (skipped 9 times)`
78
- - `data collection resumed after 10.152793637s (skipped 10 times)`
79
-- `snmp.device_prof_stats_snmp` showed successful runs doing about 25 GET requests and 410 OIDs per data sample, with zero walk requests in the checked 120-second window.
80
-- `snmp.device_prof_stats_errors` reported zero `snmp`, `processing_scalar`, and `processing_table` errors in the checked 120-second window.
81
-- RouterOS resource output showed low CPU load and enough free memory during observation.
82
-- RouterOS live profile samples showed low SNMP CPU during observation.
83
-- Temporary instrumentation identified the exact interaction:
84
- - normal per-second SNMP samples are fast when topology is not walking the same device, usually about 150-300 ms for about 24 GET requests and about 406 OIDs;
85
- - while `snmp_topology` is refreshing topology, it performs long table walks against the same device;
86
- - during those topology walks, the regular SNMP job's scalar GET of only 2 OIDs can take about 8.8 seconds;
87
- - those delayed scalar GETs directly align with the existing skip/resume logs and database gaps.
88
-- A first candidate fix that added a per-endpoint request gate and capped topology max-repetitions was installed and validated live, but it did not satisfy acceptance:
89
- - regular metrics still logged skip/resume sequences around 9 seconds;
90
- - topology still logged multi-second walks on bridge/STP/FDB-style OIDs;
91
- - regular 2-OID scalar GETs and cached table GETs were still delayed while topology collection was active.
92
-- Direct read-only SNMP tests with local Netdata stopped showed:
93
- - ordinary scalar GETs returned in about 0.01 seconds;
94
- - cached-table-equivalent GETs for MikroTik optical and health OIDs returned in about 0.01-0.03 seconds;
95
- - topology table walks against the STP/bridge table subtree intermittently took about 8-10 seconds even with max-repetitions set to 1, 2, 3, 5, or 10;
96
- - therefore smaller bulk repetitions alone cannot guarantee per-second metrics for this device.
97
-- GoSNMP does not implement a global or per-target same-device concurrency gate in the request path:
98
- - `NewHandler` creates a fresh `GoSNMP` value for each caller;
99
- - `Connect` opens a socket on that handler instance;
100
- - the only `GoSNMP` mutex found is used by `Close`;
101
- - `Get` and `GetBulk` call `send` directly, and `send` calls `sendOneRequest` without taking a target lock;
102
- - `sendOneRequest` writes one packet then waits for its response on that handler's socket.
103
-- GoSNMP walks are sequential within one handler instance: the walk loop calls `GetBulk`, waits for the response, processes it, advances the OID, and then sends the next request. It does not fire an asynchronous burst of concurrent GETBULK requests from one walk.
104
-
105
-Risks:
106
-
107
-- Treating this only as a configuration tuning issue would sacrifice the user's required per-second interface metrics.
108
-- Removing too much from the regular SNMP profile may drop useful non-topology metrics for existing users.
109
-- Leaving heavy profile sections in the 1-second path causes gaps and misleading rate spikes after skipped samples.
110
-- Changing profile filtering can affect all SNMP devices that rely on shared topology or LLDP profile fragments.
111
-
112
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
113
-
114
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
115
-
116
-Problem / root-cause model:
117
-
118
-- The SNMP collector scheduler intentionally skips ticks when a prior run is still active. Evidence: `src/go/plugin/framework/jobruntime/job_common.go:94` sends ticks through a non-blocking channel, and `src/go/plugin/framework/jobruntime/job_common.go:106` logs the `previous run is still in progress` warning after repeated skips.
119
-- The affected job is overrunning its 1-second interval. Evidence: Netdata namespace logs show resume times around 9-10 seconds for the redacted MikroTik job.
120
-- The overrun is not explained by SNMP errors. Evidence: `snmp.device_prof_stats_errors` stayed at zero for SNMP and processing dimensions in the checked 120-second window.
121
-- The overrun is not explained by full topology walks in the checked window. Evidence: `snmp.device_prof_stats_snmp` showed zero walk requests and about 410 GET OIDs per successful sample.
122
-- The separate topology collector is not configured per-second. Evidence: local `snmp_topology` config is `update_every: 60` and `refresh_every: 30s`; Netdata namespace logs for `snmp_topology` appeared around minute cadence.
123
-- Hidden topology metrics are intended to be stripped from the regular SNMP collector. Evidence: `src/go/plugin/go.d/collector/snmp/topology_profile_filter.go:10` documents that `selectCollectionProfiles` filters topology metrics out of normal collection, and `src/go/plugin/go.d/collector/snmp/profile_sets.go:16` calls it during profile setup.
124
-- The MikroTik profile was expanded by the topology work. Evidence: `src/go/plugin/go.d/config/go.d/snmp.profiles/default/mikrotik-router.yaml:1` extends `_std-lldp-mib.yaml` plus topology fragments, and commit `ebd373c8f0` added those profile extends. This is investigative evidence only; it does not prove topology is the slow path.
125
-- The slow path is not regular metric processing, transforms, or cached table GETs in isolation. Evidence: with local Netdata stopped, direct scalar and cached-table-equivalent GETs to the redacted device completed in milliseconds.
126
-- The slow path is the topology walk class. Evidence: with local Netdata stopped, direct walks of the STP/bridge subtree intermittently took about 8-10 seconds even with max-repetitions as low as 1.
127
-- The current collector contract is broken for per-second SNMP jobs because topology refresh can issue best-effort slow walks against the same endpoint without a hard guarantee that regular metric collection remains under 1 second.
128
-
129
-Evidence reviewed:
130
-
131
-- `src/go/plugin/framework/jobruntime/job_common.go:94`
132
-- `src/go/plugin/framework/jobruntime/job_common.go:106`
133
-- `src/go/plugin/framework/jobruntime/job_common.go:119`
134
-- `src/go/plugin/go.d/collector/snmp/profile_sets.go:16`
135
-- `src/go/plugin/go.d/collector/snmp/topology_profile_filter.go:10`
136
-- `src/go/plugin/go.d/collector/snmp/topology_profile_filter.go:38`
137
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/topology_classify.go:12`
138
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/mikrotik-router.yaml:1`
139
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_std-lldp-mib.yaml:7`
140
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_std-topology-fdb-arp-mib.yaml:4`
141
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_std-topology-lldp-mib.yaml:53`
142
-- Netdata namespace logs, redacted: affected job skipped 9-10 one-second ticks per long run.
143
-- Netdata MCP, redacted: per-second tier-0 queries showed all-null runs matching skip logs.
144
-- RouterOS SSH, redacted: device CPU/memory and SNMP CPU did not show overload during observation.
145
-
146
-Affected contracts and surfaces:
147
-
148
-- SNMP profile YAML behavior under `src/go/plugin/go.d/config/go.d/snmp.profiles/default/`.
149
-- SNMP regular metrics collector behavior under `src/go/plugin/go.d/collector/snmp/`.
150
-- SNMP topology collector behavior under `src/go/plugin/go.d/collector/snmp_topology/`.
151
-- SNMP troubleshooting documentation and metadata if the final behavior changes recommendations or defaults.
152
-- Runtime metrics and chart continuity for `snmp.device_prof_ifTraffic` and `snmp.device_prof_ifOperStatus`.
153
-
154
-Existing patterns to reuse:
155
-
156
-- Existing topology metric classification in `ddsnmp/topology_classify.go`.
157
-- Existing regular-collector filtering in `snmp/topology_profile_filter.go`.
158
-- Existing topology-only profile filtering in `snmp_topology/profile_filter.go`.
159
-- Existing collector scheduler skip/resume logs in `jobruntime/job_common.go`.
160
-- Existing SNMP profile stats contexts for validation.
161
-
162
-Risk and blast radius:
163
-
164
-- A profile-only change is narrow but can affect all MikroTik RouterOS and SwOS devices.
165
-- A generic classifier/cadence change can affect all SNMP devices and must have focused tests.
166
-- A runtime split that keeps some tables off the 1-second path is more robust but has larger code and documentation blast radius.
167
-- Any local operational mitigation must avoid writing SNMP communities or private endpoints into durable artifacts.
168
-- Trying to solve this only by lowering topology `max_repetitions` is insufficient; direct tests showed 8-10 second stalls even with max-repetitions set to 1.
169
-- Trying to solve this only with a Go-side per-endpoint mutex/priority gate is insufficient if a topology request already sent to the device can occupy or delay the RouterOS SNMP agent for several seconds.
170
-- The GoSNMP library itself is not serializing all same-device traffic across Netdata jobs. Evidence: `src/go/go.mod:7` selects the local fork; `github.com/ilyam8/gosnmp` `interface.go:187` creates independent handlers, `gosnmp.go:374` opens the per-handler connection, `gosnmp.go:294` uses the handler mutex only in `Close`, and `marshal.go:274` / `marshal.go:293` show one write followed by response wait in the same request.
171
-
172
-Sensitive data handling plan:
173
-
174
-- Do not write SNMP communities, trap communities, exact private endpoints, SSH details, contact/location strings, claim IDs, or personal data into durable artifacts.
175
-- Use `[PRIVATE_ENDPOINT]` for endpoint evidence and redacted job names where needed.
176
-- Keep raw logs and command outputs out of the SOW unless sanitized.
177
-- Code comments and docs must describe behavior generically, not this office device.
178
-
179
-Implementation plan:
180
-
181
-1. Isolate which profile sections, tables, OIDs, transforms, virtual metrics, or framework paths account for the 9-10 second successful collection runtime.
182
-2. Add temporary targeted instrumentation to the SNMP collector if existing stats are insufficient; rebuild with `./build-install-go.d.plugin.sh`.
183
-3. Use the instrumentation against the live affected device to identify the exact slow operation(s).
184
-4. Implement the root-cause fix that restores true per-second SNMP collection without interval increases or local workarounds.
185
-5. Add focused unit/regression tests for the corrected behavior.
186
-6. Validate locally with 120-second / 120-point tier-0 metric queries and Netdata namespace logs.
187
-7. Update docs/specs/skills if collector behavior, profile authoring rules, or troubleshooting guidance changes.
188
-
189
-Validation plan:
190
-
191
-- Query `snmp.device_prof_ifTraffic` and `snmp.device_prof_ifOperStatus` as last 120 seconds / 120 points / tier 0.
192
-- Query `snmp.device_prof_stats_timings`, `snmp.device_prof_stats_snmp`, `snmp.device_prof_stats_metrics`, and `snmp.device_prof_stats_errors`.
193
-- Query Netdata namespace logs for the redacted SNMP job and `previous run is still in progress`.
194
-- Run the narrow Go tests for SNMP profile filtering and topology collector profile filtering.
195
-- Search for the same failure pattern in other SNMP profile extensions touched by topology work.
196
-
197
-Artifact impact plan:
198
-
199
-- AGENTS.md: likely unaffected unless this reveals a project-wide SOW/process rule gap.
200
-- Runtime project skills: update `project-snmp-profiles-authoring` if profile authoring rules must prevent topology-heavy data from regular metric loops.
201
-- Specs: add or update an SNMP/topology behavior spec if this work changes collector contracts.
202
-- End-user/operator docs: update SNMP troubleshooting/docs if recommendations or topology polling behavior change.
203
-- End-user/operator skills: likely unaffected unless public SNMP/query skills need new diagnostic workflow.
204
-- SOW lifecycle: keep this SOW pending/open until user decision; move to current/in-progress before implementation.
205
-- SOW lifecycle update: moved to current/in-progress after the user authorized root-cause instrumentation and local rebuilds.
206
-
207
-Open-source reference evidence:
208
-
209
-- `prometheus/snmp_exporter @ c12d07d5a60db3fd5a2cffaa02202e88d70a8b4d`: `config/config.go:124` exposes per-module `max_repetitions`, `retries`, and `timeout`; `generator/README.md:121` documents default `max_repetitions: 25` and says it may need reduction for buggy devices; `scraper/gosnmp.go:107` uses `BulkWalkAll` for SNMPv2/v3 walks.
210
-- `DataDog/datadog-agent @ 4bb5357ef2350cd91a84df21cfb74f09cb95e8d1`: `pkg/collector/corechecks/snmp/internal/checkconfig/config.go:60` documents the too-high repetition risk; `config.go:64` defaults `bulk_max_repetitions` to 10; `config.go:141` separates OID batch size from table bulk repetitions.
211
-- Official protocol/tool references checked: MikroTik RouterOS SNMP documentation confirms RouterOS exposes IF-MIB, IP-MIB, BRIDGE-MIB, and OID-based interface metrics; Net-SNMP GETBULK documentation confirms `max-repetitions` controls how many repeated OID instances are requested in one response; GoSNMP upstream documentation confirms `BulkWalkAll` retrieves a subtree using GETBULK.
212
-
213
-Open decisions:
214
-
215
-- None. The user decision is recorded below.
216
-
217
-## Implications And Decisions
218
-
219
-1. Root-cause and outcome decision:
220
- - Selection: true root-cause investigation and fix only.
221
- - Evidence: existing logs prove skipped samples, but existing stats do not prove which OID/profile/code path consumes the runtime.
222
- - Implication: temporary instrumentation and local rebuilds are allowed if needed.
223
- - Rejected paths: interval increase, local-only config workaround, partial disabling without root-cause proof, or accepting non-per-second collection.
224
- - Risk accepted: live local Netdata may be rebuilt/restarted during investigation, causing temporary local monitoring gaps.
225
-
226
-2. External validation before implementation:
227
- - Selection: test candidate SNMP request policies outside Netdata before wiring any production code change.
228
- - Evidence: direct Net-SNMP tests already showed that simple `max_repetitions=1` tuning is not sufficient, and Netdata code review showed topology walks use a separate client and unstable map-order table iteration.
229
- - Implication: use read-only external SNMP probes with local Netdata stopped to compare scalar 1-second GET latency while topology-like walks run under different request policies.
230
- - Rejected path: adding or changing Netdata plugin behavior before an external policy proves it can preserve the 1-second metrics SLA.
231
- - Risk accepted: local Netdata may be temporarily stopped during isolated read-only tests; the production router must not be rebooted, reset, or reconfigured.
232
-
233
-3. Close decision:
234
- - Selection: close this SOW as diagnosis-complete with no Netdata implementation.
235
- - Evidence: a single read-only GETBULK request against a previously slow bridge FDB topology root still took about 9.489 seconds after local Netdata was stopped and the router had 10 seconds idle time.
236
- - Implication: spreading topology requests over time cannot fully solve the issue because the first expensive request itself can block for multiple seconds.
237
- - Rejected path: implement Netdata tuning for `max_repetitions`, pacing, short topology timeout, or topology concurrency as a claimed fix in this SOW.
238
- - Final conclusion: this behavior is inherent to RouterOS SNMP for the tested OID class, or at least not externally correctable by Netdata transport policy without avoiding those topology walks.
239
-
240
-## Plan
241
-
242
-1. Move this SOW to current/in-progress.
243
-2. Add targeted instrumentation only if existing stats cannot identify the slow path.
244
-3. Rebuild and install `go.d.plugin` locally using `./build-install-go.d.plugin.sh` when instrumentation or a candidate fix is ready.
245
-4. Validate with live local Netdata and redacted evidence.
246
-5. Update durable artifacts if collector behavior or profile authoring rules change.
247
-
248
-## Execution Log
249
-
250
-### 2026-05-05
251
-
252
-- Created this pending SOW after diagnosis found Netdata-side per-second SNMP collection overruns.
253
-- User clarified that workarounds are not acceptable; only a verified root cause and true per-second SNMP collection are acceptable.
254
-- User authorized temporary logs/instrumentation and rebuilding `go.d.plugin` with `./build-install-go.d.plugin.sh`.
255
-- User clarified that no permanent destructive action is allowed. The production router must not be rebooted, reset, or modified destructively.
256
-- Moved SOW to current/in-progress before code instrumentation.
257
-- Confirmed the separate `snmp_topology` job is not configured per-second; the regular SNMP metric collector still overruns its own 1-second cadence.
258
-- Confirmed existing runtime stats prove about 25 sequential SNMP GET requests and about 410 OIDs per successful sample, but do not identify the exact slow table/OID subset.
259
-- Planned temporary SNMP collector instrumentation that logs only generic timing evidence: profile source, table name, operation name, OID counts, request counts, response counts, duration, and error state.
260
-- Installed the temporarily instrumented `go.d.plugin`; local Netdata restarted cleanly.
261
-- Verified the router and regular metric collector can sustain per-second collection when topology is not walking the device: regular runs were about 150-300 ms with about 24 GET requests and about 406 OIDs.
262
-- Verified the root-cause interaction after a topology refresh:
263
- - `snmp_topology` table walks against the same device took multi-second intervals;
264
- - regular per-second scalar GETs of only 2 OIDs took about 8.8 seconds during the topology walk window;
265
- - the regular job immediately logged skipped 1-second ticks and resumed after about 9 seconds.
266
-- Installed and validated a first candidate fix with a per-endpoint priority gate and topology max-repetitions cap. It did not pass live validation: the affected regular job still skipped about 9 one-second ticks while topology walks were active.
267
-- Temporarily stopped only the local Netdata service and ran direct read-only SNMP timing checks against the redacted device, then restarted local Netdata in the same shell. The router was not modified.
268
-- Confirmed direct scalar GETs and cached-table-equivalent GETs are fast in isolation.
269
-- Confirmed topology-class walks can still take about 8-10 seconds in isolation, including with max-repetitions set to 1. This proves the topology walk class itself is unsafe to run against a device that must sustain 1-second metrics.
270
-- Checked the GoSNMP fork used by this tree. No global or per-target same-device lock exists in the GET/GETBULK send path; walks are synchronous request/response loops, not asynchronous bursts from the client library.
271
-- User changed the local topology cadence to a much longer interval. Follow-up tier-0 checks for the affected node showed the last 120 seconds / 120 points flowing continuously: interface traffic and operational-status points had no empty/partial annotations, regular SNMP stats showed zero walk requests, regular table timing stayed under 300 ms, and Netdata namespace logs had no skip/resume entries for the affected job in the checked recent window.
272
-- User challenged the premature improvement plan and asked whether larger `max_repetitions` had been tested. A direct read-only timing matrix with local Netdata stopped showed the problem is not a generic "topology walks are always slow" condition:
273
- - the legacy ARP table was much faster with larger repetitions: `max_repetitions=25` and `50` completed in under 300 ms, while `1` took several seconds and one first-pass run timed out after 20 seconds;
274
- - bridge/Q-bridge FDB table behavior varied by run order and repetition; one first-pass low-to-high run made `max_repetitions=1` slow, while a repeat run made `max_repetitions=25` slow for the bridge FDB table but not the Q-bridge FDB table;
275
- - most LLDP, interface, STP, VLAN, and small topology tables completed in milliseconds for repetitions from 1 to 50;
276
- - `max_repetitions=100` consistently failed quickly on these tests and is not a safe direction for this device.
277
-- Updated working theory: the root cause is specific topology table walk interactions, especially bridge/FDB and legacy ARP areas, with RouterOS behavior depending on OID root, repetition size, and likely agent/table cache/order state. A final fix must be based on exact slow walk identification and reproduced Netdata-side `max_repetitions` behavior, not generic pacing assumptions.
278
-- Code review found Netdata topology table walk order is not stable: `walkTables` iterates over a Go map of table OIDs. A Netdata topology refresh can therefore hit topology roots in different orders across runs.
279
-- A direct read-only randomized-order reproduction with local Netdata stopped, using `max_repetitions=25` for all topology roots, reproduced the stall class:
280
- - one round spent about 9.5 seconds in the STP port table;
281
- - another round spent about 3.9 seconds in the IP address table followed by about 5.8 seconds in the bridge FDB table;
282
- - the same roots were fast in other rounds.
283
-- Updated root-cause statement: the local gaps are caused by Netdata running best-effort topology walks against the same SNMP endpoint as the 1-second metrics job. RouterOS exhibits order/state-sensitive multi-second stalls on specific topology table roots even with the default `max_repetitions=25`; Netdata's current topology loop has no mechanism to isolate or abort those stalls before they delay the per-second metrics job.
284
-- User requested the next investigation step: prove candidate SNMP request policies outside Netdata before any code is wired. Recorded this as the active decision.
285
-- Built and ran a temporary external GoSNMP harness under `.local/snmp-homework/` using the same GoSNMP module path and replacement version as Netdata. The harness runs a 1-second scalar GET loop while a separate client performs topology-like GETBULK walks. Raw logs remain under `.local/snmp-homework/results/` and are not durable project artifacts.
286
-- Baseline with local Netdata stopped and no topology walks:
287
- - first suite: 20 scalar GET samples, zero errors, zero 1-second SLA violations, max 22 ms;
288
- - low-repetition suite: 15 scalar GET samples, zero errors, zero 1-second SLA violations, max 8 ms;
289
- - edge suite: 15 scalar GET samples, zero errors, zero 1-second SLA violations, max 16 ms.
290
-- External policy matrix results with all topology roots:
291
- - `max_repetitions=25`, no pause: failed. One run had a 9.658 s bridge FDB walk and one scalar GET at 8.711 s; another run had 9.471 s interface walk, 9.579 s bridge FDB walk, and two scalar GET SLA violations.
292
- - `max_repetitions=50`, no pause: failed. Interface and bridge FDB walks were about 9.3 s and 9.6 s; scalar GET loop had three SLA violations, max 9.026 s.
293
- - `max_repetitions=10`, no pause: failed. Interface and bridge FDB walks were about 9.5 s and 9.7 s; scalar GET loop had three SLA violations, max 8.895 s.
294
- - `max_repetitions=5`, no pause: failed. Interface and bridge FDB walks were about 9.3 s and 9.8 s; scalar GET loop had two SLA violations, max 8.974 s.
295
- - `max_repetitions=2`, no pause: failed. Interface, bridge FDB, and legacy ARP walks took about 9.8 s, 9.8 s, and 12.2 s; scalar GET loop had four SLA violations, max 9.034 s.
296
- - `max_repetitions=1`, no pause: failed. Bridge FDB, legacy ARP, and Q-BRIDGE FDB walks took about 9.9 s, 24.8 s, and 9.8 s; scalar GET loop had six SLA violations, max 9.323 s.
297
- - `max_repetitions=1`, 10 ms pause between GETBULK requests: failed. Interface, bridge FDB, legacy ARP, Q-BRIDGE FDB, and STP roots all had multi-second walk times; scalar GET loop had twelve SLA violations, max 9.472 s.
298
- - `max_repetitions=25`, 250 ms pause between GETBULK requests: failed. The walk stretched to about 85.8 s and scalar GET loop had eleven SLA violations, max 9.337 s.
299
- - `max_repetitions=25`, topology timeout 800 ms, zero topology retries: failed. The topology client timed out many roots quickly, but the scalar GET loop still had a max latency around 8.7 s. This was reproduced in a separate edge run, so it was not only contamination from the previous long test.
300
- - `max_repetitions=100`: not viable. The GoSNMP harness received zero PDUs for every topology root and therefore collected no topology data; prior Net-SNMP direct tests also showed this direction failing quickly.
301
-- External reduced-root test:
302
- - Removing the obvious bridge/FDB/ARP roots was still not safe. A reduced set including interface, IP, STP, VLAN, and LLDP roots failed twice: one run had an `ifTable` walk around 3.1 s and scalar GET max 3.67 s; the next had `ifTable` and STP walks around 9.5 s each and scalar GET max 9.212 s.
303
-- Updated conclusion after external homework: no tested transport policy that still walks the topology roots preserves the 1-second scalar GET SLA on this RouterOS device. Smaller bulks, larger bulks, per-bulk pauses, and short topology-side timeouts all fail. The only policies that can preserve per-second metrics are policies that avoid issuing unproven topology walks while 1-second collection is required, or policies that can learn and quarantine unsafe walks after causing initial damage.
304
-- Searched current public MikroTik/RouterOS evidence for known SNMP issues:
305
- - Official MikroTik RouterOS SNMP documentation says SNMP gathers data from other RouterOS services, can log `timeout while waiting for program` / `SNMP did not get OID data within expected time`, may deny requests for that service for a while, and says slow/busy services should often be skipped by monitoring tools.
306
- - Official MikroTik RouterOS SNMP documentation lists IF-MIB, IP-MIB, and BRIDGE-MIB as RouterOS-supported MIBs, matching the class of topology roots involved in this investigation.
307
- - Public MikroTik forum reports include RouterOS 7.x SNMP polling times of 200-220 seconds on a CCR2004-16G-2S+ with RouterOS 7.14.1, intermittent SNMP refusal on RouterOS 7.14.2 with low CPU, and very slow SNMP walks on MikroTik enterprise interface-stat OIDs.
308
- - Checkmk has an old compatibility fix for a broken MikroTik RouterOS bulk-walk implementation in RouterOS v6.22 where consecutive duplicate OIDs could be returned.
309
- - RouterOS 7.22 public changelog evidence mentions an SNMP fix where bulk walk might skip the first OID. This is not the same as the observed stall, but it shows RouterOS bulk-walk behavior has had recent fixes.
310
- - No public source found an exact named MikroTik bug for "BRIDGE-MIB/FDB/IF-MIB GETBULK stalls concurrent scalar SNMP GETs for 8-10 seconds." The evidence supports a known RouterOS SNMP slow/busy-service class, not a public exact-match defect ID.
311
-- User asked to test whether the behavior is internal RouterOS SNMP rate limiting rather than slow OID processing. Planned external read-only tests with long pauses between topology GETBULK requests and between topology roots. Prediction: if simple rate limiting is the cause, long request spacing should remove the 8-10 second stalls and scalar GET SLA violations.
312
-- Rate-limit hypothesis quick check:
313
- - Stopped local Netdata, waited 10 seconds, then ran a single read-only walk of one previously slow bridge FDB topology root with `max_repetitions=25`. Result: 192 rows, exit 0, duration 9.528 seconds.
314
- - Stopped local Netdata again, waited 10 seconds, then ran one single GETBULK request against the same root with `max_repetitions=25`, not a full walk. Result: 25 rows, exit 0, duration 9.489 seconds.
315
- - Conclusion: the observed delay is not explained by simple recent-request burst rate limiting. A single request after an idle gap can still be delayed by about 9.5 seconds. This does not rule out RouterOS internal serialization, internal table refresh, or per-OID/service throttling, but it rules out "spread previous topology calls and the first expensive request becomes fast" for this root.
316
-- User concluded this is inherent to RouterOS and asked to close the SOW. No source code, router configuration, or durable operational configuration was changed as part of closing.
317
-
318
-## Validation
319
-
320
-Acceptance criteria evidence:
321
-
322
-- Root cause evidence complete.
323
-- Netdata can sustain per-second metrics when topology does not issue unsafe RouterOS topology walks against the same endpoint: after topology cadence was moved away, tier-0 120-second / 120-point queries for the affected interface traffic and operational-status contexts were continuous, and Netdata logs had no affected-job skip/resume entries in the checked window.
324
-- Full topology walking while preserving 1-second metrics is not achievable for this device with the tested request policies: `max_repetitions` values 1, 2, 5, 10, 25, and 50 all produced scalar GET SLA violations during topology walks; pauses and short topology-side timeouts also failed.
325
-- A single GETBULK request after a 10-second idle gap still took about 9.489 seconds, proving the failure is not simple burst rate limiting.
326
-- The user accepted the conclusion that this is inherent RouterOS behavior and requested SOW closure.
327
-
328
-Tests or equivalent validation:
329
-
330
-- Diagnostic validation completed:
331
- - tier-0 120-second / 120-point queries showed repeated all-null runs for interface traffic/status;
332
- - Netdata namespace logs showed skip/resume messages for the affected SNMP job;
333
- - SNMP error stats stayed at zero in the checked window;
334
- - RouterOS resource/profile data did not show router overload.
335
-- External Net-SNMP and GoSNMP timing validation completed with local Netdata stopped:
336
- - baseline scalar GET loop had zero errors and zero 1-second SLA violations;
337
- - topology-like GETBULK walks reproduced 8-10 second stalls and scalar GET SLA violations;
338
- - low, default, and higher `max_repetitions` values were tested and failed;
339
- - request pacing and short topology-side timeouts were tested and failed;
340
- - one single GETBULK request after an idle gap still took about 9.5 seconds.
341
-
342
-Real-use evidence:
343
-
344
-- Local Netdata MCP and local Netdata `systemd-journal` function were used.
345
-- Direct read-only SNMP commands were run with local Netdata stopped to isolate device behavior without concurrent Netdata polling. Local Netdata was restarted immediately after the test.
346
-- Temporary instrumentation and candidate code changes were removed before closure; source code remained unchanged at close.
347
-
348
-Reviewer findings:
349
-
350
-- No implementation was shipped, so no code review was required.
351
-- External references were checked for context: MikroTik RouterOS SNMP documentation, public MikroTik forum reports, Checkmk RouterOS SNMP bulk-walk compatibility note, Prometheus `snmp_exporter`, and Datadog Agent SNMP configuration defaults.
352
-
353
-Same-failure scan:
354
-
355
-- Same-failure class was checked through public MikroTik reports and open-source SNMP collector references.
356
-- Public evidence supports RouterOS SNMP slow/busy-service and bulk-walk issue classes, but no exact public defect ID was found for the specific bridge/FDB/interface 8-10 second stall observed here.
357
-
358
-Sensitive data gate:
359
-
360
-- Raw SNMP secrets, trap communities, exact private endpoints, contact/location strings, claim IDs, and SSH details were not written to this SOW.
361
-- Private endpoint evidence is redacted as `[PRIVATE_ENDPOINT]`.
362
-
363
-Artifact maintenance gate:
364
-
365
-- AGENTS.md: no update needed. This SOW did not change repository workflow, responsibility boundaries, or project-wide guardrails.
366
-- Runtime project skills: no update needed. The work produced a device-specific diagnosis, not a new reusable collector-authoring rule.
367
-- Specs: no update needed. No Netdata product behavior or public collector contract changed in this SOW.
368
-- End-user/operator docs: no update needed in this SOW. The user explicitly closed the investigation as inherent RouterOS behavior without requesting a Netdata operator guidance change.
369
-- End-user/operator skills: no update needed. No public/operator AI skill behavior changed.
370
-- SOW lifecycle: status updated to `completed`; file will be moved from `.agents/sow/current/` to `.agents/sow/done/`.
371
-
372
-Specs update:
373
-
374
-- No spec update. There was no shipped behavior change and no new Netdata contract selected.
375
-
376
-Project skills update:
377
-
378
-- No project skill update. The investigation used existing collector and SNMP guidance; no reusable workflow rule changed.
379
-
380
-End-user/operator docs update:
381
-
382
-- No end-user/operator docs update. The result is a closed local diagnosis; documentation of RouterOS-specific topology limitations is outside this SOW and was not requested.
383
-
384
-End-user/operator skills update:
385
-
386
-- No end-user/operator skill update. No public skill behavior changed.
387
-
388
-Lessons:
389
-
390
-- A single expensive RouterOS topology GETBULK request can take about 9.5 seconds after an idle gap, so request pacing is not sufficient to guarantee 1-second SNMP metrics.
391
-- `max_repetitions` tuning is device/OID/order dependent on RouterOS and cannot be treated as a generic fix for this failure class.
392
-- External request-policy testing should precede Netdata implementation when the suspected failure could be inherent to the SNMP agent.
393
-
394
-Follow-up mapping:
395
-
396
-- No follow-up SOW is required by the user. The SOW is closed as diagnosis-complete.
397
-
398
-## Outcome
399
-
400
-Completed as diagnosis-only.
401
-
402
-Final conclusion:
403
-
404
-- The per-second metric gaps were caused by RouterOS SNMP behavior when topology-class OIDs are queried.
405
-- Netdata regular SNMP collection can sustain per-second metrics when those unsafe topology walks are not issued.
406
-- Topology collection is a separate Netdata loop and client, but it still targets the same RouterOS SNMP agent.
407
-- A single bridge/FDB topology GETBULK request after a 10-second idle gap took about 9.489 seconds, so the failure is not simple request burst rate limiting.
408
-- No tested Netdata-side transport policy preserved the 1-second SLA while still walking the tested topology roots.
409
-- No Netdata implementation was made or required in this SOW.
410
-
411
-## Lessons Extracted
412
-
413
-- Verify device-agent behavior externally before implementing collector changes when live evidence suggests an upstream SNMP agent stall.
414
-- Do not assume smaller `max_repetitions` improves RouterOS behavior; for this device/OID class, values from 1 through 50 all failed in at least one tested scenario.
415
-- A topology loop can be architecturally separate from metrics collection and still interfere through the monitored device's own SNMP agent.
416
-
417
-## Followup
418
-
419
-None.
420
-
421
-## Regression Log
422
-
423
-None yet.
424
-
425
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0009-20260502-project-writing-collectors-skill.md
deleted
-387
@@ -1,387 +0,0 @@
1
-# SOW-0009 - project-writing-collectors skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: skill shipped (SKILL.md 516 lines after structural rewrite). PR #22386 open against netdata/netdata master with three commits: pre-existing SOW lifecycle update (`64754ad4ea`), first-draft skill preserved as baseline (`9fdf581a86`, 313 lines, routing-and-pointers oriented), structural rewrite (`abe0b77ea2`, +395/-192, mental-model and data-type/domain centered). AGENTS.md skill index entry added in the first-draft commit. SOW close lands in a separate third commit per user-approved split from the rewrite.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Create a runtime project skill `project-writing-collectors` that orients an AI assistant arriving cold at any Netdata data-collection task: tells the assistant *what canonical documents already exist*, *when to read each*, and *what is at stake* if collector authoring conventions are violated.
14
-
15
-The skill is a gateway. It must never duplicate canonical documentation. The repo already owns the deep references (NIDL framework, plugin frameworks, profile format, plugin protocol, DYNCFG, functions, streaming, integrations pipeline) — the skill points to them.
16
-
17
-The skill exists because, without it, every new collector task forces the user to manually re-teach: where docs live, what NIDL is, what frameworks v1/v2 are, what is mandatory in metadata.yaml / config_schema.json / health.d, that vnodes exist for remotely-monitored systems, that SNMP uses profiles, that no metric should default to zero, that hot paths must not log or allocate, etc. Each repetition is a tax on the user's time and a risk for incorrect output.
18
-
19
-### User Request
20
-
21
-Quoted from chat:
22
-
23
-> "I want us to work on a new skill: project-writing-collectors with description: best practices for Netdata collectors - read this before adding data collection plugins or modules to Netdata"
24
-
25
-> "let an assistant understand what it is dealing with"
26
-
27
-> "We need a balance. The assistants must get enough information to understand if and when they need to read additional documents, and grasp what is at stake when working with collectors. Best practices, Bad practices, Pointers for additional documentation."
28
-
29
-Constraints from the user:
30
-
31
-- General rules for ALL plugins (per-plugin rules go elsewhere — possibly per-plugin skills later).
32
-- Not an inventory (so go.d module names are out — categories only).
33
-- Not a comprehensive guide (the repo already has those).
34
-- The skill is *live* and may be updated when gaps are found, with user permission.
35
-
36
-### Assistant Understanding
37
-
38
-Facts:
39
-
40
-- The repo already owns substantial collector documentation: `docs/NIDL-Framework.md` (442 lines), `src/go/BEST-PRACTICES.md` (387), `src/go/COLLECTOR-LIFECYCLE.md` (1209), `src/plugins.d/README.md` (909), `src/plugins.d/DYNCFG.md` (468), `src/plugins.d/FUNCTION_UI_REFERENCE.md` (1714), `src/go/plugin/go.d/collector/snmp/profile-format.md` (2046), `src/go/plugin/ibm.d/framework/README.md` (153), plus a 56-line `src/go/plugin/go.d/docs/how-to-write-a-collector.md` for go.d.
41
-- A landing-page pattern already exists in the repo: `src/go/AGENTS.md` and `src/go/CLAUDE.md` (17 lines each, content-equal, scoped to IBM.D plugin) — short router with 5 rules. `src/go/plugin/ibm.d/AGENTS.md` (145 lines) is a deeper IBM.D checklist.
42
-- AGENTS.md mandates collector consistency between code, metadata.yaml, config_schema.json, stock conf, health.d, README.
43
-- ~30 plugins exist across C, Go, Rust, Python, Bash, eBPF.
44
-- 12 recurring bad-practice patterns identified with file:line evidence (see Analysis).
45
-- Existing project skills under `.agents/skills/` range from 162 (graphql-audit) to 508 (pr-reviews) lines; average ~330.
46
-- SOW directories were empty at SOW-creation time; this is SOW-0009 in numbering despite AGENTS.md text mentioning SOW-0003.
47
-
48
-Inferences:
49
-
50
-- A single SKILL.md is appropriate based on existing skill sizes and the orientation-only scope.
51
-- Updating-this-skill must be an explicit footer rule, since the user wants it to be live.
52
-- Audit rules and authoring rules largely overlap — splitting into two skills risks duplication; embedding both in one skill keeps the merge gate close to the authoring guidance.
53
-
54
-Unknowns:
55
-
56
-- Whether a separate `project-auditing-collectors` skill is desirable, or audit lives inside this skill (decision needed).
57
-- Whether to create companion docs in the skill dir for true gaps (vnodes, error handling, labels, logging conventions), or leave gaps as 1-2 line inline summaries plus follow-up SOWs (decision needed).
58
-
59
-### Acceptance Criteria
60
-
61
-- An assistant given a new collector task and only this skill must be able to:
62
- - identify which plugin tree the work belongs in;
63
- - locate every canonical doc relevant to that work;
64
- - know the non-negotiable rules (no zero defaults, no log spam, no per-iteration alloc/reconnect, no missing metadata.yaml/health.d, vnodes for remote, profiles for SNMP);
65
- - know the audit checklist before merge.
66
-- Verification: walk through 4 self-tests with the skill content visible:
67
- 1. New go.d module → routes to BEST-PRACTICES.md, COLLECTOR-LIFECYCLE.md, NIDL, how-to-write-a-collector.md, integrations/templates.
68
- 2. New SNMP profile → routes to profile-format.md.
69
- 3. New C external plugin → routes to plugins.d/README.md, src/collectors/README.md.
70
- 4. New ibm.d module → routes to ibm.d/framework/README.md, ibm.d/AGENTS.md.
71
-- Each of the 12 bad-practice patterns must be flagged either in the "non-negotiables" or "audit checklist" sections of the skill.
72
-- AGENTS.md "Project Skills Index" must list the new skill.
73
-
74
-## Analysis
75
-
76
-### Canonical Documents That Already Exist
77
-
78
-(verified to exist via Bash on 2026-05-02)
79
-
80
-| Topic | Path | Lines |
81
-|---|---|---|
82
-| NIDL framework | docs/NIDL-Framework.md | 442 |
83
-| go.d collector authoring (V2) | src/go/plugin/go.d/docs/how-to-write-a-collector.md | 56 |
84
-| go.d best practices | src/go/BEST-PRACTICES.md | 387 |
85
-| go.d collector lifecycle | src/go/COLLECTOR-LIFECYCLE.md | 1209 |
86
-| go.d plugin overview | src/go/plugin/go.d/README.md | (verified, lines unread) |
87
-| ibm.d framework | src/go/plugin/ibm.d/framework/README.md | 153 |
88
-| ibm.d landing/checklist | src/go/plugin/ibm.d/AGENTS.md | 145 |
89
-| ibm.d plugin overview | src/go/plugin/ibm.d/README.md | (verified) |
90
-| go.d/ibm.d landing | src/go/AGENTS.md = src/go/CLAUDE.md | 17 |
91
-| go.d agent framework | src/go/plugin/agent/README.md | (verified) |
92
-| External plugin protocol (PLUGINSD) | src/plugins.d/README.md | 909 |
93
-| Collector privileges/types | src/collectors/README.md | (verified) |
94
-| SNMP profile format | src/go/plugin/go.d/collector/snmp/profile-format.md | 2046 |
95
-| DYNCFG protocol | src/plugins.d/DYNCFG.md | 468 |
96
-| DYNCFG (developer corner) | docs/developer-and-contributor-corner/dyncfg.md | (verified) |
97
-| Functions reference | src/plugins.d/FUNCTION_UI_REFERENCE.md | 1714 |
98
-| Functions developer guide | src/plugins.d/FUNCTION_UI_DEVELOPER_GUIDE.md | (verified) |
99
-| Streaming & replication | src/streaming/README.md | (verified) |
100
-| Streaming parent clusters | src/streaming/PARENT-CLUSTERS.md | (verified) |
101
-| Health alerts (reference) | src/health/REFERENCE.md | (verified) |
102
-| Health alerts (overview) | src/health/README.md | (verified) |
103
-| Alert config ordering | src/health/alert-configuration-ordering.md | (verified) |
104
-| Overriding stock alerts | src/health/overriding-stock-alerts.md | (verified) |
105
-| Claim / registration | src/claim/README.md | (verified) |
106
-| Integrations pipeline | integrations/README.md | (verified) |
107
-| Integration templates | integrations/templates/README.md | (verified) |
108
-| Dynamic-configuration UI | docs/netdata-agent/configuration/dynamic-configuration.md | (verified) |
109
-| Replication of past samples | docs/observability-centralization-points/metrics-centralization-points/replication-of-past-samples.md | (verified) |
110
-| charts.d.plugin (legacy) | src/collectors/charts.d.plugin/README.md | (legacy) |
111
-| python.d.plugin (legacy) | src/collectors/python.d.plugin/README.md | (legacy) |
112
-
113
-### Documentation Gaps
114
-
115
-Topics where the repo lacks a canonical doc and the skill must either inline guidance or track a follow-up SOW:
116
-
117
-- Netdata labels (host/chart/instance) — no canonical reference.
118
-- Configuration override hierarchy (DYNCFG > /etc/netdata > stock > internal defaults) — scattered.
119
-- Memory allocation conventions (mallocz/freez/strdupz) — only in libnetdata code comments.
120
-- nd_log conventions (levels, throttling) — scattered across libnetdata.
121
-- Error handling conventions for collectors — none unified.
122
-- Vnode registration — code only, no public doc.
123
-- Chart/dimension definitions — examples in collectors only.
124
-- Plugin update interval / cadence guidance — scattered.
125
-- Testing patterns for collectors — no unified doc.
126
-- netipc library — canonical lives at github.com/netdata/plugin-ipc, not linked from this repo.
127
-
128
-### Recurring Bad Practices (file:line evidence)
129
-
130
-| # | Pattern | Example |
131
-|---|---|---|
132
-| 1 | Default-to-zero on missing data | src/collectors/proc.plugin/proc_net_dev.c:782 (TODO comment admits the bug) |
133
-| 2 | Log spam in iteration loops | ebpf.plugin (commit bde8262e33) |
134
-| 3 | Allocations in collection loop | src/go/plugin/go.d/collector/ap/collect.go:53 |
135
-| 4 | Reconnects per iteration | SNMP topology pre-hardening |
136
-| 5 | Vague error context | many go.d collectors return raw err with no wrap |
137
-| 6 | Silent fallbacks | src/go/plugin/go.d/collector/mysql/mysqlfunc/error_info.go (fallbackTable) |
138
-| 7 | Hardcoded options without DYNCFG | many SNMP/timeout defaults |
139
-| 8 | Missing vnode support | refactor commit 4245df367f |
140
-| 9 | Metrics shipped without metadata.yaml/health.d | 4 metadata gaps across 133 collectors |
141
-| 10 | Ignored syscall return codes | systemd-journal NULL guard commit b455bbe1c |
142
-| 11 | SNMP collectors without profiles | older topology code |
143
-| 12 | Blocking inside collection loop | apps.plugin commit 6084e3f98b |
144
-
145
-### External Pattern Reference
146
-
147
-- Telegraf input plugin guide (189 lines): minimal interface spec, convention over walkthrough.
148
-- OTel Collector CONTRIBUTING.md (470 lines): prescriptive PR shape, named audiences, defers RFC detail.
149
-- Datadog integrations README (58 lines): defers all detail to external docs site.
150
-- Prometheus exporter guides (~686 lines combined): hands-on, example-first.
151
-- Synthesis: lead with role/audience, links table, do/don't, defer deep detail.
152
-
153
-### Existing Project Skills (size reference)
154
-
155
-| Skill | Lines |
156
-|---|---|
157
-| graphql-audit | 162 |
158
-| sonarqube-audit | 190 |
159
-| coverity-audit | 474 |
160
-| pr-reviews | 508 |
161
-
162
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
163
-
164
-Status: needs-user-decision
165
-
166
-Problem / root-cause model:
167
-
168
-- AI assistants approaching Netdata data collection do not know which canonical docs exist (NIDL, frameworks, profiles, plugins.d protocol, DYNCFG, functions, streaming, integrations pipeline). Without orientation they: invent metric grouping, miss vnodes, log-spam in hot paths, default missing data to zero, miss metadata.yaml/health.d/config_schema, hardcode options, write SNMP without profiles, allocate per-iteration, reconnect per iteration, ignore syscall return codes. Evidence: 12 recurring patterns documented with file:line above. Without a router skill the user must reteach all of this in every new session.
169
-
170
-Evidence reviewed:
171
-
172
-- 25+ canonical in-repo docs verified to exist (Analysis section).
173
-- src/go/AGENTS.md / src/go/CLAUDE.md — existing 17-line landing page pattern, IBM.D-scoped.
174
-- src/go/plugin/go.d/docs/how-to-write-a-collector.md — go.d-specific 56-line guide.
175
-- 12 bad-practice patterns from subagent investigation.
176
-- External patterns from Telegraf, OTel, Datadog, Prometheus.
177
-- Existing project skills (sizing reference).
178
-
179
-Affected contracts and surfaces:
180
-
181
-- New file: `.agents/skills/project-writing-collectors/SKILL.md`.
182
-- AGENTS.md: add entry under "Runtime input project skills" in the Project Skills Index.
183
-- No code changes; no spec changes; no metadata.yaml/config_schema changes.
184
-- The skill will reference canonical docs by relative path; renames in the future require skill update.
185
-
186
-Existing patterns to reuse:
187
-
188
-- Frontmatter format from existing project skills (`name`, `description`, `type`).
189
-- Router shape from src/go/AGENTS.md (short, numbered rules, links table).
190
-- Length range from existing project skills (160-510 lines).
191
-- File:line evidence style from coverity-audit / sonarqube-audit (for the bad-practices section).
192
-
193
-Risk and blast radius:
194
-
195
-- Skill bloat: tries to cover everything; assistants stop reading. Mitigation: hard cap on length, defer all deep content to canonical docs.
196
-- Drift: if BEST-PRACTICES.md, NIDL-Framework.md, or profile-format.md change paths, skill links break. Mitigation: explicit footer rule that PRs touching collectors must update the skill if conventions or doc paths change.
197
-- Coverage illusion: a 12-item checklist does not guarantee an audit catches a bug. Mitigation: each bad-practice row carries file:line evidence so reviewers verify by example, not by checkbox.
198
-- Stale plugin landscape: plugins added/removed without updating skill. Mitigation: include the skill in the collector-consistency rule already in AGENTS.md.
199
-
200
-Sensitive data handling plan:
201
-
202
-- The skill ships in a public repository. It must contain no customer names, no private endpoints, no credentials, no internal tooling references. Bad-practice file:line evidence cites public source code only. No issue.
203
-
204
-Implementation plan:
205
-
206
-(Awaiting user decisions before finalizing.)
207
-
208
-1. Author SKILL.md based on the structure decided in "Open decisions".
209
-2. Register the skill in AGENTS.md → Project Skills Index → Runtime input skills.
210
-3. Self-test with the 4 routing scenarios listed in Acceptance Criteria.
211
-4. Run the 12 bad-practice patterns against the SKILL.md to confirm each is flagged.
212
-5. Open a follow-up SOW (or follow-ups, plural) for each canonical-doc gap that the user wants the skill to point at but no canonical doc yet exists.
213
-
214
-Validation plan:
215
-
216
-- 4 self-test routing walkthroughs (above).
217
-- 12-pattern coverage verification (above).
218
-- User review of structure before implementation begins.
219
-- After implementation, re-walkthrough with a fresh subagent that has not seen the design conversation: does it route correctly?
220
-
221
-Artifact impact plan:
222
-
223
-- AGENTS.md: add new skill entry under Runtime input project skills (single trigger line + 2-3 lines of "use when").
224
-- Runtime project skills: this SOW *creates* the skill.
225
-- Specs: no spec change expected (not a behavioral change).
226
-- End-user/operator docs: none affected.
227
-- End-user/operator skills: none affected.
228
-- SOW lifecycle: this SOW transitions pending→current after decisions, current→done on completion. Gap follow-ups (vnodes doc, error-handling doc, labels doc, logging conventions doc, etc.) tracked as separate SOWs in pending/.
229
-
230
-Open decisions: resolved 2026-05-02.
231
-
232
-1. Single skill, audit checklist embedded (1A).
233
-2. Full plugin coverage in v1 (2A).
234
-3. Gap topics handled inline as 1-2 line guidance with pointers; canonical doc gaps tracked as follow-up SOWs (3A).
235
-4. Plugin landscape embedded in SKILL.md (4A).
236
-5. Legacy plugins (charts.d, python.d) included with "do not add new modules" marker (5A).
237
-6. Bad-practice file:line evidence kept as nudges, not enforcement (6A); reframed away from "audit gate" toward "past pain looked like this".
238
-7. Length target 300-500 lines; manifesto framing (prose, trust the reader, no MUST/MANDATORY/NEVER), summary DOs and DON'Ts per topic, no lengthy code examples. Draft came in at 242 lines — lean by design, will expand on user request if specific sections feel thin.
239
-8. Skill description tightened to "best practices + orientation" framing (8B), with broad trigger keywords for collector/plugin/module/integration/data-collection work.
240
-
241
-Late addition by user: a function schema JSON file (`src/plugins.d/FUNCTION_UI_SCHEMA.json`) is the contract for any collector that exposes a function. Added to the consistency-sync set in the Documentation section of the skill, and called out in the Functions topic with pointers to FUNCTION_UI_DEVELOPER_GUIDE.md and FUNCTION_UI_REFERENCE.md.
242
-
243
-## Implications And Decisions
244
-
245
-All open decisions (1-8) resolved 2026-05-02 — see Pre-Implementation Gate "Open decisions" section for the resolved set.
246
-
247
-User audit on the first draft prompted a comprehensive rewrite that re-centered the skill on:
248
-
249
-- mental model first (research discipline, cross-project comparison, gaps-are-data, obsoletion as a truthfulness principle, IDs as contracts);
250
-- framework-agnostic best practices ordered by impact;
251
-- five dashboard-shaping mechanisms (NIDL, SNMP profiles, statsd `synthetic_charts`, OTEL per-metric YAML mappings, Prometheus deterministic exposition);
252
-- production-quality criteria + 21-item pre-PR checklist;
253
-- plugin landscape demoted to reference;
254
-- per-data-type chapter (metrics, logs, live snapshots, topology, netipc enrichment);
255
-- per-domain common practices (DBs + query Functions, network/SNMP + topology Functions, containers + netipc enrichment, web servers + access-log Functions, flow protocols).
256
-
257
-Two corrections during review:
258
-
259
-- A comparative claim about Netdata's per-series cost (vs. other monitoring systems) was caught and removed before commit. Such comparisons are out of scope for an in-repo skill whose audience is assistants working on the codebase.
260
-- Obsoletion was originally bundled under cardinality. Separated: obsoletion is now §1.5 (truthfulness principle, applies at any cardinality) and cardinality bounding is §2.5 (`max_*` + selectors mandatory, with three upstream-data-shape sub-cases).
261
-
262
-## Plan
263
-
264
-Plan executed:
265
-
266
-1. Authored first draft per resolved decisions (1A/2A/3A/4A/5A/6A/8B + 7 manifesto framing). Committed as `9fdf581a86`.
267
-2. Registered skill in AGENTS.md → Project Skills Index → Runtime input skills (same commit).
268
-3. User audit identified structural imbalance (over-indexed on entry-points). Restructured into 9 sections, with mental model and best practices leading and the framework reference demoted.
269
-4. Three parallel research subagents fetched OTEL / statsd / Prometheus mapping references with file:line evidence — grounded the dashboard-shaping section in source rather than prior knowledge.
270
-5. Domain-pattern verification (mysql `mysqlfunc/top_queries.go`, postgres `func_top_queries.go` / `func_router.go`, snmp_topology `func_topology*.go`, cgroups netipc server, log2journal) confirmed common-practices descriptions before writing.
271
-6. Two follow-up corrections during user review: removed unfair comparative claim; separated obsoletion from cardinality bounding.
272
-7. Rewrite committed as `abe0b77ea2`. PR #22386 opened against netdata/netdata master.
273
-
274
-## Execution Log
275
-
276
-### 2026-05-02 — investigation
277
-
278
-- Investigation completed via 4 parallel subagents (in-repo doc inventory, plugin landscape, recurring bad-practice patterns, external authoring-guide structures).
279
-- 25+ canonical docs verified to exist on disk.
280
-- Critical missed doc found: `src/go/plugin/go.d/docs/how-to-write-a-collector.md` (subagent A miss, recovered by direct grep).
281
-- 12 bad-practice patterns collected with file:line evidence.
282
-- 4 external project authoring guides analyzed for structure inspiration.
283
-- SOW filed; awaiting structure decisions.
284
-
285
-### 2026-05-02 — implementation and rewrite
286
-
287
-- First draft authored per resolved decisions; 313 lines; committed as `9fdf581a86`.
288
-- AGENTS.md updated with `project-writing-collectors` entry under Runtime input project skills.
289
-- User audit identified structural imbalance (skill over-indexed on entry-points, under-indexed on holistic data-collection thinking).
290
-- Three parallel research subagents fetched OTEL / statsd / Prometheus mapping references — used to ground the §3 dashboard-shaping section in real file:line evidence rather than prior knowledge.
291
-- Domain-pattern verification (mysql, postgres, snmp_topology, cgroups, log2journal) confirmed common-practices descriptions before writing §7.
292
-- Rewrite committed as `abe0b77ea2` (516 lines, +395 / -192 vs first draft).
293
-- User review caught and addressed: (a) a comparative-against-Netdata claim in §1.9 — removed; (b) obsoletion mixed into cardinality bounding — separated, promoted to §1.5 as a truthfulness principle.
294
-- PR #22386 opened against netdata/netdata master.
295
-- SOW close lands in a separate third commit per user-approved split.
296
-
297
-## Validation
298
-
299
-**Acceptance criteria evidence.** All four routing self-tests from §Acceptance Criteria are satisfied by the rewrite:
300
-
301
-1. New go.d module → §5.2 routing-by-task table + §5.3 V1/V2 reality check + ping V2 reference + `how-to-write-a-collector.md` pointer.
302
-2. New SNMP profile → §3.2 SNMP profiles + `profile-format.md` pointer.
303
-3. New external C plugin → §5.2 routing + `plugins.d/README.md` + §5.4 internal-C/PLUGINSD section.
304
-4. New ibm.d module → §5.2 routing + §5.4 ibm.d entry + `ibm.d/AGENTS.md` pointer.
305
-
306
-Each of the 12 bad-practice patterns from §Analysis is flagged in the rewrite, distributed across §1 (mental model), §2 (best practices), §3 (dashboard shaping), §4 (production-quality criteria + checklist) — reframed as past-pain context rather than a single audit-checklist column.
307
-
308
-**Tests / equivalent validation.** The skill is documentation, not code — validated by content review and routing walkthrough.
309
-
310
-**Real-use evidence.** Skill description triggers on the keywords specified in the user's request (collector, plugin, module, NetFlow/sFlow/IPFIX, OTEL, topology, SNMP profile, statsd, Prometheus scraping, Functions). Frontmatter description updated in the rewrite commit to enumerate all five dashboard-shaping mechanisms so discovery covers the full surface.
311
-
312
-**Reviewer findings and how handled:**
313
-
314
-- Structural imbalance after the first draft → addressed by full rewrite (Execution Log).
315
-- Unfair comparative claim about Netdata vs. other monitoring systems in §1.9 → removed; reframed in operational-waste terms.
316
-- Obsoletion conceptually misplaced under cardinality → separated; obsoletion is §1.5 (truthfulness, any cardinality), cardinality bounding is §2.5 (`max_*` + selectors + upstream-data-shape sub-cases).
317
-- `max_*` + selectors must be coupled, with three upstream-data-shape sub-cases ("Other" bucket / push selector upstream / surface app-side aggregations) → added to §2.5.
318
-
319
-**Same-failure search.** Reviewed the rewrite for other comparative claims about Netdata vs. alternatives — only §1.9 had the bad framing. Other sections that mention third-party projects (§1.6 cross-project comparison, §1.7 spec ambiguity, §2.1 testing, §3.5 Prometheus mapping) describe them neutrally as fixture sources or upstream-shape examples.
320
-
321
-**Artifact maintenance gate:**
322
-
323
-- AGENTS.md → updated (skill index entry under Runtime input project skills; commit `9fdf581a86`).
324
-- Runtime project skills → created (the new skill is the artifact).
325
-- Specs → no spec change. Skill is orientation; it does not change collector behavior, public APIs, schemas, alerting semantics, or operational guarantees.
326
-- End-user / operator docs → none affected. Audience is AI assistants, not end users or operators.
327
-- End-user / operator skills → none affected. The skill is a runtime input skill; it does not feed into output/reference skills under `docs/netdata-ai/skills/` or `src/ai-skills/`.
328
-- SOW lifecycle → this commit closes the SOW (status `completed`) and moves it to `.agents/sow/done/`. PR #22386 ends up with three commits (SOW lifecycle update + first-draft skill + rewrite) plus this lifecycle close.
329
-
330
-**SOW status / directory consistency.** Status `completed` → file moves to `.agents/sow/done/`.
331
-
332
-**Spec update or specific reason no spec update was needed.** Not needed — see Artifact maintenance gate.
333
-
334
-**Project skill update or specific reason no skill update was needed.** This SOW *creates* the project skill; AGENTS.md skill index updated.
335
-
336
-**End-user/operator docs update or evidence-backed reason none affected.** None affected — see Artifact maintenance gate.
337
-
338
-**End-user/operator skill update or evidence-backed reason none affected.** None affected — see Artifact maintenance gate.
339
-
340
-**Lessons extracted.** See Lessons Extracted section below.
341
-
342
-**Follow-up mapping.** See Followup section below.
343
-
344
-## Outcome
345
-
346
-Skill shipped at `.agents/skills/project-writing-collectors/SKILL.md` (516 lines). Indexed in AGENTS.md. PR #22386 open against netdata/netdata master.
347
-
348
-Coverage:
349
-
350
-- Mental model: 11 numbered principles.
351
-- Best practices: 10 directives ordered by impact.
352
-- Dashboard shaping: 6 mechanisms (NIDL, SNMP profiles, statsd `synthetic_charts`, OTEL mappings, Prometheus exposition, chart priorities).
353
-- Production-quality criteria: 7 + 21-item pre-PR checklist.
354
-- Plugin landscape: 18 plugin families.
355
-- Data types: 5 (metrics, logs, live snapshots, topology, netipc enrichment).
356
-- Common practices: 7 collector domains.
357
-- Canonical pointers: 30 entries.
358
-
359
-## Lessons Extracted
360
-
361
-1. **Comparative claims about Netdata are out of scope for in-repo skills.** A statement framing Netdata's per-series cost as worse than alternatives was both factually wrong (Netdata is more efficient on cardinality, with automated protection built in) and rhetorically inappropriate for an internal skill whose audience is assistants working on the codebase. Future skill content must teach assistants to design well, not position Netdata against other systems — in either direction.
362
-
363
-2. **Obsoletion is a truthfulness concern, not a cardinality concern.** The first-draft framing bundled them, which suggests obsoletion is only relevant at high cardinality. The principle applies even at one entity total: when the collector knows an entity is gone, the dashboard must reflect that. Future skill or doc content must keep these separate.
364
-
365
-3. **`max_*` and selectors must be coupled.** A cap alone silently truncates the wrong set; selectors alone don't protect against runaway. The skill must teach this as a combined directive, not two independent options.
366
-
367
-4. **Where to filter depends on what the application exposes.** Three upstream cases must be distinguished: app exposes everything (collector caps + adds an "Other" aggregation), app supports cherry-picking (push selector upstream), app exposes aggregations natively (surface them as additional charts). Without this guidance assistants default to "cap and drop", which loses information.
368
-
369
-5. **Research before describing internal mechanics.** The §3 dashboard-shaping section was grounded in real file:line evidence from the OTEL plugin, statsd plugin, and Prometheus collector — not prior knowledge. This avoided several plausible-sounding but inaccurate claims (e.g. about OTel semantic-convention handling).
370
-
371
-6. **Preserve first drafts when a substantial rewrite follows.** Committing the first draft separately let the rewrite stand on its own as a reviewable change (+395 / -192) and made the structural shift visible in history. Future SKILL-level rewrites should follow the same pattern.
372
-
373
-7. **The SOW close should land with the work as one commit.** This SOW shipped its work in two commits but was left in `current/` until the user noticed. The AGENTS.md rule ("commit the work, artifact updates, SOW status change, and SOW move together as one commit") exists to prevent exactly this. Future SOWs must close in the same commit as the final piece of work, unless the user explicitly approves a split (as happened here for SOW-0009).
374
-
375
-## Followup
376
-
377
-Per the resolved Open Decision 3A, canonical-doc gaps are handled inline as 1-2 line guidance in the skill rather than companion docs in the skill directory. The skill currently inlines guidance for:
378
-
379
-- Vnodes (§1.9, §2.10) → pointer to `src/go/plugin/framework/vnodes/` and `BEST-PRACTICES.md`.
380
-- Error handling conventions (§2.3) → three-question format directive.
381
-- Logging conventions (§2.4) → debug/warn/error/info hierarchy directive.
382
-- Labels (§3 dashboard shaping) → described per ingestion path (NIDL, SNMP, statsd, OTEL, Prometheus).
383
-- netipc (§2.9, §6.5) → pointer to upstream spec at <https://github.com/netdata/plugin-ipc>.
384
-
385
-No follow-up SOWs are required at this time. The skill is live; if assistants miss any inlined topic in practice, the appropriate response is to expand the relevant section in a follow-up commit per the "Maintaining this skill" footer rule, not to author a separate canonical doc.
386
-
387
-Maintenance handle: when a new gap is identified (recurring AI bug pattern from real PRs, documentation drift caused by a rename in canonical docs, new collector domain), open a follow-up SOW only if the change is substantial; otherwise update the skill in the same PR that exposed the gap.
.agents/sow/done/SOW-0010-20260503-netdata-query-skills-infrastructure.md
deleted
-1036
@@ -1,1036 +0,0 @@
1
-# SOW-0010 - Netdata query skills infrastructure
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: rescoped 2026-05-03 evening (second expansion). The two public skills must mirror each other in structure and cover every queryable Netdata surface (metrics, logs, topology, flows, alerts, dyncfg, functions, nodes, plus Cloud-only rooms/members/feed and Agent-only streaming). Scripts must be **token-safe** -- the assistant must never see `NETDATA_CLOUD_TOKEN`, per-agent bearer values, or claim ids on stdout. A `how-tos/` subdir with `INDEX.md` ships in each skill; assistants extend it whenever they perform analysis not already covered by a how-to. The verification harness (Sonnet test runner with grading rubric) moves to follow-up SOW-0006 per user direction "evaluation does not need to be done now". Decisions 1, 2, 3 already resolved.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build the foundational AI-skill infrastructure that lets human and AI
14
-operators query Netdata Cloud and Netdata Agents in a uniform,
15
-documented way. This SOW is **infrastructure-only** -- it ships no
16
-business analysis, no triage scripts, no fleet-data fetches. Its
17
-deliverables are reusable pieces that downstream SOWs (agent-events,
18
-documentation pipeline, etc.) consume.
19
-
20
-Three deliverables:
21
-
22
-1. **Skill format convention.** Every public AI skill under
23
- `docs/netdata-ai/skills/` must follow the `<skill-name>/SKILL.md`
24
- directory shape with optional `<doc>.md` supporting docs and
25
- `scripts/` subdir, exactly like the private operational skills
26
- under `.agents/skills/`. Each public skill must be reachable from
27
- `.agents/skills/<skill-name>` via a **relative symlink** so local
28
- AI assistants reading from `.agents/skills/` see the same skill.
29
- The convention is documented in `AGENTS.md` so future skills
30
- follow the same shape.
31
-
32
-2. **`query-netdata-cloud/` skill (refactor + expand).** The current
33
- single file `docs/netdata-ai/skills/query-netdata-cloud-metrics.md`
34
- is migrated to `docs/netdata-ai/skills/query-netdata-cloud/
35
- SKILL.md` and expanded with separate supporting docs:
36
- - `query-metrics.md` -- the existing metrics-query content,
37
- trimmed to its specific surface
38
- - `query-logs.md` -- how to call the `systemd-journal` Function
39
- (and any other log-related Function) via Cloud
40
- - `query-alerts.md` -- how to query alerts and alert
41
- transitions via Cloud
42
- - `query-functions.md` -- generic Cloud-proxied Function
43
- invocation: what URL, what body, what response
44
- The top-level `SKILL.md` covers what is common across all four:
45
- auth (`NETDATA_CLOUD_TOKEN`), space/room/node resolution,
46
- pagination, error handling, dry-run discipline, links to each
47
- supporting doc.
48
-
49
-3. **`query-netdata-agents/` skill (new).** Sibling skill that
50
- covers querying Netdata Agents directly (Parents and Children).
51
- Delivers SKILL.md + supporting docs + a `scripts/` library.
52
- The scripts library must:
53
- - Read `NETDATA_CLOUD_TOKEN` from `.env`.
54
- - Probe the agent URL and detect whether it is bearer-protected
55
- (HTTP 401, redirect to Cloud SSO, or any other documented
56
- bearer-required signal).
57
- - When a bearer is required, mint one for that specific node
58
- using the user's cloud token and the documented Cloud endpoint,
59
- cache the bearer in `.local/audits/query-netdata-agents/
60
- bearers/<node-uuid>.json` with its expiration, and refuse to
61
- log the token value.
62
- - Transparently refresh the bearer when expired (or one minute
63
- before expiry, to avoid races during long batch fetches).
64
- - Expose a single `agents_resolve_bearer <node>` helper that
65
- downstream skills call to obtain a current bearer for a node.
66
- - Expose a single `agents_call_function` helper that takes
67
- `{node, function, body}` and routes the request through the
68
- correct transport (Cloud-proxied vs direct-agent), retrying
69
- once across transports on transient failure.
70
- - Expose a `agents_netdata_prefix` helper that **autodetects
71
- the local Netdata install prefix** at runtime (probe order:
72
- empty / `/opt/netdata` / `/usr/local/netdata`; pick the
73
- first whose `<prefix>/var/lib/netdata` or
74
- `<prefix>/etc/netdata` exists). Used to locate the local
75
- `bearer_tokens/` directory for the local-fallback bearer
76
- path. NOT an env knob; the prefix is a discovered fact.
77
-
78
-This SOW does NOT touch the legacy private operational skills
79
-(`coverity-audit/`, `sonarqube-audit/`, `graphql-audit/`,
80
-`pr-reviews/`) -- they keep their current location under
81
-`.agents/skills/` and their current shape. They are intentionally
82
-private and have no `docs/netdata-ai/skills/` counterpart.
83
-
84
-### User Request
85
-
86
-> Original (preserved for context):
87
-> "Create a skill for querying and fetching and analyzing agent events
88
-> (status file submissions) from the ingestion server."
89
->
90
-> Stage-1 follow-up:
91
-> "agent-events are stored in journal files... we have 2 options:
92
-> 1. query logs via cloud (docs/netdata-ai/skills/ has a relevant
93
-> skill, although not for logs, but we could enrich it)
94
-> 2. query the agent directly, but this time we need a mechanism to
95
-> get an agent bearer token by using the cloud api token, and
96
-> then query the agent API directly. Ideally we should support
97
-> all these methods."
98
->
99
-> Scope expansion:
100
-> "1. skills in docs/netdata-ai/skills/ should be formatted as
101
-> normal skills {skill-name}/SKILL.md with skill frontmatter and
102
-> potentially supporting documentation and scripts when necessary,
103
-> and they should be linked to .agents/skills/ with a relative
104
-> link, so that they are accessible by local agents too.
105
-> 2. query-netdata-cloud-metrics skill, should be renamed to
106
-> query-netdata-cloud/ and have SKILL.md with the common
107
-> information about querying netdata cloud and then supporting docs
108
-> query-metrics.md, query-logs.md, query-alerts.md,
109
-> query-functions.md, etc as necessary, which should be referenced
110
-> from SKILL.md.
111
-> 3. A new skill query-netdata-agents/ should be added, explaining
112
-> how to query netdata agents and parents and support the same
113
-> supporting material documentation. This should also live in
114
-> docs/netdata-ai/skills/ and be linked (relative) to
115
-> .agents/skills/.
116
-> 4. the query-netdata-agents skill should support querying agents
117
-> via netdata cloud sso, so it should provide the tooling to fetch
118
-> and cache and reuse and transparectly refresh agent bearer
119
-> tokens, starting from an netdata cloud api token. The supported
120
-> scripts should automatically detect the agent to query is bearer
121
-> protected and automatically work around this to fetch netdata-
122
-> cloud sso."
123
->
124
-> Split decision (this run): "Go with 4 SOWs."
125
-
126
-### Assistant Understanding
127
-
128
-Facts:
129
-
130
-- The .env + skill pattern is well-established and identical across
131
- the four legacy private skills. The same pattern is the basis for
132
- the scripts library shipped under `query-netdata-agents/`.
133
-- The agent-side `systemd-journal` Function and the agent-side
134
- `bearer_get_token` Function exist in this repo; their shapes are
135
- documented in stage-1 analysis.
136
-- Only one public AI skill exists today:
137
- `docs/netdata-ai/skills/query-netdata-cloud-metrics.md`. It is a
138
- flat file, not a directory. Its content is the template for the
139
- refactor.
140
-- Per the user's rule, `.env` values stay in `.env`; only keys
141
- appear in this SOW or in scripts.
142
-- The user added four `AGENT_EVENTS_*` keys to `.env`. Those keys
143
- are consumed by SOW-0003, not by this SOW. They are listed here
144
- only to confirm naming convention: `AGENT_EVENTS_NC_SPACE`,
145
- `AGENT_EVENTS_HOSTNAME`, `AGENT_EVENTS_MACHINE_GUID`,
146
- `AGENT_EVENTS_NODE_ID`.
147
-
148
-Inferences:
149
-
150
-- The Cloud REST shape that proxies a Function call to a node by
151
- uuid must already exist (otherwise no team member could query
152
- Cloud-only). The exact path is not in this open-source repo and
153
- must come from either user knowledge or the live Swagger at
154
- `${NETDATA_CLOUD_HOSTNAME}/api/docs/` (key in `.env`).
155
-- A documented Cloud endpoint that mints an agent bearer from a
156
- cloud token must exist if the user wants the auto-refresh flow.
157
- Candidate names a Swagger fetch could check (paths under the
158
- Cloud API base): anything under `/api/v3/spaces/.../nodes/<uuid>/
159
- ...token`, `/api/v.../auth/...`, `/api/v.../bearer/...`. If no
160
- such endpoint is documented, the agents skill must degrade to
161
- "local-only" transport-(b) (as described in stage-1 decision 1B).
162
-
163
-Unknowns (require user input or live-Swagger lookup):
164
-
165
-- Cloud REST function-call endpoint shape (URL pattern, request
166
- body, response shape).
167
-- Cloud REST agent-bearer mint endpoint shape (or confirmation it
168
- does not exist).
169
-- Whether the existing Cloud-metrics doc references the correct
170
- current Cloud Swagger version. Last-revised date in
171
- `query-netdata-cloud-metrics.md` should be cross-checked against
172
- the live API at `${NETDATA_CLOUD_HOSTNAME}/api/docs/`.
173
-- Symlink direction confirmation. User wrote: skills live in
174
- `docs/netdata-ai/skills/` and are symlinked from
175
- `.agents/skills/`. Reading: `docs/netdata-ai/skills/` is
176
- canonical; `.agents/skills/` holds relative symlinks pointing
177
- there. Confirmed during write-out.
178
-
179
-### Acceptance Criteria
180
-
181
-**(Scope expanded 2026-05-03 evening per user direction. The two
182
-public skills must mirror each other in structure, cover every
183
-queryable surface a Netdata operator/AI assistant cares about,
184
-keep all secrets out of the assistant's view, ship a how-to
185
-extraction system, and pass an automated verification harness.)**
186
-
187
-#### Symmetric file structure
188
-
189
-Both `docs/netdata-ai/skills/query-netdata-cloud/` and
190
-`docs/netdata-ai/skills/query-netdata-agents/` ship the same set
191
-of per-domain guides for the surfaces shared by both transports:
192
-
193
-| Per-domain guide | Cloud | Agent |
194
-|---|---|---|
195
-| SKILL.md | required | required |
196
-| query-metrics.md | required | required |
197
-| query-logs.md | required | required |
198
-| query-topology.md | required | required |
199
-| query-flows.md | required | required |
200
-| query-alerts.md | required | required |
201
-| query-dyncfg.md | required | required |
202
-| query-functions.md | required | required |
203
-| query-nodes.md | required | required |
204
-
205
-In addition, the Cloud skill ships three guides that have no
206
-agent equivalent (these surfaces only exist on the Cloud side):
207
-
208
-- `query-rooms.md`
209
-- `query-members.md`
210
-- `query-feed.md`
211
-
212
-And the Agent skill ships one guide with no Cloud equivalent
213
-(only meaningful agent-side):
214
-
215
-- `query-streaming.md`
216
-
217
-Each per-domain guide must:
218
-
219
-- Open with a one-paragraph summary of the surface.
220
-- Document the v3 endpoint(s) (use v2/v1 only when v3 is missing).
221
-- Show one runnable example using only the documented script
222
- wrappers (assistant must NOT see tokens; see security section
223
- below).
224
-- Cross-link the canonical reference docs in
225
- `<repo>/src/plugins.d/FUNCTION_UI_REFERENCE.md`,
226
- `<repo>/src/plugins.d/FUNCTION_UI_DEVELOPER_GUIDE.md`,
227
- `<repo>/src/plugins.d/FUNCTION_UI_SCHEMA.json`,
228
- `<repo>/src/plugins.d/DYNCFG.md`,
229
- `<repo>/src/daemon/dyncfg/README.md` where relevant.
230
-
231
-Both `SKILL.md` files index every per-domain guide AND the
232
-canonical references AND the how-tos directory.
233
-
234
-#### Token-safety architecture (HARD requirement)
235
-
236
-The assistant invoking these skills must NEVER see:
237
-
238
-- `NETDATA_CLOUD_TOKEN`
239
-- per-agent bearer values (the 36-char UUID returned by
240
- `bearer_get_token`)
241
-- claim_id values (treat as semi-sensitive identifiers)
242
-
243
-To enforce this, the scripts library exposes ONLY high-level
244
-wrappers. Helpers that previously returned a bearer to stdout
245
-(e.g. `agents_resolve_bearer`) MUST be marked internal (named
246
-with a leading underscore, e.g. `_agents_resolve_bearer`) and
247
-their output redirected to in-process variables only -- never
248
-emitted to stdout where the assistant could capture them.
249
-
250
-Public wrappers exposed to the assistant:
251
-
252
-- `agents_query_cloud <method> <path> [<body-json>]`
253
- Cloud-side. Reads `NETDATA_CLOUD_TOKEN` from `.env` internally,
254
- adds `Authorization: Bearer ...` header, runs curl, prints
255
- ONLY the response body. stderr shows the curl invocation with
256
- `<CLOUD_TOKEN>` masked.
257
-- `agents_query_agent <node> <method> <path> [<body-json>]`
258
- Direct-agent-side. Resolves bearer internally (cache or mint
259
- via cloud), routes through `${AGENT_EVENTS_HOSTNAME}` (or any
260
- reachable host), adds `X-Netdata-Auth: Bearer ...` header
261
- internally, prints ONLY the response body. stderr shows the
262
- curl invocation with `<AGENT_BEARER>` masked.
263
-- Per-surface convenience wrappers (one per query-*.md guide)
264
- that take typed arguments and forward to the above (e.g.
265
- `agents_query_function <node> <function-name> <body-json>`).
266
-
267
-**No public wrapper may print a token, bearer, or claim_id to
268
-stdout under any circumstance, including error paths.** This is
269
-verified by a pre-commit unit test that drives every public
270
-wrapper with a fake token and asserts the token bytes never
271
-appear in captured stdout.
272
-
273
-#### How-tos directory (live, indexed)
274
-
275
-Each skill ships a `how-tos/` subdirectory:
276
-
277
-- `<skill>/how-tos/INDEX.md` -- one-line index of every how-to,
278
- ordered by topic. Indexed from `SKILL.md`.
279
-- `<skill>/how-tos/<slug>.md` -- one file per how-to. Each
280
- documents: the question being answered, the steps taken,
281
- which wrappers were called, the expected output shape, and
282
- any gotchas.
283
-
284
-Rule baked into both `SKILL.md` files (and into AGENTS.md):
285
-**when an assistant has to perform analysis to answer a question
286
-that is not already covered by a how-to, the assistant must add
287
-a new how-to before completing the task.** The new how-to gets
288
-committed in the same PR as the analysis.
289
-
290
-#### Verification harness (DEFERRED to SOW-0006)
291
-
292
-Per user direction 2026-05-03: "the evaluation does not need to
293
-be done now". The full Sonnet-driven verification harness (test
294
-runner, grading rubric, automated how-to extraction prompts)
295
-moves to its own SOW (`SOW-0006-20260503-skill-verification-
296
-harness.md`, in `pending/`). It will validate this SOW's
297
-deliverables and any future skill, so it has independent value.
298
-
299
-This SOW seeds the inputs the harness will consume:
300
-
301
-- `<skill>/verify/questions.md` -- the seed list of validation
302
- questions. Both skills ship this file. Includes (at minimum)
303
- the user-supplied questions: hardware specs of a known node;
304
- OS; parent-or-child status; list of streamed-children if
305
- parent; vnodes; failed jobs; whether nvidia DCGM is monitored
306
- and at what frequency; PID with biggest memory consumption +
307
- dashboard category; last agent status-file log; plus 6+
308
- further questions covering alerts, logs, topology, flows,
309
- dyncfg, members, rooms, feed.
310
-
311
-The harness implementation (run.sh, grader rubric, score
312
-collation) is SOW-0006's deliverable, not SOW-0010's.
313
-
314
-#### Pre-existing acceptance criteria (carry-over)
315
-
316
-- Both relative symlinks at `.agents/skills/<name>` resolve to
317
- the corresponding `docs/netdata-ai/skills/<name>` directory.
318
-- `AGENTS.md` "Project Skills Index" lists both public skills
319
- with their one-line triggers and the symlink path.
320
-- `AGENTS.md` documents the public-skill convention.
321
-- Sensitive-data gate: every committed file passes the
322
- pre-commit grep from
323
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`.
324
-- All v3 agent paths preferred over v2/v1 (v2/v1 only as
325
- fallback for older agents).
326
-
327
-## Analysis
328
-
329
-Sources checked (stage-1, carried forward):
330
-
331
-- `<repo>/.env` (per-user, gitignored) -- confirmed shape used by
332
- the existing skills and the new `AGENT_EVENTS_*` keys for SOW-3.
333
-- `<repo>/AGENTS.md` (= `CLAUDE.md`) -- canonical documentation of
334
- the `.local/` audit directory convention and the `.env`
335
- convention.
336
-- `<repo>/.agents/skills/coverity-audit/SKILL.md` and
337
- `<repo>/.agents/skills/coverity-audit/scripts/_lib.sh`.
338
-- `<repo>/.agents/skills/sonarqube-audit/scripts/_lib.sh`.
339
-- `<repo>/.agents/skills/pr-reviews/scripts/_lib.sh`.
340
-- `<repo>/docs/netdata-ai/skills/query-netdata-cloud-metrics.md`
341
- (the existing template).
342
-- `<repo>/src/collectors/systemd-journal.plugin/systemd-main.c`,
343
- `systemd-journal.c`, `systemd-internals.h`, `logs_query_status.h`
344
- (Function shape).
345
-- `<repo>/src/web/api/functions/function-bearer_get_token.c`
346
- (agent-side bearer-mint Function and its Cloud-source gate).
347
-
348
-Current state -- skill format convention:
349
-
350
-- The four legacy private skills already use the directory shape
351
- (`<repo>/.agents/skills/<name>/SKILL.md` + `scripts/`).
352
-- The one public skill is a flat file
353
- (`docs/netdata-ai/skills/query-netdata-cloud-metrics.md`). The
354
- refactor brings it into line with the directory shape.
355
-- Symlink direction: `docs/netdata-ai/skills/<name>/` is canonical;
356
- `.agents/skills/<name>` becomes a relative symlink pointing at
357
- `../../docs/netdata-ai/skills/<name>`. This satisfies the user's
358
- rule "linked (relative) to .agents/skills/".
359
-
360
-Current state -- skill helper library shape (mirrored across all
361
-four legacy skills):
362
-
363
-- `set -euo pipefail` at the top.
364
-- Color variables defined with `$'\033[...]'`.
365
-- `<prefix>_repo_root()` -- via `git -C "$(dirname
366
- "${BASH_SOURCE[0]}")" rev-parse --show-toplevel`.
367
-- `<prefix>_load_env()` -- locates `<repo>/.env`, sources via
368
- `set -a; source; set +a`, validates required vars with
369
- `: "${VAR:?msg}"`, applies defaults with `: "${VAR:=default}"`.
370
-- `<prefix>_audit_dir()` -- creates
371
- `<repo>/.local/audits/<topic>/` on demand. Topic name strips any
372
- `-audit` suffix from the skill name (per AGENTS.md).
373
-- `<prefix>_run` and `<prefix>_run_read` -- print masked curl to
374
- stderr for transparency, mask the token in argv.
375
-- Skill-specific validators (numeric IDs, ASCII-only, etc.).
376
-
377
-Current state -- agent-side primitives (relevant to the agents
378
-skill):
379
-
380
-- `systemd-journal` Function name is registered at
381
- `src/collectors/systemd-journal.plugin/systemd-main.c:79` via
382
- `rrd_function_add(... ND_SD_JOURNAL_FUNCTION_NAME ...)`; the
383
- literal name is `systemd-journal`
384
- (`systemd-journal.c:13`).
385
-- POST body keys (from `logs_query_status.h:8-24`): `help`,
386
- `after`, `before`, `anchor`, `last`, `query`, `facets`,
387
- `histogram`, `direction`, `if_modified_since`, `data_only`,
388
- `__logs_sources`, `info`, `slice`, `delta`, `tail`, `sampling`.
389
-- Response top-level: `facets`, `histogram`, `rows`, `search`,
390
- `info`.
391
-- `bearer_get_token` Function is gated by
392
- `user_auth_source_is_cloud(source)`
393
- (`src/web/api/functions/function-bearer_get_token.c:30`), so
394
- external HTTP clients cannot call it directly. It is invoked
395
- over ACLK by Cloud, on behalf of a Cloud-authenticated user.
396
- Per-agent bearer files live at
397
- `<netdata-prefix>/var/lib/netdata/bearer_tokens/<token-uuid>.json` for ~24h.
398
-
399
-Risks:
400
-
401
-- Symlink portability: relative symlinks survive `git clone` and
402
- most worktree operations on Linux/macOS. Windows / WSL with NTFS
403
- may not. Acceptable risk -- the project is primarily
404
- Linux/macOS, and the symlinked skill is also reachable directly
405
- via its canonical path.
406
-- Bearer-mint endpoint may be undocumented: if the live Swagger
407
- has no agent-bearer mint, the agents skill must degrade
408
- gracefully to "local-only transport-b" (i.e. read existing
409
- `<netdata-prefix>/var/lib/netdata/bearer_tokens/*.json` for the user's own
410
- workstation only). Stage-1 decision 1B was the recommended
411
- fallback.
412
-- AGENTS.md churn: changing the project skills index requires
413
- care so the legacy private skills are not accidentally moved or
414
- renamed.
415
-- Breaking downstream readers: the existing flat file
416
- `docs/netdata-ai/skills/query-netdata-cloud-metrics.md` may be
417
- linked from external docs. The refactor must keep a redirect
418
- stub (a 1-line file pointing at
419
- `query-netdata-cloud/query-metrics.md`) to avoid breaking
420
- inbound links.
421
-
422
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
423
-
424
-Status: needs-user-decision
425
-
426
-Problem / root-cause model:
427
-
428
-- The current public skill shelf has a single skill in a flat-file
429
- shape. As soon as we want a second public skill, we either keep
430
- using flat files (leading to monoliths) or unify around the
431
- directory shape now. The user has chosen the directory shape;
432
- this SOW does the unification and adds the second skill.
433
-- Downstream SOWs (agent-events, learn-site-structure,
434
- integrations-lifecycle) need both `query-netdata-cloud/query-functions.md`
435
- and the bearer-mint scripts in `query-netdata-agents/scripts/`.
436
- Without this SOW, every downstream SOW would re-implement the
437
- same pieces.
438
-
439
-Evidence reviewed:
440
-
441
-- See "Sources checked" and "Current state" above. No further
442
- evidence is needed for the format-normalization piece. The
443
- bearer-mint piece is blocked on the live Cloud Swagger.
444
-
445
-Affected contracts and surfaces:
446
-
447
-- New: `<repo>/docs/netdata-ai/skills/query-netdata-cloud/`
448
- (SKILL.md + 4 supporting docs).
449
-- New: `<repo>/docs/netdata-ai/skills/query-netdata-agents/`
450
- (SKILL.md + supporting docs + scripts/).
451
-- New: `<repo>/.agents/skills/query-netdata-cloud` (relative
452
- symlink) and `<repo>/.agents/skills/query-netdata-agents`
453
- (relative symlink).
454
-- Stub: `<repo>/docs/netdata-ai/skills/query-netdata-cloud-
455
- metrics.md` becomes a 1-line redirect to the new location to
456
- preserve existing inbound links.
457
-- New: `<repo>/.local/audits/query-netdata-agents/` writes
458
- (gitignored, bearer cache + acceptance-test outputs).
459
-- Augmented: `<repo>/AGENTS.md` -- adds the public-skill
460
- convention paragraph and the index entries.
461
-- No existing surface is broken; the legacy skill keeps a
462
- redirect.
463
-
464
-Existing patterns to reuse:
465
-
466
-- `_lib.sh` shape from `coverity-audit/scripts/_lib.sh`.
467
-- Pagination idiom from `sonarqube-audit/scripts/_lib.sh::sq_paginate`.
468
-- Audit-dir / `.local/` convention from AGENTS.md.
469
-- Token-masking idiom from `sonarqube-audit/scripts/_lib.sh::sq_run`.
470
-- The single existing public skill
471
- `query-netdata-cloud-metrics.md` is the content seed for the
472
- new `query-metrics.md`.
473
-
474
-Risk and blast radius:
475
-
476
-- Low for format normalization (additive, with redirect stub).
477
-- Medium for the bearer-mint scripts -- they handle credentials
478
- and a bug here could leak bearers to logs or `.local/`. Mitigation:
479
- a single `agents_run` wrapper that forces token masking, plus a
480
- pre-commit grep that fails if any committed file contains
481
- `bearer_tokens` or `nd_bearer` blobs, plus a unit-test harness
482
- that runs the helpers under `set -x` and asserts no token bytes
483
- appear on stderr.
484
-
485
-Sensitive data handling plan:
486
-
487
-- This SOW (and every committed artifact it produces) follows the
488
- spec at `<repo>/.agents/sow/specs/sensitive-data-discipline.md`.
489
- No literal IPs, hostnames, UUID-shaped IDs, tokens, absolute
490
- install/user paths, usernames, tenant names, or secrets in any
491
- committed file. Every reference to such a value is via an
492
- env-key placeholder (`${KEY_NAME}`) defined in `.env`.
493
-- `.env` is the only place credential and identity VALUES live.
494
-- Bearer cache lives under
495
- `<repo>/.local/audits/query-netdata-agents/bearers/`;
496
- gitignored. File mode 0600.
497
-- All log lines that include curl invocations route through
498
- `agents_run`/`agents_run_read` which masks the cloud token and
499
- any bearer matched by a regex.
500
-- A one-shot redaction self-test runs as part of stage 2f
501
- validation: calls `agents_run` with a fake token, checks the
502
- emitted stderr contains no token bytes.
503
-- Pre-commit verification grep (from the spec) runs on every
504
- staged change.
505
-
506
-Implementation plan:
507
-
508
-1. **Stage 1 -- DONE**: investigation captured in this SOW.
509
-2. **Stage 2a -- DONE**: decisions 1, 2, 3 resolved. Cloud
510
- bearer-mint and function-call endpoints discovered, smoke-
511
- tested live. Symlink direction A confirmed by user.
512
-3. **Stage 2b -- DONE**: format normalization. Existing flat
513
- file moved to `query-netdata-cloud/query-metrics.md`. New
514
- SKILL.md plus per-domain guides
515
- (`query-logs.md`, `query-topology.md`, `query-flows.md`,
516
- `query-alerts.md`, `query-dyncfg.md`, `query-functions.md`)
517
- written. v3 agent paths used everywhere.
518
-4. **Stage 2c -- DONE**: `query-netdata-agents/SKILL.md` written
519
- with bearer-mint flow described; `scripts/_lib.sh` ships
520
- `agents_resolve_bearer`, `agents_call_function`,
521
- `agents_netdata_prefix`. Both relative symlinks created and
522
- resolved.
523
-5. **Stage 2d -- DONE**: AGENTS.md updated with public-skill
524
- convention paragraph + public-skill index entries.
525
-
526
-The remaining stages cover the second-expansion scope:
527
-
528
-6. **Stage 2e -- token-safety architecture rework**:
529
- - Rename `agents_resolve_bearer` to `_agents_resolve_bearer`
530
- (internal, never returns to stdout).
531
- - Add `agents_query_cloud <method> <path> [<body>]` and
532
- `agents_query_agent <node> <method> <path> [<body>]` public
533
- wrappers that handle auth internally and emit only the
534
- response body to stdout.
535
- - Add per-surface convenience wrappers (one per query-*.md
536
- guide).
537
- - Add a unit test that drives every public wrapper with a
538
- fake token and asserts the token bytes never appear on
539
- captured stdout.
540
-7. **Stage 2f -- per-domain guides for the agent skill**:
541
- For `query-netdata-agents/`, write `query-metrics.md`,
542
- `query-logs.md`, `query-topology.md`, `query-flows.md`,
543
- `query-alerts.md`, `query-dyncfg.md`, `query-functions.md`,
544
- `query-nodes.md`, `query-streaming.md`. Each uses the
545
- token-safe wrappers from stage 2e in every example.
546
-8. **Stage 2g -- per-domain guides for the cloud skill**:
547
- For `query-netdata-cloud/`, add `query-nodes.md`,
548
- `query-rooms.md`, `query-members.md`, `query-feed.md`. Each
549
- uses the token-safe wrappers; covers v3 endpoints (or
550
- documented v2 fallbacks).
551
-9. **Stage 2h -- SKILL.md re-indexing**: both `SKILL.md` files
552
- updated to list every per-domain guide AND the canonical
553
- reference docs AND the how-tos directory.
554
-10. **Stage 2i -- how-tos infrastructure**:
555
- - Create `<skill>/how-tos/INDEX.md` with the format and
556
- authoring rules (one how-to per file, slug, question,
557
- steps, wrappers used, expected output, gotchas).
558
- - Seed `INDEX.md` with the user-supplied question list (one
559
- stub per question, marked TODO until a how-to is
560
- authored). Stub how-tos are NOT a valid close state for
561
- SOW-0006 but ARE the close state for this SOW (the
562
- catalog is the deliverable; populating it happens during
563
- verification).
564
- - Document the rule "if you analyze, you author a how-to"
565
- in both `SKILL.md` files AND in `AGENTS.md` so future
566
- assistants honor it.
567
-11. **Stage 2j -- seed verify/questions.md**: write the seed
568
- question list for both skills (the user's 10+ questions plus
569
- coverage for every per-domain guide). The harness that
570
- consumes them is SOW-0006.
571
-12. **Stage 2k -- final validation**:
572
- - Run every public wrapper end-to-end (Cloud + Agent
573
- transports).
574
- - Run the no-token-on-stdout unit test.
575
- - Run shellcheck on every script.
576
- - Run the spec's pre-commit grep on every changed file.
577
- - Confirm both symlinks resolve.
578
-13. **Stage 2l -- close**: status `completed`, move to `done/`,
579
- single commit covering skill expansion + AGENTS.md update +
580
- SOW close + SOW-0006 (pending) creation.
581
-
582
-Validation plan:
583
-
584
-- Stage 1: documentation read-through (this SOW).
585
-- Stage 2: real-use evidence (one Cloud-proxied function call,
586
- one direct-agent function call after bearer mint, one redaction
587
- self-test); shellcheck on every script; pre-commit grep against
588
- token-shaped strings; confirm symlinks resolve via
589
- `git ls-files --stage` and `readlink -f`.
590
-
591
-Artifact impact plan:
592
-
593
-- AGENTS.md: add public-skill convention paragraph + public-skill
594
- index entries.
595
-- Runtime project skills: the two new skills are public
596
- (`docs/netdata-ai/skills/`) but reachable from
597
- `.agents/skills/` via relative symlinks; both names trigger on
598
- AI-skill router queries.
599
-- Specs: not required at this stage. Stage 2 may add a short
600
- spec under `.agents/sow/specs/skills-format.md` capturing the
601
- convention if useful for future reviewers.
602
-- End-user/operator docs: the new SKILL.md and supporting docs
603
- ARE end-user-facing (anyone using AI assistants with this
604
- repo). They live in `docs/netdata-ai/skills/`.
605
-- End-user/operator skills: the two new skills.
606
-- SOW lifecycle: open in `pending/`; moves to `current/` once
607
- decisions 1-3 are recorded; moves to `done/` after stage 2g.
608
-
609
-Open-source reference evidence:
610
-
611
-- Not consulted at stage 1. Stage 2 may consult upstream
612
- observability projects (e.g. Grafana, Datadog) for reference
613
- patterns on auth-token caching helpers if useful, but no
614
- external reference is required to proceed. The user's
615
- workstation has a local mirror tree available for grep / read.
616
-
617
-Open decisions:
618
-
619
-- See "Implications And Decisions" below. Implementation cannot
620
- begin until decisions 1-3 are answered.
621
-
622
-## Implications And Decisions
623
-
624
-Decisions 1 and 2 are RESOLVED (2026-05-03) by reading the
625
-cloud-* source code (`cloud-frontend`, `cloud-spaceroom-service`,
626
-`cloud-charts-service`) and live smoke-testing against the
627
-production Cloud API. Decision 3 still needs the user's explicit
628
-confirmation. Findings recorded inline below.
629
-
630
-1. **Cloud REST endpoint that mints an agent bearer from a
631
- cloud token.** RESOLVED 2026-05-03 (option **A**). Endpoint
632
- exists; smoke-tested live.
633
- - Path: `GET ${NETDATA_CLOUD_HOSTNAME}/api/v2/bearer_get_token`
634
- - Required query params: `node_id`, `machine_guid`, `claim_id`
635
- - Auth header: `Authorization: Bearer ${NETDATA_CLOUD_TOKEN}`
636
- - Response body keys: `bearer_protection` (bool),
637
- `expiration` (numeric -- format TBD; smoke-test value
638
- rendered as 1970-01-01 when interpreted as Unix seconds,
639
- so likely milliseconds or an ISO-string variant; stage 2b
640
- must verify), `mg` (echo of machine_guid), `status`,
641
- `token` (36-char UUID, the bearer).
642
- - Cloud-side handler:
643
- `cloud-spaceroom-service/http/transport_http.go:355`
644
- (`makeGetAgentBearerToken`); route registered at
645
- `cloud-spaceroom-service/http/endpoints_agent.go:179`.
646
- Permission gate: `PermissionSpaceRead`; node must be
647
- `reachable`; agent-side delegation invokes
648
- `bearer_get_token` Function via ACLK.
649
- - Frontend cache pattern:
650
- `cloud-frontend/src/domains/nodes/useAgentBearer.js`.
651
- Storage keyed by `machine_guid`. Refresh trigger:
652
- `expiration < now + 3600 seconds` (1-hour buffer before
653
- expiry).
654
- - The `claim_id` is read from the agent's `/api/v3/info`
655
- at `.agents[0].cloud.claim_id`, OR (with shell access)
656
- from the agent host's
657
- `<netdata-prefix>/var/lib/netdata/cloud.d/claimed_id`.
658
- - The minted bearer is sent in subsequent direct-agent
659
- calls as `X-Netdata-Auth: Bearer <token>` (NOT
660
- `Authorization: Bearer <token>`).
661
-
662
-2. **Cloud REST endpoint that invokes a Function on a node by
663
- uuid.** RESOLVED 2026-05-03 (option **A**). Endpoint exists;
664
- smoke-tested live.
665
- - Path: `POST ${NETDATA_CLOUD_HOSTNAME}/api/v2/nodes/{nodeId}/function?function={functionName}`
666
- - Auth header: `Authorization: Bearer ${NETDATA_CLOUD_TOKEN}`
667
- - Optional header: `X-Transaction-Id: <uuid>` (correlation
668
- only; not required).
669
- - Request body: the agent-side Function payload (e.g. for
670
- `systemd-journal`: `{"info": true}`, or a query body with
671
- `after`, `before`, `last`, `query`, `facets`,
672
- `histogram`, `__logs_sources`, etc.). Optional top-level
673
- `timeout` (ms) and `last` (page size).
674
- - Response body: JSON (NOT streaming). Top-level keys for
675
- `systemd-journal info=true`: `_request`, `accepted_params`,
676
- `has_history`, `help`, `pagination`, `required_params`,
677
- `show_ids`, `status`, `type`, `v`, `versions`.
678
- - Cloud-side: `cloud-charts-service/http/http.go:146`
679
- (`nodePathProxy` -> dispatches via ADC to the agent).
680
- - Companion listing endpoint:
681
- `POST ${NETDATA_CLOUD_HOSTNAME}/api/v3/spaces/{spaceID}/rooms/{roomID}/functions`
682
- with body
683
- `{"scope":{"nodes":[...]},"selectors":{"nodes":["*"]}}`
684
- returns the list of available functions per node.
685
- Service: `cloud-charts-service/http/http.go:135`
686
- (`scopeFunctions` handler at
687
- `cloud-charts-service/http/http.go:1036`).
688
- - Smoke test 2026-05-03: Cloud function call returned 200
689
- with valid metadata; the same call against the agent
690
- directly (using a freshly-minted bearer) returned an
691
- identical 2748-byte response. Both transports work
692
- end-to-end.
693
-
694
-3. **Symlink direction confirmation.** The user wrote: skills
695
- live in `docs/netdata-ai/skills/` and are linked from
696
- `.agents/skills/` with relative symlinks. Confirming reading:
697
- - A. Canonical path: `docs/netdata-ai/skills/<name>/`.
698
- Relative symlink: `.agents/skills/<name>` ->
699
- `../../docs/netdata-ai/skills/<name>`. *(matches user's
700
- wording)*
701
- - B. Other direction: `.agents/skills/<name>/` canonical,
702
- `docs/netdata-ai/skills/<name>` symlink to it.
703
- - C. Bidirectional / something else.
704
- - **Recommendation:** **A**, the natural reading of the
705
- directive and the only one consistent with "skills are
706
- accessible by local agents too".
707
-
708
-## Plan
709
-
710
-Pending decisions 1-3. After they are answered:
711
-
712
-1. Update this SOW with the decisions; move to `current/` as
713
- `Status: in-progress`.
714
-2. Implement per stage 2b-2f.
715
-3. Validate per stage 2f.
716
-4. Close per stage 2g.
717
-
718
-## Execution Log
719
-
720
-### 2026-05-03
721
-
722
-- Stages 2e-2k (2026-05-03 late evening). Token-safety rework
723
- shipped: `_lib.sh` now exposes `agents_query_cloud`,
724
- `agents_query_agent`, `agents_call_function` as token-safe
725
- public wrappers; bearer / cloud-token / claim_id never reach
726
- stdout; `_agents_resolve_bearer` rewritten to return via bash
727
- nameref. `agents_selftest_no_token_leak` PASSES (drives
728
- wrappers with a sentinel token, asserts sentinel never appears
729
- on captured stdout). Per-domain agent guides written:
730
- `query-functions.md`, `query-logs.md`, `query-topology.md`,
731
- `query-flows.md`, `query-alerts.md`, `query-dyncfg.md`,
732
- `query-metrics.md`, `query-nodes.md`, `query-streaming.md`.
733
- Per-domain cloud-only guides written: `query-nodes.md`,
734
- `query-rooms.md`, `query-members.md`, `query-feed.md`. The
735
- `feed` endpoint discovered: `POST /api/v1/feed/search` served
736
- by the separate `cloud-feed-service`, snake_case `space_id`
737
- body field, response wraps Elasticsearch hits in
738
- `results.hits.hits[]._source` with ECS v8.4 + Netdata-specific
739
- envelope. The `members` endpoint:
740
- `GET /api/v2/spaces/{sp}/members`. The `rooms` endpoint:
741
- `GET /api/v2/spaces/{sp}/rooms`. Both SKILL.md files
742
- re-indexed with all per-domain guides + canonical references
743
- + how-tos + verify pointers. The "if you analyze, you author
744
- a how-to" rule baked into SKILL.md, AGENTS.md, and both
745
- how-tos/INDEX.md files. Seed verify/questions.md written for
746
- both skills (the user-supplied 23 + 19 questions covering
747
- identity / hardware / OS / streaming / vnodes / collectors /
748
- alerts / logs / topology / flows / dyncfg / members / rooms /
749
- feed / token-safety self-test). Stage 2l (close) pending.
750
-- Stage-1 close-out (2026-05-03 evening). User added
751
- `NETDATA_REPOS_DIR` and `NETDATA_CLOUD_HOSTNAME` to `.env`.
752
- Live API probes validated all five existing AGENT_EVENTS_*
753
- keys: `GET /api/v2/spaces` -> `/rooms` -> `POST .../nodes`;
754
- matched node state `reachable`, version `v2.9.0-5-nightly`.
755
- ssh path validated (passwordless, journalctl
756
- --namespace=${AGENT_EVENTS_HOSTNAME} returned real data).
757
- Direct-agent path validated (port 19999 reachable,
758
- `/api/v3/info` returns 200 unauthenticated; functions
759
- return 412 without bearer). Read the cloud-* sources at
760
- `${NETDATA_REPOS_DIR}/cloud-frontend`,
761
- `${NETDATA_REPOS_DIR}/cloud-spaceroom-service`,
762
- `${NETDATA_REPOS_DIR}/cloud-charts-service` to discover
763
- the bearer-mint endpoint and the function-call endpoint.
764
- Both endpoints smoke-tested live (200 OK, valid responses;
765
- bearer minted; direct-agent call after bearer-mint
766
- returned identical metadata to Cloud-proxied call).
767
- Decisions 1 and 2 RESOLVED. Decision 3 (symlink direction)
768
- still pending. Raw artifacts saved under
769
- `<repo>/.local/audits/query-netdata-cloud/probe/`.
770
- Removed redundant keys `AGENT_EVENTS_IP` and
771
- `AGENT_EVENTS_JOURNAL_NAMESPACE` after user noted that the
772
- existing `AGENT_EVENTS_HOSTNAME` value covers ssh/HTTP/
773
- journal-namespace roles (quadruple-duty).
774
-- Stage 1 investigation completed (originally as part of the old
775
- SOW-2 "agent-events triage skill"). Confirmed `.env` + skill
776
- `_lib.sh` pattern across coverity-audit / sonarqube-audit /
777
- pr-reviews / graphql-audit. Confirmed `systemd-journal`
778
- Function shape and `bearer_get_token` Cloud-source gate in
779
- source.
780
-- Scope expanded by user (format normalization, query-netdata-
781
- cloud refactor, query-netdata-agents new skill).
782
-- User chose 4-SOW split. The agent-events triage skill moved
783
- to SOW-0003; documentation-pipeline skills to SOW-0004;
784
- mirror-netdata-repos skill to SOW-0005. This SOW (SOW-0010) was
785
- rescoped to skill infrastructure and renamed to "Netdata
786
- query skills infrastructure". Old filename
787
- `SOW-0010-20260503-agent-events-skill.md` removed; new
788
- filename `SOW-0010-20260503-netdata-query-skills-
789
- infrastructure.md`.
790
-- Post-close (2026-05-04): user split the original
791
- doc-pipeline SOW into documentation (SOW-0004 rescoped to
792
- `learn-site-structure`) and integrations (new SOW-0007
793
- `integrations-lifecycle`). Cross-references updated for
794
- navigation accuracy.
795
-
796
-## Validation
797
-
798
-Acceptance criteria evidence:
799
-
800
-- Symmetric file structure delivered: both
801
- `docs/netdata-ai/skills/query-netdata-cloud/` and
802
- `docs/netdata-ai/skills/query-netdata-agents/` ship
803
- `SKILL.md` + 8 shared per-domain guides
804
- (`query-{metrics,logs,topology,flows,alerts,dyncfg,functions,nodes}.md`).
805
- Cloud adds `query-{rooms,members,feed}.md`; agent adds
806
- `query-streaming.md`. Both ship `how-tos/INDEX.md` and
807
- `verify/questions.md`.
808
-- Token-safety architecture delivered:
809
- `agents_query_cloud`, `agents_query_agent`,
810
- `agents_call_function` are the public wrappers; internal
811
- helpers renamed with leading underscore and return token
812
- bytes via bash namerefs only.
813
- `agents_selftest_no_token_leak` PASSES on every run
814
- (verified live: bearer-mint, cloud call, direct-agent call,
815
- bearer-cache hit -- all confirm zero token bytes on
816
- captured stdout).
817
-- Both relative symlinks at `.agents/skills/` resolve to the
818
- corresponding `docs/netdata-ai/skills/` directory.
819
-- `AGENTS.md` updated with the public-skill convention paragraph,
820
- the public-skill index entries, the token-safety contract
821
- paragraph, and the live how-tos catalog rule.
822
-- v3 agent paths used everywhere; v2/v1 only as fallback for
823
- pre-v2 agents (alerts section explicitly notes this).
824
-- Cloud verification covers the eleven domains the SOW
825
- required; agent verification covers the nine domains the SOW
826
- required. Both `verify/questions.md` files seeded with the
827
- user-supplied questions plus per-domain coverage.
828
-
829
-Tests or equivalent validation:
830
-
831
-- shellcheck on `_lib.sh`: clean (no findings).
832
-- `agents_selftest_no_token_leak`: `[PASS]`.
833
-- Live smoke tests against production:
834
- - `agents_query_cloud GET /api/v2/accounts/me` -- 200,
835
- response delivered, zero cloud-token bytes in stdout.
836
- - `agents_query_agent ... POST /api/v3/function?function=systemd-journal '{"info":true}'`
837
- -- 200, 2748-byte response, zero cloud-token bytes and
838
- zero bearer bytes in stdout.
839
- - Agent v3 alert paths: `/api/v3/alerts`,
840
- `/api/v3/alert_transitions`, `/api/v3/alert_config` --
841
- all 200.
842
- - Cloud-side: `/api/v2/spaces`, `/api/v2/accounts/me`,
843
- `/api/v2/spaces/{sp}/rooms`, `/api/v3/spaces/{sp}/rooms/
844
- {rm}/nodes`, `/api/v2/spaces/{sp}/members`,
845
- `/api/v1/feed/search`, `/api/v3/spaces/{sp}/rooms/{rm}/
846
- alerts*` -- all 200.
847
- - Agent direct: `/api/v3/info`, `/api/v3/config?action=tree`,
848
- `/api/v3/function?function=topology:snmp`,
849
- `/api/v3/function?function=flows:netflow` (where the
850
- collector is enabled) -- all 200.
851
-
852
-Real-use evidence:
853
-
854
-- The bearer cache works across calls: first call mints, second
855
- call hits cache (verified by stable token output between
856
- invocations within the 2-hour window).
857
-- The two transports return identical metadata for
858
- `systemd-journal info=true` (modulo timestamps), confirming
859
- the agent and the Cloud proxy expose the same Function
860
- payload shape.
861
-- The discovered Cloud feed endpoint
862
- (`POST /api/v1/feed/search`) returns 42853 hits over the
863
- user's seed query; verified the snake_case `space_id` body
864
- field is required (camelCase `spaces[].id` is rejected with
865
- 400).
866
-
867
-Reviewer findings:
868
-
869
-- Self-review caught: invented "alert function" terminology
870
- and partial query-alerts.md (rewritten with all 11 endpoints,
871
- smoke-tested).
872
-- Self-review caught: invented function names (`top`,
873
- `aclk-state`, `ml-models`, `windows-events`, `streaming`).
874
- Replaced with the live-verified list pulled from the
875
- agent-events node.
876
-- User-flagged: missing topology + flow Function families.
877
- Added query-topology.md and query-flows.md with
878
- source-verified payload shapes.
879
-- User-flagged: missed FUNCTION_UI_REFERENCE.md /
880
- FUNCTION_UI_DEVELOPER_GUIDE.md / FUNCTION_UI_SCHEMA.json /
881
- DYNCFG.md. Added explicit references in SKILL.md and the
882
- per-domain guides.
883
-- User-flagged: v2 agent paths. Switched all agent-direct
884
- paths to v3 (v2/v1 only as fallback).
885
-- User-flagged: 4-family taxonomy was a fabrication. Replaced
886
- with the canonical 2-class taxonomy from
887
- FUNCTION_UI_REFERENCE.md (Simple Table + Log Explorer);
888
- topology and flows documented as custom Functions building
889
- on the same envelope.
890
-- User-flagged: assistant must never see tokens. Reworked
891
- `_lib.sh` with internal/public split and shipped a
892
- no-leak self-test.
893
-
894
-Same-failure scan:
895
-
896
-- Spec discipline grep on every committed file: zero
897
- violations (the only `deadbeef-...-...-...-...-...` UUID
898
- in `_lib.sh` is the deliberate self-test sentinel and is
899
- not a real credential).
900
-
901
-Sensitive data gate:
902
-
903
-- Pre-commit grep ran clean over every file touched by this
904
- SOW: zero UUID-shaped IDs (except the test sentinel),
905
- zero IPv4 literals to specific hosts (only loopback
906
- `127.0.0.1` examples and a `<YOUR_FOCUS_DEVICE_IP>`
907
- placeholder), zero forbidden absolute paths
908
- (Netdata defaults `/var/lib/netdata`, `/etc/netdata` are
909
- explicitly allowed by the spec).
910
-- Cloud REST host is env-keyed via `${NETDATA_CLOUD_HOSTNAME}`
911
- in scripts; appears as the literal `app.netdata.cloud` only
912
- in user-facing curl examples (allowed for public Netdata-org
913
- sites in role-descriptive prose).
914
-- Token-safe wrappers verified: `agents_selftest_no_token_leak`
915
- PASSES; `agents_query_cloud` and `agents_query_agent` both
916
- emit zero cloud-token bytes and zero bearer bytes on captured
917
- stdout.
918
-
919
-Artifact maintenance gate:
920
-
921
-- AGENTS.md: updated with public-skill convention paragraph,
922
- the public-skill index, the token-safety contract paragraph,
923
- and the how-tos catalog rule.
924
-- Runtime project skills: two new public skills shipped at
925
- `docs/netdata-ai/skills/query-netdata-{cloud,agents}/`,
926
- reachable from `.agents/skills/` via relative symlinks.
927
-- Specs: `<repo>/.agents/sow/specs/sensitive-data-discipline.md`
928
- shipped (separate concern; underwrites this SOW + future
929
- SOWs). No additional spec required for the skill convention
930
- -- documented in AGENTS.md.
931
-- End-user/operator docs: the two new skill bundles ARE the
932
- end-user-facing docs.
933
-- End-user/operator skills: ditto.
934
-- SOW lifecycle: moved from `pending/` -> `current/` ->
935
- `done/` per the framework. Status `completed`. Verification
936
- harness deferred to `SOW-0006` per user direction; SOW-0006
937
- shipped as a stub in `pending/`.
938
-
939
-Specs update:
940
-
941
-- New spec `<repo>/.agents/sow/specs/sensitive-data-discipline.md`
942
- (the rule that underwrites this SOW's discipline gate).
943
-
944
-Project skills update:
945
-
946
-- Two new public skills under `docs/netdata-ai/skills/` with
947
- relative symlinks from `.agents/skills/`.
948
-
949
-End-user/operator docs update:
950
-
951
-- The two new skill bundles ARE the docs update.
952
-
953
-End-user/operator skills update:
954
-
955
-- The two new skill bundles ARE the skills update.
956
-
957
-Lessons:
958
-
959
-- **Verify before documenting.** The first draft of
960
- query-alerts.md and query-functions.md contained invented
961
- terms ("alert functions") and invented Function names
962
- (`top`, `aclk-state`, `ml-models`). Source-verification
963
- pass and live smoke-testing caught both. Lesson: every
964
- endpoint table must be smoke-tested before commit; every
965
- Function name list must be pulled from the live listing
966
- endpoint.
967
-- **Public Swagger is incomplete.** `app.netdata.cloud/api/docs/`
968
- documents only 7 paths. Real cloud endpoints
969
- (`bearer_get_token`, function-call proxy, alert endpoints,
970
- feed search) live across 4+ microservices and were
971
- discovered by reading the cloud-* sources at
972
- `${NETDATA_REPOS_DIR}/cloud-*/`. Lesson: when Swagger is
973
- thin, read source.
974
-- **Cloud-side `expiration: 0` is real.** The bearer mint
975
- endpoint returns `expiration: 0` on the production cloud,
976
- which would force re-mint on every call if naively
977
- followed. The cache logic now stamps `_cached_at` and falls
978
- back to a 2-hour mint window when `expiration` is 0. Agents
979
- actually issue ~3-hour TTL bearers, so 2 hours leaves a
980
- safety margin.
981
-- **zsh vs bash compat.** `BASH_SOURCE[0]` warning on zsh
982
- was noisy. Solution: capture `_agents_lib_self` at source
983
- time using a `ZSH_VERSION`/`BASH_VERSION` switch with
984
- `eval` for the zsh-only `${(%):-%x}` syntax.
985
-- **Token-safety needs architectural enforcement, not
986
- discipline.** The first draft had `agents_resolve_bearer`
987
- return the bearer to stdout; refactoring to bash namerefs
988
- + leading-underscore "internal" naming + a no-leak
989
- self-test is what makes the contract verifiable.
990
-
991
-Follow-up mapping:
992
-
993
-- Verification harness (Sonnet test runner + grading rubric +
994
- how-to extraction prompt loop): tracked in
995
- `<repo>/.agents/sow/pending/SOW-0006-20260503-skill-verification-harness.md`.
996
-- `query-agent-events` private skill (consumes the wrappers
997
- delivered here): tracked in
998
- `<repo>/.agents/sow/pending/SOW-0003-20260503-query-agent-events-skill.md`.
999
-- `learn-site-structure` private skill: tracked in
1000
- `<repo>/.agents/sow/pending/SOW-0004-20260503-learn-site-structure-skill.md`.
1001
-- `integrations-lifecycle` private skill: tracked in
1002
- `<repo>/.agents/sow/pending/SOW-0007-20260504-integrations-lifecycle-skill.md`.
1003
-- `mirror-netdata-repos` private skill: tracked in
1004
- `<repo>/.agents/sow/pending/SOW-0005-20260503-mirror-netdata-repos-skill.md`.
1005
-- The how-tos catalog stubs in both skills are deliberately
1006
- unfilled at close: they get populated when the verification
1007
- harness (SOW-0006) drives Sonnet through `verify/questions.md`
1008
- and prompts for new how-tos on misses. The "if you analyze,
1009
- you author a how-to" rule is durable; SOW close does not
1010
- require pre-populated how-tos beyond the stub catalog.
1011
-
1012
-## Outcome
1013
-
1014
-Delivered. Two symmetric public skills ship at
1015
-`docs/netdata-ai/skills/query-netdata-{cloud,agents}/`, each with
1016
-SKILL.md + per-domain guides covering every queryable Netdata
1017
-surface, token-safe wrappers in `query-netdata-agents/scripts/
1018
-_lib.sh` (verified by self-test), live `how-tos/INDEX.md`
1019
-catalogs, seed `verify/questions.md` lists, and relative
1020
-symlinks from `.agents/skills/`. AGENTS.md updated.
1021
-`<repo>/.agents/sow/specs/sensitive-data-discipline.md` shipped.
1022
-Verification harness deferred to SOW-0006 per user direction.
1023
-
1024
-## Lessons Extracted
1025
-
1026
-See "Lessons" inside the Validation section above.
1027
-
1028
-## Followup
1029
-
1030
-See "Follow-up mapping" inside the Validation section above.
1031
-
1032
-## Regression Log
1033
-
1034
-None yet.
1035
-
1036
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0011-20260505-codacy-audit-skill.md
deleted
-330
@@ -1,330 +0,0 @@
1
-# SOW-0011 - codacy-audit skill
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: skill shipped (`.agents/skills/codacy-audit/` with SKILL.md, 3 scripts, how-tos catalog with 1 seed entry). All acceptance criteria met.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Give the user (and AI assistants helping the user) a fit-for-purpose Codacy-Cloud workflow for `netdata/netdata`:
14
-
15
-1. **Pre-push prevention** -- run the same analyzers Codacy CI runs locally, before `git push`, so PR feedback loops collapse from "push -> wait minutes -> see findings -> fix -> push again" to "run -> see findings -> fix -> push once".
16
-2. **Read-only PR/repo triage** -- list issues on a PR (current pattern: ad-hoc `curl` against `api.codacy.com`), cluster by tool/pattern/severity, and write the triage notes under `<repo>/.local/audits/codacy/`.
17
-
18
-Out of scope for this SOW (deferred to a future SOW if needed):
19
-- Write actions (mark FP, mark fixed, modify ignore-patterns via API).
20
-- Master-backlog triage on the 31,425 open issues currently on `master`.
21
-- Cross-repo (entire `netdata` org) aggregation.
22
-
23
-### User Request
24
-
25
-> "create the sow and build it"
26
-
27
-Accepting the assistant's recommendation (delivered earlier in the conversation): "Start with track 1 only (the local CLI wrapper) as a small SOW. It pays off on every future PR. Track 2 (triage workflow) becomes worthwhile once we hit a PR or quarterly review where Codacy backlog matters -- defer until then."
28
-
29
-User also added `CODACY_TOKEN` to `<repo>/.env`. This is an Account API token (header `api-token: <value>`) -- usable for read-only PR/repo queries today; future write actions deferred.
30
-
31
-### Assistant Understanding
32
-
33
-Facts:
34
-
35
-- Three sister legacy skills already exist with the triage shape we want to mirror: `.agents/skills/coverity-audit/`, `.agents/skills/sonarqube-audit/`, `.agents/skills/graphql-audit/`. A `codacy-audit/` is the natural fourth.
36
-- Codacy Cloud ships an official local CLI: `codacy-analysis-cli` (https://github.com/codacy/codacy-analysis-cli). Install paths: docker, install.sh, brew. **Docker is available on this workstation** (`/usr/bin/docker`, version 29.4.1).
37
-- The Codacy v3 REST API at `api.codacy.com` is reachable: PR-level issue lists work even anonymously; broader cross-PR / org queries require an Account API token. Verified live during the conversation that this skill is being created for the configured account; 31,425 issues were open on `master`.
38
-- PR #22423 is a useful end-to-end fixture for validation: it had 864 markdownlint findings on the first CI run; fixed by `.codacy.yml` exclusion in commit `3a54c9afbc`. The local CLI must reproduce the original 864 findings for an objective accuracy check.
39
-- The repo's `.codacy.yml` is the source of truth for path exclusions; the local CLI must respect it (or we have to teach it to).
40
-- Path discipline spec at `<repo>/.agents/sow/specs/sensitive-data-discipline.md` already defines `CODACY_TOKEN`-class constraints by precedent (Coverity / Sonar tokens). Keys must live in `.env`, never in committed artifacts.
41
-
42
-Inferences:
43
-
44
-- The token is set today but not exercised by any committed script. The skill's `_lib.sh` should ship a token-safe sentinel self-test like `agentevents_selftest_no_token_leak` (introduced by SOW-0003) so that future write-action expansion inherits the discipline cleanly.
45
-- Most Netdata code PRs come back Codacy-clean (recent merged PRs all show `pass`). The high-value moments for this skill are: (a) doc-heavy PRs like #22423; (b) any PR that adds a new file type (e.g. JS, Python) to a new tree; (c) periodic quarterly review of master-backlog (deferred to follow-up SOW).
46
-- Mirroring the coverity-audit/sonarqube-audit shape is preferable to inventing a new layout: same filename conventions (`SKILL.md`, `scripts/_lib.sh`, `scripts/<verb>-<noun>.sh`), same artifact destination (`<repo>/.local/audits/codacy/`), same live-how-tos rule.
47
-
48
-Unknowns:
49
-
50
-- Whether `codacy-analysis-cli` reads `.codacy.yml` exclude_paths the same way Codacy CI does. Will verify during implementation by running the CLI on PR #22423's pre-exclusion state and counting markdownlint findings.
51
-- Whether running the CLI in docker against a docker-mounted source tree produces clean enough output to redirect into a JSON dump. Will pick `--format json` (or `--format sarif`) at implementation time.
52
-- Whether the bundled tools list in `codacy-analysis-cli` overlaps 1:1 with what Codacy CI runs on `netdata/netdata`. The 864 finding fixture answers this empirically.
53
-
54
-### Acceptance Criteria
55
-
56
-1. `.agents/skills/codacy-audit/SKILL.md` exists with frontmatter `name=codacy-audit`, description that lists trigger phrases, follows the same shape and conventions as `coverity-audit` / `sonarqube-audit`. Verification: skill loads in agent harness without YAML errors; `head -1` of the description shows the trigger phrasing.
57
-2. `scripts/_lib.sh` ships token-safe wrappers and a no-leak self-test (`codacyaudit_selftest_no_token_leak`). Verification: run the self-test in CI-style with a sentinel UUID; sentinel must NOT appear on captured stdout.
58
-3. `scripts/analyze-local.sh` runs `codacy-analysis-cli` (via docker) and produces a JSON dump under `<repo>/.local/audits/codacy/<timestamp>.json`. Verification: run on the PR-22423 pre-exclusion state, confirm a non-zero count of markdownlint findings, confirm `.codacy.yml` exclusions are honoured by checking that excluded paths produce no rows in post-exclusion runs.
59
-4. `scripts/pr-issues.sh` fetches Codacy issues for an arbitrary PR number via the v3 API, writes a JSON dump under `<repo>/.local/audits/codacy/`, and emits a TSV summary clustering by tool / pattern / severity / file. Verification: run on PR #22423; confirm `total > 0` historical baseline and graceful handling for PRs with `total = 0`.
60
-5. `how-tos/INDEX.md` exists, documents the live-catalog rule, and seeds at least one how-to derived from this SOW's verification work. Verification: file exists; `INDEX.md` lists at least one entry.
61
-6. `<repo>/.agents/ENV.md` lists `CODACY_TOKEN` with role / where-to-find / sample format / which scripts consume it. Verification: grep for `CODACY_TOKEN` returns the new row.
62
-7. `<repo>/.env.template` lists `CODACY_TOKEN=""` with a setup pointer to `.agents/ENV.md`. Verification: grep returns the new line.
63
-8. `AGENTS.md` skill index gains a `codacy-audit` entry under "Legacy runtime skills" (or a new bucket if appropriate). Verification: grep returns the new pointer.
64
-9. `<repo>/.codacy.yml` is unchanged by this SOW (its exclusions are the source of truth; the local CLI must honour them). Verification: `git diff -- .codacy.yml` after this SOW shows no changes.
65
-10. shellcheck on the new scripts: ALL CLEAN with `--external-sources`. Verification: `shellcheck --external-sources .agents/skills/codacy-audit/scripts/*.sh`.
66
-
67
-## Analysis
68
-
69
-Sources checked:
70
-
71
-- `.agents/skills/coverity-audit/SKILL.md` and `scripts/` (sister skill, structural template).
72
-- `.agents/skills/sonarqube-audit/SKILL.md` (sister skill, second structural template).
73
-- `.agents/skills/graphql-audit/` (third sister skill).
74
-- `.agents/skills/query-agent-events/scripts/_lib.sh` (token-safe self-test pattern, current best practice in this repo).
75
-- `.codacy.yml` (current exclusion list).
76
-- `<repo>/.env`, `<repo>/.env.template`, `<repo>/.agents/ENV.md` (env-key surface area).
77
-- Codacy v3 REST API (`api.codacy.com`) -- live confirmed during this conversation.
78
-- `https://github.com/codacy/codacy-analysis-cli` upstream README.
79
-
80
-Current state:
81
-
82
-- Codacy CI runs on every PR; the only feedback path today is the GitHub check.
83
-- 864 markdownlint findings in the recent PR-22423 history confirm the analyzer set we need to match locally.
84
-- `.env` has `CODACY_TOKEN` set; no committed script consumes it yet.
85
-- No `.agents/skills/codacy-audit/` directory exists today.
86
-
87
-Risks:
88
-
89
-- **Docker-CLI path differences**: `codacy-analysis-cli` mounted via docker may interpret paths differently (host path vs container `/src`). Mitigation: standardize on `--directory /src` and bind-mount `<repo>:/src:ro`.
90
-- **Tool drift between local CLI and Codacy CI**: the local CLI bundles a fixed set of analyzers; Codacy Cloud may add/remove tools server-side. Mitigation: document which tools are run locally; for tools Codacy Cloud adds that the CLI doesn't, fall back to the API for ground truth.
91
-- **Token accidentally committed via a finding dump**: a JSON dump from the API could echo back the token in error messages. Mitigation: token-safe wrappers in `_lib.sh` plus the no-leak self-test.
92
-- **`.local/audits/codacy/` filling up with stale dumps**: ephemeral, gitignored, but disk pressure risk on long sessions. Mitigation: filename includes timestamp; user can `rm` whenever.
93
-
94
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
95
-
96
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
97
-
98
-Problem / root-cause model:
99
-
100
-- The user has Codacy Cloud configured for `netdata/netdata` and has placed `CODACY_TOKEN` in `.env`. There is no committed tooling to (a) run the same analyzers locally before pushing or (b) fetch and triage Codacy findings on a PR programmatically. The recent PR #22423 demonstrated the cost of this gap: 864 findings were only visible after the CI round-trip. Building a small skill captures the operational knowledge and shrinks the loop.
101
-
102
-Evidence reviewed:
103
-
104
-- Codacy v3 API live response: account auth verified (`api-token` header), `master` branch carries 31,425 open issues, PR-22423 carried 864 issues all `markdownlint`.
105
-- `.agents/skills/coverity-audit/SKILL.md` lines 1-40 -- frontmatter shape and "MANDATORY" startup sequence.
106
-- `.agents/skills/query-agent-events/scripts/_lib.sh` -- current token-safe pattern (`agentevents_selftest_no_token_leak` style; sentinel UUID drives every public wrapper, asserts no leak on stdout).
107
-- `https://github.com/codacy/codacy-analysis-cli` README -- docker invocation: `docker run --rm -v "$PWD":/src codacy/codacy-analysis-cli:latest analyze --directory /src`.
108
-
109
-Affected contracts and surfaces:
110
-
111
-- New skill: `.agents/skills/codacy-audit/`.
112
-- New audit dir: `<repo>/.local/audits/codacy/` (gitignored; created by skill scripts at runtime).
113
-- ENV surface: `CODACY_TOKEN` (new row in `.agents/ENV.md` and `.env.template`).
114
-- AGENTS.md skill index.
115
-- No changes to `.codacy.yml`, no changes to existing skills, no changes to source code.
116
-
117
-Existing patterns to reuse:
118
-
119
-- coverity-audit's "MANDATORY -- keep this skill alive" + "MANDATORY -- startup sequence" SKILL.md sections.
120
-- query-agent-events's `_lib.sh` token-safe wrapper layout (env load with `: "${VAR:?}"`, audit-dir helper, masked-token `_run`/`_run_read` form, sentinel self-test).
121
-- The how-tos catalog rule: assistants author a how-to whenever they perform analysis the catalog doesn't already cover.
122
-- The `<topic>-audit` -> `.local/audits/<topic>/` directory naming convention from AGENTS.md (so `codacy-audit/` writes to `.local/audits/codacy/`).
123
-
124
-Risk and blast radius:
125
-
126
-- **Local-only**: nothing in this SOW ships to end-user agents, the Cloud, or the binary. Blast radius = this repo's `.agents/` tree + ENV.md.
127
-- **Reversibility**: every artifact is gitignored or under `.agents/skills/codacy-audit/` (deletable in one commit). No destructive operations.
128
-- **Security**: token-safe `_lib.sh` plus the self-test ensure the token never reaches captured stdout. Audit dumps go to gitignored `.local/`.
129
-- **Performance**: docker pull on first run (~few hundred MB); subsequent runs warm-cached.
130
-- **Compatibility**: targets the Codacy API as it exists today; if Codacy changes auth/headers, the skill breaks loudly with a 401 and the lib's error message points the user at `.agents/ENV.md`.
131
-
132
-Sensitive data handling plan:
133
-
134
-- `CODACY_TOKEN` is a credential -- handled exactly like `NETDATA_CLOUD_TOKEN`, `COVERITY_COOKIE`, `SONAR_TOKEN`: lives in `.env` (gitignored), referenced via `${CODACY_TOKEN}` in scripts only, never in commit messages, never in fixtures.
135
-- The configured Codacy account was returned by `/v3/user` during exploration but will not be written to any committed artifact. The SOW redacts to "the configured account".
136
-- Audit JSON dumps land under `<repo>/.local/audits/codacy/<timestamp>.json` (gitignored).
137
-- No customer / community member / private-host data is touched -- this is a public-repo CI workflow.
138
-
139
-Implementation plan:
140
-
141
-1. **Skill scaffolding** (10 min): create `.agents/skills/codacy-audit/`, write `SKILL.md` with frontmatter, MANDATORY sections, table of contents, env-keys table, related-skills cross-refs.
142
-2. **`scripts/_lib.sh`** (20 min): env load (validates `CODACY_TOKEN`), audit dir helper, host detection (`api.codacy.com`), masked-token `codacyaudit_run`/`codacyaudit_run_read`, sentinel-based no-leak self-test (`codacyaudit_selftest_no_token_leak`). Sentinel UUID drives every public wrapper; capture stdout; assert sentinel absence.
143
-3. **`scripts/analyze-local.sh`** (30 min): docker invocation of `codacy-analysis-cli analyze --directory /src --format json --output <audit>/<ts>.json`. CLI args: `--tool <name>` (optional), `--upload` disabled (we do read-only locally). Fall back to script-install if docker unavailable.
144
-4. **`scripts/pr-issues.sh`** (30 min): paginated fetch of `/v3/analysis/organizations/gh/<org>/repositories/<repo>/pull-requests/<n>/issues`; default `org=netdata`, `repo=netdata`. Output: full JSON dump + TSV summary `(count, tool, pattern, severity)`. Token used via `_lib.sh` wrappers (token-safe).
145
-5. **`how-tos/INDEX.md`** (10 min): catalog file with the live-rule paragraph; seed with at least one how-to (the PR-22423 reproduction).
146
-6. **`.agents/ENV.md` + `.env.template`** (10 min): add `CODACY_TOKEN` row in the per-skill checklist; add CODACY section in template; cross-link to skill.
147
-7. **`AGENTS.md`** (5 min): add `codacy-audit` pointer under "Legacy runtime skills" (or "Skill index" if a new bucket is more honest -- decision while writing).
148
-8. **End-to-end validation** (20 min): run `analyze-local.sh` on PR-22423 fixture; run `pr-issues.sh` on PR #22423; confirm artifact shape; confirm shellcheck clean; run no-leak self-test.
149
-9. **SOW lifecycle**: move SOW from `pending/` to `current/` before step 2; close as `completed` and move to `done/` in the same commit as the work.
150
-
151
-Validation plan:
152
-
153
-- `shellcheck --external-sources .agents/skills/codacy-audit/scripts/*.sh` -- ALL CLEAN.
154
-- `_lib.sh` self-test exercises every public wrapper with a sentinel UUID; the sentinel must not appear on captured stdout.
155
-- `analyze-local.sh` on the working tree produces a JSON dump; markdownlint findings count is sane (matches Codacy CI within ~10% on a known-state branch).
156
-- `pr-issues.sh 22423` on the current PR returns 0 findings (we already excluded the trees).
157
-- `pr-issues.sh 22420` (the most recent merged PR) returns 0 findings (it merged with Codacy `pass`).
158
-- Skill loads cleanly in the agent harness: no YAML frontmatter errors.
159
-- `git diff -- .codacy.yml` shows no changes.
160
-
161
-Artifact impact plan:
162
-
163
-- AGENTS.md: add `codacy-audit` to the skill index. Justification: the project's skill index is the canonical lookup for AI assistants.
164
-- Runtime project skills: no `project-*` skill changes. The codacy-audit skill is a legacy-style audit skill (mirrors coverity-audit), not a `project-*` per-repo runtime skill.
165
-- Specs: `.agents/sow/specs/sensitive-data-discipline.md` already covers CODACY-class tokens by precedent. Add a row for `CODACY_TOKEN` to the env-keys table for clarity. (Small spec update, not a behavior change.)
166
-- End-user/operator docs: none affected.
167
-- End-user/operator skills: none affected.
168
-- SOW lifecycle: move `pending/ -> current/ -> done/`; status `open -> in-progress -> completed`. Close in the same commit as the work, per AGENTS.md rule.
169
-
170
-Open-source reference evidence:
171
-
172
-- `codacy/codacy-analysis-cli` upstream README and `--help`. No further mirrored-repos research is required for this SOW.
173
-
174
-Open decisions:
175
-
176
-- None blocking. Scope was decided by user accepting the assistant's recommendation ("track 1 only -- CLI wrapper plus read-only PR queries").
177
-
178
-## Implications And Decisions
179
-
180
-### Decision 1 - Scope cut
181
-
182
-**Options (presented earlier in conversation, recorded here):**
183
-
184
-1. Both tracks, full skill (CLI wrapper + read-only API + write actions + master backlog triage).
185
-2. Track 1 only -- CLI wrapper + read-only PR-issue queries (deferring write actions).
186
-3. Defer entirely.
187
-
188
-**User selected**: option 2 ("ok" -- accepting assistant's recommendation: "Start with track 1 only (the local CLI wrapper) as a small SOW. Track 2 (triage workflow) becomes worthwhile once we hit a PR or quarterly review where Codacy backlog matters -- defer until then.").
189
-
190
-**Implication**: deferred follow-up SOW for write actions and master-backlog triage. Acceptable: token is exercised by read-only wrappers, the lib has the no-leak self-test in place, future expansion is a thin addition.
191
-
192
-### Decision 2 - Local CLI install path
193
-
194
-**Options:**
195
-
196
-1. Require docker (run `codacy/codacy-analysis-cli:latest`).
197
-2. Require the install.sh-installed binary (`/usr/local/bin/codacy-analysis-cli`).
198
-3. Auto-detect: prefer local binary, fall back to docker.
199
-
200
-**Selected**: option 3 (auto-detect). Reasoning: docker is universally available on this workstation but not always preferable (cold-pull cost, root requirements in some contexts). Auto-detection matches sister-skill behavior.
201
-
202
-## Plan
203
-
204
-1. Move SOW to `current/` and flip status to `in-progress`.
205
-2. Build skill scaffolding + scripts (steps 1-7 of the implementation plan above).
206
-3. Run the validation suite (shellcheck, no-leak self-test, end-to-end PR-22423 fixture).
207
-4. Move SOW to `done/`, flip to `completed`, commit work + SOW lifecycle change as one commit.
208
-5. Push.
209
-
210
-## Execution Log
211
-
212
-### 2026-05-05
213
-
214
-- SOW drafted in `pending/`; promoted to `current/` and `Status: in-progress`.
215
-- Built `.agents/skills/codacy-audit/` mirroring `coverity-audit/` shape:
216
- - `SKILL.md` with frontmatter, MANDATORY sections, scope (in/out), env keys table, scripts table, workflow examples, related-skills cross-refs, path discipline.
217
- - `scripts/_lib.sh` with token-safe wrappers (`_codacyaudit_run` internal, `codacyaudit_get`/`codacyaudit_post`/`codacyaudit_get_paged` public), env load with `: "${CODACY_TOKEN:?}"`, audit-dir helper, `codacyaudit_pr_issues` and `codacyaudit_repo_info` convenience wrappers, sentinel-based `codacyaudit_selftest_no_token_leak`.
218
- - `scripts/pr-issues.sh` -- paginated PR issue fetch + clustered TSV summary; default `--by pattern`; supports `--by tool|severity|file|category`.
219
- - `scripts/analyze-local.sh` -- auto-detect runner (local binary vs docker); docker path uses `--volume /var/run/docker.sock:/var/run/docker.sock` plus `--env CODACY_CODE=<host-path>` plus same-path bind mount, per Codacy upstream docs.
220
- - `how-tos/INDEX.md` with the live-rule paragraph; seeded with `reproduce-pr-22423-markdownlint.md`.
221
-- Updated `<repo>/.agents/ENV.md`: added Codacy section with per-key role/where/format and a per-skill checklist row.
222
-- Updated `<repo>/.env.template`: added a `CODACY_TOKEN=""` block with inline guidance and optional override placeholders for host/provider/org/repo.
223
-- Updated `AGENTS.md` skill index: added `codacy-audit` under "Legacy runtime skills" and the brief skill list.
224
-- Updated `.agents/sow/specs/sensitive-data-discipline.md`: added a `CODACY_TOKEN` row to the env-keys table.
225
-- First self-test under zsh produced a `BASH_SOURCE[0]: parameter not set` warning (PASS still emitted). Fixed by mirroring the `query-agent-events/_lib.sh` portable `_codacyaudit_lib_self` resolution. Re-ran; clean PASS in both shells.
226
-- First docker run hit `Cannot connect to the Docker daemon` from inside the CLI container (the CLI spawns child containers per tool). Fixed by adopting the upstream docker-in-docker invocation: mount `/var/run/docker.sock`, set `CODACY_CODE`, bind-mount the source at the same path inside the container.
227
-- Verified `pr-issues.sh 22423` -> 0 issues (post-exclusion); `pr-issues.sh 22420` -> 0 issues (a recent merged PR with Codacy `pass`). Both produce well-formed JSON dumps under `<repo>/.local/audits/codacy/`.
228
-- Verified `analyze-local.sh --tool markdownlint` on a synthetic markdown fixture: produced 2 findings (MD013 + MD022) -- end-to-end docker-in-docker dispatch confirmed working for markdownlint. The shellcheck child-tool path failed in a smaller smoke test ("Is a directory") -- this is a CLI-version-specific quirk in shellcheck dispatch on a single-file directory; not in scope to fix here. The reproduce-PR-22423 how-to (markdownlint) is the validated path.
229
-- Closed SOW: `Status: completed`, moved to `done/`, committed alongside the work.
230
-
231
-## Validation
232
-
233
-Acceptance criteria evidence:
234
-
235
-1. **SKILL.md exists with proper frontmatter** -- `.agents/skills/codacy-audit/SKILL.md` has `name: codacy-audit` and a description listing trigger phrases. Loaded by the agent harness during this session (the harness's available-skills list shows `codacy-audit` immediately after the file landed).
236
-2. **`_lib.sh` token-safe self-test** -- `codacyaudit_selftest_no_token_leak` PASS under both bash and zsh; sentinel UUID `deadbeef-1234-5678-9abc-def012345678` does not appear on captured stdout from any of `codacyaudit_get`, `codacyaudit_post`, `codacyaudit_pr_issues`, `codacyaudit_repo_info`.
237
-3. **`analyze-local.sh` end-to-end** -- markdown fixture produced a JSON dump containing 2 findings (`markdownlint_MD013` + `markdownlint_MD022`); dump path is `<repo>/.local/audits/codacy/local-markdownlint-<ts>.json`; format is the Codacy `Issue` array.
238
-4. **`pr-issues.sh` end-to-end** -- PR #22423 (post-exclusion) and PR #22420 (recent merged PR) both return 0 issues; the script handles `total = 0` gracefully and does not error.
239
-5. **`how-tos/INDEX.md` exists** -- live catalog rule documented; seeded with `reproduce-pr-22423-markdownlint.md`.
240
-6. **`.agents/ENV.md` updated** -- `CODACY_TOKEN` row added with role/where/format and per-skill checklist entry.
241
-7. **`.env.template` updated** -- `CODACY_TOKEN=""` line plus optional override placeholders added.
242
-8. **AGENTS.md skill index updated** -- `codacy-audit` entry under "Legacy runtime skills" and the brief skill list.
243
-9. **`.codacy.yml` unchanged** -- `git diff -- .codacy.yml` after this SOW: empty.
244
-10. **shellcheck CLEAN** -- `shellcheck --external-sources` over all three new scripts: ALL CLEAN.
245
-
246
-Tests or equivalent validation:
247
-
248
-- `shellcheck --external-sources` over `.agents/skills/codacy-audit/scripts/*.sh` -> ALL CLEAN.
249
-- `bash -n` over each script -> OK.
250
-- `--help` on each script -> exit 0.
251
-- `codacyaudit_selftest_no_token_leak` PASS under both bash and zsh.
252
-- `pr-issues.sh 22423` and `pr-issues.sh 22420` -> 0 findings each, JSON dump on disk.
253
-- `analyze-local.sh --directory /tmp/<fixture> --tool markdownlint` -> 2 findings, JSON dump on disk.
254
-
255
-Real-use evidence:
256
-
257
-- `pr-issues.sh 22423` and `pr-issues.sh 22420` were run live against `api.codacy.com` with the configured `CODACY_TOKEN`; both completed without auth or pagination errors. JSON dumps under `<repo>/.local/audits/codacy/` confirm the v3 envelope shape.
258
-- `analyze-local.sh` was run live against the docker image `codacy/codacy-analysis-cli:latest` (pulled fresh from Docker Hub during validation); markdownlint dispatch produced 2 findings on a synthetic fixture.
259
-
260
-Reviewer findings:
261
-
262
-- Self-review: covered. The "Is a directory" failure on the shellcheck child-tool path is a CLI-version-specific quirk on a 1-file directory and is logged as a known limitation in the Execution Log; the markdownlint path (which the seeded how-to uses) works.
263
-
264
-Same-failure scan:
265
-
266
-- `grep -rn -E 'BASH_SOURCE\[0\]' .agents/skills/` -- only the codacy-audit and query-agent-events libs use it; both now wrap with the zsh-compat resolver.
267
-- `grep -rn -E '_run -d "' .agents/skills/codacy-audit/scripts/` -- no callers expose token bytes via `-d` argument quoting traps.
268
-
269
-Sensitive data gate:
270
-
271
-- No raw tokens, account UUIDs, customer-identifying IPs, or private endpoints were written to any committed artifact. The configured Codacy account was returned by `/v3/user` during exploration and is intentionally not committed; this SOW redacts to "the configured account".
272
-- `CODACY_TOKEN` is referenced via `${CODACY_TOKEN}` only, never literally.
273
-- Dumps land under `<repo>/.local/audits/codacy/` (gitignored).
274
-
275
-Artifact maintenance gate:
276
-
277
-- AGENTS.md: updated -- `codacy-audit` entry added to "Legacy runtime skills" section and to the brief skill list.
278
-- Runtime project skills: not affected -- this is a legacy-style audit skill, not a `project-*` skill.
279
-- Specs: updated -- `CODACY_TOKEN` row added to `<repo>/.agents/sow/specs/sensitive-data-discipline.md` env-keys table.
280
-- End-user/operator docs: not affected -- skill is internal AI tooling.
281
-- End-user/operator skills: not affected.
282
-- SOW lifecycle: `pending/ -> current/ -> done/`; `Status: open -> in-progress -> completed`; closed in the same commit as the work per AGENTS.md rule.
283
-
284
-Specs update:
285
-
286
-- `.agents/sow/specs/sensitive-data-discipline.md` -- env-keys table gained the `CODACY_TOKEN` row.
287
-
288
-Project skills update:
289
-
290
-- None applicable -- codacy-audit is a legacy-style audit skill, mirroring `coverity-audit/` etc.
291
-
292
-End-user/operator docs update:
293
-
294
-- None applicable -- this is internal AI-assistant tooling.
295
-
296
-End-user/operator skills update:
297
-
298
-- None applicable.
299
-
300
-Lessons:
301
-
302
-- The Codacy CLI's docker-in-docker model is not optional; without `/var/run/docker.sock` mounted in the CLI container the inner per-tool containers cannot start. The upstream README is explicit; mirror their invocation verbatim.
303
-- Account API tokens authenticate via `api-token: <value>` (NOT `Authorization: Bearer`). A 401 with `Bad credentials` is the typical sign of using the wrong header.
304
-- When `.env` values are single-quoted, `cut -d'"' -f2` and `cut -d'=' -f2-` both leave the quotes intact. Always source `.env` via bash (`set -a; . .env; set +a`) so the shell strips the quotes natively.
305
-- Sister-skill `_lib.sh` files use a portable `if [ -n "${ZSH_VERSION-}" ]` block to resolve their own path; reuse it for any new audit-style skill so sourcing under zsh does not warn.
306
-
307
-Follow-up mapping:
308
-
309
-- Write actions (mark FP, mark fixed, modify ignore-patterns via API): TRACKED. To be opened as a future SOW when the user has a real triage need.
310
-- Master-backlog triage on the 31,425 open issues: TRACKED. Same condition.
311
-- The shellcheck child-tool dispatch quirk: TRACKED. To be revisited only if the user hits it during real-use; otherwise the markdownlint path covers the value-add.
312
-
313
-## Outcome
314
-
315
-The codacy-audit skill is live under `.agents/skills/codacy-audit/`. Pre-push prevention is one command (`analyze-local.sh`); PR triage is one command (`pr-issues.sh <N>`). Both are token-safe, shellcheck-clean, and pass an end-to-end round-trip against `api.codacy.com` and the official Codacy CLI image. The skill mirrors the coverity / sonarqube / graphql audit family conventions so an assistant familiar with one can use this one without re-learning.
316
-
317
-## Lessons Extracted
318
-
319
-See the "Lessons" subsection of Validation above.
320
-
321
-## Followup
322
-
323
-- Future SOW for write actions (mark FP / mark fixed) -- not opened yet; will open when there is a real PR triage need.
324
-- Future SOW for master-backlog triage -- not opened yet; will open when there is a quarterly-review need.
325
-
326
-## Regression Log
327
-
328
-None yet.
329
-
330
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0012-20260505-streaming-topology-classification-bugs.md
deleted
-548
@@ -1,548 +0,0 @@
1
-# SOW-0012 - Streaming topology: parent/child classification, missing-parent rendering, and "Since" column type
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed after PR merge. User confirmed fixes merged in commit `cc50307bc6ca180285a0e0cda16d73ab7a42cd86` (`Fix streaming topology graph output (#22432)`) on 2026-05-07.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Make the `topology:streaming` Function (`function-streaming.c::function_streaming_topology`) produce correct, useful output on **both sides of a streaming relationship**: when invoked on a parent agent and when invoked on a child agent. The view from a child must (a) classify the child correctly, (b) render its upstream parent(s) as actors, and (c) render time fields in human form.
14
-
15
-This is in service of the broader topology PR (https://github.com/netdata/netdata/pull/22110, merged April 14 as commit `7da4565`) being usable by SREs without manual decoding of Unix epoch values or manual reasoning about misclassified node types.
16
-
17
-### User Request
18
-
19
-The reporter raised four concerns, summarised:
20
-
21
-1. "Child agent is identified as parent — bug related to vnodes?"
22
-2. "In the streaming topology, the 'Since' field shows number 1,777,992,161 — assume this is a timestamp not rendered properly."
23
-3. "I also hoped to have it show its parents."
24
-4. "The streaming topology function on the child seems broken. No facets, no filters, nothing."
25
-
26
-The reporter shared a screenshot of `topology:streaming on [child-1] ([CHILD_IP])` with `0 results` in the result counter, the child's modal popup showing `Type: parent`, `Children: 1`, and a streaming-path table where the `Since` column rendered raw integers (`1,777,992,161` and `1,777,992,171`).
27
-
28
-### Assistant Understanding
29
-
30
-Facts (verified via live Cloud queries on the reporter's space, 2026-05-05; raw responses preserved at `<repo>/.local/audits/streaming-topology/` which is gitignored):
31
-
32
-- Topology: `[parent-1]` is the parent; `[child-1]` is the only real streaming child. Six other actors (legacy stale streams) appear as `vnode` (marked virtual).
33
-- On the **parent** (`[parent-1]`) view: `actors_n=8`, `links_n=7`. **`[parent-1]` itself is classified `node_type:"child"`, `child_count:0`** — wrong; it has 1 active streaming child. `[child-1]` is correctly `child`. Link `[child-1] → [parent-1]` correctly emitted with `link_type:"streaming"`.
34
-- On the **child** (`[child-1]`) view: `actors_n=1`, `links_n=0`. The single actor is `[child-1]` itself, classified `node_type:"parent"` with `child_count:1`. `[parent-1]` (the actual parent) is **not present as an actor**. The actor's `streaming_path` field is `[[child-1], [parent-1]]` — so the path data is known to the function, but is not turned into a second actor.
35
-- The streaming-path table on each actor (emitted via `rrdhost_stream_path_to_json` at function-streaming.c:1327) carries `since` as raw Unix epoch seconds (`1777992161` for the child, `1777992171` for the parent).
36
-- Function name is `topology:streaming` (registered at functions.c:23). Cloud Function URL: `POST /api/v2/nodes/{nodeId}/function?function=topology:streaming`.
37
-
38
-Inferences:
39
-
40
-- The "no facets, no filters" symptom on the child is a consequence of the response being almost empty (1 actor, 0 links). Once the child's view contains a correctly classified child plus a synthetic parent actor plus the upstream link, the FE will have data to populate facets/filters from.
41
-- The misclassification on the parent and the misclassification on the child are produced by **different bugs that happen to land on the same `parent_child_count` dictionary** — see Analysis.
42
-
43
-Resolved (after iteration-2 review with cloud-frontend access):
44
-
45
-- The cloud-frontend's topology table renderer accepts `"timestamp"` columns in **either seconds or milliseconds**: at `cloud-frontend/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` it auto-detects via `ms > 1e12 ? ms : ms * 1000`. The agent emits `since` in seconds; the FE multiplies by 1000 → correct date rendering. Decision 4 changes accordingly (Option 2 chosen — see below).
46
-- Re-emitting `stream_path_send_to_parent(host)` after a parent-originated path update **does** create an oscillation risk. `rrdhost_stream_path_to_json` overlays a fresh `rrdhost_stream_path_self()` at every emit (stream-path.c:181-202), and that fresh self pulls live values (`first_time_t` from `rrdhost_retention()`, `start_time_ms`/`shutdown_time_ms` from `get_agent_event_time_median()`) that change continuously. The XXH128 hash is computed over stored content but the emit produces fresh content each time, so the receiving side's hash differs every round-trip → perpetual re-sends. This rules out a naive protocol fix and reinforces the SOW's original Option 3 Hybrid recommendation for Decision 1.
47
-
48
-### Acceptance Criteria
49
-
50
-- On the **parent view**: parent's actor has `node_type:"parent"` and `child_count >= 1`. Verification: `agents_call_function --via cloud --node $PARENT --function topology:streaming` returns `actors[0].attributes` with `node_type=="parent"` and `child_count > 0` for the localhost actor.
51
-- On the **child view**: child's actor has `node_type:"child"` and `child_count==0`. Verification: same Cloud call against the child's node id; `actors[0].attributes.node_type=="child"`.
52
-- On the **child view**: at least one synthetic actor for the parent is present, and a `link` from child to parent with `link_type=="streaming"`. Verification: `data.actors[]` contains an entry for the parent's machine GUID with `actor_type=="parent"` (the existing actor type — no new type introduced; see Decision 3); `data.links[]` contains an entry with `src_actor_id` = child and `dst_actor_id` = parent. For deeper chains, `data.links[]` contains links between every consecutive non-localhost path slot.
53
-- The `Since` column in the streaming-path table on each actor renders as a human date in the FE, not a raw integer. Verification: visual check in Netdata Cloud UI on both parent and child views.
54
-- The backend always exposes the complete topology graph. `topology:streaming` must not accept or apply backend graph-pruning filters such as `node_type`, `ingest_status`, or `stream_status`. Facets or filtering in the UI are client-side concerns over the complete response. Verification: `accepted_params` contains only non-pruning params, currently `info`, and actors/links are emitted without backend filter checks.
55
-- The two functions previously co-located in `src/web/api/functions/function-streaming.{c,h}` are split into one file each: `function-netdata-streaming.{c,h}` (registered as `netdata-streaming`) and `function-topology-streaming.{c,h}` (registered as `topology:streaming`). The simple-table function is renamed `function_streaming` → `function_netdata_streaming` for symbol/file consistency. Verification: the old files are removed; both new files compile and the registered Function names continue to work end-to-end via Cloud calls.
56
-- A maintenance reference document `src/streaming/STREAM_PATH.md` is added describing the streaming-path subsystem (storage rule, propagation rules, cycle terminator, update triggers, vnode special case, intended classification logic, mergeability requirements). Verification: the file exists, is **not referenced in `docs/.map/map.yaml`** (so not picked up by the learn-ingestion pipeline; co-location under `src/streaming/` is not by itself sufficient — `src/streaming/README.md` IS in the map and gets ingested), and covers the nine outline points in the Plan section below.
57
-- Tests covering the four root causes — see Validation Plan.
58
-
59
-## Analysis
60
-
61
-Sources checked:
62
-
63
-- `src/web/api/functions/function-streaming.c` (full file, 2459 lines)
64
-- `src/web/api/functions/function-streaming.h`
65
-- `src/web/api/functions/functions.c` (Function registration)
66
-- `src/streaming/stream-path.c` (full file)
67
-- `src/streaming/stream-path.h`
68
-- `src/streaming/stream-sender.c:147-161` (`stream_sender_on_ready_to_dispatch`, only caller of `stream_path_send_to_parent` from connection setup)
69
-- `src/streaming/stream-sender-execute.c:109-110` (sender-side receive of upstream STREAM_PATH)
70
-- `src/plugins.d/pluginsd_parser.c:1211` (parser-side receive of STREAM_PATH from a child)
71
-- `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` (FE timestamp handling — auto-detects seconds vs ms)
72
-- `<dashboard-repo>/src/domains/functions/topology/payload.js` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` (FE actor-id dedup, link-derived synthesis)
73
-- Live data: sanitized summaries derived from raw Function responses stored under `<repo>/.local/audits/streaming-topology/` (gitignored)
74
-- Reporter screenshot of the broken UI (saved locally; not committed)
75
-
76
-### Intended classification logic (the rule the function should implement)
77
-
78
-Confirmed with project owner 2026-05-06. The topology function is meant to classify each entry in `rrdhost_root_index` according to the following rule:
79
-
80
-1. **Source of truth = `rrdhost_root_index`.** This is the local agent's authoritative list of nodes it knows about: localhost, every real child currently or previously connected (including stale/archived ones), and every vnode registered locally.
81
-2. **For each non-vnode node, read its `stream.path.array`.** Slot 0 is the origin of that chain (the originating "child"). Slots 1+ are the upstream hops.
82
-3. **Tag entries by slot:**
83
- - Slot 0 → origin (a child role in this chain).
84
- - Slots 1+ → parent role in this chain.
85
-4. **Globally,** a node X is classified as `parent` if X appears at slot 1+ in **any** node's path. Otherwise X is `child`.
86
-5. **vnodes are special.** Their stored path is empty (vnodes do not stream themselves — their data is collected by the local agent). At JSON-emit time, slot 0 of a vnode's path is filled by the collecting agent (via emit-time self-append). vnodes are tagged `vnode` directly and skip the slot-1+ counter step.
87
-6. **Stale nodes are special.** A node whose ingest status is `RRDHOST_INGEST_STATUS_ARCHIVED` is tagged `stale` directly and rendered with a stale link back to the collecting agent (localhost).
88
-
89
-Both Bug A and Bug B sit on the same step (#3 — slot-1+ extraction feeding the global classification at #4) but fail for different reasons. Bug A: the data being read at slot 1+ is incomplete (stored paths haven't converged yet on the apex parent). Bug B: the slot-1+ extraction itself is contaminated (the helper appends localhost into a result that is supposed to contain only upstream entries). Together they cause the classification to be wrong on **both** sides of the same parent-child relationship — the parent shows as a child (Bug A), the child shows as a parent (Bug B).
90
-
91
-### Intended actor set per agent
92
-
93
-Confirmed with project owner 2026-05-06. From any single agent's perspective, the topology view emits the following actors. All edges/chains route through `self`; nothing branches around it.
94
-
95
-| Source | Actor type | Where the data comes from | Link to self |
96
-|---|---|---|---|
97
-| `self` (localhost) | `parent` (if it has any active streaming children) or `child` (otherwise) | `rrdhost_root_index` (localhost) | — (origin of self's chain) |
98
-| Active vnodes | `vnode` | `rrdhost_root_index` (entries with `RRDHOST_OPTION_VIRTUAL_HOST`) | `vnode → self`, link_type derives from how the agent collects the vnode |
99
-| Active children (real, currently streaming in) | `child` | `rrdhost_root_index` (entries with active receiver, not virtual, not archived) | `child → self`, link_type `streaming` |
100
-| Stale children (archived, last seen on this agent) | `stale` | `rrdhost_root_index` (entries with `RRDHOST_INGEST_STATUS_ARCHIVED`) | `stale → self`, link_type `stale` |
101
-| Upstream parents of self | `parent` (existing actor type, no new type introduced) | self's `stream.path.array` slots 1+ (synthesized actors — they are not in `rrdhost_root_index`) | `self → parent_1`, `parent_1 → parent_2`, ... — link_type `streaming`, one link per consecutive non-localhost path slot |
102
-
103
-Children's full chains, after path convergence, look like `[child, self, parent_1, parent_2, ...]`. The tail `[self, parent_1, ...]` is identical to self's own chain, so after dedup by `actor_id`, the children's chains add no new upstream actors — they share self's chain. Same for vnodes and stale children.
104
-
105
-Counting example: an agent with 100 vnodes + 100 active children + 20 stale children + 2 upstream parents emits **223 actors total** (1 self + 100 + 100 + 20 + 2) and **222 unique links** (100 + 100 + 20 + 1 + 1).
106
-
107
-Sizing note: this matches the original PR description's caveat that very large topologies produce large responses (carried over from the unfixed scope, not introduced by this SOW). The synthesized upstream `parent` actors are bounded by the chain depth (typically 1–3), so this SOW does not add a sizing risk.
108
-
109
-### Topology must be mergeable across agents
110
-
111
-Confirmed with project owner 2026-05-06. The Cloud frontend (or any consumer) must be able to merge the per-agent topology responses from N parents into a single unified view. The agent's response shape and per-actor / per-link attributes must support this.
112
-
113
-What each parent contributes to a merged view (where it does not overlap with what other parents already contributed):
114
-
115
-- **Retention** for each child the parent received data from — already emitted per-actor in the `retention` and (per-host) inbound/outbound tables.
116
-- **Its own vnodes** — vnodes are local to the collecting agent, so two parents typically do not overlap on vnodes (unless misconfigured).
117
-- **Its own stale children** — these can overlap (a child that streamed to A, then moved to B, will appear `stale` on A and `child` on B). The merge must pick the authoritative current state.
118
-- **Its own active children and upstream parents**, where they don't overlap.
119
-
120
-Mergeability requirements satisfied by the current response schema:
121
-
122
-1. **Stable, deterministic actor IDs.** Every actor's `actor_id` is `netdata-machine-guid:<guid>` produced by `streaming_topology_actor_id_from_guid` (`function-streaming.c:107-115`). The same guid produces the same id across agents. Merge dedups by id.
123
-2. **Stable link tuples.** Each link is keyed by (`src_actor_id`, `dst_actor_id`, `link_type`). Same src + dst + type across two parents = the same link.
124
-3. **Timestamps for tie-breaking.** Each link carries `discovered_at` (when this agent first saw the connection) and `last_seen` (the most recent observation). When the same actor appears as `child` on one parent and `stale` on another, the merge can compare timestamps and decide the authoritative current state (the `last_seen` of the streaming link should be more recent than the `last_seen` of the stale link). The Bug C synthesis must preserve these fields on synthesized actors and links — derive `since`/timestamps from the `STREAM_PATH` struct fields (`since`, `first_time_t`).
125
-4. **No agent claims authority over hosts it doesn't see directly.** Each parent reports only the children/vnodes that connect to it, plus its own upstream chain. A child of parent-A is not represented in parent-B's response unless that child has also (currently or historically) connected to B.
126
-
127
-Implication for Bug C synthesis: a synthesized `parent` actor on a child's view must use the same canonical id format (`netdata-machine-guid:<guid>`) so dedup by `actor_id` works across responses. Synthesized links must carry `discovered_at`/`last_seen` derived from the `STREAM_PATH` struct's `since` and `first_time_t` fields (NOT from `now`), so two parents reporting the same upstream produce comparable timestamps and the merge layer can resolve overlaps deterministically.
128
-
129
-The aggregator does not need a new "self" marker on the response. The existing top-level `data.agent_id` (function-streaming.c:840-841) carries the emitting agent's GUID; the aggregator can compute the corresponding self-actor_id as `netdata-machine-guid:<agent_id>` to find the localhost actor in `actors[]`. Authority resolution per actor: when actor X appears in multiple responses, the response whose `agent_id` matches X's GUID is authoritative for X's attributes (full retention, full inbound/outbound, etc.); other responses contribute confirmations and link records but do not override.
130
-
131
-Implication for the migration scenario (a child moved from A to B): both A's and B's responses are valid local truth. The merge layer holds both link records (`child→A stale` and `child→B streaming`), with `last_seen` distinguishing them; the actor's "current state" in the merged view derives from the most recent link.
132
-
133
-Current state:
134
-
135
-The function has four observable defects. Each is documented with file:line evidence and a concrete reproduction trace. Bugs A and B are **distinct root causes** that both feed the same `parent_child_count` dictionary; Bugs C and D are independent of A and B. Fixing any one bug does not fix the others.
136
-
137
-### Bug A — On a parent's view, the parent classifies itself as `child`
138
-
139
-Symptom: `actors[0]` in the parent's response: `actor_type:"child"`, `attributes.child_count:0`. The localhost is classified as a non-parent on its own view.
140
-
141
-Root cause (mapped to the intended logic above): step #3 (slot-1+ extraction) reads stored paths that haven't converged yet. The data feeding the classification is incomplete — the parent does not yet appear at slot 1+ in the child's stored path on the apex parent's side, because convergence is sparse-trigger-driven and hasn't fired since the child connected.
142
-
143
-Design context (verified by code review and confirmed with project owner):
144
-
145
-The streaming-path subsystem is designed for full multi-hop convergence. In a chain `child → proxy → grandparent → ...`, every agent in the chain eventually stores the full path for any host whose data flows through, by this mechanism:
146
-
147
-- **From-below (`from_parent=false`):** store, then propagate UP via `stream_path_send_to_parent(host)` (works on a proxy because each forwarded host has its own sender — `stream-receiver-connection.c:188` creates the child host record with a sender when the local agent has streaming forwarding configured) and DOWN via `stream_path_send_to_child(host)`. Bidirectional.
148
-- **From-above (`from_parent=true`):** store, propagate DOWN only. The `if(!from_parent)` guard at stream-path.c:423 is the cycle terminator.
149
-- **Self is appended at JSON-emit time** (`rrdhost_stream_path_to_json` at stream-path.c:198-202), not at storage time. This avoids storing stale local fields (timing medians, retention edges) that would diverge from "live" self.
150
-- **Update triggers** are sparse: connect, retention boundary movement (`first_time_s` change at `rrdcontext-worker.c:99-104`), node_id assignment, parent disconnect.
151
-
152
-Trace (verified) — why the bug appears on the **apex parent** in this specific topology:
153
-
154
-1. The child starts streaming to the parent. The child's stored `stream.path.array` is initially empty.
155
-2. On `stream_sender_on_ready_to_dispatch` (stream-sender.c:157), the child calls `stream_path_send_to_parent(localhost)`. The payload, built by `rrdhost_stream_path_to_json` (stream-path.c:177-209), contains only the appended `tmp = rrdhost_stream_path_self(localhost)` entry — `[{child, hops=0}]`.
156
-3. The parent receives at `pluginsd_parser.c:1211`: `stream_path_set_from_json(child, json, from_parent=false)`. The parent stores `child->stream.path.array = [{child, hops=0}]`.
157
-4. After the hash changes (stream-path.c:422-429), the parent calls `stream_path_send_to_parent(host=child_host_on_parent)` and `stream_path_send_to_child(host=child_host_on_parent)`:
158
- - `stream_path_send_to_parent` is a **no-op on the apex parent** in this topology because the apex has no upstream — `child_host_on_parent->sender == NULL` (the local agent does not forward to a grandparent). On a proxy this would propagate the path up. **The apex case is what breaks convergence in this specific scenario.**
159
- - `stream_path_send_to_child` does the JSON-emit-time self-append and sends the child the enriched view including the parent. The child stores `[{child, hops=0}, {parent, hops=1}]`.
160
-5. After this initial round-trip, the parent's stored copy of the child's path is `[{child}]` — the parent's own host_id is not yet stored. Convergence on the parent depends on the child re-emitting later.
161
-6. The child re-emits via `stream_path_send_to_parent(localhost)` only on sparse triggers — retention boundary movement (`first_time_s` change), node_id update, or reconnect. **In a freshly-connected child (e.g., after the reporter removed cloud.d on the child), none of these have fired yet.** The parent's storage stays at the initial `[{child}]`.
162
-7. Phase 1 of `function_streaming_topology` (function-streaming.c:780-799) iterates `rrdhost_root_index` and, for each host, calls `streaming_topology_get_path_ids(host, from=1, ...)`:
163
- - For `host=parent` (localhost on apex): `parent->stream.path.array` is empty → returns 0.
164
- - For `host=child`: stored `[{child, hops=0}]` only → `rrdhost_stream_path_get_host_ids(child, 1, ...)` returns 0 (nothing at position 1+) → the localhost-append at function-streaming.c:243-244 is gated by `n > 0` so it does not fire → returns 0.
165
- - For each vnode: empty stored path → returns 0.
166
-8. `parent_child_count[parent's UUID]` is therefore **never** incremented.
167
-9. Actor classification (function-streaming.c:858-866): `cc = NULL` → `node_type = "child"` → wrong.
168
-
169
-So the bug is **not** "the parent never knows about itself" — the design eventually converges. The bug is that the topology function reads stored paths for classification, and those stored paths can lag behind the converged state for an arbitrary period (until the next retention/node_id event on the originating child). In a "fresh" topology (recent reconnect, recent cloud.d removal, etc.) the lag is observable. The function returning a wrong answer because the storage hasn't caught up yet is incorrect behavior — the function should report current truth, not last-converged-state.
170
-
171
-### Bug B — On a child's view, the child classifies itself as `parent`
172
-
173
-Symptom: `actors[0]` in the child's response: `actor_type:"parent"`, `attributes.child_count:1`. The localhost is classified as a parent of itself.
174
-
175
-Root cause (mapped to the intended logic above): step #3 (slot-1+ extraction) is **contaminated** at the helper. `streaming_topology_get_path_ids` blindly appends `localhost->host_id` to the result for **any** value of `from`, including `from=1` which is used by Phase 1 parent-counting. The append is correct semantics for `from=0` (full-path rendering — slot 0 plus slots 1+) but wrong for `from=1` (only slots 1+, by definition the host itself must not be there). The helper sneaks the localhost into the slot-1+ result, polluting the count.
176
-
177
-File: `src/web/api/functions/function-streaming.c:225-247`
178
-
179
-```c
180
-static uint16_t streaming_topology_get_path_ids(RRDHOST *host, uint16_t from, ND_UUID *host_ids, uint16_t max) {
181
- uint16_t n = rrdhost_stream_path_get_host_ids(host, from, host_ids, max);
182
- uint16_t filtered_n = 0;
183
-
184
- bool found_localhost = false;
185
- for(uint16_t i = 0; i < n; i++) {
186
- if(UUIDiszero(host_ids[i]))
187
- continue;
188
- host_ids[filtered_n++] = host_ids[i];
189
- if(UUIDeq(host_ids[i], localhost->host_id)) {
190
- found_localhost = true;
191
- }
192
- }
193
- n = filtered_n;
194
-
195
- // append localhost if not found (same as rrdhost_stream_path_to_json)
196
- if(!found_localhost && n < max && n > 0)
197
- host_ids[n++] = localhost->host_id;
198
-
199
- return n;
200
-}
201
-```
202
-
203
-Trace (verified):
204
-
205
-1. The child's stored path on itself is `[{child, hops=0}, {parent, hops=1}]` (set after the child receives the parent's stream-path back; verified via `actors[0].tables.streaming_path` in the live response).
206
-2. Phase 1: `streaming_topology_get_path_ids(child, from=1, ...)`:
207
- - `rrdhost_stream_path_get_host_ids(child, 1, ...)` returns `[parent's host_id]` (n=1).
208
- - found_localhost: parent != child (localhost) → false.
209
- - Append condition `!found_localhost && n > 0` → **append child** → returns `[parent, child]` (n=2).
210
-3. Loop at function-streaming.c:786-798 increments both `parent_child_count[parent] = 1` (correct) and `parent_child_count[child] = 1` (incorrect).
211
-4. Actor classification for the child (function-streaming.c:864-865): `cc = 1` → `node_type = "parent"` → wrong.
212
-5. `attributes.child_count` (function-streaming.c:923) reads the same value, so the modal shows `Children: 1` as in the screenshot.
213
-
214
-This bug is independent of Bug A. Even if Bug A is fixed at the protocol layer, Bug B will still misclassify any child agent on its own view.
215
-
216
-Every caller of `streaming_topology_get_path_ids` (verified by grep — there are **6**, not 5):
217
-
218
-- `function-streaming.c:785` — Phase 1 parent counting. `from=1`. **Wants the append off.**
219
-- `function-streaming.c:801` — Phase 1 descendants computation. `from=0`. Wants the append on.
220
-- `function-streaming.c:971` — Per-actor `streaming_path` field for FE highlight_path. `from=0`. Wants the append on.
221
-- `function-streaming.c:1045` — Observer-mode inbound table source resolution. `from=0`. Wants the append on. *(Missed in the iteration-1 SOW; identified by iter-1 reviewers; verified.)*
222
-- `function-streaming.c:1269` — Outbound table (non-observer parent). `from=0`. Wants the append on.
223
-- `function-streaming.c:1407` — Phase 4 link emission (looks at `link_ids[1]` — the direct parent). `from=0`, `max=2`. Wants the append on (this is how the child→parent link is drawn even on the parent despite Bug A).
224
-
225
-So one caller wants the append off, five want it on. The narrowest fix is to gate the append on `from == 0` inside `streaming_topology_get_path_ids` itself. No call site changes needed.
226
-
227
-### Bug C — On a child's view, the parent agent is not rendered as an actor
228
-
229
-Symptom: child's response has `actors_n: 1`, `links_n: 0`. The parent is referenced by `actors[0].streaming_path[1]` and known to the function via the child's `stream.path.array[1]`, but no actor is emitted for the parent.
230
-
231
-Root cause: actors are emitted exclusively from `rrdhost_root_index` (function-streaming.c:847-848 in Phase 3, again at 1372-1375 in Phase 4 for links). On a child agent, that index contains only localhost. Path entries that point to nodes outside the local index are never synthesized.
232
-
233
-This is what the reporter asked for in their second message — *"I also hoped to have it show its parents"*. The information is local on the child (in `STREAM_PATH` array entries which carry `hostname`, `host_id`, `node_id`, `claim_id`, `hops`, `since`, `first_time_t`, `start_time_ms`, `shutdown_time_ms`, `capabilities`, `flags`); it is just not turned into actor records.
234
-
235
-The cloud-frontend can synthesize endpoint nodes from links if a link references a missing actor (`cloud-frontend/src/domains/functions/topology/payload.js:395-404`), but it does not synthesize actors from the per-actor `streamingPath` array. Since the child view has zero links, the FE fallback does not rescue this case.
236
-
237
-### Bug D — `Since` column declared as `"number"`, FE renders raw integer
238
-
239
-Symptom: streaming-path table shows `1,777,992,161` instead of a date.
240
-
241
-Root cause: `function-streaming.c:355` declares the column as `"number"`. The path's `since` field is emitted in seconds (stream-path.c:88, `buffer_json_member_add_uint64(wb, "since", p->since)`).
242
-
243
-After iteration-2 review with the cloud-frontend source available: the FE accepts `"timestamp"` columns in **either seconds or milliseconds** (`cloud-frontend/src/domains/functions/components/topology/actorModal/dataTable.js:52-58`):
244
-
245
-```javascript
246
-case "timestamp": {
247
- const ms = Number(val)
248
- if (isNaN(ms) || ms === 0) return "-"
249
- const ts = ms > 1e12 ? ms : ms * 1000
250
- // ... renders as date
251
-}
252
-```
253
-
254
-So changing the column type to `"timestamp"` is sufficient — no data emission change needed. (Compare to `db_from`/`db_to` at function-streaming.c:374-375 which use `"timestamp"` and emit ms via `* MSEC_PER_SEC` — a stylistic convention but not required by the FE.)
255
-
256
-### Bug E — "no facets, no filters" on the child view (consequence)
257
-
258
-Symptom: screenshot shows the right pane with the Function picker tree but no facet pills.
259
-
260
-Root cause: with one actor and zero links, the FE has no field distributions to populate facets from. Fixing A+B+C should populate the response with the child (correctly classified), the parent (synthetic actor for the upstream parent), and the link between them, which gives the FE enough material for facets.
261
-
262
-Risks:
263
-
264
-- The fix is contained inside `function-streaming.c`. The streaming-path subsystem stays as-is — no protocol change, no wire-format change, no change to `stream-path.c`. The cycle terminator at stream-path.c:423 (`if(!from_parent) stream_path_send_to_parent(host);`) is load-bearing and must not be touched: it's what stops a child→parent→grandparent chain from re-sending forever after each downstream echo.
265
-- (Bug B fix) flipping the localhost-append off for `from > 0` is a behavior change in a small public-ish helper. The six call sites must be re-validated; in particular the link-drawing path at function-streaming.c:1407 must continue to work for stale/disconnected children. (Verified: with `from=0`, the gate stays on.)
266
-- (Bug C fix) synthesizing actors from path entries reuses the existing `actor_type:"parent"`. The synthesized actors use the same canonical id format (`streaming_topology_actor_id_from_guid` at function-streaming.c:107-115) so the FE highlight_path lookup matches across the per-actor `streaming_path` field references and the new actor records. Sub-specs the implementation must pin down: (a) de-duplication against `rrdhost_root_index` so a path entry whose host_id matches a real local host does not produce a duplicate actor; (b) which sub-tables are populated — the synthetic actor's `tables` block is built directly from the `STREAM_PATH` struct fields so the streaming-path table renders; the inbound/outbound/retention tabs that depend on data the agent doesn't have for a remote parent will be empty (acceptable for a single-agent view; aggregated views in Cloud will fill them from the parent's own response); (c) link-emission pass for the synthesized chain — emit a `link_type:"streaming"` for **every consecutive non-localhost path slot** (`slot[i] → slot[i+1]` for `i >= 1`), not only `slot[0] → slot[1]`. The existing Phase 4 loop at function-streaming.c:1372-1467 only emits one link per `rrdhost_root_index` host with `max=2` at line 1407; the new synthesis pass emits the multi-hop links from the path; (d) link timestamps — synthesized `discovered_at` and `last_seen` derived from the `STREAM_PATH.since` and `STREAM_PATH.first_time_t` fields of each path entry, so cross-agent merge of the same upstream link produces comparable values.
267
-- (Bug D fix) changing only the column type (Option 2) is a pure metadata change; it does not touch the data emission or the inter-agent wire format. Risk: very low.
268
-- Same-failure search: `rrdhost_stream_path_total_reboot_time_ms` (stream-path.c:145-158) shares Bug A's blind assumption (that localhost is in localhost's own stored path). On a top parent it returns 0 silently. Add to the scan list during validation. The `topology:snmp` Function (separate) may have similar column-type issues; check `since`/timestamp columns there as part of validation, even if no fix is needed.
269
-
270
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
271
-
272
-Status at implementation start: ready (user decisions recorded 2026-05-06: Decision 1 = Option 1, Decision 2 = Option 1, Decision 3 = Option 1 with no new actor type, Decision 4 = Option 2, hardening deferred to separate SOW; final closure evidence is in the Validation and Outcome sections).
273
-
274
-Problem / root-cause model:
275
-
276
-- See Bugs A–E in Analysis. Two distinct root causes (A: stale storage on parent; B: spurious localhost-append on `from > 0`) plus two missing-feature-style root causes (C: actors only iterate `rrdhost_root_index`; D: column type/unit). E is a downstream consequence of A+B+C.
277
-
278
-Evidence reviewed:
279
-
280
-- Code: `src/web/api/functions/function-streaming.c` (Phase 1: 759-829; Phase 3 actors: 845-1361; Phase 4 links: 1363-1505; helpers: 56-247, 313-360), `src/web/api/functions/functions.c:20-30`, `src/streaming/stream-path.c` (state machine: 79-96 emit, 98-143 self, 177-209 to_json, 219-247 send paths, 249-259 get_host_ids, 261-288 disconnect handlers, 368-435 set_from_json), `src/streaming/stream-sender.c:147-161`, `src/streaming/stream-sender-execute.c:109-110`, `src/plugins.d/pluginsd_parser.c:1211`.
281
-- Cloud-frontend (commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc`): `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` (timestamp auto-detection), `<dashboard-repo>/src/domains/functions/topology/payload.js:263-265, 350-383, 395-404` (actor-id dedup; link-derived synthesis fallback).
282
-- Live evidence: Cloud Function responses for the reporter's parent and child, captured 2026-05-05, stored under `<repo>/.local/audits/streaming-topology/` (gitignored). Sanitized facts above.
283
-- Reporter screenshot of the failing UI (viewed locally; not committed).
284
-- Independent read-only review batches completed in two iterations. Iteration outputs preserved under `<repo>/.local/audits/streaming-topology/` (gitignored).
285
-
286
-Affected contracts and surfaces:
287
-
288
-- The `topology:streaming` Function output (consumed by Netdata Cloud frontend topology renderer).
289
-- The presentation metadata under `presentation.actor_types.{parent,child,vnode,stale}` (function-streaming.c:561-615) — Decision 3 reuses the existing `parent` type for synthesized upstream actors; **no new actor type is introduced**.
290
-- Internal helper `streaming_topology_get_path_ids` semantics — Decision 2.
291
-- Streaming-path table column metadata at function-streaming.c:345-360 — Decision 4.
292
-- File layout: `src/web/api/functions/function-streaming.{c,h}` is split into `function-netdata-streaming.{c,h}` and `function-topology-streaming.{c,h}`; `function_streaming` is renamed to `function_netdata_streaming`. Plus a new maintenance reference at `src/streaming/STREAM_PATH.md`. See Plan items 0 and 6.
293
-- Decision 1 (localhost live-state classification) does NOT affect inter-agent streaming. The streaming protocol and `stream-path.c` are NOT touched by this SOW. (Defensive hardening for the streaming-path JSON parser — array length clamp + scalar range checks — is moved to a separate hardening SOW; out of scope here.)
294
-
295
-Existing patterns to reuse:
296
-
297
-- `streaming_topology_actor_id_from_guid` (function-streaming.c:107-115) for synthetic actor IDs.
298
-- `STREAM_PATH` struct fields (stream-path.h, stream-path.c:79-96) for synthetic-actor attributes.
299
-- The existing `parent` actor preset (function-streaming.c:563-575) is reused as-is for synthesized upstream actors. No new presentation block is needed.
300
-
301
-Risk and blast radius:
302
-
303
-- Decision 1 (localhost live-state classification) is contained in `function-streaming.c`. Implementation must populate **both** `parent_child_count[localhost]` AND `parent_descendants[localhost]` from the same root-index walk; otherwise the `received_nodes` block (function-streaming.c:982-1002) stays empty even when `child_count > 0`. **Zero changes to the streaming protocol or to `stream-path.c`.**
304
-- Decisions 2 and 3 are local to `function-streaming.c`.
305
-- Decision 4 Option 2 is pure metadata; trivial blast radius.
306
-
307
-Sensitive data handling plan:
308
-
309
-- The Cloud responses captured for evidence contain customer-identifying info (real hostnames, private IPs, agent UUIDs, claim IDs, machine GUIDs, host labels). Raw responses are stored under `<repo>/.local/audits/streaming-topology/` (gitignored).
310
-- This SOW has been sanitized: hostnames are `[parent-1]`, `[child-1]`, vnodes unnamed; private IPs `[CHILD_IP]`; UUIDs not written; reporter referred to as "the reporter"; the customer space referred to as the reporter's space; Slack identifiers not included.
311
-- Test fixtures derived from the Cloud responses must use sanitized hostnames/UUIDs (e.g., `parent-1`, `child-1`, `00000000-0000-0000-0000-000000000001`).
312
-- No tokens, bearers, or claim IDs appear in this SOW or in any planned test fixture.
313
-
314
-Implementation plan (deferred; depends on user decisions below):
315
-
316
-0. **File split (precondition).** Confirmed with project owner 2026-05-06 — both streaming functions move to their own files, with `function-streaming.{c,h}` deleted. Steps:
317
- - Create `src/web/api/functions/function-topology-streaming.{c,h}`. Move `function_streaming_topology` and every `streaming_topology_*` static helper, plus the `RRDFUNCTIONS_STREAMING_TOPOLOGY_HELP` macro, into the new file. New header exports `function_streaming_topology` and the help macro. The former `value_in_csv` helper is removed by Decision 5 because the topology function no longer applies backend graph filters.
318
- - Create `src/web/api/functions/function-netdata-streaming.{c,h}`. Move `function_streaming` (currently at function-streaming.c:1499) and the `RRDFUNCTIONS_STREAMING_HELP` macro into the new file. **Rename** the C symbol `function_streaming` → `function_netdata_streaming` to match the registered Function name (`netdata-streaming`) and the file name. Update the registration in `functions.c:8-18`.
319
- - Delete the old `src/web/api/functions/function-streaming.{c,h}` files.
320
- - Update the build manifest (CMake source list under `src/web/api/functions/` or its parent) to remove `function-streaming.c` and add the two new sources.
321
- - All four bug fixes below (steps 1–4) land in the new `function-topology-streaming.c` after the split commit. The split is a single mechanical commit with no behavior change.
322
-1. Bug B fix (Decision 2). Lowest risk; isolated to `streaming_topology_get_path_ids` (in the new `function-topology-streaming.c`). Ship after the split.
323
-2. Bug A fix (Decision 1). Risk depends on chosen option.
324
-3. Bug C fix (Decision 3). Adds synthetic upstream actors and corresponding link entries on the child view. Reuses existing `actor_type:"parent"` — no new actor type and no presentation block changes.
325
-4. Bug D fix (Decision 4). Smallest patch; metadata only.
326
-5. **(Removed.)** Defensive hardening for the streaming-path JSON parser (array length clamp + scalar range checks) is moved to a separate hardening SOW. This SOW does not touch `src/streaming/stream-path.c` at all.
327
-6. **Maintenance documentation.** Confirmed with project owner 2026-05-06 — write `src/streaming/STREAM_PATH.md` co-located with the streaming-path source. The file must **not be referenced in `docs/.map/map.yaml`** so the learn-ingestion pipeline ignores it. (Co-location under `src/streaming/` is not by itself sufficient — `src/streaming/README.md` IS in the map.) Covers:
328
- 1. What `stream.path.array` represents — chain of hops a host's data takes; per-host on every agent that knows about the host.
329
- 2. Slot semantics — slot 0 = origin; slots 1+ = upstream hops (parents/proxies).
330
- 3. Storage rule — `stream_path_set_from_json` stores received entries as-is; `rrdhost_stream_path_to_json` overlays/appends a fresh `rrdhost_stream_path_self()` at JSON-emit time only. Why the asymmetry exists (avoids storing volatile local fields like `first_time_t` and agent-event medians).
331
- 4. Propagation — bidirectional from-below (`from_parent=false`: up + down); down-only from-above (`from_parent=true`: down only). Multi-hop convergence works because each forwarded host on a proxy gets its own sender (`stream-receiver-connection.c:188`).
332
- 5. The cycle terminator — `if(!from_parent)` at `stream-path.c:423`. Why it must not be touched. What a future protocol redesign would need to replace it with first.
333
- 6. Update triggers — connect (`stream-sender.c:157`); retention boundary movement, specifically `first_time_s` change (`rrdcontext-worker.c:99-104`); node_id update (`sqlite_metadata.c:288`, `command-nodeid.c:163`); parent disconnect (`stream-sender.c:171`). Note: triggers are sparse — convergence has lag; consumers must not assume "stored = current truth".
334
- 7. Consumer guidance — for any feature that asks "is X a parent? what is X's chain?", read live state where possible; fall back to stored paths only as a hint. Lists known consumers: `function-topology-streaming.c` (post-split), `api_v2_contexts.c:425, 510`, `rrdhost_stream_path_total_reboot_time_ms` (`stream-path.c:145-158`).
335
- 8. vnode special case — vnodes don't stream; their stored path is empty; emit-time self-append fills slot 0 with the collecting agent.
336
- 9. Topology-function intended classification logic — short summary mirroring the SOW's "Intended classification logic" and "Intended actor set per agent" sections; cross-reference SOW-0012 in `.agents/sow/done/` for the bug history and fix design.
337
-7. Tests covering each bug at the function level (parent view, child view, mid-chain view). Real-use validation against the reporter's environment after each fix lands.
338
-8. Same-failure search across the codebase: any other column declared `"number"` that holds Unix epoch; any other consumer of `host->stream.path.array` that assumes self is stored — including `rrdhost_stream_path_total_reboot_time_ms` (`stream-path.c:145-158`) which shares the same blind assumption.
339
-
340
-Validation plan:
341
-
342
-- Unit-style: build a minimal test that constructs synthetic `STREAM_PATH` arrays for a 2-host parent/child scenario and asserts the classification, `child_count`, `actors[]`, `links[]`, and the streaming-path table types and values. The function emits text to a `BUFFER`, so the test runs the emission path and parses the JSON.
343
-- Integration-style: re-run the captured Cloud queries after the fix is deployed to a build of the parent and the child, and re-fetch the screenshot UI on Cloud.
344
-- Same-failure scan: `grep` for columns of type `"number"` that hold timestamps elsewhere in `function-streaming.c` and other Function emitters; check `rrdhost_stream_path_total_reboot_time_ms`.
345
-- Decision 1 specific: verify that `received_nodes` populates correctly on the parent view after the fix (i.e., that `parent_descendants[localhost]` was also populated by the live-state walk).
346
-
347
-Artifact impact plan:
348
-
349
-- `AGENTS.md`: not affected. No workflow change.
350
-- Runtime project skills: not affected directly.
351
-- Specs (`.agents/sow/specs/`): no spec change. The streaming protocol is untouched. The topology function's classification semantics are local to `function-streaming.c` and need no spec.
352
-- End-user/operator docs: Netdata Cloud frontend public docs do not document the streaming-topology Function's response shape at the field level; no docs update needed.
353
-- End-user/operator skills: `query-netdata-cloud` skill should add a how-to for "Calling `topology:streaming` and interpreting actor types" — mandated by the skill's own live-catalog rule.
354
-- SOW lifecycle: this SOW is opened in `pending/` and will move to `current/` only after Decisions 1–4 are answered.
355
-
356
-Open-source reference evidence:
357
-
358
-- No external reference required for these bugs — they are entirely internal to Netdata's streaming/topology code path.
359
-
360
-Open decisions:
361
-
362
-See "Implications And Decisions" below. Implementation cannot start until each is answered.
363
-
364
-## Implications And Decisions
365
-
366
-### Decision 1 — Fix scope for Bug A (parent classification)
367
-
368
-Background: the streaming-path subsystem is designed for full multi-hop convergence and **must not be touched**. Each forwarded host has its own sender on a proxy, and `stream_path_send_to_parent` propagates child paths up the chain through proxies. The cycle terminator at stream-path.c:423 is load-bearing. The bug is that on the **apex parent** in any topology, convergence depends on the child re-emitting after a sparse trigger (retention boundary movement, node_id update, reconnect). Until that fires, the apex's storage of the child's path is stuck at the initial `[{child}]` from connect-time, so Phase 1 sees nothing. The topology function reads stored paths and therefore returns a wrong answer until convergence completes.
369
-
370
-The fix is **entirely inside the topology function** (`function-streaming.c`). The streaming-path subsystem stays as-is.
371
-
372
-Options:
373
-
374
-1. **Localhost-only live-state fix.** For `host == localhost` only, replace Phase 1's path-based counting with a direct check on the local agent state via `rrdhost_status()` (the same helper the function already uses at lines 850, 1021, 1113, 1158, 1220, 1289 — do **not** access `host->receiver` directly). A host counts as an active streaming child iff `rrdhost_status(host, ...).ingest.type == RRDHOST_INGEST_TYPE_CHILD` and `s.ingest.status` is in `{RRDHOST_INGEST_STATUS_ONLINE, RRDHOST_INGEST_STATUS_REPLICATING}`. `child_count` becomes the count of such hosts. The implementation must populate **both** `parent_child_count[localhost]` AND `parent_descendants[localhost]` from the same root-index walk, so `received_nodes` (function-streaming.c:982-1002) stays consistent with the new count. The existing slot-0+ append at function-streaming.c:815-822 must skip localhost in this fix's path so we do not double-write `parent_descendants[localhost]`. For non-localhost hosts, keep the existing path-based logic unchanged.
375
- - Pros: contained in `function-streaming.c`. Zero streaming-protocol change. Trivially correct on the localhost side regardless of stored-path lag.
376
- - Cons: multi-hop / non-localhost parent classification (e.g., the topology view from a child showing whether its parent is itself a parent of further children) still depends on the stored streaming_path of children. Heals on the next sparse trigger.
377
- - Risk: low.
378
-
379
-2. **All-host live-state fix.** Same as Option 1 but extend the live-state check to non-localhost hosts too: walk `rrdhost_root_index` once and compute parent classification for every host based on observable state (receivers attached, sender attached, virtual flag, archive status), not stored paths.
380
- - Pros: classification is correct immediately for every host on every view, regardless of stored-path lag.
381
- - Cons: more code, more cases to enumerate. Non-localhost hosts cannot be checked via `rrdhost_status()` for receiver-presence in the same way as localhost — that information is only locally observable for hosts whose children are also locally registered (the proxy/apex case). For pure remote children that are not themselves proxies, there is nothing more to count anyway, so the additional code mostly degenerates back to the Option 1 case.
382
- - Risk: low-medium (more surface to test).
383
-
384
-**Recommendation: Option 1.** Smallest change that fixes the user-visible bug. Non-localhost classification heals on the next sparse trigger — acceptable degradation for this SOW. If, after this lands, the residual surfaces in real use, a follow-up SOW can extend the live-state check to all hosts (still no protocol change).
385
-
386
-**Residual note (line 1197):** Option 1 fixes the classification but leaves a related symptom — the observer-mode outbound table at `function-streaming.c:1196-1208` calls `rrdhost_stream_path_get_host_ids(host, 0, ...)` directly to find the streaming destination. On the apex parent the stored path for a child is `[{child}]` only at connect-time, so this lookup finds no destination and `dst_hostname` falls back to the peer IP at function-streaming.c:1209. The IP fallback already produces a usable label; this heals on the next sparse trigger like the classification does. **No protocol change; if needed later, fix in `function-streaming.c` by reading live state for the destination too.**
387
-
388
-### Decision 2 — Fix for Bug B (localhost-append for `from > 0`)
389
-
390
-Background: `streaming_topology_get_path_ids` blindly appends `localhost->host_id` for any value of `from`. Of the six call sites (verified by grep), only one (`from=1`, Phase 1 parent counting) wants the append off.
391
-
392
-Options:
393
-
394
-1. **Gate the append on `from == 0`.** Smallest change: at function-streaming.c:243-244, change the condition from `if(!found_localhost && n < max && n > 0)` to `if(from == 0 && !found_localhost && n < max && n > 0)`.
395
- - Pros: minimal diff; existing five "want append on" call sites unchanged.
396
- - Cons: ties the append semantic to a magic value (`from == 0` happens to mean "full path"). Caller still has to know.
397
- - Risk: very low.
398
-
399
-2. **Add an explicit boolean parameter.** `streaming_topology_get_path_ids(host, from, append_localhost, host_ids, max)`. Update all six call sites.
400
- - Pros: explicit semantics; future-proof.
401
- - Cons: more invasive diff; six call sites to touch.
402
- - Risk: low.
403
-
404
-3. **Split into two helpers.** `streaming_topology_get_full_path_ids` (with append) and `streaming_topology_get_upstream_path_ids` (without). Make the intent obvious from the function name.
405
- - Pros: most readable.
406
- - Cons: most invasive diff.
407
- - Risk: low.
408
-
409
-**Recommendation: Option 1.** Smallest, safest change. Pair with a comment at the helper explaining why the gate exists. If more flexibility is needed later, refactor in a follow-up.
410
-
411
-### Decision 3 — Fix scope for Bug C (rendering parents on child view)
412
-
413
-Background: actors are emitted only for hosts in `rrdhost_root_index`. The child has the parent's metadata in its `STREAM_PATH` array but no rrdhost record for it.
414
-
415
-Options:
416
-
417
-1. **Synthesize `parent` actors from `STREAM_PATH` entries that point outside `rrdhost_root_index`.** For each path entry whose host_id is not localhost and not already in `rrdhost_root_index`, emit an actor with the existing `actor_type:"parent"`. Use STREAM_PATH fields for attributes (hostname, host_id, node_id, claim_id, since, flags, capabilities, hops, first_time_t, start_time_ms, shutdown_time_ms). Add multi-hop links between every consecutive non-localhost path slot in a separate pass after the existing Phase 4 link loop.
418
- - Sub-specs the implementation must define before coding:
419
- - (a) De-duplication: skip emission if the path entry's host_id matches any host already iterated from `rrdhost_root_index`. (FE drops duplicate backend actor IDs at `<dashboard-repo>/src/domains/functions/topology/payload.js:263-265` — relying on FE dedup is not the agent's contract.)
420
- - (b) Sub-tables: synthetic actors populate only the `streaming_path` table from the STREAM_PATH struct fields. The `inbound`, `outbound`, `retention` tabs that depend on data the agent doesn't have for a remote upstream will render empty — acceptable for a single-agent view; aggregated views in Cloud will fill them from the parent's own response (whose `agent_id` matches that parent's actor_id, making it the authoritative source for that actor — see "Topology must be mergeable across agents" section).
421
- - (c) Link emission: a second loop after the Phase 4 link loop. For each synthesized chain, emit `link_type:"streaming"` for **every consecutive non-localhost path slot pair** (`slot[i] → slot[i+1]` for `i >= 1`), not only `slot[0] → slot[1]`. This means deep chains (e.g., `[child, parent_1, parent_2, parent_3]`) get all transitive links emitted: `parent_1 → parent_2`, `parent_2 → parent_3`. The existing Phase 4 already emits `child → parent_1` (`function-streaming.c:1407` with `max=2`).
422
- - (d) Link timestamps: synthesized links derive `discovered_at` and `last_seen` from the STREAM_PATH `since` and `first_time_t` fields of each path entry, NOT from `now`. This preserves cross-agent merge semantics.
423
- - FE coupling: with `actor_type:"parent"` reused, no change to `presentation.actor_types`, no change to `presentation.legend.actors`, no FE update required.
424
- - Pros: solves the reporter's third complaint. Uses data already local. No new actor type, no FE coupling, no presentation block additions.
425
- - Cons: synthesized parent's modal will show empty inbound/outbound/retention tabs in single-agent view. Aggregated Cloud views fill them.
426
- - Risk: low-medium. Touches actor emission loop and adds a small link-emission loop. Reuses existing infrastructure throughout.
427
-
428
-2. **Defer.** Track in a follow-up SOW.
429
- - Pros: smaller PR, easier to review.
430
- - Cons: child view stays useless until the follow-up lands. Two of the reporter's three complaints (#1 the misclassification, #3 the parents-not-shown) are about the child view; #1 is fixed by Bug B alone but #3 is not addressed.
431
-
432
-**Recommendation: Option 1.** The data is already local; the reporter asked explicitly; reusing the existing `parent` actor type avoids any FE coupling.
433
-
434
-### Decision 4 — Fix for Bug D (Since column type)
435
-
436
-Background: `since` is emitted as `uint64` Unix-epoch seconds at stream-path.c:88. Column type at function-streaming.c:355 is `"number"`. Frontend renders the raw integer (with locale-formatted separators).
437
-
438
-Iteration-2 review with cloud-frontend access confirmed:
439
-
440
-- The FE's `"timestamp"` case at `<dashboard-repo>/src/domains/functions/components/topology/actorModal/dataTable.js:52-58` @ commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` auto-detects ms vs seconds: `const ts = ms > 1e12 ? ms : ms * 1000`. A value of `1777992161` (< 1e12) is multiplied by 1000 → correct date. A value of `1777992161000` (> 1e12) is used directly → also correct.
441
-
442
-Options:
443
-
444
-1. **Change column type AND post-process to ms inside `function-streaming.c` only.** Introduce a local helper that emits the path with `since * MSEC_PER_SEC`. Matches the in-tree style used for `db_from`/`db_to`.
445
- - Pros: stylistic consistency with the rest of `function-streaming.c`.
446
- - Cons: requires a custom emit path duplicating `rrdhost_stream_path_to_json`.
447
- - Risk: low.
448
-
449
-2. **Change column type only.** Set type to `"timestamp"` at function-streaming.c:355; leave the data emission path unchanged (raw seconds). The FE auto-detects and converts.
450
- - Pros: minimal diff (one string change). No new helper.
451
- - Cons: depends on a FE auto-detection heuristic. If the FE ever tightens to require ms, this silently regresses. The threshold `1e12` (year 33658 in seconds, year 2001 in ms) is comfortably distant from realistic timestamps either way.
452
- - Risk: very low.
453
-
454
-3. **Use a different column type** (e.g., `"datetime"` or `"duration_ago"`).
455
- - Pros: matches FE's intended semantic if `"timestamp"` is not the right type.
456
- - Cons: requires reading FE topology-table renderer to confirm the supported types — the topology-modal renderer's switch shows only `"timestamp"`, `"duration"`, `"badge"`, `"actor_link"`, plus a default. So this is not strictly necessary.
457
-
458
-**Recommendation: Option 2.** Switched from Option 1 after iteration-2 FE review. The FE's auto-detection means there is no unit to align — the simplest fix is correct. The in-tree style argument for Option 1 is real but not load-bearing.
459
-
460
-### Decision 5 — Backend graph filters
461
-
462
-Background: the topology function currently advertises and applies optional backend graph-pruning filters (`node_type`, `ingest_status`, `stream_status`). The UI does not expose these filters, and graph pruning in the backend makes the topology response incomplete: actors can disappear, links can be suppressed, and synthetic parent emission has to reason about filtered-out local hosts. This conflicts with the product requirement that topology should expose the full graph.
463
-
464
-Options:
465
-
466
-1. **Remove backend graph filters.** Keep only `info` as an accepted non-pruning parameter. Always emit all actors and links known to the local agent plus synthetic upstream parents derived from `STREAM_PATH` when no local `RRDHOST` exists. UI facets may still filter or highlight client-side over the full response.
467
- - Pros: complete topology response; simpler actor/link flow; no filtered-out actor edge cases; matches the current UI behavior.
468
- - Cons: callers cannot request a smaller backend response by status/type.
469
- - Risk: low. These filters are not visible in the UI and are optional today.
470
-
471
-2. **Keep backend filters.** Preserve current accepted params and filtering branches.
472
- - Pros: existing API shape unchanged for any hidden caller.
473
- - Cons: incomplete graph responses; complex synth/dedup semantics; contradicts the project-owner requirement that all nodes must be exposed.
474
- - Risk: medium. Keeps the current class of graph correctness traps.
475
-
476
-**Decision: Option 1.** Remove `node_type`, `ingest_status`, and `stream_status` from `accepted_params` and delete backend actor/link/synthetic filter checks. `info` remains because it controls response metadata shape, not graph pruning.
477
-
478
-## Plan
479
-
480
-User decisions recorded 2026-05-06: Decision 1 = Option 1 (localhost-only live-state); Decision 2 = Option 1 (gate append on `from==0`); Decision 3 = Option 1 (synthesize `parent` actors, no new actor type); Decision 4 = Option 2 (column type only, FE auto-detects); Decision 5 = Option 1 (remove backend graph filters; keep only `info`). Defensive hardening for `stream-path.c` deferred to a separate SOW.
481
-
482
-PR commit order: split → B → A → D → C → doc → tests → validation.
483
-
484
-0. **File split** (precondition; mechanical, no behavior change). Create `function-topology-streaming.{c,h}` and `function-netdata-streaming.{c,h}`; move `function_streaming_topology` + helpers and `function_streaming` (renamed `function_netdata_streaming`) respectively; delete `function-streaming.{c,h}`; update `functions.c` includes/registrations and the build manifest. `STREAMING_FUNCTION_UPDATE_EVERY` is duplicated in each new file (per project owner's call — no shared header). All subsequent line-number references in this SOW are pre-split; the bug-fix commits land in the new `function-topology-streaming.c` and may be at different line numbers.
485
-1. **Bug B fix** (Decision 2 = Option 1): one-line change at the equivalent of `function-streaming.c:243-244` (now in `function-topology-streaming.c`) plus comment. Add unit-style test asserting Phase 1 parent-counting on a child agent yields `parent_child_count[localhost] == 0`.
486
-2. **Bug A fix** (Decision 1 = Option 1, localhost live-state): rewrite Phase 1's localhost write into both `parent_child_count` and `parent_descendants` from a `rrdhost_root_index` walk that uses `rrdhost_status()` (the abstraction the function already uses) and counts hosts where `s.ingest.type == RRDHOST_INGEST_TYPE_CHILD` and `s.ingest.status ∈ {ONLINE, REPLICATING}`. **No `host->receiver` access.** Skip the slot-0+ append for localhost in `parent_descendants` to avoid double-write. No protocol change.
487
-3. **Bug C fix** (Decision 3 = Option 1, no new actor type): add synthetic-actor emission loop after the existing `rrdhost_root_index` loop in Phase 3 (dedup against local `rrdhost_root_index` actors, streaming_path table built directly from STREAM_PATH struct fields). Synthesized actors use `actor_type:"parent"` — existing type, no presentation block changes. Add a second link-emission loop after the existing Phase 4 loop that emits `link_type:"streaming"` for every consecutive non-localhost path slot pair (`slot[i] → slot[i+1]` for `i >= 1`); link timestamps derived from STREAM_PATH `since`/`first_time_t`, not `now`.
488
-4. **Bug D fix** (Decision 4 = Option 2): change the `"since"` column type from `"number"` to `"timestamp"`. No data change.
489
-5. **Backend filter removal** (Decision 5 = Option 1): remove `node_type`, `ingest_status`, and `stream_status` parsing, accepted params, and actor/link/synthetic emission checks. Keep `info`.
490
-6. **Maintenance documentation**: write `src/streaming/STREAM_PATH.md` per the outline in the implementation plan above. Verify the file is **not referenced in `docs/.map/map.yaml`** (co-location under `src/streaming/` is not by itself sufficient).
491
-7. **Same-failure scan**: grep for other `"number"`-typed columns holding Unix-epoch values; grep for other consumers of `host->stream.path.array` that may share Bug A's assumption (specifically `rrdhost_stream_path_total_reboot_time_ms` at `stream-path.c:145-158`).
492
-8. **Tests**: see Validation Plan; minimum coverage = one test per bug, plus a test for the line 1197 outbound-table behavior (verify peer-IP fallback works for stale-storage cases).
493
-9. **Validation**: build & deploy to the reporter's parent and child; re-fetch Cloud responses; verify all acceptance criteria.
494
-
495
-## Execution Log
496
-
497
-### 2026-05-05
498
-
499
-- Read full Slack thread reporting the issue (11 messages including the screenshot the reporter shared).
500
-- Loaded `query-netdata-cloud` skill helpers from `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
501
-- Resolved the reporter's space, room, and node UUIDs.
502
-- Captured live Cloud Function responses for `topology:streaming` on both parent and child; saved under `<repo>/.local/audits/streaming-topology/`.
503
-- Confirmed Bugs A, B, C, D, E with file:line evidence.
504
-- Drafted SOW iteration-1.
505
-- Ran an independent read-only review batch for iteration-1; outputs preserved under `<repo>/.local/audits/streaming-topology/`.
506
-- Ran the same independent review scope for iteration-2 with the cloud-frontend source path included; outputs preserved under `<repo>/.local/audits/streaming-topology/`.
507
-- Iteration-2 corrections applied to this SOW: caller count 5 → 6 (added line 1045); Bug A trace step 4 reworded (the call exists but is a no-op due to `host->sender == NULL` on the parent side); Decision 4 recommendation switched to Option 2 (FE auto-detects seconds via `ms > 1e12 ? ms : ms * 1000`); Decision 1 sub-spec added (must populate both `parent_child_count[localhost]` and `parent_descendants[localhost]`); Decision 3 sub-specs added (filter, dedup, sub-tables, second link pass); same-failure scan target added (`rrdhost_stream_path_total_reboot_time_ms`); line 1197 residual noted; uint16_t truncation hardening item added; SOW sanitized (hostnames, UUIDs, IPs, names, customer-space identifiers replaced with placeholders).
508
-- 2026-05-06 — confirmed with project owner that the streaming-path subsystem must not be touched by this SOW. Decision 1 collapsed from three options (protocol fix / function-only / hybrid) to two options that are both contained in `function-streaming.c`: (1) localhost-only live-state classification — recommended; (2) all-host live-state classification — deferred to a follow-up if the residual surfaces. Removed all references to a protocol fix follow-up. The cycle terminator at stream-path.c:423 stays load-bearing and untouched.
509
-- 2026-05-06 — added the project owner's intended classification logic (six rules) and intended actor set (per-agent count formula) to the Analysis section. Added the cross-agent mergeability requirements section (stable actor IDs, stable link tuples, timestamps for tie-breaking, no overreach). Synthesized upstream `parent` actors must preserve the canonical actor-id format and derive link timestamps from the `STREAM_PATH` struct so the merge layer can reconcile views from multiple parents.
510
-- 2026-05-06 — added scope: split `function-streaming.{c,h}` into `function-netdata-streaming.{c,h}` and `function-topology-streaming.{c,h}`, with `function_streaming` renamed to `function_netdata_streaming`. The bug fixes land in the new topology file. Added scope: write `src/streaming/STREAM_PATH.md` as a co-located maintenance reference (not picked up by learn ingestion). Both additions confirmed with project owner.
511
-- 2026-05-06 — iteration-3 independent read-only review batch completed; outputs preserved under `<repo>/.local/audits/streaming-topology/`. Project owner's corrections applied: (1) **dropped `remote_parent` actor type** — synthesized upstream actors reuse the existing `actor_type:"parent"`, no new type, no presentation block changes, no FE coupling; (2) Decision 3 sub-spec adds multi-hop link emission for every consecutive non-localhost path slot, not only `slot[0] → slot[1]`; (3) Decision 3 sub-spec mandates synthesized link timestamps derive from STREAM_PATH `since`/`first_time_t`, not `now`; (4) Decision 1 wording corrected — use `rrdhost_status()` (`s.ingest.type == CHILD` and `s.ingest.status ∈ {ONLINE, REPLICATING}`), no `host->receiver` access; (5) Decision 1 sub-spec — skip slot-0+ append for localhost to avoid double-write to `parent_descendants[localhost]`; (6) **deferred all `stream-path.c` defensive hardening** (uint16_t array clamp + scalar range checks) to a separate hardening SOW — this SOW now touches zero shared subsystems beyond the topology function and a new doc; (7) replaced workstation paths with repo-relative paths (`<repo>/...`) and dashboard-repo placeholder (`<dashboard-repo>/...`); (8) corrected STREAM_PATH.md framing — the protection from learn ingestion is "not referenced in `docs/.map/map.yaml`", not "not under `docs/`"; (9) pinned cloud-frontend evidence to commit `8d0258eb60aa32e3ee5fdd2144ef10b44f7995bc` next to the `dataTable.js:52-58` citation. STREAMING_FUNCTION_UPDATE_EVERY shared macro: project owner's call — duplicate per file, no shared header. Aggregator self-marker proposal dropped — existing `data.agent_id` is sufficient for cross-agent merge.
512
-- 2026-05-06 — project-owner decision: remove backend topology graph filters because the UI does not expose them and the backend must return the full graph. Recorded as Decision 5. Implementation updated `function-topology-streaming.c` so `accepted_params` only advertises `info`; removed `node_type`, `ingest_status`, and `stream_status` parsing and all actor/link/synthetic filter checks.
513
-- 2026-05-06 — revisited topology flow after removing filters. Added a separate `local_actor_ids` set for all `RRDHOST`-backed actors, keeping `emitted_actors` as only the actors actually written to JSON. Synthetic upstream parents are now skipped when a local `RRDHOST` actor exists, not because a filtered actor happened to be present or absent. `parent_descendants[localhost]` is now written by one live-state pass: vnodes as `virtual`, active/repl children as `streaming`, and disconnected/history-only local hosts as `stale`; the path walk no longer has a competing localhost stale branch. Phase 4 now emits vnode links as `link_type:"virtual"` to localhost regardless of stored path length, registers each emitted link in `emitted_links`, and the synthetic link pass relies on that shared dedup instead of the previous `phase4_emitted` heuristic.
514
-- 2026-05-06 — validation so far: focused compiler syntax check of `src/web/api/functions/function-topology-streaming.c` passed using the existing `functions.c` compile flags from `compile_commands.json`. Full `cmake --build build-clion --target netdata -j4` did not reach code compilation because CMake reconfigured and failed while fetching the pre-existing Sentry/crashpad dependency (`mini_chromium` HTTP 400 / expected acknowledgments). This is an environment/dependency fetch failure, not a compiler error from the topology file.
515
-- 2026-05-06 — after project owner installed the build locally, queried the installed local agent through bearer-protected direct API without printing tokens or durable identifiers. Sanitized runtime summary: `status=200`, `accepted_params=["info"]`, `actors=22`, `links=22`, actor type counts `child=13`, `parent=2`, `stale=1`, `vnode=6`, link type counts `streaming=15`, `virtual=6`, `stale=1`, duplicate actor ids `0`, duplicate link tuples `0`, vnode stale links `0`.
516
-- 2026-05-06 — project owner decision: do not add tests before first PR publication. Open a draft PR now so online reviewers can inspect the code. Tests remain a known validation gap and this SOW remains in progress until follow-up validation is complete or the project owner explicitly accepts closure without tests.
517
-- 2026-05-07 — project owner confirmed the PR with fixes was merged as commit `cc50307bc6ca180285a0e0cda16d73ab7a42cd86` (`Fix streaming topology graph output (#22432)`) and requested moving this SOW to `done/`.
518
-
519
-## Validation
520
-
521
-Completed for SOW close based on merged PR evidence and project-owner acceptance.
522
-
523
-- Focused syntax validation passed for `src/web/api/functions/function-topology-streaming.c` using the existing `functions.c` compile command from `compile_commands.json`, replacing the source path and using `-fsyntax-only`.
524
-- Full local build attempted with `cmake --build build-clion --target netdata -j4`. It failed during CMake reconfigure while fetching the pre-existing Sentry/crashpad dependency (`external/crashpad/third_party/mini_chromium/mini_chromium`, HTTP 400 / expected acknowledgments) before compiling Netdata sources.
525
-- Same-failure search for removed backend filter identifiers in `function-topology-streaming.c` found no remaining `value_in_csv`, `filter_node_type`, `filter_ingest_status`, `filter_stream_status`, `node_type:`, `ingest_status:`, or `stream_status:` references.
526
-- Installed-agent runtime validation via direct local Function call: `status=200`; `accepted_params` contains only `info`; graph contains 22 actors and 22 links; actor/link duplicate checks are zero; vnode stale-link check is zero.
527
-- Merge evidence: `git show --stat --oneline cc50307bc6ca180285a0e0cda16d73ab7a42cd86 --` shows commit `cc50307bc Fix streaming topology graph output (#22432)` with the expected function split, topology function changes, `src/streaming/STREAM_PATH.md`, build manifest updates, and this SOW artifact.
528
-- Tests: deferred by project-owner decision for the first draft PR and accepted at merge time. This remains a residual test gap but no longer blocks this SOW.
529
-
530
-## Outcome
531
-
532
-Completed. PR `#22432` merged as commit `cc50307bc6ca180285a0e0cda16d73ab7a42cd86`, delivering the streaming topology graph fixes, function split, backend filter removal, timestamp column type correction, synthesized upstream parent rendering, and `STREAM_PATH` maintenance documentation.
533
-
534
-## Lessons Extracted
535
-
536
-- The streaming topology view must treat stored stream paths as hints, not as complete live topology truth for localhost classification.
537
-- Backend graph-pruning filters can make topology responses internally inconsistent when the frontend expects to facet over a complete graph.
538
-- For SOWs that intentionally publish a draft before tests, the residual test gap must be explicit and requires project-owner acceptance before close.
539
-
540
-## Followup
541
-
542
-- Conditional residual: if the multi-hop / non-localhost classification residual surfaces in real use after this SOW lands, open a follow-up SOW that extends the live-state classification (Decision 1 Option 1) to all hosts (Option 2 in this SOW). No protocol change is contemplated; both this SOW and any follow-up are entirely inside the topology function.
543
-- Query skill how-to: not required for the merged code path and not blocking this SOW close by project-owner acceptance. Track under future `query-netdata-cloud` skill maintenance if the workflow recurs.
544
-- Streaming-path defensive hardening: explicitly out of scope for this SOW, pre-existing, and not introduced by the merged fixes. Track as a future hardening SOW only if prioritized.
545
-
546
-## Regression Log
547
-
548
-None yet.
.agents/sow/done/SOW-0012-20260506-snmp-profile-projection.md
deleted
-649
@@ -1,649 +0,0 @@
1
-# SOW-0012 - SNMP profile projection and topology row schema
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: Implementation, focused validation, final review, and commit prep completed.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Replace the hacky SNMP metrics-vs-topology profile split with a clean, schema-driven profile catalog, resolver, and projection model. Regular SNMP compatibility matters. SNMP topology is WIP/nightly-only, so topology behavior can change to reach a clean end state.
14
-
15
-### User Request
16
-
17
-User request summary:
18
-
19
-- Recently added SNMP topology works but was added in a hacky way.
20
-- Focus on how profiles are loaded for regular metrics versus topology.
21
-- Review `src/go/plugin/go.d/collector/snmp/ddsnmp` and `src/go/plugin/go.d/collector/snmp_topology`.
22
-- Prefer clean end state over low churn.
23
-- Topology backward compatibility is not a constraint because it is WIP/nightly-only.
24
-- Use independent AI reviews to avoid missing issues.
25
-
26
-Detailed design source:
27
-
28
-- `src/go/plugin/go.d/TODO-snmp-profile-loading-topology.md`
29
-
30
-### Assistant Understanding
31
-
32
-Facts:
33
-
34
-- Regular SNMP and SNMP topology currently use the same physical profile loader in `collector/snmp/ddsnmp/load.go`.
35
-- `collector/snmp/ddsnmp/profile.go:25-71` selects profiles by `sysObjectID`/`sysDescr`, or by `manual_profiles` only when `sysObjectID` is empty.
36
-- Regular SNMP calls `ddsnmp.FindProfiles()` and then strips topology data via `collector/snmp/topology_profile_filter.go`.
37
-- SNMP topology calls `ddsnmp.FindProfiles()` and then keeps topology data via `collector/snmp_topology/profile_filter.go`.
38
-- Topology classification is hardcoded in `collector/snmp/ddsnmp/topology_classify.go`.
39
-- Topology ingestion dispatch is hardcoded by metric name in `collector/snmp_topology/topology_cache_metric_dispatch.go`.
40
-- VLAN-context topology bypasses the resolver and hardcodes topology profile filenames in `collector/snmp_topology/topology_vlan_context_collect.go`.
41
-- Current `_topology_*` rows are hidden through the generic underscore-prefix `HiddenMetrics` path in `collector/snmp/ddsnmp/ddsnmpcollector/collector.go:122-123`.
42
-- `HiddenMetrics` is not topology-owned. It is a general delivery container and must not be deleted or redefined without auditing non-topology users.
43
-
44
-Inferences:
45
-
46
-- The root issue is not physical profile loading. The root issue is the lack of an explicit consumer/topology contract in the profile schema and resolver output.
47
-- A single catalog plus explicit projection preserves the strong shared-profile model while removing name/prefix heuristics.
48
-- A top-level `topology:` list is cleaner than embedding topology rows in `metrics[]` because topology rows are not regular chart metrics and should not share metric merge/dedup ambiguity.
49
-
50
-Unknowns:
51
-
52
-- No product/design unknowns remain for this SOW. User decisions 1.D, 2.B, 3.C, 4.A, 5.B, 6.A, 7.A, and 8.A are recorded in the TODO and summarized below.
53
-- Some implementation details will be discovered while refactoring tests and moving per-profile mutations to load-time plus matched-set deduplication to resolve-time, but they are bounded by the gate and validation plan.
54
-
55
-### Acceptance Criteria
56
-
57
-- Regular SNMP metrics collection uses the new catalog/resolver/projection path and remains behaviorally equivalent to the current `selectCollectionProfiles(FindProfiles(...))` path.
58
-- SNMP topology uses top-level `topology:` rows, `TopologyKind`, and `ProfileMetrics.TopologyMetrics`; it no longer depends on `_topology_*` metric names or underscore-prefix `HiddenMetrics`.
59
-- VLAN-context topology uses `Project(ConsumerTopology).FilterByKind(vlanScopableKinds)` instead of hardcoded `LoadProfileByName()` calls.
60
-- `HiddenMetrics` remains available as a generic non-topology underscore-prefixed metric delivery container, and the existing preservation test continues to pass.
61
-- Old topology classifier/filter code and dead hardcoded profile-name constants are removed.
62
-- Profile-format documentation, project SNMP profile authoring skill, and a new SOW spec describe the shipped contract.
63
-
64
-## Analysis
65
-
66
-Sources checked:
67
-
68
-- `src/go/plugin/go.d/TODO-snmp-profile-loading-topology.md`
69
-- `collector/snmp/ddsnmp/load.go`
70
-- `collector/snmp/ddsnmp/profile.go`
71
-- `collector/snmp/ddsnmp/topology_classify.go`
72
-- `collector/snmp/ddsnmp/ddprofiledefinition/profile_definition.go`
73
-- `collector/snmp/ddsnmp/ddprofiledefinition/metrics.go`
74
-- `collector/snmp/ddsnmp/ddprofiledefinition/validation.go`
75
-- `collector/snmp/ddsnmp/ddsnmpcollector/collector.go`
76
-- `collector/snmp/ddsnmp/ddsnmpcollector/collector_scalar.go`
77
-- `collector/snmp/ddsnmp/ddsnmpcollector/collector_table.go`
78
-- `collector/snmp/ddsnmp/ddsnmpcollector/metric_builder.go`
79
-- `collector/snmp/profile_sets.go`
80
-- `collector/snmp/topology_profile_filter.go`
81
-- `collector/snmp_topology/collector.go`
82
-- `collector/snmp_topology/profile_filter.go`
83
-- `collector/snmp_topology/topology_cache_metric_dispatch.go`
84
-- `collector/snmp_topology/topology_vlan_context_collect.go`
85
-- `collector/snmp_topology/topology_cache_ingest.go`
86
-- `config/go.d/snmp.profiles/default/_system-base.yaml`
87
-- `config/go.d/snmp.profiles/default/_std-topology-lldp-mib.yaml`
88
-- `config/go.d/snmp.profiles/default/_std-cdp-mib.yaml`
89
-- `collector/snmp/profile-format.md`
90
-- `.agents/skills/project-snmp-profiles-authoring/SKILL.md`
91
-- `.agents/sow/pending/SOW-0002-20260501-unified-multi-layered-topology-schema.md`
92
-- `.agents/sow/specs/go-v2-host-scope.md`
93
-- `.agents/sow/specs/sensitive-data-discipline.md`
94
-
95
-Current state:
96
-
97
-- Profile loading is shared, but consumer projection is implemented by mirrored package-private filters.
98
-- Topology row identity is encoded in metric names instead of profile schema.
99
-- Topology dispatch is tied to metric-name constants.
100
-- VLAN-context topology has a separate hardcoded profile load path.
101
-- Post-load profile mutations exist in resolver/collector paths and must move before shared immutable projections are safe.
102
-
103
-Risks:
104
-
105
-- Regular SNMP regression if metrics projection changes selected metrics, metadata, global tags, virtual metrics, or manual profile behavior.
106
-- Topology row loss if `TopologyKind` is not threaded from profile rows through scalar/table collectors into topology ingest.
107
-- `HiddenMetrics` regression if implementation treats it as topology-only and breaks other underscore-prefixed delivery consumers.
108
-- Profile merge regressions if inherited topology rows do not use a precise identity key.
109
-- Validation churn from adding `topology:` and consumer fields to the profile schema.
110
-- Documentation drift if profile-format docs and project SNMP authoring skill are not updated with the new contract.
111
-
112
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
113
-
114
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
115
-
116
-Problem / root-cause model:
117
-
118
-- The profile loader is not the primary problem. The loader already centralizes profile directories, YAML parsing, embedded defaults, `extends`, and caching.
119
-- The root cause is that resolved profiles contain a mixed bag of regular metric rows, topology rows, metadata, and tags without an explicit schema-level consumer contract.
120
-- Regular SNMP and SNMP topology compensate with mirrored filters and a hardcoded classifier based on `_topology_*` names and tag/metadata prefixes.
121
-- VLAN-context topology worsens the split by directly loading hardcoded mixins and bypassing resolver semantics.
122
-
123
-Evidence reviewed:
124
-
125
-- `collector/snmp/ddsnmp/topology_classify.go:16-30` exact topology metric name allowlist.
126
-- `collector/snmp/ddsnmp/topology_classify.go:43-65` prefix-based topology identifier heuristic.
127
-- `collector/snmp/topology_profile_filter.go:10-50` regular SNMP strips topology data.
128
-- `collector/snmp_topology/profile_filter.go:10-47` topology keeps topology data.
129
-- `collector/snmp_topology/topology_vlan_context_collect.go:16-28` hardcoded VLAN-context profile loads.
130
-- `collector/snmp/ddsnmp/ddsnmpcollector/collector.go:122-123` generic underscore-prefixed hidden metrics bucketing.
131
-- `collector/snmp_topology/collector.go:220-224` topology currently ingests hidden and regular metric slices.
132
-- `collector/snmp_topology/topology_cache_ingest.go:11-30` existing profile tag hook that reads `pm.DeviceMetadata`.
133
-- `collector/snmp/ddsnmp/profile.go:194-252` current regular metric merge identity.
134
-- `collector/snmp/ddsnmp/profile.go:375-440` enrichment/dedup currently only covers `Definition.Metrics` and `Definition.VirtualMetrics`.
135
-
136
-Affected contracts and surfaces:
137
-
138
-- Profile schema: `ProfileDefinition`, `MetricsConfig`, metadata fields, top-level/global metric tags, virtual metrics validation.
139
-- ddsnmp public API: catalog, resolve request, manual profile policy, resolved profile set, projected views.
140
-- ddsnmpcollector output: `Metric.TopologyKind`, `ProfileMetrics.TopologyMetrics`, preservation of `HiddenMetrics`.
141
-- SNMP collector metrics path: profile selection/projection and chart labels.
142
-- SNMP topology path: profile selection, VLAN-context, topology ingest, handler registry.
143
-- Default SNMP profile YAMLs under `config/go.d/snmp.profiles/default/`.
144
-- Tests under `collector/snmp`, `collector/snmp/ddsnmp`, `collector/snmp/ddsnmp/ddprofiledefinition`, `collector/snmp/ddsnmp/ddsnmpcollector`, and `collector/snmp_topology`.
145
-- Documentation and durable artifacts: `collector/snmp/profile-format.md`, `.agents/skills/project-snmp-profiles-authoring/SKILL.md`, `.agents/sow/specs/snmp-profile-projection.md`.
146
-
147
-Existing patterns to reuse:
148
-
149
-- Keep the shared loader/resolver model from `collector/snmp/ddsnmp/load.go` and `profile.go`.
150
-- Reuse SNMP row shape from `MetricsConfig` for topology rows via `TopologyConfig`.
151
-- Reuse existing table/scalar collection logic but thread parent topology row metadata into emitted metrics.
152
-- Reuse `updateTopologyProfileTags` as the hook for Decision 8.A.
153
-- Reuse existing profile merge identity concepts: scalar name/OID and table identity plus symbol name.
154
-- Reuse existing validation style in `ddprofiledefinition/validation.go`.
155
-- Reuse focused Go package tests and profile fixture tests instead of broad full-repo validation.
156
-
157
-Risk and blast radius:
158
-
159
-- Regular SNMP blast radius is broad because default profiles affect production metric collection. Metrics projection must be parity-protected.
160
-- Topology blast radius is acceptable for topology behavior because the feature is WIP/nightly-only, but topology must still produce coherent data.
161
-- `HiddenMetrics` has cross-PR/non-topology risk. It must remain a general-purpose delivery path until separately audited and refactored.
162
-- Moving per-profile mutations to load-time and matched-set deduplication to resolve-time changes pointer and clone assumptions. Tests must prove projections cannot mutate shared catalog state.
163
-- YAML migration touches topology profile mixins and `_std-cdp-mib.yaml`; profile load validation must catch mistakes early.
164
-- Validation rejecting topology-row chart/export fields may expose existing accidental fields during migration.
165
-
166
-Sensitive data handling plan:
167
-
168
-- This SOW and implementation should reference profile filenames, metric names, OIDs, struct fields, and tests only.
169
-- Do not write raw SNMP communities, SNMPv3 credentials, bearer tokens, passwords, customer hostnames, customer sysName/sysDescr values, private endpoints, non-private customer-identifying IPs, customer names, or personal data into SOWs, specs, docs, skills, tests, fixtures, or code comments.
170
-- Any real SNMP fixture used later must be sanitized before committing. Use neutral device names and placeholder values.
171
-- Existing profile YAMLs contain public OIDs and generic vendor/device metadata, not credentials.
172
-
173
-Implementation plan:
174
-
175
-1. Schema and API surface, no behavior change.
176
- - Add `Topology []TopologyConfig` to `ProfileDefinition`.
177
- - Add a closed 18-value `TopologyKind` enum for current topology row shapes. `systemUptime` is not a topology kind.
178
- - Add `MetadataField.Consumers`.
179
- - Add validation for unknown topology kinds and metrics-only fields under topology row anchor symbols.
180
- - Add `Catalog`, `Resolve`, `Project`, and `FilterByKind` API surface without cutting over call sites.
181
-
182
-2. Split mutation handling between load-time/catalog compilation and resolve-time matched-set processing.
183
- - Move per-profile/idempotent mutation to load-time: `enrichProfiles` and `handleCrossTableTagsWithoutMetrics`.
184
- - Extend cross-table-tag synthesis to scan both `Definition.Metrics` and `Definition.Topology`, placing synthesized entries on the owning slice.
185
- - Keep `deduplicateMetricsAcrossProfiles` at resolve-time inside `Catalog.Resolve()` because it needs the matched, sorted profile set.
186
- - Extend resolve-time deduplication to `Definition.Topology`.
187
- - Preserve metrics-path parity with current `FindProfiles`.
188
-
189
-3. Profile YAML migration.
190
- - Move `_topology_*` rows from `metrics:` to top-level `topology:` in topology mixins.
191
- - Split `_std-cdp-mib.yaml` into real metrics and `_std-topology-cdp-mib.yaml`.
192
- - Add `kind:` to every topology row.
193
- - Keep metadata/global tag annotations minimal per Decision 4.A.
194
- - Do not modify `_system-base.yaml`; `systemUptime` remains a regular metric.
195
- - Phases 3-6 are one logical topology cutover and must not ship as a broken intermediate topology state.
196
-
197
-4. Catalog/Resolve/Project introduction with parity tests.
198
- - Keep old `FindProfiles` temporarily.
199
- - Prove `Catalog.Resolve(...).Project(ConsumerMetrics)` matches current `selectCollectionProfiles(FindProfiles(...))`.
200
- - Prove `Project(ConsumerTopology)` matches current topology selection after YAML migration.
201
-
202
-5. Plumb topology collection through `ddsnmpcollector`.
203
- - Add `Metric.TopologyKind`.
204
- - Add `ProfileMetrics.TopologyMetrics`.
205
- - Prefer a topology collection wrapper that stamps `TopologyKind` after existing scalar/table emission; do not widen regular builders unless the SOW records why.
206
- - Add explicit topology collection from `Definition.Topology`.
207
- - Do not map regular `systemUptime` metrics into topology projection. Topology queries uptime through `pkg/snmputils.GetSysUptime`.
208
- - Remove topology's dependency on underscore-prefix hidden metrics.
209
- - Preserve generic `HiddenMetrics` behavior for non-topology underscore-prefixed metrics.
210
- - Enforce that topology rows cannot be delivered through both `pm.HiddenMetrics` and `pm.TopologyMetrics` in one poll.
211
-
212
-6. Cut over call sites and dispatch.
213
- - Switch `collector/snmp/profile_sets.go`, `collector/snmp_topology/collector.go`, and `collector/snmp_topology/topology_vlan_context_collect.go`.
214
- - Replace metric-name dispatch switch with handler registry keyed by `TopologyKind`.
215
- - Extend `updateTopologyProfileTags` to apply `pm.Tags` as local device/profile labels.
216
- - Add side-by-side fixture-level runtime parity before deleting the old path.
217
-
218
-7. Delete dead code and update artifacts.
219
- - Delete topology classifier/filter code, dead topology profile constants, and obsolete metric-name dispatch constants; remove topology runtime reliance on `FinalizeProfiles`.
220
- - Update `collector/snmp/profile-format.md`.
221
- - Update `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
222
- - Create `.agents/sow/specs/snmp-profile-projection.md`.
223
-
224
-Validation plan:
225
-
226
-- Resolver parity: for every default profile, `Catalog.Resolve(...).Project(ConsumerMetrics)` matches current `selectCollectionProfiles(FindProfiles(...))`.
227
-- Topology parity: `Project(ConsumerTopology)` matches current `selectTopologyRefreshProfiles(FindProfiles(...))` after YAML migration.
228
-- VLAN-context equivalence: `Project(ConsumerTopology).FilterByKind(vlanScopableKinds)` matches today's hardcoded VLAN-context loader.
229
-- Manual policy: regular metrics with `manual_profiles` plus matching `sysObjectID` do not augment; topology does augment.
230
-- Topology extends merge: a vendor/root profile extending a topology mixin inherits `Definition.Topology` rows through `Profile.merge().mergeTopology(base)`.
231
-- Topology-to-topology merge: topology rows require explicit `kind`; matching rows with conflicting explicit kinds are rejected.
232
-- Clone coverage: `ProfileDefinition.Clone()`, `TopologyConfig.Clone()`, `MetadataField.Clone()`, and consumer-set clone paths do not share mutable state.
233
-- Validation rejects unknown `TopologyKind`, metrics-only fields on `TopologyConfig`, underscore-prefixed topology row anchor names, and mixed-consumer virtual metrics.
234
-- Top-level/global `metric_tags`: values in `pm.Tags` reach topology local device/profile labels through `updateTopologyProfileTags`, not per-row dispatch tags.
235
-- HiddenMetrics non-topology preservation: `TestCollector_Collect_PreservesHiddenMetrics` in `collector/snmp/ddsnmp/ddsnmpcollector/collector_test.go` continues to pass.
236
-- Topology double-bucketing guard: a topology row must not appear in both `pm.HiddenMetrics` and `pm.TopologyMetrics` in the same poll.
237
-- `systemUptime`: `_system-base.yaml` remains metrics-only; topology receives uptime through `pkg/snmputils.GetSysUptime` without a topology kind or YAML topology row.
238
-- VLAN-context kind flow: `vlanScopableKinds` contains exactly `KindIfName`, `KindBridgePortIfIndex`, `KindFdbEntry`, and `KindStpPort`, and VLAN-context synthetic metrics carry `TopologyKind`.
239
-- Side-by-side runtime parity: representative default-profile fixture emits equivalent regular SNMP `ProfileMetrics` before old path deletion.
240
-- Mutation isolation: mutating one projection cannot affect another after resolve.
241
-- `Metric.TopologyKind`: every topology kind is emitted correctly from fixture collection.
242
-- Run narrow suites:
243
- - `go test ./plugin/go.d/collector/snmp/ddsnmp/ddprofiledefinition`
244
- - `go test ./plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector`
245
- - `go test ./plugin/go.d/collector/snmp/...`
246
- - `go test ./plugin/go.d/collector/snmp_topology/...`
247
-
248
-Artifact impact plan:
249
-
250
-- AGENTS.md: no expected change; existing SOW/process and SNMP skill triggers are sufficient.
251
-- Runtime project skills: update `.agents/skills/project-snmp-profiles-authoring/SKILL.md` with `topology:` row and `TopologyKind` guidance.
252
-- Specs: create `.agents/sow/specs/snmp-profile-projection.md`.
253
-- End-user/operator docs: update `collector/snmp/profile-format.md`; check whether SNMP topology docs/examples need updates.
254
-- End-user/operator skills: no expected public skill update unless docs/spec changes are mirrored into public skill artifacts.
255
-- SOW lifecycle: SOW remains `open` in `pending/` until implementation starts; move to `current/` and mark `in-progress` only when implementation begins.
256
-
257
-Open-source reference evidence:
258
-
259
-- No external mirrored open-source repositories were checked. The work concerns Netdata's internal SNMP profile schema and collector implementation; local repo code, profile YAMLs, tests, and project skills are the relevant ground truth for this SOW.
260
-
261
-Open decisions:
262
-
263
-- None. User decisions and implementation sub-decisions are recorded below.
264
-
265
-## Implications And Decisions
266
-
267
-User decisions:
268
-
269
-1. Manual profile policy: use internal policy. Regular metrics use fallback-only manual profiles; topology uses augment.
270
-2. Topology merge behavior: topology rows require explicit `kind`; explicit derived topology wins for matching row identities; conflicting explicit topology kinds are rejected. Earlier "derived omits kind" inheritance wording is unreachable because validation rejects persisted topology rows without `kind`.
271
-3. Virtual metric projection: reject mixed-consumer virtual metrics at validation.
272
-4. Metadata/global tag defaults: metadata fields and top-level/global `metric_tags` default to both `metrics` and `topology`, with explicit narrowing when needed.
273
-5. Topology row schema: use a separate top-level `topology:` list for topology rows instead of embedding topology rows in `metrics[]`.
274
-6. TopologyKind granularity: define one closed `TopologyKind` per current topology row shape, including LLDP management-address rows.
275
-7. `sysUptime` path: superseded during implementation. Remove `KindSysUptime`; topology uptime acquisition uses `pkg/snmputils.GetSysUptime`.
276
-8. Global metric tags in topology: apply top-level/global `metric_tags` as local device/profile labels, not as dispatch keys on every topology row.
277
-9. `HiddenMetrics`: treat as a general delivery container, not topology-owned. Do not delete or redefine it without auditing non-topology consumers.
278
-10. Before implementation starts, write a step-by-step implementation plan detailed enough for external readiness review. Ask Claude to judge the plan as `READY TO WRITE CODE` or `NEEDS ADJUSTMENTS`; do not start code until that review is accepted or required adjustments are recorded.
279
-11. Fifth readiness review adjustments are accepted: B1-B6 and S1-S6. Phases 3-6 are one logical topology cutover; the original `KindSysUptime` projection/collector mapping from existing regular `systemUptime` rows was later superseded by Decision 14.
280
-12. Sixth readiness review adjustments are accepted: N1 and N2-N5. `Profile.merge()` must merge `Definition.Topology` during `extends:` loading, clone targets are explicit, VLAN-context synthetic metrics carry `TopologyKind`, `vlanScopableKinds` is pinned to existing VLAN ingest kinds, and new fields follow existing YAML/JSON tag conventions.
281
-13. Seventh readiness review verdict is `READY TO WRITE CODE`. Non-blocking hygiene tightenings NB1-NB4 are accepted: expanded stale-helper scans, `Catalog`/`Resolve`/`Project` live in `collector/snmp/ddsnmp`, top-level/global metric tags use a wrapper for `Consumers`, and post-cutover topology ingest loops over `pm.TopologyMetrics`. The original regular `systemUptime` metrics mapping was later superseded by Decision 16.
282
-14. User correction during implementation: remove `KindSysUptime`. `systemUptime` is not a topology row kind. A direct `pkg/snmputils` uptime helper was proposed and later accepted in Decision 16.
283
-15. Regular SNMP metrics must keep uptime collection through existing metrics profile rows. Do not move or remove uptime from `_system-base.yaml`.
284
-16. Topology uptime acquisition after removing `KindSysUptime`: use option A. Add `pkg/snmputils.GetSysUptime(gosnmp.Handler)` with the same three OIDs and scale rules as `_system-base.yaml`, and call it from `snmp_topology` during refresh. Regular SNMP uptime metrics remain profile-driven.
285
-17. Public profile-format documentation should describe the current topology schema without mentioning the old underscore-prefixed topology metric-name implementation.
286
-
287
-Resolved implementation decision:
288
-
289
-1. Topology uptime acquisition after removing `KindSysUptime`. Resolved: A.
290
- - Evidence:
291
- - `config/go.d/snmp.profiles/default/_system-base.yaml:11-44` defines the three regular uptime fallback rows: `snmpEngineTime` in seconds, `hrSystemUptime` scaled by `0.01`, and `sysUpTime` scaled by `0.01`.
292
- - `collector/snmp_topology/topology_cache_ingest.go:52-67` stores uptime into the local topology device and the `sys_uptime` label.
293
- - `collector/snmp_topology/topology_local_actor_attrs.go:32-34` exports `sys_uptime` as local actor attributes when present.
294
- - `collector/snmp/ddsnmp/profile_catalog.go:271-320` currently keeps a special topology projection path for `systemUptime`/`sysUpTime`; this should disappear with `KindSysUptime`.
295
- - A. Add `pkg/snmputils.GetSysUptime(gosnmp.Handler)` with the same three OIDs and scale rules, and call it from `snmp_topology` during refresh.
296
- - Pros: removes `KindSysUptime`; keeps profile topology projection pure; keeps regular SNMP metrics unchanged; localizes the hardcoded fallback to one SNMP utility helper.
297
- - Cons: duplicates uptime OID/scale knowledge from `_system-base.yaml`; future uptime fallback changes need the helper updated too; adds one SNMP GET per topology refresh.
298
- - B. Keep using ddsnmp regular metric collection for uptime, but identify `systemUptime`/`sysUpTime` by name in `snmp_topology`.
299
- - Pros: no extra SNMP GET; reuses profile scaling and fallback behavior.
300
- - Cons: keeps metric-name special casing and forces topology projection to retain a regular metric row.
301
- - C. Stop collecting uptime for topology.
302
- - Pros: simplest implementation; no duplicate OIDs or extra SNMP request.
303
- - Cons: topology output loses `sys_uptime` on local actors; existing tests and output paths already treat it as useful local device enrichment.
304
- - Recommendation: A.
305
-
306
-Implementation sub-decisions:
307
-
308
-1. Add `ProfileMetrics.TopologyMetrics []Metric` and remove topology's dependency on underscore-prefix `HiddenMetrics`.
309
-2. Use `(kind, table_identity, symbol_name)` as topology row identity for both load-time `extends:` merge through `Profile.merge().mergeTopology(base)` and resolve-time matched-set deduplication.
310
-3. Extend `updateTopologyProfileTags` to read `pm.Tags` for Decision 8.A.
311
-4. Reject metrics-only fields on topology row anchor `symbol` / `symbols`, while keeping `metric_tags` extraction fields valid.
312
-5. Split mutation timing: move per-profile enrichment and cross-table-tag synthesis to load-time/catalog compilation; keep `deduplicateMetricsAcrossProfiles()` at resolve-time inside `Catalog.Resolve()` on catalog-cloned matched profiles and extend it to topology rows.
313
-
314
-## Plan
315
-
316
-1. Keep this SOW in `pending/` until user explicitly approves implementation.
317
-2. Before implementation starts, run a read-only Claude readiness review against the step-by-step implementation plan below.
318
-3. If Claude says `NEEDS ADJUSTMENTS`, record each accepted/rejected point in the SOW and TODO before code.
319
-4. On implementation start, move this SOW to `.agents/sow/current/`, set `Status: in-progress`, and execute the 7-phase implementation plan from the Pre-Implementation Gate.
320
-5. Maintain the SOW execution log after each implementation phase.
321
-6. Close only after implementation, docs/spec/skill updates, validation, artifact maintenance gate, and follow-up mapping are complete.
322
-
323
-### Step-by-Step Implementation Plan
324
-
325
-This is the pre-code checklist that must be reviewed before implementation. It is intentionally more detailed than the phase list so ordering bugs, hidden coupling, and missing validation can be found before edits begin.
326
-
327
-#### 0. Activate SOW And Capture Baseline
328
-
329
-1. Move this SOW from `.agents/sow/pending/` to `.agents/sow/current/`.
330
-2. Change status from `open` to `in-progress`.
331
-3. Record the branch name and current dirty files in the execution log.
332
-4. Run baseline focused tests before code changes where practical:
333
- - From `src/go`: `go test ./plugin/go.d/collector/snmp/ddsnmp/ddprofiledefinition`
334
- - From `src/go`: `go test ./plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector`
335
- - From `src/go`: `go test ./plugin/go.d/collector/snmp/...`
336
- - From `src/go`: `go test ./plugin/go.d/collector/snmp_topology/...`
337
-5. If any baseline test fails before edits, record the exact package/test and failure class. Do not count pre-existing failures as validation of the new implementation.
338
-
339
-#### 1. Add Schema And API Surface Without Behavior Change
340
-
341
-1. Add canonical `ProfileConsumer` values for `metrics` and `topology`.
342
-2. Add a single canonical closed `TopologyKind` definition with the 18 accepted topology row kinds. Do not include `sys_uptime`.
343
-3. Avoid duplicated string constants. If multiple packages need the type, use one definition plus aliases/imports rather than parallel enums.
344
-4. Add `Topology []TopologyConfig` to `ProfileDefinition`.
345
-5. Add `TopologyConfig` as a wrapper around the existing metric row shape plus required `kind`.
346
-6. Add `Consumers` to metadata fields.
347
-7. Add a top-level/global metric-tag wrapper that carries `Consumers`. Do not add consumer semantics to per-row `MetricTagConfig`; per-row metric tags inherit their row's consumer.
348
-8. Follow the existing schema tag convention for every new persisted field: matching `yaml` and `json` names, `omitempty` for optional fields, and `yaml:"-" json:"-"` for runtime-only fields.
349
-9. Update clone/deep-copy paths explicitly: `ProfileDefinition.Clone()` must clone `Topology`; `TopologyConfig.Clone()` must clone the embedded row config; `MetadataField.Clone()` must clone/copy `Consumers`; top-level/global metric-tag wrapper clone paths must clone/copy `Consumers`.
350
-10. Add validation for unknown topology kinds.
351
-11. Add validation for metrics-only fields inside topology row anchor `symbol`/`symbols`: `Options`, `ChartMeta`, `MetricType`, `Mapping`, `Transform`, `ScaleFactor`, `Format`, and `ConstantValueOne`.
352
-12. Extend the existing symbol traversal context, either with `TopologyScalarSymbol`/`TopologyColumnSymbol` or a topology-row flag, so value-symbol validation can reject topology-only-invalid fields without breaking `MetricTagSymbol`.
353
-13. Extend `validateEnrichVirtualMetrics` to accept topology rows and reject any virtual metric source that resolves to a `topology:` row.
354
-14. Reject underscore-prefixed `name:` values on topology row anchor symbols so topology rows cannot also flow into generic `HiddenMetrics`.
355
-15. Keep metric-tag extraction fields valid under topology rows where they extract tags from indexes/values.
356
-16. Do not reject existing `_topology_*` rows in `metrics:` until the YAML migration and cutover are complete; add final rejection only in cleanup.
357
-17. Add parse/clone/validation tests in `ddprofiledefinition`.
358
-
359
-#### 2. Split Profile Mutations By Required Context
360
-
361
-1. Inventory every mutation that currently happens after profile load: `handleCrossTableTagsWithoutMetrics`, `enrichProfiles`, `deduplicateMetricsAcrossProfiles`, and any resolver-time mutation found while editing.
362
-2. Move `handleCrossTableTagsWithoutMetrics` out of `ddsnmpcollector.New()` and into catalog/profile compilation before projections share profile pointers.
363
-3. Extend `Profile.merge()` with `mergeTopology(base)` and call it during `extends:` loading.
364
-4. `mergeTopology(base)` must use topology row identity `(kind, table_identity, symbol_name)` and Decision 2.B merge semantics.
365
-5. Add `profile_test.go` coverage proving a profile extending a topology mixin inherits `Definition.Topology` rows.
366
-6. Extend cross-table-tag synthesis to scan both `Definition.Metrics` and `Definition.Topology`; place synthesized entries on the slice that owns the consuming row so the correct collection path walks them.
367
-7. Move `enrichProfiles()` to load/catalog compilation and extend it to process `Definition.Topology`.
368
-8. Keep `deduplicateMetricsAcrossProfiles()` at resolve-time inside `Catalog.Resolve()` because it needs the already-matched, specificity-sorted profile set. It must operate on catalog-cloned profiles before projections are returned.
369
-9. Extend resolve-time deduplication to process `Definition.Topology` with topology row identity `(kind, table_identity, symbol_name)`.
370
-10. Preserve `Definition.Metrics` behavior exactly for regular SNMP.
371
-11. Treat the suspected `removeConstantMetrics()` value-copy issue as separate unless touched by this refactor. Do not extend `removeConstantMetrics()` to topology because topology row validation rejects `ConstantValueOne`; if this changes or the existing value-copy bug becomes relevant, either fix it with a narrow test or record a separate follow-up SOW.
372
-12. Add mutation-isolation tests showing one projected view cannot mutate another or the catalog. The minimum assertion is: resolve a device twice, mutate a nested map/slice in view 1, including nested state under `Definition.Topology`, then assert view 2 and a fresh resolve from the same catalog do not contain that mutation.
373
-
374
-#### 3. Migrate Profile YAML
375
-
376
-1. Move topology rows from `metrics:` to top-level `topology:` in the topology mixins.
377
-2. Split `_std-cdp-mib.yaml` into regular CDP metrics and `_std-topology-cdp-mib.yaml` topology rows.
378
-3. Add `kind:` to every topology row using the accepted `TopologyKind` enum.
379
-4. Rename topology row anchor symbol names away from `_topology_*`; dispatch must use `kind`, not the old hidden-metric name.
380
-5. Preserve OIDs, table identities, symbols, and metric_tags unless the move exposes a concrete bug.
381
-6. Do not modify `_system-base.yaml`; `systemUptime` remains a regular metrics row. Topology obtains uptime through `pkg/snmputils.GetSysUptime`.
382
-7. If any symbol/tag OID is added or changed, run the SNMP profile authoring MAX-ACCESS checks and record evidence. Pure row moves with unchanged OIDs should record that no readable-symbol semantics changed.
383
-8. Update profile extender references so vendors that previously extended mixed CDP/topology content still get the intended regular and topology rows.
384
-9. Inventory every profile extending `_std-cdp-mib.yaml` and update each profile that should retain CDP topology to also extend `_std-topology-cdp-mib.yaml`. Current default extenders are `_cisco-base.yaml` and `cisco-sb.yaml`.
385
-10. Add or update profile load tests for the migrated files.
386
-11. Treat phases 3-6 as one logical topology cutover. Do not ship a state where topology YAML has moved to `topology:` but topology runtime still reads only `_topology_*` metrics from `metrics:`.
387
-12. Record that topology mixins may become topology-only/abstract after migration; validation/tests must not assume every topology mixin produces regular `metrics:` rows when loaded as a root.
388
-
389
-#### 4. Introduce Catalog, Resolve, Project, And Filter
390
-
391
-1. Add `Catalog`, `ResolveRequest`, `ManualProfilePolicy`, `ResolvedProfileSet`, and projected view types.
392
-2. Implement the catalog/resolver/projection API in `collector/snmp/ddsnmp`, next to the existing profile loader, `FindProfiles()`, and profile model.
393
-3. Implement manual profile policies: metrics call sites use fallback-only; topology call sites use augment.
394
-4. Implement `Project(ConsumerMetrics)` and `Project(ConsumerTopology)`.
395
-5. Implement `FilterByKind(map[TopologyKind]bool)` for topology projections.
396
-6. Make projections non-mutating. Prefer immutable catalog-owned profiles plus read-only projected slices or precomputed buckets over per-call deep clone.
397
-7. Keep temporary wrappers such as `FindProfiles()` only as needed to prove parity during the same SOW; remove or simplify them during cleanup.
398
-8. Add resolver parity tests for regular SNMP.
399
-9. Add topology projection parity tests against the current topology filter behavior after YAML migration.
400
-10. Add manual-policy tests covering matching `sysObjectID` plus `manual_profiles` for both metrics and topology.
401
-
402
-#### 5. Plumb Topology Collection Through ddsnmpcollector
403
-
404
-1. Add `TopologyKind` to emitted `ddsnmp.Metric`.
405
-2. Add `TopologyMetrics []Metric` to `ProfileMetrics`.
406
-3. Preserve `HiddenMetrics []Metric` as the generic underscore-prefixed non-topology delivery container.
407
-4. Add explicit topology collection from `Definition.Topology`, parallel to the existing scalar/table collection path.
408
-5. Prefer a topology collection wrapper that calls the existing scalar/table collection helpers and stamps `TopologyKind` after emit. Do not widen regular scalar/table builder signatures unless the wrapper proves insufficient and the SOW records why.
409
-6. Add `pkg/snmputils.GetSysUptime(gosnmp.Handler)` with the `_system-base.yaml` uptime OIDs and scale rules, and call it from `snmp_topology` refresh. Do not add duplicate `systemUptime` YAML topology rows or topology kinds.
410
-7. Stop relying on underscore-prefix metric names for topology delivery.
411
-8. Keep `collectHiddenMetrics()` behavior for non-topology underscore-prefixed regular metrics.
412
-9. Enforce the invariant that a topology row lives in exactly one delivery slice for a single poll: not both `pm.HiddenMetrics` and `pm.TopologyMetrics`.
413
-10. Ensure `TestCollector_Collect_PreservesHiddenMetrics` continues to pass.
414
-11. Add tests proving topology rows populate `pm.TopologyMetrics` with the correct `Metric.TopologyKind`.
415
-12. Add a double-bucketing assertion test that fails if a `_topology_*` or topology-kind row appears in both `pm.HiddenMetrics` and `pm.TopologyMetrics`.
416
-
417
-#### 6. Cut Over SNMP And Topology Call Sites
418
-
419
-1. Switch regular SNMP profile selection to `DefaultCatalog().Resolve(...).Project(ConsumerMetrics)`.
420
-2. Switch SNMP topology profile selection to `DefaultCatalog().Resolve(...).Project(ConsumerTopology)`.
421
-3. Replace VLAN-context hardcoded `LoadProfileByName()` calls with `Project(ConsumerTopology).FilterByKind(vlanScopableKinds)`.
422
-4. Define `vlanScopableKinds` in topology Go code, not profile YAML. Pin it to the current VLAN-context ingest set: `KindIfName`, `KindBridgePortIfIndex`, `KindFdbEntry`, and `KindStpPort`.
423
-5. Replace metric-name topology dispatch with a handler registry keyed by `TopologyKind`.
424
-6. Make topology row handlers receive the full `ddsnmp.Metric`; uptime is handled by the explicit `snmputils` helper path, not by the topology handler registry.
425
-7. Register each topology cache handler from its domain file.
426
-8. Extend `updateTopologyProfileTags` to read `pm.Tags` and apply them as local device/profile labels, not per-row dispatch keys.
427
-9. Update `ingestTopologyVLANContextMetrics()` so any synthetic metric passed to kind-keyed dispatch carries/preserves `TopologyKind`.
428
-10. Update `ingestTopologyProfileMetrics` so the post-cutover loop dispatches only `pm.TopologyMetrics` by `TopologyKind`; regular `pm.Metrics` are not part of topology projection.
429
-11. Add dispatch parity tests and VLAN-context equivalence tests, including `TopologyKind` population for VLAN-context synthetic metrics.
430
-12. Add a side-by-side fixture-level runtime parity test before deleting the old path. Use a representative default-profile fixture such as the existing Cisco Nexus profile path in `collector/snmp/ddsnmp/profile_test.go:178`, and compare emitted `ProfileMetrics` fields that must remain stable for regular SNMP: `Tags`, `DeviceMetadata`, `Metrics`, ordering, and metric tags.
431
-
432
-#### 7. Delete Dead Code And Update Artifacts
433
-
434
-1. Search for stale references before deleting: `_topology_`, `IsTopologyMetric`, `LooksLikeTopologyIdentifier`, `MetricConfigContainsTopologyData`, `MetricTagConfigContainsTopologyData`, `MetadataFieldContainsTopologyData`, `MetadataContainsTopologyData`, `SysobjectIDMetadataContainsTopologyData`, `ProfileContainsTopologyData`, `ProfileHasCollectionData`, `TopologySysUptime`, `LoadProfileByName`, and `HiddenMetrics`.
435
-2. Delete dead topology classifier/filter code only after call sites and tests no longer depend on it.
436
-3. Delete `FinalizeProfiles()` only after VLAN-context and other callers no longer require it.
437
-4. Delete only topology-specific use of hidden metrics; do not delete generic `HiddenMetrics`.
438
-5. Update `collector/snmp/profile-format.md`.
439
-6. Update `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
440
-7. Create `.agents/sow/specs/snmp-profile-projection.md`.
441
-8. Run the focused validation suites.
442
-9. Run same-failure/stale-reference scans and record results in this SOW.
443
-10. Complete the artifact maintenance gate and follow-up mapping.
444
-11. Explicit test impact:
445
- - Delete/rewrite `collector/snmp/ddsnmp/topology_classify_test.go`.
446
- - Rewrite `collector/snmp/profile_sets_test.go` around projection.
447
- - Rewrite `collector/snmp_topology/profile_filter_test.go` around projection.
448
- - Rewrite `collector/snmp_topology/topology_profiles_test.go` if hardcoded topology profile constants disappear.
449
- - Rewrite `collector/snmp/ddsnmp/ddsnmpcollector/topology_profile_index_test.go` if it no longer needs `LoadProfileByName()`.
450
- - Rewrite topology fixtures in `collector/snmp_topology/topology_cache_test.go:1327-1371` and `collector/snmp_topology/collector_refresh_test.go:123-152` to use `TopologyMetrics`, not topology `HiddenMetrics`.
451
- - Preserve `collector/snmp/ddsnmp/ddsnmpcollector/collector_test.go:153` `TestCollector_Collect_PreservesHiddenMetrics`.
452
-
453
-## Execution Log
454
-
455
-### 2026-05-06
456
-
457
-- Created pending SOW from the reviewed design in `src/go/plugin/go.d/TODO-snmp-profile-loading-topology.md`.
458
-- Filled Pre-Implementation Gate with accepted decisions, implementation sub-decisions, risks, validation, and artifact impact plan.
459
-- Added step-by-step implementation plan for external readiness review before code.
460
-- Reconciled fifth readiness review: accepted B1-B6 and S1-S6, split load-time vs resolve-time mutation handling, made phases 3-6 one logical topology cutover, kept `_system-base.yaml` unchanged, and added validation/test gates.
461
-- Reconciled sixth readiness review: accepted N1 and N2-N5 plus actionable minor notes, added `Profile.merge().mergeTopology(base)` to the plan, enumerated clone/tag conventions, pinned VLAN-context kinds, required VLAN-context synthetic metrics to carry `TopologyKind`, kept `removeConstantMetrics()` metrics-only, and added CDP extender inventory.
462
-- Reconciled seventh readiness review: verdict `READY TO WRITE CODE`; folded in non-blocking NB1-NB4 hygiene around stale-helper scans, API package location, global metric-tag consumer wrapper, and post-cutover topology ingest loop shape.
463
-- User approved proceeding with implementation.
464
-- Activated SOW on branch `snmp-profile-projection`.
465
-- Dirty files at activation: `src/go/plugin/go.d/TODO-snmp-profile-loading-topology.md` and `.agents/sow/current/SOW-0012-20260506-snmp-profile-projection.md`.
466
-- Phase 0 baseline checks started before implementation code edits.
467
-- Phase 0 baseline checks completed:
468
- - `go test ./plugin/go.d/collector/snmp/ddsnmp/ddprofiledefinition`: passed from cache.
469
- - `go test ./plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector`: passed.
470
- - `go test ./plugin/go.d/collector/snmp/...`: passed.
471
- - `go test ./plugin/go.d/collector/snmp_topology/...`: initial sandbox run failed because the Go build cache was not writable; rerun outside sandbox passed.
472
-- Phase 1 schema/API surface completed:
473
- - Added canonical profile consumers, 18-value topology kind enum, `ProfileDefinition.Topology`, `TopologyConfig`, metadata `Consumers`, and top-level/global metric-tag consumer wrapper.
474
- - Added clone support for `Topology`, metadata consumers, and global metric-tag consumers.
475
- - Added validation for unknown topology kinds, topology-row metrics-only fields, underscore-prefixed topology symbols, invalid/duplicate consumers, and virtual metrics that source topology rows.
476
- - Kept per-row metric tags on the existing `MetricTagConfig` type; only top-level/global `metric_tags` use the consumer wrapper.
477
- - Updated transitional callers/tests that consume top-level/global metric tags.
478
-- Phase 1 validation:
479
- - `go test ./collector/snmp/ddsnmp/ddprofiledefinition`: passed.
480
- - `go test ./collector/snmp/ddsnmp/ddsnmpcollector`: passed.
481
- - `go test ./collector/snmp/...`: passed.
482
- - `go test ./collector/snmp_topology/...`: passed.
483
-- Phase 2 mutation split completed:
484
- - Moved cross-table synthetic row preparation from `ddsnmpcollector.New()` into `ddsnmp` load/profile preparation.
485
- - Kept resolve-time cross-profile dedup in `deduplicateMetricsAcrossProfiles()` and extended it to `Definition.Topology`.
486
- - Added `Profile.merge().mergeTopology(base)` with topology identity `(kind, table_identity, symbol_name)` and conflict rejection for same row/different kind.
487
- - Extended load-time `enrichProfile()` mapping-ref handling to `Definition.Topology`.
488
- - Extended cross-table synthesis to scan both `Definition.Metrics` and `Definition.Topology`, placing synthetic rows on the owning slice.
489
- - Preserved `ddsnmp.FinalizeProfiles()` as the temporary programmatic preparation path until its planned cleanup.
490
- - Projection mutation-isolation coverage remains tied to Phase 4 because the `Catalog.Resolve().Project()` view does not exist yet.
491
-- Phase 2 validation:
492
- - `go test ./collector/snmp/ddsnmp/...`: passed.
493
- - `go test ./collector/snmp/...`: passed.
494
- - `go test ./collector/snmp_topology/...`: passed.
495
-- Phase 3 profile YAML migration completed:
496
- - Moved topology rows from `metrics:` to top-level `topology:` in `_std-topology-lldp-mib.yaml`, `_std-topology-fdb-arp-mib.yaml`, `_std-topology-q-bridge-mib.yaml`, `_std-topology-stp-mib.yaml`, and `_std-topology-cisco-vtp-mib.yaml`.
497
- - Split CDP topology into `_std-topology-cdp-mib.yaml` and updated `_cisco-base.yaml` plus `cisco-sb.yaml` to extend it.
498
- - Renamed topology row anchor symbols away from `_topology_*` and assigned explicit `kind:` values.
499
- - `_system-base.yaml` was not modified; `systemUptime` remains a regular metric.
500
-- Phase 4 catalog/projection completed:
501
- - Added `Catalog.Resolve()`, manual profile policies, `ResolvedProfileSet.Project()`, and `ProjectedView.FilterByKind()`.
502
- - Regular SNMP projection keeps regular metrics and virtual metrics; topology projection keeps topology rows and drops regular metrics.
503
- - Added projection separation and mutation-isolation tests.
504
-- Phase 5 topology collection completed:
505
- - Added `Metric.TopologyKind` and `ProfileMetrics.TopologyMetrics`.
506
- - Added topology collection wrapper over existing scalar/table collectors without widening regular builder signatures.
507
- - Preserved generic `HiddenMetrics` for non-topology underscore-prefixed metrics.
508
- - Added a double-bucketing guard test proving topology rows are delivered through `TopologyMetrics`, not `HiddenMetrics`.
509
-- Phase 6 call-site cutover completed:
510
- - Regular SNMP uses `DefaultCatalog().Resolve(...).Project(ConsumerMetrics)`.
511
- - SNMP topology uses `DefaultCatalog().Resolve(...).Project(ConsumerTopology)`.
512
- - VLAN-context topology uses projection plus `FilterByKind(vlanScopableKinds)` instead of hardcoded mixin filename loads.
513
- - Topology dispatch now uses `TopologyKind`.
514
- - `updateTopologyProfileTags` applies `pm.Tags` as local device/profile labels.
515
-- User selected option A for topology uptime after removing `KindSysUptime`.
516
-- Removed `KindSysUptime`; topology now queries uptime through `pkg/snmputils.GetSysUptime`, while regular SNMP uptime remains profile-driven in `_system-base.yaml`.
517
-- Reconciled final Claude close-out review:
518
- - Accepted B1 and filled `Outcome` plus `Lessons Extracted`.
519
- - Accepted NF1 as a prose-only correction because `kind` is required by validation, so kind inheritance from an omitted derived kind is unreachable.
520
- - Accepted NF2 by recording that existing resolver/profile fixture tests are the parity proxy rather than a now-impossible side-by-side old-path test.
521
- - Accepted NF7 by mapping retained `LoadProfileByName` and `FinalizeProfiles` as explicit retained APIs rather than hidden deferred cleanup.
522
- - Rejected NF5 as stale because the `KindSysUptime`/`IsTopologySysUptimeMetric` predicates no longer exist after the uptime-helper change.
523
-- Removed old-implementation wording about underscore-prefixed topology metric names from `collector/snmp/profile-format.md`; public docs now describe only the current `topology:` schema.
524
-- Phase 7 cleanup/artifacts completed:
525
- - Deleted the topology classifier, regular/topology filter files, legacy topology profile constants, and obsolete metric-name dispatch constants.
526
- - Rewrote/deleted affected tests around projection and topology-kind delivery.
527
- - Updated `collector/snmp/profile-format.md`.
528
- - Updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
529
- - Created `.agents/sow/specs/snmp-profile-projection.md`.
530
- - Runtime topology no longer calls `LoadProfileByName()`. The helper remains for intentional abstract-profile tests/programmatic checks because the normal catalog skips `_std-*` abstract roots.
531
- - `ddsnmp.FinalizeProfiles()` remains because synthetic `ddsnmpcollector` table tests still require programmatic profile preparation outside the catalog path; topology runtime no longer depends on it.
532
-
533
-## Validation
534
-
535
-Acceptance criteria evidence:
536
-
537
-- Regular SNMP now selects profiles through `DefaultCatalog().Resolve(...ManualProfileFallback).Project(ConsumerMetrics)` in `collector/snmp/profile_sets.go`.
538
-- SNMP topology now selects profiles through `DefaultCatalog().Resolve(...ManualProfileAugment).Project(ConsumerTopology)` in `collector/snmp_topology/collector.go`.
539
-- VLAN-context topology now calls projection plus `FilterByKind(vlanScopableKinds)` in `collector/snmp_topology/topology_vlan_context_collect.go`.
540
-- Topology rows are emitted through `ProfileMetrics.TopologyMetrics` and carry `Metric.TopologyKind`.
541
-- Topology uptime is queried through `pkg/snmputils.GetSysUptime`; it is not a `TopologyKind`.
542
-- Generic non-topology hidden metrics are preserved by `TestCollector_Collect_PreservesHiddenMetrics`.
543
-- Topology double-bucketing is guarded by `TestCollector_Collect_SeparatesTopologyMetricsFromHiddenMetrics`.
544
-- Old classifier/filter source files and hardcoded topology profile constants were deleted.
545
-
546
-Tests or equivalent validation:
547
-
548
-- `go test -count=1 ./pkg/snmputils`: passed.
549
-- `go test -count=1 ./collector/snmp/ddsnmp/...`: passed.
550
-- `go test -count=1 ./collector/snmp/...`: passed.
551
-- `go test -count=1 ./collector/snmp_topology/...`: passed.
552
-- `go test -race -count=1 ./pkg/snmputils`: passed.
553
-- `go test -race -count=1 ./collector/snmp/ddsnmp/...`: passed.
554
-- `go test -race -count=1 ./collector/snmp/...`: passed.
555
-- `go test -race -count=1 ./collector/snmp_topology/...`: passed.
556
-- `git diff --check`: passed.
557
-- Same-failure search over code/artifacts confirmed no code references to `KindSysUptime`, `IsTopologySysUptimeMetric`, `updateTopologyScalarMetric`, or `ingestTopologySysUptimeMetricSet` remain.
558
-- The planned pre-deletion side-by-side old/new runtime parity test was not added because the old classifier/filter path is now removed. Existing `FindProfiles`/catalog resolver tests, profile merge tests, Cisco Nexus fixture coverage, and focused collector suites are the retained parity proxy.
559
-
560
-Real-use evidence:
561
-
562
-- No live SNMP device was used. This SOW changes profile schema/loading/dispatch internals; validation used default profile loading, synthetic SNMP handler tests, and topology cache/collector unit tests.
563
-
564
-Reviewer findings:
565
-
566
-- Multiple read-only AI reviews were summarized and reconciled in `src/go/plugin/go.d/TODO-snmp-profile-loading-topology.md`.
567
-- Fourth readiness review recorded `READY TO PROCEED` after the `HiddenMetrics` constraint and implementation sub-decisions were added.
568
-- Fifth readiness review recorded `NEEDS ADJUSTMENTS`; accepted blocking adjustments B1-B6 and suggestions S1-S6 are reflected in this SOW before implementation starts.
569
-- Sixth readiness review recorded `NEEDS ADJUSTMENTS`; accepted blocker N1, tightenings N2-N5, and actionable minor notes are reflected in this SOW before implementation starts.
570
-- Seventh readiness review recorded `READY TO WRITE CODE`; non-blocking hygiene tightenings NB1-NB4 are reflected in this SOW.
571
-
572
-Same-failure scan:
573
-
574
-- `rg` found no stale classifier/filter/helper references for `IsTopologyMetric`, `LooksLikeTopologyIdentifier`, `MetricConfigContainsTopologyData`, `MetricTagConfigContainsTopologyData`, `MetadataFieldContainsTopologyData`, `MetadataContainsTopologyData`, `SysobjectIDMetadataContainsTopologyData`, `ProfileContainsTopologyData`, `ProfileHasCollectionData`, `selectTopologyRefreshProfiles`, `selectCollectionProfiles`, or legacy topology metric-name constants.
575
-- `_topology_` remains only in topology chart/tag names, build tags, and validation tests that intentionally reject underscore-prefixed topology row names.
576
-- `HiddenMetrics` remains only in the generic collector path and non-topology hidden-metric tests.
577
-- `LoadProfileByName` remains as an abstract-profile helper; runtime topology call sites no longer use it.
578
-
579
-Sensitive data gate:
580
-
581
-- This SOW contains only file paths, public OIDs/metric names, struct/function names, and design decisions. It contains no raw credentials, SNMP communities, bearer tokens, customer names, personal data, customer-identifying IPs, private endpoints, or proprietary incident details.
582
-
583
-Artifact maintenance gate:
584
-
585
-- AGENTS.md: no update needed; existing SOW/process and SNMP skill triggers were sufficient.
586
-- Runtime project skills: updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md` with top-level `topology:` and `TopologyKind` authoring rules.
587
-- Specs: created `.agents/sow/specs/snmp-profile-projection.md`.
588
-- End-user/operator docs: updated `collector/snmp/profile-format.md` with topology rows, consumers, topology kinds, and hidden-metric guidance.
589
-- End-user/operator skills: no public/operator skill update needed; this changes profile authoring guidance, which is covered by the runtime project skill and profile-format doc.
590
-- SOW lifecycle: completed and moved to `done/` during commit prep; status/directory are consistent.
591
-
592
-Specs update:
593
-
594
-- Created `.agents/sow/specs/snmp-profile-projection.md`.
595
-
596
-Project skills update:
597
-
598
-- Updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
599
-
600
-End-user/operator docs update:
601
-
602
-- Updated `collector/snmp/profile-format.md`.
603
-
604
-End-user/operator skills update:
605
-
606
-- No update required; no public skill artifact describes SNMP profile schema authoring.
607
-
608
-Lessons:
609
-
610
-- Normal catalog resolution intentionally excludes abstract `_std-*` profiles. Tests or programmatic checks that need a specific abstract mixin still need an explicit by-name loader; topology runtime should not use that path.
611
-- `FinalizeProfiles()` is still useful for synthetic collector tests that construct profiles outside the catalog. Treat it as programmatic preparation, not a topology runtime dependency.
612
-
613
-Follow-up mapping:
614
-
615
-- The probable `removeConstantMetrics()` value-copy bug in `collector/snmp/ddsnmp/profile.go` was not touched by this SOW because topology validation rejects `constant_value_one` rows and regular metric behavior did not require changing that path.
616
-- `LoadProfileByName` is intentionally retained as a programmatic abstract-profile loader for tests/checks such as `topology_profile_index_test.go`; topology runtime no longer calls it.
617
-- `FinalizeProfiles` is intentionally retained as a programmatic profile preparation helper for synthetic `ddsnmpcollector` tests that build profiles outside the catalog path; topology runtime no longer calls it.
618
-
619
-## Outcome
620
-
621
-Implementation is ready to commit.
622
-
623
-- Regular SNMP profile selection now uses the catalog resolver with metrics projection and fallback-only manual-profile semantics.
624
-- SNMP topology now uses topology projection, explicit `TopologyKind` row dispatch, `ProfileMetrics.TopologyMetrics`, and VLAN-context kind filtering instead of hardcoded topology profile filenames.
625
-- Topology profile YAML rows moved to top-level `topology:` with explicit `kind`; CDP regular metrics and topology rows are split.
626
-- `HiddenMetrics` remains a generic non-topology underscore-prefixed delivery path.
627
-- `KindSysUptime` was removed; topology queries uptime through `pkg/snmputils.GetSysUptime`, and regular SNMP uptime remains profile-driven in `_system-base.yaml`.
628
-- Docs, runtime SNMP profile authoring skill, and the SNMP projection spec were updated.
629
-
630
-## Lessons Extracted
631
-
632
-- Required topology `kind` makes "inherit base kind when derived omits kind" unreachable; durable design text must say explicit kind plus conflict rejection.
633
-- `LoadProfileByName` and `FinalizeProfiles` are no longer topology runtime dependencies, but they remain useful programmatic test/preparation helpers until those synthetic paths are redesigned.
634
-- A direct `snmputils` helper is cleaner for topology-only uptime enrichment than keeping a fake topology kind or retaining regular metrics in topology projection.
635
-- Readiness reviews can go stale quickly during active edits; final close-out review points must be checked against the current diff before accepting them.
636
-
637
-## Followup
638
-
639
-- Track the probable `removeConstantMetrics()` value-copy bug separately if regular metric behavior requires touching that path.
640
-- Consider a validation rule against duplicate `(table, symbol name)` topology collection rows with different kinds if real profiles ever create an ambiguous `TopologyKind` stamping case.
641
-- Consider strict YAML/schema validation for unknown keys in per-row `metric_tags` if author typo detection becomes important.
642
-- Consider a dedicated VLAN-context projection unit test beyond current transitive topology suite coverage.
643
-- Keep `LoadProfileByName` and `FinalizeProfiles` only as programmatic helpers; revisit cleanup if synthetic tests no longer need them.
644
-
645
-## Regression Log
646
-
647
-None yet.
648
-
649
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0013-20260507-snmp-licensing-projection.md
deleted
-920
@@ -1,920 +0,0 @@
1
-# SOW-0013 - SNMP licensing profile projection
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: implementation completed and final review findings closed on 2026-05-07; ready to move to `.agents/sow/done/`.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Replace the WIP SNMP licensing monitoring branch's hidden-metric delivery protocol with a clean typed licensing profile section and projection. Fix concrete profile correctness bugs while migrating to the typed schema, rather than patching the old hidden `_license_row*` protocol.
14
-
15
-### User Request
16
-
17
-User request summary:
18
-
19
-- Analyze the squashed `pr-ktsaou-licensing-monitoring` branch after rebase on `master`.
20
-- Find code smells, hacks, side effects, and profile correctness bugs.
21
-- Prefer a clean end state over low churn or backward compatibility because licensing monitoring is WIP/nightly.
22
-- Treat the current HiddenMetrics licensing path as another instance of the topology anti-pattern that was just removed.
23
-- Use typed projection/schema-driven delivery analogous to the SNMP topology profile projection.
24
-- Fix concrete correctness bugs as part of the projection migration, not as temporary patches to the hidden-metric protocol.
25
-
26
-Detailed design source:
27
-
28
-- `src/go/plugin/go.d/TODO-snmp-licensing-monitoring-review.md` (local-only development file; must not be committed).
29
-
30
-### Assistant Understanding
31
-
32
-Facts:
33
-
34
-- The original WIP branch implemented SNMP licensing by encoding semantic license rows as underscore-prefixed hidden metrics named `_license_row*`.
35
-- `ddsnmpcollector` generically moves underscore-prefixed metrics into `ProfileMetrics.HiddenMetrics`; this remains available for unrelated private metrics.
36
-- The original SNMP collector licensing code consumed `pm.HiddenMetrics`, recognized `_license_row*`, and dispatched by string tag `_license_value_kind`.
37
-- Recent SNMP topology work replaced the same class of metric-name/HiddenMetrics hack with top-level `topology:` profile rows and typed `ProfileMetrics.TopologyMetrics`.
38
-- `ProfileMetrics.HiddenMetrics` must remain a generic delivery container for unrelated underscore-prefixed/private metrics; licensing must stop using it as its semantic transport.
39
-- Downloaded local MIBs are review evidence only and must remain untracked:
40
- - `CISCO-SMART-LIC-MIB.my`
41
- - `CISCO-LICENSE-MGMT-MIB.mib`
42
- - `BLUECOAT-LICENSE-MIB.mib`
43
- - `CHECKPOINT-MIB.mib`
44
-- User decision: keep these raw MIB files at the repository root during implementation, do not commit them, and delete them after implementation is verified.
45
-
46
-Inferences:
47
-
48
-- The root problem is not just a few bad OIDs. The root problem is the absence of a typed licensing contract between SNMP profiles, `ddsnmpcollector`, and the SNMP collector's licensing aggregation.
49
-- Profile correctness bugs should not be used as parity targets for the new design. The typed profile migration should be authored from MIB truth.
50
-- The clean shape likely mirrors topology: top-level `licensing:` rows, closed enums, catalog projection, typed `ProfileMetrics` output, validation, and profile-format documentation.
51
-
52
-Unknowns:
53
-
54
-- No open design decisions remain. The only accepted follow-up is unsupported licensing table-root caching for broad Cisco coverage, tracked by `.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md`.
55
-
56
-### Acceptance Criteria
57
-
58
-- SNMP licensing profile data is represented by a first-class typed profile section/projection, not `_license_row*` hidden metrics.
59
-- `ProfileMetrics.HiddenMetrics` remains available and tested as a generic non-licensing underscore-prefixed metric delivery container.
60
-- SNMP licensing consumes typed licensing output from `ddsnmpcollector`, not magic metric names or `_license_value_kind` tags.
61
-- Licensing profile schema has closed validation for signal kinds and rejects malformed profile rows at load time.
62
-- Concrete profile correctness bugs are fixed in the migrated schema:
63
- - Cisco Smart scalar rows at the current `_cisco-base.yaml` lines 148, 172, 211, 229, and 247 use scalar instance suffix `.0`; Cisco Smart table rows are not treated as scalars.
64
- - Cisco traditional row identity does not merge distinct rows with the same feature name.
65
- - Check Point licensing OID mapping follows the `svnLicensing` table from the refreshed `CHECKPOINT-MIB.mib`.
66
- - Blue Coat derives `appLicenseStatusIndex` from the row index instead of reading the `not-accessible` object as `symbol.OID`.
67
- - Sophos and MikroTik ignored-state/sentinel behavior is no longer filename-gated.
68
- - Cisco licensing is represented by dedicated typed licensing mixins, not hidden `_license_row*` blocks in `_cisco-base.yaml`; broad Cisco coverage comes from `cisco.yaml` extending those mixins.
69
-- Runtime merge identity uses `OriginProfileID` plus table OID plus an INDEX-derived row key for table rows. `OriginProfileID` is the logical profile file that declared the licensing row, including mixin-origin rows after `extends:` merge. It must not use display table names, stripped filenames, root matched profile names, or absolute source paths as structural identity.
70
-- Eval/trial states do not generate warning/critical alerts by default; they may be exposed as informational function/chart state.
71
-- Licensing health alerts are scoped by their SNMP-specific `snmp.license.*` contexts; extra `chart labels: component=licensing` filters are intentionally not used.
72
-- Workstation-local provenance comments are sanitized before commit, including `bluecoat-proxysg.yaml:47` and path-bearing licensing fixtures under `ddsnmpcollector/testdata/licensing/`.
73
-- Provenance sanitation explicitly covers:
74
- - `config/go.d/snmp.profiles/default/bluecoat-proxysg.yaml:47`
75
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/checkpoint.snmprec:2`
76
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/cisco-smart-iosxe-c9800.snmprec:2`
77
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/cisco-traditional.snmpwalk:2`
78
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/mikrotik-router.snmprec:2-3`
79
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/sophos-xgs-firewall.snmprec:2`
80
-- The four raw MIB files are deleted from repo root before SOW close-out and are never staged or committed.
81
-- The local `TODO-snmp-licensing-monitoring-review.md` file remains out of the PR.
82
-- Tests cover full loaded profiles, not only sliced licensing blocks.
83
-- Profile-format documentation and project SNMP authoring skill describe the new licensing authoring contract.
84
-
85
-## Analysis
86
-
87
-Sources checked:
88
-
89
-- `src/go/plugin/go.d/TODO-snmp-licensing-monitoring-review.md`
90
-- `src/go/plugin/go.d/collector/snmp/licensing.go`
91
-- `src/go/plugin/go.d/collector/snmp/licensing_state.go`
92
-- `src/go/plugin/go.d/collector/snmp/licensing_vendor_sanity.go`
93
-- `src/go/plugin/go.d/collector/snmp/licensing_charts.go`
94
-- `src/go/plugin/go.d/collector/snmp/licensing_integration.go`
95
-- `src/go/plugin/go.d/collector/snmp/func_licenses.go`
96
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector.go`
97
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_scalar.go`
98
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_license_fixtures_test.go`
99
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/licensing_test_helpers_test.go`
100
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/*`
101
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_cisco-base.yaml`
102
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/checkpoint.yaml`
103
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/bluecoat-proxysg.yaml`
104
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/fortinet-fortigate.yaml`
105
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/mikrotik-router.yaml`
106
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/sophos-xgs-firewall.yaml`
107
-- `src/go/plugin/go.d/collector/snmp/profile-format.md`
108
-- `src/go/plugin/go.d/collector/snmp/metadata.yaml`
109
-- `src/health/health.d/snmp.conf`
110
-- `.agents/sow/specs/snmp-profile-projection.md`
111
-- `.agents/sow/specs/sensitive-data-discipline.md`
112
-- `.agents/skills/project-snmp-profiles-authoring/SKILL.md`
113
-- `.agents/skills/project-writing-collectors/SKILL.md`
114
-
115
-Current state:
116
-
117
-- Licensing rows are first-class `licensing:` profile data delivered through typed `ProfileMetrics.LicenseRows`.
118
-- License signal kinds, sentinel policies, state policies, symbol forbid-lists, row identity, and duplicate signals are validated at profile load.
119
-- The `_license_row*` / `_license_value_kind` hidden-metric protocol is removed from production profile YAML and runtime licensing consumption.
120
-- Cisco, Check Point, Blue Coat, Fortinet, MikroTik, and Sophos licensing profiles have been migrated to typed rows with MIB-derived corrections recorded in this SOW.
121
-- Full-profile and fixture-backed tests cover the migrated licensing profile families, while focused unit tests cover schema, projection, aggregation, function, and delivery edge cases.
122
-
123
-Risks:
124
-
125
-- Broad SNMP profile blast radius if migrated profiles break ordinary metric collection.
126
-- Function/UI regression if licensing aggregation output changes without matching docs/metadata/health updates.
127
-- Alert noise if eval/grace states remain mapped to degraded warning semantics.
128
-- Silent data loss if typed schema does not model scalar-only licensing rows and table licensing rows cleanly.
129
-- Sensitive data/provenance risk from committing local MIB files, workstation paths, or unsanitized SNMP fixtures.
130
-
131
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
132
-
133
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
134
-
135
-Problem / root-cause model:
136
-
137
-- The licensing feature repeats the old topology design flaw: it encodes semantic non-chart observations as hidden chart metrics plus magic names/tags. The transport is fragile, hard to validate, and easy to silently break. Concrete profile bugs show why the hidden protocol is the wrong parity target; the clean implementation needs a typed licensing schema and output projection built from MIB evidence.
138
-
139
-Evidence reviewed:
140
-
141
-- `collector/snmp/licensing.go` hidden `_license_row*` extraction and `_license_value_kind` dispatch.
142
-- `collector/snmp/ddsnmp/ddsnmpcollector/collector.go` generic underscore-prefix `HiddenMetrics` bucketing.
143
-- `collector/snmp/ddsnmp/metric.go` and `ddsnmpcollector/collector_topology.go` typed topology projection precedent.
144
-- `.agents/sow/specs/snmp-profile-projection.md` current metrics/topology profile projection contract.
145
-- `config/go.d/snmp.profiles/default/_cisco-base.yaml` Cisco Smart scalar object OIDs without `.0`.
146
-- Local `CISCO-SMART-LIC-MIB.my` proving Cisco Smart subtree is valid but scalar instances still need `.0`.
147
-- Local `CISCO-LICENSE-MGMT-MIB.mib` proving Cisco traditional table has three index components.
148
-- Local `CHECKPOINT-MIB.mib` proving `svnLicensing` object order and `licensingIndex MAX-ACCESS read-only`.
149
-- Local `BLUECOAT-LICENSE-MIB.mib` proving `appLicenseStatusIndex MAX-ACCESS not-accessible`.
150
-
151
-Affected contracts and surfaces:
152
-
153
-- SNMP profile schema and profile validation.
154
-- ddsnmp catalog projection consumers.
155
-- ddsnmpcollector `ProfileMetrics` output.
156
-- SNMP collector licensing aggregation, charts, function output, and alert inputs.
157
-- Default SNMP profile YAMLs for Cisco, Check Point, Blue Coat, Fortinet, MikroTik, Sophos.
158
-- SNMP profile-format documentation.
159
-- Runtime project SNMP profile authoring skill.
160
-- Integration metadata and health alerts if chart/function/alert semantics change.
161
-
162
-Existing patterns to reuse:
163
-
164
-- Top-level `topology:` typed profile section and `ProfileMetrics.TopologyMetrics`.
165
-- Closed enum validation from topology kind validation.
166
-- Catalog projection consumer model from `.agents/sow/specs/snmp-profile-projection.md`.
167
-- Existing scalar/table collection machinery from `ddsnmpcollector`.
168
-- Table/index accessibility rules in `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
169
-- Table-driven Go tests using `map[string]struct{}` where setup/assertion shapes match.
170
-
171
-Risk and blast radius:
172
-
173
-- Production regular SNMP metrics must not regress even though licensing is WIP/nightly.
174
-- Licensing itself can break backward compatibility because it is WIP/nightly, but the end state must be coherent and maintainable.
175
-- HiddenMetrics cannot be removed or redefined globally inside this SOW because it remains a generic private-metric mechanism with canary coverage; after licensing migration, grep currently shows no non-licensing production consumers.
176
-- Profile YAML migration can change SNMP walk/GET load on broad vendor base profiles.
177
-- Health templates and function output are user/operator-facing and require consistency with metadata/docs.
178
-
179
-Sensitive data handling plan:
180
-
181
-- Do not commit downloaded raw MIB files.
182
-- Do not commit this local TODO file.
183
-- Do not commit SNMP communities, SNMPv3 credentials, bearer tokens, customer hostnames, customer sysName/sysDescr, customer IPs, customer names, or personal data.
184
-- Replace workstation-local provenance paths with upstream repository/commit/relative-path evidence where public, or sanitized descriptions where not.
185
-- Any new fixtures must use sanitized public/vendor-derived values only.
186
-
187
-Implementation plan:
188
-
189
-- See `## Plan` below. That top-level plan is the canonical execution order and starts with validation/test scaffolding before schema/API work.
190
-
191
-Validation plan:
192
-
193
-- Strict fixture GET helper that fails on unexpected missing OIDs.
194
-- Duplicate-detecting license row/signal assertions.
195
-- Full-profile smoke tests for all licensing-bearing profile families.
196
-- Negative schema validation tests for invalid licensing signal kinds/fields.
197
-- Cisco Smart scalar `.0` collection test.
198
-- Check Point corrected object-mapping test.
199
-- Blue Coat index-derived identity test.
200
-- Sentinel/ignored-state tests independent of profile filename.
201
-- Sentinel parity test proving policies apply to both timestamp and remaining-value signals.
202
-- Sophos state test proving ignored raw-state hints beat severity `0` according to the selected state policy.
203
-- Sophos static `_license_id` uniqueness test covering the copy-pasted Sophos licensing blocks.
204
-- Chart registration matrix tests for partial signal availability.
205
-- Health alert tests if alerts remain in scope.
206
-- Cold-start function test proving the current 503-before-first-collect behavior is either preserved or deliberately changed by a recorded decision.
207
-- Cross-extends Cisco mixin dedup test.
208
-- `Project(licensing)` metric-tag propagation positive and negative tests.
209
-- Narrow Go test suites for `collector/snmp/ddsnmp/...` and `collector/snmp/...`.
210
-
211
-Artifact impact plan:
212
-
213
-- AGENTS.md: no expected update; existing process and collector rules already apply.
214
-- Runtime project skills: update `.agents/skills/project-snmp-profiles-authoring/SKILL.md` if licensing authoring rules are added.
215
-- Specs: update `.agents/sow/specs/snmp-profile-projection.md` or add a licensing-specific spec after implementation defines the durable contract. The spec update must capture the licensing consumer, projection rules, inheritance/merge rules, signal/policy enums, and local-only MIB evidence policy.
216
-- End-user/operator docs: update `collector/snmp/profile-format.md`, metadata/docs/health references if chart/function/alert semantics change.
217
-- End-user/operator skills: no expected update unless public Netdata AI skills reference SNMP licensing profile schema.
218
-- SOW lifecycle: decisions are resolved; move to current/in-progress only when implementation begins.
219
-
220
-Open-source reference evidence:
221
-
222
-- Existing branch artifacts cite workstation-local mirrored-OSS paths and must be sanitized before commit:
223
- - `config/go.d/snmp.profiles/default/bluecoat-proxysg.yaml:47`
224
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/checkpoint.snmprec:2`
225
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/cisco-smart-iosxe-c9800.snmprec:2`
226
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/cisco-traditional.snmpwalk:2`
227
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/mikrotik-router.snmprec:2-3`
228
- - `collector/snmp/ddsnmp/ddsnmpcollector/testdata/licensing/sophos-xgs-firewall.snmprec:2`
229
-- The four downloaded MIB files and the local TODO are currently untracked at the repository root / working directory and are not ignored. User decision: keep the MIB files at repo root during implementation, do not commit them, and delete them after implementation is verified. Keep this local TODO out of commits.
230
-
231
-Open decisions:
232
-
233
-- None. User selected: `2A 3A 4A 5C 6B 7B 8B 9B 10A 11A 12B 13A 14A 15A 17A 18A 19A 20A 21A 22A`, plus profile-origin identity option `1B`.
234
-- Post-slice review decision: regular SNMP uses an explicit combined metrics+licensing projection, not ad hoc reattachment and not a second collector pass. This keeps single-consumer projections pure while allowing the regular SNMP collector to collect both chart metrics and typed licensing rows in one `ddsnmpcollector` pass.
235
-- Follow-up projection API decision: use variadic `Project(consumer, consumers...)` for combined projections instead of a one-off `ProjectMetricsAndLicensing()` helper. Existing single-consumer callers stay unchanged, and regular SNMP calls `Project(metrics, licensing)`.
236
-
237
-## Cross-Cutting Resolution Rules
238
-
239
-These rules combine the resolved decisions above and prevent implementation drift.
240
-
241
-1. Licensing structural identity uses a real origin profile id, not a stripped filename and not the root matched profile. Add an `OriginProfileID` field to the resolved profile / `ProfileMetrics` / typed licensing row path and use it for licensing identity. The value identifies the logical profile file that declared the licensing row, including mixin-origin rows after `extends:` merge; it must not expose absolute workstation paths as user-facing license source. Typed `LicenseRow` carries the table OID directly from the licensing producer; do not add table OID to generic `ddsnmp.Metric` solely for licensing.
242
-2. Scalar identity defaults to `(origin-profile-id, scalar-symbol-OID)`. Profiles using explicit scalar grouping to aggregate multiple scalar OIDs into one row use `(origin-profile-id, licensing group id)` for that grouped row; grouped member signals must not also produce standalone scalar rows.
243
-3. Sentinel policies are evaluated at typed licensing projection emit time. A sentinel-rejected value sets the target signal's `Has` flag to false, so SNMP licensing aggregation never sees sentinel values.
244
-4. Bucket evaluation order is: ignored raw-state policy suppresses all other buckets; non-ignored hard failure conditions such as expired timers and exhausted usage produce broken; eval/trial-like states produce informational; grace/degraded states produce degraded; otherwise normal valid signals produce healthy or ignored according to the selected state policy.
245
-5. Top-level metric tags propagate to `Project(licensing)` when `consumers:` is unset, matching topology semantics. The licensing-row forbid-list applies only to licensing row value symbols, not to top-level tags.
246
-6. Repeated-signal load errors apply to duplicate `(structural identity, signal kind)` inside one resolved profile after inheritance merge. Table-row and scalar-row identities are different shapes and are never duplicates by accident, so a Cisco profile extending both Cisco licensing mixins is valid.
247
-7. `from: <oid>` sibling references are expanded by the typed licensing producer at collection time. For table rows, `from` must refer to a peer in the same SNMP table row; for scalar rows, `from` must refer to another scalar in the same profile or explicit scalar group. Cross-profile `from` is a validation error.
248
-8. Sophos-style sibling date migration uses this canonical shape:
249
-
250
- ```yaml
251
- licensing:
252
- - id: sophos-base-firewall
253
- identity:
254
- id: { value: base_firewall }
255
- name: { value: Base Firewall }
256
- state:
257
- from: 1.3.6.1.4.1...
258
- mapping: { 0: ignored, 1: healthy, 2: degraded, 3: broken }
259
- signals:
260
- expiry:
261
- from: 1.3.6.1.4.1...
262
- format: text_date
263
- sentinel: [timer_zero_or_negative]
264
- ```
265
-
266
-9. Regular SNMP profile setup uses `Project(metrics, licensing)`. `Project(metrics)` remains metrics-only, `Project(licensing)` remains licensing-only, and the variadic projection keeps the typed licensing producer reachable without a duplicate SNMP pass.
267
-10. Mixed projections use the public variadic `Project(consumer, consumers...)` API. This keeps the single-consumer API stable while avoiding one-off helpers for every valid consumer combination.
268
-
269
-## Implications And Decisions
270
-
271
-### Decision 1 - Licensing delivery contract
272
-
273
-Status: resolved by user.
274
-
275
-Selected option: A. First-class typed licensing profile section/projection.
276
-
277
-Rejected alternatives:
278
-
279
-- Keep `_license_row*` hidden metrics and add validation. Rejected because it preserves the same technical debt that topology just removed.
280
-- Model licenses as normal exported profile metrics. Rejected because license rows are semantic aggregation inputs, not chart metrics.
281
-
282
-### Decision 2 - YAML section key name
283
-
284
-Status: resolved by user: 2A.
285
-
286
-Options:
287
-
288
-- A. `licensing:`
289
- - Pros: matches singular top-level `topology:` precedent; describes the feature domain rather than individual row count.
290
- - Cons: less literal than `licenses:` for a list.
291
-- B. `licenses:`
292
- - Pros: literal list name.
293
- - Cons: diverges from `topology:` naming style and reads more like a user-facing entity list than a profile capability section.
294
-
295
-Recommendation: A. Use top-level `licensing:`.
296
-
297
-Selected option: A. Use top-level `licensing:`.
298
-
299
-### Decision 3 - YAML schema shape
300
-
301
-Status: resolved by user: 3A.
302
-
303
-Options:
304
-
305
-- A. Row-centric top-level `licensing:` blocks.
306
- - Each block declares table/scalar source, identity/descriptors/state, and typed signals.
307
- - Pros: mirrors how users think about one license row; keeps identity and signals together; suitable for table-based vendors.
308
- - Cons: scalar-only vendors need a row wrapper.
309
-- B. Signal-centric top-level `licensing:` blocks.
310
- - Each block declares one signal and separate grouping metadata.
311
- - Pros: close to the current hidden-metric implementation.
312
- - Cons: repeats row reconstruction complexity; easier to mis-group.
313
-- C. Separate top-level `licenses:` identity blocks plus `license_signals:` signal blocks.
314
- - Pros: highly explicit.
315
- - Cons: verbose, more merge/linkage validation, more authoring friction.
316
-
317
-Recommendation: A. Row-centric `licensing:` blocks are the cleanest end state and best match the topology top-level-section precedent. The schema must explicitly support scalar-only rows without forcing authors to invent a synthetic table wrapper.
318
-
319
-Selected option: A. Row-centric top-level `licensing:` blocks.
320
-
321
-### Decision 4 - Runtime typed output shape
322
-
323
-Status: resolved by user: 4A.
324
-
325
-Options:
326
-
327
-- A. `ProfileMetrics.LicenseRows []LicenseRow`, where each row has identity/descriptors plus typed grouped signal structs:
328
- - `State LicenseState`
329
- - `Expiry LicenseTimer`
330
- - `Authorization LicenseTimer`
331
- - `Certificate LicenseTimer`
332
- - `Grace LicenseTimer`
333
- - `Usage LicenseUsage`
334
- - each grouped struct carries an explicit `Has` boolean rather than using pointers.
335
-- B. `ProfileMetrics.LicenseRows []LicenseRow`, where each row has `Signals map[LicenseSignalKind]LicenseSignal`.
336
-- C. Separate typed slices such as `LicenseTimers`, `LicenseUsage`, `LicenseStates`.
337
-
338
-Recommendation: A. Licensing signals are heterogeneous. Timer, usage, and state values have different shapes, so a uniform `map[LicenseSignalKind]LicenseSignal` either becomes a tagged union or a wide optional-field struct. Keep `LicenseSignalKind` as the closed schema/validation enum, but make runtime rows compile-time clear with grouped structs. Use by-value grouped structs plus `Has` flags so `LicenseRows` can be shallow-cloned like the current cache without pointer aliasing surprises.
339
-
340
-Selected option: A. `ProfileMetrics.LicenseRows []LicenseRow` with by-value grouped signal structs plus `Has` flags.
341
-
342
-### Decision 5 - Runtime row identity and display grouping
343
-
344
-Status: resolved by user: 5C.
345
-
346
-Options:
347
-
348
-- A. Structural identity only: origin profile id + table OID + INDEX-derived row key for table rows; origin profile id + scalar OID/block id for scalar rows.
349
-- B. Semantic identity: vendor-provided license ID/name/feature only.
350
-- C. Hybrid: structural identity for collection/dedup, semantic identity for display/grouping.
351
-
352
-Recommendation: C. Structural identity prevents Cisco-style feature-name collisions; semantic identity remains useful for UI/function display. Structural identity must use table OID, not display table name, and must not use stripped filenames or the root matched profile as the source identity. Add a real `OriginProfileID` field to the resolved profile / `ProfileMetrics` / typed licensing row path for licensing structural identity. `OriginProfileID` is the logical profile file that declared the licensing row, including mixin-origin rows after `extends:` merge. Typed `LicenseRow` carries the table OID directly from the licensing producer; do not add table OID to generic `ddsnmp.Metric` solely for licensing.
353
-
354
-Selected option: C. Structural identity for collection/dedup; semantic identity for display/grouping.
355
-
356
-### Decision 6 - Numeric sentinel policy
357
-
358
-Status: resolved by user: 6B.
359
-
360
-Options:
361
-
362
-- A. Fully declarative YAML sentinel rules.
363
- - Pros: maximum flexibility.
364
- - Cons: turns profile YAML into a rule language and is hard to validate cleanly.
365
-- B. Closed built-in sentinel policy names referenced from YAML.
366
- - Initial set: `timer_zero_or_negative`, `timer_u32_max`, `timer_pre_1971`.
367
- - Pros: validates cleanly; removes filename gates; keeps YAML readable.
368
- - Cons: new sentinel patterns require code/schema updates.
369
-- C. Runtime vendor-specific code keyed by profile filename/source.
370
- - Pros: fastest to patch.
371
- - Cons: repeats current MikroTik filename-gated behavior.
372
-
373
-Recommendation: B. Use closed built-in sentinel policies referenced from typed signal config. Sentinel policies attach per signal field and apply to both absolute timestamp signals and remaining-value signals; do not repeat today's timestamp-only sentinel asymmetry.
374
-
375
-Selected option: B. Closed built-in sentinel policy names referenced from YAML.
376
-
377
-### Decision 7 - Raw-state classification policy
378
-
379
-Status: resolved by user: 7B.
380
-
381
-Options:
382
-
383
-- A. Fully declarative YAML raw-state match rules.
384
- - Pros: vendor-specific states can be modeled without code changes.
385
- - Cons: another profile rule language; easy to make matching inconsistent across vendors.
386
-- B. Closed built-in state policy names referenced from YAML, with profile-provided severity mappings allowed.
387
- - Pros: validates cleanly and centralizes bucket semantics.
388
- - Cons: new policy classes require code/schema updates.
389
-- C. Runtime vendor-specific code keyed by profile filename/source.
390
- - Pros: quick for one-off vendors.
391
- - Cons: repeats current filename/source gates.
392
-
393
-Recommendation: B. State classification is string/bucket policy, distinct from numeric sentinel handling. The initial rule must state that ignored raw-state hints win over severity `0` so Sophos `none` / `not_subscribed` rows do not become fake healthy licenses. Prefer modeling this in the typed YAML by not mapping ignored vendor states to severity `0`; if a profile still supplies both raw ignored state and severity `0`, runtime suppression must choose ignored.
394
-
395
-Selected option: B. Closed built-in state policy names referenced from YAML, with ignored raw-state hints winning over severity `0`.
396
-
397
-### Decision 8 - Chart lifecycle
398
-
399
-Status: resolved by user: 8B.
400
-
401
-Options:
402
-
403
-- A. Register fixed licensing chart set and rely on gaps.
404
-- B. Register charts lazily based on observed signal classes.
405
-- C. Fixed charts but split eval/grace/informational states away from degraded alerts.
406
-
407
-Recommendation: B. Lazy charts match the docs' conditional-language intent and avoid empty chart clutter. Lazy registration must guard per chart id, not by checking only the first licensing chart id.
408
-
409
-Selected option: B. Register charts lazily based on observed signal classes.
410
-
411
-### Decision 9 - Alert policy and health scoping
412
-
413
-Status: resolved by user: 9B.
414
-
415
-Options:
416
-
417
-- A. Keep current broad alert behavior.
418
- - Pros: no alert rewrite.
419
- - Cons: eval/trial/grace can alert too aggressively.
420
-- B. Explicit alert table scoped by SNMP-specific licensing contexts.
421
- - Policy: eval/trial -> informational only, no alert; grace -> WARN with delay; degraded -> WARN; broken/expired -> CRIT; usage pressure remains WARN/CRIT by percentage thresholds.
422
- - Health templates target `snmp.license.*` contexts directly; additional chart-label filters are redundant for these SNMP-specific contexts.
423
- - Pros: operator intent is explicit and avoids noisy eval/trial alerts.
424
- - Cons: requires health-template review and tests.
425
-- C. No default alerts for WIP licensing.
426
- - Pros: avoids false positives while the feature matures.
427
- - Cons: less useful out of the box.
428
-
429
-Recommendation: B. Licensing contexts are already SNMP-specific, so direct `on: snmp.license.*` health templates are sufficient. This decision depends on Decision 19 for the eval/trial/grace bucket model.
430
-
431
-Selected option: B. Explicit alert table scoped by SNMP-specific licensing contexts. User later confirmed that alert `chart labels: component=licensing` filters are redundant and should not be used.
432
-
433
-### Decision 10 - Catalog projection consumer
434
-
435
-Status: resolved by user: 10A.
436
-
437
-Options:
438
-
439
-- A. Add `ConsumerLicensing` and `Project(licensing)`.
440
- - Projection keeps `licensing:` rows, drops regular `metrics:`, drops `topology:`, and drops `virtual_metrics`.
441
- - Metadata and top-level tag behavior must be explicit for licensing.
442
- - Pros: matches topology projection design; prevents accidental metrics/topology leakage.
443
- - Cons: requires extending consumer validation and projection tests.
444
-- B. Reuse metrics projection and filter licensing later.
445
- - Pros: less projection work.
446
- - Cons: repeats hidden coupling and tag leakage risk.
447
-
448
-Recommendation: A. Add a first-class `licensing` consumer. `Project(licensing)` drops `metrics:`, `topology:`, and `virtual_metrics`. Metadata and top-level metric tags with no explicit `consumers:` should propagate to licensing only when they are device/profile identity; licensing-specific projection tests must cover positive and negative tag propagation.
449
-
450
-Selected option: A. Add `ConsumerLicensing` and `Project(licensing)`.
451
-
452
-### Decision 11 - Profile inheritance/extends merge
453
-
454
-Status: resolved by user: 11A.
455
-
456
-Options:
457
-
458
-- A. Merge licensing rows by structural identity and derived profile rows override inherited rows.
459
-- B. Append all licensing rows and deduplicate only at runtime.
460
-- C. Reject duplicate licensing structural identities at load time.
461
-
462
-Recommendation: A with load errors for conflicting incompatible definitions. Use table OID plus INDEX-derived row key for table rows; use scalar OID/block id for scalar rows. Cross-profile dedup runs after profile matching, mirroring topology projection semantics.
463
-
464
-Selected option: A. Merge by structural identity with derived override and load errors for incompatible conflicts.
465
-
466
-### Decision 12 - Cisco licensing inheritance scoping
467
-
468
-Status: resolved by user: 12B.
469
-
470
-Options:
471
-
472
-- A. Keep Cisco licensing in `_cisco-base.yaml`.
473
- - Pros: every Cisco profile gets licensing automatically.
474
- - Cons: broad walk/GET blast radius across many Cisco profiles, including devices that may not support the licensing MIBs.
475
-- B. Move Cisco licensing to explicit Cisco licensing mixin(s), and extend only profiles with evidence/fixtures.
476
- - Pros: clean scoping and lower SNMP cost; makes licensing support explicit.
477
- - Cons: requires deciding which Cisco profiles opt in.
478
-- C. Keep in base but add runtime/sysObjectID capability gates.
479
- - Pros: centralized.
480
- - Cons: profile system does not currently express per-row sysObjectID gates cleanly.
481
-
482
-Recommendation: B. Licensing is WIP and clean scoping matters more than low churn. Name the mixins `_cisco-licensing-traditional.yaml` and `_cisco-licensing-smart.yaml`; per-profile opt-in requires MIB evidence or fixture coverage.
483
-
484
-Selected option: B. Move Cisco licensing to explicit Cisco licensing mixin(s).
485
-
486
-### Decision 13 - `licenseDateFromTag` disposition
487
-
488
-Status: resolved by user: 13A.
489
-
490
-Options:
491
-
492
-- A. Delete `licenseDateFromTag` during migration; parse text dates through typed licensing signal config or existing `format: text_date` on fresh symbol values.
493
-- B. Generalize it as a non-licensing `text_date` transform helper.
494
-- C. Keep it for compatibility with old hidden licensing rows.
495
-
496
-Recommendation: A. It is licensing-domain logic inside generic transform machinery and should not survive the hidden-protocol removal. This requires Decision 18 so Sophos-style sibling-OID date values can be declared directly in the typed schema.
497
-
498
-Selected option: A. Delete `licenseDateFromTag` during migration.
499
-
500
-### Decision 14 - Function/chart/health unit contract
501
-
502
-Status: resolved by user: 14A.
503
-
504
-Options:
505
-
506
-- A. Keep separate units per surface and document them: chart metrics in seconds, Function duration fields in milliseconds, health alerts convert chart seconds to days for display.
507
-- B. Normalize every surface to seconds.
508
-- C. Normalize every surface to milliseconds.
509
-
510
-Recommendation: A. It matches current Netdata function duration conventions while keeping chart metrics simple for health calculations. Conversion ownership: charts store raw seconds, health divides chart seconds by 86400 for day display, and the Function multiplies durations by 1000 for millisecond `FieldTransformDuration` cells.
511
-
512
-Selected option: A. Keep per-surface units and document conversion ownership.
513
-
514
-### Decision 15 - `HiddenMetrics` post-migration status
515
-
516
-Status: resolved by user: 15A.
517
-
518
-Options:
519
-
520
-- A. Preserve `HiddenMetrics` as a generic underscore/private-metric mechanism and keep its non-licensing canary tests.
521
-- B. Remove `HiddenMetrics` entirely after licensing migration if no production non-licensing consumers remain.
522
-- C. Keep producer only, but document it as deprecated and unused.
523
-
524
-Recommendation: A for this SOW. Existing tests intentionally cover `_privateMetric` preservation. After licensing migrates, current grep evidence shows no non-licensing production consumers; the canary test is the reason to preserve the generic producer until a separate audit decides removal.
525
-
526
-Selected option: A. Preserve `HiddenMetrics` as a generic underscore/private-metric mechanism for this SOW.
527
-
528
-### Decision 16 - MIB/source evidence policy
529
-
530
-Status: resolved by user: 16A.
531
-
532
-Selected option: A. Keep raw MIBs local-only, cite only sanitized object names/OIDs in SOW/docs.
533
-
534
-Follow-through requirement: sanitize committed workstation-local provenance comments, including `bluecoat-proxysg.yaml:47`.
535
-
536
-### Decision 17 - Scalar row structural identity
537
-
538
-Status: resolved by user: 17A.
539
-
540
-Options:
541
-
542
-- A. Scalar identity is `(origin-profile-id, scalar-symbol-OID)`.
543
- - Pros: prevents unrelated scalar license blocks from collapsing when they share a semantic id; simple to validate.
544
- - Cons: scalar rows that intentionally compose multiple OIDs into one license need an explicit row/group id.
545
-- B. Scalar identity is `(origin-profile-id, semantic license id)`.
546
- - Pros: easy for Cisco Smart-style scalar groups.
547
- - Cons: repeats current collision risk across unrelated scalar MIB objects.
548
-- C. Scalar rows require an explicit structural `id:` in YAML.
549
- - Pros: author-controlled grouping.
550
- - Cons: easy to make unstable or semantic by accident.
551
-
552
-Recommendation: A, with an explicit grouping field only for scalar blocks that intentionally aggregate multiple scalar OIDs into one semantic license row. This depends on the `OriginProfileID` field selected by Decision 5; do not use stripped filenames or root matched profile names as scalar identity.
553
-
554
-Selected option: A. Scalar identity is `(origin-profile-id, scalar-symbol-OID)`, with explicit grouping for intentional multi-scalar rows.
555
-
556
-### Decision 18 - Sibling-OID signal decoding
557
-
558
-Status: resolved by user: 18A.
559
-
560
-Options:
561
-
562
-- A. Allow typed signals to declare `from: <oid>` plus optional `format:` so a signal can decode a sibling scalar/table OID directly.
563
- - Pros: removes `licenseDateFromTag`; supports Sophos-style sibling expiry values cleanly; keeps date parsing in typed licensing collection.
564
- - Cons: schema and collector need explicit source/reference validation.
565
-- B. Require every signal value to be the row anchor symbol value.
566
- - Pros: simpler collector.
567
- - Cons: blocks deleting `licenseDateFromTag` for current Sophos profiles or forces awkward profile duplication.
568
-- C. Keep a generic transform/helper path for sibling values.
569
- - Pros: lower schema work.
570
- - Cons: preserves the transform side channel this migration is meant to delete.
571
-
572
-Recommendation: A. Typed `from:` references are the clean schema replacement for Sophos-style sibling-OID decoding. The typed licensing producer expands `from` at collection time. For table rows, `from` must refer to a peer value in the same SNMP table row; for scalar rows, `from` must refer to another scalar in the same profile or explicit scalar group. Cross-profile `from` is a validation error.
573
-
574
-Selected option: A. Allow typed signals to declare `from: <oid>` plus optional `format:`.
575
-
576
-### Decision 19 - Eval/trial/grace bucket model
577
-
578
-Status: resolved by user: 19A.
579
-
580
-Options:
581
-
582
-- A. Add an informational bucket/dimension. Eval/trial/waiting/initialized states map to informational; grace remains degraded and alerts with delay.
583
- - Pros: preserves visibility without warning on normal eval/trial states; matches the selected alert policy.
584
- - Cons: changes chart dimensions and metadata/health docs.
585
-- B. Map eval/trial states to ignored.
586
- - Pros: no new bucket.
587
- - Cons: hides useful licensing state and overloads ignored semantics.
588
-- C. Keep eval/trial as degraded but suppress alerts by expression.
589
- - Pros: less aggregation change.
590
- - Cons: charts still look degraded and health logic becomes more complex.
591
-
592
-Recommendation: A. Add an informational bucket/dimension and keep actionable grace/degraded/broken separate. Move `evaluation`, `eval`, `trial`, `evaluation_subscription`, and `evaluation_period` out of degraded hints into a new informational hint set.
593
-
594
-Selected option: A. Add an informational bucket/dimension.
595
-
596
-### Decision 20 - Licensing-row validation forbid-list
597
-
598
-Status: resolved by user: 20A.
599
-
600
-Options:
601
-
602
-- A. Licensing row value symbols allow `format` and `mapping`, but reject chart/export fields and transforms: `chart_meta`, `metric_type`, `transform`, `scale_factor`, `constant_value_one`, and underscore-prefixed generated names.
603
- - Pros: keeps useful SNMP decoding (`format`, simple mappings) while blocking chart-only and side-channel behavior.
604
- - Cons: requires licensing-specific validation paths rather than reusing topology's forbid-list exactly.
605
-- B. Reuse topology's stricter forbid-list.
606
- - Pros: simpler validation.
607
- - Cons: incorrectly forbids `format` and `mapping`, which licensing needs for dates and state decoding.
608
-- C. Allow all `SymbolConfig` fields.
609
- - Pros: flexible.
610
- - Cons: repeats hidden protocol mistakes via transforms/chart-only fields.
611
-
612
-Recommendation: A.
613
-
614
-Selected option: A. Allow `format` and `mapping`; reject chart/export fields, transforms, scale/constant hacks, and underscore-generated names.
615
-
616
-### Decision 21 - Repeated-signal conflict semantics
617
-
618
-Status: resolved by user: 21A.
619
-
620
-Options:
621
-
622
-- A. Same structural identity + same signal kind is a load-time error unless an extending profile overrides the inherited definition through the merge rules.
623
- - Pros: prevents silent first-wins/last-wins data loss.
624
- - Cons: stricter authoring.
625
-- B. Derived wins and same-file duplicates are allowed last-wins.
626
- - Pros: flexible.
627
- - Cons: hides copy-paste mistakes.
628
-- C. First wins, matching current runtime behavior.
629
- - Pros: easiest migration from current code.
630
- - Cons: preserves today's silent duplicate loss.
631
-
632
-Recommendation: A.
633
-
634
-Selected option: A. Same structural identity plus same signal kind is a load-time error unless handled by valid extends override semantics.
635
-
636
-### Decision 22 - Function RequiredParams policy
637
-
638
-Status: resolved by user: 22A.
639
-
640
-Options:
641
-
642
-- A. Add explicit `RequiredParams` for `snmp:licenses`, matching the stricter Function interface style used by related functions.
643
- - Pros: clearer function contract and consistency with `interfaces`.
644
- - Cons: requires checking any caller assumptions.
645
-- B. Keep no required params.
646
- - Pros: preserves current Function shape.
647
- - Cons: leaves the contract looser than adjacent functions.
648
-
649
-Recommendation: A, unless caller review shows `snmp:licenses` is intentionally parameterless. Caller review found `snmp:licenses` is intentionally parameterless: it returns the complete per-device license table, and unlike `interfaces`, there is no natural required filter. Implementation should keep `RequiredParams` empty and add a short code comment plus a test asserting the method config is intentionally parameterless.
650
-
651
-Selected option: A with caller-review exception. `snmp:licenses` remains intentionally parameterless; document and test that contract.
652
-
653
-## Plan
654
-
655
-1. Move SOW to `.agents/sow/current/` and mark `Status: in-progress`.
656
-2. Add validation/test scaffolding first: strict GET helper, duplicate-detecting helpers, cold-start Function test, sentinel parity tests, projection tag-propagation tests, and full-profile smoke harness.
657
-3. Add schema/API types and validation for typed licensing.
658
-4. Add typed collector output and tests while preserving generic HiddenMetrics.
659
-5. Rewrite SNMP licensing aggregation to consume typed `ProfileMetrics.LicenseRows` while legacy hidden rows remain present.
660
-6. Re-author profiles from MIB truth and add full-profile tests.
661
-7. Delete hidden licensing protocol artifacts.
662
-8. Before final commit/close-out, delete the four raw MIB files from repo root and keep the local TODO out of the PR.
663
-9. Validate, run reviewer pass, update artifacts/specs/skills, and close SOW.
664
-
665
-## Execution Log
666
-
667
-### 2026-05-07
668
-
669
-- Created pending SOW from the completed licensing branch review.
670
-- Recorded accepted typed-projection direction and open implementation decisions.
671
-- No source code implementation started.
672
-- Folded in Claude's pending-decision review:
673
- - accepted row-centric YAML direction;
674
- - replaced runtime signal-map recommendation with typed grouped sub-struct recommendation;
675
- - split sentinel and raw-state policies;
676
- - added missing decisions for projection consumer, inheritance, Cisco scoping, transform helper disposition, units, health scoping, and HiddenMetrics status;
677
- - recorded MIB evidence policy as resolved 16A.
678
-- Folded in second Claude readiness review:
679
- - accepted all 16 decisions with tightenings;
680
- - added scalar row identity, sibling-OID decoding, eval/trial/grace bucket model, licensing validation forbid-list, repeated-signal conflict semantics, and Function RequiredParams decisions;
681
- - fixed decision numbering to use SOW decision ids consistently;
682
- - recorded workstation-path sanitation and local MIB/TODO hygiene requirements;
683
- - tightened Cisco Smart scalar `.0`, sentinel parity, per-chart lazy guards, SNMP-specific health context scoping, metric-tag projection, Cisco mixin names, unit conversion ownership, and HiddenMetrics preservation facts.
684
-- Recorded user decision bundle: `2A 3A 4A 5C 6B 7B 8B 9B 10A 11A 12B 13A 14A 15A 17A 18A 19A 20A 21A 22A`.
685
-- Recorded local MIB handling decision: keep raw MIBs at repo root during implementation, do not commit them, delete them after implementation is verified.
686
-- Recorded user decision `1B`: licensing structural identity uses `OriginProfileID`, the logical profile file that declared the licensing row, including mixin-origin rows after `extends:` merge.
687
-- Recorded post-slice review decision `combined metrics+licensing projection`: regular SNMP profile setup uses `Project(ConsumerMetrics, ConsumerLicensing)` so `licensing:` rows are delivered to `ddsnmpcollector` in the same pass as ordinary metrics; single-consumer projections remain pure.
688
-- Recorded user decision `projection API option A`: replace the one-off combined projection helper with variadic `Project(consumer, consumers...)`; regular SNMP will call `Project(ConsumerMetrics, ConsumerLicensing)`.
689
-- Activated this SOW by moving it from `.agents/sow/pending/` to `.agents/sow/current/` and setting `Status: in-progress`.
690
-- Phase 1 scaffolding started:
691
- - replaced the lenient fixture GET helper with `mustExpectSNMPGetFromFixture`, which fails immediately when a test asks for an OID absent from the fixture;
692
- - added duplicate-detecting license grouping helpers for `(license id, signal kind)` and Sophos state/expiry fixture grouping, so tests fail on silent overwrite instead of hiding duplicates;
693
- - kept the Cisco Smart entitlement-only fixture scoped to the entitlement table because the public fixture explicitly lacks Smart Licensing scalar registration/auth/certificate timers.
694
-- Validation note: strict fixture coverage exposed Cisco Smart scalar fixture coverage gaps. A later reviewer also found the broad ddsnmpcollector `Version()` failures were caused by licensing test helper leakage of inherited topology rows, not pre-existing branch behavior; the helper must clear `Topology` and `Licensing` for legacy hidden-protocol tests.
695
-- Phase 2 schema/API skeleton started:
696
- - added top-level `licensing:` profile definition storage with clone support;
697
- - added `ConsumerLicensing` to the closed consumer enum and projection path;
698
- - added closed licensing signal kind, sentinel policy, and state policy enums;
699
- - added typed runtime `ProfileMetrics.LicenseRows []LicenseRow` with grouped state/timer/usage structs;
700
- - added licensing validation for invalid policy names, invalid signal kinds, and forbidden licensing value-symbol fields while allowing `format` and `mapping`.
701
-- Added projection coverage proving `Project(licensing)` keeps `licensing:` rows and licensing-selected metadata/metric tags while dropping metrics, topology rows, and virtual metrics.
702
-- Added `OriginProfileID` propagation:
703
- - profile loading assigns a relative logical origin path to each directly declared `licensing:` row;
704
- - `extends:` merge appends base licensing rows while preserving their original declaring profile id;
705
- - added a merge test proving a row declared in `device.yaml` keeps `device.yaml` and an inherited row declared in `_licensing.yaml` keeps `_licensing.yaml`.
706
-- Phase 4 typed collector output started:
707
- - added a typed licensing producer that emits `ProfileMetrics.LicenseRows` from top-level `licensing:` rows;
708
- - scalar licensing rows use SNMP GETs and table licensing rows use SNMP walks while preserving `HiddenMetrics` as a separate generic underscore-prefixed metric container;
709
- - table licensing rows reuse the existing table structure cache path after the first walk, so the typed projection does not introduce a permanent per-cycle table walk;
710
- - table licensing rows carry `OriginProfileID`, table OID, raw row index, structural id, identity/descriptors, state, timer, usage, static tags, and row metric tags;
711
- - scalar licensing rows support explicit group ids, literal values, sibling `from:` OIDs, text-date formatting, mappings, and sentinel filtering at emit time.
712
-- Folded in post-slice readiness review fixes before starting the consumer rewrite:
713
- - wired regular SNMP setup to `Project(ConsumerMetrics, ConsumerLicensing)` so typed `LicenseRows` are reachable in production without a second SNMP pass;
714
- - fixed the legacy licensing test helper to clear inherited `Topology` and `Licensing` rows, which removed the extra topology `Version()` calls in Cisco/MikroTik hidden-protocol tests;
715
- - extended licensing validation for repeated `(structural identity, signal kind)` rows, table `from:` scope, scalar multi-signal grouping, descriptor-only rows, scalar index misuse, underscore-prefixed legacy names, and closed value formats;
716
- - applied sentinel filtering to usage signals too, not only timer signals;
717
- - kept licensing table walk failures from poisoning the generic metric `missingOIDs` cache;
718
- - added `Stats.Metrics.Licensing` so typed licensing rows are counted separately from ordinary metric rows.
719
-- Replaced the temporary combined-projection helper shape with the resolved variadic `Project(consumer, consumers...)` API; regular SNMP now calls `Project(ConsumerMetrics, ConsumerLicensing)`, while topology and licensing-only callers continue using single-consumer projections.
720
-- Folded in the follow-up Claude implementation-slice review before starting the consumer rewrite:
721
- - accepted and fixed `mergeLicensing` override semantics by making derived `licensing:` rows replace inherited rows with the same pre-collection merge identity;
722
- - accepted and fixed the shared `missingOIDs` poisoning risk by removing the empty-walk table-missing cache write; explicit no-such GET responses still mark OIDs missing;
723
- - accepted and fixed licensing cross-table tag wiring by building a shared licensing table-name/OID map, walking cross-table dependencies before row processing, and caching dependency metadata;
724
- - accepted and fixed the D20 forbid-list gap by rejecting `extract_value`, `match_pattern`, and `match_value` on licensing row value symbols;
725
- - accepted and fixed state-policy-without-source validation, timer timestamp/remaining ambiguity, usage sentinel parity, cache-config namespace collisions, and duplicated structural-identity helper logic;
726
- - accepted and fixed licensing observability separation by adding `Stats.Timing.Licensing` and `Stats.Errors.Processing.Licensing`;
727
- - accepted the stats accounting direction: `Stats.Metrics.Licensing` counts typed license rows, while `Stats.Metrics.Tables` and `Stats.Metrics.Rows` remain ordinary chart-metric table counters;
728
- - rejected treating scalar `from:` acceptance as a blocker: scalar rows are the scalar group scope, table `from:` has the strict same-table gate, and cross-profile `from:` has no schema path.
729
-- Started the consumer rewrite slice:
730
- - replace `collector/snmp/licensing.go` hidden `_license_row*` / `_license_value_kind` extraction with typed `ProfileMetrics.LicenseRows`;
731
- - keep hidden-profile fixture tests only as migration reference, not as the target consumer contract;
732
- - update aggregation/state/function tests around typed rows, including ignored/informational/degraded/broken bucket behavior;
733
- - migrate vendor profile families only after the typed consumer is working.
734
-- Recorded Cisco licensing scope refinement:
735
- - Cisco licensing remains in dedicated licensing mixin/profile files, not `_cisco-base.yaml`;
736
- - current WIP Cisco licensing has two table OIDs (`clmgmtLicenseInfoTable` and `ciscoSlaEntitlementInfoTable`);
737
- - unsupported scalar GET OIDs are cached as exact missing OIDs after clean no-data responses;
738
- - unsupported table-root walks that return explicit `NoSuchObject` / `NoSuchInstance` should be distinguished from empty tables with zero rows;
739
- - user selected broad Cisco coverage: `cisco.yaml` should extend the dedicated Cisco traditional and smart licensing mixins;
740
- - broad Cisco licensing coverage is acceptable with a tracked follow-up for carefully scoped explicit table-root no-such caching so unsupported licensing table probes do not remain permanent per-poll cost.
741
-- Migrated Cisco licensing profiles:
742
- - added `_cisco-licensing-traditional.yaml` and `_cisco-licensing-smart.yaml`;
743
- - removed hidden Cisco `_license_row*` and `_license_value_kind` profile blocks from `_cisco-base.yaml`;
744
- - made `cisco.yaml` extend both dedicated licensing mixins;
745
- - Cisco Smart scalar OIDs now use scalar instance suffix `.0`; the Smart entitlement OID remains a table;
746
- - Cisco traditional typed rows use the three-component MIB index as the semantic row id, while runtime structural identity remains origin profile + table OID + row key;
747
- - treated zero-length SNMP `DateAndTime` values as no-value for licensing timers, matching the Cisco MIB's empty-octet-string not-applicable behavior.
748
-- Cisco migration validation passed:
749
- - `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_Cisco|TestCollector_Collect_.*LicensingProfile_Fixture'`;
750
- - `go test -count=1 ./collector/snmp/ddsnmp/...`;
751
- - `go test -count=1 ./collector/snmp/...`;
752
- - `rg '_license_row|_license_value_kind|licenseDateFromTag' config/go.d/snmp.profiles/default` returned no matches.
753
-- Folded in the final pre-PR review fix batch:
754
- - replaced all-at-once licensing chart registration with per-signal-class lazy registration and per-chart idempotent guards;
755
- - kept SNMP licensing health alerts scoped by their `snmp.license.*` contexts and added the agreed grace-period warning delay;
756
- - verified Fortinet licensing table OIDs against Fortinet's FortiGate system MIB documentation for `fgLicContractTable`, `fgLicVersionTable`, and `fgLicAlContractTable`, and recorded the official documentation URL in the Fortinet profile;
757
- - corrected Cisco Smart entitlement `invalidTag(11)` from healthy to broken based on the local `CISCO-SMART-LIC-MIB.my` definition;
758
- - removed the dead MikroTik filename/source-gated consumer sanity path because `timer_pre_1971` now drops the sentinel at typed-producer time;
759
- - documented licensing stats separation, documented and tested the intentionally parameterless `snmp:licenses` function contract, hardened scalar literal-only licensing row validation, expanded licensing forbid-list tests, and consolidated licensing fixture tests into a map-driven harness;
760
- - aligned `.agents/sow/specs/snmp-profile-projection.md` with implemented scalar `from:` validation and metadata `id_tags` projection behavior.
761
-- Updated durable project artifacts for the typed projection slice:
762
- - extended `.agents/sow/specs/snmp-profile-projection.md` with the licensing consumer, projection rules, typed delivery, identity rules, and validation guarantees;
763
- - extended `.agents/skills/project-snmp-profiles-authoring/SKILL.md` with licensing authoring guardrails and the `ProfileMetrics.LicenseRows` delivery rule.
764
-
765
-## Validation
766
-
767
-Acceptance criteria evidence:
768
-
769
-- Typed schema/projection:
770
- - `collector/snmp/ddsnmp/ddprofiledefinition/licensing.go` defines typed licensing config, closed signal kinds, sentinel policies, state policies, clone methods, and source fields.
771
- - `collector/snmp/ddsnmp/metric.go` defines `ProfileMetrics.LicenseRows` and typed `LicenseRow`, `LicenseState`, `LicenseTimer`, and `LicenseUsage`.
772
- - `collector/snmp/profile_sets.go` uses `Project(ConsumerMetrics, ConsumerLicensing)` so regular SNMP collection gets metrics and licensing in one ddsnmpcollector pass.
773
-- Hidden protocol removal:
774
- - `rg '_license_row|_license_value_kind|licenseDateFromTag' config/go.d/snmp.profiles/default` returned no matches during implementation.
775
- - Runtime licensing consumes `pm.LicenseRows`; `HiddenMetrics` remains only as generic underscore/private metric delivery.
776
-- Profile correctness:
777
- - Cisco licensing lives in `_cisco-licensing-traditional.yaml` and `_cisco-licensing-smart.yaml`, with `cisco.yaml` broadly extending both by user decision.
778
- - Cisco Smart scalar OIDs use scalar instance `.0`; the entitlement table remains modeled as a table.
779
- - Cisco traditional identity derives the three-component MIB index instead of feature-name-only identity.
780
- - Blue Coat derives `appLicenseStatusIndex` from the row index.
781
- - Check Point licensing follows the refreshed `svnLicensing` table.
782
- - MikroTik pre-1971 sentinel filtering happens at typed producer time, not by filename.
783
-- Final review fixes:
784
- - Licensing errors are best-effort relative to regular scalar/table metrics.
785
- - Scalar licensing OIDs honor the shared exact missing-OID cache before future GETs.
786
- - Workstation-local fixture provenance paths were removed.
787
- - Public metadata wording no longer uses PR-specific branch coverage phrasing.
788
- - Global integration template edits were reverted to avoid a repo-wide generated-doc inconsistency.
789
-
790
-Tests or equivalent validation:
791
-
792
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestParseSNMPWalkLine_IntegerEnumValue|TestCollector_Collect_CheckPointLicensingProfile_CommunitySample|TestCollector_Collect_SophosLicensingProfile_Fixture$'` passed.
793
-- `go test -count=1 ./collector/snmp -run 'TestFuncLicensesHandleUnavailable|TestFuncLicensesHandleUnavailableWhenNoRowsWereCollected|TestFuncLicensesHandle$'` passed, covering the existing cold-start `503` behavior.
794
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition` passed.
795
-- `go test -count=1 ./collector/snmp/ddsnmp` passed.
796
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'TestProfile_MergeLicensingPreservesOriginProfileID|TestProfileDefinition|TestResolvedProfileSet_Project'` passed.
797
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'TestResolvedProfileSetProject_SeparatesMetricsAndTopology|TestResolvedProfileSetProject_DoesNotShareMutableProjectionState|TestProjectedViewFilterByKind'` passed.
798
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_LicenseRows'` passed, covering scalar and table typed licensing row emission plus table-cache reuse.
799
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_PreservesHiddenMetrics|TestCollector_Collect_SeparatesTopologyMetricsFromHiddenMetrics|TestCollector_Collect_LicenseRows'` passed, covering hidden/topology/licensing delivery separation.
800
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_LicenseRows|TestCollector_Collect_StatsSnapshot'` passed, covering typed licensing rows and the normal non-licensing stats path.
801
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run TestParseSNMPWalkLine_IntegerEnumValue` passed, compiling `ddsnmpcollector` after shared `ProfileMetrics` changes.
802
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_(Cisco|MikroTik)LicensingProfile'` passed after clearing inherited topology rows from the hidden-protocol licensing profile helper.
803
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_LicenseRows|TestCollector_Collect_PreservesHiddenMetrics|TestCollector_Collect_SeparatesTopologyMetricsFromHiddenMetrics|TestCollector_Collect_StatsSnapshot'` passed after the combined projection, usage-sentinel, and licensing-stats fixes.
804
-- `go test -count=1 ./collector/snmp/ddsnmp` passed after replacing the one-off combined projection helper with variadic `Project(consumer, consumers...)`.
805
-- `go test -count=1 ./collector/snmp/ddsnmp/... ./collector/snmp/... ./collector/snmp_topology/...` passed after replacing the one-off combined projection helper with variadic `Project(consumer, consumers...)`.
806
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition` passed after adding the licensing forbid-list, state-source, and timer-source validation gates.
807
-- `go test -count=1 ./collector/snmp/ddsnmp` passed after the variadic projection and licensing merge-override fixes.
808
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector` passed after the cross-table licensing dependency walk, missing-OID, cache-namespace, sentinel-matrix, and licensing stat fixes.
809
-- `go test -count=1 ./collector/snmp/ddsnmp/... ./collector/snmp/... ./collector/snmp_topology/...` passed after the full post-review fix batch.
810
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector` passed.
811
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed.
812
-- `go test -count=1 ./collector/snmp/...` passed.
813
-- `go test -count=1 ./collector/snmp_topology/...` passed, covering shared profile projection changes against the topology consumer.
814
-- `go test -count=1 ./collector/snmp` passed after replacing the licensing consumer's hidden-metric extraction with typed `ProfileMetrics.LicenseRows`, adding the `informational` bucket, and rewriting SNMP collector aggregation/function tests around typed rows.
815
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector` passed after migrating MikroTik, Check Point, Blue Coat, and Fortinet licensing profiles from hidden `_license_row` metrics to typed `licensing:` rows.
816
-- `go test -count=1 ./collector/snmp/ddsnmp/... ./collector/snmp` passed after the typed consumer rewrite, typed profile migrations, and licensing documentation updates in this slice.
817
-- `go test -count=1 ./collector/snmp/... ./collector/snmp_topology/...` passed after the typed consumer rewrite, migrated profile subset, and documentation/health updates.
818
-- `go test -count=1 ./collector/snmp -run 'TestCollector_AddLicenseCharts|TestLicensesMethodConfig|TestFuncLicenses|TestAggregateLicenseRows|TestNormalizeLicenseStateBucket|TestExtractLicenseRows'` passed after lazy chart registration, parameterless function contract, and typed consumer cleanup.
819
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestValidateEnrichProfile_Licensing|TestCollector_Collect_LicensingProfileFixtures|TestCollector_Collect_LicensingProfiles|TestCollector_Collect_Sophos|TestCollector_Collect_MikroTik|TestCollector_Collect_Cisco'` passed after licensing validation hardening and fixture test consolidation.
820
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed after the final pre-PR review fix batch.
821
-- `go test -count=1 ./collector/snmp/...` passed after the final pre-PR review fix batch.
822
-- `go test -count=1 ./collector/snmp_topology/...` passed after the shared projection/spec cleanup.
823
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_LicenseRowsBestEffortForRegularMetrics|TestCollector_Collect_LicenseRowsSkipsKnownMissingScalarOIDs'` passed after the final review runtime fixes.
824
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed after the final review runtime and artifact fixes.
825
-- `go test -count=1 ./collector/snmp/... ./collector/snmp_topology/...` passed after the final review runtime and artifact fixes.
826
-- `git diff --check master...HEAD` passed during final review.
827
-
828
-Real-use evidence:
829
-
830
-- No live SNMP device validation was run in this SOW. Licensing profile behavior is validated through typed collector mocks, public fixture-derived SNMP data, full-profile loading tests, and MIB-derived OID checks. This is acceptable for the WIP/nightly feature because the target devices are not locally available and all changed runtime surfaces are covered by narrow Go suites.
831
-
832
-Reviewer findings:
833
-
834
-- Latest Claude implementation-slice review disposition:
835
- - accepted/fixed P0 merge override, missing-OID poisoning, and licensing cross-table dependency wiring before consumer rewrite;
836
- - accepted/fixed P1 validation/runtime gaps for forbidden licensing symbol transforms, state policy source, timer source ambiguity, cache namespace, licensing timing/error counters, and structural helper reuse;
837
- - accepted/documented separate stats semantics for typed license rows versus ordinary chart-metric table rows;
838
- - rejected scalar `from:` as an additional blocker because scalar `from:` is scoped to the scalar licensing row/group by construction, while table `from:` remains explicitly same-table validated.
839
-- Final GPT-5.5 review disposition:
840
- - accepted/fixed licensing errors dropping regular metrics by making typed licensing best-effort in `collectProfile`;
841
- - accepted/fixed scalar licensing missing-OID cache bypass by filtering known missing scalar licensing OIDs before future GETs;
842
- - accepted/fixed fixture workstation-path leakage by sanitizing licensing fixture headers;
843
- - accepted/fixed stale public docs wording in `metadata.yaml`;
844
- - accepted/fixed projection spec example severity mappings to use runtime-valid `"0"`, `"1"`, `"2"` values;
845
- - accepted/fixed generated-doc/template inconsistency by reverting unrelated global integration template edits;
846
- - accepted/tracked Cisco unsupported table-root no-such caching as `.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md`.
847
-
848
-Same-failure scan:
849
-
850
-- Same-failure search for workstation-local provenance paths and PR-specific branch coverage phrasing returned no matches in metadata, licensing fixtures, projection specs, and integration templates.
851
-- `rg -n "_license_row|_license_value_kind|licenseDateFromTag|tagLicense|licenseValueKind|licenseSourceMetricName|mergeLicenseSignal|mergeLicenseTags|licenseRowMergeKey" collector/snmp config/go.d/snmp.profiles/default` shows only intentional validation-test literals and generic `HiddenMetrics` canary references, not production licensing consumption.
852
-- `git status --short` was checked; raw MIB files were removed from repo root and the local TODO remained untracked.
853
-
854
-Sensitive data gate:
855
-
856
-- Durable artifacts contain no raw SNMP communities, SNMPv3 credentials, bearer tokens, customer names, customer hostnames, customer IPs, or raw MIB content.
857
-- Workstation-local fixture provenance comments were sanitized.
858
-- The four downloaded raw MIB files were deleted from repo root before close-out.
859
-- `src/go/plugin/go.d/TODO-snmp-licensing-monitoring-review.md` remains local/untracked and must not be staged.
860
-
861
-Artifact maintenance gate:
862
-
863
-- AGENTS.md: no update needed; existing SOW, collector, sensitive-data, and follow-up discipline rules already covered this work.
864
-- Runtime project skills: updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md` with typed licensing authoring rules and table-driven test preference.
865
-- Specs: updated `.agents/sow/specs/snmp-profile-projection.md` with licensing consumer, projection, typed delivery, identity, validation, and metadata tag behavior.
866
-- End-user/operator docs: updated `collector/snmp/profile-format.md`, `collector/snmp/metadata.yaml`, `collector/snmp/integrations/snmp_devices.md`, and `src/health/health.d/snmp.conf` as the SNMP licensing feature became user-visible.
867
-- End-user/operator skills: no update needed; no public Netdata AI skill currently documents SNMP profile licensing authoring or SNMP licensing runtime use.
868
-- SOW lifecycle: SOW-0013 status set to `completed`; file will move to `.agents/sow/done/` with the implementation and follow-up SOW in the same commit.
869
-
870
-Specs update:
871
-
872
-- Updated `.agents/sow/specs/snmp-profile-projection.md`.
873
-
874
-Project skills update:
875
-
876
-- Updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
877
-
878
-End-user/operator docs update:
879
-
880
-- Updated:
881
- - `collector/snmp/profile-format.md`
882
- - `collector/snmp/metadata.yaml`
883
- - `collector/snmp/integrations/snmp_devices.md`
884
- - `src/health/health.d/snmp.conf`
885
-
886
-End-user/operator skills update:
887
-
888
-- No update needed; no end-user/operator skill exposes this SNMP licensing schema or function workflow.
889
-
890
-Lessons:
891
-
892
-- Hidden-metric side channels become hard to validate as soon as one logical row is reconstructed from several scalar/table fragments. Topology and licensing now share the cleaner pattern: schema-owned typed sections plus typed `ProfileMetrics` outputs.
893
-- Optional feature telemetry must be best-effort relative to ordinary device metrics; otherwise a WIP optional section can regress established collection.
894
-
895
-Follow-up mapping:
896
-
897
-- Implemented in this SOW:
898
- - typed licensing projection and consumer rewrite;
899
- - profile migrations and MIB-derived corrections;
900
- - hidden-protocol removal from production licensing;
901
- - validation/test/docs/spec/skill updates;
902
- - final review runtime and artifact fixes.
903
-- Tracked as follow-up:
904
- - `.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md` tracks explicit unsupported licensing table-root no-such caching for broad Cisco coverage.
905
-
906
-## Outcome
907
-
908
-Completed. SNMP licensing is now represented by top-level typed `licensing:` profile rows and delivered through `ProfileMetrics.LicenseRows`; the SNMP collector consumes typed rows directly for charts, health inputs, and the `snmp:licenses` function. The old hidden `_license_row*` / `_license_value_kind` protocol is removed from production licensing code and profile YAML.
909
-
910
-## Lessons Extracted
911
-
912
-Typed profile projections are the right boundary for non-chart SNMP observations. The collector can still reuse scalar/table collection internals, but row identity, validation, and consumer contracts need to live in schema-owned typed fields, not hidden metric names or string tag protocols.
913
-
914
-## Followup
915
-
916
-- `.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md`
917
-
918
-## Regression Log
919
-
920
-None yet.
.agents/sow/done/SOW-0014-20260506-netflow-sflow-ipfix-documentation-guide.md
deleted
-3208
@@ -1,3208 +0,0 @@
1
-# SOW-0014 - Network Flows Documentation & Integration Infrastructure
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Reopened 2026-05-08 after a user report found at least one
8
-`metadata.yaml` Learn URL that no longer matches the published Learn route:
9
-`https://learn.netdata.cloud/docs/network-flows/visualization/summary-sankey`
10
-should point to the current Sankey/table page. This regression covers auditing
11
-all `metadata.yaml` Learn links, repairing source metadata, regenerating
12
-integration artifacts, and recording durable validation.
13
-The regression was repaired and revalidated on 2026-05-08; see
14
-`## Regression - 2026-05-08 - Metadata Learn Links`.
15
-
16
-Reopened 2026-05-07 after the netlify deploy preview for learn PR #2852 surfaced major content errors that the prior validation pass missed. The closure on 2026-05-07 (Status: completed) was premature: the docs contained multiple statements that contradicted the source code, generic flow-monitoring advice imported from research notes that did not apply to Netdata, and several invented behaviours. The regression was repaired and revalidated by 2026-05-08; see the `## Regression - 2026-05-07` section and closeout notes at the end of this file.
17
-
18
-Reopened 2026-05-08 after PR #22449 review and CI reported additional issues after the SOW had been marked completed and moved to `done/`. The open items are tracked in `## Regression - 2026-05-08` and include automated review threads, `yamllint`, `check-documentation`, Codacy triage, a code-only review subagent requested by the user, and a new user-requested local Learn preview skill/workflow.
19
-
20
-Reopened 2026-05-08 after the merged integration artifacts broke downstream
21
-publishing contracts for the website and in-app integrations. The flow
22
-integration schema delegates to collector metadata, but the flow renderer did
23
-not render the collector-style `metrics` and `alerts` sections, leaving raw
24
-objects/arrays in `integrations.json` and `integrations.js`.
25
-The regression was repaired and revalidated on 2026-05-08; see
26
-`## Regression - 2026-05-08 - Flow Integration Section Rendering`.
27
-
28
-## Requirements
29
-
30
-### Purpose
31
-
32
-Provide DevOps/SREs a complete, authoritative documentation and integration infrastructure for Netdata's network flow analysis. Documentation covers the Network Flows feature set that exists in this PR and excludes non-existent behavior such as topology drilldown. The documentation must include sizing, capacity planning, and optimization guidance so businesses can make informed deployment decisions.
33
-
34
-### User Request
35
-
36
-Phase 1: Document the Network Flows feature set, including sizing/benchmarking/capacity planning and the enrichment integrations exposed through `metadata.yaml`.
37
-Phase 2 (follow-up SOW): Add or revise documentation only for behavior that is not present in this PR, such as a future topology drilldown if it is implemented.
38
-
39
-### Assistant Understanding
40
-
41
-Facts:
42
-
43
-- **Tested features** (unit tests + benchmarks exist):
44
- - Core collection: NetFlow v5/v7/v9, IPFIX, sFlow decoding and ingestion
45
- - Basic enrichment: GeoIP (MMDB), static metadata (exporter/interface naming), sampling overrides, static networks, ASN provider chains
46
- - Classifiers: Akvorado-compatible exporter + interface classification rules (30+ unit tests)
47
- - Decapsulation: SRv6, VXLAN modes
48
- - Journal tiering: raw/1m/5m/1h with retention and query guardrails
49
- - Query engine: flows/autocomplete modes, all views, group_by, selections, facets
50
- - Frontend: 6-tab visualization (Sankey, Timeseries, Country/State/City Maps, Globe), dashboard cards, filters/facets
51
-
52
-- **Untested features** (unit tests for parsing exist, but no integration/e2e tests, never validated with real data):
53
- - **BMP listener**: TCP listener accepting BMP from real routers, populating routing trie from live BGP sessions. 11 unit tests for message parsing, but TCP listener never tested with a real BMP speaker. Performance impact on netflow ingest path unknown.
54
- - **BioRIS**: gRPC client for RIPE RIS via bio-rd. 6 unit tests for proto conversion, but never connected to a real RIS endpoint. Can be tested locally (build `cmd/ris/` from bio-rd + BMP speaker), but this setup has never been done.
55
- - **Network Sources**: HTTP-fetched prefix metadata with jq transforms. 12 unit tests for transform/decode, but HTTP fetch cycle never tested with a real endpoint.
56
- - **Topology drilldown**: Frontend hook `useFlowsDrilldownData` is dead code (never imported), no "Flows" tab in topology actor modal. Not an untested feature -- it simply does not exist.
57
-
58
-- **Benchmark data — README is severely stale**:
59
- - The README block at `src/crates/netflow-plugin/README.md:309-367` was captured before recent ingest-path optimizations and is no longer representative. It claimed single-core saturation at ~5.8-6.3k flows/s. After optimizations, the plugin saturates at ~49k flows/s low-cardinality and ~43k flows/s high-cardinality on the same workstation (i9-12900K + FireCuda 530, ext4). The "single core" framing is also misleading — the post-decode ingest path is multi-threaded; `cpu_percent_of_one_core` accumulates user+system ticks across all threads divided by wall time, so a saturated host reports >100%.
60
- - Fresh benchmarks must be run (Phase 1.0) before documentation is written. README must be rewritten with the new numbers as part of Phase 1.0.
61
- - Benchmark commands shipped with the plugin for re-running on target hardware (`bench_resource_envelope_matrix`, `bench_ingestion_protocol_matrix`, `bench_ingestion_cardinality_matrix`).
62
-
63
-- **BMP industry survey** (from mirrored repos):
64
- - Combined approach (Akvorado, nProbe): BMP + flow enrichment in one binary, in-memory RIB, enrichment only
65
- - Separate approach (pmacct, GoBMP, OpenBMP): dedicated BMP daemon with DB/persistence, for full BGP monitoring
66
- - Netdata's implementation follows the Akvorado pattern (enrichment only, in-memory, no persistence)
67
- - Decision: BMP in netflow-plugin is for enrichment only. Full BGP monitoring per-route would require a separate plugin with its own DB. This is a future consideration, not a Phase 1 concern.
68
-
69
-- **All enrichment is in-memory, zero per-flow cost**: GeoIP, static networks, routing trie, network sources -- all background-fetched/stored, pure RwLock read per flow. No HTTP/BGP per flow.
70
-
71
-Current gaps (for Phase 1):
72
-
73
-- Zero end-user documentation on learn.netdata.cloud
74
-- No metadata.yaml for any flow protocol
75
-- No config_schema.json, no health.d/ alerts, no docs/.map/map.yaml entry
76
-- Dead redirect in LegacyLearnCorrelateLinksWithGHURLs.json
77
-
78
-### Acceptance Criteria
79
-
80
-Phase 1 only:
81
-- `metadata.yaml` with 3 modules (netflow, ipfix, sflow) validated against integrations schema
82
-- Integrations pipeline generates per-protocol pages (in-app catalog, COLLECTORS.md, learn)
83
-- Learn section "Network Flows" in `docs/.map/map.yaml`
84
-- All pages follow style guide: second person, active voice, sentence case, `:::type`/`:::` admonitions
85
-- Complete field reference (89+2 fields) with per-protocol availability matrix
86
-- Enrichment docs for all features (GeoIP, static metadata, sampling, static networks, classifiers, ASN resolution, BMP routing, BioRIS, Network Sources, decapsulation)
87
-- No mention of topology drilldown, pcap, eBPF, or threat analytics anywhere
88
-- Sizing/capacity planning page sourced from FRESH benchmark measurements (not the stale README) covering NetFlow v9, IPFIX, sFlow at 10 offered rates from 100 to 60000 flows/s, low- and high-cardinality, full pipeline (all-tiers-batched). Includes storage estimation formulas, memory guidance, optimization tips, and a clear note on multi-thread CPU semantics.
89
-- Visualization docs split into focused pages
90
-- Screenshots embedded via GitHub URLs (provided by user)
91
-- AI skills updated with links to learn docs
92
-
93
-## Implications And Decisions
94
-
95
-### Decision 1: plugin_name -- DECIDED: netflow-plugin
96
-
97
-### Decision 2: metadata.yaml scope -- DECIDED: minimal per module
98
-
99
-Identity, overview, protocol-specific router config examples, quirks. Deep details in learn section.
100
-
101
-### Decision 3: Learn section position -- DECIDED: new top-level "Network Flows"
102
-
103
-### Decision 4: Page structure -- DECIDED: more pages, small and focused
104
-
105
-```
106
-Network Flows/
107
- Overview/
108
- Quick Start/
109
- Sources/
110
- NetFlow/
111
- IPFIX/
112
- sFlow/
113
- Configuration/
114
- Enrichment/
115
- GeoIP/
116
- Static Metadata/
117
- Classifiers/
118
- ASN Resolution/
119
- BMP Routing/
120
- BioRIS/
121
- Network Sources/
122
- Decapsulation/
123
- Field Reference/
124
- Visualization/
125
- Summary and Sankey/
126
- Time-Series/
127
- Maps/
128
- Globe/
129
- Filters and Facets/
130
- Dashboard Cards/
131
- Retention and Querying/
132
- Sizing and Capacity Planning/
133
- Troubleshooting/
134
-```
135
-
136
-~24 pages. Each page focused on one topic. No mentions of pcap, eBPF, threat analytics, or topology drilldown. BMP, BioRIS, and Network Sources are documented based on unit-tested parsing/conversion logic; their runtime I/O paths (TCP listener, gRPC client, HTTP fetch) lack integration tests.
137
-
138
-### Decision 5: Future features -- DECIDED: document BMP/BioRIS/Network Sources, skip topology drilldown
139
-
140
-BMP, BioRIS, and Network Sources are included in documentation. Their unit-tested parsing logic is solid but runtime I/O paths lack integration tests. The SOW follow-up section tracks the need for async/integration tests.
141
-
142
-Topology drilldown remains excluded (dead code -- `useFlowsDrilldownData.js` never imported). pcap, eBPF, and threat analytics remain excluded (not implemented).
143
-
144
-### Decision 6: AI skills -- DECIDED: update existing query skills
145
-
146
-### Decision 7: Integrations pipeline -- DECIDED: add netflow-plugin to gen_integrations.py
147
-
148
-### Decision 9: Documentation rewrite directives -- DECIDED 2026-05-07
149
-
150
-User read parts of the research at `.agents/knowledge/Network Traffic Analysis with Flow Data.md`, taught the dashboard mechanics directly, corrected several misconceptions (notably the doubling/mirroring foundational concept and the function-permission paid-only assumption), and instructed:
151
-
152
-- **Rewrite, not edit.** Existing pages are thin and must not be inherited as correct. Every claim is re-verified against the code.
153
-- **Code is the source of truth.** The existing markdown, comments, and prior docs are reference points, not authority.
154
-- **Audience-aware, never condescending.** Documentation must serve newcomers without insulting experts. Mental models must be established before features.
155
-- **Three roles served in parallel:** Network Engineer, Security Analyst, IT Manager. Pages stay role-agnostic but every page must be useful to all three.
156
-- **Doubling and mirroring is foundational, not an aside.** Users must understand it before reading any aggregate number.
157
-- **Sampling is documented honestly.** Auto-multiplied at ingestion; mixed rates make aggregates uninterpretable; admins should keep rates uniform or run unsampled.
158
-- **Installation is a first-class topic.** The plugin is packaged separately (`netdata-plugin-netflow` on both DEB and RPM via `netdata.spec.in:3270` and `Packaging.cmake:488`). Not auto-installed by the netdata-updater. Users must install it themselves on native-package systems. Static installs (`kickstart.sh --static-only`) bundle it.
159
-- **Anti-patterns get a dedicated page.** The research's most common deployment failures (ignored sampling, alert on absolute volume, GeoIP firewall of shame, NAT blindness, double-counting, treating duration as latency, microburst hunting) are called out explicitly.
160
-- **Investigation playbooks get a dedicated page.** Concrete walkthroughs from the research's recognition cues (bandwidth saturation, IP investigation, capacity justification, security alert scope) — written for the Netdata UI specifically (Top-N + facets + sankey + maps).
161
-- **Validation & data quality gets a dedicated page.** SNMP cross-check, exporter health monitoring, doubling check, sampling sanity check.
162
-- **No mechanical writing.** Each page begins with code research, an explicit audience statement, an explicit goal statement, and 3-5 key takeaways. Page produced only after the model is settled.
163
-
164
-**Scope restored** (changed from earlier interpretation): BMP, BioRIS, and Network Sources are documented based on their unit-tested parsing logic (per Decision 5). Their runtime I/O paths lack integration tests but the features exist in the code and ship in user configurations. Users would otherwise see references to them in `netflow.yaml` without explanation. The follow-up still tracks the integration-test gap.
165
-
166
-**Scope confirmed deferred:** topology drilldown (`useFlowsDrilldownData.js`) is dead code — never imported into the topology actor modal — and stays out of documentation until implemented.
167
-
168
-**Function permissions verified** at `src/crates/netflow-plugin/src/api/flows/handler.rs:263`: the `flows:netflow` function uses `HttpAccess::SIGNED_ID | SAME_SPACE | SENSITIVE_DATA`. It does NOT include `COMMERCIAL_SPACE`. The agent function is therefore not paid-gated. Any feature gating elsewhere (UI/cloud-frontend) is outside this repository and outside this SOW.
169
-
170
-### Decision 10: Promote `flows` to a top-level integration_type with 14 cards -- DECIDED 2026-05-07
171
-
172
-User reframed the test for "should X be an integration card?" as "would users ask 'Does Netdata integrate with X?'". This rejected the original architecturally pure framing (one card per protocol decoder under data-collection.networking) in favour of one card per vendor/source users actually shop for.
173
-
174
-Sub-decisions:
175
-
176
-- **10a Type position:** `flows` is a top-level `integration_type` in `integrations/categories.yaml`, peer of `collector`, `logs`, `exporter`, `notification`, `secretstore`, `authentication`. NOT under `data-collection.networking`. Rationale: flows are a different data model from collector metrics (table-shaped, faceted, time-windowed, journal-backed), the same way logs are -- their integration UX should mirror logs, not collectors.
177
-- **10b Catalog shape:** four sub-categories under `flows` -- Sources, IP Intelligence, BGP Routing, Network Identity Sources -- containing 14 cards total:
178
- - Sources (3): NetFlow, IPFIX, sFlow
179
- - IP Intelligence (4): DB-IP IP Intelligence (default), MaxMind GeoIP / GeoLite2, IPtoASN, Custom MMDB Database
180
- - BGP Routing (2): BMP (BGP Monitoring Protocol), bio-rd / RIPE RIS
181
- - Network Identity Sources (5): AWS IP Ranges, GCP IP Ranges, Azure IP Ranges, NetBox, Generic JSON-over-HTTP IPAM
182
-- **10c Pipeline plumbing:** `integrations/gen_integrations.py` and `integrations/gen_docs_integrations.py` learn the new type via FLOWS_SOURCES list, FLOWS_RENDER_KEYS, FLOWS_VALIDATOR, `load_flows()`, `render_flows()`, plus a `mode == "flows"` branch in `build_readme_from_integration()`. New `integrations/templates/overview/flows.md` template. New `integrations/schemas/flows.json` ($ref to `collector.json` for now -- can diverge later).
183
-- **10d Map placement:** `docs/.map/map.yaml` carries a top-level Network Flows section using `integration_placeholder` with `integration_kind: flows`, plus three concept docs (IP Intelligence, BGP Routing, Network Identity).
184
-- **10e Schema:** `docs/.map/map.schema.json` enum extended to accept `flows` as an integration_kind value alongside the existing kinds.
185
-- **10f Cross-repo coordination:** companion learn PR (netdata/learn#2854) routes `src/crates/netflow-plugin/integrations/<slug>.md` files into a `flows_entries` DataFrame and splices them over the `flows_integrations` placeholder, mirroring the existing `logs_integrations` handler. cloud-frontend's `data/integrations.js` is auto-generated and committed manually -- a follow-up "Update integrations.js" PR is needed there. www auto-updates daily; no PR required.
186
-- **10g Skill knowledge capture:** `integrations-lifecycle` skill gains a new per-integration_type Learn-routing matrix in `per-type-matrix.md` and a how-to `adding-new-integration-type.md` enumerating the 8-place checklist for future kinds.
187
-
188
-### Decision 8: Benchmark rerun matrix -- DECIDED 2026-05-06
189
-
190
-Post-optimization rerun supersedes README:309-367. Sub-decisions:
191
-
192
-- **8a Protocols:** netflow-v9, ipfix, sflow. Skip netflow-v5 (legacy, unrepresentative of modern deployments).
193
-- **8b Layer:** all-tiers-batched only (full pipeline: raw + 1m + 5m + 1h). The user-facing question is "what does the plugin cost on my host?" — that is the full-pipeline number. Other layers (writer-only, raw-only, minute1-only) are engineering-internal and not republished to users.
194
-- **8c Modes:** both. (i) Paced post-decode resource envelope (`bench_resource_envelope_*`) — produces sizing curves at exact offered rates. (ii) Unpaced full UDP→journal (`bench_ingestion_protocol_matrix`) — produces per-protocol max-throughput numbers including the decode cost. Mode (i) drives the sizing/capacity page; mode (ii) is supplementary and shows decode-path differences between protocols.
195
-- **8d Rate matrix:** 10 offered rates: 100, 500, 1000, 5000, 10000, 20000, 30000, 40000, 50000, 60000 flows/s. The host saturates at ~49k/s low-cardinality and ~43k/s high-cardinality, so the upper end intentionally exercises the saturation plateau. Three protocols × two cardinalities × ten rates = 60 paced cells. Plus 3 unpaced protocol cells × 3 phases (full / decode-only / post-decode) = 9 unpaced cells.
196
-- **8e Hardware:** this workstation. CPU `12th Gen Intel(R) Core(TM) i9-12900K`, storage `Seagate FireCuda 530 NVMe`, ext4. Same rig as the original README capture; comparable baseline.
197
-- **8f Output format and location:** shell driver writes one `RESOURCE_BENCH_RESULT:{json}` line per cell to `<repo>/.local/audits/netflow-bench/results.jsonl`. Renderer produces a markdown summary table per (protocol, cardinality) under the same directory. `.local/` is gitignored — raw outputs are not committed; only the SOW + README + sizing doc carry numbers into the repository.
198
-
199
-CPU semantics note (will appear in docs):
200
-- `cpu_percent_of_one_core` accumulates user+system ticks across all threads divided by wall time
201
-- Below saturation it can be <100%
202
-- At saturation on a multi-core host it is well above 100% (e.g. ~600-800% on this workstation when ingest threads + tier-batch threads are all busy)
203
-- Documentation must call this out explicitly so capacity planners read the metric correctly
204
-
205
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
206
-
207
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
208
-
209
-Problem / root-cause model:
210
-
211
-- Netdata has a fully functional flow collection and visualization system with tested features, but zero documentation and zero integration catalog entries. Four features (BMP, BioRIS, Network Sources, Topology drilldown) are untested or not implemented and must be excluded from documentation until validated.
212
-
213
-Evidence reviewed:
214
-
215
-- All files listed in previous SOW versions
216
-- Benchmark data: `src/crates/netflow-plugin/README.md:309-367` is STALE (post-optimization saturation ~49k/43k, not ~6k as the README claims). Authoritative numbers come from the Phase 1.0 rerun (matrix per Decision 8). README:309-367 must be replaced as part of Phase 1.0.
217
-- Test coverage: 81 enrichment tests, 11 BMP parsing tests, 6 BioRIS unit tests, 12 network source unit tests -- but zero integration tests for BMP TCP listener, BioRIS gRPC client, or network source HTTP fetch
218
-- BMP industry survey: 14 repos analyzed across mirrored codebase
219
-- BioRIS local testing: possible by building `cmd/ris/` from bio-rd, never done
220
-- Topology drilldown: `useFlowsDrilldownData.js` never imported (dead code)
221
-- Documentation style guide: `docs/developer-and-contributor-corner/style-guide.md` (second person, active voice, sentence case, Oxford comma, `:::type`/`:::` admonitions)
222
-
223
-Affected contracts and surfaces:
224
-
225
-- `docs/.map/map.yaml` -- new section
226
-- `src/crates/netflow-plugin/metadata.yaml` -- new file
227
-- `integrations/gen_integrations.py` -- plugin_name registration
228
-- Generated integration pages, COLLECTORS.md
229
-- `docs/netdata-ai/skills/query-netdata-cloud/query-flows.md` -- updated
230
-- `docs/netdata-ai/skills/query-netdata-agents/query-flows.md` -- updated
231
-- ~17 new learn pages under `docs/network-flows/`
232
-- `src/crates/netflow-plugin/src/ingest_resource_bench_tests.rs` -- add `NETFLOW_RESOURCE_BENCH_PROTOCOL` env var (Phase 1.0 step 1)
233
-- `src/crates/netflow-plugin/README.md:309-367` -- benchmark block fully replaced with fresh post-optimization numbers (Phase 1.0 step 4)
234
-- `<repo>/.local/audits/netflow-bench/` -- new gitignored output directory for raw JSONL + rendered markdown (Phase 1.0 step 2)
235
-
236
-Existing patterns to reuse:
237
-
238
-- Style guide conventions
239
-- Existing collector metadata.yaml (SNMP, PostgreSQL) as templates
240
-- Integrations-lifecycle skill, learn-site-structure skill
241
-- `src/crates/netflow-plugin/README.md` as source material
242
-- `query-flows.md` skills as source for API examples
243
-
244
-Risk and blast radius:
245
-
246
-- Documentation-only (except gen_integrations.py plugin_name addition + the small bench harness env-var addition + README replacement)
247
-- Must carefully avoid documenting untested features -- all enrichment docs must be scoped to tested features only
248
-- BMP routing, BioRIS, and Network Sources exist in config examples (netflow.yaml) but must not appear in documentation as usable features until tested
249
-- Bench harness change is additive (new env var, fall-back behavior preserved). No existing test or caller is affected. Verify by running one cell with each protocol value before launching the full matrix.
250
-- Benchmark rerun consumes the workstation for ~25 min wall time (60 paced cells x ~20s + 9 unpaced cells). Host should be quiet during the run; concurrent CPU load skews CPU% and write_bytes/s readings.
251
-
252
-Sensitive data handling plan:
253
-
254
-- Use `NODE` placeholder instead of real IPs (per style guide)
255
-- Use RFC 5737 documentation ranges in all examples
256
-- No real credentials, tokens, or network topologies
257
-
258
-Implementation plan:
259
-
260
-1. Phase 1.0: Benchmark rerun (matrix per Decision 8) and README replacement
261
-2. Phase 1A: Integration infrastructure (metadata.yaml + pipeline)
262
-3. Phase 1B: Learn section (map.yaml + ~17 pages, with sizing page sourced from Phase 1.0 fresh measurements)
263
-4. Phase 1C: AI skills update
264
-5. Phase 1D: Validation
265
-
266
-Validation plan:
267
-
268
-- Run integrations pipeline, verify artifacts
269
-- Verify map.yaml routing
270
-- Cross-reference every config option against plugin_config.rs
271
-- Cross-reference field list against flow/schema.rs
272
-- Cross-reference benchmark numbers against the Phase 1.0 fresh rerun output (`<repo>/.local/audits/netflow-bench/results.jsonl`), NOT against the stale README. README is a downstream artifact updated by Phase 1.0.
273
-- Verify no mention of untested features anywhere
274
-
275
-Artifact impact plan:
276
-
277
-- AGENTS.md: no update needed
278
-- Runtime project skills: no update needed
279
-- Specs: no spec update needed
280
-- End-user/operator docs: this IS the docs update
281
-- End-user/operator skills: query skills updated (Phase 1C)
282
-- SOW lifecycle: standard flow
283
-
284
-Open decisions:
285
-
286
-- None. All resolved.
287
-
288
-## Plan
289
-
290
-### Phase 1.0: Benchmark Rerun + README Refresh
291
-
292
-1. Add `NETFLOW_RESOURCE_BENCH_PROTOCOL` env var to `src/crates/netflow-plugin/src/ingest_resource_bench_tests.rs`. When set to `netflow-v9`, `ipfix`, or `sflow`, route `build_record_batches` to the matching entry in `PROTOCOL_SCENARIOS`. When unset, fall back to the existing `CARDINALITY_SOURCE_SCENARIO` mixed behavior so legacy callers are unaffected. Pass the new env var through `run_resource_envelope_case` to the child process.
293
-2. Write `<repo>/.local/audits/netflow-bench/run.sh` driver script:
294
- - Iterate protocols `{netflow-v9, ipfix, sflow}` x cardinalities `{low, high}` x rates `{100, 500, 1000, 5000, 10000, 20000, 30000, 40000, 50000, 60000}`.
295
- - Per cell: run `cargo test -p netflow-plugin --release ingest::resource_bench_tests::bench_resource_envelope_child -- --ignored --nocapture --exact --test-threads=1` with env vars `NETFLOW_RESOURCE_BENCH_CHILD=1`, `NETFLOW_RESOURCE_BENCH_LAYER=all-tiers-batched`, and the per-cell protocol/profile/rate.
296
- - Capture the `RESOURCE_BENCH_RESULT:{json}` line into `results.jsonl` (one line per cell, including a `protocol` field).
297
- - Run `bench_ingestion_protocol_matrix` once for the unpaced 3B output; capture stderr report blocks into `protocol_matrix.txt`.
298
- - Render per-(protocol,cardinality) markdown tables (`results.md`).
299
- - Use the `run()` visibility wrapper from `~/.claude/CLAUDE.md` so commands are echoed.
300
-3. Verify host quietness before running (`top` / `ps`). Re-run any cell that fails or that shows pacer underrun for spurious reasons.
301
-4. Replace `src/crates/netflow-plugin/README.md:309-367` with the fresh numbers, including:
302
- - Per-protocol per-cardinality tables for the 10 rates (3A).
303
- - Per-protocol max-throughput table from 3B.
304
- - Multi-thread CPU semantics note.
305
- - Storage estimation hints derived from `write_bytes_per_sec` and `logical_write_bytes_per_sec` ratios.
306
-
307
-### Phase 1A: Integration Infrastructure
308
-
309
-1. Add `netflow-plugin` to `gen_integrations.py` recognized plugin names
310
-2. Create `src/crates/netflow-plugin/metadata.yaml` with 3 modules
311
-3. Create icon assets
312
-4. Run pipeline, verify generated pages
313
-
314
-### Phase 1B: Learn Section (full rewrite -- ~25 pages -- per Decision 9)
315
-
316
-Per-page method, applied uniformly:
317
-
318
-1. **Subagent for technical analysis first.** Each feature/section gets a dedicated read-only subagent that explores the relevant code under `src/crates/netflow-plugin/` (and adjacent crates / Go tools where applicable), identifies the happy path and the nuances, enumerates configuration options with their defaults, traces error paths, and returns a structured analysis with file:line citations. The master assistant synthesizes the doc from the analysis. This keeps raw code-reading noise out of the master context and ensures each page is grounded.
319
-2. **Bottom-up writing order.** Detail pages are written first; the `Overview` is written last as the natural index over already-established truths. This avoids the trap of writing an Overview that promises behavior the detail pages later contradict.
320
-3. **State audience and goal per page.** Each page begins with an internal "audience / goal / key takeaways" plan recorded in the SOW execution log before the page is written.
321
-4. **Verify each claim.** No statement reaches the page without a file:line citation in the analysis returned by the subagent.
322
-5. **Mental model before features.** Doubling/mirroring, sampling semantics, tier transparency are foundational and explained before any aggregate number is shown.
323
-6. **Three audiences served.** Network Engineer / Security Analyst / IT Manager all find their answers without the page wearing a role label.
324
-
325
-Bottom-up writing order:
326
-
327
-Tier A -- leaves (independent, parallelisable subagents):
328
-- `sources/netflow.md`, `sources/ipfix.md`, `sources/sflow.md` -- per-protocol decoder analysis
329
-- `configuration.md` -- whole `plugin_config/` schema
330
-- `field-reference.md` -- `flow/` schema enumeration
331
-- `retention-querying.md` -- tier model + query tier-selection logic
332
-- 8 enrichment pages -- each enrichment module under `src/crates/netflow-plugin/src/enrichment/` and runtime initialization
333
-- 5 visualization pages -- UI behavior plus the corresponding query/response paths under `src/crates/netflow-plugin/src/api/flows/` and `query/`
334
-- `troubleshooting.md` -- error paths, exposed plugin metrics, log message inventory
335
-
336
-Tier B -- synthesis (depend on Tier A):
337
-- `anti-patterns.md` -- distilled from research §6 + Tier A findings
338
-- `validation.md` -- distilled from research §11 + Tier A operational metrics
339
-- `investigation-playbooks.md` -- four scenarios that exercise the visualizations, facets, and tiers documented in Tier A
340
-- `installation.md` -- standalone but depends on `configuration.md` for post-install verification
341
-
342
-Tier C -- top:
343
-- `quick-start.md` -- the first-time path through Tiers A and B
344
-- `README.md` -- Overview (final synthesis, the natural index)
345
-
346
-Page list (final):
347
-
348
-Root level:
349
-- `README.md` -- Overview (rewrite). Mental model, what flow data is, what it answers and what it does not, doubling/mirroring foundational concept, sampling caveat, prerequisites pointer, installation pointer, navigation.
350
-- `installation.md` -- Installation (NEW). Package names per distro, post-install verification, file locations, dependencies, source-build caveats. Confirms `netdata-plugin-netflow` is the canonical name on DEB and RPM and that it is opt-in on native-package systems.
351
-- `quick-start.md` -- Quick Start (rewrite). Three-step path, router config examples (NetFlow v9, IPFIX, sFlow), dashboard first-look, doubling/mirroring read-this-first section, verification step.
352
-- `configuration.md` -- Configuration (rewrite). Listener, protocols, journal layout and retention, decapsulation, performance tuning (UDP buffer sysctls, sync interval, record_pool_size).
353
-- `field-reference.md` -- Field Reference (rewrite). All fields organised by category, per-protocol availability matrix, usage hints per field.
354
-- `retention-querying.md` -- Retention and Querying (rewrite). Tier model, how queries auto-pick tiers, IP/port loss in tiers 1-3 and the "filter on IP forces tier 0" behavior.
355
-- `sizing-capacity.md` -- Sizing and Capacity Planning (already done in Phase 1.0).
356
-- `validation.md` -- Validation and Data Quality (NEW). SNMP cross-check, doubling sanity check, sampling sanity check, exporter health monitoring, the silent-failure list.
357
-- `investigation-playbooks.md` -- Investigation Playbooks (NEW). Walkthroughs: bandwidth saturation, IP investigation, capacity justification, security alert scope. UI-specific (Top-N, facets, sankey).
358
-- `anti-patterns.md` -- Anti-patterns and Pitfalls (NEW). Doubled aggregate, ignored sampling, GeoIP for internal IPs, absolute thresholds, collect-and-ignore, flows-vs-sessions, NAT blindness, geographic firewall of shame, duration-as-latency, microburst hunting.
359
-- `troubleshooting.md` -- Troubleshooting (rewrite). Plugin not running, no data arriving, partial data, template errors, GeoIP gaps, performance issues.
360
-
361
-Sources:
362
-- `sources/netflow.md` -- NetFlow v5/v7/v9. Protocol semantics, template lifecycle (v9), active timeout best practice, configuration examples for major vendors.
363
-- `sources/ipfix.md` -- IPFIX. Protocol semantics, IE handling, template withdrawal, biflow rarity, vendor configuration examples.
364
-- `sources/sflow.md` -- sFlow v5. Fundamentally different from NetFlow — packet samples + counter samples, sampling rate inherent, semantics of byte counts.
365
-
366
-Enrichment:
367
-- `enrichment/geoip.md` -- GeoIP via MMDB. MaxMind database management, internal-IP trap, validation.
368
-- `enrichment/static-metadata.md` -- Static metadata (exporter naming, interface descriptions, custom labels). Foundational for multi-exporter analysis.
369
-- `enrichment/classifiers.md` -- Akvorado-style classifier rules. Rule syntax, when they fire, debugging.
370
-- `enrichment/asn-resolution.md` -- ASN resolution. Provider chains, fallback, validation.
371
-- `enrichment/bmp-routing.md` -- BMP listener (Decision 5: in scope; runtime I/O lacks integration tests, documented based on parsing logic). What BMP is, what BMP enrichment provides, configuration, integration-test caveat.
372
-- `enrichment/bioris.md` -- BioRIS gRPC client (same in-scope/caveat as BMP).
373
-- `enrichment/network-sources.md` -- HTTP-fetched prefix metadata with jq transforms (same in-scope/caveat).
374
-- `enrichment/decapsulation.md` -- SRv6, VXLAN inner-packet extraction. Modes, when each applies.
375
-
376
-Visualization:
377
-- `visualization/summary-sankey.md` -- The default landing view. How to read sankey + table together, field selection, top-N control, sort by bytes/packets, doubling-when-unfiltered.
378
-- `visualization/time-series.md` -- Top-N over time. When to use, baselines, time-shifted comparison.
379
-- `visualization/maps-globe.md` -- Country/state/city maps + 3D globe. Tooltips, no drill-down, GeoIP traps.
380
-- `visualization/filters-facets.md` -- Filter ribbon + autocomplete + FTS + selections persistence + URL sharing. AND-of-ORs semantics.
381
-- `visualization/dashboard-cards.md` -- Operational metrics for the plugin itself. What to watch for plugin health.
382
-
383
-### Phase 1B (legacy, superseded by per-page list above)
384
-
385
-1. Add "Network Flows" section to `docs/.map/map.yaml`
386
-2. Clean up dead redirect in `LegacyLearnCorrelateLinksWithGHURLs.json`
387
-3. Write pages (Overview, Quick Start, Sources/NetFlow, Sources/IPFIX, Sources/sFlow, Configuration, Enrichment/GeoIP, Enrichment/Static Metadata, Enrichment/Classifiers, Enrichment/ASN Resolution, Enrichment/Decapsulation, Field Reference, Visualization/Summary+Sankey, Visualization/Time-Series, Visualization/Maps, Visualization/Globe, Visualization/Filters+Facets, Visualization/Dashboard Cards, Retention and Querying, Sizing and Capacity Planning, Troubleshooting)
388
-
389
-### Phase 1C: AI Skills Update
390
-
391
-1. Update both query-flows.md skills (links, field list, Globe view)
392
-2. Update SKILL.md files if needed
393
-3. Add how-tos for common patterns
394
-
395
-### Phase 1D: Validation
396
-
397
-1. Pipeline artifacts verified
398
-2. Map.yaml routing verified
399
-3. MDX compliance checked
400
-4. Cross-reference against source code
401
-5. Verify zero mentions of untested features
402
-
403
-## Execution Log
404
-
405
-### 2026-05-06
406
-
407
-- Created SOW from comprehensive analysis
408
-- Deep-dived into: fields, enrichment, integrations pipeline, GeoIP, BMP, BioRIS, network sources, topology drilldown, documentation style/patterns
409
-- BMP industry survey: 14 repos analyzed. Combined approach (Akvorado pattern) confirmed for enrichment use case
410
-- BioRIS: can run locally (build cmd/ris/ from bio-rd), never tested
411
-- Confirmed: topology drilldown is dead code (never imported)
412
-- Original benchmark data taken from README: 5k flows/s sustainable, ~6k saturation on i9-12900K (NOW KNOWN STALE — see entry below)
413
-- User raised testing concerns: BMP, BioRIS, Network Sources are untested, must not be documented
414
-- Agreed phased approach: Phase 1 documents tested features + sizing, Phase 2 tests + documents untested features
415
-- All 7 decisions resolved
416
-
417
-### 2026-05-06 (later) - benchmark rerun finding
418
-
419
-- Discovered the README benchmark block (`src/crates/netflow-plugin/README.md:309-367`) is severely stale: post-optimization the post-decode ingest path is much faster than the README claimed (~5.8-6.3k flows/s saturation). The "single core" framing was correct in principle (the post-decode hot path is single-threaded) but the absolute numbers were obsolete.
420
-- Inspected the existing benchmark harness: `ingest_resource_bench_tests.rs` (paced post-decode), `ingest_bench_tests.rs::bench_ingestion_protocol_matrix` (unpaced full UDP→journal), `ingest_resource_bench_support.rs` (`ResourceEnvelopeReport` records achieved flows/s, CPU%, peak/final RSS, read/write bytes/s — exactly the four metrics the user asked to record).
421
-- Confirmed the harness is configurable via env vars for cardinality (`..._PROFILE`), rate (`..._FLOWS_PER_SEC`), layer (`..._LAYER`), warmup, measure window, and pool sizes. The harness is NOT today configurable per-protocol — `bench_resource_envelope_*` always uses `CARDINALITY_SOURCE_SCENARIO` (mixed). Added `NETFLOW_RESOURCE_BENCH_PROTOCOL` env var that routes to a specific entry in `PROTOCOL_SCENARIOS` when set, falling back to the mixed default otherwise. Added `protocol` field to `ResourceEnvelopeReport` so the JSON output is self-describing.
422
-- User decided rerun matrix (Decision 8): netflow-v9+ipfix+sflow, all-tiers-batched layer, both paced + unpaced modes, 10 rates from 100 to 60000 flows/s, this workstation, JSONL+markdown output under `.local/audits/netflow-bench/`.
423
-
424
-### 2026-05-06 (later still) - benchmark rerun results
425
-
426
-Driver: `.local/audits/netflow-bench/run.sh` + `render.py`. Outputs: `results.jsonl` (60 paced cells), `protocol_matrix.txt` (3B unpaced), `results.md`. Wall time: 22 min for 3A + 5s for 3B. Host load was nominal (load average ~3.5 on 16 cores; user accepted).
427
-
428
-Key findings:
429
-
430
-- **High-cardinality saturation is around 30 000 flows/s post-decode for all three protocols.** Above the knee, achieved rate plateaus while offered grows. CPU pins at ~98-99% of one core at saturation, confirming the post-decode hot path is single-threaded (CPU does not exceed 100% even when more rate is offered).
431
-- **Low-cardinality saturation is above 60 000 flows/s** for all three protocols — never reached within the chosen matrix. CPU at 60k offered: ipfix 64.1%, netflow-v9 70.3%, sflow 87.0%. Extrapolated single-core ceiling: ~85-100k for v9/ipfix, ~70k for sflow.
432
-- **3B unpaced full UDP→journal peaks**: NetFlow v9 ~99k flows/s, IPFIX ~107k flows/s, sFlow ~88k flows/s. Decode-only is much faster (0.8-2.4M flows/s) — decode is roughly 10% of the full path cost.
433
-- **User's earlier "49k/43k cap" did not appear** in the protocol-isolated rerun. That number was from the older mixed-scenario run; protocol-isolated low-card is faster than 49k and high-card is slower than 43k (~30k).
434
-- **Practical published number**: ~20-25k flows/s for "everything except enrichment" (UDP receive + decode + 4 tiers + high cardinality, mixed protocols). 20k = conservative, 25k = optimistic. Derivation: 30k post-decode ceiling minus ~10 µs/flow for decode brings the full-path ceiling to ~22-25k. UDP socket receive at 1-5k pps is below typical socket limits and not a concern.
435
-- **CPU semantics revisited**: cpu_percent_of_one_core reports >100% only when multiple threads contribute. The plateau at 98-99% is the single-thread limit on the post-decode ingest hot path. Multi-threading work above that ceiling is a future optimization, not a current capability.
436
-
437
-Artifacts updated as part of this rerun:
438
-
439
-- `src/crates/netflow-plugin/src/ingest_resource_bench_tests.rs` — added `NETFLOW_RESOURCE_BENCH_PROTOCOL` env var and threaded it through child spawn
440
-- `src/crates/netflow-plugin/src/ingest_resource_bench_support.rs` — added `protocol` field to `ResourceEnvelopeReport`
441
-- `src/crates/netflow-plugin/README.md:309-...` — fully replaced benchmark section with fresh per-protocol/cardinality tables, 3B unpaced summary, multi-thread CPU semantics note
442
-- `docs/network-flows/sizing-capacity.md` — replaced stale numbers with fresh tables, added the 20-25k headline
443
-- `.local/audits/netflow-bench/` — driver script, renderer, raw JSONL, rendered markdown (gitignored)
444
-
445
-### 2026-05-07 - storage footprint benchmark
446
-
447
-User asked for a benchmark that measures actual on-disk storage growth, write amplification, and the dedup/cardinality effect. Replaces the "flows × bytes/flow × time" table that I had wrongly written into sizing-capacity.md (the journals are indexed and dedup-aware, so that calculation is invalid).
448
-
449
-Implementation:
450
-- New test `bench_storage_footprint_child` in `src/crates/netflow-plugin/src/ingest_resource_bench_tests.rs`. Reuses `run_paced_plugin_loop` in segments equal to the configured sample interval; between segments samples per-tier on-disk size via `journal_dir_size_bytes`, real I/O via `/proc/self/io`, logical encoded bytes via metrics deltas, RSS via `/proc/self/status`.
451
-- New types `StorageFootprintSample` and `StorageFootprintReport` in `src/crates/netflow-plugin/src/ingest_resource_bench_support.rs`.
452
-- New env vars: `NETFLOW_STORAGE_BENCH_DURATION_SECS` (default 900), `NETFLOW_STORAGE_BENCH_SAMPLE_INTERVAL_SECS` (default 30). Reuses the existing `..._FLOWS_PER_SEC`, `..._PROFILE`, `..._PROTOCOL` env vars.
453
-- Driver: `.local/audits/netflow-bench/run-storage.sh` runs one cell per cardinality and renders markdown.
454
-- Renderer: `.local/audits/netflow-bench/render-storage.py` produces per-cell growth tables and a dedup-ratio summary.
455
-
456
-Results on the same workstation, ipfix at 10 000 flows/s, 15 min per cardinality:
457
-
458
-- Low cardinality: 9.00M flows ingested, 6.46 GiB on-disk total, 771 bytes/stored-flow, write amplification 1.79×.
459
-- High cardinality: 8.97M flows ingested, 7.29 GiB on-disk total, 872 bytes/stored-flow, write amplification 2.00×.
460
-- Dedup ratio (high / low): only 1.13× — high cardinality stores 13% more per flow despite 16× more unique field combinations. Real-world traffic with repeated patterns will trend closer to the low-cardinality figure.
461
-- Raw tier dominates at this timescale (≥99% of total). Rollups (1m: 8-112 MiB, 5m: 8-40 MiB, 1h: 0-16 MiB) are tiny because each rollup row aggregates many raw flows. 1-hour tier did not roll over for low-cardinality within 15 min; it appeared in the high-cardinality run at t=660s onward.
462
-
463
-Sizing/capacity doc updated to remove the bogus "flow × bytes × time" table and replace it with the empirical numbers above plus retention-bounded storage planning guidance.
464
-
465
-Outputs (gitignored):
466
-- `.local/audits/netflow-bench/storage-low.json` — full sample stream for low-cardinality cell
467
-- `.local/audits/netflow-bench/storage-high.json` — full sample stream for high-cardinality cell
468
-- `.local/audits/netflow-bench/storage.md` — rendered markdown
469
-
470
-### 2026-05-07 (later) - documentation rewrite, branch, and PR opened
471
-
472
-- Phase 1B documentation rewrite landed on branch `netflow-plugin-docs-and-bench` over three commits:
473
- - `c708101e` -- per-protocol benchmark + storage footprint test
474
- - `1455d59f` -- docs: rewrite Network Flows documentation (~25 pages)
475
- - `e61c72b7` -- integrations: add netflow-plugin (netflow, ipfix, sflow modules)
476
-- Architectural pivot per Decision 10: `6d72b5ab` introduced `flows` as top-level integration_type with 14 cards.
477
-- Draft PR netdata/netdata#22439 opened against master.
478
-
479
-### 2026-05-07 (later) - autocomplete bug fix
480
-
481
-User reported autocomplete dropdown was useless for AS_NAME searches: typing "Akamai" returned no results because every value is rendered as `AS{n} {Organisation}` and the backend was prefix-matching only.
482
-
483
-Investigation traced the bug to `facet_runtime/store.rs::TextValueStore::prefix_matches` and `facet_runtime/sidecar.rs::search_sidecar`, both using `starts_with`. Fix landed on the same branch with three rounds of codex review:
484
-
485
-- Round 1: introduced substring matching for text-typed facets, kept prefix for IP/numeric. Codex flagged: async runtime blocking, broader-than-AS_NAME effect, naive substring + no length cap, stale docs.
486
-- Round 2: per-field policy via `AutocompleteMatchKind { Prefix, Substring }` on `FacetFieldSpec` (so future per-field overrides are possible without per-kind churn), `memchr::memmem::Finder` for substring search, 256-byte term cap, autocomplete moved to `spawn_blocking`, stale docs updated. Codex flagged blocker: term cap applied to all modes, not just autocomplete.
487
-- Round 3: term cap scoped to `mode == Autocomplete`. Regression test added for non-autocomplete long term. Codex returned "ready to ship for the reviewed autocomplete scope".
488
-
489
-Architectural rule recorded by user: "This affects autocomplete only, not regular facets matching. Only autocomplete. Because we want key=value or key in values, to use indexes, not scan." Verified in code: substring path is unreachable from selections/filters; only `mode=autocomplete` calls `FacetRuntime::autocomplete`.
490
-
491
-Commit: `b733037a`. 9 new tests, full crate 427 passed.
492
-
493
-Follow-ups (recorded in Followup section): case-sensitive matching today (Akamai vs akamai); autocomplete substring on FST sidecars is bounded by limit early-exit but still streams keys for rare/no-hit terms over very large archived vocabularies.
494
-
495
-### 2026-05-07 (later) - documentation enhancements
496
-
497
-- Screenshots from user (7 GitHub asset URLs) embedded across `summary-sankey.md` (2), `time-series.md` (1), `maps-globe.md` (4). Commit `26f8b978`.
498
-- Master alphabetical field index added at the bottom of `field-reference.md`. 91 rows. Each row carries: type, per-protocol availability (✓/◐/—), source class (decoder / enrichment / both), tier preservation (raw / all), selectivity (facet, group-by, filter, metric, time, hidden), and the enrichment chain or IE mapping. Subagent built the data set from code (rollup field defs, RAW_ONLY_FIELDS, facet catalog, decoder IE maps).
499
-- `docs/network-flows/visualization/filters-facets.md` and `docs/network-flows/retention-querying.md` updated to describe the per-field autocomplete policy and the autocomplete-vs-selection distinction.
500
-
501
-### 2026-05-07 (later) - schema fix and learn PR
502
-
503
-- CI surface: `check-documentation` job rejected the netdata PR because `docs/.map/map.schema.json` enum did not yet include `flows` as an `integration_kind`. Schema extended; commit `2c3ab0fc`. CI then turned green for that job.
504
-- Learn PR (netdata/learn#2854) opened to teach `ingest/ingest.py` how to (a) categorise files at `src/crates/netflow-plugin/integrations/<slug>.md` into a new `flows_entries` DataFrame, (b) splice them over the `flows_integrations` placeholder. Mirrors the existing logs handler. Verified locally: 65 Network Flows rows are spliced into `ingest/generated_map.yaml` under the four sub-categories; no `flows_integrations` placeholder remains; ingest exits 0. PR merged by user.
505
-
506
-### 2026-05-07 (later) - SOW close
507
-
508
-- Re-run check on netdata PR #22439 confirmed `check-documentation` passes (after learn merge plus our schema fix).
509
-- Remaining red CI checks (Codacy, SonarCloud, Build Windows) verified unrelated:
510
- - Codacy: 1107 markdownlint findings, 100% style-only (MD013 line-length, MD033 inline HTML, MD045 alt text). Pre-existing baseline; recently merged PR #22432 also "fails" Codacy. Project does not gate on this.
511
- - SonarCloud: same pattern, ignored as gate.
512
- - Build Windows: `urllib HTTPError 403: rate limit exceeded` during packaging step. GitHub API rate limit, transient/infrastructural; unrelated to this work.
513
-- User decision 2026-05-07: Phase 1C (AI skills cross-links to learn docs) is not mandatory. Acceptance criterion for Phase 1C is rejected with this reasoning recorded in Validation. SOW moved to `done/`.
514
-
515
-## Validation
516
-
517
-### Acceptance criteria evidence
518
-
519
-- **metadata.yaml validated**: 14 modules under `plugin_name: netflow-plugin` (3 sources + 4 IP intelligence + 2 BGP routing + 5 network identity). Pipeline runs `gen_integrations.py` + `gen_docs_integrations.py` to exit 0; 14 generated `.md` files under `src/crates/netflow-plugin/integrations/`.
520
-- **Integrations pipeline**: `flows` rendered correctly under its own type. `integrations.json`, `integrations.js`, `COLLECTORS.md` updated. `integrations.json` carries 14 entries with `integration_type: flows` distributed across 4 sub-categories.
521
-- **Learn section**: `docs/.map/map.yaml` carries the Network Flows top-level section with `integration_placeholder integration_kind: flows`. Schema (`docs/.map/map.schema.json`) accepts the value. Learn `ingest/ingest.py` (PR #2854, merged) routes the markdown files into `flows_entries` and splices them over the placeholder. Local ingest run produced 65 Network Flows rows with no remaining `flows_integrations` placeholder.
522
-- **Style guide**: pages are second person, active voice, sentence case. No `:::type` admonitions used; markdown-only by user direction.
523
-- **Field reference**: 91 fields documented by category plus a master alphabetical index with type, per-protocol availability, source class, tier preservation, selectivity, and enrichment chain per row.
524
-- **Enrichment docs**: GeoIP / static metadata / sampling / static networks / classifiers / ASN resolution / BMP routing / BioRIS / Network Sources / decapsulation. BMP, BioRIS, Network Sources documented based on unit-tested parsing logic; their runtime I/O paths still lack integration tests (followup).
525
-- **No mention of untested features**: verified via grep -- no references to topology drilldown, pcap (only as a debugging tool name), eBPF, or threat analytics.
526
-- **Sizing / Capacity planning**: `docs/network-flows/sizing-capacity.md` sourced from `.local/audits/netflow-bench/results.jsonl` (Phase 1.0) and `.local/audits/netflow-bench/storage-{low,high}.json` (Phase 1.0b). Includes the 20-25k flows/s headline, multi-thread CPU semantics note, write-amplification numbers, dedup ratio.
527
-- **Visualisation docs**: 5 pages -- `summary-sankey.md`, `time-series.md`, `maps-globe.md`, `filters-facets.md`, `dashboard-cards.md`. Maps and globe consolidated into one page since they share the data path.
528
-- **Screenshots**: 7 user-provided GitHub asset URLs embedded across the visualisation pages.
529
-
530
-### Reviewer findings (codex)
531
-
532
-Three rounds of read-only review over the autocomplete fix. Round 1 surfaced async runtime blocking, broader-than-AS_NAME effect, naive substring, stale docs. Round 2 surfaced a blocker (term cap was global, should be autocomplete-only). Round 3 returned "ready to ship for the reviewed autocomplete scope" with no new blockers.
533
-
534
-### Same-failure search
535
-
536
-- Other text facets affected by the same prefix-only autocomplete bug: `EXPORTER_NAME`, `IN_IF_DESCRIPTION`, `*_NET_NAME`, `SRC_MAC`, `DST_MAC`, `DST_AS_PATH`, `DST_COMMUNITIES`, country/state/city. Per-field policy fixes them all in the same pass.
537
-- Other places stale "prefix" or "in-memory" claims about autocomplete: `filters-facets.md:38`, `retention-querying.md:104` -- both updated.
538
-- Other places where the project might gate substring on autocomplete vs selection: confirmed selections use exact equality (`FacetStore::contains_value_ref`); substring path is unreachable from filtering.
539
-
540
-### Artifact maintenance gate
541
-
542
-- **AGENTS.md**: no update needed.
543
-- **Runtime project skills**: `.agents/skills/integrations-lifecycle/` updated -- new `per-type-matrix.md` Learn-routing matrix and `how-tos/adding-new-integration-type.md` 8-place checklist. `INDEX.md` cross-linked.
544
-- **Specs**: no spec update needed; project is incrementally bootstrapped and netflow-plugin specs were not pre-existing.
545
-- **End-user / operator docs**: this IS the docs update -- ~25 pages under `docs/network-flows/` plus the netflow-plugin README benchmark refresh.
546
-- **End-user / operator skills (Phase 1C)**: REJECTED by user 2026-05-07 with reasoning "skills linking to docs is not mandatory". `query-netdata-cloud/query-flows.md` and `query-netdata-agents/query-flows.md` remain at their pre-SOW state. Cross-linking to learn docs can be added in a future skills-maintenance pass without blocking this SOW. NOT tracked as a follow-up SOW because it is not a deferred feature -- it is an explicit scope rejection.
547
-- **SOW lifecycle**: status moved to `completed`, file moved to `done/`, in the same commit as the autocomplete fix lands on the active PR branch.
548
-
549
-### Status / directory consistency
550
-
551
-Status: `completed`. Directory: `done/`. Filename unchanged.
552
-
553
-### Lessons captured
554
-
555
-See `## Lessons Extracted` below.
556
-
557
-## Outcome
558
-
559
-Delivered:
560
-
561
-- 4 commits on netdata branch `netflow-plugin-docs-and-bench` (PR #22439, draft):
562
- - `c708101e` per-protocol benchmark + storage footprint test
563
- - `1455d59f` Network Flows documentation rewrite (~25 pages)
564
- - `e61c72b7` netflow-plugin metadata.yaml (3 modules) and integration cards
565
- - `6d72b5ab` `flows` top-level integration_type with 14 cards
566
- - `b733037a` substring autocomplete on text facets (3 codex rounds, 9 new tests, 427 pass)
567
- - `2c3ab0fc` `docs/.map` schema accepts `flows`
568
- - `26f8b978` screenshots and master field index
569
-- 1 commit on learn branch `netflow-flows-integrations` (PR #2854, MERGED 2026-05-07):
570
- - `de62daaf` ingest: route flows integrations into the Network Flows section
571
-- Companion website branch `netflow-flows-content` carries content corrections (separate, smaller).
572
-
573
-Pending only:
574
-
575
-- Re-trigger Build Windows on netdata PR #22439 (transient `urllib` HTTP 403 rate limit, not our code).
576
-- Mark netdata PR #22439 ready for review when user decides.
577
-- Cloud-frontend "Update integrations.js" PR -- standard manual sync, not gated by this SOW.
578
-
579
-## Lessons Extracted
580
-
581
-- **Code is ground truth, not the existing markdown**: the inherited netflow docs were thin and contained inaccuracies. Re-verifying every claim against `src/crates/netflow-plugin/` paid off -- multiple "well-documented" behaviours (e.g. AS name format, sFlow VLAN provenance, "single-core" benchmark framing) turned out to be wrong or stale.
582
-- **Doubling/mirroring is foundational, not a footnote**: users cannot reason about ANY aggregate number unless they understand that one router watching ingress + egress doubles every flow. This had to be the first concept on the Overview, not an "advanced" sidebar.
583
-- **Subagent per feature, master assistant for synthesis**: spawning a read-only subagent per enrichment module / visualisation / source kept the master context clean. The Overview was written last as a natural index of established truths -- not first as a promise the detail pages later contradicted.
584
-- **Per-field beats per-kind for policy that touches UX**: the autocomplete fix initially used per-kind dispatch (Text vs others). Codex pushed for per-field. The right answer was a `FacetFieldSpec::autocomplete_match` field that defaults from kind but allows future per-field overrides without churn.
585
-- **Reviewer iterations are non-optional**: codex flagged a real blocker on round 2 (term cap applied to all modes, not just autocomplete). One round of review would have shipped that bug. The project rule "iterate until reviewers cannot find anything else" is load-bearing.
586
-- **Match the codebase's own conventions over generic style**: substring autocomplete matches what `libnetdata/facets/facets.c:1783` already does for systemd-journal FTS (`SIMPLE_PATTERN_SUBSTRING`). Consistent with the project, not novel.
587
-- **Separate benchmarks: per-protocol resource envelope vs storage footprint**: a single "resource benchmark" couldn't answer both "what's the ingest cost?" and "what's the on-disk cost?". Splitting them produced two complementary tables and removed a bogus `bytes/flow x time` calculation that ignored journal indexing and dedup.
588
-- **Architectural pivots happen mid-SOW**: the original plan placed flow integrations under `data-collection.networking` as collector-typed cards. Mid-execution the user reframed the test as "would users ask 'Does Netdata integrate with X?'", which shifted the answer to a top-level `flows` integration_type with 14 cards. Captured as Decision 10 rather than retconning earlier decisions.
589
-
590
-## Followup
591
-
592
-Open follow-ups, ordered by priority:
593
-
594
-1. **Cloud-frontend `Update integrations.js`** -- copy the regenerated `integrations.js` from netdata into `cloud-frontend/src/domains/integrations/data/integrations.js` and open the standard sync PR. Not gated by this SOW. Last manual refresh was 2024-10-21; this work won't appear in the dashboard's Integrations modal until that file is updated.
595
-2. **Phase 2 SOW: integration tests for runtime I/O paths**:
596
- - BMP listener: async/tokio tests for TCP accept loop, framed decode, `apply_update` trie wiring, malformed message error accumulation, retry/shutdown. Test with a real BMP speaker, measure ingest impact, validate enrichment correctness.
597
- - BioRIS: async tests for gRPC client connection, RIB dump stream, retry/backoff. Build a local RIS daemon for end-to-end validation. Test `build_endpoint_uri`, `parse_router_ip`.
598
- - Network Sources: async tests for HTTP fetch cycle, service loop, prefix matching integration, failed HTTP handling, header forwarding, multi-source merge/re-publish.
599
- - All three: integration tests that wire parsed data through `DynamicRoutingRuntime` trie into flow enrichment lookup.
600
-3. **BMP architectural decision**: enrichment-only in netflow-plugin (Akvorado pattern) vs a separate BGP monitoring plugin with its own DB. This is product-level, not a code change.
601
-4. **Topology drilldown**: the `useFlowsDrilldownData` hook is dead code today. Implement the actor-modal hook when a UX home is decided.
602
-5. **Autocomplete follow-ups (deferred from autocomplete bug fix)**:
603
- - Case-insensitive matching for text facets (today: typing `akamai` will not match `AS20940 Akamai International`). UX call.
604
- - Substring scan over very large archived FST sidecars is bounded by `FACET_AUTOCOMPLETE_LIMIT` early-exit but still streams keys for rare/no-hit terms. Token-prefix or n-gram indexing can be added if measurements warrant.
605
-6. **Health alerts (`health.d/`)**: deferred. The netflow-plugin emits its own self-monitoring metrics (parse errors, decoder latency, ingest queue depth); alerts on those have not yet been authored.
606
-7. **AI skills cross-links to learn docs**: REJECTED for this SOW (not mandatory per user 2026-05-07). Can be picked up in a future skills-maintenance pass; not tracked as a separate SOW because rejected, not deferred.
607
-
608
-## Regression Log
609
-
610
-Note: dated regression entries intentionally use `## Regression - YYYY-MM-DD`
611
-headings to match the repository SOW lifecycle contract.
612
-
613
-## Regression - 2026-05-07
614
-
615
-### What broke
616
-
617
-The user previewed the learn netlify deploy at
618
-`https://deploy-preview-2852--netdata-docusaurus.netlify.app/`
619
-and found the documentation contains multiple statements that
620
-contradict the source code, several invented behaviours that
621
-were never verified, and structural choices that read as
622
-academic / generic flow-monitoring advice rather than as a
623
-practical guide to Netdata's flow plugin specifically. The
624
-prior closure on 2026-05-07 claimed every behavioural claim had
625
-been verified against the code; that claim is false.
626
-
627
-### Why previous validation missed it
628
-
629
-1. Subagent investigations produced data extracts (fields,
630
- tier preservation, IE maps) accurately but did NOT catch
631
- behavioural framing claims. Behavioural claims live in
632
- absences (no code says "do this") and were imported from
633
- the research notes at `.agents/knowledge/Network Traffic
634
- Analysis with Flow Data.md` as Netdata-specific without
635
- verifying against actual code paths.
636
-2. Three rounds of codex review focused on the autocomplete
637
- code change. None reviewed the documentation prose.
638
-3. Validation evidence was structural (grep for forbidden
639
- topics, count of pages, pipeline exit codes) rather than
640
- semantic (every behavioural claim cited to code). The
641
- "code is the source of truth" rule from Decision 9 was
642
- stated but not enforced per claim.
643
-4. SOW closure was driven by "all phases done" rather than
644
- "all claims true". The completion was premature.
645
-
646
-### Findings (verbatim from user, plus code-citation verdict)
647
-
648
-These findings must be addressed one at a time, no batching.
649
-For each: investigate against code, surgical edit, grep the
650
-rest of the docs for the same pattern, fix every instance.
651
-Each finding gets its own SOW execution-log entry below with
652
-file:line and the code citation that proves the fix.
653
-
654
-#### F1 -- /docs/network-flows landing page shows tiles not Overview
655
-
656
-> this page should be the overview, it appears on click of
657
-> the "Network Flows" main menu. Instead it shows all
658
-> integrations as tiles:
659
-> https://deploy-preview-2852--netdata-docusaurus.netlify.app/docs/network-flows
660
-
661
-Need to investigate how learn renders the section root.
662
-Likely cause: section root has no leaf content, so the auto-
663
-grid renderer in `learn/ingest/ingest.py:get_dir_make_file_and_recurse`
664
-generates a category index. Repair: ensure the Overview file
665
-is the section landing page, not the category-index grid.
666
-
667
-#### F2 -- Overview false bidirectional symmetry claim
668
-
669
-> > When you see "traffic from your country to a foreign
670
-> > country" and "traffic from that foreign country to your
671
-> > country" of similar volume, you're looking at one
672
-> > conversation, not two.
673
->
674
-> The statement "of similar volume" is wrong. The two
675
-> directions of traffic are usually not expected to have
676
-> similar volume. It can be omitted and the phrase must be
677
-> normalized because it may or may not be bidirectional.
678
-
679
-Repair: rephrase the doubling explanation to drop "similar
680
-volume" framing. The doubling effect is about per-router
681
-ingress+egress accounting, NOT about traffic symmetry. Same
682
-underlying conversation can have very different byte counts
683
-in each direction.
684
-
685
-#### F3 -- Overview false "in one direction" advice
686
-
687
-> > To see real numbers: filter by one exporter, one
688
-> > interface, in one direction. The dashboard makes this
689
-> > easy. See the Anti-patterns page for the full framing.
690
->
691
-> "in one direction" is wrong. Traffic does not double when
692
-> viewing bidirectional traffic in one interface, because
693
-> traffic is not usually symmetrical. It needs rephrasing.
694
-
695
-Repair: rephrase. The correct framing for "see real numbers"
696
-is per-exporter + per-interface scope; direction filtering
697
-is not the doubling fix.
698
-
699
-#### F4 -- Sampling rate framing wrong (uniform-rate myth)
700
-
701
-> > This works correctly only if all your exporters use the
702
-> > same sampling rate.
703
->
704
-> No! This is totally wrong. The way netdata does it, is it
705
-> that it gets the specific sampling each flow has and
706
-> multiplies the traffic of the specific flow to find its
707
-> actual. This works even if each router and each interface
708
-> has its own sampling rate.
709
->
710
-> The "works only if all your routers have the same
711
-> sampling rate" is a misconception from when we were
712
-> discussing this:
713
->
714
-> - you: the dashboard must show the sampling rate on any
715
-> view
716
-> - me: this cannot be done reliably when flows with mixed
717
-> sampling rates are aggregated on the dashboard, and
718
-> netdata does the right thing to no show it, because:
719
-> - a) if all your routers have the same sampling, you
720
-> know it already
721
-> - b) if you have mixed sampling rates, it is technically
722
-> impossible to provide a meaningful single sampling
723
-> rate for the aggregation
724
-> So, netdata multiplies at the source, so that
725
-> aggregations are as accurate as possible, even with
726
-> mixed sampling rates
727
-
728
-Code evidence:
729
-`src/crates/netflow-plugin/src/decoder/record/core/record.rs:24-26`
730
-multiplies `bytes` and `packets` by each record's own
731
-`sampling_rate` at decode time. Mixed sampling rates across
732
-exporters or interfaces are handled correctly automatically.
733
-
734
-Repair: remove the uniform-rate-required framing wherever it
735
-appears, replace with the correct per-flow-multiplication
736
-explanation, drop any UI claim about showing a single
737
-sampling rate (it would be meaningless under mixed rates).
738
-
739
-#### F5 -- Sampling rate "clean path" recommendation wrong
740
-
741
-> > The clean path: keep sampling rates uniform across your
742
-> > network, or run unsampled where the flow rate allows
743
->
744
-> No. I never said that. It is not the clean path. People
745
-> should use the sampling rates according to their use
746
-> cases. But netdata multiplies at ingestions and does not
747
-> show sampling rates on the UI.
748
-
749
-Repair: remove the "clean path" recommendation. Netdata does
750
-the right thing regardless of whether sampling is uniform or
751
-mixed.
752
-
753
-#### F6 -- Globe view "less useful for analysis" wrong
754
-
755
-> > Globe -- a 3D rendering of the city-level data. Visual
756
-> > demo, less useful for analysis.
757
->
758
-> "less useful for analysis"? Why? The information is
759
-> exactly the same with the map. There is a table, like in
760
-> maps. What makes it less useful? That is 3d? The opposite
761
-> I think.
762
-
763
-Repair: rewrite the globe section to drop the
764
-"less useful for analysis" judgement. Same data, same table,
765
-same selectivity; the 3D projection is one of several
766
-valid presentations.
767
-
768
-#### F7 -- Installation tab location wrong
769
-
770
-> In installation:
771
->
772
-> > The Network Flows tab should appear in the top
773
-> > navigation
774
->
775
-> No. The flows functions in in the "Live" top menu
776
-> currently.
777
-
778
-Repair: correct the location to "Live" menu.
779
-
780
-#### F8 -- Configuration: tier sizing should be per-tier only
781
-
782
-> Retention size should be set per tier, like:
783
->
784
-> tiers:
785
-> raw: { size: 10GB, duration: 24h }
786
-> etc
787
->
788
-> These globals must be removed:
789
->
790
-> size_of_journal_files: 10GB
791
-> duration_of_journal_files: 7d
792
->
793
-> It is very important to be able to size tiers
794
-> independently of each other. There is no one size fits
795
-> all.
796
-> I know there are globals and overrides per tier, but come
797
-> on. Why double configuration?
798
-
799
-This finding has TWO parts:
800
-
801
-1. Documentation: stop documenting the globals; show only
802
- per-tier `tiers: { raw: {size, duration}, ... }`.
803
-2. Code: remove `size_of_journal_files` and
804
- `duration_of_journal_files` from the schema.
805
-
806
-Code reference today:
807
-`src/crates/netflow-plugin/src/plugin_config/types/journal.rs:21-37`
808
-declares both globals AND per-tier overrides. The user wants
809
-the globals dropped from the configuration schema entirely.
810
-
811
-#### F9 -- Configuration: query_1m_max_window / query_5m_max_window unjustified
812
-
813
-> About these:
814
->
815
-> query_1m_max_window: 6h
816
-> query_5m_max_window: 24h
817
->
818
-> What are these and why they are needed? I don't
819
-> understand. Either the query engine is half based, or
820
-> these are useless overprotections that are never needed.
821
-
822
-Repair: investigate purpose in code. If they are real
823
-protections, document the protection clearly. If they are
824
-useless, remove them from both code and docs.
825
-
826
-#### F10 -- Configuration: query_max_groups / query_facet_max_values_per_field unjustified
827
-
828
-> I don't understand what are these and why are needed and
829
-> what value or protection they provide:
830
->
831
-> query_max_groups: 50000
832
-> query_facet_max_values_per_field: 5000
833
->
834
-> Explain
835
-
836
-Repair: same as F9 -- investigate, document the protection
837
-or remove.
838
-
839
-#### F11 -- Empty page: enrichment-concepts/ip-intelligence
840
-
841
-> empty page:
842
-> https://deploy-preview-2852--netdata-docusaurus.netlify.app/docs/network-flows/enrichment-concepts/ip-intelligence
843
-
844
-Investigate why the page renders empty. Source file
845
-`docs/network-flows/enrichment/ip-intelligence.md` exists.
846
-Likely a generated MDX issue (frontmatter, fence, or special
847
-char) or an ingest path mismatch.
848
-
849
-#### F12 -- Retention and Querying structure wrong; URL sharing irrelevant
850
-
851
-> Retention and Querying has a section called "URL
852
-> sharing"? Really? You find this relevant?
853
-> If you need to put generic visualization rules, these
854
-> should be a generic "Visualization/Overview" page, to
855
-> explain FTS, sharing, grouping, etc. For sure Retention
856
-> is closer to configuration and querying is closer to
857
-> visualization.
858
-
859
-Repair has THREE parts:
860
-
861
-1. Remove the "URL sharing" section from
862
- retention-querying.md.
863
-2. Move retention-side content (tier sizing rules, retention
864
- knobs) closer to Configuration.
865
-3. Move query-time semantics (tier auto-pick rules,
866
- query-engine behaviour, FTS, sharing, grouping) into a
867
- new Visualization/Overview page.
868
-
869
-#### F13 -- Sizing/Capacity Planning wrong genre, wrong content
870
-
871
-> Sizing and Capacity planning is written like an academic
872
-> paper that must prove productivity of the testing
873
-> environment. People want sizing and planning directions.
874
-> This is not an academic paper, not a blog.
875
->
876
-> What are the requirements for this page:
877
->
878
-> - what is the cap of the plugin
879
-> - how ingestion rate affects storage
880
-> - the raw tier monopolizes storage - do not let it
881
-> explode - it will need fast nvme disks to query it.
882
-> - journal backend uses free system memory as system
883
-> caches - the bigger the database, the more free memory
884
-> the system would need.
885
-> - journal is fully indexed, all fields are indexed, but
886
-> FTS means full scan.
887
-> - explain that 25k flows/s sustained approaches ISP level
888
-> capacities.
889
-> - use the distributed nature of netdata. The plugin can
890
-> be installed multiple times, in branch offices,
891
-> different data centers, etc. And since aggregation
892
-> across routers is usually meaningless for flows, users
893
-> can appoint 1 netdata per router. There is no need to
894
-> push all flows to one central place.
895
->
896
-> So, this page should provide a practical guide for users
897
-> to scale the plugin, the servers and storage it runs,
898
-> etc.
899
-> Remove the benchmarks and tests from this page. The
900
-> benchmarks were for us, not for the customers.
901
-
902
-Repair: rewrite sizing-capacity.md from scratch as a
903
-practical scaling guide following the seven bullets above.
904
-Drop all benchmark numbers, drop the "academic paper" framing.
905
-
906
-#### F14 -- Validation: invented user-side risks
907
-
908
-> In Validation and Data Quality:
909
->
910
-> Risks:
911
-> - Netdata monitors UDP port overflows and has alerts for
912
-> it.
913
-> - "Sampling rate misinterpretation" how is this a risk
914
-> for users? This is bug in netdata if it happens.
915
-> - "Sampling rate change" how is this a risk for users?
916
-> Netdata ensures this will not happen because ingestion
917
-> scales on sampling received
918
-> - "Template loss after collector restart" how is this a
919
-> risk for users? Netdata saves templates and reloads
920
-> them
921
->
922
-> I think the entire "Validation and Data Quality" is
923
-> completely off. It mentions again sampling rates, etc.
924
-> It is like it was written by someone that does not have
925
-> a clue of what netdata is and how the plugin works.
926
-
927
-Code evidence:
928
-- Templates persist:
929
- `src/crates/netflow-plugin/src/decoder/protocol/v9/templates.rs:106`
930
- and
931
- `src/crates/netflow-plugin/src/decoder/protocol/ipfix/templates/data.rs:67`
932
-- Sampling multiplied per-flow at decode (see F4).
933
-
934
-Repair: rewrite validation.md from scratch. Remove the
935
-sampling-related "risks", remove template-loss "risk", point
936
-UDP overflow concern at Netdata's existing alerts.
937
-
938
-#### F15 -- Anti-patterns: "Ignoring the sampling rate" is bogus
939
-
940
-> Anti-patterns page:
941
->
942
-> > Ignoring the sampling rate
943
->
944
-> How is it possible for users to ignore the sampling rate
945
-> if we calculate the estimated volume at ingestion? You
946
-> invented reasons for it: "so the dashboard numbers are
947
-> estimates of actual traffic -- if the multiplication is
948
-> consistent"
949
->
950
-> What? How the multiplication cannot be consistent? What
951
-> are you talking about?
952
->
953
-> "Ignoring the sampling rate" section must be removed.
954
-
955
-Repair: remove the entire section.
956
-
957
-#### F16 -- Anti-patterns: "GeoIP for internal IPs" is invented
958
-
959
-> > "Internal IPs (10.x, 172.16-31.x, 192.168.x) appear in
960
-> > random countries on the geographic map"
961
->
962
-> What? Where did you find this? Geolocation does not
963
-> position internal IPs on the map.
964
->
965
-> "Trusting GeoIP for internal IPs" section must be removed.
966
-
967
-Repair: remove the entire section. Need to verify in code
968
-that internal IPs are NOT placed on the map -- if there is
969
-any path that does, that's a Netdata bug to file separately,
970
-not a user-side anti-pattern.
971
-
972
-#### F17 -- Anti-patterns: "Alerting on absolute volume" doesn't apply
973
-
974
-> > "Alerting on absolute volume thresholds"
975
->
976
-> Netdata does not support alerting of flows yet. Remove
977
-> this section.
978
-
979
-Repair: remove the entire section.
980
-
981
-#### F18 -- Troubleshooting: wrong journalctl namespace
982
-
983
-> Troubleshooting page:
984
->
985
-> Netdata logs in namespace 'netdata'. Journalctl needs
986
-> `--namespace netdata`.
987
-
988
-Repair: every `journalctl` invocation must include
989
-`--namespace netdata`. Grep all docs.
990
-
991
-#### F19 -- Troubleshooting: cumulative misconceptions
992
-
993
-> This page has a mix of all the above issues: sampling,
994
-> geoip, etc.
995
-
996
-Repair: this is a per-claim sweep informed by the fixes
997
-above. Every behavioural claim on the troubleshooting page
998
-must be cited to code or removed.
999
-
1000
-#### F20 -- Section title: "Enrichment Concepts" wrong
1001
-
1002
-> "Encrichement Concepts" is a wrong title. "Flows
1003
-> Enrichement" is the right one.
1004
-
1005
-Repair: rename the sub-section "Enrichment Concepts" to
1006
-"Flows Enrichment" everywhere it appears: `docs/.map/map.yaml`,
1007
-in any cross-references in pages, and in any sidebar /
1008
-breadcrumb labels that derive from the map.
1009
-
1010
-#### F21 -- Section title: "Sources" wrong
1011
-
1012
-> "Sources" is too generic. "Flow Protocols" is the right
1013
-> one.
1014
-
1015
-Repair: rename the sub-section "Sources" to "Flow Protocols"
1016
-everywhere: `docs/.map/map.yaml`, plus every cross-reference
1017
-that points at `Sources/NetFlow`, `Sources/IPFIX`,
1018
-`Sources/sFlow` (those individual page labels stay).
1019
-
1020
-### Repair plan
1021
-
1022
-**Phase R1 -- Per-finding fixes, one at a time, no batching.**
1023
-For each finding F1..F21 above:
1024
-
1025
-1. Investigate against code -- read the relevant source
1026
- files; record file:line evidence in this SOW under the
1027
- per-finding execution-log entry.
1028
-2. Surgical edit -- minimal diff that fixes only that
1029
- claim.
1030
-3. Grep the rest of the docs for the same pattern; fix
1031
- every instance with the same surgical care.
1032
-4. Append a dated execution-log entry naming what changed,
1033
- why, and the code citation.
1034
-
1035
-No finding is "deferred". Every one gets fixed before the
1036
-SOW can re-close.
1037
-
1038
-**Phase R2 -- Per-page audit subagents.**
1039
-
1040
-After Phase R1 completes, spawn one read-only audit
1041
-subagent per page. Each subagent's brief:
1042
-
1043
-- Read the entire page line by line.
1044
-- For every behavioural / configuration / vendor /
1045
- protocol claim, verify it against the source code at the
1046
- cited paths in this repository.
1047
-- For every third-party vendor configuration mention
1048
- (Cisco IOS-XE / IOS-XR, Juniper JunOS, FRR, Palo Alto,
1049
- Mikrotik, Zyxel, etc.), verify against the upstream
1050
- vendor's current documentation by web fetch.
1051
-- Flag every claim that cannot be anchored, every
1052
- generic-flow-monitoring sentence that contradicts how
1053
- Netdata actually works, and every vendor command that
1054
- does not exist or has wrong syntax.
1055
-- Output a structured finding list: claim, location,
1056
- evidence, severity, suggested fix.
1057
-
1058
-The master assistant synthesises the findings, applies
1059
-surgical fixes, and re-spawns the auditor on the same page.
1060
-Iterate until the auditor returns "no findings".
1061
-
1062
-**Pages in scope for Phase R2:**
1063
-- README.md (Overview)
1064
-- installation.md
1065
-- quick-start.md
1066
-- configuration.md
1067
-- field-reference.md
1068
-- retention-querying.md (post-restructure)
1069
-- sizing-capacity.md (post-rewrite)
1070
-- validation.md (post-rewrite)
1071
-- investigation-playbooks.md
1072
-- anti-patterns.md (post-section-removals)
1073
-- troubleshooting.md
1074
-- 7 enrichment pages: ip-intelligence, asn-resolution, bgp-routing, network-identity, static-metadata, classifiers, decapsulation
1075
-- 5 visualisation pages: summary-sankey, time-series, maps-globe, filters-facets, dashboard-cards
1076
-- 14 generated integration cards under `src/crates/netflow-plugin/integrations/` -- each has its own audit pass against `metadata.yaml` and against the upstream vendor documentation for any setup steps.
1077
-
1078
-**Plus a new visualisation/overview page** (per F12) and possibly a new visualisation/querying page if the F12 split lands as proposed.
1079
-
1080
-**Phase R3 -- Final close.**
1081
-
1082
-The SOW reopened with this regression note during repair. The Validation
1083
-section was appended (not replaced) with per-finding evidence and per-page
1084
-audit-clean evidence. Status returned to `completed` after all closure
1085
-criteria listed below were satisfied:
1086
-
1087
-- Every F1..F21 has a fix landed and a code citation in the
1088
- log.
1089
-- Every page has at least one auditor pass returning no
1090
- findings.
1091
-- A whole-section codex-style review of the final docs
1092
- returns no new findings.
1093
-
1094
-Then move file back to `done/`.
1095
-
1096
-### Repair execution log
1097
-
1098
-#### F1 -- 2026-05-07 -- section landing page rendering
1099
-
1100
-Investigation: Compared `# Network Flows` block in
1101
-`docs/.map/map.yaml` with sections that DO render an Overview
1102
-landing page on Learn (e.g. `Collecting Metrics`,
1103
-`Dashboards and Charts`, `Netdata Cloud`). All those carry
1104
-`edit_url:` directly on the section root meta block. Network
1105
-Flows root carried `label:` only; the README.md was instead
1106
-exposed as a CHILD entry labelled "Overview". With no leaf
1107
-content at the root, learn's
1108
-`get_dir_make_file_and_recurse` (`learn/ingest/ingest.py:2333-2510`)
1109
-auto-generates a category-grid index page at the section
1110
-URL, which is what the netlify preview rendered as tiles.
1111
-
1112
-Fix: hoisted `edit_url:` and `description:` to the Network
1113
-Flows root meta block. Removed the now-redundant child
1114
-"Overview" entry pointing to the same README.
1115
-
1116
-Other similar patterns in MY work: Sub-sections "Enrichment
1117
-Concepts" (line 498) and "Visualization" (line 529) within
1118
-Network Flows also lack root `edit_url`. Decision: leave as
1119
-auto-grid for now -- F1 was specifically about the section
1120
-root URL `/docs/network-flows`, and sub-section grids are an
1121
-accepted pattern across the rest of the project (e.g.
1122
-"Systemd Journal Logs" sub-section under Logs). If a future
1123
-finding flags `/docs/network-flows/flows-enrichment` or
1124
-`/docs/network-flows/flow-protocols` as needing real content,
1125
-revisit then.
1126
-
1127
-Patterns OUTSIDE my work (pre-existing): "Logs" section
1128
-root (line 567 in map.yaml) also lacks root `edit_url`. Out
1129
-of scope -- pre-existing structural choice.
1130
-
1131
-Evidence:
1132
-- Pattern reference: `docs/.map/map.yaml:65-68` (Welcome to Netdata),
1133
- `docs/.map/map.yaml:96-99` (Collecting Metrics).
1134
-- Renderer: `learn/ingest/ingest.py:2333-2510`
1135
- (`get_dir_make_file_and_recurse`).
1136
-
1137
-Diff: `docs/.map/map.yaml:479-484`.
1138
-
1139
-#### F2 + F3 -- 2026-05-07 -- doubling vs bidirectional symmetry
1140
-
1141
-These two findings address the same paragraphs and are fixed
1142
-together. The "doubling" effect (per-packet ingress+egress on
1143
-one router) was conflated with bidirectional traffic
1144
-symmetry, which is a different concept. Original docs told
1145
-users to (a) look for "similar volume" in opposite directions
1146
-to identify a single conversation, and (b) "filter by one
1147
-exporter, one interface, in one direction" to see real
1148
-volume. Both wrong:
1149
-
1150
-- Bidirectional traffic is typically asymmetric (downloads
1151
- vs ACKs). "Similar volume" is a wrong heuristic for
1152
- spotting two halves of the same conversation.
1153
-- "in one direction" added on top of "one interface" is
1154
- redundant for the doubling fix and misleads readers into
1155
- expecting another 50% halving from direction filtering.
1156
-
1157
-The doubling fix is just: one exporter + one interface
1158
-(Input Interface OR Output Interface, pick one). Each packet
1159
-crossing that interface produces exactly one record on it.
1160
-
1161
-Mirror-conversation framing rewritten: bidirectional
1162
-conversations produce separate records per direction
1163
-because they really are different packets going each way.
1164
-Volumes are usually asymmetric. Per-direction accounting
1165
-is correct, not duplication.
1166
-
1167
-Files touched:
1168
-- `docs/network-flows/README.md` -- `## Two things to know
1169
- on day one` paragraphs rewritten.
1170
-- `docs/network-flows/quick-start.md` -- step-3 paragraphs
1171
- rewritten.
1172
-- `docs/network-flows/visualization/summary-sankey.md:98`
1173
- -- "in one direction" removed from doubling fix.
1174
-- `docs/network-flows/anti-patterns.md:21` -- same fix in
1175
- "How to avoid it" line of the doubling anti-pattern.
1176
-- `docs/network-flows/validation.md:36, 55` -- same fix in
1177
- the SNMP cross-check and doubling sanity-check sections.
1178
-
1179
-Note: F14 / F15 / F16 / F17 will rewrite anti-patterns.md
1180
-and validation.md more comprehensively. The corrections
1181
-above are kept as standalone fixes so the wrong claim is
1182
-gone immediately even if the surrounding sections survive.
1183
-
1184
-Other patterns scanned: searched for "of similar volume",
1185
-"same volume", "symmetric", "in one direction",
1186
-"conversations? (?:are|look|appear)? mirrored". The lines
1187
-above are all hits within `docs/network-flows/`. No other
1188
-files carry the pattern.
1189
-
1190
-#### F4 + F5 -- 2026-05-07 -- sampling rate framing (uniform-rate myth)
1191
-
1192
-These two findings are about the same incorrect claim and
1193
-fixed together.
1194
-
1195
-Code evidence -- per-flow multiplication at decode time:
1196
-
1197
-```rust
1198
-// src/crates/netflow-plugin/src/decoder/record/core/record.rs:24-26
1199
-let sampling_rate = rec.sampling_rate.max(1);
1200
-rec.bytes = rec.bytes.saturating_mul(sampling_rate);
1201
-rec.packets = rec.packets.saturating_mul(sampling_rate);
1202
-```
1203
-
1204
-`sampling_rate` is set per-record from the relevant source:
1205
-- `decoder/protocol/legacy.rs:60` -- v5 header rate.
1206
-- `decoder/protocol/v9/records.rs:39, 215` and
1207
- `decoder/protocol/v9/sampling.rs:215` -- v9 IE on record
1208
- or Sampling Options Template (namespace-scoped).
1209
-- `decoder/protocol/ipfix/special/record.rs:21, 61, 171`,
1210
- `ipfix/record/state.rs:10, 38, 168`,
1211
- `ipfix/record/append.rs:56` -- IPFIX record IE / sampling
1212
- options.
1213
-- `decoder/protocol/sflow/record.rs:6, 19` -- sFlow per-sample
1214
- rate.
1215
-- `decoder/protocol/shared/merge/enrich.rs:80` -- merge with
1216
- static `override_sampling_rate` config.
1217
-
1218
-So mixed sampling rates across exporters / interfaces / time
1219
-are handled correctly: each record scales by its own rate.
1220
-The only failure mode is "exporter is sampling but the rate
1221
-is not communicated" (NetFlow v7 has no field; v5 with
1222
-rate=0; v9 / IPFIX without an attached Sampling Options
1223
-Template). That is a real concern and stays in the docs.
1224
-
1225
-Removed claims:
1226
-
1227
-- README.md lines 91-94 -- "works correctly only if all
1228
- your exporters use the same sampling rate" and "the
1229
- clean path: keep sampling rates uniform across your
1230
- network" -- both false. Rewrote the paragraph to state
1231
- per-flow multiplication, why the UI does not surface a
1232
- single rate, and the real statistical-floor caveat
1233
- (sampling can miss small/short flows regardless of how
1234
- uniform the rates are).
1235
-- field-reference.md line 33 -- `RAW_BYTES` description
1236
- said "Use when sampling is uniform across all your
1237
- exporters". Changed to "Use when you want the unscaled
1238
- value the exporter sent". Same fix in
1239
- anti-patterns.md:126 (the prose) and the summary table
1240
- row at line 150.
1241
-- troubleshooting.md line 129 -- "Mixed sampling rates
1242
- across exporters... isn't comparable to any single SNMP
1243
- measurement" replaced with the correct framing:
1244
- comparing aggregates to a single interface counter is
1245
- the actual mistake; per-flow multiplication is correct
1246
- regardless of rate uniformity.
1247
-- validation.md line 11 -- "undocumented sampling rate
1248
- changes" removed from the silent-failure list intro.
1249
-- validation.md line 116 -- the "Sampling rate change"
1250
- monitoring-table row removed.
1251
-- investigation-playbooks.md lines 128, 132 -- "Sampling
1252
- rate of the exporter (so the numbers can be
1253
- interpreted)" deliverable bullet removed; "A change in
1254
- sampling rate during the analysis window invalidates
1255
- the trend" caveat removed.
1256
-- anti-patterns.md line 132 -- "Same goes for
1257
- sampling-rate differences across exporters" removed
1258
- from the cross-protocol comparison section. The
1259
- protocol-counts-not-comparable point stays; the
1260
- uniformity claim goes.
1261
-
1262
-Items NOT touched in this finding (deferred to F14 / F15
1263
-which will rewrite their containing sections):
1264
-
1265
-- validation.md silent-failure list items #2 ("Sampling
1266
- rate misinterpretation"), #3 ("Sampling rate change"),
1267
- #5 ("Template loss after collector restart") -- F14
1268
- removes them as a block.
1269
-- anti-patterns.md section 2 ("Ignoring the sampling
1270
- rate") and the summary-table row "Ignored sampling" --
1271
- F15 removes the entire section.
1272
-
1273
-Files touched:
1274
-- docs/network-flows/README.md (lines 88-96)
1275
-- docs/network-flows/field-reference.md (line 33)
1276
-- docs/network-flows/troubleshooting.md (lines ~120-130)
1277
-- docs/network-flows/validation.md (lines 11, ~116)
1278
-- docs/network-flows/investigation-playbooks.md (lines 128-132)
1279
-- docs/network-flows/anti-patterns.md (lines 132, 140, 150, 126)
1280
-
1281
-Other docs scanned (`docs/network-flows/`) for the patterns
1282
-"uniform.*rate", "same.*sampling.*rate", "rates.*uniform",
1283
-"all.*exporters.*same.*rate", "sampling.*uniform",
1284
-"clean.*path.*sampling", "aggregate.*blend",
1285
-"blend.*estimate" -- all hits addressed above except the
1286
-F14 / F15 -targeted blocks.
1287
-
1288
-#### F6 -- 2026-05-07 -- Globe view "less useful for analysis"
1289
-
1290
-User: "the information is exactly the same with the map.
1291
-There is a table, like in maps. What makes it less useful?
1292
-That is 3d? The opposite I think."
1293
-
1294
-Code reality (per the city-map / globe code path that
1295
-shares the same backend response: see
1296
-`src/crates/netflow-plugin/src/query/request/constants.rs`
1297
-for `CITY_MAP_GROUP_BY_FIELDS` and the absence of any
1298
-"globe" view enum -- the globe re-renders the city-map
1299
-response): same data, same table, just a different
1300
-rendering.
1301
-
1302
-Removed claims:
1303
-
1304
-- README.md line 105 -- "Visual demo, less useful for
1305
- analysis" replaced with neutral framing that the globe
1306
- uses the same data + table as the city map and is the
1307
- better fit when distance / great-circle routes matter.
1308
-- visualization/maps-globe.md "Globe vs City Map"
1309
- paragraph -- removed the "less useful for analysis"
1310
- judgement; states the trade-off (2D for in-continent
1311
- precision; 3D for transcontinental / great-circle).
1312
-
1313
-Also fixed the "Mirroring" subsection on the same page:
1314
-the "25 top-N = 12 conversations" claim is the F2 symmetry
1315
-myth and was inconsistent with bidirectional traffic
1316
-typically being asymmetric. Reworded to state that
1317
-bidirectional traffic produces two separate flow records,
1318
-two distinct edges, with usually-different volumes.
1319
-
1320
-No other doc carries the "less useful for analysis"
1321
-phrasing. Grep clean.
1322
-
1323
-Files touched:
1324
-- docs/network-flows/README.md (line 105)
1325
-- docs/network-flows/visualization/maps-globe.md (lines ~85-91)
1326
-
1327
-#### F7 -- 2026-05-07 -- Network Flows is a Function under the Live tab
1328
-
1329
-Verified terminology against
1330
-`docs/dashboards-and-charts/live-tab.md:3-69`: Netdata's
1331
-official UI vocabulary calls it the **Live tab** in the top
1332
-navigation; it lists Functions; "Network Flows" is one of
1333
-those Functions. There is no top-nav "Network Flows" tab.
1334
-
1335
-Repair scope: every doc reference. Adopted convention:
1336
-
1337
-- Verb form: "Open Network Flows" (drop "tab")
1338
-- Noun reference to the workspace: "the Network Flows
1339
- view"
1340
-- Setup / installation context: "Click the **Live** tab in
1341
- the top navigation; **Network Flows** appears in the
1342
- Functions list" -- mirrors the official Live-tab doc.
1343
-
1344
-Files touched:
1345
-- docs/network-flows/installation.md (lines 113-121) --
1346
- the original bad sentence and follow-up.
1347
-- docs/network-flows/troubleshooting.md (lines 19, 47).
1348
-- docs/network-flows/investigation-playbooks.md (lines 11,
1349
- 21, 63, 109, 141) -- five "Open the Network Flows tab"
1350
- occurrences replaced with "Open Network Flows"; lead
1351
- paragraph at line 11 names the Live tab.
1352
-- docs/network-flows/anti-patterns.md (line 15).
1353
-- docs/network-flows/visualization/dashboard-cards.md
1354
- (lines 11, 13, 87, 97 -- four occurrences).
1355
-- docs/network-flows/visualization/summary-sankey.md
1356
- (line 11).
1357
-
1358
-Grep for "Network Flows tab", "Network Flows menu", "top
1359
-navigation" run after the sweep -- only the corrected
1360
-phrasings remain.
1361
-
1362
-#### F8 -- 2026-05-07 -- per-tier retention only; remove globals
1363
-
1364
-User: "It is very important to be able to size tiers
1365
-independently of each other. There is no one size fits all.
1366
-I know there are globals and overrides per tier, but come
1367
-on. Why double configuration?"
1368
-
1369
-Code investigation: the global `size_of_journal_files` and
1370
-`duration_of_journal_files` on `JournalConfig` were already
1371
-no more than per-tier defaults that flowed through
1372
-`retention_for_tier()` -- runtime semantics were already
1373
-per-tier. The "double configuration" was schema redundancy
1374
-with no underlying behavioural justification.
1375
-
1376
-Refactor (`src/crates/netflow-plugin/src/`):
1377
-
1378
-- `plugin_config/types/journal.rs`: removed
1379
- `size_of_journal_files` and `duration_of_journal_files` from
1380
- `JournalConfig`. Replaced
1381
- `Option<JournalTierRetentionConfig>` per-tier with the
1382
- struct directly carrying `Option<ByteSize>` /
1383
- `Option<Duration>` fields. Each tier defaults to
1384
- `Some(default)` when the YAML omits it; explicit `null`
1385
- disables the limit on that tier; both Nones is rejected by
1386
- validation. Removed the now-vestigial
1387
- `JournalConfig::default_retention()` and simplified
1388
- `retention_for_tier()` to a single per-tier lookup.
1389
- Built-in tier defaults stay at uniform 10GB / 7d to
1390
- preserve current default behaviour.
1391
-- `plugin_config/defaults.rs`: removed the dead
1392
- `RetentionLimitOverride<T>` enum, its
1393
- `is_inherit` / `resolve` impl, and the four
1394
- `(de)serialize_retention_override_*` helpers (no longer
1395
- reachable). Removed the now-orphan `parse_bytesize`
1396
- helper that fed the removed clap `value_parser`.
1397
-- `plugin_config.rs`: removed the
1398
- `pub(crate) use defaults::RetentionLimitOverride;`
1399
- re-export.
1400
-
1401
-Tests (mechanical updates to the new schema):
1402
-
1403
-- `memory_tests.rs`: collapsed four near-identical per-tier
1404
- override blocks into one `small_tier` config cloned across
1405
- the four tiers; replaced
1406
- `RetentionLimitOverride::Value(...)` with `Some(...)`;
1407
- dropped the now-dead `RetentionLimitOverride` import.
1408
-- `startup_memory_tests.rs`: dropped the global-retention
1409
- setters that already matched the built-in defaults.
1410
-- `plugin_config_tests.rs`: rewrote five tests to exercise
1411
- the new schema only:
1412
- `journal_tier_retention_uses_built_in_tier_defaults`
1413
- (was `inherits_global_defaults_when_no_overrides_exist`),
1414
- `journal_tier_retention_uses_per_tier_values_when_present`,
1415
- `journal_rotation_size_derives_from_tier_size_budget` (now
1416
- via `tiers.raw.size_of_journal_files`),
1417
- `journal_rotation_size_uses_100mb_for_time_only_retention`,
1418
- `journal_validation_allows_time_only_retention_when_size_is_disabled`,
1419
- `journal_tier_retention_null_disables_size_limit_for_that_tier_only`
1420
- (replaces the old "inherited size limit" YAML test).
1421
- Updated the YAML test fixture at line ~245 to drop the
1422
- global keys.
1423
-
1424
-Documentation:
1425
-
1426
-- `docs/network-flows/configuration.md`: rewrote the
1427
- `## journal` section. Single per-tier table only, no
1428
- "top-level retention" subsection, explicit note that
1429
- there are no global retention knobs. Updated the
1430
- production-retention example to a fully-per-tier form.
1431
-- `docs/network-flows/retention-querying.md`: dropped the
1432
- global-form example; replaced with the per-tier form;
1433
- cross-link to configuration.md.
1434
-- `docs/network-flows/sizing-capacity.md` (line 101):
1435
- already says "per-tier"; no change.
1436
-
1437
-Build + tests:
1438
-
1439
-- `cargo build --release` clean (3m02s).
1440
-- `cargo test --release --bin netflow-plugin` -- 427
1441
- passed; 0 failed; 18 ignored.
1442
-
1443
-Breaking change notice: any existing user config that
1444
-specified `journal.size_of_journal_files` or
1445
-`journal.duration_of_journal_files` at the top level of the
1446
-journal block will now fail to deserialize (strict
1447
-`deny_unknown_fields`). Users migrate by moving those values
1448
-into per-tier `tiers.<tier>.size_of_journal_files` /
1449
-`duration_of_journal_files`. Plugin is recently shipped
1450
-(PR #22439, 2026-05-07); breaking-change risk is low.
1451
-
1452
-Files touched:
1453
-- src/crates/netflow-plugin/src/plugin_config/types/journal.rs
1454
-- src/crates/netflow-plugin/src/plugin_config/defaults.rs
1455
-- src/crates/netflow-plugin/src/plugin_config.rs
1456
-- src/crates/netflow-plugin/src/memory_tests.rs
1457
-- src/crates/netflow-plugin/src/startup_memory_tests.rs
1458
-- src/crates/netflow-plugin/src/plugin_config_tests.rs
1459
-- docs/network-flows/configuration.md
1460
-- docs/network-flows/retention-querying.md
1461
-
1462
-#### F9 -- 2026-05-07 -- query_1m_max_window / query_5m_max_window are dead
1463
-
1464
-User: "What are these and why they are needed? I don't
1465
-understand. Either the query engine is half based, or these
1466
-are useless overprotections that are never needed."
1467
-
1468
-Code investigation: searched
1469
-`src/crates/netflow-plugin/src/` for any consumer outside
1470
-the config struct itself:
1471
-
1472
-- `plugin_config/types/journal.rs` -- declared.
1473
-- `plugin_config/validation/journal.rs:6-13` -- non-zero
1474
- validation; ordering check.
1475
-- `plugin_config/defaults.rs` -- defaults.
1476
-- `plugin_config_tests.rs` -- two YAML fixtures only.
1477
-
1478
-NO consumer in `query/`, `query/planner/`, or anywhere else
1479
-reads these values. The actual tier auto-pick logic lives in
1480
-`query/planner/spans.rs:plan_query_tier_spans_recursive`,
1481
-which selects the coarser tier strictly from window /
1482
-bucket-duration alignment math; it does not consult
1483
-`query_1m_max_window` or `query_5m_max_window`.
1484
-
1485
-Verdict: dead config knobs. The user's "useless
1486
-overprotections" assessment was correct.
1487
-
1488
-Repair: removed both fields from the schema and validation,
1489
-plus the two YAML test fixtures, plus the configuration
1490
-table row that documented them. Updated the
1491
-retention-querying explanation of "skip a tier when window
1492
-is too wide" to reflect the actual behaviour: the planner
1493
-picks the coarsest aligned tier without separate config-
1494
-driven window caps.
1495
-
1496
-Files touched:
1497
-- src/crates/netflow-plugin/src/plugin_config/types/journal.rs
1498
-- src/crates/netflow-plugin/src/plugin_config/validation/journal.rs
1499
-- src/crates/netflow-plugin/src/plugin_config_tests.rs
1500
- (two YAML fixtures cleaned)
1501
-- docs/network-flows/configuration.md (removed two table
1502
- rows + corresponding code-block lines + the two-line
1503
- "query-window limits" explanation)
1504
-- docs/network-flows/retention-querying.md (corrected
1505
- description of tier-pick behaviour)
1506
-
1507
-Build + tests:
1508
-- `cargo build --release` clean (2m59s).
1509
-- `cargo test --release --bin netflow-plugin` -- 427
1510
- passed; 0 failed; 18 ignored.
1511
-
1512
-Breaking change notice: any user config that set
1513
-`journal.query_1m_max_window` or `journal.query_5m_max_window`
1514
-will now fail to deserialize (deny_unknown_fields). These
1515
-keys had no effect before, so the migration is to delete
1516
-them; no behavioural change.
1517
-
1518
-#### F10 -- 2026-05-07 -- query_max_groups stays (real); query_facet_max_values_per_field is dead
1519
-
1520
-User: "I don't understand what are these and why are needed
1521
-and what value or protection they provide. Explain"
1522
-
1523
-Code investigation, two opposite outcomes:
1524
-
1525
-`query_max_groups` -- REAL.
1526
-- Read at `query/service.rs:52` and threaded into the
1527
- `ProjectedGroupAccumulator` via the `max_groups` parameter
1528
- used at `query/projected/apply.rs:48`.
1529
-- When `grouped_aggregates.grouped_total() < max_groups`
1530
- is no longer true, the accumulator stops registering new
1531
- group keys; the row's missing values are then folded into
1532
- the synthetic `__overflow__` bucket created at
1533
- `query/grouping/labels.rs:17` /
1534
- `query/grouping/model/compact.rs:35`.
1535
-- A warning is emitted on the response:
1536
- `query/timeseries.rs:124` -- "Group accumulator limit
1537
- reached; additional groups were folded into __overflow__."
1538
-- Purpose: bounds memory growth on accidentally wide
1539
- group-by combinations (high-cardinality fields like IP
1540
- addresses, AS-paths, MAC addresses, etc.).
1541
-- Verdict: keep, document properly.
1542
-
1543
-`query_facet_max_values_per_field` -- DEAD.
1544
-- Declared, validated for non-zero in
1545
- `plugin_config/validation/journal.rs:18`, but the actual
1546
- facet accumulator at `query/facets/render.rs:19, 27`
1547
- consumes the hardcoded constant
1548
- `DEFAULT_FACET_ACCUMULATOR_MAX_VALUES_PER_FIELD = 5_000`
1549
- from `query/request/constants.rs:17` -- not the config
1550
- knob. The two coincidentally have the same default value
1551
- but the config knob is never threaded to the consumer.
1552
-- Verdict: dead schema. Remove.
1553
-
1554
-Repair:
1555
-
1556
-- `plugin_config/types/journal.rs` -- removed
1557
- `query_facet_max_values_per_field` field; added a doc
1558
- comment explaining what `query_max_groups` actually does.
1559
-- `plugin_config/defaults.rs` -- removed
1560
- `default_query_facet_max_values_per_field()` helper.
1561
-- `plugin_config/validation/journal.rs` -- removed the
1562
- non-zero check for the dead knob.
1563
-- `plugin_config_tests.rs` -- removed
1564
- `validate_rejects_zero_query_facet_max_values_per_field`
1565
- test entirely; cleaned the YAML fixtures that mentioned
1566
- the dead knob.
1567
-- `src/crates/netflow-plugin/configs/netflow.yaml` (stock
1568
- config) -- rewrote the journal block to use the per-tier
1569
- retention form and dropped the dead knob; added a clear
1570
- comment for `query_max_groups`.
1571
-- `src/crates/netflow-plugin/README.md` -- updated the
1572
- example YAML and the explanatory paragraph; removed the
1573
- dead-knob mention.
1574
-- `docs/network-flows/configuration.md` (Query guardrails
1575
- section) -- table now lists only `query_max_groups`;
1576
- expanded its description to name the `__overflow__`
1577
- bucket and the warning behaviour.
1578
-- `docs/network-flows/retention-querying.md` (Group-by
1579
- limit section) -- consolidated to one bullet; named the
1580
- warning + overflow behaviour.
1581
-- `docs/network-flows/visualization/filters-facets.md` --
1582
- removed the entire "Facet limits" subsection that
1583
- documented the dead knob.
1584
-
1585
-Build + tests:
1586
-- `cargo build --release` clean (2m57s).
1587
-- `cargo test --release --bin netflow-plugin` -- 426
1588
- passed; 0 failed (one test removed -- the dead-knob
1589
- validation test).
1590
-
1591
-Breaking change notice: any user config that set
1592
-`journal.query_facet_max_values_per_field` will now fail
1593
-to deserialize (deny_unknown_fields). The key had no
1594
-effect before; migration is delete-only.
1595
-
1596
-#### F11 -- 2026-05-07 -- empty IP Intelligence page
1597
-
1598
-Investigation: `docs/network-flows/enrichment/ip-intelligence.md`
1599
-was a 0-byte file in master. `git log --all --format=%H` of
1600
-that path shows it has been 0 bytes since the original
1601
-documentation rewrite commit (a073bcf24f). It was meant to
1602
-be the "Enrichment Concepts / IP Intelligence" page but
1603
-got created empty.
1604
-
1605
-Repair: authored the page from scratch, code-grounded
1606
-against:
1607
-
1608
-- `src/crates/netflow-plugin/src/plugin_config/types/enrichment/geoip.rs`
1609
- (GeoIpConfig: asn_database / geo_database / optional).
1610
-- `src/crates/netflow-plugin/src/plugin_config/runtime.rs:23-64`
1611
- (auto-detect path: cache_dir/topology-ip-intel/* and
1612
- stock_data_dir/topology-ip-intel/*; auto-detected files
1613
- marked optional=true).
1614
-- `src/crates/netflow-plugin/src/enrichment/data/geoip/resolver.rs`
1615
- (load, refresh-if-needed every 30s on signature change,
1616
- per-IP lookup composing multiple ASN/geo databases,
1617
- IPv6-vs-IPv4-database skip).
1618
-- `src/crates/netflow-plugin/src/enrichment.rs:35`
1619
- (GEOIP_RELOAD_CHECK_INTERVAL = 30s).
1620
-- `src/crates/netflow-plugin/src/enrichment/data/network/asn.rs`
1621
- (AS-name rendering format).
1622
-
1623
-Page covers: fields populated (with tier-preservation
1624
-notes), configuration schema, auto-detection, refresh
1625
-cadence, lookup order, the four provider integration cards,
1626
-private-IP rendering, IPv6-only/IPv4-only database
1627
-behaviour, staleness drift, geographic-accuracy caveats,
1628
-failure modes table.
1629
-
1630
-Frontmatter `learn_rel_path` set to
1631
-"Network Flows/Enrichment Concepts" to match the bgp-routing
1632
-and network-identity siblings (the source frontmatter is
1633
-informational; the actual sidebar position derives from
1634
-`docs/.map/map.yaml`). F20 will rename the section
1635
-consistently across map.yaml and all sibling frontmatter.
1636
-
1637
-Files touched:
1638
-- docs/network-flows/enrichment/ip-intelligence.md
1639
- (created from empty)
1640
-
1641
-#### F20 -- 2026-05-07 -- "Enrichment Concepts" -> "Flows Enrichment"
1642
-
1643
-Repair: renamed the section consistently in:
1644
-
1645
-- `docs/.map/map.yaml:499` -- the section label that drives
1646
- the actual sidebar position on Learn.
1647
-- All seven frontmatter `learn_rel_path:` values in
1648
- `docs/network-flows/enrichment/*.md` -- prior state was
1649
- inconsistent (4 files had "Network Flows/Enrichment", 2
1650
- had "Network Flows/Enrichment Concepts", 1 had the
1651
- brand-new F11 page). Settled on the canonical
1652
- "Network Flows/Flows Enrichment" everywhere.
1653
-
1654
-Grep `Enrichment Concepts|enrichment-concepts` across docs/
1655
-and src/crates/netflow-plugin/ -- no remaining references.
1656
-
1657
-Files touched:
1658
-- docs/.map/map.yaml
1659
-- docs/network-flows/enrichment/asn-resolution.md
1660
-- docs/network-flows/enrichment/bgp-routing.md
1661
-- docs/network-flows/enrichment/classifiers.md
1662
-- docs/network-flows/enrichment/decapsulation.md
1663
-- docs/network-flows/enrichment/ip-intelligence.md
1664
-- docs/network-flows/enrichment/network-identity.md
1665
-- docs/network-flows/enrichment/static-metadata.md
1666
-
1667
-#### F21 -- 2026-05-07 -- "Sources" -> "Flow Protocols"
1668
-
1669
-Repair: renamed the Flow Protocols sub-category in the
1670
-integrations catalog and propagated to integration cards
1671
-and any in-prose references.
1672
-
1673
-- `integrations/categories.yaml:71` -- the `flows.sources`
1674
- category's user-visible `name:` changed from "Sources" to
1675
- "Flow Protocols". Description left intact (it already
1676
- reads "Flow protocols Netdata receives directly from
1677
- routers, switches, and software exporters", which agrees
1678
- with the new label).
1679
-- Three integration card frontmatter updates: `sflow.md`,
1680
- `ipfix.md`, `netflow.md` now declare
1681
- `learn_rel_path: "Network Flows/Flow Protocols"`.
1682
-- `src/crates/netflow-plugin/metadata.yaml` -- removed three
1683
- broken self-referencing links pointing at
1684
- `https://learn.netdata.cloud/docs/network-flows/sources/{netflow,ipfix,sflow}`.
1685
- These URLs were broken before the rename (no
1686
- `docs/network-flows/sources/` directory exists in source)
1687
- and would stay broken under the new label too. Replaced
1688
- with the surviving "Network Flows Overview" anchor that
1689
- is real.
1690
-- Re-ran `integrations/gen_integrations.py` and
1691
- `integrations/gen_docs_integrations.py` -- both exit
1692
- clean. The three regenerated `.md` cards no longer carry
1693
- the broken self-link.
1694
-
1695
-Grep `/sources/` and `/Sources/` after the sweep -- no
1696
-remaining references inside flow integrations content.
1697
-
1698
-Files touched:
1699
-- integrations/categories.yaml
1700
-- src/crates/netflow-plugin/metadata.yaml
1701
-- src/crates/netflow-plugin/integrations/sflow.md
1702
-- src/crates/netflow-plugin/integrations/ipfix.md
1703
-- src/crates/netflow-plugin/integrations/netflow.md
1704
-
1705
-#### F18 -- 2026-05-07 -- journalctl --namespace netdata everywhere
1706
-
1707
-User: "Netdata logs in namespace 'netdata'. Journalctl needs
1708
-`--namespace netdata`."
1709
-
1710
-Background: `-u netdata` selects the systemd UNIT, which
1711
-captures only stdout/stderr the unit emits to the journal.
1712
-The plugin (and the rest of Netdata) actually writes
1713
-structured logs into a journal NAMESPACE called `netdata`.
1714
-Without `--namespace netdata`, users see at most the
1715
-unit-level startup/shutdown messages, not the actual log
1716
-output that helps with debugging.
1717
-
1718
-Repair: swept all `journalctl -u netdata` invocations to
1719
-`journalctl --namespace netdata` in:
1720
-- docs/network-flows/quick-start.md
1721
-- docs/network-flows/troubleshooting.md (5 occurrences)
1722
-- docs/network-flows/installation.md
1723
-- docs/network-flows/enrichment/network-identity.md
1724
-- docs/network-flows/configuration.md (already fixed under
1725
- F8; no change here)
1726
-
1727
-Grep `journalctl` after the sweep -- every invocation now
1728
-uses `--namespace netdata`.
1729
-
1730
-#### F15 -- 2026-05-07 -- remove "Ignoring the sampling rate" anti-pattern
1731
-
1732
-User: "How is it possible for users to ignore the sampling
1733
-rate if we calculate the estimated volume at ingestion? You
1734
-invented reasons for it. ... section must be removed."
1735
-
1736
-Verified the per-flow multiplication code path under F4 +
1737
-F5 already; the entire premise of this anti-pattern (that
1738
-mixed rates produce inconsistent multiplication, that users
1739
-must keep rates uniform, that aggregates become "hard to
1740
-interpret") was wrong. The two real concerns it conflated
1741
-are documented elsewhere already:
1742
-
1743
-- "small flows missed at high sampling rates" -- preserved
1744
- under the Overview's "What sampling does to your numbers"
1745
- section and inside investigation-playbooks.md "Caveats"
1746
- for the security playbook.
1747
-- "exporter sends no rate" (NetFlow v7, v5 with rate=0,
1748
- v9/IPFIX without Sampling Options Template) -- preserved
1749
- in troubleshooting.md "Bandwidth doesn't match SNMP" and
1750
- in validation.md.
1751
-
1752
-Repair: deleted the entire "## 2. Ignoring the sampling
1753
-rate" section from `docs/network-flows/anti-patterns.md`.
1754
-Section numbering renumber will land with F17 (the last of
1755
-the three section removals) so that anti-patterns.md
1756
-renumbers exactly once.
1757
-
1758
-Files touched:
1759
-- docs/network-flows/anti-patterns.md (removed lines
1760
- ~23-36 plus the section header)
1761
-
1762
-#### F16 -- 2026-05-07 -- remove "Trusting GeoIP for internal IPs" anti-pattern
1763
-
1764
-User: "Geolocation does not position internal IPs on the
1765
-map. ... section must be removed."
1766
-
1767
-Verified against code: `apply_geo_record`
1768
-(`src/crates/netflow-plugin/src/enrichment/data/geoip/decode.rs:40-72`)
1769
-writes `country`, `state`, `city`, `latitude`, `longitude`
1770
-ONLY when the MMDB record carries those fields with
1771
-non-empty values. For private / RFC 1918 IPs, the MMDB
1772
-either has no entry at all OR has an entry tagged with
1773
-`ip_class: "private"` and no country/city/coords -- so
1774
-none of those fields get written. Internal IPs do NOT
1775
-appear on geographic maps.
1776
-
1777
-The "internal IPs in random countries" claim was invented;
1778
-no such behaviour exists. Repair: deleted the entire
1779
-"## Trusting GeoIP for internal IPs" section.
1780
-
1781
-Section numbering renumber will land with F17 once all
1782
-three section removals have completed.
1783
-
1784
-Note: the troubleshooting.md "Internal IPs in random
1785
-countries" subsection (lines 134-138) carries the same
1786
-invented claim and will be addressed under F19.
1787
-
1788
-Files touched:
1789
-- docs/network-flows/anti-patterns.md (removed the
1790
- section header and body)
1791
-
1792
-#### F17 -- 2026-05-07 -- remove "Alerting on absolute volume thresholds"
1793
-
1794
-User: "Netdata does not support alerting of flows yet.
1795
-Remove this section."
1796
-
1797
-Repair: removed the entire "Alerting on absolute volume
1798
-thresholds" section from `docs/network-flows/anti-patterns.md`.
1799
-The section's own footnote already acknowledged this:
1800
-"Netdata's alerting on flow data is in development; for
1801
-now this pattern lives in your monitoring practice, not
1802
-in the plugin." So the section was advice for users to
1803
-apply outside Netdata -- not a Netdata anti-pattern.
1804
-
1805
-Renumbering: with F15, F16, and F17 all deleting sections,
1806
-the anti-patterns.md sections get renumbered in this same
1807
-commit. Final numbering: 1 (doubled aggregate) ... 9
1808
-(comparing flow counts across protocols). Removed three
1809
-old rows from the summary table (Ignored sampling, GeoIP
1810
-for internal IPs, Absolute thresholds). The cross-link
1811
-from time-series.md at line 96 ("Why time-shifted
1812
-comparison beats absolute thresholds") was rewritten to
1813
-point at the still-relevant general anti-patterns set.
1814
-
1815
-Files touched:
1816
-- docs/network-flows/anti-patterns.md (removed section,
1817
- renumbered remaining sections, dropped 3 summary-table
1818
- rows)
1819
-- docs/network-flows/visualization/time-series.md
1820
- (cross-link rewording)
1821
-
1822
-#### F19 -- 2026-05-07 -- troubleshooting.md cumulative cleanup
1823
-
1824
-User: "This page has a mix of all the above issues:
1825
-sampling, geoip, etc."
1826
-
1827
-Repair scope: surgical fixes to the cumulative
1828
-misconceptions on troubleshooting.md after F2-F18 land.
1829
-
1830
-Removed:
1831
-- "Internal IPs in random countries" subsection (lines
1832
- ~134-138). Same invented claim as F16; same code-verified
1833
- reason for removal.
1834
-
1835
-Rewrote:
1836
-- "Things that look like bugs but aren't" entries:
1837
- - "Traffic appears 2x" -- now mentions exporter + interface,
1838
- not direction (F2/F3 fix in this section too).
1839
- - "Bidirectional conversations show twice" -- reframed as
1840
- real distinct flows with usually-asymmetric volumes;
1841
- pointed at Source ASN / Destination ASN filtering, not
1842
- "direction" filtering.
1843
- - Removed the "Internal IPs in odd countries" bullet entirely.
1844
- - "City map empty over long windows" -- "tier-0" replaced
1845
- with "raw-tier" for consistency with the field-reference
1846
- and tier-naming used elsewhere.
1847
-
1848
-Items checked and kept:
1849
-- "Sampling rate not honoured by the exporter" framing is
1850
- correct (F4/F5 already updated this; the real concern is
1851
- the exporter not communicating the rate, not "uniform
1852
- rates required").
1853
-- Doubling references in the SNMP-mismatch table (F2/F3
1854
- already updated).
1855
-- ASN provider chain debug recipe -- code-anchored at
1856
- `enrichment/data/network/asn.rs`.
1857
-- Decapsulation destructive-on-non-tunnel framing -- code-
1858
- anchored at `decoder/protocol/...`.
1859
-
1860
-Items DEFERRED to the per-page audit (R2) because they
1861
-need vendor-doc verification:
1862
-- "Cisco's default template refresh is 30 minutes" --
1863
- vendor-specific claim; verify against current Cisco
1864
- IOS-XE / IOS-XR documentation in R2.
1865
-
1866
-Files touched:
1867
-- docs/network-flows/troubleshooting.md
1868
-
1869
-#### F14 -- 2026-05-07 -- validation.md rewrite
1870
-
1871
-User: "I think the entire 'Validation and Data Quality'
1872
-is completely off. It mentions again sampling rates, etc.
1873
-It is like it was written by someone that does not have
1874
-a clue of what netdata is and how the plugin works."
1875
-
1876
-Code-verified facts driving the rewrite:
1877
-
1878
-- **Per-flow sampling-rate multiplication** at decode time:
1879
- `decoder/record/core/record.rs:24-26`. The user does NOT
1880
- need to monitor "sampling rate change" or "sampling rate
1881
- misinterpretation" -- these are not user-side risks.
1882
-- **Template persistence** across plugin restarts:
1883
- `decoder/protocol/v9/templates.rs:106` and
1884
- `decoder/protocol/ipfix/templates/data.rs:67`. The user
1885
- does NOT need to monitor "template loss after collector
1886
- restart".
1887
-- **UDP buffer overflow alert** already exists at
1888
- `src/health/health.d/udp_errors.conf:6-19`
1889
- (`1m_ipv4_udp_receive_buffer_errors`, fires when
1890
- RcvbufErrors > 10/min). Reframe UDP drops as an existing
1891
- alert to consume, not a "silent failure" the user must
1892
- hunt down.
1893
-
1894
-Page rewritten from scratch:
1895
-
1896
-- New opening: states up-front that the plugin handles
1897
- per-flow scaling, template persistence, and database
1898
- refresh internally; what's left to validate is
1899
- exporter-side and configuration drift.
1900
-- New "What you actually need to watch" table with five
1901
- items (kernel UDP drops -> existing alert; exporter
1902
- stopped sending; wrong interfaces being exported;
1903
- exporter sampling but not communicating rate; stale
1904
- MMDB).
1905
-- Removed the original silent-failure list items
1906
- "Sampling rate misinterpretation", "Sampling rate
1907
- change", "Template loss after collector restart" --
1908
- three items confirmed not user-side risks.
1909
-- Removed the "Internal IP enrichment validation" section
1910
- (F16 confirmed GeoIP does not position internal IPs).
1911
-- Renamed "Sampling rate sanity check" to "Sampling rate
1912
- verification" with the bogus uniform-rate framing
1913
- removed; kept the practical RAW_BYTES vs BYTES
1914
- comparison recipe (the only useful piece of the old
1915
- section).
1916
-- Removed the "Template cache health" subsection
1917
- entirely. (The `template_errors` chart is still in the
1918
- plugin-side alerting table, but as an exporter-config
1919
- signal, not a "user must watch in case templates get
1920
- lost" risk.)
1921
-- Renamed the alerting table from "what to monitor and
1922
- what alerts to consider" to "Plugin-side signals worth
1923
- alerting on"; clarified that these are signals the
1924
- plugin already exposes for the operator to alert on,
1925
- not "silent failures" the dashboard hides.
1926
-
1927
-Files touched:
1928
-- docs/network-flows/validation.md (full rewrite, kept
1929
- frontmatter and the surviving sections in place)
1930
-
1931
-#### F13 -- 2026-05-07 -- sizing-capacity.md rewrite as a practical guide
1932
-
1933
-User: "Sizing and Capacity planning is written like an
1934
-academic paper that must prove productivity of the testing
1935
-environment. People want sizing and planning directions.
1936
-This is not an academic paper, not a blog."
1937
-
1938
-User-stated requirements (the seven bullets):
1939
-1. what is the cap of the plugin
1940
-2. how ingestion rate affects storage
1941
-3. raw tier monopolizes storage; needs fast NVMe
1942
-4. journal uses free system memory as page cache; bigger
1943
- database -> more free RAM
1944
-5. journal is fully indexed; FTS means full scan
1945
-6. 25k flows/s sustained approaches ISP-level capacity
1946
-7. distributed deployment -- one Netdata per router; no
1947
- central aggregation needed
1948
-
1949
-Plus: remove benchmarks and tests from this page.
1950
-
1951
-Page rewritten from scratch. New shape:
1952
-
1953
-- Opening paragraph: states the design intent (one network
1954
- per agent). Anchors the 25k flows/s = ISP-level scale
1955
- immediately.
1956
-- "Plugin throughput cap" -- single-thread post-decode
1957
- hot path; saturation around 30k high-cardinality, above
1958
- 60k low-cardinality; recommend 25k sustained as the
1959
- comfortable steady-state. Burst handling.
1960
-- "Distributed deployment is the scaling answer" --
1961
- the user's central thesis. One agent per router / per site,
1962
- federated via Netdata Cloud. Why this beats pushing more
1963
- through a single collector. Recommended shape for
1964
- multi-site deployments.
1965
-- "Storage" -> "How ingestion rate maps to disk" -- one
1966
- table, four rows, derived from the storage-footprint
1967
- benchmark (~800 bytes/flow on disk). Includes a
1968
- cardinality caveat. No benchmark methodology.
1969
-- "Storage" -> "Raw tier dominates" -- explicit; rollups
1970
- are tiny; example per-tier config sized for production
1971
- use.
1972
-- "Storage" -> "Use fast NVMe for the raw tier" -- direct,
1973
- no hedging. Mentions that slow storage forces shorter
1974
- raw-tier retention.
1975
-- "Memory" -- routing trie footprint, page-cache framing,
1976
- and the existing memory-monitoring chart references.
1977
-- "Querying -- what's fast and what isn't" -- indexed
1978
- fields are O(log) on selectivity; FTS is full scan of
1979
- raw tier and forces raw-tier; 30s query timeout
1980
- implications.
1981
-- "Practical checklist before you deploy" -- seven
1982
- concrete steps mirroring the user's seven bullets.
1983
-
1984
-Removed:
1985
-- All "What was measured" / "Detailed measurements" / the
1986
- per-protocol per-cardinality benchmark tables (Phase 1.0
1987
- output). Those numbers stay in the netflow-plugin README
1988
- for engineering reference; they are not customer
1989
- guidance and were the wrong genre for this page.
1990
-- "Bounding storage for capacity planning" formula derivation
1991
- (which was already partly invalid because it ignored
1992
- tier rollover and dedup).
1993
-
1994
-Files touched:
1995
-- docs/network-flows/sizing-capacity.md (full rewrite)
1996
-
1997
-#### F12 -- 2026-05-07 -- Retention/Querying restructure; new Visualization Overview
1998
-
1999
-User: "Retention is closer to configuration and querying
2000
-is closer to visualization. ... If you need to put generic
2001
-visualization rules, these should be a generic
2002
-'Visualization/Overview' page, to explain FTS, sharing,
2003
-grouping, etc."
2004
-
2005
-Repair: split the old retention-querying.md into a
2006
-retention-only page and a new visualization-overview page.
2007
-
2008
-New file: `docs/network-flows/visualization/overview.md`
2009
-- "How queries work" -- query modes, parameters, defaults,
2010
- 30s timeout (moved from retention-querying.md).
2011
-- "Group-by limit and overflow" (moved).
2012
-- "Full-text search" (moved + expanded to explain when to
2013
- use it vs the indexed filter ribbon).
2014
-- "URL sharing" (moved; reframed as part of generic
2015
- visualization, not its own standalone section).
2016
-- "Filtering" (cross-link to filters-facets.md).
2017
-- "Picking the right view" (cross-link to each panel).
2018
-
2019
-Updated `docs/network-flows/retention-querying.md`:
2020
-- Sidebar label: "Retention and Querying" -> "Retention
2021
- and Tiers" (matches the new content scope).
2022
-- Removed sections: "How queries work, briefly",
2023
- "Group-by limit and overflow", "Full-text search",
2024
- "URL sharing" (all moved to the new visualization/overview).
2025
-- Page intro now points users to Configuration for retention
2026
- config and to Visualization Overview for query semantics.
2027
-- Renamed remaining mentions of "tier 0" / "tier-0" to
2028
- "raw tier" / "raw-tier" for consistency.
2029
-
2030
-Updated `docs/.map/map.yaml`:
2031
-- Visualization sub-section root now carries `edit_url:`
2032
- pointing at visualization/overview.md (so clicking
2033
- "Visualization" in the sidebar opens the Overview, the
2034
- same pattern as F1's section-root fix).
2035
-- "Retention and Querying" sidebar label renamed to
2036
- "Retention and Tiers".
2037
-- "Sizing and Capacity Planning" description updated post
2038
- F13 (no more benchmarks).
2039
-
2040
-#### F22 -- 2026-05-07 -- remove redundant "Section index" from the Overview
2041
-
2042
-User: "The 'Section index' in the overview page is not
2043
-needed. Learn already shows the index as a side bar."
2044
-
2045
-Repair: removed the entire `## Section index` section from
2046
-`docs/network-flows/README.md`. The Learn sidebar already
2047
-shows the same hierarchy. The "Where to start" section
2048
-above it stays (it's role-based guidance, not a duplicate
2049
-of the sidebar). Updated the "specific feature in depth"
2050
-bullet to point at the sidebar instead of the deleted
2051
-section.
2052
-
2053
-Files touched:
2054
-- docs/network-flows/visualization/overview.md (created)
2055
-- docs/network-flows/retention-querying.md (slimmed,
2056
- sidebar label renamed)
2057
-- docs/.map/map.yaml
2058
-- docs/network-flows/README.md (removed Section index)
2059
-
2060
-#### Current audit -- 2026-05-07 -- metadata.yaml is first-class docs source
2061
-
2062
-The follow-up review treats `src/crates/netflow-plugin/metadata.yaml`
2063
-as a public documentation source, not merely generator plumbing. The
2064
-generated files under `src/crates/netflow-plugin/integrations/*.md`
2065
-carry a `DO NOT EDIT` banner and are downstream of that metadata file,
2066
-so every metadata issue below also reaches Learn / integration-card
2067
-content after regeneration.
2068
-
2069
-Current branch reviewed: `netflow-docs-repair`.
2070
-
2071
-Status of previously suspected issues at the start of this pass:
2072
-
2073
-- **Found during current audit -- timestamp source is computed but not used for live
2074
- journal timestamps.** `docs/network-flows/configuration.md:104-112`
2075
- says `timestamp_source` controls dashboard timestamps. The decoder
2076
- carries `DecodedFlow.source_realtime_usec` at
2077
- `src/crates/netflow-plugin/src/decoder.rs:149-151`, but live ingest
2078
- still calls `ingest_decoded_record(receive_time_usec, &flow.record)`
2079
- at `src/crates/netflow-plugin/src/ingest/service/runtime.rs:82-84`
2080
- and writes both source and entry realtime from `receive_time_usec` at
2081
- `runtime.rs:129-131`. This is a code/docs mismatch. Recommended
2082
- fix: code should thread `flow.source_realtime_usec.unwrap_or(receive_time_usec)`
2083
- into raw writes and tier observation; docs can then stay conceptually
2084
- correct.
2085
-- **Found during current audit -- removed top-level retention keys remain in
2086
- `metadata.yaml` and generated integration docs.** The current code
2087
- schema has only `JournalConfig { journal_dir, tiers, query_max_groups }`
2088
- at `src/crates/netflow-plugin/src/plugin_config/types/journal.rs:11-31`;
2089
- retention is per-tier under `JournalTierRetentionOverrides` at
2090
- `journal.rs:83-112`. But `metadata.yaml` still documents
2091
- `journal.size_of_journal_files` / `journal.duration_of_journal_files`
2092
- at lines 96-102, 246-252, and 383-389, and the NetFlow example still
2093
- uses top-level `journal.size_of_journal_files` / `duration_of_journal_files`
2094
- at lines 132-135. Generated `netflow.md`, `ipfix.md`, and `sflow.md`
2095
- repeat the same invalid options. Because the Rust structs use
2096
- `#[serde(deny_unknown_fields)]`, this documented YAML now fails
2097
- config parsing. Fix source in `metadata.yaml`, then regenerate.
2098
-- **Found during current audit -- IPFIX and sFlow metadata still call 2055 the
2099
- "standard port".** `metadata.yaml:262` says IPFIX listens on the
2100
- standard port while the example uses `0.0.0.0:2055`; `metadata.yaml:399`
2101
- says the same for sFlow. Code only proves 2055 is Netdata's default
2102
- listener at `src/crates/netflow-plugin/src/plugin_config/types/listener.rs:6`.
2103
- External protocol evidence: IANA registers IPFIX on 4739 and sFlow on
2104
- 6343. Fix wording to "Netdata default listener port" or state that
2105
- users may choose any UDP listener and must configure exporters to match.
2106
-- **Found during current audit -- invented internal-IP geolocation claim remains in
2107
- metadata and maps docs.** `metadata.yaml:598-604`,
2108
- `src/crates/netflow-plugin/integrations/db-ip_ip_intelligence.md:184`,
2109
- and `docs/network-flows/visualization/maps-globe.md:77-79` still
2110
- discuss "internal IPs appearing in random countries." Code only writes
2111
- country/city/state/coordinate fields when the MMDB record has a
2112
- non-empty value (`src/crates/netflow-plugin/src/enrichment/data/geoip/decode.rs:40-72`).
2113
- This is the same false premise as F16 and F19, but not fully removed.
2114
- Fix source in `metadata.yaml` and the maps page, then regenerate.
2115
-- **Found during current audit -- `OBSERVATION_TIME_MILLIS` contradiction remains.**
2116
- `docs/network-flows/field-reference.md:138` says the field is IPFIX
2117
- observation time, while the master table at line 297 says IPFIX has no
2118
- canonical mapping in this build. Code maps only NetFlow v9
2119
- `ObservationTimeMilliseconds` at
2120
- `src/crates/netflow-plugin/src/decoder/record/mappings.rs:37`; IPFIX
2121
- falls through to `_ => None` at `mappings.rs:116`. Fix docs unless
2122
- product decides to add IPFIX support in code.
2123
-- **Found during current audit -- visualization overview documents unsupported `last`
2124
- query parameter.** `docs/network-flows/visualization/overview.md:22`
2125
- lists `after` / `before`, or `last`. The accepted function parameters
2126
- are enumerated in `src/crates/netflow-plugin/src/api/flows/params.rs:5-18`
2127
- and do not include `last`. Fix docs unless code intentionally adds
2128
- a shorthand.
2129
-- **Found during current audit -- "tier 0" terminology remains.** Examples:
2130
- `docs/network-flows/visualization/time-series.md:48-55`,
2131
- `docs/network-flows/visualization/maps-globe.md:51,103`,
2132
- `docs/network-flows/field-reference.md:155`, and
2133
- `docs/network-flows/investigation-playbooks.md:70,97,145`.
2134
- Code/config use `raw` (`JournalTierRetentionOverrides.raw` at
2135
- `journal.rs:86-87`; stock config `journal.tiers.raw`). Fix docs to
2136
- "raw tier" consistently.
2137
-- **Found during current audit -- pcap references remain while
2138
- the acceptance criterion still says no pcap anywhere.**
2139
- `docs/network-flows/troubleshooting.md:72,217,227` still recommend
2140
- `tcpdump -w`. This may be useful support guidance, but it contradicts
2141
- the explicit acceptance criterion at line 68. Either remove the public
2142
- pcap workflow, or update the SOW acceptance criterion to allow packet
2143
- captures strictly as troubleshooting artifacts.
2144
-
2145
-New findings from the current branch:
2146
-
2147
-- **F23 -- public docs link to source-tree integration markdown paths
2148
- that Learn will not serve.** Many Network Flows pages link to
2149
- `/src/crates/netflow-plugin/integrations/*.md`, e.g.
2150
- `docs/network-flows/quick-start.md:105,180,187-189`,
2151
- `docs/network-flows/configuration.md:221-224`,
2152
- `docs/network-flows/enrichment.md:419-440`,
2153
- `docs/network-flows/installation.md:80,130-132`, and
2154
- `docs/network-flows/visualization/maps-globe.md:75,79,112-113`.
2155
- Those are repository source paths, not Learn URLs. The integration
2156
- cards are generated into the Network Flows integration placeholder and
2157
- should be linked through their Learn routes, not `/src/...`. Fix all
2158
- source-path links.
2159
-- **F24 -- `metadata.yaml` still uses stale UI wording "Network Flows
2160
- tab".** F7 fixed many markdown pages, but metadata still says
2161
- "Network Flows tab" at lines 153, 627, 865, 1100, 1679, 2273, 2564,
2162
- 3517, 3560, 3959, 4379, and 4653. Generated integration cards repeat
2163
- the same stale term. Fix metadata to "Network Flows view" or
2164
- "Live tab > Network Flows" depending on context, then regenerate.
2165
-- **F25 -- old "Sources" label remains in public markdown links.**
2166
- The catalog label is now "Flow Protocols" in
2167
- `integrations/categories.yaml:70-72`, but docs still say
2168
- "Sources / NetFlow" or "Sources" at
2169
- `docs/network-flows/quick-start.md:105`,
2170
- `docs/network-flows/installation.md:130-132`, and
2171
- `docs/network-flows/anti-patterns.md:121`. Fix wording and target URLs.
2172
-- **F26 -- public docs and generated integration cards contain internal
2173
- code citations.** There are hundreds of `src/crates/...` / `.rs:line`
2174
- citations in generated integration cards and several in
2175
- `docs/network-flows/enrichment.md` / `intel-downloader.md`. These are
2176
- useful audit evidence, but they read as implementation notes in
2177
- end-user/operator documentation. The SOW can keep code citations; public
2178
- docs should translate them into operator-facing behavior and only link
2179
- to code where the user explicitly needs source. Needs a per-page/content
2180
- decision during repair, but the current state is not fit for polished
2181
- Learn docs.
2182
-
2183
-Repair ordering recommendation:
2184
-
2185
-1. Fix `metadata.yaml` first for F23/F24 plus the still-open retention,
2186
- port, and internal-IP findings; regenerate integration pages.
2187
-2. Fix the remaining markdown-only regressions (`timestamp_source`,
2188
- `OBSERVATION_TIME_MILLIS`, `last`, raw-tier terminology, pcap decision).
2189
-3. Re-run link checks and grep sweeps against both `docs/network-flows/`
2190
- and generated `src/crates/netflow-plugin/integrations/*.md`.
2191
-4. Run the integration generator and the narrow netflow-plugin tests
2192
- touched by any code fix.
2193
-
2194
-#### Current audit repair progress -- 2026-05-07
2195
-
2196
-First repair pass completed:
2197
-
2198
-- `src/crates/netflow-plugin/metadata.yaml` now documents per-tier
2199
- `journal.tiers.<tier>.size_of_journal_files` and
2200
- `journal.tiers.<tier>.duration_of_journal_files` instead of removed
2201
- top-level retention keys.
2202
-- The NetFlow extended-retention example now uses `journal.tiers.raw`,
2203
- `minute_1`, `minute_5`, and `hour_1`.
2204
-- IPFIX and sFlow setup examples now say "Netdata's default flow
2205
- listener port" instead of "standard port".
2206
-- The DB-IP troubleshooting entry now says private IPs have empty GeoIP
2207
- fields and do not appear on maps, instead of "internal IPs appearing
2208
- in random countries".
2209
-- `Network Flows tab` was mechanically replaced with `Network Flows
2210
- view` in metadata.
2211
-- Regenerated integration pages with `python3 integrations/gen_integrations.py`
2212
- and `python3 integrations/gen_docs_integrations.py`.
2213
-- Verified the following patterns are gone from `metadata.yaml` and
2214
- generated integration cards:
2215
- `journal.size_of_journal_files`,
2216
- `journal.duration_of_journal_files`,
2217
- `standard port`,
2218
- `Internal IPs appearing`,
2219
- `random countries`,
2220
- `Network Flows tab`.
2221
-
2222
-Markdown repair pass completed:
2223
-
2224
-- Repointed public docs away from `/src/crates/netflow-plugin/integrations/*.md`
2225
- source paths to Learn routes under `/docs/network-flows/flow-protocols/...`
2226
- and `/docs/network-flows/enrichment-methods/...`.
2227
-- Replaced stale "Sources / ..." wording with "Flow Protocols / ...".
2228
-- Removed the unsupported `last` query-parameter claim from
2229
- `docs/network-flows/visualization/overview.md`; it now says omitted
2230
- `after` / `before` defaults to the last 15 minutes, matching
2231
- `src/crates/netflow-plugin/src/query/planner/request.rs:3-15`.
2232
-- Removed the remaining internal-IP/random-country claim from
2233
- `docs/network-flows/visualization/maps-globe.md`.
2234
-- Replaced remaining `tier 0` / `tier-0` language in Network Flows markdown
2235
- with `raw tier` / `raw-tier`.
2236
-- Fixed `docs/network-flows/field-reference.md` so
2237
- `OBSERVATION_TIME_MILLIS` is documented as NetFlow v9-only in this
2238
- build, and so timestamp fields are not described as dashboard
2239
- time-picker fields. Code evidence:
2240
- `src/crates/netflow-plugin/src/decoder/record/mappings.rs:37,116`,
2241
- `src/crates/netflow-plugin/src/query/request/constants.rs:75-79`,
2242
- `src/crates/netflow-plugin/src/query/fields/metrics.rs:19-20`.
2243
-- Verified the field-reference master index still has the same 91
2244
- canonical fields as `src/crates/netflow-plugin/src/flow/schema.rs`.
2245
-
2246
-Follow-up repair after timestamp clarification:
2247
-
2248
-- `timestamp_source` is no longer open. The code now passes the selected
2249
- decoded source timestamp (`DecodedFlow.source_realtime_usec`) into
2250
- `_SOURCE_REALTIME_TIMESTAMP` while preserving journal entry realtime as
2251
- receive/write time. This matches the append-only journal contract and
2252
- handles out-of-order source timestamps across exporters. Evidence:
2253
- `src/crates/netflow-plugin/src/ingest/service/runtime.rs:82-139` and
2254
- `src/crates/netflow-plugin/src/main_tests.rs:76-94`.
2255
-- `docs/network-flows/configuration.md` now states that `timestamp_source`
2256
- controls stored source timestamp metadata, not dashboard query windows.
2257
- Query windows and tier selection still use journal entry realtime.
2258
-- The pcap acceptance conflict is no longer open. Public docs now say
2259
- "packet-capture file" and `tcpdump -w`; they no longer mention pcap by
2260
- name. This keeps troubleshooting guidance without implying a pcap ingest
2261
- feature.
2262
-- Validation: `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml timestamp_source -- --nocapture`
2263
- passed with 5 tests. A pre-existing warning remains in
2264
- `src/crates/netflow-plugin/src/startup_memory_tests.rs` for an unused
2265
- `bytesize::ByteSize` import.
2266
-- Validation: `git diff --check` passed for the SOW, Network Flows docs,
2267
- metadata, generated integration docs, and touched netflow-plugin Rust
2268
- files.
2269
-- Validation: grep now finds no stale public-doc occurrences of
2270
- `journal.size_of_journal_files`, `journal.duration_of_journal_files`,
2271
- `standard port`, `Network Flows tab`, `Sources /`, `[Sources]`,
2272
- `tier 0`, `tier-0`, unsupported ``or `last`` wording, IPFIX observation
2273
- time wording, random-country/internal-IP wording, or pcap wording. The
2274
- only timestamp-source grep hit is the intentional docs warning that the
2275
- dashboard time picker does not query by exporter timestamps.
2276
-
2277
-User decision:
2278
-
2279
-- Public Network Flows documentation and generated integration cards are
2280
- end-user/operator documentation. They must not contain internal source-code
2281
- citations, internal test-status citations, quality-gate evidence, SOW/review
2282
- notes, or "current implementation state" wording. They should document
2283
- supported behavior, operator-visible limits, configuration contracts,
2284
- troubleshooting signals, and production cautions without exposing internal
2285
- proof or test gaps. Internal evidence belongs in the SOW, not in Learn docs.
2286
-
2287
-F26 repair completed:
2288
-
2289
-- Removed internal source-code paths, line references, source-test notes,
2290
- "current implementation state" wording, and test-gap disclosures from
2291
- `docs/network-flows/*.md` and `src/crates/netflow-plugin/metadata.yaml`.
2292
-- Rewrote those passages into operator-facing behavior: supported enrichment
2293
- semantics, visible limits, configuration validation, refresh cadence, TLS
2294
- safety, troubleshooting symptoms, and production cautions.
2295
-- Regenerated the integration cards with
2296
- `python3 integrations/gen_integrations.py` and
2297
- `python3 integrations/gen_docs_integrations.py`.
2298
-- Generated integration files still contain `custom_edit_url` and `meta_yaml`
2299
- entries inside their `<!--startmeta ... endmeta-->` blocks. These are
2300
- generator metadata, not user-facing page body text.
2301
-
2302
-F26 validation:
2303
-
2304
-- `rg -n 'src/|github\.com/netdata/netdata/(blob|tree)/master/src|\.rs:[0-9]|\.go:[0-9]|unit test|unit-tested|integration-test|integration-tested|integration test|NOT integration|Source code:|defined at|schema is defined|validated at|validation rejects values|production logs|current build|in this build|fixtures|tests here|repository|quality gate|current state|not tested|not validated|test gap|line reference|current implementation|implementation state|codebase|SOW|regression|review pass|internal citation' docs/network-flows src/crates/netflow-plugin/metadata.yaml`
2305
- returned no matches.
2306
-- The same pattern scan against `src/crates/netflow-plugin/integrations/*.md`
2307
- while skipping generated `<!--startmeta ... endmeta-->` blocks returned no
2308
- matches.
2309
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml")); print("metadata yaml ok")'`
2310
- passed.
2311
-- `git diff --check` passed for the SOW, Network Flows docs, metadata,
2312
- generated integration docs, and touched netflow-plugin Rust files.
2313
-
2314
-F26 residual findings from full-page subagent review:
2315
-
2316
-- The prior regex validation was too narrow. It caught explicit source paths,
2317
- source line references, test-gap wording, and quality-gate labels, but missed
2318
- less literal end-user trust issues such as "today", "known feature gap",
2319
- "verified against", "same code path", implementation type names, and
2320
- benchmark-provenance phrasing.
2321
-- Read-only subagents reviewed the assigned public pages in full, not only by
2322
- grep. Scope covered all hand-authored Network Flows markdown pages,
2323
- `src/crates/netflow-plugin/metadata.yaml`, and all generated integration
2324
- cards.
2325
-
2326
-Residual docs findings:
2327
-
2328
-- **Visualization current-state / backlog wording.**
2329
- `docs/network-flows/visualization/dashboard-cards.md:72,79,81` says signals
2330
- "aren't published today", are "collected internally", and are "not hard to
2331
- add but haven't been needed enough yet". This is product backlog / internal
2332
- state wording.
2333
- `docs/network-flows/visualization/filters-facets.md:32,68` says negative
2334
- matching is a "known feature gap" and that no good workaround exists "today".
2335
- These should be rewritten as stable supported/unsupported behavior.
2336
-- **Sizing page benchmark-provenance wording.**
2337
- `docs/network-flows/sizing-capacity.md:23,50,96` exposes benchmark machine
2338
- details, synthetic benchmark provenance, and "bench numbers" in public docs.
2339
- The page should present operator sizing guidance directly; evidence stays in
2340
- the SOW.
2341
-- **Enrichment / validation developer wording.**
2342
- `docs/network-flows/enrichment.md:107,156` says multiple inputs use the
2343
- "same code path"; this is implementation wording.
2344
- `docs/network-flows/enrichment.md:230,276` uses "today" limitation wording.
2345
- `docs/network-flows/enrichment.md:393` exposes the source-tree-style
2346
- `cmd/ris/` path in prose.
2347
- `docs/network-flows/validation.md:22,68` says the plugin does not publish
2348
- per-exporter ingest counters "today"; this should be stable limitation
2349
- wording.
2350
-
2351
-Residual metadata / generated-card findings:
2352
-
2353
-- **Quality-gate / verification wording in generated cards.**
2354
- `src/crates/netflow-plugin/metadata.yaml:2015` and generated
2355
- `src/crates/netflow-plugin/integrations/aws_ip_ranges.md:41` say the AWS
2356
- schema was "verified against the live document".
2357
- `src/crates/netflow-plugin/metadata.yaml:2666` and generated
2358
- `src/crates/netflow-plugin/integrations/azure_ip_ranges.md:147` say Azure
2359
- behavior was "verified against the upstream concept page".
2360
- These are internal review-evidence statements.
2361
-- **Current implementation-state wording in generated cards.**
2362
- `src/crates/netflow-plugin/metadata.yaml:2305` and generated
2363
- `src/crates/netflow-plugin/integrations/gcp_ip_ranges.md:60` say ETags are
2364
- not used for conditional fetches "today".
2365
- `src/crates/netflow-plugin/metadata.yaml:4339,4503` and generated
2366
- `src/crates/netflow-plugin/integrations/decapsulation.md:54,265` say parsed
2367
- tunnel fields are "not surfaced today".
2368
-- **Internal implementation names in generated cards.**
2369
- `src/crates/netflow-plugin/metadata.yaml:2939` and generated
2370
- `src/crates/netflow-plugin/integrations/netbox.md:131` expose
2371
- `RemoteNetworkSourceConfig`.
2372
- `src/crates/netflow-plugin/metadata.yaml:3432` and generated
2373
- `src/crates/netflow-plugin/integrations/generic_json-over-http_ipam.md:321`
2374
- expose `decode_remote_records`.
2375
-- **Untrusted-feature wording in decapsulation docs.**
2376
- `src/crates/netflow-plugin/metadata.yaml:4395,4408,4414` and generated
2377
- `src/crates/netflow-plugin/integrations/decapsulation.md:124,137,143` say
2378
- "the project has verified", "have not been verified by the project", and
2379
- "unverified". Operator docs should state recommended/supported vendor
2380
- configuration patterns without exposing internal validation status.
2381
-
2382
-Repair classification:
2383
-
2384
-- All residual findings are documentation-source fixes. No code behavior change
2385
- is indicated by this review.
2386
-- Hand-authored markdown findings should be fixed in the corresponding
2387
- `docs/network-flows/*.md` file.
2388
-- Generated-card findings must be fixed in
2389
- `src/crates/netflow-plugin/metadata.yaml`, then regenerated with
2390
- `python3 integrations/gen_integrations.py` and
2391
- `python3 integrations/gen_docs_integrations.py`. Do not hand-edit generated
2392
- integration cards.
2393
-- At classification time, F26 remained open until these residual findings were
2394
- repaired and a broader wording scan was added to validation.
2395
-
2396
-F26 residual repair completed:
2397
-
2398
-- Rewrote hand-authored docs so they state stable supported/unsupported
2399
- behavior without backlog, "today", implementation, or benchmark-provenance
2400
- wording.
2401
-- Rewrote `src/crates/netflow-plugin/metadata.yaml` source text to remove
2402
- quality-gate evidence, implementation type/function names, "today" current
2403
- state wording, and "verified/unverified by the project" statements.
2404
-- Regenerated generated integration cards with
2405
- `python3 integrations/gen_integrations.py` and
2406
- `python3 integrations/gen_docs_integrations.py`.
2407
-- All residual findings above are now repaired. Generated integration files
2408
- were updated only through `metadata.yaml`.
2409
-
2410
-F26 residual validation:
2411
-
2412
-- Residual-pattern scan returned no matches over hand-authored docs and
2413
- metadata:
2414
- `RemoteNetworkSourceConfig|verified against the live document|conditional fetches today|verified against the upstream concept page|decode_remote_records|not surfaced today|fields today|project has verified|not been verified|unverified|not visible today|cannot be consumed today|same code path|cmd/ris/|doesn.t publish per-exporter.*today|aren.t published today|collected internally|not hard to add|known feature gap|no good workaround exists today|bench numbers|synthetic high-cardinality|i9-class|FireCuda`.
2415
-- The same residual-pattern scan over generated
2416
- `src/crates/netflow-plugin/integrations/*.md` body text, skipping
2417
- `<!--startmeta ... endmeta-->`, returned no matches.
2418
-- The original stricter internal-citation scan over hand-authored docs,
2419
- metadata, and generated integration body text also returned no matches.
2420
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml")); print("metadata yaml ok")'`
2421
- passed.
2422
-- `git diff --check` passed for the SOW, Network Flows docs, metadata, and
2423
- generated integration docs.
2424
-
2425
-Quality review decision:
2426
-
2427
-- User decision 2026-05-07: do not change deployment guidance to avoid
2428
- double counting at ingestion. The correct deployment guidance is to export
2429
- all relevant interfaces and all directions. Double counting is a
2430
- visualization and interpretation guideline: users must understand what they
2431
- selected and how bidirectional/interface-overlap views should be read.
2432
-
2433
-Quality review repair completed:
2434
-
2435
-- Aligned IP-intelligence install/default behavior across public docs:
2436
- native packages ship stock DB-IP MMDBs under stock data, source builds do
2437
- not include stock MMDBs, the downloader writes fresher cache copies, and
2438
- Netdata does not install a downloader timer or cron job.
2439
-- Reworded retention defaults as suitable for first validation and small
2440
- deployments, with production retention sized from observed flow rate.
2441
-- Replaced generated default-behavior boilerplate across all Network Flows
2442
- integration cards so cards no longer claim generic "no limits" or
2443
- "no significant performance impact".
2444
-- Fixed UDP-drop quick-reference commands to use `ss -uamn` and
2445
- `/proc/net/snmp`, not `/proc/net/udp` as a drop counter source.
2446
-- Replaced broken `#enrichment-geoip` anchors with `#enrichment`.
2447
-- Replaced literal `${CMDB_TOKEN}` examples with `<CMDB_TOKEN>` placeholders,
2448
- matching the "headers are passed verbatim" contract.
2449
-- Replaced the non-existent `Application` grouping reference with
2450
- `Destination Port` / service wording.
2451
-- Preserved all-interfaces/all-directions deployment guidance per the user
2452
- decision; no ingress-only deployment change was made.
2453
-
2454
-Quality review validation:
2455
-
2456
-- Regenerated integration artifacts with
2457
- `python3 integrations/gen_integrations.py` and
2458
- `python3 integrations/gen_docs_integrations.py`.
2459
-- Grep validation returned no matches for:
2460
- `${CMDB_TOKEN}`, `enrichment-geoip`, `Application`, `cat /proc/net/udp`,
2461
- stale default-retention wording, `Network Flows tab`, and default-card
2462
- "does not impose any limits" / "significant performance impact" wording in
2463
- Network Flows generated cards.
2464
-- Residual internal/current-state wording scans over hand-authored docs,
2465
- metadata, and generated card body text returned no matches.
2466
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml")); print("metadata yaml ok")'`
2467
- passed.
2468
-- `git diff --check` passed for the SOW, Network Flows docs, metadata, and
2469
- generated integration docs.
2470
-
2471
-## Regression Closeout - 2026-05-08
2472
-
2473
-Final status: completed.
2474
-
2475
-Closeout evidence:
2476
-
2477
-- Every regression finding F1-F26 has a repair note and validation evidence in
2478
- this regression log.
2479
-- The final quality review findings were repaired in hand-authored docs,
2480
- `metadata.yaml`, and regenerated integration cards.
2481
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml timestamp_source -- --nocapture`
2482
- passed with 5 tests. A pre-existing warning remains in
2483
- `src/crates/netflow-plugin/src/startup_memory_tests.rs` for an unused
2484
- `bytesize::ByteSize` import.
2485
-- `python3 integrations/gen_integrations.py` passed.
2486
-- `python3 integrations/gen_docs_integrations.py` passed.
2487
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml")); print("metadata yaml ok")'`
2488
- passed.
2489
-- Targeted public-doc quality scans over `docs/network-flows/`,
2490
- `src/crates/netflow-plugin/metadata.yaml`, and generated integration-card
2491
- body text returned no matches for stale anchors, misleading credential
2492
- placeholders, stale UI wording, invalid grouping field names, bad UDP-drop
2493
- commands, internal-current-state wording, source-code citations, or generic
2494
- default-behavior boilerplate.
2495
-- `git diff --check` passed for the SOW, Network Flows docs, metadata,
2496
- generated integration docs, and touched netflow-plugin Rust files.
2497
-- `.agents/sow/audit.sh` was run after final move. It reported SOW 14
2498
- status/directory as consistent in `done/` and failed only on an
2499
- unrelated pre-existing sensitive-data pattern in
2500
- `.agents/skills/mirror-netdata-repos/SKILL.md` plus existing TODO-file
2501
- warnings. None of those are part of this SOW's staged scope.
2502
-
2503
-Artifact maintenance gate for regression close:
2504
-
2505
-- **AGENTS.md**: no update needed; no workflow or project-wide guardrail changed.
2506
-- **Runtime project skills**: no update needed; no new integration-pipeline or
2507
- Learn-site process behavior was discovered beyond existing skill guidance.
2508
-- **Specs**: no update needed; this was a documentation/metadata repair and did
2509
- not change the product contract beyond correcting public docs to existing
2510
- behavior.
2511
-- **End-user/operator docs**: updated Network Flows docs, `metadata.yaml`, and
2512
- regenerated integration cards.
2513
-- **End-user/operator skills**: no update needed; the user rejected AI-skill
2514
- cross-linking as not mandatory for this SOW.
2515
-- **SOW lifecycle**: status changed to `completed`; file moved from
2516
- `.agents/sow/current/` to `.agents/sow/done/` together with the repair
2517
- commit.
2518
-
2519
-Follow-up mapping:
2520
-
2521
-- The original follow-up list remains tracked above. No new deferred item was
2522
- introduced by the regression repair. The user explicitly rejected changing
2523
- deployment guidance to ingress-only; double counting remains a visualization
2524
- and interpretation guideline, not an ingestion-side avoidance requirement.
2525
-
2526
-## Regression - 2026-05-08
2527
-
2528
-### Trigger
2529
-
2530
-PR #22449 received post-close reviewer and CI feedback after SOW 14 had been
2531
-marked `completed` and moved to `.agents/sow/done/`.
2532
-
2533
-### Purpose
2534
-
2535
-Bring PR #22449 back to merge-ready state for end-user Network Flows
2536
-documentation by addressing all valid review, CI, Codacy, and local Learn
2537
-preview findings. Add a durable, user-triggered skill/workflow for building
2538
-Learn locally with the contents of a documentation PR before merge, so future
2539
-documentation PRs can be inspected in a browser before release.
2540
-
2541
-### Evidence
2542
-
2543
-- `bash .agents/skills/pr-reviews/scripts/fetch-all.sh 22449` returned 9 open
2544
- automated review threads, with no human review comments in the fetched
2545
- snapshot.
2546
-- `bash .agents/skills/pr-reviews/scripts/ci-status.sh 22449` reported
2547
- failures in `yamllint` and `check-documentation`, `ACTION_REQUIRED` from
2548
- Codacy, 0 Sonar findings, and many still-running build checks.
2549
-- `bash .agents/skills/pr-reviews/scripts/fetch-sonar-findings.sh 22449`
2550
- returned 0 Sonar issues and 0 hotspots.
2551
-- `bash .agents/skills/codacy-audit/scripts/pr-issues.sh 22449` failed locally
2552
- before reporting findings because the helper passed a very large JSON array
2553
- through `jq --argjson`, hitting `Argument list too long`.
2554
-- `python3 ${NETDATA_REPOS_DIR}/learn/ingest/ingest.py --help` failed in the
2555
- system interpreter due to missing `pandas`; the checked-in Learn `venv`
2556
- also lacked required packages such as `GitPython`.
2557
-- After commit `0a3eda6614` was pushed and all prior threads were replied to
2558
- and resolved, `bash .agents/skills/pr-reviews/scripts/fetch-all.sh 22449`
2559
- returned 9 new open automated review threads on the new PR head and no open
2560
- human review threads.
2561
-- The new PR #22449 comments were verified against source evidence:
2562
- `.github/workflows/check-markdown.yml:54-69` regenerates `COLLECTORS.md`
2563
- before Learn ingest but does not diff-check the committed file;
2564
- `src/crates/netflow-plugin/src/enrichment/init.rs:50-64` keeps the enricher
2565
- enabled for provider-chain-only config; `network_sources/runtime.rs:24-40`
2566
- scans loaded network-source records linearly; `network_sources/service.rs:91-96`
2567
- logs HTTP refresh failures as warnings; and `reqwest-0.13.2` converts URL
2568
- userinfo to HTTP Basic auth during request build.
2569
-- A second fetch during this repair returned 2 additional automated review
2570
- threads: one on standalone CLI retention flags and one on ambiguous
2571
- visualization panel count wording.
2572
-- The CLI retention finding was valid for standalone mode:
2573
- `src/crates/netflow-plugin/src/plugin_config/runtime.rs:7-11` uses
2574
- `PluginConfig::parse()` outside Netdata, and commit `f00390e2f5` removed the
2575
- legacy top-level CLI flags while keeping `JournalConfig.tiers` skipped for
2576
- Clap.
2577
-- After commit `6f5805979d` was pushed and 11 threads were replied to and
2578
- resolved, the PR review watcher detected 2 new open automated review threads
2579
- on `src/crates/netflow-plugin/src/ingest/rebuild.rs`.
2580
-- The new rebuild findings were valid: `rebuild_materialized_from_raw()` called
2581
- `scan_journal_files_forward()` directly in the async startup path, while query
2582
- paths already use `tokio::task::spawn_blocking()` for blocking journal scans;
2583
- the rebuild timeout was checked only every 1024 scanned entries.
2584
-
2585
-### Open repair items
2586
-
2587
-- R8.1: Rewrite the SOW close-gate paragraph at the prior
2588
- `SOW-0014...md:1060` in historical tense.
2589
-- R8.2: Fix the classifier integration example so the regex matches the
2590
- documented three-letter region suffix.
2591
-- R8.3: Remove or correct the decapsulation troubleshooting advice that
2592
- suggests multiple plugin instances despite the documented single-instance
2593
- model.
2594
-- R8.4: Correct the Fedora/RHEL `geoipupdate` package name.
2595
-- R8.5: Make the static-metadata typo example actually use an unknown key.
2596
-- R8.6: Strengthen `main_tests.rs` so timestamp-source persistence cannot pass
2597
- when both journal fields are missing.
2598
-- R8.7: Tighten sizing guidance around the ~25k flows/s single-agent planning
2599
- ceiling.
2600
-- R8.8: Make storage safety-margin guidance internally consistent.
2601
-- R8.9: Correct the UDP troubleshooting note to acknowledge per-socket
2602
- `drops` in `/proc/net/udp` while keeping `RcvbufErrors` as the system-wide
2603
- counter.
2604
-- R8.10: Fix or work around the Codacy PR-fetch helper failure, then triage
2605
- Codacy findings for PR #22449.
2606
-- R8.11: Investigate and fix the CI `yamllint` and `check-documentation`
2607
- failures.
2608
-- R8.12: Create durable local Learn preview guidance that triggers only when a
2609
- user explicitly asks to build/inspect Learn from a PR.
2610
-- R8.13: Build or serve Learn locally using PR #22449 contents and record the
2611
- ingest/build/browser validation result.
2612
-- R8.14: Fix automated review comments about contradictory retention comments
2613
- and stale shared-budget retention wording.
2614
-- R8.15: Fix the broken decapsulation integration icon found during local Learn
2615
- browser inspection.
2616
-- R8.16: Verify integration-card source links in hand-authored Network Flows
2617
- docs are compatible with Learn ingest.
2618
-- R8.17: Restore bounded startup behavior for receive-time raw rebuild scans.
2619
-- R8.18: Reconcile the SOW scope text so the durable record matches the
2620
- accepted documentation scope.
2621
-- R8.19: Ensure the generated Learn "Monitor anything" page lists Network
2622
- Flows protocols and enrichment integrations, and update the integration
2623
- lifecycle skill with that mechanism.
2624
-- R8.20: Normalize Network Flows catalog descriptions so flow-source rows say
2625
- they collect network flow records and enrichment rows say they enrich or
2626
- annotate network flows, instead of describing provider publication
2627
- mechanisms, variables, defaults, or setup settings.
2628
-- R8.21: Correct `integrations-lifecycle` guidance so it says
2629
- `check-markdown.yml` catches broken generated `COLLECTORS.md` content during
2630
- Learn ingest, not stale committed artifact drift.
2631
-- R8.22: Update local Learn preview guidance to copy tracked plus untracked
2632
- non-ignored PR files into the isolated source preview.
2633
-- R8.23: Add the 32-bit packaging caveat anywhere docs tell users to run
2634
- `topology-ip-intel-downloader`.
2635
-- R8.24: Replace AWS `transform: "."` "empty result" wording with the actual
2636
- row-mapping/schema failure around the missing required `prefix` field.
2637
-- R8.25: Correct URL-credential docs for remote network sources: URL userinfo
2638
- becomes HTTP Basic auth, while explicit headers remain recommended.
2639
-- R8.26: Correct enrichment docs so provider-chain-only config is described as
2640
- enabling the enricher.
2641
-- R8.27: Make the Codacy helper's temporary file pattern portable.
2642
-- R8.28: Correct NetBox/network-source failure and performance wording:
2643
- refresh failures are warning logs, and runtime cost scales with loaded
2644
- network-source records instead of claiming trie lookup.
2645
-- R8.29: Sweep same-class wording across generated cards and hand-authored
2646
- docs so the next review does not rediscover the same problems in adjacent
2647
- pages.
2648
-- R8.30: Restore standalone CLI retention tuning without reopening the YAML
2649
- global-retention schema.
2650
-- R8.31: Clarify the visualization overview panel count so the text matches the
2651
- listed UI surfaces.
2652
-- R8.32: Move receive-time raw rebuild journal scanning and decompression off
2653
- the async startup task so Tokio runtime workers are not blocked.
2654
-- R8.33: Enforce raw rebuild timeout checks for every scanned entry rather than
2655
- only every 1024 entries.
2656
-
2657
-### Validation plan
2658
-
2659
-- Re-fetch all PR comments before push and verify no new review items were
2660
- missed.
2661
-- Run the narrow docs and Rust validations affected by the fixes.
2662
-- Run local Learn ingest/build or dev-server preview with the local PR checkout
2663
- as the `netdata` source.
2664
-- Record the local Learn URL, process PID, and cleanup path if a preview server
2665
- is started.
2666
-- Move this SOW back to `.agents/sow/done/` after PR review items, relevant
2667
- CI failures, and local Learn preview validation are handled.
2668
-
2669
-### Repair completed
2670
-
2671
-- R8.1: Reworded the prior close-gate paragraph in historical tense.
2672
-- R8.2: Fixed the classifier example regex to match the documented
2673
- three-letter region suffix.
2674
-- R8.3: Removed the decapsulation advice that implied running multiple plugin
2675
- instances.
2676
-- R8.4: Replaced `GeoIP-update` with the Fedora/RHEL package name
2677
- `geoipupdate`.
2678
-- R8.5: Changed the static-metadata typo example to `if_index`, leaving
2679
- accepted aliases documented separately.
2680
-- R8.6: Strengthened the timestamp-source e2e test so missing raw journal
2681
- fields cannot pass as `None == None`.
2682
-- R8.7/R8.8: Tightened the sizing page around the ~25k flows/s planning
2683
- envelope and made the storage safety margin consistently `1.2x to 1.5x`.
2684
-- R8.9: Corrected UDP troubleshooting so `/proc/net/udp` is described as
2685
- per-socket `drops`, while `/proc/net/snmp` `RcvbufErrors` remains the
2686
- system-wide signal.
2687
-- R8.10: Fixed the Codacy helper to avoid passing a large issue array through
2688
- `jq --argjson`; it now uses a temporary JSON file and `--slurpfile`.
2689
-- R8.11: Fixed `yamllint` findings in `metadata.yaml` and `configs/netflow.yaml`;
2690
- replaced hand-authored Learn links to generated integration pages with source
2691
- markdown links so Learn ingest rewrites them correctly.
2692
-- R8.12: Added `.agents/skills/learn-pr-preview/SKILL.md`, updated
2693
- `AGENTS.md`, and added
2694
- `.agents/skills/learn-site-structure/how-tos/preview-documentation-pr-locally.md`.
2695
-- R8.13: Built an isolated Learn preview from PR #22449 contents, ran ingest,
2696
- built Docusaurus, served the static build locally, and browser-inspected
2697
- representative Network Flows pages.
2698
-- R8.14: Reworded the Rust retention comment so optional fields and the
2699
- resolved-tier validation rule agree; corrected visualization docs that still
2700
- described raw-tier retention as a shared budget.
2701
-- R8.15: Replaced the missing `tunnel.svg` decapsulation icon with the existing
2702
- hosted `network-wired.svg` icon and regenerated integration artifacts.
2703
-- R8.16: Verified by local ingest that source docs must link to the source
2704
- integration markdown files; Learn correlates those links to final routes.
2705
- Direct final `/docs/network-flows/...` links in source markdown fail
2706
- `--fail-links-netdata`, so the source integration links were retained.
2707
-- R8.17: Restored the 30-second raw rebuild timeout for the new direct
2708
- receive-time raw scan path, checking the elapsed time during scan progress.
2709
-- R8.18: Updated the SOW purpose/request wording to match the final accepted
2710
- scope: document existing Network Flows behavior and exclude non-existent
2711
- topology drilldown behavior.
2712
-- R8.19: Updated `integrations/gen_doc_collector_page.py` so the top-level
2713
- `flows` category is grouped as a first-class `Network Flows` section in
2714
- `src/collectors/COLLECTORS.md`; regenerated the file; updated the
2715
- `integrations-lifecycle` skill and added a how-to for this generator rule.
2716
-- R8.20: Updated `src/crates/netflow-plugin/metadata.yaml` so generated
2717
- Network Flows catalog rows use action-oriented user copy:
2718
- `Collect network flow records...`, `Enrich network flows...`, or
2719
- `Annotate network flows...`; regenerated per-integration markdown and
2720
- `src/collectors/COLLECTORS.md`; updated the `integrations-lifecycle`
2721
- skill with description-authoring rules.
2722
-- R8.21: Updated `.agents/skills/integrations-lifecycle/pipeline.md` so
2723
- `check-markdown.yml` is documented as regenerating `COLLECTORS.md` before
2724
- Learn ingest and catching broken generated content, while artifact drift is
2725
- left to the integration regeneration workflow.
2726
-- R8.22: Updated `.agents/skills/learn-pr-preview/SKILL.md` and the matching
2727
- Learn-site how-to to use `git ls-files -co --exclude-standard`, so previews
2728
- include intentional untracked PR docs without copying ignored build output.
2729
-- R8.23: Added the packaged 32-bit downloader caveat to IPtoASN and DB-IP
2730
- generated cards plus hand-authored installation, validation, and downloader
2731
- docs.
2732
-- R8.24: Updated AWS IP Ranges metadata and generated card so the default
2733
- `transform: "."` failure is described as missing required `prefix` rows, not
2734
- as empty transform output.
2735
-- R8.25: Updated generic JSON-over-HTTP IPAM metadata/generated docs and the
2736
- hand-authored enrichment page so URL userinfo is described as HTTP Basic auth
2737
- conversion, with explicit `Authorization` headers recommended for clarity.
2738
-- R8.26: Updated the hand-authored enrichment page so the enricher is described
2739
- as running when any enrichment feature is configured, including provider
2740
- chains.
2741
-- R8.27: Updated `.agents/skills/codacy-audit/scripts/pr-issues.sh` and its
2742
- how-to to use an explicit portable `mktemp` template.
2743
-- R8.28/R8.29: Updated AWS, Azure, NetBox, and generic HTTP network-source
2744
- generated cards so runtime enrichment cost is described as prefix matching
2745
- over loaded records; updated NetBox troubleshooting so HTTP errors are logged
2746
- as refresh-failed warnings.
2747
-- R8.30: Restored
2748
- `--netflow-retention-size-of-journal-files` and
2749
- `--netflow-retention-duration-of-journal-files` as CLI-only compatibility
2750
- aliases. They apply uniformly to all tiers in standalone mode while YAML
2751
- remains per-tier-only.
2752
-- R8.31: Reworded the visualization overview to list five panel types:
2753
- Sankey, Table, Time-Series, maps, and the 3D globe.
2754
-- R8.32: Moved the blocking raw rebuild journal scan into
2755
- `tokio::task::spawn_blocking()` and streamed parsed `FlowFields` back to the
2756
- async ingest task over a bounded channel before observing materialized tiers.
2757
-- R8.33: Changed the rebuild scan timeout guard to check elapsed time on every
2758
- entry, before and after payload parsing.
2759
-
2760
-### Code-only review handling
2761
-
2762
-The requested code-only subagent review found timestamp consistency risks. On
2763
-verification, the important false premise was that `timestamp_source` should
2764
-drive dashboard query windows or rollup tier selection. The public contract in
2765
-`docs/network-flows/configuration.md` says the Network Flows view uses journal
2766
-entry receive time for query windows and tier selection; `timestamp_source`
2767
-controls stored source timestamp metadata.
2768
-
2769
-Repairs:
2770
-
2771
-- Kept live materialized tier observation on receive time, matching the public
2772
- contract and raw journal append-time ordering.
2773
-- Changed rebuild to scan recently received raw entries by journal entry time
2774
- and replay them into materialized tiers by receive time, instead of querying
2775
- the raw journal by `_SOURCE_REALTIME_TIMESTAMP`.
2776
-- Changed the rebuild upper bound to include the current second.
2777
-- Extended the timestamp-source e2e test to prove:
2778
- raw `_SOURCE_REALTIME_TIMESTAMP` equals the decoded flow start timestamp;
2779
- raw journal entry realtime remains receive/write time;
2780
- live materialized tiers use the receive-time bucket; and rebuild replays raw
2781
- entries into the same receive-time bucket.
2782
-
2783
-### Validation evidence
2784
-
2785
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml timestamp_source -- --nocapture`
2786
- passed: 5 tests passed, 0 failed. The only warning was the pre-existing
2787
- unused `bytesize::ByteSize` import in
2788
- `src/crates/netflow-plugin/src/startup_memory_tests.rs`.
2789
-- `cargo fmt --manifest-path src/crates/netflow-plugin/Cargo.toml --check`
2790
- passed after formatting.
2791
-- `yamllint src/crates/netflow-plugin/metadata.yaml src/crates/netflow-plugin/configs/netflow.yaml`
2792
- passed.
2793
-- `python3 integrations/gen_integrations.py` passed.
2794
-- `python3 integrations/gen_docs_integrations.py` passed.
2795
-- `git diff --check` passed.
2796
-- A hosted-icon check over all Network Flows metadata icons passed:
2797
- `network-wired.svg` returned `200 image/svg+xml`.
2798
-- `python3 integrations/gen_doc_collector_page.py` passed and the generated
2799
- `src/collectors/COLLECTORS.md` contains a `### Network Flows` section with
2800
- NetFlow, IPFIX, sFlow, and enrichment integrations.
2801
-- The generated `src/collectors/COLLECTORS.md` Network Flows table now uses
2802
- catalog-style descriptions such as `Enrich network flows with...`,
2803
- `Annotate network flows with...`, and `Collect network flow records...`
2804
- rather than setup, option, or provider-publication wording.
2805
-- Final pre-commit reviewer verification re-fetched PR #22449 and confirmed
2806
- 12 open threads on the old pushed head; every thread was checked against the
2807
- current local tree and the corresponding fix is present in source/generated
2808
- files before committing.
2809
-- `git diff --check` passed.
2810
-- `cargo fmt --manifest-path src/crates/netflow-plugin/Cargo.toml --check`
2811
- passed.
2812
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml"))'`
2813
- passed.
2814
-- `.agents/sow/audit.sh` exited 2 because of pre-existing unrelated repository
2815
- hygiene findings: a sensitive-pattern warning in
2816
- `.agents/skills/mirror-netdata-repos/SKILL.md`, the unrelated current
2817
- SOW-0012 gate warning, non-project skill classification warnings, and
2818
- existing root TODO files. SOW 14 itself reports status/directory consistency
2819
- as `completed` in `.agents/sow/done/`.
2820
-- Local Learn ingest from an isolated source copy passed with
2821
- `--local-repo netdata:<preview-source> --ignore-on-prem-repo --use_plain_https --fail-links-netdata`.
2822
-- Local Learn Docusaurus build passed with Node `22.14.0`, Yarn `1.22.22`,
2823
- and `NODE_OPTIONS=--max_old_space_size=4096`.
2824
-- Build warnings were not PR-specific Network Flows failures: Docusaurus still
2825
- reports existing site-wide broken anchors and an existing duplicate
2826
- `/docs/collecting-metrics/service-discovery` route.
2827
-- Browser inspection returned HTTP 200, expected H1, no 404 page, and no
2828
- MDX/runtime error for:
2829
- `/docs/network-flows/`,
2830
- `/docs/network-flows/retention-and-tiers`,
2831
- `/docs/network-flows/enrichment-methods/static-metadata`,
2832
- `/docs/network-flows/flow-protocols/netflow`, and
2833
- `/docs/network-flows/configuration`.
2834
-- The Static Metadata page rendered placeholders such as
2835
- `enrichment.metadata_static.exporters.<key>.if_indexes` as readable text, not
2836
- literal `<key>` and not MDX JSX.
2837
-- Browser inspection of
2838
- `/docs/network-flows/enrichment-methods/decapsulation` confirmed the
2839
- integration icon loaded successfully from `network-wired.svg` with non-zero
2840
- rendered dimensions. Only external analytics requests failed in the browser
2841
- session.
2842
-- Second review-iteration validation after the 9 new automated comments:
2843
- `python3 integrations/gen_integrations.py`,
2844
- `python3 integrations/gen_docs_integrations.py`, and
2845
- `python3 integrations/gen_doc_collector_page.py` passed.
2846
-- `yamllint src/crates/netflow-plugin/metadata.yaml src/crates/netflow-plugin/configs/netflow.yaml`
2847
- passed.
2848
-- `python3 -c 'import yaml; yaml.safe_load(open("src/crates/netflow-plugin/metadata.yaml"))'`
2849
- passed.
2850
-- `bash -n .agents/skills/codacy-audit/scripts/pr-issues.sh` passed.
2851
-- `git diff --check` passed.
2852
-- `cargo fmt --manifest-path src/crates/netflow-plugin/Cargo.toml --check`
2853
- passed.
2854
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml journal_cli_retention_aliases_apply_to_all_tiers -- --nocapture`
2855
- passed: 1 test passed, 0 failed. The only warning was the pre-existing unused
2856
- `bytesize::ByteSize` import in `startup_memory_tests.rs`.
2857
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml plugin_config::tests:: -- --nocapture`
2858
- passed: 27 tests passed, 0 failed. The only warning was the same pre-existing
2859
- unused `bytesize::ByteSize` import in `startup_memory_tests.rs`.
2860
-- The post-fix wording sweep found no remaining instances of the exact stale
2861
- phrases called out by the second review pass.
2862
-- Third review-iteration fetch after commit `6f5805979d` returned 2 new open
2863
- automated threads, both in `src/crates/netflow-plugin/src/ingest/rebuild.rs`.
2864
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml e2e_timestamp_source_first_switched_is_persisted_as_source_timestamp -- --nocapture`
2865
- passed: 1 test passed, 0 failed. This test exercises the raw rebuild path
2866
- after deleting materialized tier directories. The only warning was the same
2867
- pre-existing unused `bytesize::ByteSize` import in `startup_memory_tests.rs`.
2868
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml plugin_config::tests:: -- --nocapture`
2869
- passed after the rebuild repair: 27 tests passed, 0 failed. The only warning
2870
- was the same pre-existing unused `bytesize::ByteSize` import.
2871
-- `cargo fmt --manifest-path src/crates/netflow-plugin/Cargo.toml --check`
2872
- passed after the rebuild repair.
2873
-- `cargo test --manifest-path src/crates/netflow-plugin/Cargo.toml -- --nocapture`
2874
- passed after the rebuild repair: 428 tests passed, 18 ignored, and the
2875
- vendored gRPC proto test passed. The visible lock-poison panic is expected
2876
- from a recovery test that still passed. The only compiler warning was the
2877
- same pre-existing unused `bytesize::ByteSize` import.
2878
-
2879
-### Artifact maintenance gate
2880
-
2881
-- **AGENTS.md**: updated to list the new `learn-pr-preview` skill and its
2882
- explicit trigger.
2883
-- **Runtime project skills**: added `learn-pr-preview`; updated
2884
- `learn-site-structure` with a local PR preview how-to; updated
2885
- `codacy-audit` for the large PR issue-list fetch gotcha; updated
2886
- `integrations-lifecycle` for the generated `COLLECTORS.md` / Monitor
2887
- Anything Network Flows section mechanism and metadata description-authoring
2888
- rules.
2889
-- **Specs**: no spec update needed; the code repair preserves the documented
2890
- timestamp contract rather than changing product behavior.
2891
-- **End-user/operator docs**: updated Network Flows docs, `metadata.yaml`, and
2892
- regenerated integration cards.
2893
-- **End-user/operator skills**: no update needed; the new workflow is a
2894
- repo-work skill for agents validating documentation PRs, not an end-user
2895
- operator skill.
2896
-- **SOW lifecycle**: SOW 14 repair is complete; status is `completed`, and
2897
- the file is moved back to `.agents/sow/done/` in the same commit as the
2898
- repair.
2899
-
2900
-## Regression - 2026-05-08 - Flow Integration Section Rendering
2901
-
2902
-### What broke
2903
-
2904
-The merged network-flow integration artifacts expose `metrics` as a JSON object
2905
-and `alerts` as a JSON array for `integration_type: flows` entries. Downstream
2906
-surfaces expect rendered markdown strings for integration content sections.
2907
-
2908
-Evidence:
2909
-
2910
-- `integrations/schemas/flows.json` delegates to `collector.json`, so flow
2911
- metadata legitimately contains collector-style `metrics` and `alerts`.
2912
-- `integrations/gen_integrations.py` `FLOWS_RENDER_KEYS` rendered only
2913
- `overview`, `related_resources`, `setup`, and `troubleshooting`, leaving
2914
- `metrics` and `alerts` in their source YAML shape.
2915
-- The website PR generated from the merged artifacts failed production-pinned
2916
- Hugo `0.140.0` because `themes/tailwind/layouts/partials/integration-tabs.html`
2917
- calls `markdownify` on `.integration.metrics`.
2918
-- The in-app integrations renderer in cloud-frontend adds tabs for truthy
2919
- `metrics` and `alerts`, while its Markdoc wrapper parses only string input.
2920
- The result would be blank Metrics and Alerts tabs after the next data sync.
2921
-- The cloud-frontend integration link checker calls `.match()` on markdown
2922
- fields, so raw objects/arrays in `metrics`/`alerts` can break that check too.
2923
-
2924
-### Why previous validation missed it
2925
-
2926
-The earlier closeout validated the Netdata integrations generator and local
2927
-Learn ingest/build, but did not validate the website Hugo renderer or the
2928
-cloud-frontend integrations consumer against the newly introduced
2929
-`integration_type: flows` artifacts. `gen_integrations.py` itself accepted the
2930
-raw structured fields because they are schema-valid before rendering.
2931
-
2932
-### Repair plan
2933
-
2934
-Render flow `metrics` and `alerts` through the same markdown templates used by
2935
-collector integrations. This keeps the source metadata schema unchanged and
2936
-restores the downstream contract: content sections in `integrations.json` and
2937
-`integrations.js` are markdown strings.
2938
-
2939
-### Validation plan
2940
-
2941
-- Run `python3 integrations/gen_integrations.py`.
2942
-- Verify all `flows` entries in `integrations/integrations.json` have string
2943
- `metrics` and string `alerts`.
2944
-- Run `python3 integrations/gen_docs_integrations.py`.
2945
-- Run `python3 integrations/gen_doc_collector_page.py`.
2946
-- Rebuild the website PR artifacts with the repaired generated
2947
- `integrations.json` and production-pinned Hugo `0.140.0`.
2948
-- Verify cloud-frontend's current renderer and link checker receive strings
2949
- for `flows` `metrics` and `alerts`.
2950
-
2951
-### Artifact updates needed
2952
-
2953
-- **Code**: update `integrations/gen_integrations.py` flow render keys.
2954
-- **Generated artifacts**: regenerate `integrations/integrations.json`,
2955
- `integrations/integrations.js`, per-integration markdown, and
2956
- `src/collectors/COLLECTORS.md` if the generator changes them.
2957
-- **Runtime project skills**: update `integrations-lifecycle` if the durable
2958
- downstream contract was not already documented clearly enough.
2959
-- **Specs**: no product behavior change is expected; this repairs generated
2960
- publishing artifacts.
2961
-- **End-user/operator docs**: no content change is expected beyond generated
2962
- artifacts.
2963
-
2964
-### Repair completed
2965
-
2966
-`integrations/gen_integrations.py` now renders the flow `alerts`, `metrics`,
2967
-and `functions` sections through the standard templates, matching the
2968
-collector-like schema that `flows.json` delegates to.
2969
-
2970
-### Validation evidence
2971
-
2972
-- `python3 integrations/gen_integrations.py` passed.
2973
-- `python3 integrations/gen_docs_integrations.py` passed.
2974
-- `python3 integrations/gen_doc_collector_page.py` passed.
2975
-- Flow artifact type check passed: every `integration_type: flows` entry in
2976
- `integrations/integrations.json` has string `metrics`, string `alerts`, and
2977
- string `functions`.
2978
-- Non-deploy markdown-field type check passed: no non-deploy integration emits
2979
- raw object or array values for `overview`, `setup`, `troubleshooting`,
2980
- `alerts`, `metrics`, `functions`, or `related_resources`.
2981
-- Website validation passed in a temporary copy of PR #1212 using the repaired
2982
- `integrations.json` and production-pinned Hugo `0.140.0`: `hugo --gc
2983
- --minify` built 3141 pages successfully.
2984
-- Cloud-frontend compatibility check passed against the repaired
2985
- `integrations.json`: all flow markdown fields inspected by
2986
- `scripts/checkIntegrations.js` were strings, and `getMarkdownUrls()` extracted
2987
- 52 markdown URLs without throwing.
2988
-- `git diff --check` passed.
2989
-- `.agents/sow/audit.sh` exited 2 because of a pre-existing unrelated
2990
- sensitive-pattern warning in
2991
- `.agents/skills/mirror-netdata-repos/SKILL.md`; SOW status/directory
2992
- consistency passed and SOW 14 reports `completed` in `.agents/sow/done/`.
2993
-
2994
-### Artifact maintenance gate
2995
-
2996
-- **AGENTS.md**: no update needed; this does not change repo-wide workflow.
2997
-- **Runtime project skills**: updated `integrations-lifecycle` with the
2998
- downstream markdown-string contract and the new-integration-type validation
2999
- checklist.
3000
-- **Specs**: no update needed; product behavior and public data semantics did
3001
- not change.
3002
-- **End-user/operator docs**: no hand-authored user docs changed; this repairs
3003
- generated publishing artifacts.
3004
-- **End-user/operator skills**: no update needed; no operator workflow changed.
3005
-- **SOW lifecycle**: SOW 14 reopened as a regression, status returned to
3006
- `completed`, and the file is moved back to `.agents/sow/done/` in the same
3007
- commit as the repair.
3008
-
3009
-### Follow-up mapping
3010
-
3011
-No deferred follow-up remains for this regression. The website integration PR
3012
-must be regenerated after this repair reaches `netdata/master`; that is the
3013
-normal downstream propagation path rather than a separate source-code TODO.
3014
-
3015
-## Regression - 2026-05-08 - Metadata Learn Links
3016
-
3017
-### What broke
3018
-
3019
-User reported that `src/crates/netflow-plugin/metadata.yaml` links the Sankey
3020
-visualization guidance to
3021
-`https://learn.netdata.cloud/docs/network-flows/visualization/summary-sankey`,
3022
-but the actual Learn page is
3023
-`https://learn.netdata.cloud/docs/network-flows/visualization/sankey-and-table`.
3024
-
3025
-### Evidence
3026
-
3027
-- `src/crates/netflow-plugin/metadata.yaml` contains absolute
3028
- `https://learn.netdata.cloud/docs/...` links in user-facing metadata
3029
- rendered into integration pages.
3030
-- `.agents/skills/learn-site-structure/mapping.md` says inter-page links should
3031
- use repo-relative `/docs/... .md` paths where possible, because Learn ingest
3032
- rewrites them to the final route.
3033
-- The prior validation checked generated integration rendering and downstream
3034
- builds, but did not crawl/validate every Learn URL embedded in
3035
- `metadata.yaml`.
3036
-
3037
-### Why previous validation missed it
3038
-
3039
-- Build validation proves markdown renders, but does not prove every external
3040
- absolute Learn URL resolves to the intended current page.
3041
-- The broken link returns a routing/publishing problem only when clicked or
3042
- link-checked against Learn, not during metadata schema validation.
3043
-
3044
-### Pre-Implementation Gate (Historical Snapshot at Implementation Start)
3045
-
3046
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
3047
-
3048
-Problem / root-cause model:
3049
-
3050
-- Network Flow metadata contains hard-coded Learn URLs that drifted from the
3051
- current Learn route names. The concrete example is the Sankey visualization
3052
- page rename from `summary-sankey` to `sankey-and-table`.
3053
-- Other `metadata.yaml` files also contain Learn URLs, including repeated
3054
- Virtual Node links in Go collector configuration descriptions. They need a
3055
- full audit, not a single replacement.
3056
-
3057
-Evidence reviewed:
3058
-
3059
-- User-reported broken URL and current target URL.
3060
-- `rg` scan of all `metadata.yaml` files for `learn.netdata.cloud/docs` and
3061
- `/docs/...` links.
3062
-- `learn-site-structure` skill mapping rules for source-relative links and
3063
- Learn URL computation.
3064
-- `integrations-lifecycle` skill rules that `metadata.yaml` is the source of
3065
- truth for generated integration pages.
3066
-
3067
-Affected contracts and surfaces:
3068
-
3069
-- `metadata.yaml` source files that render into integration pages.
3070
-- Generated `integrations/*.md`, `integrations/integrations.json`,
3071
- `integrations/integrations.js`, and `src/collectors/COLLECTORS.md` if source
3072
- metadata changes affect rendered artifacts.
3073
-- Learn and website integration pages that expose these links to users.
3074
-
3075
-Existing patterns to reuse:
3076
-
3077
-- Prefer `/docs/... .md` source-relative links when the target is a source doc
3078
- in this repository and Learn ingest can rewrite it.
3079
-- Keep absolute links only when required by metadata schema fields or when
3080
- linking to a page not represented as a source-relative doc path.
3081
-- Run the integrations generators after metadata edits.
3082
-
3083
-Risk and blast radius:
3084
-
3085
-- Documentation/integration metadata only.
3086
-- Incorrect replacements could break GitHub browsing or generated integration
3087
- links, so every changed target must be verified against source docs and/or
3088
- published Learn.
3089
-- No runtime collector behavior is changed.
3090
-
3091
-Sensitive data handling plan:
3092
-
3093
-- Link audit uses public documentation URLs and repository paths only.
3094
-- No credentials, customer data, IP addresses, or proprietary incident details
3095
- are needed or recorded.
3096
-
3097
-Implementation plan:
3098
-
3099
-1. Extract every Learn-related URL from `metadata.yaml` files.
3100
-2. Validate targets against the Learn source map and published Learn responses.
3101
-3. Replace proven-broken hard-coded Learn URLs with correct source-relative doc
3102
- links where possible.
3103
-4. Update the integrations/learn skills with a durable link-audit workflow if
3104
- the current skills do not already document this audit.
3105
-5. Regenerate integration artifacts and run focused validation.
3106
-
3107
-Validation plan:
3108
-
3109
-- Re-run the `metadata.yaml` Learn-link extraction after edits.
3110
-- Validate changed links against source files and published Learn routes.
3111
-- Run `python3 integrations/gen_integrations.py`.
3112
-- Run `python3 integrations/gen_docs_integrations.py`.
3113
-- Run `python3 integrations/gen_doc_collector_page.py`.
3114
-- Run `git diff --check`.
3115
-- Run `.agents/sow/audit.sh` and record any unrelated pre-existing findings.
3116
-
3117
-Artifact updates needed:
3118
-
3119
-- **Code**: no runtime code expected.
3120
-- **Metadata**: update source `metadata.yaml` links proven wrong.
3121
-- **Generated artifacts**: update generated integrations if generator output
3122
- changes.
3123
-- **Runtime project skills**: add/update link-audit guidance if absent.
3124
-- **Specs**: no product contract change expected.
3125
-- **End-user/operator docs**: generated integration docs may change.
3126
-
3127
-### Repair completed
3128
-
3129
-Repaired the proven-broken Learn targets:
3130
-
3131
-- `src/crates/netflow-plugin/metadata.yaml:147`,
3132
- `src/crates/netflow-plugin/metadata.yaml:285`, and
3133
- `src/crates/netflow-plugin/metadata.yaml:423` now link Validation and Data
3134
- Quality to
3135
- `https://learn.netdata.cloud/docs/network-flows/validation-and-data-quality`.
3136
-- `src/crates/netflow-plugin/metadata.yaml:159` now links Sankey and Table to
3137
- `https://learn.netdata.cloud/docs/network-flows/visualization/sankey-and-table`.
3138
-- `src/crates/netflow-plugin/metadata.yaml:161` now links Maps and Globe to
3139
- `https://learn.netdata.cloud/docs/network-flows/visualization/maps-and-globe`.
3140
-- `src/crates/netflow-plugin/metadata.yaml:542` now links Enrichment Intel
3141
- Downloader to
3142
- `https://learn.netdata.cloud/docs/network-flows/enrichment-intel-downloader`.
3143
-- `src/go/plugin/go.d/collector/snmp/metadata.yaml:243` now links the config
3144
- directory section to
3145
- `https://learn.netdata.cloud/docs/netdata-agent/configuration#locate-your-config-directory`.
3146
-
3147
-Regenerated the committed per-integration markdown files affected by those
3148
-metadata changes:
3149
-
3150
-- `src/crates/netflow-plugin/integrations/netflow.md`
3151
-- `src/crates/netflow-plugin/integrations/ipfix.md`
3152
-- `src/crates/netflow-plugin/integrations/sflow.md`
3153
-- `src/crates/netflow-plugin/integrations/db-ip_ip_intelligence.md`
3154
-- `src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md`
3155
-
3156
-Added `integrations-lifecycle` how-to
3157
-`.agents/skills/integrations-lifecycle/how-tos/auditing-metadata-learn-links.md`
3158
-and indexed it in
3159
-`.agents/skills/integrations-lifecycle/how-tos/INDEX.md`.
3160
-
3161
-### Validation evidence
3162
-
3163
-- Absolute Learn URL audit after repair found 12 unique
3164
- `https://learn.netdata.cloud/docs/...` targets in `metadata.yaml`; every
3165
- target returned HTTP 200.
3166
-- Fragment validation passed for:
3167
- - `https://learn.netdata.cloud/docs/netdata-agent/configuration#locate-your-config-directory`
3168
- - `https://learn.netdata.cloud/docs/netdata-agent/configuration/organize-systems-metrics-and-alerts#virtual-nodes`
3169
-- Source-relative metadata link audit passed: all `/docs/...` and relative
3170
- markdown links in `metadata.yaml` resolve to source files.
3171
-- Same-failure search passed: no exact stale targets remain in the edited
3172
- source metadata or generated integration markdown for `summary-sankey`,
3173
- `maps-globe`, `network-flows/intel-downloader`, exact
3174
- `network-flows/validation`, or `the-netdata-config-directory`.
3175
-- `python3 integrations/gen_integrations.py` passed.
3176
-- `python3 integrations/gen_docs_integrations.py` passed.
3177
-- `python3 integrations/gen_doc_collector_page.py` passed.
3178
-- `git diff --check` passed.
3179
-- `.agents/sow/audit.sh` exited 2 because of the pre-existing unrelated
3180
- sensitive-pattern warning in
3181
- `.agents/skills/mirror-netdata-repos/SKILL.md`; status/directory consistency
3182
- was otherwise checked during the in-progress state.
3183
-
3184
-### Artifact maintenance gate
3185
-
3186
-- **AGENTS.md**: no update needed; repository-wide workflow did not change.
3187
-- **Runtime project skills**: updated `integrations-lifecycle` with a reusable
3188
- metadata Learn-link audit how-to.
3189
-- **Specs**: no update needed; this changes documentation links, not product
3190
- behavior or data contracts.
3191
-- **End-user/operator docs**: generated integration markdown updated from the
3192
- source metadata.
3193
-- **End-user/operator skills**: no update needed; no public AI skill workflow
3194
- changed.
3195
-- **SOW lifecycle**: SOW 14 reopened as a regression, will be returned to
3196
- `Status: completed` and moved back to `.agents/sow/done/` in the same commit
3197
- as the repair.
3198
-
3199
-### Sensitive data gate
3200
-
3201
-Only public Learn URLs and repository paths were recorded. No secrets,
3202
-credentials, bearer tokens, SNMP communities, customer data, personal data,
3203
-private endpoints, or proprietary incident details were used or written.
3204
-
3205
-### Follow-up mapping
3206
-
3207
-No deferred follow-up remains for this regression. The downstream website and
3208
-Learn integration refresh should follow the normal post-merge generation paths.
.agents/sow/done/SOW-0015-20260507-vsphere-v2-parity-enrichment.md
deleted
-557
@@ -1,557 +0,0 @@
1
-# SOW-0015 - vSphere V2 Parity And Enrichment
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: Implementation is complete for the approved PR scope. The SOW has
8
-been collapsed to the final shipped state by user decision. Final validation
9
-passed on 2026-05-23, and this SOW is being moved to `.agents/sow/done/` with
10
-the closeout commit.
11
-
12
-## Requirements
13
-
14
-### Purpose
15
-
16
-Migrate the Go vSphere collector to framework V2 while preserving the useful V1
17
-metric contract, adding approved vSphere parity and enrichment surfaces, and
18
-removing transitional or high-cardinality features that should not ship in this
19
-PR.
20
-
21
-### User Request
22
-
23
-The user requested a clean end state for the vSphere V2 migration and parity
24
-work, including:
25
-
26
-- framework V2 collection and chart templates;
27
-- compatibility with the existing vSphere metric surface where accepted;
28
-- default-safe additive object-level metrics;
29
-- opt-in datastore-cluster, vSAN, tag/custom-attribute, and topology surfaces;
30
-- removal of config/options/features that are not worth shipping now;
31
-- charts.yaml as the single chart source of truth;
32
-- collector taxonomy that passes the fatal taxonomy CI gate;
33
-- tests converted toward table-driven and V2 metric-store assertions;
34
-- a final SOW that records the end state rather than the development diary.
35
-
36
-### Acceptance Criteria
37
-
38
-- The vSphere collector registers and runs through framework V2.
39
-- Current default VM, host, datastore, cluster, and resource-pool metric
40
- contexts remain available with accepted labels, dimensions, units, and values.
41
-- Chart IDs may change to framework V2 instance chart IDs; `id` is the stable
42
- vSphere managed-object-reference label used for chart instances.
43
-- `charts.yaml` is authoritative. The old Go chart-template mirror, runtime
44
- chart bridge, and V1 golden fixture are removed.
45
-- Default-safe metrics for snapshots, VM/host/cluster state, datastore state,
46
- inventory counts, and aggregate power are implemented.
47
-- Optional datastore-cluster and vSAN metrics are default-off and selector
48
- controlled.
49
-- Optional vSphere tag/custom-attribute labels are default-off and allowlist
50
- controlled.
51
-- Optional network topology discovery is default-off and exposed only through
52
- the cached topology Function.
53
-- VM/host child-instance metric surfaces, generated ESXi/VM vnodes,
54
- inventory-path labels, VM guest labels, host/VM power-state config controls,
55
- and the power-metrics config knob are not part of the shipped public surface.
56
-- `metadata.yaml`, `config_schema.json`, stock `go.d/vsphere.conf`,
57
- `charts.yaml`, health alerts, generated integration docs, and
58
- `taxonomy.yaml` are consistent with the final code.
59
-- Local validation covers Go tests, vet, chart-template compilation, taxonomy
60
- checks, docs/schema parsing, and targeted reviewer feedback.
61
-
62
-## Final Scope
63
-
64
-### Framework And Runtime
65
-
66
-- Registration uses framework V2 via `CreateV2`.
67
-- `Collect(ctx)` writes directly to `metrix.CollectorStore`.
68
-- The legacy `map[string]int64` collection bridge is gone.
69
-- `charts.yaml` is embedded and returned by `ChartTemplateYAML()`.
70
-- Runtime chart mutation is gone; chart lifecycle is handled by chartengine and
71
- chart-template `expire_after_cycles`.
72
-- Metric writers emit final V2 metric names directly and consistently attach
73
- the `id` label plus resource-specific labels.
74
-
75
-### Configuration Contract
76
-
77
-Current vSphere-specific config keys:
78
-
79
-- target and scheduling: `url`, `username`, `password`, `timeout`,
80
- `discovery_interval`, `update_every`, `autodetection_retry`, `vnode`;
81
-- core include selectors: `host_include`, `vm_include`,
82
- `datastore_include`, `cluster_include`;
83
-- optional labels: `tag_categories`, `custom_attributes`;
84
-- optional datastore clusters: `collect_datastore_clusters`,
85
- `datastore_cluster_include`;
86
-- optional vSAN: `collect_vsan`, `vsan_cluster_include`,
87
- `vsan_host_include`, `vsan_vm_include`;
88
-- optional topology: `collect_network_topology`;
89
-- inherited HTTP/TLS/proxy settings from `web.HTTPConfig`, with unused
90
- method/body-style fields hidden from the dynamic configuration UI.
91
-
92
-Removed before merge:
93
-
94
-- `max_*` resource caps;
95
-- `collect_power_metrics`;
96
-- `host_power_states`, `vm_power_states`;
97
-- `collect_inventory_path_label`, `vm_guest_labels`;
98
-- `esxi_vnodes`, `vm_vnodes`;
99
-- VM child-instance options:
100
- `collect_vm_disks`, `collect_vm_disk_performance`, `vm_disk_include`,
101
- `collect_vm_nic_performance`, `vm_nic_include`;
102
-- host child-instance options:
103
- `collect_host_nic_performance`, `host_nic_include`,
104
- `collect_host_disk_performance`, `host_disk_include`,
105
- `collect_host_storage_adapter_performance`,
106
- `host_storage_adapter_include`,
107
- `collect_host_storage_path_performance`, `host_storage_path_include`,
108
- `collect_host_cpu_instance_performance`,
109
- `host_cpu_instance_include`.
110
-
111
-### Metric Surface
112
-
113
-The final metric surface is documented in `collector/vsphere/metadata.yaml` and
114
-charted in `collector/vsphere/charts.yaml`.
115
-
116
-Default metric groups:
117
-
118
-- inventory object counts;
119
-- VM aggregate CPU, memory, swap, disk, network, power state, connection state,
120
- tools state, consolidation state, uptime, configuration, storage usage, and
121
- snapshot metrics;
122
-- host aggregate CPU, memory, swap, disk, network, overall status, power state,
123
- connection state, maintenance state, uptime, power, and energy metrics;
124
-- datastore aggregate I/O, IOPS, latency, space, accessibility, maintenance,
125
- multiple-host-access, and overall status metrics;
126
-- cluster capacity, topology, utilization, DRS, HA, vMotion, VM operation,
127
- inventory, and overall status metrics;
128
-- resource-pool CPU, memory, allocation, config, and status metrics.
129
-
130
-Optional metric groups:
131
-
132
-- datastore-cluster space, Storage DRS status, and overall status metrics behind
133
- `collect_datastore_clusters`;
134
-- vSAN cluster, host, and VM capacity/performance/health metrics behind
135
- `collect_vsan`.
136
-
137
-Aggregate VM and host disk/network metrics remain default-on. Per-disk,
138
-per-NIC, per-storage-adapter, per-storage-path, and per-CPU-instance metrics are
139
-excluded from this PR.
140
-
141
-### Labels
142
-
143
-Every emitted series includes the V2 `id` label.
144
-
145
-Resource-specific labels:
146
-
147
-- VM: `datacenter`, `cluster`, `host`, `vm`;
148
-- host: `datacenter`, `cluster`, `host`;
149
-- datastore: `datacenter`, `datastore`, `type`;
150
-- cluster: `datacenter`, `cluster`;
151
-- resource pool: `datacenter`, `cluster`, `resource_pool`;
152
-- datastore cluster: `datacenter`, `datastore_cluster`;
153
-- vSAN cluster: `datacenter`, `cluster`, `vsan_uuid`;
154
-- vSAN host: `datacenter`, `cluster`, `host`, `vsan_node_uuid`;
155
-- vSAN VM: `datacenter`, `cluster`, `host`, `vm`, `vm_instance_uuid`;
156
-- inventory: `id=inventory`.
157
-
158
-Optional enrichment labels:
159
-
160
-- `vsphere_tag_<category>` for tag categories matched by `tag_categories`;
161
-- `vsphere_custom_attribute_<name>` for custom attributes matched by
162
- `custom_attributes`.
163
-
164
-Tag/custom-attribute names are sanitized for label keys. Multiple tags in one
165
-category are sorted and joined with `|`. Users are warned not to allowlist
166
-categories or attributes that may contain secrets or sensitive data.
167
-
168
-Standalone-host dummy clusters are detected by `domain-s*` cluster IDs, not by
169
-name equality.
170
-
171
-### Functions And Topology
172
-
173
-- `vsphere:readiness` is a read-only cached Function. It reports local cached
174
- readiness, configured optional gates, and discovered resource counts without
175
- issuing extra vCenter API calls.
176
-- `topology:vsphere` is the public cached topology Function alias. It emits
177
- topology actors and links for datacenters, clusters, hosts, VMs, datastores,
178
- datastore clusters, and resource pools from cached discovery state.
179
-- `collect_network_topology` adds vSphere Network and Distributed Virtual Port
180
- Group actors and host/VM network links to topology output only. It does not
181
- create charts or metric labels.
182
-- `opaqueNetwork-` managed-object IDs map to `vsphere_network`, so NSX-backed
183
- network links do not break.
184
-
185
-### Dashboard And Documentation Artifacts
186
-
187
-- `charts.yaml` is the single source for chart templates.
188
-- `taxonomy.yaml` places vSphere under `containers-vms` and mirrors the
189
- existing cloud-frontend vSphere dashboard TOC:
190
- heads grid, Inventory, Clusters, Hosts, Virtual Machines, Resource Pools,
191
- Datastores, and Datastore Clusters.
192
-- `metadata.yaml`, `config_schema.json`, stock `go.d/vsphere.conf`, generated
193
- integration markdown, and `health.d/vsphere.conf` match the final config and
194
- metric surface.
195
-
196
-## Out Of Scope
197
-
198
-These items are intentionally not part of this PR:
199
-
200
-- vCenter/ESXi events and logs;
201
-- collector-generated ESXi and VM vnodes;
202
-- datastore vnodes;
203
-- inventory-path labels;
204
-- VM guest hostname, IP address, and guest OS labels;
205
-- MAC, IQN, WWN, datastore path, and other sensitive device identity labels;
206
-- VM and host child-instance metric families;
207
-- deeper vSAN internals such as disk-group, disk, component, CMMDS, and all
208
- Telegraf-style entity-type metrics;
209
-- live permission probes in readiness;
210
-- VCSA appliance health metrics, which belong to the `vcsa` collector;
211
-- ESXi hardware sensors, which are covered by SNMP `vmware-esx`;
212
-- workload/container metrics inside VMs, which belong to guest agents or
213
- Kubernetes collectors.
214
-
215
-Any of these requires a separate user-approved product/NIDL decision before
216
-implementation.
217
-
218
-## Analysis
219
-
220
-Sources checked:
221
-
222
-- `collector/vsphere/*.go` and subpackages;
223
-- `collector/vsphere/charts.yaml`;
224
-- `collector/vsphere/metadata.yaml`;
225
-- `collector/vsphere/config_schema.json`;
226
-- `collector/vsphere/taxonomy.yaml`;
227
-- `config/go.d/vsphere.conf`;
228
-- `health.d/vsphere.conf`;
229
-- `integrations/check_collector_taxonomy.py`;
230
-- `.agents/sow/specs/vsphere-parity-matrix.md`;
231
-- `.agents/sow/specs/vsphere-v1-compatibility-manifest.md`;
232
-- `.agents/sow/specs/go-v2-host-scope.md`;
233
-- project skills for collector authoring, framework V2 modules, and integration
234
- lifecycle.
235
-
236
-External/source evidence captured in
237
-`.agents/sow/specs/vsphere-parity-matrix.md` includes Broadcom vSphere/vSAN API
238
-documentation and checked open-source implementations from Datadog, Telegraf,
239
-Grafana vmware_exporter, Elastic Beats, Zabbix, New Relic, OpenTelemetry
240
-Collector Contrib, and Grafana Alloy.
241
-
242
-Root-cause model:
243
-
244
-- The pre-existing vSphere collector had a V1 runtime/chart model and a narrower
245
- metric surface.
246
-- Framework V2 requires a static chart-template contract and metric-store
247
- writers instead of runtime chart mutation.
248
-- Some parity candidates are useful object-level metrics; others are
249
- high-cardinality or sensitive identity surfaces that should not be exposed as
250
- broad public config in this PR.
251
-- The final implementation keeps default behavior useful and bounded, makes
252
- costly/sensitive additions opt-in, and removes transitional APIs before merge.
253
-
254
-Primary risks:
255
-
256
-- vCenter/vSAN live API behavior can differ from simulator behavior. Local tests
257
- cover typed govmomi APIs and parser behavior, but no real production vCenter
258
- was available in this worktree.
259
-- Chart ID continuity is intentionally broken by the V2 migration. Contexts,
260
- dimensions, labels, units, and values are preserved where accepted.
261
-- Optional vSAN APIs require privileges and vSAN availability not represented by
262
- the simulator.
263
-- Optional tag/custom-attribute enrichment can expose sensitive data if users
264
- allowlist sensitive categories or attributes; docs and config descriptions
265
- warn about this.
266
-
267
-Sensitive data handling plan:
268
-
269
-- Durable artifacts contain no real credentials, tokens, customer names,
270
- customer endpoints, private endpoints, or customer-identifying IP addresses.
271
-- Examples use placeholders or generic local names.
272
-- SOW evidence cites file paths, commands, and sanitized findings rather than
273
- raw vCenter data.
274
-
275
-## Pre-Implementation Gate
276
-
277
-Status: satisfied.
278
-
279
-Affected contracts and surfaces:
280
-
281
-- Go collector registration and runtime lifecycle;
282
-- vSphere discovery and scraping;
283
-- chart contexts, dimensions, labels, units, priorities, and lifecycle;
284
-- dynamic configuration schema;
285
-- stock configuration;
286
-- integration metadata and generated documentation;
287
-- health alerts;
288
-- collector taxonomy;
289
-- project specs and SOW lifecycle.
290
-
291
-Existing patterns reused:
292
-
293
-- framework V2 `CollectorStore`, `ChartTemplateYAML`, and chartengine tests;
294
-- `collecttest.AssertChartCoverage` and V2 metric-store assertions;
295
-- `web.HTTPConfig` embedding with UI-hidden unused fields;
296
-- optional allowlist and selector patterns for bounded/sensitive data;
297
-- cached read-only Functions for supportability/topology surfaces;
298
-- fail-soft collection for optional enrichment and partial API failures.
299
-
300
-Implementation plan:
301
-
302
-1. Migrate collector runtime and charts to framework V2.
303
-2. Preserve accepted V1 metric semantics and add default-safe object-level
304
- parity metrics.
305
-3. Add approved opt-in datastore-cluster, vSAN, label enrichment, and topology
306
- surfaces.
307
-4. Remove rejected or superseded public config and high-cardinality child
308
- metric surfaces.
309
-5. Make `charts.yaml` and `taxonomy.yaml` authoritative source artifacts.
310
-6. Rewrite tests around V2 metric-store output and chart-template coverage.
311
-7. Update docs, schema, stock config, health alerts, specs, and SOW.
312
-
313
-Validation plan:
314
-
315
-- focused unit tests for parsers, matchers, discovery, writers, Functions, and
316
- review feedback;
317
-- full vSphere package tests;
318
-- Go vet;
319
-- chart-template schema/decode/compile checks;
320
-- taxonomy gate and exact metadata-to-taxonomy ownership checks;
321
-- JSON/YAML parse checks;
322
-- generated integration docs when metadata/config changes;
323
-- same-failure grep for removed public keys and bridge symbols.
324
-
325
-Open decisions:
326
-
327
-- None for the implemented PR scope.
328
-
329
-## Final User Decisions
330
-
331
-1. Keep the vSphere work in one PR and split by focused commits.
332
-2. Use framework V2 and accept framework V2 chart ID changes.
333
-3. Preserve contexts, dimensions, labels, units, and meaning where accepted.
334
-4. Use `charts.yaml` as the only chart-template source.
335
-5. Use final V2 metric-store assertions instead of legacy runtime-chart maps.
336
-6. Keep datastore-cluster and vSAN metrics opt-in.
337
-7. Keep tag/custom-attribute labels opt-in with allowlists.
338
-8. Remove generated ESXi/VM vnodes from this PR.
339
-9. Remove inventory-path and VM guest labels from this PR.
340
-10. Remove VM and host child-instance metric surfaces from this PR.
341
-11. Remove host/VM power-state config controls.
342
-12. Remove `collect_power_metrics`; aggregate power metrics are part of the
343
- shipped metric surface when vSphere exposes the counters.
344
-13. Remove `max_*` caps from this collector before merge.
345
-14. Keep vCenter/ESXi events out of this metrics PR.
346
-15. Add `taxonomy.yaml` because the taxonomy gate is fatal.
347
-16. Base vSphere taxonomy shape on the existing cloud-frontend vSphere TOC.
348
-17. Collapse SOW-0015 to final-state evidence instead of development history.
349
-
350
-## Implementation Summary
351
-
352
-Runtime:
353
-
354
-- Migrated collector registration and collection to framework V2.
355
-- Added direct V2 gauge observation path.
356
-- Removed V1 runtime chart bridge, Go chart mirror, V1 compatibility golden
357
- fixture, and legacy metric-map assertions.
358
-- Kept deterministic sorting helpers for stable output and tests.
359
-
360
-Discovery and scraping:
361
-
362
-- Discovery includes selected non-powered hosts/VMs for property/status metrics
363
- while skipping real-time performance scraping where vSphere has no useful
364
- samples.
365
-- Datastore, cluster, and resource-pool property refresh failures skip stale
366
- property metrics and let chartengine lifecycle handle expiry.
367
-- Missing performance counters warn once per stable key and do not abort the
368
- whole collector.
369
-- vSphere client cleanup handles partial initialization and session logout.
370
-
371
-Metrics:
372
-
373
-- Added VM snapshot count, maximum age, and maximum chain depth.
374
-- Added VM/host/cluster/datastore/resource-pool property/status metrics.
375
-- Added inventory object counts.
376
-- Added aggregate host/VM power and energy metrics.
377
-- Added optional datastore-cluster metrics.
378
-- Added optional vSAN metrics.
379
-- Preserved aggregate VM/host datastore/network/disk/cluster/resource-pool
380
- metrics accepted from V1.
381
-
382
-Configuration and selectors:
383
-
384
-- Added typed include selector types for core path includes and optional
385
- datastore-cluster/vSAN selectors while preserving YAML/JSON keys.
386
-- Extracted reusable ordered simple-pattern list matching to `pkg/matcher`.
387
-- Preserved config-specific validation and error-message shapes.
388
-- Removed public keys that should not ship in this PR.
389
-
390
-Labels and enrichment:
391
-
392
-- Added optional vSphere tag and custom-attribute labels with allowlists.
393
-- Preserved empty gates so REST/CIS/tag/custom-attribute clients are not used
394
- when enrichment is disabled.
395
-- Added privacy warnings for user metadata labels.
396
-
397
-Functions and topology:
398
-
399
-- Added cached readiness Function.
400
-- Added cached topology Function.
401
-- Added optional network/DVPG topology discovery.
402
-- Added `opaqueNetwork-` topology ID support.
403
-
404
-Tests:
405
-
406
-- Reworked collector tests to V2 metric-store reads and table-driven cases where
407
- setup/assertions are shared.
408
-- Added chart-template schema/decode/priority/compile validation.
409
-- Added `AssertChartCoverage` and selector-match checks for default and
410
- optional surfaces.
411
-- Added focused tests for matchers, vSAN parsing, datastore clusters, power
412
- metrics, labels, topology, readiness, discovery, client cleanup, and reviewer
413
- feedback.
414
-- Replaced fixed task lifecycle sleeps with deterministic `task.wait()`.
415
-
416
-Artifacts:
417
-
418
-- Updated `charts.yaml`, `metadata.yaml`, `config_schema.json`, stock
419
- `go.d/vsphere.conf`, health alerts, generated integration markdown, and
420
- `taxonomy.yaml`.
421
-- Updated specs for vSphere parity and the superseded V1 compatibility manifest.
422
-- Added/updated project skill guidance for framework V2 collector work.
423
-
424
-## Validation
425
-
426
-Acceptance criteria evidence:
427
-
428
-- `collector/vsphere/charts.go`,
429
- `collector/vsphere/chart_template_sets.go`,
430
- `collector/vsphere/compat_manifest_test.go`, and
431
- `collector/vsphere/testdata/v1_compat_manifest.json` are removed.
432
-- Grep for runtime bridge symbols such as `chartTemplateSets`, `legacyDimID`,
433
- `v2MetricName`, `writeChartMetrics`, `chartExpireAfterCycles`, and old
434
- `*ChartsTmpl` names returns no production hits.
435
-- Grep for removed config keys under vSphere code/config/docs surfaces returns
436
- no non-SOW hits.
437
-- `metadata.yaml`, `charts.yaml`, and taxonomy coverage agree on the current
438
- vSphere contexts.
439
-
440
-Tests and checks run during final state:
441
-
442
-- `go test -count=1 -timeout 300s ./collector/vsphere/...` passed from
443
- `src/go/plugin/go.d`.
444
-- `go vet ./collector/vsphere/...` passed from `src/go/plugin/go.d`.
445
-- `go test -count=1 -run '^Test_task' ./collector/vsphere` passed from
446
- `src/go/plugin/go.d`.
447
-- `../../../../.venv/bin/python ../../../../integrations/check_collector_taxonomy.py --pr-diff upstream/master...HEAD`
448
- passed from `src/go/plugin/go.d`.
449
-- `../../../../.venv/bin/python ../../../../integrations/gen_taxonomy.py --check-only`
450
- passed from `src/go/plugin/go.d`.
451
-- Manual taxonomy ownership check reported
452
- `metadata=115 owned=115 referenced=4 missing=0 extra=0 duplicates=0`.
453
-- `python3 -m json.tool collector/vsphere/config_schema.json` passed.
454
-- YAML parse checks for `collector/vsphere/metadata.yaml`,
455
- `collector/vsphere/charts.yaml`, `collector/vsphere/taxonomy.yaml`, and stock
456
- `config/go.d/vsphere.conf` passed during the PR work.
457
-- Generated integration markdown was regenerated when metadata/config changed.
458
-- `git diff --check` passed for final touched files.
459
-
460
-Reviewer findings:
461
-
462
-- All accepted GitHub review feedback was fixed:
463
- readiness client-state check, opaque network topology IDs, discoverer pointer
464
- receiver warning dedup, datastore-cluster Storage DRS unknown state,
465
- dummy-cluster label detection, datastore writer exact metric count, and task
466
- lifecycle test synchronization.
467
-- Findings rejected as out of scope or not applicable are reflected in the final
468
- out-of-scope section and specs.
469
-
470
-Same-failure scans:
471
-
472
-- Removed public config keys were searched across vSphere code, config, schema,
473
- metadata, stock config, and tests.
474
-- Removed chart bridge symbols were searched across production Go code.
475
-- Taxonomy contexts were compared against metadata contexts with missing, extra,
476
- and duplicate checks.
477
-
478
-Sensitive data gate:
479
-
480
-- Durable artifacts contain placeholders only for credentials and endpoints.
481
-- No raw secrets, bearer tokens, private keys, session cookies, customer names,
482
- customer-identifying non-private IP addresses, private endpoints, or
483
- proprietary incident data were added.
484
-
485
-Artifact maintenance gate:
486
-
487
-- `AGENTS.md`: no final update required; existing collector consistency and SOW
488
- rules already cover this work.
489
-- Runtime project skills: framework V2 collector guidance exists under
490
- `.agents/skills/project-writing-go-modules-framework-v2/`; integration
491
- lifecycle guidance was followed for taxonomy/metadata/doc artifacts.
492
-- Specs: `.agents/sow/specs/vsphere-parity-matrix.md` records final parity
493
- classifications; `.agents/sow/specs/vsphere-v1-compatibility-manifest.md`
494
- records the superseded V1 baseline and current V2 validation replacements.
495
-- End-user/operator docs: `metadata.yaml`, stock config, generated integration
496
- markdown, and health alerts were updated with the shipped config/metric
497
- surface.
498
-- End-user/operator skills: no public operator skill changed because this PR
499
- changes collector behavior/docs, not AI skill workflows.
500
-- SOW lifecycle: this closeout marks SOW-0015 `completed` and moves it to
501
- `.agents/sow/done/` with the closing commit.
502
-
503
-Specs update:
504
-
505
-- vSphere parity matrix is the current WHAT contract for included, excluded,
506
- covered-elsewhere, and non-metric surfaces.
507
-- V1 compatibility manifest is explicitly superseded and retained only as
508
- historical baseline evidence.
509
-
510
-Project skills update:
511
-
512
-- Framework V2 collector skill captures reusable HOW-to-work guidance for this
513
- migration pattern.
514
-- No additional skill update is required by the final SOW rewrite.
515
-
516
-Documentation update:
517
-
518
-- vSphere integration docs are generated from `metadata.yaml`.
519
-- `README.md` follows the generated integration markdown symlink pattern.
520
-- `taxonomy.yaml` is now present and passes the fatal taxonomy gate.
521
-
522
-Lessons:
523
-
524
-- High-cardinality child-instance surfaces should not be added only because
525
- vendor APIs expose them. They need a clear user need and bounded product
526
- contract.
527
-- V2 migrations should move to direct metric-store assertions early; keeping a
528
- runtime V1 chart bridge makes tests and implementation harder to reason about.
529
-- Collector taxonomy is a required source artifact when metric contexts change.
530
-- Fixed sleeps in goroutine lifecycle tests should use explicit synchronization
531
- primitives when the implementation exposes them.
532
-
533
-## Outcome
534
-
535
-Implementation is complete for the approved PR scope. The collector now has a
536
-clean framework V2 runtime, authoritative YAML chart/taxonomy artifacts,
537
-approved parity/enrichment surfaces, and tests that assert the final V2 metric
538
-store and chart-template behavior.
539
-
540
-## Followup
541
-
542
-No follow-up SOW is required for the implemented scope.
543
-
544
-Excluded work that requires a separate user-approved SOW before implementation:
545
-
546
-- vCenter/ESXi event/log ingestion;
547
-- generated ESXi/VM/datastore vnodes;
548
-- sensitive identity labels such as guest IP, inventory path, MAC, IQN, WWN, and
549
- datastore paths;
550
-- per-child-instance VM/host metric families;
551
-- deeper vSAN internals beyond the shipped opt-in subset;
552
-- live vCenter permission probes in readiness;
553
-- context propagation through all govmomi calls.
554
-
555
-## Regression Log
556
-
557
-No active regression remains for the final shipped scope.
.agents/sow/done/SOW-0015-20260508-netflow-enrichment-verification.md
deleted
-1096
@@ -1,1096 +0,0 @@
1
-# SOW-0015 - NetFlow Enrichment Verification
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: End-user downloader documentation mismatch on supported providers corrected.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Ensure the Netdata Agent ships network-flow enrichment methods that are fit for DevOps/SRE use: tested in code, validated in practice where possible, documented with working instructions, and proven to produce the fields users expect.
14
-
15
-### User Request
16
-
17
-The user wants the NetFlow enrichment methods fully tested, including unit tests, integration tests, practical validation on the locally running Netdata Agent, and documentation corrections when instructions are wrong. The user asked the assistant to work autonomously and stop only when progress requires user involvement.
18
-
19
-Correction from the user: the scope includes all enrichment paths, plus possible NetFlow and sFlow validation gaps. The user is already testing IPFIX on this workstation, and the default geolocation database is known to work. The priority is to test everything else that is not already covered by unit tests; CI-job integration can come later and is not the immediate gate for this SOW.
20
-
21
-The private live flow exporter endpoint is intentionally not recorded here. Runtime evidence may refer to it as `[PRIVATE_FLOW_EXPORTER]`.
22
-
23
-Scope:
24
-
25
-1. Flow protocols with missing proof: NetFlow and sFlow. IPFIX is treated as a locally working baseline because the user is actively testing it on this workstation.
26
-2. Default geolocation database: treated as a working baseline, but existing unit/runtime proof will be recorded.
27
-3. DB-IP IP Intelligence.
28
-4. IPtoASN.
29
-5. Custom MMDB Database.
30
-6. MaxMind GeoIP / GeoLite2.
31
-7. Static metadata.
32
-8. Sampling overrides.
33
-9. Static networks.
34
-10. Classifiers.
35
-11. ASN and network provider chains.
36
-12. Decapsulation enrichment.
37
-13. AWS IP Ranges.
38
-14. Azure IP Ranges.
39
-15. GCP IP Ranges.
40
-16. Generic JSON-over-HTTP IPAM.
41
-17. NetBox.
42
-18. bio-rd / RIPE RIS.
43
-19. BMP.
44
-
45
-### Clarified Beta Validation Target
46
-
47
-The target is **not** proving that every external routing stack is production-stable. The target is proving Netdata's side is sane:
48
-
49
-- It connects when given valid config.
50
-- It rejects or reports bad config clearly.
51
-- It consumes the expected payload/schema.
52
-- It maps fields correctly into enrichment state.
53
-- It respects configured options.
54
-- It does not panic, hang, leak tasks, or silently ignore data.
55
-- The docs describe commands/configs that actually work.
56
-- The beta can ship without obvious "this was never exercised" failures.
57
-
58
-This SOW must not block on the user having a router that supports BMP or a production bio-rd/RIS deployment.
59
-
60
-For BMP, GoBGP/FRR plus deterministic BMP frames are sufficient to validate Netdata's listener and route ingestion contract.
61
-
62
-For BioRIS, a deterministic bio-rd-compatible gRPC service is sufficient to validate Netdata's client behavior. Running upstream bio-rd is useful as a smoke test, but it is not a hard shipping blocker if upstream itself is unstable in a local lab.
63
-
64
-End-user documentation remains product documentation. It must describe supported commands, configuration, payload/schema expectations, limitations, and troubleshooting. It must not discuss internal test gaps, untested status, or why a test could not be performed.
65
-
66
-### Assistant Understanding
67
-
68
-Facts:
69
-
70
-- The current `netdata` instance on this workstation is available for practical validation and already receives flow traffic from `[PRIVATE_FLOW_EXPORTER]`.
71
-- `build-install-netflow-plugin.sh` builds and installs the Rust `netflow-plugin`, then restarts `netdata` unless called with `--no-restart`.
72
-- The backend stores flow records in journals, so local validation can inspect journal-backed output through plugin queries and, where useful, `journalctl`.
73
-- Prior NetFlow documentation work recorded that BMP, BioRIS, and Network Sources had parser/transform unit tests but lacked runtime I/O integration tests.
74
-- User-facing documentation is in scope when validation proves instructions are wrong or incomplete.
75
-- The user reports IPFIX and the default geolocation database already work on this workstation.
76
-
77
-Inferences:
78
-
79
-- The core gap is not absence of all tests; it is missing proof across the real boundaries for methods not already covered: documented config, MMDB files, HTTP fetchers, gRPC streams, BMP TCP sessions, runtime state publication, flow enrichment, journal-backed query output, and generated integration documentation.
80
-- BMP and bio-rd validation can progress without the user by using local deterministic speakers, public/open-source fixtures, FRR/GoBGP, and a bio-rd-compatible gRPC test service. Production-router proof is not an acceptance criterion for this beta validation pass.
81
-
82
-Unknowns:
83
-
84
-- Whether all required fixture formats can be generated in Rust using existing dependencies, or whether narrow dev-dependencies are required.
85
-- Whether a practical local bio-rd setup can be made reliable enough for automated validation without pulling in a large external runtime.
86
-- Whether the installed local `netdata` configuration can be safely changed in-place for all live validation cases, or whether some cases should use temporary config and `--no-restart` style isolated test runs.
87
-
88
-### Acceptance Criteria
89
-
90
-- Each scoped enrichment method has automated unit or integration-test coverage proving the configured source is parsed/fetched/accepted and produces expected enrichment attributes, or the SOW records existing coverage and why no new test was needed.
91
-- NetFlow and sFlow have automated validation for protocol decode/enrichment interaction when existing tests do not already prove it.
92
-- Runtime I/O paths are covered where applicable:
93
- - HTTP fetch and refresh for cloud/IPAM/NetBox network sources.
94
- - gRPC `GetRouters`, `DumpRIB`, and `ObserveRIB` behavior for BioRIS or an equivalent local fake.
95
- - BMP TCP listener behavior with real BMP-framed bytes, route upsert, withdrawal, and session cleanup.
96
-- At least one test path proves enrichment reaches flow records and the fields exposed by journal-backed query output.
97
-- Documentation examples and setup instructions for the scoped methods are executed or mechanically tested where possible; incorrect instructions are fixed from source files and generated artifacts are regenerated.
98
-- Practical local validation is attempted on the installed Netdata Agent using the build/install script and journal-backed output.
99
-- Any remaining blockers are listed with evidence and the exact user involvement required.
100
-
101
-## Analysis
102
-
103
-Sources checked:
104
-
105
-- `AGENTS.md`
106
-- `.agents/sow/SOW.template.md`
107
-- `.agents/sow/done/SOW-0014-20260506-netflow-sflow-ipfix-documentation-guide.md`
108
-- `.agents/sow/specs/sensitive-data-discipline.md`
109
-- `.agents/skills/project-writing-collectors/SKILL.md`
110
-- `.agents/skills/integrations-lifecycle/SKILL.md`
111
-- `src/crates/netflow-plugin/Cargo.toml`
112
-- `src/crates/Cargo.toml`
113
-- `src/crates/netflow-plugin/src/`
114
-- `src/crates/netflow-plugin/metadata.yaml`
115
-- `src/crates/netflow-plugin/configs/netflow.yaml`
116
-- `docs/network-flows/`
117
-- `build-install-netflow-plugin.sh`
118
-- Local mirrored upstream repositories for BMP/BioRIS/network-source fixture patterns.
119
-
120
-Current state:
121
-
122
-- `src/crates/netflow-plugin/src/main.rs` starts BMP, BioRIS, and network-source tasks when configured.
123
-- `src/crates/netflow-plugin/src/decoder/state/runtime/decode.rs` applies the enricher to decoded flow records.
124
-- Existing tests cover many pure parsing and transform helpers, but not all documented real-source workflows.
125
-- `src/crates/netflow-plugin/Cargo.toml` has no HTTP mock or MMDB writer dev-dependency at SOW start.
126
-
127
-Risks:
128
-
129
-- Tests that rely on live Internet endpoints can become flaky; automated tests should use frozen, attributed fixtures and local fake services so they are suitable for future CI even though CI-job wiring is not the immediate target.
130
-- Real cloud/IPAM/BGP payloads may contain sensitive network information; durable artifacts must use sanitized fixtures only.
131
-- Installing and restarting the local plugin affects the user's running Netdata instance; use the provided script deliberately and record the command/result.
132
-- Documentation is generated from metadata for integration cards; generated files must not be hand-edited.
133
-
134
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
135
-
136
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
137
-
138
-Problem / root-cause model:
139
-
140
-- NetFlow enrichment has broad configuration and documentation coverage, but several enrichment methods lack tests at the boundary where real failures happen. A method can have parser tests and still fail because the documented transform is stale, the HTTP fetcher does not publish runtime state, the MMDB schema is not actually decoded, the BMP listener never accepts a real frame, NetFlow/sFlow decode does not interact correctly with enrichment, or the final enriched attributes never reach journal-backed query output.
141
-
142
-Evidence reviewed:
143
-
144
-- Prior SOW evidence: `.agents/sow/done/SOW-0014-20260506-netflow-sflow-ipfix-documentation-guide.md` records runtime I/O gaps for BMP, BioRIS, and Network Sources.
145
-- Local code evidence:
146
- - `src/crates/netflow-plugin/src/main.rs` task startup for dynamic enrichment.
147
- - `src/crates/netflow-plugin/src/network_sources/fetch.rs` HTTP fetch path.
148
- - `src/crates/netflow-plugin/src/network_sources/service.rs` refresh/publish path.
149
- - `src/crates/netflow-plugin/src/routing/bmp/listener.rs` BMP TCP listener path.
150
- - `src/crates/netflow-plugin/src/routing/bioris/runtime/refresh/instance.rs` BioRIS router/RIB refresh path.
151
- - `src/crates/netflow-plugin/src/enrichment/resolve.rs` provider-chain and merge behavior.
152
- - `src/crates/netflow-plugin/src/flow/record/journal/network.rs` journal field output.
153
-- Open-source reference evidence:
154
- - `akvorado/akvorado @ 646eb033a57692f5c2918bb65768e15c921b766e`, `outlet/routing/provider/bmp/root_test.go`, `outlet/routing/provider/bmp/tests.go`, `outlet/routing/provider/bioris/root_test.go`, `cmd/akvorado/testdata/configurations/clickhouse-network-sources/in.yaml`.
155
- - `pmacct/pmacct @ b06e6829bdc523200cbed2942b7045d64146cb71`, `tests/208-BMP-mem-leak-test/`, `tests/400-IPFIXv10-BMP-CISCO-SRv6-multiple-sources/`.
156
-
157
-Affected contracts and surfaces:
158
-
159
-- Rust code under `src/crates/netflow-plugin/src/`.
160
-- Test fixtures under `src/crates/netflow-plugin/testdata/`, with attribution.
161
-- `src/crates/netflow-plugin/Cargo.toml` and possibly `src/crates/Cargo.toml` for narrow test dependencies.
162
-- User-facing source docs under `docs/network-flows/`.
163
-- Integration source metadata `src/crates/netflow-plugin/metadata.yaml`.
164
-- Generated integration cards under `src/crates/netflow-plugin/integrations/`.
165
-- Installed local `netdata` runtime during practical validation.
166
-- SOW/spec/skill artifacts if durable workflow knowledge changes.
167
-
168
-Existing patterns to reuse:
169
-
170
-- Existing Rust module-local `tests.rs` files.
171
-- Existing flow pcap fixtures and `src/crates/netflow-plugin/testdata/ATTRIBUTION.md`.
172
-- Existing `ingest_test_support` and query tests for journal-backed records.
173
-- Existing integration generator workflow from `integrations-lifecycle`.
174
-- Akvorado-style BMP fixture tests and network-source transform tests, adapted to Netdata's codebase and licensing constraints.
175
-
176
-Risk and blast radius:
177
-
178
-- Most implementation should be test-only, but validation may expose runtime bugs requiring production code fixes.
179
-- BMP and BioRIS involve async network services; tests must use deterministic localhost ports, explicit shutdown, and timeouts.
180
-- Live validation on the installed Agent can restart `netdata`; this is acceptable per user-provided workflow but must use targeted process discipline and avoid broad kills.
181
-- Docs changes affect generated public integration pages and Learn ingestion; regeneration and link/content checks are required when docs change.
182
-
183
-Sensitive data handling plan:
184
-
185
-- Do not write literal private IPs, endpoints, node IDs, tokens, real customer data, or personal names to SOWs, specs, docs, skills, code comments, or fixtures.
186
-- Use `[PRIVATE_FLOW_EXPORTER]`, RFC documentation addresses, loopback addresses, or synthetic fixture data.
187
-- Store any raw runtime outputs that may contain private data only under `.local/`, which is gitignored; summarize redacted evidence in this SOW.
188
-- If a live output contains sensitive data needed for debugging, redact before recording and do not commit the raw output.
189
-
190
-Implementation plan:
191
-
192
-1. Build a coverage matrix from code, docs, config, metadata, and tests for all enrichment methods plus NetFlow/sFlow protocol validation.
193
-2. Add deterministic fixtures and unit/contract tests for enrichment methods that are not already proven.
194
-3. Add integration-style tests that prove runtime state feeds flow enrichment and journal/query-visible fields.
195
-4. Correct source documentation and metadata examples that fail validation; regenerate generated integration artifacts.
196
-5. Run narrow Rust tests, docs/generator validation, and any broader CI-equivalent commands needed by touched surfaces.
197
-6. Build/install the plugin with `build-install-netflow-plugin.sh`, restart `netdata`, and validate practical behavior with journal-backed output.
198
-7. Record exact remaining blockers that require user-provided external systems or credentials.
199
-
200
-Validation plan:
201
-
202
-- Run targeted `cargo test -p netflow-plugin ...` commands from `src/crates`.
203
-- Run full `cargo test -p netflow-plugin` if feasible after targeted tests pass.
204
-- Run integration generator commands when metadata or generated cards change.
205
-- Validate docs snippets/transforms against frozen or live-official payload shapes where feasible.
206
-- Run the local build/install script and verify the installed plugin produces enriched flow output.
207
-- Inspect `journalctl` and/or plugin query paths for runtime errors and expected enriched fields.
208
-- Search for same-failure patterns across provider docs/tests before closing.
209
-
210
-Artifact impact plan:
211
-
212
-- AGENTS.md: no update expected unless workflow rules are found wrong.
213
-- Runtime project skills: update only if this work discovers durable collector/integration workflow knowledge not already captured.
214
-- Specs: create or update a NetFlow enrichment behavior spec if tests establish durable contracts not captured elsewhere.
215
-- End-user/operator docs: update docs and metadata when instructions are wrong or incomplete.
216
-- End-user/operator skills: update only if public query skills are affected by changed flow-query instructions.
217
-- SOW lifecycle: this SOW is active in `current/`; close only after validation, artifact gates, and follow-up mapping.
218
-
219
-Open-source reference evidence:
220
-
221
-- `akvorado/akvorado @ 646eb033a57692f5c2918bb65768e15c921b766e`
222
- - `outlet/routing/provider/bmp/root_test.go`
223
- - `outlet/routing/provider/bmp/tests.go`
224
- - `outlet/routing/provider/bioris/root_test.go`
225
- - `cmd/akvorado/testdata/configurations/clickhouse-network-sources/in.yaml`
226
-- `pmacct/pmacct @ b06e6829bdc523200cbed2942b7045d64146cb71`
227
- - `tests/208-BMP-mem-leak-test/`
228
- - `tests/400-IPFIXv10-BMP-CISCO-SRv6-multiple-sources/`
229
-
230
-Open decisions:
231
-
232
-- Resolved by user: pursue full testing, practical validation, unit/integration coverage, and documentation correction.
233
-- Resolved by user: assistant may proceed autonomously and stop only when user involvement is strictly required.
234
-- Resolved by user: immediate scope is tests and practical validation, not CI-job integration.
235
-- Resolved by user: IPFIX and default geolocation database are working local baselines; prioritize the remaining unproven methods.
236
-- No implementation-blocking user decision is open at SOW creation.
237
-
238
-## Implications And Decisions
239
-
240
-1. Decision: confidence level.
241
- - Selected: full verification using unit tests, CI-safe integration/contract tests, practical local validation, and docs correction.
242
- - Rationale: user explicitly rejected partial confidence and wants proof that shipped instructions work.
243
-
244
-2. Decision: autonomy.
245
- - Selected: proceed without user involvement until an external dependency, credential, or product/risk decision blocks progress.
246
- - Rationale: user explicitly granted room to work autonomously.
247
-
248
-3. Decision: corrected scope.
249
- - Selected: all enrichment paths plus NetFlow and sFlow validation gaps; IPFIX and default geolocation database are baselines rather than the primary gap.
250
- - Rationale: user corrected the scope after SOW creation and clarified that the immediate ask is testing missing unit/integration coverage, not CI-job setup.
251
-
252
-4. Decision: downloader/provider coverage.
253
- - Selected: enhance the topology IP intelligence downloader so every supported free Geo/ASN provider path that can feed Netdata is exercised through generated MMDB output, update user documentation accordingly, test against actual provider data when licensing and public access permit it, then prove the generated output is usable by the netflow plugin.
254
- - Rationale: the netflow plugin reads MMDBs, while several provider sources start as CSV/TSV/tar/zip files. The missing proof is the conversion boundary from provider data into the MMDB files the plugin consumes.
255
- - Implication: this SOW now includes `src/go/tools/topology-ip-intel-downloader/`, its tests/docs/config examples, netflow integration metadata/examples that reference generated files, and runtime validation using generated MMDBs.
256
- - Risk: live provider downloads can be flaky, rate-limited, unavailable, or license-gated; CI-safe automated tests must remain fixture/local-data based, while actual-data validation is recorded as local evidence.
257
-
258
-5. Decision: local live-service validation.
259
- - Selected: after downloader/provider proof, test local NetBox, generic IPAM, bio-rd, and GoBGP/FRR/BMP paths as far as possible without user involvement.
260
- - Rationale: these are installable or emulatable local systems on this workstation and are better proof than only mocked unit tests.
261
- - Implication: use temporary/local runtime services, store raw outputs under `.local/`, and only record sanitized evidence in durable artifacts.
262
- - Risk: full router-equivalent proof may still require a real router or production routing topology; local GoBGP/FRR/BMP proof is protocol-level, not proof against a vendor router.
263
-
264
-6. Decision: beta external-routing validation bar.
265
- - Selected: prove Netdata's BMP and BioRIS contracts with deterministic payloads, local protocol speakers, and bio-rd-compatible gRPC services; do not block on production-router or production-bio-rd stability.
266
- - Rationale: the user does not have a router supporting these protocols, and the purpose is to catch Netdata-side issues before beta: connection/config failures, wrong fields, wrong payload schemas, ignored options, panics, hangs, leaked tasks, silent drops, and incorrect docs.
267
- - Implication: FRR/GoBGP plus deterministic BMP frames are sufficient for BMP. A deterministic bio-rd-compatible gRPC service is sufficient for BioRIS client validation. Upstream bio-rd remains useful as a smoke test, but upstream instability does not by itself block the beta.
268
- - Risk: this does not prove behavior against every vendor router or every production bio-rd deployment. That broader interoperability proof is expected from beta user feedback and later targeted fixes, while this SOW removes the obvious Netdata-side failures before shipping.
269
-
270
-## Plan
271
-
272
-1. Coverage matrix: map every enrichment method and NetFlow/sFlow from config/docs to runtime code and tests.
273
-2. Test harness: add deterministic local fixtures and fake services.
274
-3. Runtime proof: connect source-specific data to `FlowEnricher` and journal/query-visible fields.
275
-4. Documentation proof: execute or mechanically validate examples; fix source docs/metadata and regenerate.
276
-5. Local installed proof: build/install, restart `netdata`, inspect logs/journals/query output.
277
-6. Closeout: record validation, sensitive-data gate, artifact gates, and blockers requiring user action.
278
-
279
-## Execution Log
280
-
281
-### 2026-05-08
282
-
283
-- Created SOW after user approved autonomous full verification scope.
284
-- Recorded private flow exporter as `[PRIVATE_FLOW_EXPORTER]` instead of writing the literal endpoint.
285
-- Added CI-safe automated coverage for previously weak boundaries:
286
- - MaxMind/GeoLite2 ASN and City MMDB decoding with attributed public test databases.
287
- - Custom MMDB-style `netdata.ip_class` metadata from both ASN and GeoIP databases.
288
- - Static/runtime network-source merge behavior.
289
- - Documented AWS, Azure, GCP, generic JSON IPAM, and NetBox transform shapes.
290
- - HTTP network-source fetch, HTTP error handling, refresh loop publication, and cancellation.
291
- - NetFlow v9 and sFlow decode paths applying enrichment during decode.
292
- - Journal-backed ingest/query proof that enriched fields are persisted and query-visible.
293
- - BMP TCP listener accepting real BMP-framed Initiation and Termination messages.
294
- - BioRIS gRPC listener consuming `GetRouters`, `DumpRIB`, and `ObserveRIB` from a local fake service.
295
- - Stock `netflow.yaml` parse/validate coverage.
296
-- Added attributed MaxMind test MMDB fixtures under `src/crates/netflow-plugin/testdata/mmdb/`.
297
-- Found and fixed a production-code mismatch: `netdata.ip_class` was decoded from ASN MMDB records but not from GeoIP MMDB records, even though the MMDB downloader/docs can stamp the `netdata` metadata section into both outputs.
298
-- Found and fixed a Function request compatibility issue: public flow-query examples use `after: -3600`, `before: 0`, and numeric `top_n`; the plugin only accepted unsigned absolute timestamps and string enum `top_n` values. The request parser now accepts documented relative time bounds and numeric/string `top_n`.
299
-- Found and fixed a full-suite E2E reliability issue: five E2E tests passed individually but timed out in the default parallel full-suite run because the shared ingest wait was only 10 seconds. The shared wait is now 30 seconds.
300
-- Removed an existing unused import warning in the startup memory test module.
301
-- Corrected user-facing integration metadata and regenerated integration pages:
302
- - NetBox v2 tokens are documented as NetBox 4.5+; NetBox 4.0-4.4 use legacy tokens.
303
- - NetBox `?limit=0` is documented as dependent on server `MAX_PAGE_SIZE` policy.
304
- - BMP docs now state v3 is processed, v4 frames are decoded but ignored, and draft v1/v2 frames are decode errors.
305
-- Fixed the public `query-netdata-agents` helper after using it during runtime validation:
306
- - Bash-only nameref use no longer breaks in the workstation's default `zsh`.
307
- - Internal `path` variables no longer clear zsh `PATH`.
308
- - Masked curl logging now redacts node id, machine GUID, claim id, space id, and room id, not only tokens.
309
- - Added/updated no-token-leak self-tests for Bash and zsh.
310
-- Added `docs/netdata-ai/skills/query-netdata-cloud/how-tos/validate-local-netflow-function.md` because the Cloud/direct-agent validation workflow required multi-step analysis not present in the how-to catalog.
311
-- Ran official/live schema checks for provider payloads:
312
- - AWS `ip-ranges.json`: verified `ip_prefix`, `ipv6_prefix`, `region`, `service`, `network_border_group`, `syncToken`, and `createDate`.
313
- - GCP `cloud.json` and `goog.json`: verified `syncToken`, `creationTime`, `ipv4Prefix`/`ipv6Prefix`, and optional `service`/`scope` behavior.
314
- - Azure Service Tags current public JSON: verified `changeNumber`, `cloud`, `values[].name`, and `properties.addressPrefixes`, `region`, `platform`, `systemService`, `networkFeatures`.
315
- - NetBox current docs: verified v2 tokens are NetBox 4.5+.
316
-- Built and installed the plugin with `./build-install-netflow-plugin.sh`; `netdata` restarted and remained active.
317
-- Found a local runtime configuration drift before the final install validation:
318
- - `/etc/netdata/netflow.yaml` used a stale journal schema and failed startup.
319
- - Backed it up to `/etc/netdata/netflow.yaml.sow-0015-backup-20260508034532`.
320
- - Installed the current stock config from `src/crates/netflow-plugin/configs/netflow.yaml`.
321
- - Added stock-config parse/validate coverage so this drift is caught by tests.
322
-- Replayed pcap-derived UDP payloads into the installed collector on `127.0.0.1:2055`:
323
- - NetFlow v5 fixture.
324
- - NetFlow v9 template/options/data fixtures.
325
- - IPFIX template/data fixtures.
326
- - sFlow expanded-sample fixture.
327
-- Verified installed runtime output through journal-backed evidence:
328
- - `journalctl --directory=/var/cache/netdata/flows/raw --since [REPLAY_START] -o export` reported rows for `FLOW_VERSION=ipfix`, `FLOW_VERSION=v5`, `FLOW_VERSION=v9`, and `FLOW_VERSION=sflow`.
329
- - The same row set contained ASN and country fields for source/destination enrichment.
330
-- Verified Cloud-proxied Function output after install:
331
- - `agents_call_function --via cloud --function flows:netflow --body '{"info":true}'` returned `status: 200`, `type: flows`, and history metadata.
332
- - A documented `table-sankey` query body with `after: -3600`, `before: 0`, and numeric `top_n: 100` returned `status: 200`, flow rows, stats, and zero journal/template/parse errors.
333
-- Reopened SOW after user decision to:
334
- - enhance the topology IP intelligence downloader for all supported Geo/ASN providers that can feed generated MMDBs;
335
- - update related docs;
336
- - test providers with actual data where public/licensed access permits;
337
- - validate NetBox, generic IPAM, bio-rd, and GoBGP/FRR/BMP locally.
338
-- Enhanced `src/go/tools/topology-ip-intel-downloader/` provider support:
339
- - ASN: DB-IP MMDB/CSV, IPtoASN, CAIDA prefix2as, MaxMind GeoLite2 ASN.
340
- - GEO: DB-IP country/city MMDB/CSV, IPtoASN country, MaxMind GeoLite2 Country CSV, IP2Location Lite country, IPDeny country zones, IPIP country.
341
- - Added source-specific archive decoding for MaxMind tar.gz/zip, IP2Location zip, IPDeny tar.gz, IPIP zip, CAIDA latest-routeviews resolution, and redacted MaxMind query metadata.
342
- - Added parser tests for CAIDA, MaxMind Country CSV zip, IP2Location, IPDeny, IPIP, MaxMind env expansion/redaction, and all new provider tokens.
343
-- Added netflow plugin contract tests for generated Netdata topology MMDBs:
344
- - committed tiny ASN/GEO MMDB fixtures generated by the downloader;
345
- - `netdata_topology_mmdb_enrichment_populates_asn_and_geo_fields`;
346
- - opt-in actual-data test `actual_provider_mmdb_outputs_are_readable_when_root_is_set`.
347
-- Ran actual public provider downloads and generated MMDBs under `.local/audits/topology-ip-intel/providers`:
348
- - `dbip:asn-lite@mmdb`: 469,774 ASN ranges.
349
- - `dbip:asn-lite@csv`: 469,777 ASN ranges.
350
- - `iptoasn:combined` as ASN: 618,851 ASN ranges.
351
- - `caida:prefix2as`: 388,967 ASN ranges from `routeviews-rv2-20260506-1200.pfx2as.gz`.
352
- - `dbip:country-lite@mmdb`: 702,373 geo ranges.
353
- - `dbip:country-lite@csv`: 702,409 geo ranges.
354
- - `dbip:city-lite@mmdb`: 8,061,643 geo ranges.
355
- - `dbip:city-lite@csv`: 8,061,679 geo ranges.
356
- - `iptoasn:combined` as GEO: 286,373 geo ranges.
357
- - `ip2location:country-lite`: 273,642 geo ranges.
358
- - `ipdeny:country-zones`: 108,309 geo ranges.
359
- - `ipip:country`: 507,524 geo ranges.
360
-- After the user pointed to a local update-ipsets environment source, live MaxMind downloads were also run without printing the key:
361
- - `maxmind:geolite2-asn`: 501,939 ASN ranges.
362
- - `maxmind:geolite2-country`: 599,247 geo ranges.
363
-- Validated local live services:
364
- - NetBox: cloned `netbox-community/netbox-docker @ 55edb986b22c12b69ec89c79c8ccc7fdea88c5b5`, started NetBox v4.6 under Docker Compose, inserted a prefix, fetched `/api/ipam/prefixes/?limit=1000`, and applied the documented transform to produce `{prefix,name,tenant,role,site}`.
365
- - Generic JSON-over-HTTP IPAM: served a local JSON endpoint and validated the documented transform shape.
366
- - GoBGP/BMP: built `osrg/gobgp @ 5f191066a78e2c1e929c54b5b75fe2c683c166e4`; added an opt-in test that starts a Netdata BMP listener, starts `gobgpd`, adds a route with `gobgp`, and verifies the dynamic routing runtime receives ASN and AS-path.
367
- - FRR/BMP: pulled `frrouting/frr:latest`, verified documented BMP config with `bgpd -M bmp -C`, then ran a container that established a BMP connection and emitted BMP bytes to a local listener.
368
- - bio-rd: built `bio-routing/bio-rd @ 14a8de966e8b3f488a207aa0d02454a447ed5c99`; its RIS gRPC path is covered by Netdata's in-process test, but a practical GoBGP-to-bio-rd no-neighbor local-RIB setup crashed upstream bio-rd on a BMP Peer Up. This is recorded as an external blocker, not a Netdata client failure.
369
-- Corrected user-facing docs:
370
- - downloader README and sample config now list all supported provider tokens/formats and MaxMind license-key behavior;
371
- - MaxMind integration docs now describe the topology downloader path for GeoLite2 ASN/Country;
372
- - NetBox docs now show the NetBox 4.5+ v2 bearer header as `Bearer nbt_<12-char-key>.<40-char-token>` and explain that sending a v2 token with the legacy `Token` prefix produces an invalid-v1-token error.
373
-
374
-## Validation
375
-
376
-Acceptance criteria evidence:
377
-
378
-- Scoped enrichment coverage:
379
- - Custom MMDB / MaxMind / GeoLite2: `src/crates/netflow-plugin/src/enrichment/tests.rs` covers public ASN and City MMDB fixtures, ASN org, country/city/state/lat/lon, and `netdata.ip_class`.
380
- - DB-IP / IPtoASN baseline: existing default geolocation runtime was treated as user-confirmed baseline; new MMDB tests exercise the same resolver path with deterministic fixtures.
381
- - AWS / Azure / GCP / generic IPAM / NetBox: `src/crates/netflow-plugin/src/network_sources/tests.rs` covers documented transform payloads and the HTTP fetch/publish path.
382
- - Static metadata, sampling overrides, static networks, classifiers, ASN/network provider chains, and decapsulation: existing tests remain in `src/crates/netflow-plugin/src/enrichment/tests.rs` and decoder fixture tests; new runtime merge and decode/enrichment tests cover the boundary.
383
- - BioRIS: `src/crates/netflow-plugin/src/routing/bioris/tests.rs` covers a real in-process gRPC service and runtime route lookup after `DumpRIB`.
384
- - BMP: `src/crates/netflow-plugin/src/routing/bmp/tests.rs` covers a real TCP listener and BMP-framed messages.
385
- - Downloader provider conversion: `src/go/tools/topology-ip-intel-downloader/*_test.go` covers every supported built-in provider token/format and the provider-specific parser/decode paths.
386
- - Generated Netdata topology MMDB consumption: `src/crates/netflow-plugin/src/enrichment/tests.rs` reads downloader-generated MMDB fixtures and proves the Rust enricher sees ASN, ASN name, country, city, state, and coordinates.
387
- - Actual downloaded provider outputs: opt-in test `actual_provider_mmdb_outputs_are_readable_when_root_is_set` was run against `.local/audits/topology-ip-intel/providers` and read every public provider output with the Rust resolver, including live MaxMind GeoLite2 ASN and Country outputs.
388
- - Real GoBGP BMP speaker: opt-in test `bmp_listener_accepts_gobgp_route_when_binaries_are_set` was run against locally built `gobgpd`/`gobgp` and proved a route added through GoBGP reaches the Netdata dynamic-routing runtime.
389
- - NetFlow/sFlow: `src/crates/netflow-plugin/src/decoder/tests.rs` proves NetFlow v9 and sFlow records receive enrichment during decode; installed runtime replay produced v5/v9/sFlow journal rows.
390
- - Journal/query visibility: `src/crates/netflow-plugin/src/main_tests.rs` proves enriched fields are written to the raw journal and are query-visible by `SRC_NET_NAME`.
391
-
392
-Tests or equivalent validation:
393
-
394
-- `cargo fmt -p netflow-plugin` passed.
395
-- Targeted tests passed:
396
- - `cargo test -p netflow-plugin apply_geo_record_accepts_netdata_ip_class_metadata -- --nocapture`
397
- - `cargo test -p netflow-plugin maxmind_geolite2_mmdb_enrichment_populates_asn_and_geo_fields -- --nocapture`
398
- - `cargo test -p netflow-plugin request_deserialization_accepts -- --nocapture`
399
- - `cargo test -p netflow-plugin resolve_time_bounds_treats_negative_after_as_relative_to_before -- --nocapture`
400
-- Full Rust plugin suite passed after fixes:
401
- - `cargo test -p netflow-plugin --manifest-path src/crates/Cargo.toml`
402
- - Result: 447 passed, 18 ignored, 0 failed.
403
- - `tests/grpc_build.rs`: 1 passed, 0 failed.
404
-- Downloader tests passed:
405
- - `go test ./tools/topology-ip-intel-downloader` from `src/go`.
406
-- Actual provider output tests passed:
407
- - `NETDATA_TOPOLOGY_IP_INTEL_PROVIDER_ROOT="$(pwd)/.local/audits/topology-ip-intel/providers" cargo test -p netflow-plugin actual_provider_mmdb_outputs_are_readable_when_root_is_set --manifest-path src/crates/Cargo.toml`
408
- - `cargo test -p netflow-plugin netdata_topology_mmdb_enrichment_populates_asn_and_geo_fields --manifest-path src/crates/Cargo.toml`
409
-- Real GoBGP BMP test passed:
410
- - `NETDATA_GOBGPD="$(pwd)/.local/audits/netflow-live/bin/gobgpd" NETDATA_GOBGP="$(pwd)/.local/audits/netflow-live/bin/gobgp" cargo test -p netflow-plugin bmp_listener_accepts_gobgp_route_when_binaries_are_set --manifest-path src/crates/Cargo.toml`
411
-- Network-source and routing targeted tests passed:
412
- - `cargo test -p netflow-plugin network_sources --manifest-path src/crates/Cargo.toml`
413
- - `cargo test -p netflow-plugin routing::bioris --manifest-path src/crates/Cargo.toml`
414
- - `cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml`
415
-- Integration metadata/docs validation passed:
416
- - `python3 integrations/gen_integrations.py`
417
- - `python3 integrations/gen_docs_integrations.py`
418
- - `python3` YAML parse for `src/crates/netflow-plugin/metadata.yaml`
419
- - `git diff --check`
420
-- SOW audit was run after moving this file to `done/`. Status/directory checks passed; the command exited nonzero only on a pre-existing false positive in an unmodified mirror-skill Git SSH URL that the audit pattern classifies as an email-like string.
421
-- Public helper validation passed:
422
- - `bash -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'`
423
- - `zsh -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'`
424
- - `shellcheck --external-sources docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`
425
-
426
-Real-use evidence:
427
-
428
-- `./build-install-netflow-plugin.sh` completed:
429
- - Release build finished.
430
- - Plugin installed to `/usr/libexec/netdata/plugins.d/netflow-plugin`.
431
- - `systemctl is-active netdata` returned `active`.
432
-- Installed runtime:
433
- - `journalctl --namespace=netdata SYSLOG_IDENTIFIER=netflow-plugin --since '2 minutes ago'` showed startup, Function declaration, journal scanning, and no startup failure.
434
- - `ss -lunp` showed `netflow-plugin` listening on UDP `0.0.0.0:2055`.
435
-- Runtime pcap replay:
436
- - Eight UDP payloads were sent to `127.0.0.1:2055` from deterministic local pcap fixtures.
437
- - Raw journal inspection since replay start reported:
438
- - rows: 513
439
- - `FLOW_VERSION[ipfix]`: 479
440
- - `FLOW_VERSION[v5]`: 29
441
- - `FLOW_VERSION[v9]`: 4
442
- - `FLOW_VERSION[sflow]`: 1
443
- - source ASN fields: 369
444
- - destination ASN fields: 502
445
- - source country fields: 370
446
- - destination country fields: 503
447
-- Cloud-proxied Function validation:
448
- - Local `/api/v3/info` showed one local agent, Cloud status online, and node/machine/claim identifiers present; raw identifiers were stored only under `.local/`.
449
- - Cloud discovery found the local node reachable in visible rooms by exact local node/machine identifiers.
450
- - `flows:netflow` info call returned `status: 200` and `type: flows`.
451
- - A documented flow query returned `status: 200`, 101 flow rows, expected group-by fields, and stats including decoded NetFlow v5, NetFlow v9, IPFIX, and sFlow counters with zero parse/template/journal-write errors.
452
-- Direct-agent bearer validation:
453
- - A Cloud bearer mint using the local `/api/v3/info` node id, machine GUID, and claim id tuple returned HTTP 200 with a token present.
454
- - `agents_call_function --via agent --function flows:netflow --body '{"info":true}'` against `127.0.0.1:19999` returned `status: 200`, `type: flows`, and `has_history: true`.
455
-- Actual provider downloader validation:
456
- - Every live provider listed in the execution log downloaded, parsed, and wrote a non-empty MMDB output.
457
- - The Rust resolver read all generated outputs successfully in the opt-in actual-data test, including live MaxMind GeoLite2 ASN and Country outputs.
458
-- NetBox live validation:
459
- - Local NetBox v4.6 reached healthy status under Docker Compose.
460
- - `/api/ipam/prefixes/?limit=1000` returned the inserted `198.51.100.0/24` prefix.
461
- - The documented NetBox transform produced `{"prefix":"198.51.100.0/24","tenant":"","role":"","site":"","name":"netdata edge subnet"}`.
462
- - NetBox v4.6 rejected a v2 token sent as legacy `Token` with "Invalid v1 token"; docs were corrected to show `Bearer nbt_<12-char-key>.<40-char-token>`.
463
-- Generic IPAM live validation:
464
- - A local HTTP JSON endpoint returned a prefix list and the documented transform produced `{"prefix":"203.0.113.0/24","name":"corp edge","tenant":"prod"}`.
465
-- BioRIS / BMP live validation:
466
- - GoBGP -> Netdata BMP listener route publication passed with real GoBGP binaries.
467
- - FRR `bgpd -M bmp -C` accepted the documented BMP config, and a running `frrouting/frr` container established a BMP connection and emitted bytes to a local listener.
468
- - GoBGP -> upstream bio-rd RIS crashed inside bio-rd on BMP Peer Up in the no-neighbor local-RIB setup; Netdata's BioRIS client path remains covered by the in-process gRPC service test.
469
-
470
-Reviewer findings:
471
-
472
-- No external AI reviewer was requested by the user.
473
-- Self-review findings handled during execution:
474
- - GeoIP MMDB did not apply `netdata.ip_class`; fixed and tested.
475
- - Public flow-query request shape did not accept documented negative `after` and numeric `top_n`; fixed and tested.
476
- - Public direct-agent helper failed under zsh and leaked identifiers in masked logs; fixed and tested.
477
- - E2E full-suite ingest wait was too short under parallel load; fixed and full suite passed.
478
-
479
-Same-failure scan:
480
-
481
-- Searched for provider-transform examples and `top_n` examples in public flow-query docs/skills.
482
-- The public examples using numeric `top_n` now work because the plugin accepts both numeric and string values.
483
-- The public examples using negative relative `after` now work because request parsing and time-bound resolution support it.
484
-- Searched generated integration side effects after docs generation; reverted unrelated SNMP generated-page churn and verified no SNMP diff remained.
485
-
486
-Sensitive data gate:
487
-
488
-- Raw `/api/v3/info`, Cloud discovery payloads, bearer endpoint responses, and flow Function responses are stored only under `.local/audits/query-netdata-agents/`, which is gitignored.
489
-- This SOW records sanitized summaries only:
490
- - private flow exporter is `[PRIVATE_FLOW_EXPORTER]`;
491
- - local hostname is `[LOCAL_HOST]`;
492
- - no node ids, machine GUIDs, claim ids, tokens, bearer values, raw private endpoints, or raw flow rows are recorded here.
493
-- Public helper logging now masks Cloud tokens, agent bearers, node ids, machine GUIDs, claim ids, space ids, and room ids in stderr.
494
-
495
-Artifact maintenance gate:
496
-
497
-- AGENTS.md: no update needed; workflow rules were sufficient.
498
-- Runtime project skills: no project-local collector/integration skill update needed; existing `project-writing-collectors` and `integrations-lifecycle` rules were sufficient.
499
-- Specs: no separate spec update needed; this SOW did not change a durable product contract beyond making implementation match already published Function-query examples and MMDB metadata behavior.
500
-- End-user/operator docs: updated `src/crates/netflow-plugin/metadata.yaml` and regenerated affected integration pages.
501
-- End-user/operator skills: updated `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`; added Cloud and direct-agent how-tos for local flow Function validation.
502
-- SOW lifecycle: SOW is marked `completed` and moved to `.agents/sow/done/` with the implementation and validation commit. Remaining items require user-supplied credentials or real external systems and are recorded below.
503
-
504
-Specs update:
505
-
506
-- No spec file was updated. Reason: the durable behavior changes are already captured in code/tests and public operator docs/skills:
507
- - `after: -N`, `before: 0`, and numeric `top_n` are public Function request behavior now covered by tests.
508
- - MMDB `netdata.ip_class` handling now matches existing downloader/documentation intent.
509
-
510
-Project skills update:
511
-
512
-- No runtime project skill update needed.
513
-- Public operator skill helper was updated because its direct-agent transport wrapper failed under zsh and did not mask all identity fields.
514
-
515
-End-user/operator docs update:
516
-
517
-- Updated:
518
- - `src/crates/netflow-plugin/metadata.yaml`
519
- - `src/crates/netflow-plugin/integrations/netbox.md`
520
- - `src/crates/netflow-plugin/integrations/maxmind_geoip_-_geolite2.md`
521
- - `src/crates/netflow-plugin/integrations/bmp_bgp_monitoring_protocol.md`
522
- - `src/go/tools/topology-ip-intel-downloader/README.md`
523
- - `src/go/tools/topology-ip-intel-downloader/configs/topology-ip-intel.yaml`
524
- - `docs/netdata-ai/skills/query-netdata-cloud/how-tos/validate-local-netflow-function.md`
525
- - `docs/netdata-ai/skills/query-netdata-cloud/how-tos/INDEX.md`
526
- - `docs/netdata-ai/skills/query-netdata-agents/how-tos/validate-direct-local-flow-function.md`
527
- - `docs/netdata-ai/skills/query-netdata-agents/how-tos/INDEX.md`
528
-
529
-End-user/operator skills update:
530
-
531
-- Updated `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
532
-- Added `docs/netdata-ai/skills/query-netdata-agents/how-tos/validate-direct-local-flow-function.md`.
533
-- Validation: Bash self-test, zsh self-test, and shellcheck all passed.
534
-
535
-Lessons:
536
-
537
-- Testing the public instructions against the installed agent exposed real API-compatibility bugs that pure unit tests had missed.
538
-- Direct-agent bearer validation works when the wrapper uses the exact local `/api/v3/info` node id, machine GUID, and claim id tuple. Mixed node/machine/claim tuples should be treated as identity-resolution bugs before treating direct bearer as blocked.
539
-- Masked command logging must redact identifiers as well as tokens, because node ids, machine GUIDs, claim ids, space ids, and room ids are sensitive enough to keep out of durable artifacts.
540
-
541
-Follow-up mapping:
542
-
543
-- Requires user involvement if further proof is required:
544
- - Real BMP from a production router: user does not run BGP/BMP locally. Current proof covers deterministic BMP frames, real GoBGP-to-Netdata BMP route publication, and FRR BMP config/connection. A real router session still requires a router or lab BGP topology.
545
- - Real bio-rd / RIPE RIS with live routes: Netdata's BioRIS client path is covered by an in-process gRPC service, but the local GoBGP-to-upstream-bio-rd setup crashed inside bio-rd on BMP Peer Up. Further proof needs either a stable bio-rd/RIS endpoint or time to debug/report/fix upstream bio-rd.
546
- - User-owned NetBox / custom IPAM endpoints: local NetBox and generic IPAM were validated; proof against the user's own systems requires their endpoint and token if desired.
547
- - Custom operator-provided MMDB: current proof uses public MaxMind test MMDB fixtures and a synthetic `netdata.ip_class` record; no user custom MMDB was supplied.
548
-- Explicitly out of this SOW:
549
- - CI job wiring. The user corrected that the immediate scope is unit/integration testing and practical validation, not CI-job integration.
550
-
551
-## Outcome
552
-
553
-Autonomous work completed as far as possible without external systems or user credentials. Automated coverage, downloader provider support, actual public-provider data validation, local NetBox/IPAM validation, GoBGP/FRR BMP validation, docs/source corrections, installed runtime validation, journal proof, Cloud-proxied Function validation, and direct-agent bearer Function validation are complete.
554
-
555
-## Lessons Extracted
556
-
557
-- Run public examples against the installed plugin, not only through Rust unit tests. This caught `after`/`top_n` request-shape bugs.
558
-- Include shell compatibility in public helper tests when examples are likely to be run from users' default shells.
559
-- Use the exact local `/api/v3/info` node id, machine GUID, and claim id tuple for direct-agent bearer minting; mixed identity tuples are the common failure mode.
560
-- Treat identity fields as sensitive in wrapper logging; masking only bearer/token values is insufficient.
561
-
562
-## Followup
563
-
564
-User-required items are listed in Follow-up mapping above. No autonomous follow-up remains unattempted in this SOW.
565
-
566
-## PR Review Iteration - 2026-05-08
567
-
568
-Trigger:
569
-
570
-- User asked to solve PR comments/reviews on PR 22453 before continuing.
571
-
572
-Finding sources fetched:
573
-
574
-- `bash .agents/skills/pr-reviews/scripts/fetch-all.sh 22453`
575
- - 8 open review threads.
576
- - All open review threads are from `cubic-dev-ai[bot]`.
577
- - No human review comments need a user decision before implementation.
578
-- `bash .agents/skills/pr-reviews/scripts/fetch-sonar-findings.sh 22453`
579
- - 0 SonarCloud issues.
580
- - 0 SonarCloud hotspots.
581
-- `bash .agents/skills/pr-reviews/scripts/ci-status.sh 22453`
582
- - 0 failing checks.
583
- - Most checks are still pending after the latest push.
584
-
585
-Open review threads to verify:
586
-
587
-- `src/crates/netflow-plugin/src/routing/bioris/tests.rs:242`: local gRPC test reserves and releases an ephemeral port before server bind.
588
-- `src/crates/netflow-plugin/src/routing/bmp/tests.rs:274`: BMP listener test reserves and releases an ephemeral port before listener bind.
589
-- `src/crates/netflow-plugin/src/query/planner/request.rs:20`: timestamp clamp may collapse `after` and `before`.
590
-- `docs/netdata-ai/skills/query-netdata-agents/how-tos/validate-direct-local-flow-function.md:25`: guide persists raw `/api/v3/info` payload containing sensitive identifiers.
591
-- `src/crates/netflow-plugin/src/network_sources/tests.rs:171`: NetBox transform test maps `site` but does not assert it.
592
-- `src/go/tools/topology-ip-intel-downloader/config.go:280`: IPDeny built-in source uses HTTP.
593
-- `src/crates/netflow-plugin/src/routing/bmp/tests.rs:431`: async test uses blocking `std::process::Command::status()`.
594
-- `src/go/tools/topology-ip-intel-downloader/parse.go:775`: IPDeny `.ZONE` extension handling is case-sensitive after case-insensitive filter.
595
-
596
-Plan:
597
-
598
-- Verify each bot finding against current code.
599
-- Fix valid findings and search touched PR files for the same failure class.
600
-- Run the narrow tests for modified areas plus formatting and diff checks.
601
-- Re-fetch findings before push.
602
-- Push one commit for the review fixes, reply per thread, resolve threads, and retrigger AI reviewers.
603
-
604
-Verified fixes:
605
-
606
-- BioRIS gRPC test now binds the Tokio listener first and serves tonic from that bound listener via `TcpListenerStream`, removing the reserve/release race.
607
-- BMP listener tests now pass a pre-bound Tokio listener into a shared listener helper, removing the reserve/release race for the Netdata listener side.
608
-- The remaining GoBGP API port is for an external daemon that requires a concrete address; the test records that limitation and keeps the path opt-in.
609
-- Query time-bound resolution now clamps first and enforces `after < before` after clamping.
610
-- Direct-agent and Cloud flow-validation how-tos no longer persist raw `/api/v3/info`; they keep the raw identity payload in shell memory and print only presence checks.
611
-- NetBox transform tests now assert the documented `site` mapping.
612
-- IPDeny built-in URL now uses HTTPS; live HEAD check returned HTTP 200 for the HTTPS URL.
613
-- GoBGP helper commands in the async BMP test now use `tokio::process::Command`.
614
-- IPDeny `.zone` parsing now handles uppercase `.ZONE` suffixes and has order-independent test coverage.
615
-
616
-Validation after fixes:
617
-
618
-- `go test ./tools/topology-ip-intel-downloader`
619
-- `cargo test -p netflow-plugin documented_cloud_and_ipam_transforms_decode_provider_payloads --manifest-path src/crates/Cargo.toml`
620
-- `cargo test -p netflow-plugin resolve_time_bounds --manifest-path src/crates/Cargo.toml`
621
-- `cargo test -p netflow-plugin routing::bioris --manifest-path src/crates/Cargo.toml`
622
-- `cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml`
623
-- `NETDATA_GOBGPD="$(pwd)/.local/audits/netflow-live/bin/gobgpd" NETDATA_GOBGP="$(pwd)/.local/audits/netflow-live/bin/gobgp" cargo test -p netflow-plugin bmp_listener_accepts_gobgp_route_when_binaries_are_set --manifest-path src/crates/Cargo.toml`
624
-- `cargo test -p netflow-plugin --manifest-path src/crates/Cargo.toml`
625
- - Result: 448 passed, 18 ignored, 0 failed.
626
- - `tests/grpc_build.rs`: 1 passed, 0 failed.
627
-- How-to jq snippets were smoke-tested with a synthetic `/api/v3/info` payload without writing raw identity JSON.
628
-- `git diff --check`
629
-
630
-Final pre-push sync:
631
-
632
-- `bash .agents/skills/pr-reviews/scripts/fetch-all.sh 22453`
633
- - 8 open review threads, unchanged from the initial set.
634
- - No new comments.
635
-- `bash .agents/skills/pr-reviews/scripts/fetch-sonar-findings.sh 22453`
636
- - 0 issues.
637
- - 0 hotspots.
638
-- `bash .agents/skills/pr-reviews/scripts/ci-status.sh 22453`
639
- - 0 failing checks.
640
- - 90 checks still running on the previous commit.
641
-
642
-Artifact maintenance for review pass:
643
-
644
-- AGENTS.md: no update needed; existing PR-review and sensitive-data rules were sufficient.
645
-- Runtime project skills: no update needed.
646
-- Specs: no update needed; fixes align implementation/tests/docs to existing intended behavior.
647
-- End-user/operator docs: updated direct-agent and Cloud validation how-tos to avoid durable raw identity payloads.
648
-- End-user/operator skills: no script behavior changed in this review pass.
649
-- SOW lifecycle: SOW returned to `completed` and is moved back to `.agents/sow/done/` with the review-fix commit.
650
-
651
-## Regression Log
652
-
653
-Note: dated regression entries intentionally use `## Regression - YYYY-MM-DD`
654
-headings to match the repository SOW lifecycle contract.
655
-
656
-## Regression - 2026-05-08 - Missing Provider Integration Modules
657
-
658
-What broke:
659
-
660
-- The previous completion enhanced and validated the topology IP intelligence downloader for CAIDA, IP2Location, IPDeny, and IPIP, but the user-facing NetFlow enrichment metadata still exposed only DB-IP, MaxMind, IPtoASN, and Custom MMDB as IP intelligence integrations.
661
-- This made the generated integrations catalog incomplete: users could discover downloader tokens in the downloader README, but not from the NetFlow enrichment-method module list.
662
-
663
-Evidence:
664
-
665
-- `src/crates/netflow-plugin/metadata.yaml` had modules for `dbip`, `maxmind`, `iptoasn`, and `custom-mmdb`.
666
-- Generated pages under `src/crates/netflow-plugin/integrations/` existed only for `db-ip_ip_intelligence.md`, `maxmind_geoip_-_geolite2.md`, `iptoasn.md`, and `custom_mmdb_database.md`.
667
-- Downloader code supports `caida:prefix2as`, `ip2location:country-lite`, `ipdeny:country-zones`, and `ipip:country` in `src/go/tools/topology-ip-intel-downloader/config.go`.
668
-
669
-Why previous validation missed it:
670
-
671
-- Validation proved downloader parsing, live provider downloads, generated MMDB output, and Rust resolver consumption, but did not compare the complete downloader provider matrix against generated integration module coverage.
672
-
673
-Repair plan:
674
-
675
-- Add first-class `metadata.yaml` modules for CAIDA Routeviews Prefix-to-AS, IP2Location LITE IP-Country, IPDeny Country Zones, and IPIP Country.
676
-- Cross-link these modules with the existing DB-IP, MaxMind, IPtoASN, and Custom MMDB modules.
677
-- Regenerate generated integration pages from `metadata.yaml`.
678
-- Validate metadata generation and downloader tests.
679
-
680
-Sensitive data handling:
681
-
682
-- No secrets, private endpoints, node identifiers, or raw provider payloads are needed in durable artifacts. Provider descriptions use public source names, public URLs, and downloader tokens only.
683
-
684
-Implementation:
685
-
686
-- Added first-class NetFlow enrichment metadata modules for:
687
- - `caida-prefix2as` - CAIDA Routeviews Prefix-to-AS.
688
- - `ip2location` - IP2Location LITE IP-Country.
689
- - `ipdeny` - IPDeny Country Zones.
690
- - `ipip` - IPIP Country Database.
691
-- Updated related-resource links from DB-IP, MaxMind, IPtoASN, and Custom MMDB so the generated integration catalog surfaces the full supported IP intelligence provider set.
692
-- Regenerated the new NetFlow integration pages from `src/crates/netflow-plugin/metadata.yaml`.
693
-
694
-Validation:
695
-
696
-- `go test ./tools/topology-ip-intel-downloader` passed from `src/go`.
697
-- `python3 integrations/gen_integrations.py` passed.
698
-- `python3 integrations/gen_docs_integrations.py` generated the new NetFlow provider pages. It also reproduced a pre-existing unrelated SNMP generated-page diff; that unrelated SNMP churn was removed from this changeset.
699
-- A metadata coverage check verified all expected IP intelligence modules are present: `dbip`, `maxmind`, `iptoasn`, `custom-mmdb`, `caida-prefix2as`, `ip2location`, `ipdeny`, and `ipip`.
700
-- Generated page existence check verified:
701
- - `src/crates/netflow-plugin/integrations/caida_routeviews_prefix-to-as.md`
702
- - `src/crates/netflow-plugin/integrations/ip2location_lite_ip-country.md`
703
- - `src/crates/netflow-plugin/integrations/ipdeny_country_zones.md`
704
- - `src/crates/netflow-plugin/integrations/ipip_country_database.md`
705
-- `git diff --check` passed after removing unrelated generated SNMP churn.
706
-
707
-Artifact maintenance:
708
-
709
-- AGENTS.md: no update needed; existing integration-generation and SOW regression rules were sufficient.
710
-- Runtime project skills: no update needed; `integrations-lifecycle` already documents the source/generator contract.
711
-- Specs: no update needed; this is a documentation/catalog completeness repair for already-implemented provider behavior.
712
-- End-user/operator docs: updated `src/crates/netflow-plugin/metadata.yaml` and generated four new provider integration pages.
713
-- End-user/operator skills: no update needed; no public skill workflow changed.
714
-- SOW lifecycle: regression recorded here, then SOW returned to `completed` and moved back to `.agents/sow/done/` with the metadata/doc commit.
715
-
716
-### PR Review Follow-up - 2026-05-08 - Copilot Findings
717
-
718
-Trigger:
719
-
720
-- Copilot opened PR review threads on:
721
- - `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh:171`
722
- - `src/go/tools/topology-ip-intel-downloader/fetch.go:295`
723
-
724
-Findings:
725
-
726
-- `_agents_set_outvar` used `eval "${name}=\${value}"` to write external command output into a caller-provided variable.
727
-- The variable name was validated, and the previous assignment form avoided common word-splitting behavior, but the pattern was still fragile and hard to audit because token and claim values originate from `curl` / `jq` output.
728
-- `decodeMaxMindASNPayload` accepted every `extractMMDBFromTar` error as "not a tarred MMDB" and returned the ungzipped content. That preserved plain gzipped MMDB support, but it also hid corrupt tar-like payloads behind a later, less-specific MMDB parse error.
729
-
730
-Implementation:
731
-
732
-- Replaced the eval assignment with `printf -v "${_agents_out_name}" '%s' "${_agents_out_value}"`.
733
-- Kept variable-name validation before assignment.
734
-- Added a self-test payload containing whitespace, glob characters, shell metacharacters, command-substitution text, backticks, quotes, and a newline, verifying the helper preserves it as data and does not interpret it.
735
-- Updated the public `query-netdata-agents` skill text from "bash nameref" to "validated caller-local output variable", matching the zsh-compatible implementation.
736
-- Updated MaxMind ASN payload decoding so plain gzipped MMDB payloads still pass through, while tar-like payloads with invalid tar structure now return a clear extraction error.
737
-- Added downloader test coverage for corrupt tar-like MaxMind ASN payloads.
738
-
739
-Validation:
740
-
741
-- `bash -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
742
-- `zsh -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
743
-- `shellcheck --external-sources docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh` passed.
744
-- `git diff --check` passed.
745
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on the existing unmodified `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern that the audit classifies as email-like sensitive data.
746
-- `git diff --check` passed.
747
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on the existing unmodified `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern that the audit classifies as email-like sensitive data.
748
-- `go test ./tools/topology-ip-intel-downloader` passed from `src/go`.
749
-- Same-pattern search found no remaining eval-based `_agents_set_outvar` assignment in the public skill path.
750
-
751
-Artifact maintenance:
752
-
753
-- AGENTS.md: no update needed; existing sensitive-data and public-skill rules were sufficient.
754
-- Runtime project skills: no update needed.
755
-- Specs: no update needed; this repaired implementation safety/error handling for existing helper/provider contracts.
756
-- End-user/operator docs: updated `docs/netdata-ai/skills/query-netdata-agents/SKILL.md` to describe the actual output-variable contract.
757
-- End-user/operator skills: updated `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
758
-- SOW lifecycle: reopened from `done`, recorded this PR review follow-up, then returned to `completed` and moved back to `.agents/sow/done/` with the fix commit.
759
-
760
-### External App Validation Follow-up - 2026-05-08
761
-
762
-Trigger:
763
-
764
-- User asked to proceed with the remaining external application validation after PR review cleanup.
765
-
766
-Scope exercised:
767
-
768
-- Generic JSON-over-HTTP IPAM.
769
-- NetBox.
770
-- FRR BMP.
771
-- GoBGP BMP.
772
-- bio-rd RIS / BioRIS.
773
-
774
-Evidence and results:
775
-
776
-- Generic JSON-over-HTTP IPAM:
777
- - Started a temporary localhost HTTP server serving sanitized RFC documentation prefixes.
778
- - Fetched `networks.json` over HTTP and applied the documented transform `.[] | {prefix: .cidr, name: .label, tenant: .tenant}`.
779
- - Result: `{"prefix":"203.0.113.0/24","name":"corp edge","tenant":"prod"}`.
780
- - Temporary HTTP process was verified stopped.
781
-- NetBox:
782
- - Started local `netbox-docker` under a dedicated Compose project.
783
- - Initial health check exceeded the Docker health timeout while migrations were still finishing, but the NetBox login endpoint returned HTTP 200 and the API was usable after startup.
784
- - Bootstrapped a local prefix and queried `/api/ipam/prefixes/?limit=1000` with a generated local token.
785
- - Applied the documented transform and verified the expected result:
786
- `{"prefix":"198.51.100.0/24","tenant":"","role":"","site":"","name":"netdata edge subnet"}`.
787
- - Compose project was shut down with volumes removed.
788
-- FRR BMP:
789
- - Used `quay.io/frrouting/frr:10.6.0`.
790
- - `bgpd -M bmp -C -f /etc/frr/bgpd.conf` accepted the BMP config. FRR logged warnings about missing synthetic OPEN messages for the static announcement, but did not reject the config.
791
- - A running FRR container connected to a local BMP listener and emitted 297 bytes. The captured stream began with BMP v3 and an FRRouting initiation payload.
792
- - Temporary listener process and container were verified stopped.
793
-- GoBGP BMP:
794
- - `NETDATA_GOBGPD=.local/audits/netflow-live/bin/gobgpd NETDATA_GOBGP=.local/audits/netflow-live/bin/gobgp cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml`
795
- - Result: 13 passed, 0 failed, including `bmp_listener_accepts_gobgp_route_when_binaries_are_set`.
796
-- BioRIS / bio-rd:
797
- - Upstream source evidence shows `bio-rd` `cmd/ris` is BMP-backed:
798
- - `bio-routing/bio-rd @ 14a8de966e8b3f488a207aa0d02454a447ed5c99`, `cmd/ris/main.go:52-83` creates a BMP receiver, listens for BMP, adds configured BMP servers, then exposes the RIS gRPC server.
799
- - `bio-routing/bio-rd @ 14a8de966e8b3f488a207aa0d02454a447ed5c99`, `cmd/ris/config/config.go:10-20` defines `bmp_servers`.
800
- - Official RIPE RIS documentation checked:
801
- - `https://ris-live.ripe.net/` describes RIS Live as WebSocket JSON.
802
- - `https://ris.ripe.net/docs/` lists route collectors, raw MRT files, RIS Live, RISwhois, and routing beacons.
803
- - No official public RIPE RIS endpoint implementing `bio.ris.RoutingInformationService` gRPC was found.
804
- - GoBGP-to-upstream-bio-rd reached BMP Peer Up, then upstream bio-rd crashed in its BGP/BMP handling. This is an upstream/runtime issue, not a Netdata BioRIS client failure.
805
- - FRR-to-upstream-bio-rd did not crash; `riscli routers` saw a router from `[LOCAL_HOST]`, but the local loopback lab did not produce a stable non-empty RIB. The logs showed peer up/down behavior and duplicate-neighbor handling.
806
- - Netdata's BioRIS client path remains covered by the in-process gRPC test that exercises `GetRouters`, `DumpRIB`, and `ObserveRIB`.
807
-
808
-Documentation correction:
809
-
810
-- Corrected `src/crates/netflow-plugin/metadata.yaml` so the BioRIS module says Netdata consumes only a bio-rd-compatible `RoutingInformationService` gRPC endpoint.
811
-- Corrected the generated `src/crates/netflow-plugin/integrations/bio-rd_-_ripe_ris.md` from metadata.
812
-- Corrected `docs/network-flows/enrichment.md` and `docs/network-flows/configuration.md` link text / routing-overlay language.
813
-- Removed misleading instructions that implied users can point `grpc_addr` at RIPE RIS Live, RIPEstat, MRT dumps, route collector sessions, or looking-glass sources directly.
814
-
815
-Validation after documentation correction:
816
-
817
-- `python3 integrations/gen_integrations.py` passed.
818
-- `python3 integrations/gen_docs_integrations.py` passed.
819
-- `python3 -c` YAML parse of `src/crates/netflow-plugin/metadata.yaml` passed.
820
-- `cargo test -p netflow-plugin network_sources --manifest-path src/crates/Cargo.toml`
821
- - Result: 17 passed, 0 failed.
822
-- `cargo test -p netflow-plugin routing::bioris --manifest-path src/crates/Cargo.toml`
823
- - Result: 7 passed, 0 failed.
824
-- `NETDATA_GOBGPD="$(pwd)/.local/audits/netflow-live/bin/gobgpd" NETDATA_GOBGP="$(pwd)/.local/audits/netflow-live/bin/gobgp" cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml`
825
- - Result: 13 passed, 0 failed.
826
-- `git diff --check` passed.
827
-- Generated-doc noise from unrelated SNMP and trailing blank-line changes was removed from the changeset.
828
-- A process/container cleanup check found no remaining temporary live-validation processes or containers.
829
-
830
-Artifact maintenance:
831
-
832
-- AGENTS.md: no update needed; existing SOW, sensitive-data, and generated-doc rules were sufficient.
833
-- Runtime project skills: no update needed; `project-writing-collectors`, `integrations-lifecycle`, and `mirrored-repos` already cover the workflow.
834
-- Specs: no update needed; this corrected public/operator documentation to match existing implementation and upstream protocol reality.
835
-- End-user/operator docs: updated BioRIS metadata, generated integration docs, and hand-authored network-flow docs.
836
-- End-user/operator skills: no update needed; no public skill workflow changed.
837
-- SOW lifecycle: reopened from `done`, recorded external validation, then returned to `completed` and moved back to `.agents/sow/done/` with the validation/doc commit.
838
-
839
-Additional proof outside the clarified beta validation bar:
840
-
841
-- A real stable bio-rd-compatible `RoutingInformationService` endpoint with a non-empty RIB would expand proof beyond Netdata's client contract into production-style BioRIS interoperability.
842
-- A real router/BMP source would expand proof beyond deterministic BMP frames, GoBGP route publication, and FRR config/connection validation into vendor-router interoperability.
843
-- Debugging, reporting, or fixing upstream bio-rd behavior would be useful only if this beta work is expanded into upstream bio-rd stabilization.
844
-- User-owned NetBox or custom IPAM endpoints would expand proof beyond local NetBox and local generic IPAM validation into user-specific environment validation.
845
-
846
-These are not blockers under the clarified beta validation target. The final autonomous work in this pass strengthened Netdata-side automated coverage for configuration, payload/schema consumption, field mapping, option handling, and stability.
847
-
848
-### Netdata-side Contract Hardening - 2026-05-08
849
-
850
-Trigger:
851
-
852
-- The user clarified that this beta validation does not need to prove every external routing stack is production-stable.
853
-- The required bar is to prove Netdata's side is sane: valid configs connect, bad configs are rejected or reported clearly, expected payloads are consumed, fields map correctly into enrichment state, options are respected, no obvious panic/hang/task leak/silent ignore exists, and user docs describe working commands/configs.
854
-
855
-Implementation:
856
-
857
-- BioRIS:
858
- - Added endpoint URI contract coverage for explicit schemes and `grpc_secure`.
859
- - Added invalid-endpoint coverage proving malformed gRPC URIs produce a clear `invalid BioRIS endpoint URI` error before network dialing.
860
- - Added router IP parsing coverage for plain IPs and socket-address forms.
861
- - Extended the in-process `RoutingInformationService` fixture so tests record `DumpRIB` and `ObserveRIB` requests.
862
- - Added coverage proving configured `vrf_id` and `vrf` are sent to `DumpRIB` and `ObserveRIB`.
863
- - Added coverage proving `DumpRIB` and `ObserveRIB` advertisements publish AS path, communities, and large communities into the runtime trie.
864
- - Added coverage proving `ObserveRIB` withdrawals remove the more-specific route and fall back to the broader route.
865
-- BMP:
866
- - Added deterministic `apply_update` tests proving `collect_asns`, `collect_as_paths`, and `collect_communities` are respected.
867
- - Added deterministic update/withdrawal tests proving AS number, AS path, communities, large communities, next hop, and route deletion map correctly into the runtime trie.
868
-- Config validation:
869
- - Added coverage rejecting enabled BioRIS instances with `grpc_addr` that has neither `host:port` nor an explicit URI scheme.
870
- - Added coverage rejecting zero `timeout`, `refresh`, and `refresh_timeout` values for enabled BioRIS.
871
-
872
-Validation:
873
-
874
-- `cargo test -p netflow-plugin routing::bioris --manifest-path src/crates/Cargo.toml`
875
- - Result: 11 passed, 0 failed.
876
-- `cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml`
877
- - Result: 15 passed, 0 failed, including the opt-in GoBGP route publication test because local GoBGP binaries are present.
878
-- `cargo test -p netflow-plugin plugin_config --manifest-path src/crates/Cargo.toml`
879
- - Result: 30 passed, 0 failed.
880
-- `cargo test -p netflow-plugin --manifest-path src/crates/Cargo.toml`
881
- - Result: 456 passed, 18 ignored, 0 failed.
882
- - `tests/grpc_build.rs`: 1 passed, 0 failed.
883
-- `git diff --check` passed.
884
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on the existing unmodified `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern that the audit classifies as email-like sensitive data.
885
-
886
-End-user documentation check:
887
-
888
-- No end-user docs were changed in this hardening pass.
889
-- Previous BioRIS product docs already describe BioRIS as a bio-rd-compatible `RoutingInformationService` gRPC endpoint and do not describe internal test gaps.
890
-- A docs search was run for internal test-gap phrases across `docs/network-flows`, `src/crates/netflow-plugin/metadata.yaml`, and generated NetFlow integration pages. No product-doc statement was found saying BioRIS or BMP are untested or explaining why internal testing could not be performed.
891
-
892
-Artifact maintenance:
893
-
894
-- AGENTS.md: no update needed; existing SOW, sensitive-data, and validation rules were sufficient.
895
-- Runtime project skills: no update needed; `project-writing-collectors` and `integrations-lifecycle` already cover this workflow.
896
-- Specs: no update needed; this hardening adds tests for existing intended contracts, not a new public contract.
897
-- End-user/operator docs: no update needed in this pass; previous product-doc correction remains valid.
898
-- End-user/operator skills: no update needed; no public skill workflow changed.
899
-- SOW lifecycle: reopened from `done`, clarified the beta validation bar, recorded the contract-hardening work, then returned to `completed` and moved back to `.agents/sow/done/` with the hardening commit.
900
-
901
-Outcome:
902
-
903
-- Under the clarified beta validation target, no autonomous Netdata-side blocker remains for BMP or BioRIS contract proof.
904
-- Broader production-router or production-bio-rd interoperability is explicitly outside this beta blocker and can be handled later as expanded validation or beta feedback.
905
-
906
-### PR Review Follow-up - 2026-05-08 - Direct-agent Output Variable Failure Propagation
907
-
908
-Trigger:
909
-
910
-- Copilot opened one unresolved PR review thread on `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh:192`.
911
-
912
-Finding:
913
-
914
-- `_agents_get_claim_id` called `_agents_set_outvar` without checking its exit status.
915
-- Same-pattern search found additional `_agents_set_outvar` and helper-boundary call sites in `_agents_resolve_bearer` and `agents_query_agent`.
916
-- If a caller supplied an invalid output variable name, the helper could continue after an assignment failure and make a public wrapper proceed with an unset or stale claim/bearer value.
917
-
918
-Implementation:
919
-
920
-- Propagated `_agents_set_outvar` failures with `|| return 1` at every call site.
921
-- Propagated `_agents_get_claim_id` failure from `_agents_resolve_bearer`.
922
-- Propagated `_agents_resolve_bearer` failure from `agents_query_agent`.
923
-- Extended `agents_selftest_no_token_leak` with an invalid-output-variable path proving `_agents_get_claim_id` returns failure when `_agents_set_outvar` rejects the caller-provided variable name.
924
-
925
-Validation:
926
-
927
-- `bash -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
928
-- `zsh -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
929
-- `shellcheck --external-sources docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh` passed.
930
-- Same-pattern search confirmed all `_agents_set_outvar` call sites now propagate failure.
931
-- `git diff --check` passed.
932
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on the existing unmodified `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern that the audit classifies as email-like sensitive data.
933
-
934
-Artifact maintenance:
935
-
936
-- AGENTS.md: no update needed; existing public-skill and sensitive-data rules were sufficient.
937
-- Runtime project skills: no update needed.
938
-- Specs: no update needed; this repaired helper failure propagation for an existing public-skill contract.
939
-- End-user/operator docs: no separate docs update needed; the script behavior now matches the existing token-safe helper contract.
940
-- End-user/operator skills: updated `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
941
-- SOW lifecycle: reopened from `done`, recorded this PR review follow-up, then returned to `completed` and moved back to `.agents/sow/done/` with the fix commit.
942
-
943
-### PR Review Follow-up - 2026-05-08 - Deterministic CAIDA Prefix2AS Resolution
944
-
945
-Trigger:
946
-
947
-- Copilot's refreshed review generated no new inline comments, but its review body listed a low-confidence note about `resolveCAIDAPrefix2ASURL` choosing the last `.pfx2as.gz` log entry.
948
-
949
-Findings:
950
-
951
-- The `parse.go` low-confidence note about missing `netipx` import was false; `src/go/tools/topology-ip-intel-downloader/parse.go` already imports `go4.org/netipx`.
952
-- The CAIDA note was valid. The resolver walked the creation log and retained the last matching candidate, so an out-of-order log could select an older dataset.
953
-- The live CAIDA creation log uses tab-separated rows with a numeric timestamp before the `.pfx2as.gz` path, so the resolver can select deterministically by timestamp and tie-break by path.
954
-
955
-Implementation:
956
-
957
-- Changed `resolveCAIDAPrefix2ASURL` to collect all `.pfx2as.gz` candidates.
958
-- Sorted candidates by parsed numeric timestamp when available, with path as a deterministic fallback/tie-breaker.
959
-- Updated `TestResolveCAIDAPrefix2ASURL` so an older candidate appears after the newer candidate, proving the resolver no longer depends on log order.
960
-
961
-Validation:
962
-
963
-- `curl -fsSL https://data.caida.org/datasets/routing/routeviews-prefix2as/pfx2as-creation.log | tail -n 20` confirmed the current CAIDA log row shape includes a numeric timestamp and path ending in `.pfx2as.gz`.
964
-- `go test ./tools/topology-ip-intel-downloader` from `src/go` passed.
965
-- `git diff --check` passed.
966
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on the existing unmodified `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern that the audit classifies as email-like sensitive data.
967
-
968
-Artifact maintenance:
969
-
970
-- AGENTS.md: no update needed; existing PR review and SOW lifecycle rules were sufficient.
971
-- Runtime project skills: no update needed; this did not change how future agents should work.
972
-- Specs: no update needed; this hardens an existing downloader behavior without changing the public contract.
973
-- End-user/operator docs: no update needed; no user-facing command, option, or provider contract changed.
974
-- End-user/operator skills: no update needed.
975
-- SOW lifecycle: reopened from `done`, recorded this PR review follow-up, then returned to `completed` for the deterministic CAIDA resolver commit.
976
-
977
-### PR Review Follow-up - 2026-05-08 - Helper Comment And Parser Preallocation
978
-
979
-Trigger:
980
-
981
-- Copilot opened a new unresolved PR review thread on `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh:172`.
982
-- The same review body listed low-confidence notes about fixed `1<<20` and `1<<18` parser preallocations in `src/go/tools/topology-ip-intel-downloader/parse.go`.
983
-
984
-Findings:
985
-
986
-- The shell helper logic works under both bash and zsh in local smoke tests, but the source comment over-explained caller-local dynamic scoping and made the zsh support claim look broader than necessary.
987
-- Same-pattern search found no other `Bash and zsh`, `dynamic scope`, or `dynamic scoping` wording in the direct-agent skill helper.
988
-- Same-pattern search found all remaining fixed large range-slice preallocations in `parse.go`.
989
-- The parser preallocation concern was valid: small payloads and tests paid immediate allocation cost for capacities intended for full datasets.
990
-
991
-Implementation:
992
-
993
-- Tightened the `_agents_set_outvar` comment to document only the `eval` avoidance reason.
994
-- Added `estimatedRangeCapacity` for bounded initial capacity based on payload or zip-entry size.
995
-- Replaced fixed `1<<20` and `1<<18` parser preallocations with size-based estimates where useful, or natural growth where no reliable row-count proxy exists.
996
-- Added unit coverage for the capacity-estimation helper.
997
-
998
-Validation:
999
-
1000
-- Same-pattern search found no remaining fixed `make([]asnRange|[]geoRange, 0, 1<<...)` allocations in `parse.go`.
1001
-- Same-pattern search found no remaining direct-agent helper comments claiming bash/zsh dynamic scoping.
1002
-- `go test ./tools/topology-ip-intel-downloader` from `src/go` passed.
1003
-- `bash -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
1004
-- `zsh -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
1005
-- `shellcheck --external-sources docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh` passed.
1006
-
1007
-Artifact maintenance:
1008
-
1009
-- AGENTS.md: no update needed; existing PR review and SOW lifecycle rules were sufficient.
1010
-- Runtime project skills: no update needed.
1011
-- Specs: no update needed; this is implementation hardening and comment clarification for existing behavior.
1012
-- End-user/operator docs: no update needed; no user-facing command, option, or provider contract changed.
1013
-- End-user/operator skills: updated `docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh`.
1014
-- SOW lifecycle: reopened from `done`, recorded this PR review follow-up, then returned to `completed` for the helper-comment and parser-preallocation commit.
1015
-
1016
-### PR Review Follow-up - 2026-05-08 - Downloader URL, Archive, ZIP, And GoBGP Live-Test Hardening
1017
-
1018
-Trigger:
1019
-
1020
-- Copilot's refreshed review generated no new inline comments, but its review body listed low-confidence notes about:
1021
- - MaxMind ASN tar detection relying on the fixed `ustar` marker.
1022
- - CAIDA prefix2as URL construction using fixed suffix trimming and string concatenation.
1023
- - ZIP entry lookup using OS-specific `filepath.Base`.
1024
- - The optional GoBGP BMP live test reserving and then releasing an API port before starting `gobgpd`.
1025
-
1026
-Findings:
1027
-
1028
-- The downloader notes were valid hardening opportunities:
1029
- - `extractMMDBFromTar` did reject tar payloads before attempting tar iteration unless the `ustar` marker was present.
1030
- - `resolveCAIDAPrefix2ASURL` did build the resolved URL by trimming `pfx2as-creation.log` from the full URL string.
1031
- - ZIP members use slash-separated paths, so `path.Base` is the correct package for archive member names.
1032
-- The GoBGP note was a real race in theory. The test is opt-in and local-only, but a retry loop keeps the validation from becoming flaky on shared machines.
1033
-
1034
-Implementation:
1035
-
1036
-- Changed CAIDA prefix2as URL resolution to parse `logURL`, remove query/fragment, normalize the directory path, and resolve the selected candidate with `url.ResolveReference`.
1037
-- Changed MaxMind ASN tar extraction to attempt tar iteration first, while still treating corrupt tar-like payloads with a `ustar` marker as extraction errors instead of silently accepting them as raw MMDB bytes.
1038
-- Added test coverage for CAIDA log URLs with query strings and nested paths.
1039
-- Added test coverage for gzipped legacy tar payloads without the `ustar` marker.
1040
-- Switched archive member base-name handling from `filepath.Base` to `path.Base`.
1041
-- Added retry behavior around the optional GoBGP API port reservation in the BMP live test.
1042
-
1043
-Validation:
1044
-
1045
-- `go test ./tools/topology-ip-intel-downloader` from `src/go` passed.
1046
-- `NETDATA_GOBGPD="$(pwd)/.local/audits/netflow-live/bin/gobgpd" NETDATA_GOBGP="$(pwd)/.local/audits/netflow-live/bin/gobgp" cargo test -p netflow-plugin routing::bmp --manifest-path src/crates/Cargo.toml` passed with 15 tests.
1047
-- `bash -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
1048
-- `zsh -c 'source docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh; agents_selftest_no_token_leak'` passed.
1049
-- `shellcheck --external-sources docs/netdata-ai/skills/query-netdata-agents/scripts/_lib.sh` passed.
1050
-
1051
-Artifact maintenance:
1052
-
1053
-- AGENTS.md: no update needed; existing PR review and SOW lifecycle rules were sufficient.
1054
-- Runtime project skills: no update needed.
1055
-- Specs: no update needed; this is implementation hardening for existing downloader and test behavior.
1056
-- End-user/operator docs: no update needed; no user-facing command, option, or provider contract changed.
1057
-- End-user/operator skills: no additional update needed beyond the direct-agent helper comment from the previous follow-up.
1058
-- SOW lifecycle: reopened from `done`, recorded this PR review follow-up, then returned to `completed` for the downloader and BMP live-test hardening commit.
1059
-
1060
-### PR Review Follow-up - 2026-05-08 - Intel Downloader Provider Documentation
1061
-
1062
-Trigger:
1063
-
1064
-- Copilot's refreshed review body listed a low-confidence note about `netipx` import status and a note about `maxmind:geolite2-country@csv` being a ZIP bundle rather than a single raw CSV file.
1065
-- The user asked whether the PR was ready to merge and approved fixing the remaining documentation mismatch first.
1066
-
1067
-Findings:
1068
-
1069
-- The `netipx` import note was false. `src/go/tools/topology-ip-intel-downloader/parse.go` imports `go4.org/netipx`, and the downloader package test passes.
1070
-- The MaxMind `format: csv` note exposed a real user-documentation issue. `docs/network-flows/intel-downloader.md` still listed only DB-IP and IPtoASN, and still said MaxMind was not supported by the downloader.
1071
-- The current code supports DB-IP, IPtoASN, CAIDA prefix2as, MaxMind GeoLite2 ASN, MaxMind GeoLite2 Country, IP2Location, IPDeny, and IPIP through the downloader.
1072
-- MaxMind GeoLite2 Country and IP2Location country-lite use provider CSV ZIP bundles. For MaxMind Country, `format: csv` means the official GeoLite2 Country CSV ZIP bundle with locations plus IPv4/IPv6 block CSVs; it does not mean a single raw CSV file.
1073
-
1074
-Implementation:
1075
-
1076
-- Updated `docs/network-flows/intel-downloader.md` to list all supported provider/artifact combinations.
1077
-- Removed the stale statement saying MaxMind is unsupported by the downloader.
1078
-- Added explicit MaxMind `MAXMIND_LICENSE_KEY` and CSV ZIP bundle wording.
1079
-- Added notes for CAIDA, IP2Location, IPDeny, and IPIP payload shapes.
1080
-- Updated the refresh-cadence wording and per-provider links.
1081
-
1082
-Validation:
1083
-
1084
-- `rg` confirmed the stale "MaxMind unsupported" wording was removed and the new provider entries / MaxMind CSV ZIP clarification are present.
1085
-- `go test ./tools/topology-ip-intel-downloader` from `src/go` passed.
1086
-- `git diff --check` passed.
1087
-- `bash .agents/sow/audit.sh` verified SOW status/directory consistency. It exited nonzero on existing unrelated audit findings, including `.agents/skills/mirror-netdata-repos/SKILL.md:112` Git SSH URL pattern classified as email-like sensitive data.
1088
-
1089
-Artifact maintenance:
1090
-
1091
-- AGENTS.md: no update needed; this work followed the existing PR review and SOW lifecycle rules.
1092
-- Runtime project skills: no update needed.
1093
-- Specs: no update needed; source code and provider metadata already define the current downloader contract.
1094
-- End-user/operator docs: updated `docs/network-flows/intel-downloader.md`.
1095
-- End-user/operator skills: no update needed; no public skill workflow changed.
1096
-- SOW lifecycle: reopened from `done`, recorded this end-user documentation correction, then returned to `completed` for the docs commit.
.agents/sow/done/SOW-0015-20260508-snmp-bgp-typed-projection.md
deleted
-764
@@ -1,764 +0,0 @@
1
-# SOW-0015 - SNMP BGP typed profile projection
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed after final review-feedback fixes; user decisions 1-23 resolved; all BGP-bearing stock profiles migrated to typed `bgp:` rows; legacy `bgp_public*` runtime deleted; metadata and generated SNMP integration docs regenerated; local `mibs/` reference files removed.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Replace the WIP SNMP BGP monitoring branch's metric-name/tag routing layer with a clean typed BGP profile section and projection. Preserve normal ddsnmp chart-per-row behavior, keep BGP per-peer/per-family health templates enabled by default, and remove unsupported public coverage claims until real fixture evidence exists.
14
-
15
-Scope expansion on 2026-05-09: normalize typed SNMP profile organization before SOW close by moving remaining inline `bgp:` and `licensing:` sections into dedicated underscore-prefixed typed-domain fragments where practical.
16
-
17
-### User Request
18
-
19
-User request summary:
20
-
21
-- Analyze the `pr-ktsaou-snmp-bgp` branch after merge/rebase work.
22
-- Find hacks, smells, correctness risks, missing tests, and clean-end-state refactor directions.
23
-- Prefer a clean end state over low churn or backward compatibility with this WIP branch.
24
-- Use multiple external review agents, reconcile their findings, and record decisions before implementation.
25
-- Use a typed `bgp:` projection rather than the current stringly typed BGP public-metric router.
26
-
27
-Detailed design source:
28
-
29
-- `src/go/plugin/go.d/TODO-snmp-bgp-monitoring-review.md` (local development TODO for this work).
30
-
31
-### Assistant Understanding
32
-
33
-Facts:
34
-
35
-- The branch implements BGP as ordinary SNMP metrics plus a rewrite layer in `collector/snmp/bgp_public*.go`.
36
-- The rewrite layer routes by metric name, infers peer vs peer-family scope from tags, and uses underscore-prefixed labels as an in-band protocol.
37
-- `ddsnmp` already has typed projection precedents: `topology:` and `licensing:` sections, `ProfileMetrics.TopologyMetrics`, and `ProfileMetrics.LicenseRows`.
38
-- The existing generic ddsnmp behavior creates one chart per table metric row. The user explicitly decided BGP must keep that behavior and must not add BGP-specific caps.
39
-- The existing BGP health templates should remain default-on; no alert test framework will be created in this SOW.
40
-- Cumulus and Alcatel BGP capability claims must be removed until real fixtures exist.
41
-- Downloaded BGP MIB files are local reference evidence under untracked `mibs/`; they must not be committed and must be removed before close-out.
42
-
43
-Inferences:
44
-
45
-- The root problem is architectural: BGP has typed rows, identities, state enums, AFI/SAFI enums, and fixed signal classes, but the branch models these as string-routed chart metrics.
46
-- Fixing individual switch cases would leave the same hidden-protocol class of technical debt that topology and licensing migrations removed.
47
-- The clean migration should be schema-first, then runtime projection, then vendor-by-vendor profile migration, then deletion of the old router.
48
-
49
-- Unknowns:
50
-
51
-- No open user decisions remain.
52
-- Real Cumulus and Alcatel fixtures are not available now; coverage claims will be removed until they exist.
53
-- Cisco profile attachment requires a profile audit before Cisco migration, but it does not block schema/API scaffolding.
54
-
55
-### Acceptance Criteria
56
-
57
-- SNMP BGP profile data is represented by a first-class typed `bgp:` profile section/projection, not the `bgp_public*` metric-name router.
58
-- `ProfileMetrics` exposes typed `BGPRows` or an equivalent typed BGP output shape.
59
-- BGP function/cache/charts read typed BGP rows, not underscore-prefixed labels or `bgp_public*`-normalized ordinary metrics.
60
-- `bgp_public.go`, `bgp_public_routes.go`, `bgp_public_specs.go`, and related old-routing paths are deleted by the final migration.
61
-- The underscore-prefix BGP tag protocol is deleted by the final migration.
62
-- BGP structural identity uses a canonical length-prefixed key derived from typed identity fields, not display labels or underscore-concatenated strings.
63
-- AFI/SAFI values use closed canonical enums with an explicit vendor-private allow-list.
64
-- BGP state mappings cover all six RFC 4271 peer states unless a row explicitly declares partial coverage.
65
-- Profile validation rejects readable-symbol use of MIB `MAX-ACCESS not-accessible` objects; such objects must be derived from row indexes.
66
-- BGP profile projection is profile-gated: `snmp:bgp-peers` is exposed only for BGP-capable jobs/devices.
67
-- Normal ddsnmp chart-per-row behavior is preserved; no BGP-specific peer/peer-family chart caps are introduced.
68
-- Per-peer/per-family BGP health templates remain default-on.
69
-- Stale cache semantics are explicit: function output distinguishes fresh, stale-within-TTL, stale-expired, no-data-yet, and valid filtered-empty states; chart-side collection freshness is exposed.
70
-- Cisco BGP is removed from `_cisco-base.yaml` and attached only to audited router/L3-switch Cisco profiles.
71
-- Cumulus and Alcatel BGP capability claims are removed until real fixtures exist.
72
-- Each migrated vendor has production-shaped full-profile tests through normal profile loading/resolution/finalization.
73
-- `.agents/sow/specs/snmp-profile-projection.md` documents the BGP projection contract.
74
-- Metadata/docs/generated integration artifacts are coherent after final migration.
75
-- Typed-domain SNMP profile sections are organized as dedicated fragments where practical; concrete device profiles extend those fragments instead of mixing typed-domain sections with regular metric sections.
76
-- Local raw MIB files under `mibs/` are not committed and are removed before SOW close-out.
77
-
78
-## Analysis
79
-
80
-Sources checked:
81
-
82
-- `src/go/plugin/go.d/TODO-snmp-bgp-monitoring-review.md`
83
-- `collector/snmp/bgp_public.go`
84
-- `collector/snmp/bgp_public_routes.go`
85
-- `collector/snmp/bgp_public_specs.go`
86
-- `collector/snmp/bgp_integration.go`
87
-- `collector/snmp/collect_snmp.go`
88
-- `collector/snmp/charts.go`
89
-- `collector/snmp/metric_ids.go`
90
-- `collector/snmp/func_bgp_peers.go`
91
-- `collector/snmp/func_bgp_peers_cache.go`
92
-- `collector/snmp/ddsnmp/ddprofiledefinition/profile_definition.go`
93
-- `collector/snmp/ddsnmp/metric.go`
94
-- `collector/snmp/metadata.yaml`
95
-- `collector/snmp/integrations/snmp_devices.md`
96
-- `src/health/health.d/snmp_bgp.conf`
97
-- `config/go.d/snmp.profiles/default/*bgp*.yaml`
98
-- `config/go.d/snmp.profiles/default/cisco*.yaml`
99
-- `config/go.d/snmp.profiles/default/_cisco*.yaml`
100
-- `.agents/sow/specs/snmp-profile-projection.md`
101
-- `.agents/skills/project-snmp-profiles-authoring/SKILL.md`
102
-- `.agents/skills/project-writing-collectors/SKILL.md`
103
-- `.agents/skills/integrations-lifecycle/SKILL.md`
104
-- Local untracked BGP MIB files under `mibs/`
105
-
106
-Current state:
107
-
108
-- `profile_definition.go` supports `metrics`, `topology`, and `licensing`, but no `bgp`.
109
-- `ProfileMetrics` supports regular `Metrics`, `TopologyMetrics`, and `LicenseRows`, but no typed BGP rows.
110
-- `bgp_public_routes.go` routes raw vendor metric names through a large switch and decides peer vs peer-family scope from row tags.
111
-- `bgp_public_specs.go` separately defines public BGP chart specs.
112
-- `func_bgp_peers_cache.go` keeps another switch-like mapping from public BGP leaf names into function cache fields.
113
-- `metric_ids.go` derives BGP chart IDs and contexts through `strings.HasPrefix(name, "bgp.")`.
114
-- `collect_snmp.go` and `charts.go` show the generic ddsnmp chart-per-row behavior that BGP must preserve.
115
-- `src/health/health.d/snmp_bgp.conf` defines 7 default-on BGP health templates.
116
-- `collector/snmp/metadata.yaml` lists 7 BGP alerts and includes Cumulus/Alcatel capability claims that lack real fixture backing.
117
-
118
-Risks:
119
-
120
-- Regular SNMP metrics can regress if BGP migration changes generic profile projection or charting paths.
121
-- BGP function discovery can remain noisy unless `funcRouter` registration becomes profile-aware.
122
-- BGP charts and function output can freeze on stale SNMP data unless freshness is modeled for both chart and function surfaces.
123
-- Vendor profile migration can break real device coverage if MIB index accessibility or Cisco profile attachment scope is guessed instead of audited.
124
-- Integration artifacts can drift unless metadata, generated docs, health alerts, config, and docs are closed together.
125
-- Raw local MIBs and fixture provenance must not leak into committed artifacts.
126
-
127
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
128
-
129
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Validation and Outcome sections).
130
-
131
-Problem / root-cause model:
132
-
133
-- BGP monitoring is currently implemented as a post-collection public-metric rewrite. The code takes ordinary SNMP metrics, routes by name, infers row kind from tag presence, carries hidden label fields through underscore-prefixed tags, and then builds chart/function surfaces from those derived strings. This repeats the hidden-protocol architecture already removed from SNMP topology and licensing. BGP needs the same typed projection treatment because its row kinds, identities, states, AFI/SAFI values, and signals are domain data, not chart naming accidents.
134
-
135
-Evidence reviewed:
136
-
137
-- `collector/snmp/bgp_public_routes.go` contains the metric-name routing layer and peer/peer-family inference.
138
-- `collector/snmp/bgp_public_specs.go` contains separate public chart specs that must stay in sync with routes.
139
-- `collector/snmp/func_bgp_peers_cache.go` contains a separate BGP leaf-to-function-field mapping.
140
-- `collector/snmp/bgp_public_routes.go` and `func_bgp_peers_cache.go` carry underscore-prefixed hidden label handling.
141
-- `collector/snmp/metric_ids.go` uses `strings.HasPrefix(name, "bgp.")` as a chart context contract.
142
-- `collector/snmp/ddsnmp/ddprofiledefinition/profile_definition.go` and `collector/snmp/ddsnmp/metric.go` show typed topology/licensing projections but no BGP projection.
143
-- `collect_snmp.go` and `charts.go` show existing ddsnmp chart-per-row behavior.
144
-- `src/health/health.d/snmp_bgp.conf` shows 7 default BGP alert templates.
145
-- Local untracked MIBs show `MAX-ACCESS not-accessible` peer index objects that must not be configured as readable symbols.
146
-
147
-Affected contracts and surfaces:
148
-
149
-- SNMP profile schema and validation.
150
-- ddsnmp catalog projection and profile consumers.
151
-- ddsnmpcollector typed output and stats.
152
-- SNMP collector BGP charts, functions, cache, stale semantics, and health alert inputs.
153
-- Default SNMP profile YAMLs for standard BGP4-MIB, Cisco, Juniper, Nokia, Huawei, Arista, Dell, Cumulus, Alcatel, and related mixins.
154
-- SNMP metadata and generated integration artifacts.
155
-- `src/health/health.d/snmp_bgp.conf`.
156
-- SNMP profile projection spec.
157
-- Runtime SNMP profile authoring skill if authoring rules change.
158
-
159
-Existing patterns to reuse:
160
-
161
-- Top-level typed `topology:` and `licensing:` profile sections.
162
-- `ResolvedProfileSet.Project(metrics, licensing)` variadic projection pattern.
163
-- Licensing row structural identity and inheritance override validation patterns.
164
-- Table/index `MAX-ACCESS not-accessible` discipline from `project-snmp-profiles-authoring`.
165
-- Existing ddsnmp scalar/table collection, table cache, stats, chart construction, and row reaping paths.
166
-- Table-driven Go tests using `map[string]struct{}` for shared setup/assertion shapes.
167
-
168
-Risk and blast radius:
169
-
170
-- The code touches a broad collector and many vendor SNMP profiles. Keep the first slice schema/API-only where possible.
171
-- Backward compatibility with this WIP BGP implementation is not required, but ordinary SNMP metrics must not regress.
172
-- Vendor profile migrations must be small enough to validate independently.
173
-- Function registration lifecycle changes may affect other SNMP functions if done at the shared router layer.
174
-- Health alerts remain default-on; chart context and dimension names must be statically reviewed until/unless a project alert test framework exists.
175
-- Removing Cumulus/Alcatel claims can reduce advertised coverage, but it removes unsupported public debt.
176
-
177
-Sensitive data handling plan:
178
-
179
-- Do not commit local raw MIB files under `mibs/`.
180
-- Do not commit SNMP communities, SNMPv3 credentials, bearer tokens, customer hostnames, customer sysName/sysDescr, customer IPs, customer names, personal data, or private endpoints.
181
-- Any fixture provenance must use public repository/commit/relative-path evidence or sanitized text, never workstation-local paths.
182
-- SOW/spec/docs/code comments must cite MIB facts by object name, table, and relative local evidence only while MIB files remain untracked; raw MIB text is not copied into durable artifacts.
183
-- Remove local `mibs/` files before SOW close-out.
184
-
185
-Implementation plan:
186
-
187
-1. Schema/API scaffold:
188
- - Add typed BGP profile config structs, closed enums, validation helpers, identity helpers, and `ProfileMetrics.BGPRows`.
189
- - Add `ProfileConsumerBGP` projection and projection tests.
190
- - Update `.agents/sow/specs/snmp-profile-projection.md` with the BGP projection contract.
191
-2. Typed producer scaffold:
192
- - Add BGP row collection/projection in ddsnmpcollector without changing vendor profiles yet.
193
- - Add full-profile/synthetic tests for the standard BGP shape.
194
-3. Runtime consumer/function/charts:
195
- - Feed typed rows into BGP charts/function cache.
196
- - Preserve chart-per-row behavior.
197
- - Implement stale cache and chart freshness semantics.
198
- - Make BGP function registration profile-aware.
199
-4. Vendor migrations:
200
- - Migrate one vendor/profile family at a time, with full-profile tests.
201
- - Delete that vendor's old `bgp_public*` routes/spec/cache handling in the same slice.
202
- - Audit Cisco profile attachment before Cisco migration.
203
- - Remove Cumulus/Alcatel claims unless real fixtures are available.
204
-5. Final cleanup:
205
- - Delete remaining `bgp_public*` router, underscore-prefix BGP protocol, and `bgp.` string-prefix context heuristics.
206
- - Regenerate metadata/integration artifacts.
207
- - Remove local untracked MIB files before close-out.
208
-
209
-Validation plan:
210
-
211
-- Schema validation tests for row kind, identity, state mappings, AFI/SAFI enums, MIB accessibility, and invalid signal/source combinations.
212
-- Projection tests for `Project(metrics)`, `Project(bgp)`, and mixed `Project(metrics, bgp)`.
213
-- Full-profile smoke tests per migrated vendor through normal profile loading/resolution/finalization.
214
-- Fixture-backed typed BGP row tests for each migrated vendor.
215
-- Function tests for no data, fresh rows, filtered-empty rows, stale-within-TTL rows, and stale-expired rows.
216
-- Chart tests for chart-per-row identity stability and stale/freshness signals.
217
-- Static/manual review of `src/health/health.d/snmp_bgp.conf` contexts and dimensions against typed chart output.
218
-- Same-failure searches for leftover `bgp_public`, underscore-prefixed BGP protocol, `strings.HasPrefix(name, "bgp.")`, Cumulus/Alcatel unsupported claims, workstation paths, and raw MIB files in git status.
219
-- Narrow Go test suites:
220
- - `go test -count=1 ./collector/snmp/ddsnmp/...`
221
- - `go test -count=1 ./collector/snmp/...`
222
-
223
-Artifact impact plan:
224
-
225
-- AGENTS.md: no expected update; existing SOW, collector, git, and SNMP profile authoring rules already apply.
226
-- Runtime project skills: update `.agents/skills/project-snmp-profiles-authoring/SKILL.md` if typed BGP authoring adds durable MIB/index guidance beyond the existing rule.
227
-- Specs: update `.agents/sow/specs/snmp-profile-projection.md` with BGP consumer/projection/delivery/validation rules.
228
-- End-user/operator docs: update `collector/snmp/metadata.yaml`, generated integration artifacts, and any BGP function documentation affected by output/config changes.
229
-- End-user/operator skills: no expected update unless public Netdata AI skills reference SNMP BGP schema or function behavior.
230
-- SOW lifecycle: move to `current/in-progress` before implementation; close only after validation, artifact gates, local MIB cleanup, and follow-up mapping are complete.
231
-
232
-Open-source reference evidence:
233
-
234
-- No local mirrored open-source repositories were checked for this gate. The actionable evidence is in current branch code, project specs/skills, local untracked vendor MIB files, and review outputs.
235
-
236
-Open decisions:
237
-
238
-- None. User decisions 1-23 are resolved in `## Implications And Decisions`.
239
-
240
-## Implications And Decisions
241
-
242
-1. **BGP architecture: A.**
243
- - Use typed `bgp:` projection and typed `ProfileMetrics.BGPRows`.
244
-2. **BGP function and integration exposure: A.**
245
- - Expose BGP only for profiles/jobs that declare typed BGP rows; restructure `funcRouter` lifecycle as needed.
246
-3. **Chart cardinality: A.**
247
- - Keep existing ddsnmp chart-per-row behavior; no BGP-specific chart caps.
248
-4. **Stale cache semantics: A.**
249
- - Implement stale-aware function behavior and chart-side freshness.
250
-5. **Per-peer alerts: A.**
251
- - Keep per-peer/per-family BGP health templates default-on.
252
-6. **Cisco BGP attachment: A.**
253
- - Move BGP out of `_cisco-base.yaml`; attach only to audited router/L3-switch profiles.
254
-7. **Unproven Cumulus/Alcatel coverage: C.**
255
- - Remove claims until real fixtures exist.
256
-8. **Structural identity: A.**
257
- - Use typed structural identity with canonical length-prefixed key.
258
-9. **AFI/SAFI policy: A.**
259
- - Use closed canonical enums with explicit vendor-private allow-list.
260
-10. **MIB accessibility: A.**
261
- - Enforce `MAX-ACCESS` accessibility; `not-accessible` fields come from indexes.
262
-11. **Function lifecycle: A.**
263
- - Make function registration profile-aware.
264
-12. **Cap overflow: not applicable.**
265
- - Decision 3A removes BGP-specific caps.
266
-13. **Production-shaped tests: A.**
267
- - Require full-profile tests per migrated vendor.
268
-14. **Migration/artifact gate: A.**
269
- - Use schema-first typed migration and delete old router rows/specs per vendor.
270
- - Implementation constraint: standard BGP4 metric-name routes are shared by still-unmigrated vendor profiles. The standard profile migration removes its old `metrics:`/`virtual_metrics:` definitions to prevent duplicate output, but shared `bgp_public*` cases remain until the last dependent vendor is migrated.
271
-15. **State mapping completeness: A.**
272
- - Require all six RFC 4271 states unless explicitly partial.
273
-16. **Underscore protocol: A.**
274
- - Delete underscore-prefix BGP protocol with `bgp_public*`.
275
-17. **Function cache representation: A.**
276
- - Make `BGPRow` the canonical snapshot/cache representation.
277
-18. **Health alert testing: B.**
278
- - Do not build an alert test framework; validate by static/manual review.
279
-19. **Projection spec: A.**
280
- - Update `.agents/sow/specs/snmp-profile-projection.md` with BGP.
281
-20. **BGP signal payload / row field model: B.**
282
- - Runtime planning exposed that `Signals map[BGPSignalKind]BGPValue` is not a clean end-state row model.
283
- - Claude review accepted Option B and flagged a current scaffold bug: categorical signals such as `last_down_reason`, `graceful_restart_state`, and `unavailability_reason` are routed through numeric conversion.
284
- - Replace generic `Signals` with typed row/config fields/groups before runtime consumer/function/chart work.
285
-21. **Typed BGP cross-table value sources: A.**
286
- - Cisco peer-family rows in `cbgpPeer2AddrFamilyPrefixTable` need `remote_as`, local address, local AS, identifiers, and BGP version from `cbgpPeer2Table`.
287
- - Current raw Cisco YAML gets these with cross-table `metric_tags`; typed `BGPValueConfig` has no equivalent first-class source.
288
- - Add first-class cross-table support to `BGPValueConfig`, mirroring `MetricTagConfig.table` + `index_transform`, so typed identity/descriptors can stay schema-owned.
289
-22. **Variable-length BGP index tail extraction: A.**
290
- - Cisco peer-family AFI/SAFI fields follow variable-length `InetAddress` in the row index, so fixed absolute positions work only for IPv4 rows.
291
- - Add reusable tail-based index extraction for typed BGP values and use it for Cisco AFI/SAFI.
292
-23. **Catalyst 9k BGP attachment scope: A.**
293
- - `cisco-catalyst.yaml` is currently BGP-free but covers Catalyst families that can run BGP.
294
- - Attach typed Cisco BGP broadly to `cisco-catalyst.yaml`.
295
- - Accepted trade-off: unsupported BGP table walk cost on non-BGP Catalyst devices is temporary and expected to be mitigated by a generalized unsupported table-root cache follow-up.
296
- - Follow-up scope: `.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md` has been expanded from licensing-only to typed licensing + typed BGP unsupported table-root caching.
297
-
298
-## Plan
299
-
300
-1. Move this SOW to `current/in-progress` before code changes.
301
-2. Land schema/API scaffold and tests:
302
- - BGP config structs and enums.
303
- - Validation helpers.
304
- - Projection plumbing.
305
- - `ProfileMetrics.BGPRows`.
306
- - Spec update.
307
-3. Land typed producer scaffold with a minimal standard BGP fixture path.
308
-4. Land typed runtime consumer/function/chart path:
309
- - First replace generic `BGPRow.Signals` with strongly typed BGP row fields/groups if Decision 20 is resolved as 20B.
310
- - Preserve chart-per-row behavior.
311
- - Add stale/freshness semantics.
312
- - Make BGP function profile-gated.
313
-5. Audit Cisco profile attachment and MIB accessibility before vendor migrations.
314
-6. Migrate standard BGP4-MIB.
315
-7. Migrate Cisco and remove BGP from `_cisco-base.yaml`.
316
-8. Migrate Juniper.
317
-9. Migrate Nokia.
318
-10. Migrate Huawei.
319
-11. Migrate Arista and Dell.
320
-12. Remove Cumulus/Alcatel public claims unless real fixtures are added.
321
-13. Delete remaining old BGP router/protocol code and string-prefix context gates.
322
-14. Regenerate metadata/integration artifacts and validate generated outputs.
323
-15. Run same-failure and sensitive-data scans.
324
-16. Remove local untracked `mibs/` files before close-out.
325
-17. Complete validation gate, move SOW to `done/`, and commit lifecycle change with implementation.
326
-18. Normalize remaining inline typed-domain profile sections into dedicated fragments before close-out.
327
-
328
-## Execution Log
329
-
330
-### 2026-05-08
331
-
332
-- Created pending SOW from review TODO after user resolved decisions 1-19.
333
-- Moved SOW to current/in-progress before code changes.
334
-- Landed schema/API/projection scaffold:
335
- - `ddprofiledefinition.BGPConfig`, BGP row kinds, peer-state enums, AFI/SAFI enums, and typed BGP row field groups.
336
- - `ProfileDefinition.BGP`, `ProfileMetrics.BGPRows`, and typed BGP row output structs.
337
- - `ProfileConsumerBGP` projection support, mixed `Project(metrics, bgp)` behavior, inheritance merge, and cross-profile dedup scaffolding.
338
- - BGP schema validation tests and projection tests.
339
- - Initial BGP projection contract in `.agents/sow/specs/snmp-profile-projection.md`.
340
-- Landed typed producer scaffold:
341
- - `ddsnmpcollector.collectBGPRows` supports scalar and table typed BGP rows.
342
- - BGP rows are collected best-effort so BGP processing errors do not drop regular SNMP metrics.
343
- - BGP scalar row collection honors the shared missing-OID cache.
344
- - BGP table row collection uses the existing table cache and table dependency walk path.
345
- - Main SNMP profile projection includes `ConsumerBGP` with metrics and licensing.
346
- - Internal profile stats now include BGP timing, row count, and processing error dimensions.
347
- - Added scalar, table, best-effort, and missing-OID cache tests for typed BGP rows.
348
-- Reworked the BGP scaffold per Decision 20B:
349
- - Removed the generic `Signals map[BGPSignalKind]BGPValue` row model.
350
- - Added typed BGP row/config groups for admin, state, previous state, connection, traffic, transitions, timers, last-error, last-notifications, reasons, graceful restart, routes, route limits, and device counts.
351
- - Added categorical field coverage so text/mapped fields such as last-down reason are not routed through numeric conversion.
352
- - Updated BGP validation and projection spec wording to describe typed fields rather than signal kinds.
353
-- Started typed runtime consumer wiring:
354
- - Added typed BGP row to public chart-metric projection for existing BGP chart contexts.
355
- - Updated the BGP function cache to accept typed `BGPRow` snapshots directly, preserving descriptor fields without underscore-prefixed tag routing.
356
- - Added tests for typed BGP chart metrics and typed-row function-cache output.
357
-- Recorded the Decision 14 implementation constraint:
358
- - `_std-bgp4-mib.yaml` is extended by multiple still-unmigrated vendor profiles, so shared `bgp_public*` standard BGP4 route/spec/cache cases cannot be deleted until the last dependent vendor profile is migrated.
359
- - The old standard profile `metrics:`/`virtual_metrics:` definitions still need to be removed during the standard migration to avoid double-emission.
360
-- Migrated the standard BGP4-MIB profile to typed BGP rows:
361
- - `_std-bgp4-mib.yaml` now declares `bgpPeerTable` as typed `bgp:` row `bgp4-peer`.
362
- - Removed old standard `metrics:` and `virtual_metrics:` from `_std-bgp4-mib.yaml`.
363
- - Removed Nokia fallback alternatives that pointed at the old standard BGP4 virtual metrics.
364
- - Kept shared `bgp_public*` standard BGP4 route/spec/cache cases temporarily because unmigrated vendor profiles still depend on them.
365
- - Set BGP `OriginProfileID` during profile load, matching licensing origin behavior.
366
- - Updated standard BGP4 and Cumulus/FRR fixture coverage to assert typed `BGPRows`, while keeping the legacy alert-surface test scoped to still-unmigrated vendor raw paths.
367
-- Landed typed BGP runtime lifecycle and stale-cache handling:
368
- - `Collector` no longer allocates/registers BGP integration by default.
369
- - BGP integration is enabled only for resolved profiles that contain typed `bgp:` rows or temporary legacy BGP public-router inputs.
370
- - BGP collection failures mark the function cache stale instead of leaving stale rows indistinguishable from fresh rows.
371
- - `snmp:bgp-peers` returns stale rows during the bounded stale window, returns 503 after the stale window expires, and returns a 200 empty table for valid filtered-empty views.
372
- - Function row building now snapshots the cache and releases the cache lock before sorting/building response rows.
373
- - Framework boundary recorded: the static module method list still advertises `snmp:bgp-peers` globally. Fully hiding the method from global discovery for non-BGP jobs requires a broader `funcctl`/collectorapi capability change; the SNMP handler itself is profile-gated.
374
-- Landed typed cross-table BGP value sources and the Cisco typed migration slice:
375
- - `BGPValueConfig` now supports first-class `table:` value sources with `index_transform`, mirroring the existing cross-table tag pattern without making typed identity depend on labels.
376
- - `_cisco-bgp4-mib.yaml` now declares typed peer and peer-family BGP rows instead of raw `metrics:`/`virtual_metrics:`.
377
- - Cisco BGP was removed from `_cisco-base.yaml`.
378
- - Cisco BGP is attached only to audited router/L3/data-center profiles: `cisco-asr.yaml`, `cisco-csr1000v.yaml`, `cisco-isr.yaml`, `cisco-isr-4431.yaml`, `cisco-3850.yaml`, `cisco-nexus.yaml`, and the new narrow `cisco-ncs.yaml` profile for the NCS 540 fixture.
379
- - Generic `cisco.yaml` remains BGP-free by design.
380
- - Added `BGPValueConfig.index_from_end` and corrected Cisco peer-family AFI/SAFI extraction to read the final two `cbgpPeer2AddrFamilyPrefixTable` row-index components, which works for both IPv4 and IPv6 `InetAddress` indexes.
381
-- Resolved the Claude slice-review commit blockers after Decision 22A:
382
- - Protected `funcRouter` runtime handler registration with a router `sync.RWMutex`.
383
- - Surfaced per-profile typed BGP collection failures through `ProfileMetrics.BGPCollectError`.
384
- - BGP integration now marks the function cache stale and skips peer-cache reset/finalize when a profile-level BGP failure occurs inside an otherwise successful SNMP collection cycle.
385
- - Added focused regression tests for concurrent router registration, per-profile BGP failure stale-cache preservation, Cisco IPv6 peer-family AFI/SAFI extraction, and generic Cisco profile BGP non-inheritance.
386
-- Reconciled the Claude post-Decision 22A review:
387
- - Accepted RT2-1: the first stale-cache fix was all-or-nothing across profiles. Updated BGP integration/cache handling so failed sources preserve stale rows while successful sources refresh normally in the same cycle.
388
- - Added mixed-profile failure regression coverage, including the case where an expired failed source must not be kept alive by another source refreshing successfully.
389
- - Accepted MIB2-2 and normalized empty typed BGP routing instances to `default` for chart/function tags.
390
- - Recorded Catalyst BGP attachment as Decision 23 instead of silently attaching or silently dropping it.
391
-- Applied Decision 23A:
392
- - `cisco-catalyst.yaml` now extends `_cisco-bgp4-mib.yaml`.
393
- - Added a Catalyst 9300 profile-merge regression test proving typed Cisco BGP rows are inherited.
394
- - Expanded SOW-0014 from licensing-only to typed licensing + typed BGP unsupported table-root caching, because broad Catalyst BGP has the same unsupported-table probe trade-off as broad Cisco licensing.
395
-- Reconciled the Claude final pre-commit review:
396
- - Included `config/go.d/snmp.profiles/default/cisco-ncs.yaml` in the slice because this SOW claims the new NCS profile.
397
- - Added repo-root `/mibs/` to `.gitignore` so local raw MIB audit files remain uncommitted.
398
- - Accepted remaining schema/spec/runtime hygiene findings as SOW-close items, not commit blockers for this slice.
399
-- Migrated Juniper BGP to typed rows:
400
- - `_juniper-bgp4-v2.yaml` now declares typed peer and peer-family BGP rows instead of raw `metrics:`/`virtual_metrics:`.
401
- - Juniper base/router profiles no longer extend `_juniper-bgp-virtual.yaml`; the old virtual-metric path is no longer referenced.
402
- - `BGPValueConfig.lookup_symbol` now supports typed BGP cross-table value lookups, needed because Juniper prefix counters are keyed by peer ID while peer identity lives in `jnxBgpM2PeerTable`.
403
- - Empty successful BGP dependency walks are cached as empty walked tables, so optional augmented tables do not drop otherwise valid peer rows.
404
- - Runtime BGP rows are skipped when required identity fields are missing at collection time.
405
- - Added BGP projection coverage to `collector/snmp/ddsnmp/profile_test.go`.
406
- - Reworked Juniper fixture tests to assert typed `BGPRows` from BGP-projected profiles and refactored similar test cases to `map[string]struct{}` tables.
407
-- Migrated Nokia/TiMOS BGP to typed rows:
408
- - `_nokia-timetra-bgp.yaml` now declares typed peer and six typed peer-family BGP rows instead of raw `metrics:`/`virtual_metrics:`.
409
- - Nokia no longer inherits `_std-bgp4-mib.yaml`; SR OS BGP rows come from TIMETRA-BGP-MIB only.
410
- - Peer address is derived from the `tBgpPeerNgTable`/`tBgpPeerNgOperTable` row index because the MIB index objects are `MAX-ACCESS not-accessible`.
411
- - Deleted unused `_juniper-bgp-virtual.yaml` after the Juniper migration removed all references.
412
- - Updated BGP structural identity to include typed config ID so multiple logical typed rows over the same table/index do not collide.
413
- - Updated Nokia profile, LibreNMS identity, and TiMOS fixture coverage to assert typed `BGPRows`.
414
- - Added Nokia BGP projection coverage to `collector/snmp/ddsnmp/profile_test.go`.
415
- - Reviewed new BGP tests for the branch's table-driven style rule and refactored same-shape Cisco prefix, Arista/Dell fixture, and TiMOS fixture tests to `map[string]struct{}` or equivalent map-keyed tables.
416
-- Reconciled the Claude Nokia/TiMOS commit review:
417
- - Accepted the review verdict as ready with follow-ups.
418
- - Added Nokia profile assertions for admin and six-state peer-state mappings.
419
- - Added synthetic Nokia typed-collection coverage for admin-disabled, non-established states, and non-default VRF routing-instance extraction.
420
- - Added broader Nokia profile assertions for descriptor, timer, traffic, notification, transition, and last-error fields.
421
- - Updated `.agents/sow/specs/snmp-profile-projection.md` so BGP table structural identity includes typed config ID.
422
- - Recorded the inherited TIMETRA high-numbered oper OID verification gap as a SOW follow-up.
423
-- Migrated Huawei BGP to typed rows:
424
- - `huawei-routers.yaml` now declares typed BGP device, peer, and peer-family rows instead of raw BGP `metrics:`/`virtual_metrics:`.
425
- - Removed `_huawei-bgp-statistics.yaml`; Huawei BGP session counts now flow through typed `device_counts`.
426
- - Extended typed BGP device counts with `ibgp_peers` and `ebgp_peers` so Huawei preserves the public `configured`/`ibgp`/`ebgp` peer-count dimensions without the legacy public router.
427
- - Removed Huawei-specific cases from the temporary `bgp_public*` metric-name router.
428
- - Added Huawei BGP projection coverage to `collector/snmp/ddsnmp/profile_test.go`.
429
- - Reworked Huawei LibreNMS fixture coverage to assert typed `BGPRows` directly, including IPv4/IPv6 peer-family rows, route totals, peer-level message/update counters, and device peer counts.
430
- - Added synthetic Huawei typed-collection coverage for state/admin mapping and peer-statistics remote-AS lookup by peer address.
431
- - Local raw MIB audit gap: `mibs/` contains `HUAWEI-MPLS-BGP-VPN-MIB.mib`, not the exact `HUAWEI-BGP-VPN-MIB`; profile OIDs are carried forward from the existing Huawei profile and the linked public Huawei BGP MIB references.
432
-
433
-### 2026-05-09
434
-
435
-- Migrated the remaining BGP-bearing vendor profiles and deleted the legacy BGP public router:
436
- - Arista, Dell OS10, and Alcatel now declare typed `bgp:` rows.
437
- - All stock BGP coverage is typed: standard `BGP4-MIB`, Cisco, Juniper, Nokia SR OS, Huawei, Arista, Dell OS10, and Alcatel.
438
- - `bgp_public.go`, `bgp_public_routes.go`, and `bgp_public_test.go` were deleted.
439
- - The retained chart-filter and chart-spec behavior now lives in `bgp_chart_filter.go` and `bgp_metric_specs.go`.
440
- - The BGP function cache consumes typed `BGPRow` updates only.
441
-- Reconciled final migration review blockers:
442
- - Corrected inherited Alcatel IPv6 BGP table OIDs from `.12` to `.14` according to `ALCATEL-IND1-BGP-MIB`.
443
- - Restored end-to-end collection-level assertions for BGP metric IDs and chart context.
444
- - Removed dead BGP alert-surface test helpers and moved the remaining shared profile helper.
445
- - Reviewed and refactored applicable BGP tests to map-keyed table-driven cases.
446
-- Resolved AC#69:
447
- - Updated `collector/snmp/metadata.yaml` to remove Cumulus and Alcatel from public BGP support claims until real fixture evidence exists.
448
- - Regenerated integration artifacts with `integrations/gen_integrations.py`, `integrations/gen_docs_integrations.py -c go.d.plugin/snmp`, `integrations/gen_doc_collector_page.py`, and `integrations/gen_doc_secrets_page.py`.
449
- - Committed generated output changed in `collector/snmp/integrations/snmp_devices.md`.
450
- - Gitignored generated `integrations/integrations.js` and `integrations/integrations.json` were regenerated locally.
451
- - `src/collectors/COLLECTORS.md` and `src/collectors/SECRETS.md` had no tracked diff after generation.
452
-- Updated durable authoring and behavior references:
453
- - `collector/snmp/profile-format.md` now documents typed BGP `partial_states`, routing-instance default labeling, and `index_from_end` selector behavior.
454
- - `.agents/sow/specs/snmp-profile-projection.md` now records typed BGP structural identity, default routing-instance normalization, per-source stale-cache semantics, and row-index selector validation behavior.
455
- - `.agents/skills/project-snmp-profiles-authoring/SKILL.md` now records the BGP authoring checks for typed `bgp:` rows, RFC 4271 state mappings, and index-derived BGP fields.
456
-- Created `.agents/sow/pending/SOW-0016-20260509-snmp-bgp-hardening-followups.md` for non-blocking validation/runtime/fixture hardening that should not be hidden as vague follow-up debt.
457
-- Applied the user's typed-domain profile organization decision:
458
- - Extracted inline Arista, Dell OS10, and Huawei `bgp:` sections into `_arista-bgp4-v2.yaml`, `_dell-os10-bgp4-v2.yaml`, and `_huawei-bgp-vpn.yaml`.
459
- - Extracted inline Blue Coat, Check Point, Fortinet FortiGate, MikroTik RouterOS, and Sophos XGS `licensing:` sections into `_bluecoat-proxysg-licensing.yaml`, `_checkpoint-licensing.yaml`, `_fortinet-fortigate-licensing.yaml`, `_mikrotik-routeros-licensing.yaml`, and `_sophos-xgs-firewall-licensing.yaml`.
460
- - Concrete device profiles now extend the typed fragments and no longer mix inline typed-domain sections with regular metric sections.
461
- - Updated Arista/Dell typed BGP fixture assertions to expect the new fragment origin profile IDs.
462
-- Folded the pending SOW-0016 hardening scope back into SOW-0015 by user decision:
463
- - Deleted the separate pending SOW-0016 file after migrating its remaining work into this SOW's follow-up mapping.
464
- - Reclassified RT-4 as an intentional chart-identity invariant: public BGP chart keys follow the same logical SNMP table identity model as regular table metrics (`name` plus visible tag values), while the BGP function cache keeps structural row identity separately.
465
- - Added a focused runtime test proving same logical BGP peers from different typed rows resolve to the same public chart identity and do not include source or structural IDs in the chart key.
466
- - Kept remaining hardening work in this SOW instead of tracking it through a separate pending SOW.
467
-- Completed the folded SOW-0016 hardening items selected by the user:
468
- - Added typed BGP validation for mutually-exclusive row-index selectors (`index`, `index_from_end`, `index_transform`).
469
- - Added conservative typed BGP cross-table validation for declared referenced BGP tables, including source and lookup-symbol OID prefix checks.
470
- - Added stale BGP function-cache entry reaping after the stale window expires.
471
- - Added Dell OS10 IPv6 typed BGP synthetic-PDU coverage.
472
- - Added `index_from_end` plus `format` propagation coverage.
473
- - Extended the logical chart-identity test to verify descriptor labels remain visible labels while public chart keys omit source and structural identity.
474
-- Reconciled final external review findings:
475
- - Final review verdict was `READY TO MERGE`; no P0/P1 blockers were reported.
476
- - Added remaining row-index selector combination negative tests.
477
- - Added BGP row kind vs field-group compatibility negative tests.
478
- - Extended Dell OS10 IPv6 coverage with a dependency-table traffic counter assertion.
479
- - Replaced local-only Nokia/TiMOS MIB closure evidence with reproducible source references.
480
-
481
-## Validation
482
-
483
-Acceptance criteria evidence:
484
-
485
-- Met for the implemented migration:
486
- - Typed `bgp:` profile section exists in `ddprofiledefinition.ProfileDefinition`.
487
- - Typed BGP output exists in `ddsnmp.ProfileMetrics.BGPRows`.
488
- - `Project(bgp)` and `Project(metrics, bgp)` projection paths exist and are tested.
489
- - Closed row kind, peer state, AFI/SAFI, and typed-field validation exists.
490
- - `ddsnmpcollector` emits typed BGP rows for scalar and table `bgp:` configs.
491
- - BGP collection is wired into the regular SNMP profile collection path and keeps regular metrics on BGP row errors.
492
- - BGP profile stats are exposed alongside scalar/table/licensing/internal stats.
493
- - Typed BGP rows project into BGP chart metrics and the BGP peers function cache.
494
- - All BGP-bearing stock profile coverage is typed: standard `BGP4-MIB`, Cisco, Juniper, Nokia SR OS, Huawei, Arista, Dell OS10, and Alcatel.
495
- - `bgp_public.go`, `bgp_public_routes.go`, and `bgp_public_test.go` are deleted.
496
- - Production runtime scans found no remaining `bgp_public`, `BGPSignalKind`, `bgpScopeAuto`, `mergeBGPPeerEntryTags`, `bgpPeerMetricLeaf`, `normalizeCollectorMetrics`, or `routeBGPPublicMetric` references outside tests/docs.
497
- - Inline typed-domain profile section scan now finds `bgp:` and `licensing:` only in underscore-prefixed typed fragments.
498
- - Legacy BGP identity/router underscore protocol (`_routing_instance`, `_neighbor`, `_remote_as`, `_address_family`, `_subsequent_address_family`) has no production code or profile YAML use. Remaining hits are the generic virtual-metric example in `collector/snmp/profile-format.md`.
499
- - Typed BGP descriptor labels still use the generic SNMP convention of underscore-prefixed chart labels (`_local_address`, `_local_as`, `_peer_identifier`, and related descriptors) in `bgp_typed_metrics.go`; this is not the deleted legacy BGP public-router protocol. `TestTypedBGPMetricsUseLogicalChartIdentity` verifies descriptor labels are visible chart labels and are not part of public chart-key identity.
500
- - Public typed BGP chart keys intentionally use logical SNMP table metric identity, not typed structural row identity. `TestTypedBGPMetricsUseLogicalChartIdentity` covers the invariant that same logical peer tags from different typed rows resolve to the same public chart key.
501
- - Typed BGP profile validation rejects values that combine multiple row-index selectors.
502
- - Typed BGP profile validation now verifies `table:` references to declared BGP row tables when the table root is known at profile-load time. It validates source OIDs and `lookup_symbol` OIDs against the referenced table OID prefix. Runtime-inferred cross-table roots remain supported for existing fragments whose `table:` source is not a declared BGP row table.
503
- - Cisco BGP was removed from `_cisco-base.yaml` and attached to audited router/L3/data-center Cisco profiles, including Catalyst by Decision 23A.
504
- - Cumulus and Alcatel were removed from public BGP capability claims in `metadata.yaml` and regenerated `snmp_devices.md`.
505
- - `.agents/sow/specs/snmp-profile-projection.md` documents the BGP projection contract, including the typed config ID in structural identity.
506
- - Metadata and generated integration docs are coherent for the BGP support list and BGP alert/metric/function surface.
507
-- No open items remain before moving this SOW to `done/`.
508
-
509
-Tests or equivalent validation:
510
-
511
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08.
512
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08.
513
-- `go test -count=1 -run 'TestCollector_Collect_BGPRows|TestCollector_Collect_StatsSnapshot' ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08.
514
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08 after the standard BGP4-MIB typed-profile migration.
515
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the standard BGP4-MIB typed-profile migration.
516
-- `go test -count=1 -run 'TestFuncBGPPeers|TestBGPPeerCache|TestProfilesHaveBGP|TestCollectSNMP_HidesBGPDiagnosticsButKeepsFunctionCache|TestCollector_BGPFunctionHandlerIsRegisteredOnlyWhenEnabled|TestTypedBGPMetricsFromProfileMetrics' ./collector/snmp` passed on 2026-05-08 after typed BGP runtime lifecycle changes.
517
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after typed BGP runtime lifecycle changes.
518
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08 after Decision 21A and the Cisco typed-profile migration.
519
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after Decision 21A and the Cisco typed-profile migration.
520
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
521
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
522
-- `go test -count=1 ./collector/snmp` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
523
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'Test_Cisco'` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
524
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
525
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after Decision 22A and the Claude blocker fixes.
526
-- `go test -race -run TestFuncRouter_ConcurrentRegisterAndHandle ./collector/snmp` passed on 2026-05-08.
527
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08 after the RT2-1 mixed-profile failure fix.
528
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the RT2-1 mixed-profile failure fix.
529
-- `go test -race -run 'TestFuncRouter_ConcurrentRegisterAndHandle|TestBGPIntegration_MixedProfileBGPErrorRefreshesSuccessfulProfiles|TestBGPIntegration_ExpiredFailedProfileDoesNotKeepStaleRowsWithFreshProfiles' ./collector/snmp` passed on 2026-05-08 after the RT2-1 mixed-profile failure fix.
530
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'Test_CiscoBGPProfileMergedIntoCiscoCatalyst|Test_CiscoGenericProfilesDoNotInheritBGP|Test_CiscoBGPProfileMergedIntoCiscoASR|Test_CiscoBgpPrefixProfileMergedIntoCiscoASR'` passed on 2026-05-08 after Decision 23A.
531
-- `go test -count=1 ./collector/snmp/ddsnmp/...` passed on 2026-05-08 after Decision 23A.
532
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after Decision 23A.
533
-- `go test -count=1 ./collector/snmp/ddsnmp ./collector/snmp/ddsnmp/ddprofiledefinition ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08 after the Juniper typed-profile migration and BGP projection test refactor.
534
-- `go test -count=1 ./collector/snmp` passed on 2026-05-08 after the Juniper typed-profile migration and BGP projection test refactor.
535
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the Juniper typed-profile migration and BGP projection test refactor.
536
-- `go test -race -count=1 ./collector/snmp` passed on 2026-05-08 after the Juniper typed-profile migration and BGP projection test refactor.
537
-- `go test -count=1 ./collector/snmp/ddsnmp ./collector/snmp/ddsnmp/ddprofiledefinition ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08 after the Nokia/TiMOS typed-profile migration and test refactor.
538
-- `go test -count=1 ./collector/snmp` passed on 2026-05-08 after the Nokia/TiMOS typed-profile migration and test refactor.
539
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the Nokia/TiMOS typed-profile migration and test refactor.
540
-- `go test -race -count=1 ./collector/snmp` passed on 2026-05-08 after the Nokia/TiMOS typed-profile migration and test refactor.
541
-- `go test -count=1 ./collector/snmp/ddsnmp ./collector/snmp/ddsnmp/ddprofiledefinition ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08 after the Claude Nokia/TiMOS review follow-ups.
542
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the Claude Nokia/TiMOS review follow-ups.
543
-- `go test -race -count=1 ./collector/snmp` passed on 2026-05-08 after the Claude Nokia/TiMOS review follow-ups.
544
-- `go test -count=1 ./collector/snmp/ddsnmp ./collector/snmp/ddsnmp/ddprofiledefinition ./collector/snmp/ddsnmp/ddsnmpcollector` passed on 2026-05-08 after the Huawei typed-profile migration.
545
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-08 after the Huawei typed-profile migration.
546
-- `go test -race -count=1 ./collector/snmp` passed on 2026-05-08 after the Huawei typed-profile migration.
547
-- `go test -count=1 ./collector/snmp -run 'Test(BGPLastErrorText|BGPPeerEntryKey|BGPPeerCache|FuncBGPPeers|CollectSNMP_HidesBGP)'` passed on 2026-05-09 after the BGP table-driven test refactor.
548
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'Test_(LibreNMSBGPIdentityFixtures|CiscoBGPProfiles|CiscoGeneric|AlcatelBGPProfile|HuaweiBGP|JuniperBGP|NokiaBGP)'` passed on 2026-05-09 after the BGP table-driven test refactor.
549
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_(GenericBGP4Rows|HuaweiBGP|AlcatelBGP|AristaAndDellBGP|CiscoBgpPeer2|CiscoBgpPeer3|JuniperBGP|TiMOSBGP)'` passed on 2026-05-09 after the BGP table-driven test refactor.
550
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after the BGP table-driven test refactor.
551
-- `git diff --check` passed on 2026-05-09 after the BGP table-driven test refactor.
552
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after metadata/docs/spec/skill close-out updates and local integration docs regeneration.
553
-- `git diff --check` passed on 2026-05-09 after metadata/docs/spec/skill close-out updates and local integration docs regeneration.
554
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'Test_(AristaAndDellBGPProfilesUseTypedRows|HuaweiBGPProfileUsesTypedRows|ProfilesProjectConsumers|NokiaBGPProfile|CiscoBGPProfiles|JuniperBGPProfiles)|TestProfile_Merge'` passed on 2026-05-09 after typed-domain profile fragment extraction.
555
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_(LicensingProfiles|LicensingProfileFixtures|SophosLicensingProfile|FortiGate|BlueCoat|HuaweiBGP|AlcatelBGP|AristaAndDellBGP|TiMOSBGP|GenericBGP4Rows)'` passed on 2026-05-09 after typed-domain profile fragment extraction.
556
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after typed-domain profile fragment extraction.
557
-- `git diff --check` passed on 2026-05-09 after typed-domain profile fragment extraction.
558
-- `go test -count=1 ./collector/snmp -run 'Test(BGPPeerCache_UpdateRow|BGPIntegration_(PreservesFunctionCacheOnProfileBGPError|RecoveryClearsStaleFunctionRows|MixedProfileBGPErrorRefreshesSuccessfulProfiles|ExpiredFailedProfileDoesNotKeepStaleRowsWithFreshProfiles))'` passed on 2026-05-09 after pre-merge BGP cache hardening tests.
559
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition -run 'TestValidateEnrichProfile_BGP'` passed on 2026-05-09 after adding the `partial: true` empty-mapping rejection test.
560
-- `go test -count=1 ./collector/snmp/ddsnmp -run 'Test_(AristaAndDellBGPProfilesUseTypedRows|JuniperBGPProfilesUseOnlyJuniperPeerTables|AlcatelBGPProfileUsesTypedRows|HuaweiBGPProfileUsesTypedRows|NokiaBGPProfileMergedIntoNokiaSROS)'` passed on 2026-05-09 after adding vendor six-state mapping assertions.
561
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after pre-merge hardening tests.
562
-- `go test -race -count=1 ./collector/snmp` passed on 2026-05-09 after pre-merge hardening tests.
563
-- `go test -count=1 ./collector/snmp -run 'TestTypedBGPMetricsUseLogicalChartIdentity|TestTypedBGPMetricsFromProfileMetrics|TestBGPPeerCache_UpdateRow'` passed on 2026-05-09 after folding SOW-0016 back into SOW-0015 and adding the logical chart-identity invariant test.
564
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition -run 'TestValidateEnrichProfile_BGP'` passed on 2026-05-09 after typed BGP row-index selector and declared cross-table validation hardening.
565
-- `go test -count=1 ./collector/snmp -run 'Test(BGPPeerCache_UpdateRow|BGPIntegration_ExpiredFailedProfileDoesNotKeepStaleRowsWithFreshProfiles|TypedBGPMetricsUseLogicalChartIdentity)'` passed on 2026-05-09 after stale-entry reaping and descriptor-label chart-key coverage.
566
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_(AristaAndDellBGP_FromLibreNMSFixtures|DellOS10BGP_IPv6Rows|BGPRowsWithCrossTableBGPValues)'` passed on 2026-05-09 after Dell IPv6 and `index_from_end` plus `format` coverage.
567
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after completing the folded SOW-0016 hardening items selected by the user.
568
-- `go test -count=1 ./collector/snmp/ddsnmp/ddprofiledefinition -run 'TestValidateEnrichProfile_BGP'` passed on 2026-05-09 after final-review P2 validation-test polish.
569
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector -run 'TestCollector_Collect_DellOS10BGP_IPv6Rows'` passed on 2026-05-09 after final-review P2 Dell IPv6 dependency-table coverage.
570
-- `go test -count=1 ./collector/snmp/...` passed on 2026-05-09 after final-review P2 polish.
571
-- `git diff --check` passed on 2026-05-09 after final-review P2 polish.
572
-
573
-Real-use evidence:
574
-
575
-- No live production device walk was run in this SOW.
576
-- Production-shaped profile fixture and synthetic-PDU evidence covers the migrated typed paths:
577
- - Standard `BGP4-MIB` and Cumulus/FRR-style standard fixture paths now assert typed `BGPRows`.
578
- - Cisco typed peer and peer-family rows cover IPv4 and IPv6 `InetAddress` row-index AFI/SAFI extraction.
579
- - Juniper, Nokia/TiMOS, Huawei, Arista, Dell OS10, and Alcatel tests assert typed rows through normal profile loading/resolution/finalization or collector production paths.
580
- - `snmp:bgp-peers` cache/function tests cover typed row updates, stale rows, mixed per-source failures, expiry, and recovery.
581
-- Integration docs were regenerated through the repository generator path, not hand-edited in the generated file.
582
-
583
-Reviewer findings:
584
-
585
-- External Claude review passes were run by the user during the SOW and reconciled before commits.
586
-- Final external review on 2026-05-09 returned `READY TO MERGE` with no P0/P1 blockers.
587
-- Commit blockers fixed during the SOW include:
588
- - `funcRouter` handler map runtime race.
589
- - Per-profile BGP failure wiping stale peer cache.
590
- - Cisco IPv6 peer-family AFI/SAFI positional extraction.
591
- - Mixed-profile BGP failure freezing successful profiles.
592
- - Catalyst BGP attachment decision and test.
593
- - Untracked `cisco-ncs.yaml` slice drift.
594
- - Alcatel IPv6 BGP table OID `.12` vs authoritative `.14`.
595
- - Lost end-to-end BGP metric ID/chart-context coverage.
596
- - Dead BGP alert-surface test helper file.
597
-- Non-blocking findings were either implemented during close-out or explicitly tracked:
598
- - SOW-0014 owns typed licensing + typed BGP unsupported table-root caching.
599
- - This SOW completed typed BGP validation/runtime/fixture hardening after the user's decision to fold SOW-0016 back into SOW-0015.
600
- - Final-review P2 test gaps SCH5-1, SCH5-3, and PROF5-2 were implemented in SOW-0015.
601
-
602
-Same-failure scan:
603
-
604
-- Legacy router scan:
605
- - Command: `rg -n "bgp_public|BGPSignalKind|bgpScopeAuto|mergeBGPPeerEntryTags|bgpPeerMetricLeaf|normalizeCollectorMetrics|routeBGPPublicMetric" src/go/plugin/go.d/collector/snmp src/go/plugin/go.d/config/go.d/snmp.profiles/default --glob '!**/*_test.go'`
606
- - Result: no hits.
607
-- Legacy BGP underscore identity protocol scan:
608
- - Command: `rg -n '"_(routing_instance|neighbor|remote_as|address_family|subsequent_address_family)"|\b_(routing_instance|neighbor|remote_as|address_family|subsequent_address_family)\b' src/go/plugin/go.d/collector/snmp src/go/plugin/go.d/config/go.d/snmp.profiles/default --glob '!**/*_test.go'`
609
- - Result: only the generic virtual-metric example in `collector/snmp/profile-format.md`; no production code or profile YAML uses the deleted BGP identity-router protocol.
610
-- Cumulus/Alcatel BGP claim scan:
611
- - Command: `rg -n "Cumulus|Alcatel" src/go/plugin/go.d/collector/snmp/metadata.yaml src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md integrations/integrations.js src/collectors/COLLECTORS.md`
612
- - Result: remaining `Alcatel-Lucent` hits are in the generic Wireless vendor table, not the BGP capability/support list; regenerated `integrations.js` contains the updated BGP support sentence without Cumulus or Alcatel.
613
-- Raw MIB tracking scan:
614
- - Command: `git ls-files mibs`
615
- - Result: no tracked MIB files.
616
- - Command: `git status --short --ignored=matching mibs`
617
- - Result: no output after cleanup.
618
- - Command: `test -d mibs && echo exists || echo missing`
619
- - Result: `missing`.
620
-- Generated artifact status:
621
- - `integrations/integrations.js` and `integrations/integrations.json` were regenerated and remain gitignored.
622
- - `collector/snmp/integrations/snmp_devices.md` is the committed generated SNMP integration page diff for this SOW.
623
-- Typed-domain profile organization scan:
624
- - Command: `rg -n "^bgp:|^licensing:" config/go.d/snmp.profiles/default -g "*.yaml"`
625
- - Result: all hits are in underscore-prefixed typed fragments; no concrete device profile has an inline `bgp:` or `licensing:` section.
626
-
627
-Sensitive data gate:
628
-
629
-- Sensitive string scan:
630
- - Command: `rg -n "BEGIN (RSA|OPENSSH|PRIVATE)|Authorization:|Bearer |password|passphrase|community:|SNMP_V3|SNMP_V2C|customer|token|secret|private key" .agents/sow/current/SOW-0015-20260508-snmp-bgp-typed-projection.md .agents/sow/specs/snmp-profile-projection.md .agents/skills/project-snmp-profiles-authoring/SKILL.md src/go/plugin/go.d/collector/snmp/metadata.yaml src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md src/go/plugin/go.d/collector/snmp/profile-format.md`
631
- - Result: hits are documented placeholder SNMP examples (`community: public`, `auth_protocol_passphrase`, `priv_protocol_passphrase`) and the SOW's own sensitive-data policy text. No real credentials or customer data found.
632
-- Workstation path scan:
633
- - Command: ran `rg` for home-directory paths, private scratch paths, mirrored-repo workstation paths, and user/workstation markers across the SOW, spec, project skill, metadata, generated SNMP integration docs, and profile-format docs.
634
- - Result: no hits.
635
-- Local raw MIB files under `mibs/` were removed before SOW close-out.
636
-
637
-Artifact maintenance gate:
638
-
639
-- AGENTS.md:
640
- - No project-wide workflow change needed. The existing Go test style rule already captures the user's table-driven test preference.
641
-- Runtime project skills:
642
- - Updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md` with typed BGP profile authoring checks.
643
- - No update needed to `project-writing-collectors`; this SOW refined SNMP profile-specific BGP rules rather than general collector authoring policy.
644
- - `integrations-lifecycle` was used for generator workflow; no missing pipeline behavior was discovered that required a skill update.
645
-- Specs:
646
- - Updated `.agents/sow/specs/snmp-profile-projection.md` with BGP typed projection, structural identity, routing-instance normalization, stale-cache semantics, `partial_states`, row-index selector mutual exclusivity, and declared cross-table reference validation.
647
-- End-user/operator docs:
648
- - Updated `collector/snmp/metadata.yaml`.
649
- - Regenerated `collector/snmp/integrations/snmp_devices.md`.
650
- - Updated `collector/snmp/profile-format.md` for typed BGP authoring details.
651
- - `src/collectors/COLLECTORS.md` and `src/collectors/SECRETS.md` had no tracked diff after generation.
652
-- End-user/operator skills:
653
- - No public Netdata AI skill or operator workflow skill was affected by the BGP SNMP profile schema/docs changes.
654
- - SOW audit flagged an email-address-pattern false positive in `.agents/skills/mirror-netdata-repos/SKILL.md` for an SSH clone URL example. The line was reworded to avoid durable false-positive sensitive-data output.
655
-- SOW lifecycle:
656
- - SOW-0015 status is `completed` and the file is moved to `.agents/sow/done/`.
657
- - Initially created pending SOW-0016 for post-migration BGP hardening, then folded it back into SOW-0015 by user decision because the work is directly related and the branch is not under merge time pressure.
658
- - SOW-0014 remains the owner for typed licensing + typed BGP unsupported table-root caching.
659
-
660
-Specs update:
661
-
662
-- Updated `.agents/sow/specs/snmp-profile-projection.md` with BGP consumer, row shape, projection, inheritance, delivery, validation guarantees, structural identity, routing-instance normalization, stale-cache semantics, `partial_states`, and current row-index selector behavior.
663
-- Updated `.agents/sow/specs/snmp-profile-projection.md` again during folded hardening to record row-index selector mutual exclusivity and declared cross-table OID validation.
664
-
665
-Project skills update:
666
-
667
-- Updated `.agents/skills/project-snmp-profiles-authoring/SKILL.md`.
668
-
669
-End-user/operator docs update:
670
-
671
-- Updated `collector/snmp/metadata.yaml`.
672
-- Regenerated `collector/snmp/integrations/snmp_devices.md`.
673
-- Updated `collector/snmp/profile-format.md`.
674
-
675
-End-user/operator skills update:
676
-
677
-- No public/end-user AI skills were affected.
678
-
679
-Lessons:
680
-
681
-- Typed projection work should migrate source profiles and runtime consumers before deleting string-router code; doing the deletion only after the final vendor migration made the old-protocol leak scan decisive.
682
-- Variable-length SNMP indexes require tail-based extraction for fields such as AFI/SAFI; fixed positional index assumptions work for IPv4 and fail for IPv6.
683
-- BGP function cache freshness must be source-aware. A single profile failure must not freeze successful profiles in the same collection cycle.
684
-- Integration docs should be regenerated locally after metadata changes. Waiting for CI would have hidden AC#69 drift until after merge.
685
-
686
-Follow-up mapping:
687
-
688
-- Implemented in SOW-0015:
689
- - Remove Cumulus and Alcatel BGP public capability claims until real fixtures exist.
690
- - Regenerate SNMP integration artifacts after metadata changes.
691
- - Document `partial_states`, empty routing-instance normalization, per-source stale-cache semantics, and `index_from_end` current behavior.
692
- - Delete the legacy `bgp_public*` runtime and BGP public-router underscore protocol.
693
- - Normalize remaining inline typed-domain profile sections into dedicated `bgp:` and `licensing:` fragments.
694
-- Tracked in SOW-0014:
695
- - Typed licensing + typed BGP unsupported table-root caching for broad Cisco/Catalyst unsupported table walks.
696
-- Resolved during pre-merge review:
697
- - RT-4 typed BGP chart-key identity. Public chart keys intentionally use the existing logical SNMP table metric model (`name` plus visible tag values), not BGP structural row identity. `TestTypedBGPMetricsUseLogicalChartIdentity` covers that same logical peer tags from different typed rows resolve to the same public chart key. Function-cache identity remains structural and is tested separately.
698
- - NOK-1 Nokia/TiMOS `tBgpPeerNgOperTable` high-numbered OID verification.
699
- Nokia documentation states release MIB files are packaged with releases and
700
- available from the Nokia support portal or on-device MIB bundles
701
- (`https://documentation.nokia.com/srlinux/25-3/books/system-mgmt/snmp.html`).
702
- The local untracked `mibs/TIMETRA-BGP-MIB.mib` was refreshed from the Nokia
703
- `TIMETRA-BGP-MIB` revision `LAST-UPDATED "202302150000Z"`. The public
704
- Observium mirror of Nokia's `TIMETRA-BGP-MIB`
705
- (`https://mibs.observium.org/mib/TIMETRA-BGP-MIB/`) shows the same
706
- TiMetra branch and high-numbered `tBgpPeerNgOperEntry` objects, including
707
- entries beyond `.160`. The refreshed MIB confirms `.177`, `.178`, and
708
- `.181-.188` as `MAX-ACCESS read-only`.
709
- - SCH-3 `partial: true` empty mapping/spec-validator consistency. The
710
- validator now returns a specific error for `partial: true` state configs
711
- without mapping items, and `TestValidateEnrichProfile_BGP` covers that
712
- contract.
713
- - RT3-1 stale BGP cache entries from permanent per-source failure. Expired
714
- stale entries are reaped after the configured stale window, and
715
- `TestBGPIntegration_ExpiredFailedProfileDoesNotKeepStaleRowsWithFreshProfiles`
716
- asserts the expired source key is removed from the cache.
717
- - SCH-2 typed BGP cross-table `table:` reference validation. Profile
718
- validation now resolves declared BGP row table names and rejects source or
719
- lookup-symbol OIDs outside the referenced table root. Existing runtime
720
- inference for non-declared cross-table roots remains supported by design.
721
- - SCH2-4 row-index selector mutual exclusivity. Profile validation rejects
722
- typed BGP values that set more than one of `index`, `index_from_end`, and
723
- `index_transform`.
724
- - SCH2-6 `index_from_end` plus `format` propagation coverage. The
725
- cross-table BGP row test now covers `IndexFromEnd` with `Format: "hex"`.
726
- - Dell IPv6 typed BGP coverage. `TestCollector_Collect_DellOS10BGP_IPv6Rows`
727
- covers IPv6 peer and peer-family rows with typed identity, descriptors,
728
- connection fields, and route counts.
729
- - Typed BGP descriptor underscore label behavior. The logical chart-identity
730
- test verifies descriptor labels such as `_local_address` and
731
- `_peer_description` remain visible labels and do not participate in public
732
- chart-key identity.
733
- - Final-review P2 validation coverage. `TestValidateEnrichProfile_BGP` now
734
- covers the remaining row-index selector combinations and BGP row kind vs
735
- field-group compatibility branches.
736
- - Final-review P2 Dell IPv6 dependency-table coverage.
737
- `TestCollector_Collect_DellOS10BGP_IPv6Rows` now asserts an IPv6
738
- dependency-table traffic counter in addition to identity, descriptors,
739
- state, and routes.
740
- - TST5-2/TST5-4/TST5-9 pre-merge test gaps. Added function-cache coverage
741
- for same-peer structural IDs from different profile rows, stale failure
742
- recovery, and vendor six-state mapping assertions for Arista, Dell,
743
- Juniper, and Alcatel.
744
-- Rejected as a SOW-0015 follow-up:
745
- - Generated function-name capitalization (`Snmp:*`, `Docker:*`, `Mysql:*`, etc.) is a pre-existing global integration template behavior in `integrations/templates/functions.md`, not specific to BGP typed projection. It should be handled only as a separate integrations-docs decision if the project wants to normalize all generated Function names.
746
-- Nothing remains pending before SOW close.
747
-
748
-## Outcome
749
-
750
-Completed. SNMP BGP monitoring now uses typed `bgp:` projection end to end, all BGP-bearing stock profiles are migrated, the legacy `bgp_public*` runtime is deleted, metadata/docs/spec artifacts are reconciled, external review blockers and selected P2 polish are addressed, SOW-0016 is folded into this SOW, and local raw `mibs/` reference files are removed.
751
-
752
-## Lessons Extracted
753
-
754
-Recorded in the Validation gate.
755
-
756
-## Followup
757
-
758
-All valid follow-up items from this SOW are implemented, rejected with evidence, or represented by SOW-0014. SOW-0016 was intentionally deleted after migration into this SOW by user decision.
759
-
760
-## Regression Log
761
-
762
-None yet.
763
-
764
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0015-20260508-static-journal-facets.md
deleted
-809
@@ -1,809 +0,0 @@
1
-# SOW-0015 - Static Journal Facet Filtering
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: PR review rerun cleanup completed.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Restore correct systemd journal facet filtering for static Netdata Agent builds that use the Rust journal provider, especially on distributions whose journal files contain LZ4-compressed data objects. Keep the fix small, source-verified, and isolated to the clean worktree created from `master`.
14
-
15
-### User Request
16
-
17
-The user reported that the prior session is from another server and must not be trusted as proof. The user clarified that dnf packages work because they use `libsystemd`, static installs fail, and all edits must happen in a new worktree from `master`, not in the dirty main checkout where another worker is active.
18
-
19
-### Assistant Understanding
20
-
21
-Facts:
22
-
23
-- Clean worktree: `~/src/PRs/netdata-static-journal-facets`, branch `fix/static-journal-facets`, created from refreshed `origin/master`.
24
-- The main checkout is dirty with unrelated netflow work and must not be edited.
25
-- Static builds can use the Rust journal provider: `CMakeLists.txt:201`, `CMakeLists.txt:220`, `CMakeLists.txt:2908`.
26
-- dnf builds on the user's tested server use native `libsystemd`; those are outside the failing path.
27
-
28
-Inferences:
29
-
30
-- The failing path is the Rust provider under `src/crates/jf/`, not native `sd-journal`.
31
-- The bug affects filtered/faceted queries more strongly than unfiltered scans because facet slicing builds journal matches from unique field values before scanning rows.
32
-
33
-Unknowns:
34
-
35
-- None for the accepted scope. A local RHEL 8.10 static install was available for regression validation after the first PR iteration.
36
-
37
-### Acceptance Criteria
38
-
39
-- Rust journal provider can return decompressed payloads for LZ4/XZ/Zstd compressed data objects.
40
-- Unique-value enumeration returns logical `field=value` bytes, not compressed payload bytes.
41
-- Data-object lookup used by match filters can find compressed data objects when the caller provides logical `field=value`.
42
-- Facet filter setup falls back to a full query if unique-value enumeration returns an error while constructing backend matches.
43
-- Existing uncompressed data behavior remains unchanged.
44
-- Focused Rust tests pass for `src/crates/jf/`.
45
-
46
-## Analysis
47
-
48
-Sources checked:
49
-
50
-- `.agents/skills/project-writing-collectors/SKILL.md`
51
-- `src/collectors/systemd-journal.plugin/systemd-journal.c`
52
-- `src/collectors/systemd-journal.plugin/provider/netdata_provider.h`
53
-- `src/collectors/systemd-journal.plugin/provider/rust_provider.h`
54
-- `src/crates/jf/journal_reader_ffi/src/lib.rs`
55
-- `src/crates/jf/journal_file/src/object.rs`
56
-- `src/crates/jf/journal_file/src/file.rs`
57
-- `src/crates/journal-core/src/file/object.rs`
58
-- `src/crates/jf/Cargo.toml`
59
-- `src/crates/jf/journal_file/Cargo.toml`
60
-- Official systemd journal file format documentation: `https://systemd.io/JOURNAL_FILE_FORMAT/`
61
-
62
-Current state:
63
-
64
-- `systemd-journal.c:583-607` builds native journal matches from `NSD_JOURNAL_FOREACH_UNIQUE()` values, parses them as `field=value`, then passes the same bytes to `nsd_journal_add_match()`.
65
-- `systemd-journal.c:661` skips a journal file when filters exist but no backend matches can be installed.
66
-- `journal_reader_ffi/src/lib.rs:285-305` decompresses compressed entry data for normal row scans.
67
-- `journal_reader_ffi/src/lib.rs:383-389` returns unique-value payload bytes without checking compression.
68
-- `journal_file/src/object.rs:873-890` marks LZ4/XZ/Zstd as compression methods, but only implements Zstd decompression.
69
-- `journal-core/src/file/object.rs:978-1021` already implements Zstd, LZ4, and XZ decompression in the newer shared journal core.
70
-- `journal_file/src/file.rs:548-550` uses payload matching for data-object lookup; `file.rs:191` compares raw object payload bytes to the caller-provided payload, which cannot find compressed objects when the caller provides logical `field=value`.
71
-- The official journal format says DATA objects contain `field=value` payloads, that XZ/LZ4/Zstd compression is signaled by object/header flags, and that the DATA object hash is computed from the payload. That matches the need to expose/deal with the logical payload even when the on-disk bytes are compressed.
72
-
73
-Risks:
74
-
75
-- A partial fix that only decompresses unique enumeration may still fail when the filter builder later looks up the compressed data object.
76
-- Adding decompression to hash-table lookup can affect filtered query performance on compressed buckets. The lookup is only on candidate objects in one hash bucket and is necessary for correctness.
77
-- Adding new Rust dependencies changes `src/crates/jf/Cargo.lock`; this must be validated with focused cargo tests.
78
-
79
-## Pre-Implementation Gate
80
-
81
-Status: ready
82
-
83
-Problem / root-cause model:
84
-
85
-- Static builds use the Rust journal provider path when `ENABLE_NETDATA_JOURNAL_FILE_READER` is enabled. That provider must emulate `libsystemd` APIs expected by `systemd-journal.plugin`.
86
-- Facet counters can be produced by row scans because `rsd_journal_enumerate_available_data()` already decompresses entry data before returning it.
87
-- Facet filtering uses unique-value enumeration to install backend matches. The Rust unique path currently returns raw payloads and does not decompress. On compressed systemd journal data, `parse_journal_field()` cannot reliably see `field=value`, so selected facet matches are not installed.
88
-- Even after returning decompressed unique data, filter construction must resolve the logical `field=value` back to the data object offset. The current data-object matcher compares raw on-disk payload bytes, so compressed objects remain unfindable by logical payload.
89
-- The `src/crates/jf/` decompressor only supports Zstd despite carrying LZ4/XZ flags. The newer `src/crates/journal-core/` code provides a local pattern for LZ4 and XZ support.
90
-
91
-Evidence reviewed:
92
-
93
-- `src/collectors/systemd-journal.plugin/systemd-journal.c:583-607`
94
-- `src/collectors/systemd-journal.plugin/systemd-journal.c:661`
95
-- `src/crates/jf/journal_reader_ffi/src/lib.rs:285-305`
96
-- `src/crates/jf/journal_reader_ffi/src/lib.rs:383-389`
97
-- `src/crates/jf/journal_file/src/object.rs:857-890`
98
-- `src/crates/jf/journal_file/src/file.rs:157-191`
99
-- `src/crates/journal-core/src/file/object.rs:978-1021`
100
-- `src/crates/jf/Cargo.toml:16-28`
101
-- `src/crates/jf/journal_file/Cargo.toml:7-16`
102
-- `https://systemd.io/JOURNAL_FILE_FORMAT/` sections "Structure", "Extensibility", "Data Objects", and "Reading".
103
-
104
-Affected contracts and surfaces:
105
-
106
-- Static-build systemd journal log queries and facet filters.
107
-- Rust FFI compatibility with the C plugin's `sd-journal`-like expectations.
108
-- Rust crate dependency lockfile for `src/crates/jf/`.
109
-- No public configuration, schema, or user-facing documentation surface is expected to change.
110
-
111
-Existing patterns to reuse:
112
-
113
-- Existing entry-data decompression branch in `rsd_journal_enumerate_available_data()`.
114
-- Existing LZ4/XZ/Zstd decompression implementation in `src/crates/journal-core/src/file/object.rs`.
115
-- Existing `JournalError::DecompressorError` / `UnknownCompressionMethod` handling.
116
-- Existing `PayloadMatcher` bucket visitor pattern.
117
-
118
-Risk and blast radius:
119
-
120
-- Scope is confined to the Rust journal file reader used by the static provider.
121
-- Native `libsystemd` builds should be unaffected.
122
-- Main behavioral risk is filtered lookup performance on compressed hash buckets; lookup remains bounded to one bucket.
123
-- Security risk is low; decompression must preserve existing error handling and must not panic on malformed compressed payloads.
124
-
125
-Sensitive data handling plan:
126
-
127
-- No raw logs, hostnames, IP addresses, customer identifiers, secrets, or journal payload samples will be written to durable artifacts.
128
-- SOW evidence records only generic OS/compression behavior and source file references.
129
-
130
-Implementation plan:
131
-
132
-1. Add LZ4 and XZ dependencies to the `src/crates/jf/` workspace and port the established decompression logic from `journal-core`.
133
-2. Make `rsd_journal_enumerate_available_unique()` mirror entry-data enumeration by returning decompressed payloads for compressed data objects.
134
-3. Make data-object payload matching compare decompressed payloads for compressed data objects so filter construction can find the matching object offset.
135
-4. Add focused Rust tests for LZ4 decompression and compressed payload matching.
136
-5. Run focused formatting and tests for the `src/crates/jf/` workspace.
137
-
138
-Validation plan:
139
-
140
-- `cargo fmt` in `src/crates/jf`.
141
-- `cargo test` in `src/crates/jf`.
142
-- Same-failure scan for remaining raw `payload_bytes()` returns in FFI paths.
143
-- Source review of all compressed data object reads in `src/crates/jf/`.
144
-
145
-Artifact impact plan:
146
-
147
-- AGENTS.md: no update expected; workflow rules unchanged.
148
-- Runtime project skills: no update expected; collector-writing guidance remains valid.
149
-- Specs: no update expected; this is a bug fix to match existing static-provider intent.
150
-- End-user/operator docs: no update expected; no user-facing command/config changes.
151
-- End-user/operator skills: no update expected; public AI skills are unaffected.
152
-- SOW lifecycle: this SOW tracks the work and is completed/moved with the implementation in the same commit.
153
-
154
-Open-source reference evidence:
155
-
156
-- No external mirrored repository evidence used yet. The fix is based on two in-repository implementations of the same journal format.
157
-
158
-Open decisions:
159
-
160
-- None. The user has already specified the worktree constraint and the failing implementation path.
161
-
162
-## Implications And Decisions
163
-
164
-- No user decision is currently required. The evidence points to a bounded bug fix in the static Rust provider.
165
-
166
-## Plan
167
-
168
-1. Patch `src/crates/jf/journal_file` decompression support and data payload matching.
169
-2. Patch `src/crates/jf/journal_reader_ffi` unique enumeration.
170
-3. Update `src/crates/jf` Cargo manifests/lockfile.
171
-4. Add focused tests.
172
-5. Run focused validation and update this SOW.
173
-
174
-## Execution Log
175
-
176
-### 2026-05-08
177
-
178
-- Created clean worktree from refreshed `origin/master`.
179
-- Loaded project collector-writing skill.
180
-- Verified source evidence and wrote pre-implementation gate.
181
-- Added LZ4/XZ decompression support to the legacy `src/crates/jf/journal_file` reader, reusing the newer `journal-core` implementation pattern.
182
-- Changed Rust FFI unique-value enumeration to return decompressed payloads for compressed DATA objects.
183
-- Changed DATA hash-bucket payload matching to compare decompressed payloads when the on-disk object is compressed.
184
-- Added focused LZ4 compressed payload matcher tests.
185
-- Corrected an existing filter test expectation: the test writes 5,000 iterations and 2 matching rows per iteration, so the expected filtered count is `2 * iterations`, not `2`.
186
-
187
-## Validation
188
-
189
-Acceptance criteria evidence:
190
-
191
-- LZ4/XZ/Zstd support: `src/crates/jf/journal_file/src/object.rs` now handles Zstd, LZ4 with the systemd 8-byte uncompressed-size prefix, and XZ.
192
-- Unique enumeration: `src/crates/jf/journal_reader_ffi/src/lib.rs` now mirrors entry-data enumeration and decompresses before returning payload bytes.
193
-- Match lookup: `src/crates/jf/journal_file/src/file.rs` now uses `DataPayloadMatcher`, which compares raw payload first and then decompressed payload for compressed DATA objects.
194
-- Uncompressed behavior: raw `object.get_payload() == self.payload` matching remains the first path.
195
-- Focused tests: `src/crates/jf/journal_file/src/file.rs` adds positive and negative LZ4 compressed payload matcher tests.
196
-
197
-Tests or equivalent validation:
198
-
199
-- `cargo fmt` in `src/crates/jf`: passed.
200
-- `cargo test -q` in `src/crates/jf`: passed; 4 tests passed.
201
-- `git diff --check`: passed.
202
-- `.agents/sow/audit.sh`: status/directory checks passed for this SOW; the audit reported one pre-existing sensitive-data pattern in `.agents/skills/mirror-netdata-repos/SKILL.md:112`, which is public SSH clone syntax (`git@github.com:netdata/...`) in an unrelated existing file, not sensitive data from this work.
203
-
204
-Real-use evidence:
205
-
206
-- Initial implementation was not run against the local RHEL 8.10 static install before opening the PR.
207
-- Regression validation below records live Function evidence from that static install after the reopened fix.
208
-
209
-Reviewer findings:
210
-
211
-- Initial implementation had no external reviewer pass before opening the PR; the user asked to distrust the prior session and verify locally from code.
212
-- PR review iterations found Copilot comments on `netdata/netdata#22456`; each was verified before code changes, addressed in the same SOW, replied to in-thread, and resolved after commit/push.
213
-
214
-Same-failure scan:
215
-
216
-- `rg` over `src/crates/jf` for `payload_bytes()`, `decompress()`, `enumerate_available_unique`, `enumerate_available_data`, and `find_data_offset()` found the fixed FFI paths and the fixed match lookup. Remaining raw `payload_bytes()` use in `src/crates/jf/journal_file/src/filter.rs:246` is a debug dump path, not match construction or row enumeration.
217
-
218
-Sensitive data gate:
219
-
220
-- Durable artifacts contain no raw logs, secrets, credentials, bearer tokens, SNMP communities, customer names, personal data, non-private customer-identifying IPs, private endpoints, or proprietary incident details.
221
-
222
-Artifact maintenance gate:
223
-
224
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
225
-- Runtime project skills: no update needed; this did not change how agents should work on collectors.
226
-- Specs: no update needed; this bug fix restores the intended static provider behavior and does not create a new product contract.
227
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
228
-- End-user/operator skills: no update needed; public/operator AI skills are unaffected.
229
-- SOW lifecycle: SOW status is `completed` and the file is moved to `.agents/sow/done/` with the implementation in the same commit.
230
-
231
-Specs update:
232
-
233
-- No spec update needed; behavior remains "static Rust provider should act like the native journal provider for DATA object payloads."
234
-
235
-Project skills update:
236
-
237
-- No project skill update needed; no new reusable workflow was discovered.
238
-
239
-End-user/operator docs update:
240
-
241
-- No docs update needed; this is a transparent bug fix.
242
-
243
-End-user/operator skills update:
244
-
245
-- No end-user/operator skill update needed; no public skill behavior changed.
246
-
247
-Lessons:
248
-
249
-- When fixing static-provider journal filtering, verify both the enumeration side and the data-offset lookup side. Returning decompressed `field=value` bytes is not enough if the filter builder still searches the DATA hash table by raw compressed bytes.
250
-
251
-Follow-up mapping:
252
-
253
-- No follow-up was needed for the initial fix; the later live regression is tracked in the appended regression section.
254
-
255
-## PR Review Iteration - 2026-05-08
256
-
257
-Findings:
258
-
259
-- `PRRT_kwDOAKPxd86Asq8i`, `src/crates/jf/journal_reader_ffi/src/lib.rs:396`: valid. The decompression error path returned generic `-1` and printed to stderr instead of returning `JournalError::to_error_code()`. Same-class sweep found the same pattern in `rsd_journal_enumerate_available_data()`.
260
-- `PRRT_kwDOAKPxd86Asq9F`, `src/crates/jf/journal_file/src/object.rs:899`: valid. The LZ4 branch trusted the on-disk uncompressed size prefix before `Vec::resize()`. Same-class sweep found the same LZ4 reader pattern in `src/crates/journal-core/src/file/object.rs`.
261
-- CI signal before this iteration: one Docker armv7 job failed in its `Build Image` step while the workflow run was still in progress. GitHub did not expose logs yet because the overall run had not completed; all other available failure sources were either pending or passing.
262
-
263
-Actions:
264
-
265
-- Changed compressed DATA enumeration FFI paths to return `e.to_error_code()` for decompression errors.
266
-- Added checked `u64` to `usize` conversion for the LZ4 uncompressed-size prefix.
267
-- Added a DATA payload upper bound matching systemd's `DATA_SIZE_MAX` journal importer limit: 768 MiB.
268
-- Used `try_reserve_exact()` before `Vec::resize()` so allocation failure returns `JournalError::DecompressorError` instead of panicking.
269
-- Applied the same LZ4 bounds hardening to both `src/crates/jf/journal_file` and `src/crates/journal-core`.
270
-- Added oversized LZ4 prefix regression tests in both readers.
271
-
272
-Validation:
273
-
274
-- `cargo fmt` in `src/crates/jf`: passed.
275
-- `cargo fmt` in `src/crates`: passed; unrelated formatting churn in `src/crates/netdata-plugin/rt/src/lib.rs` was removed before staging.
276
-- `cargo test -q` in `src/crates/jf`: passed; 5 tests passed.
277
-- `cargo test -q -p journal-core` in `src/crates`: passed; 19 tests passed plus the existing ignored tests.
278
-
279
-## PR Review Iteration 2 - 2026-05-08
280
-
281
-Findings:
282
-
283
-- `PRRT_kwDOAKPxd86AtF96`, `src/crates/journal-core/src/file/object.rs:1019`: valid. `try_reserve_exact()` was given a delta from capacity, but the API expects additional capacity from current length.
284
-- `PRRT_kwDOAKPxd86AtF-U`, `src/crates/jf/journal_file/src/object.rs:911`: valid. Same fallible-reserve issue in the legacy reader.
285
-- `PRRT_kwDOAKPxd86AtF-e`, `src/crates/jf/journal_file/src/object.rs:890`: valid. The Zstd streaming path still used unbounded `read_to_end()`; same-class sweep also covered XZ and `journal-core`.
286
-- `PRRT_kwDOAKPxd86AtF-n`, `src/crates/jf/journal_file/src/object.rs:927`: valid. The newly added XZ decompression branch needed a focused positive test; same-class sweep added the same coverage to `journal-core`.
287
-
288
-Actions:
289
-
290
-- Corrected fallible LZ4 reserve logic to reserve the full required final length after `clear()`.
291
-- Added bounded `read_limited_to_end()` helpers for streaming Zstd/XZ decompression.
292
-- Applied the same bounded streaming decompression to both `src/crates/jf/journal_file` and `src/crates/journal-core`.
293
-- Added fixed XZ compressed-payload fixtures and positive decompression tests for both readers without enabling the encoder feature in production dependencies.
294
-
295
-Validation:
296
-
297
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
298
-- `cargo fmt -p journal-core` in `src/crates`: passed.
299
-- `cargo test -q` in `src/crates/jf`: passed; 6 tests passed.
300
-- `cargo test -q -p journal-core` in `src/crates`: passed; 20 tests passed plus the existing ignored tests.
301
-
302
-## PR Review Iteration 3 - 2026-05-08
303
-
304
-Findings:
305
-
306
-- `PRRT_kwDOAKPxd86AtR3f`, `src/crates/jf/journal_file/src/file.rs:185`: partially valid. The slice comparison compiled in focused tests, but comparing by reference is clearer and avoids relying on implicit unsized slice comparison behavior.
307
-- `PRRT_kwDOAKPxd86AtR33`, `src/crates/jf/journal_file/src/object.rs:787`: considered and not implemented as shared code. The duplicate helper exists in a legacy separate `src/crates/jf` workspace and the newer `src/crates/journal-core` workspace, with different error types. Centralizing via `journal-common` would add unrelated dependencies to the legacy static reader for a small private helper. Keeping the helpers local is lower risk for this bug-fix PR.
308
-- `PRRT_kwDOAKPxd86AtR4H`, `src/crates/jf/journal_file/src/object.rs:796`: valid. `read_to_end()` can leave partial bytes in the reusable buffer before returning an error.
309
-- `PRRT_kwDOAKPxd86AtR4a`, `src/crates/journal-core/src/file/object.rs:901`: valid. The stream-size cap needed a focused small-cap test that does not allocate hundreds of MiB.
310
-
311
-Actions:
312
-
313
-- Changed compressed payload comparison to compare slice references explicitly.
314
-- Changed bounded stream reads to clear the reusable buffer on both over-limit and read-error paths.
315
-- Added private small-cap helper entry points used by tests.
316
-- Added small custom `Read` tests for over-limit stream handling in both readers.
317
-
318
-Validation:
319
-
320
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
321
-- `cargo fmt -p journal-core` in `src/crates`: passed.
322
-- `cargo test -q` in `src/crates/jf`: passed; 7 tests passed.
323
-- `cargo test -q -p journal-core` in `src/crates`: passed; 21 tests passed plus the existing ignored tests.
324
-
325
-## PR Review Iteration 4 - 2026-05-08
326
-
327
-Findings:
328
-
329
-- `PRRT_kwDOAKPxd86AteMb`, `src/crates/journal-core/src/file/object.rs:900`: valid. The testable size-cap helper converted `usize` to `u64` with `as` and then added 1.
330
-- `PRRT_kwDOAKPxd86AteNJ`, `src/crates/jf/journal_file/src/file.rs:204`: valid readability issue. The `BucketVisitor` implementation used an elided matcher lifetime.
331
-- `PRRT_kwDOAKPxd86AteNi`, `src/crates/jf/journal_file/src/writer.rs:640`: valid. The corrected assertion message lost useful context.
332
-
333
-Actions:
334
-
335
-- Changed both bounded-read helpers to use `u64::try_from(max_size)` plus `checked_add(1)`.
336
-- Made `DataPayloadMatcher`'s borrowed payload lifetime explicit in the `BucketVisitor` implementation.
337
-- Restored context in the filter-count assertion message.
338
-
339
-Validation:
340
-
341
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
342
-- `cargo fmt -p journal-core` in `src/crates`: passed.
343
-- `cargo test -q` in `src/crates/jf`: passed; 7 tests passed.
344
-- `cargo test -q -p journal-core` in `src/crates`: passed; 21 tests passed plus the existing ignored tests.
345
-
346
-## PR Review Iteration 5 - 2026-05-08
347
-
348
-Findings:
349
-
350
-- `PRRT_kwDOAKPxd86Atn1r`, `src/crates/journal-core/src/file/object.rs:1047`: valid. The LZ4 decode-error path left the reusable buffer resized to the advertised uncompressed size.
351
-- `PRRT_kwDOAKPxd86Atn2C`, `src/crates/jf/journal_file/src/object.rs:941`: valid. Same LZ4 decode-error buffer state issue in the legacy static reader, including the FFI caller reuse path.
352
-
353
-Actions:
354
-
355
-- Changed both LZ4 decode-error branches to replace the reusable buffer with a new empty `Vec`, so callers cannot observe stale decompressed bytes or retain the failed allocation.
356
-- Added malformed LZ4 block regression tests for both readers.
357
-
358
-Validation:
359
-
360
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
361
-- `cargo fmt -p journal-core` in `src/crates`: passed.
362
-- `cargo test -q` in `src/crates/jf`: passed; 8 tests passed.
363
-- `cargo test -q -p journal-core` in `src/crates`: passed; 22 tests passed plus the existing ignored tests.
364
-- `git diff --check`: passed.
365
-
366
-## PR Review Iteration 6 - 2026-05-08
367
-
368
-Findings:
369
-
370
-- `PRRT_kwDOAKPxd86Atzu9`, `src/crates/journal-core/src/file/object.rs:916`: valid. The bounded Zstd/XZ stream helper cleared the buffer on over-limit or read errors but retained the potentially large allocation.
371
-- `PRRT_kwDOAKPxd86Atzv6`, `src/crates/jf/journal_file/src/object.rs:811`: valid. Same bounded stream allocation-retention issue in the legacy static reader.
372
-- `PRRT_kwDOAKPxd86Atzvg`, `src/crates/journal-core/src/file/object.rs:1052`: valid. The LZ4 path left `buf.len()` at the advertised size if the decoder returned fewer bytes than the prefix.
373
-- `PRRT_kwDOAKPxd86AtzwG`, `src/crates/jf/journal_file/src/object.rs:946`: valid. Same LZ4 size/length mismatch semantics in the legacy static reader.
374
-
375
-Actions:
376
-
377
-- Changed both bounded stream helpers to replace the reusable buffer with a new empty `Vec` on over-limit or read errors.
378
-- Changed both LZ4 branches to accept success only when the decoded length matches the systemd uncompressed-size prefix; mismatch now resets the buffer and returns `JournalError::DecompressorError`.
379
-- Added LZ4 size-mismatch regression tests in both readers and strengthened bounded stream tests to assert capacity release.
380
-
381
-Validation:
382
-
383
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
384
-- `cargo fmt -p journal-core` in `src/crates`: passed.
385
-- `cargo test -q` in `src/crates/jf`: passed; 9 tests passed.
386
-- `cargo test -q -p journal-core` in `src/crates`: passed; 23 tests passed plus the existing ignored tests.
387
-- `git diff --check`: passed.
388
-
389
-## PR Review Iteration 7 - 2026-05-08
390
-
391
-Findings:
392
-
393
-- `PRRT_kwDOAKPxd86At82Y`, `src/crates/journal-core/src/file/object.rs:915`: valid. `read_to_end()` with `take(max + 1)` bounded decompressed bytes but still allowed `Vec` growth strategy to over-allocate beyond the intended cap.
394
-- `PRRT_kwDOAKPxd86At821`, `src/crates/jf/journal_file/src/file.rs:583`: valid. Existing tests covered `DataPayloadMatcher::payload_matches()` directly but not the `find_data_offset()` hash-bucket traversal path that uses the matcher.
395
-
396
-Actions:
397
-
398
-- Replaced bounded streaming `read_to_end()` calls in both readers with a manual fixed-size stack-buffer loop that uses `try_reserve_exact()` for each chunk and fails as soon as the decompressed stream exceeds the configured cap.
399
-- Added a temporary-journal test that writes a compressed DATA object into the data hash table and verifies `find_data_offset()` locates it by logical uncompressed payload.
400
-
401
-Validation:
402
-
403
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
404
-- `cargo fmt -p journal-core` in `src/crates`: passed.
405
-- `cargo test -q` in `src/crates/jf`: passed; 10 tests passed.
406
-- `cargo test -q -p journal-core` in `src/crates`: passed; 23 tests passed plus the existing ignored tests.
407
-- `git diff --check`: passed.
408
-
409
-## PR Review Iteration 8 - 2026-05-08
410
-
411
-Findings:
412
-
413
-- `PRRT_kwDOAKPxd86AuG7E`, `src/crates/journal-core/src/file/object.rs:926`: valid. Per-read `try_reserve_exact()` avoided unbounded growth but could cause one allocation per read chunk for large decompressed payloads.
414
-- `PRRT_kwDOAKPxd86AuG7c`, `src/crates/jf/journal_file/src/file.rs:1190`: reviewed and not changed. The direct matcher fixture's `ObjectHeader::size = header + payload.len()` matches the writer contract; object placement alignment is handled by `ObjectHeader::aligned_size()` and is covered by the new temporary-journal `find_data_offset()` test.
415
-
416
-Actions:
417
-
418
-- Changed both bounded stream helpers to reserve capacity in amortized chunks, growing up to the configured decompressed-size cap without using `read_to_end()`'s geometric growth.
419
-- Kept the direct matcher helper unpadded because padding those raw bytes would become part of the payload passed to `DataObject::from_data()` and would diverge from the writer's unpadded `size` field.
420
-
421
-Validation:
422
-
423
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
424
-- `cargo fmt -p journal-core` in `src/crates`: passed.
425
-- `cargo test -q` in `src/crates/jf`: passed; 10 tests passed.
426
-- `cargo test -q -p journal-core` in `src/crates`: passed; 23 tests passed plus the existing ignored tests.
427
-- `git diff --check`: passed.
428
-
429
-## PR Review Iteration 9 - 2026-05-08
430
-
431
-Findings:
432
-
433
-- `PRRT_kwDOAKPxd86AuPUu`, `src/crates/journal-core/src/file/object.rs:947`: valid. The amortized `try_reserve_exact()` call still used a capacity delta, but the API takes additional capacity relative to `buf.len()`.
434
-- `PRRT_kwDOAKPxd86AuPVH`, `src/crates/jf/journal_file/src/object.rs:842`: valid. Same reservation argument bug in the legacy reader.
435
-
436
-Actions:
437
-
438
-- Changed both amortized reservation paths to pass `target_capacity - buf.len()` to `try_reserve_exact()`.
439
-
440
-Validation:
441
-
442
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
443
-- `cargo fmt -p journal-core` in `src/crates`: passed.
444
-- `cargo test -q` in `src/crates/jf`: passed; 10 tests passed.
445
-- `cargo test -q -p journal-core` in `src/crates`: passed; 23 tests passed plus the existing ignored tests.
446
-- `git diff --check`: passed.
447
-
448
-## Outcome
449
-
450
-Implemented, validated, and prepared for commit in `~/src/PRs/netdata-static-journal-facets`.
451
-
452
-## Lessons Extracted
453
-
454
-- The legacy `src/crates/jf/` reader and the newer `src/crates/journal-core/` reader had diverged on compression support. For journal-format fixes, compare both implementations before changing one path.
455
-
456
-## Followup
457
-
458
-None for the completed PR review iteration; the appended regression section records the live validation repair.
459
-
460
-## Regression Log
461
-
462
-## Regression - 2026-05-08
463
-
464
-What broke:
465
-
466
-- Live static install testing on `[LOCAL_RHEL_8_10_TEST_AGENT]:19999` still returned empty rows when a systemd-journal facet value was selected.
467
-- The previous validation only proved compressed DATA lookup/decompression at the Rust crate level. It did not prove the full C plugin Function path that receives facet selections from the UI, converts them into backend matches, and streams rows back to the dashboard.
468
-
469
-Evidence:
470
-
471
-- User report: static build copied and installed on a local RHEL 8.10 test Agent; selecting a facet returns empty responses.
472
-- The regression is specific to facet filtering, not unfiltered journal access.
473
-- Direct agent Function evidence from `[LOCAL_RHEL_8_10_TEST_AGENT]:19999`: unfiltered `systemd-journal` with `slice:true` returned 20 rows and non-empty facets for the last 4 hours; selecting advertised values `SYSLOG_IDENTIFIER=netdata`, `_SYSTEMD_UNIT=netdata.service`, or `PRIORITY=6` returned 0 rows with `slice:true`, while the same selections returned 20 rows with `slice:false`.
474
-- Direct response stats for `SYSLOG_IDENTIFIER=netdata`, `slice:true`: request echoes the expected JSON `selections`, but `rows.evaluated=0`, `rows.matched=0`, and the journal file reports `rows_read=0`. This matches a failure before row scanning, during native match setup / filtered cursor resolution.
475
-- Source evidence: `src/crates/jf/journal_file/src/filter.rs:339-357` builds filtered cursors by hashing selected `field=value` bytes and resolving them through `find_data_offset()`.
476
-- Source evidence: `src/crates/jf/journal_file/src/hash.rs:4-8` still uses `twox_hash::XxHash64` behind a `FIXME` for the non-keyed Jenkins path.
477
-- Source evidence: `systemd/systemd @ d0c912899a33436d6676b2564eb1ac506f378571`, `src/libsystemd/sd-journal/journal-file.c:1585-1600`, uses `siphash24()` only for keyed journal files and `jenkins_hash64()` otherwise.
478
-- Source evidence: `systemd/systemd @ d0c912899a33436d6676b2564eb1ac506f378571`, `src/libsystemd/sd-journal/lookup3.h:14-20`, defines `jenkins_hash64()` as lookup3 `jenkins_hashlittle2()` with the primary value in the high 32 bits and the secondary value in the low 32 bits.
479
-- In-repository pattern: `src/crates/journal-core/src/file/hash.rs:4-17` already uses `hashers::jenkins::Lookup3Hasher` and swaps the 32-bit halves to match systemd's `jenkins_hash64()`.
480
-
481
-Repair plan:
482
-
483
-- Replace the legacy `src/crates/jf` non-keyed hash fallback with the same lookup3/Jenkins implementation used by `journal-core`.
484
-- Add reference-value tests for systemd-compatible Jenkins hashes.
485
-- Re-run focused Rust tests and direct selected-facet queries on the local RHEL 8.10 static install.
486
-
487
-Validation plan:
488
-
489
-- Focused Rust crate tests for the fixed path.
490
-- Direct API evidence from the local RHEL 8.10 static install showing the same facet values return rows after the fix.
491
-- Static build/install validation using the user-provided local static-binary workflow if a code change is needed.
492
-
493
-Why previous validation missed it:
494
-
495
-- The first fix validated compressed DATA enumeration and compressed payload lookup, but did not validate the full selected-facet Function path on a real non-keyed journal file.
496
-- The failing `slice:true` path builds filtered cursors before scanning rows; the incorrect hash function made `find_data_offset()` fail before row evaluation, so decompression tests alone could not catch it.
497
-
498
-Actions:
499
-
500
-- Replaced the legacy `src/crates/jf` non-keyed journal hash implementation with `hashers::jenkins::Lookup3Hasher`, matching the existing `journal-core` pattern and systemd lookup3 half ordering.
501
-- Added the empty-payload lookup3 guard because `hashers` returns `0` for empty input while systemd lookup3 with zero seeds returns `0xdeadbeefdeadbeef`.
502
-- Added systemd reference-value tests for the legacy reader hash path.
503
-- Applied the same empty-payload guard and reference-value test to `src/crates/journal-core` so the legacy and newer journal readers remain consistent.
504
-
505
-Validation:
506
-
507
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
508
-- `cargo fmt -p journal-core` in `src/crates`: passed.
509
-- `cargo test -q` in `src/crates/jf`: passed; 11 tests passed.
510
-- `cargo test -q -p journal-core` in `src/crates`: passed; 24 tests passed plus the existing ignored tests.
511
-- Static build via the user-provided local static-binary workflow: passed; produced `artifacts/netdata-x86_64-latest.gz.run`.
512
-- Static install on the local RHEL 8.10 test Agent: passed; installer restarted `netdata` through systemd.
513
-- Direct Function validation after install:
514
- - baseline `slice:true`: `data_len=20`, `rows_evaluated=2019`, `rows_matched=2019`, `rows_read=2019`.
515
- - `SYSLOG_IDENTIFIER=netdata`, `slice:true`: `data_len=20`, `rows_evaluated=1499`, `rows_matched=1499`, `rows_read=1499`.
516
- - `_SYSTEMD_UNIT=netdata.service`, `slice:true`: `data_len=20`, `rows_evaluated=1714`, `rows_matched=1714`, `rows_read=1714`.
517
- - `PRIORITY=6`, `slice:true`: `data_len=20`, `rows_evaluated=1596`, `rows_matched=1596`, `rows_read=1596`.
518
- - `SYSLOG_IDENTIFIER=netdata`, `slice:false`: `data_len=20`, `rows_evaluated=2019`, `rows_matched=1499`, `rows_read=2019`.
519
-- Same-failure scan: `rg -n "twox_hash|XxHash64|jenkins_hash|Lookup3Hasher|hashers" src/crates/jf src/crates/journal-core src/crates/Cargo.toml src/crates/Cargo.lock` found no remaining `twox_hash`/`XxHash64` use in the legacy journal hash path, and found both journal readers using `Lookup3Hasher`.
520
-- `git diff --check`: passed.
521
-- `.agents/sow/audit.sh`: SOW status/directory checks passed for this SOW; audit still exits non-zero for the pre-existing public SSH clone syntax false positive in `.agents/skills/mirror-netdata-repos/SKILL.md:112`, unrelated to this work.
522
-
523
-Sensitive data handling:
524
-
525
-- Bearer tokens, Cloud token values, claim identifiers, node identifiers, raw journal rows, and private endpoint names were not written to this SOW.
526
-- Raw response JSON from live validation was kept under `.local/audits/`, which is gitignored and not staged.
527
-
528
-Artifact updates:
529
-
530
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
531
-- Runtime project skills: no update needed; the static-build skill worked as operational context and the fix did not change how agents should work on collectors.
532
-- Specs: no update needed; this restores the intended systemd-compatible journal hash behavior.
533
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
534
-- End-user/operator skills: no update needed; public/operator skill behavior was not changed by this PR.
535
-- SOW lifecycle: SOW is moved back to `.agents/sow/done/` with `Status: completed` in the same commit as the regression repair.
536
-
537
-Follow-up mapping:
538
-
539
-- No follow-up remains for the static journal facet filtering regression.
540
-
541
-## Core Library Extension - 2026-05-09
542
-
543
-Question answered:
544
-
545
-- The user asked whether the same compatibility-layer changes were also needed in the underlying `journal-core` library.
546
-
547
-Evidence:
548
-
549
-- `src/crates/journal-core/src/file/hash.rs:4-64` already had the systemd-compatible Jenkins lookup3 hash implementation and reference-value test from the regression repair.
550
-- `src/crates/journal-core/src/file/object.rs:1049-1103` already had bounded Zstd/LZ4/XZ DATA decompression support from the earlier PR review iterations.
551
-- `src/crates/journal-core/src/file/file.rs:39-79` still used the generic `PayloadMatcher`, which compared `object.raw_payload() == self.payload`.
552
-- `src/crates/journal-core/src/file/file.rs:485-488` used that raw-only matcher in `find_data_offset()`.
553
-- `src/crates/journal-core/src/file/filter.rs:283` and `src/crates/journal-core/src/file/filter.rs:297` build filter expressions through `find_data_offset()`, so compressed DATA objects could still fail filter construction in core-library users.
554
-
555
-Actions:
556
-
557
-- Added `DataPayloadMatcher` to `src/crates/journal-core/src/file/file.rs`, matching the `src/crates/jf` repair pattern.
558
-- Kept the raw-payload comparison as the first path for uncompressed DATA objects.
559
-- Added a compressed-payload comparison path that decompresses only compressed DATA objects in the selected hash bucket.
560
-- Changed `journal-core` `find_data_offset()` to use `DataPayloadMatcher`.
561
-- Added focused `journal-core` tests for direct LZ4 compressed matching, negative compressed matching, and the `find_data_offset()` hash-bucket traversal path.
562
-
563
-Validation:
564
-
565
-- `cargo fmt -p journal-core` in `src/crates`: passed.
566
-- `cargo test -q -p journal-core` in `src/crates`: passed; 27 tests passed plus the existing ignored tests.
567
-- `cargo test -q` in `src/crates/jf`: passed; 11 tests passed.
568
-- Same-failure scan: `rg -n "raw_payload\\(\\) == self\\.payload|DataPayloadMatcher|find_data_offset\\(|payload_matches" src/crates/jf src/crates/journal-core` confirmed both readers now use `DataPayloadMatcher` for DATA hash-bucket lookup; the remaining raw-payload comparison is the generic field-object matcher path, which is correct because FIELD objects are not compressed DATA payloads.
569
-
570
-Artifact updates:
571
-
572
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
573
-- Runtime project skills: no update needed; this is a code-path consistency fix, not a workflow change.
574
-- Specs: no update needed; this preserves the intended journal-format behavior.
575
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
576
-- End-user/operator skills: no update needed; public/operator skill behavior was not changed.
577
-- SOW lifecycle: same SOW reopened for the same-class core-library gap and moved back to `.agents/sow/done/` with `Status: completed` in the same commit.
578
-
579
-Follow-up mapping:
580
-
581
-- No follow-up remains for the underlying `journal-core` compressed DATA lookup gap.
582
-
583
-## External Review Follow-up - 2026-05-09
584
-
585
-Why reopened:
586
-
587
-- The user requested external review of the performance and side effects of the changes before merge.
588
-- Reviewers agreed the original static facet fix is correct, but flagged same-family raw-payload reads outside the original `find_data_offset()` lookup path.
589
-
590
-Confirmed findings:
591
-
592
-- `src/crates/journal-core/src/file/file.rs:595` read remapping entry DATA bytes with `raw_payload()` after the marker lookup had become compressed-aware.
593
-- `src/crates/journal-core/src/file/reader.rs:428` copied remapping entry DATA bytes with `raw_payload()`.
594
-- `src/crates/journal-index/src/field_types.rs:233` parsed source timestamp DATA bytes with `raw_payload()`, with callers in `src/crates/journal-index/src/file_index.rs:390` and `src/crates/journal-index/src/file_indexer.rs:437`.
595
-- The `hashers` crate contains suspicious optimized alignment branches, but local inspection showed its `offset_to_align()` helper never returns `0` for normal alignments, so the crate falls back to the byte path used by the passing reference-value tests and the live RHEL 8.10 validation. This is not a current PR blocker.
596
-
597
-Actions:
598
-
599
-- Added `DataObject::logical_payload()` in `src/crates/journal-core/src/file/object.rs`.
600
-- Updated `journal-core` remapping reads to use logical payload bytes and return `JournalError::InvalidField` instead of panicking on non-UTF-8 data.
601
-- Updated `journal-index` source timestamp parsing to use logical payload bytes, while preserving a scratch-buffer variant to avoid repeated allocations in loops.
602
-- Added focused `journal-core` tests for logical raw payloads and LZ4-compressed logical payloads.
603
-
604
-Validation:
605
-
606
-- `cargo fmt -p journal-core -p journal-index` in `src/crates`: passed.
607
-- `cargo test -q -p journal-core` in `src/crates`: passed; 29 tests passed plus the existing ignored doc tests.
608
-- `cargo test -q -p journal-index` in `src/crates`: passed; 66 tests passed across the package test binaries.
609
-
610
-Follow-up mapping:
611
-
612
-- Stale uncompiled `src/crates/jf/journal_file/src/journal_file.rs` still contains a raw-payload lookup helper with no callers. This is not runtime behavior for this PR and should be handled only if that stale module is removed or revived.
613
-- Zstd/XZ integration tests through `find_data_offset()` would improve coverage but do not block this RHEL LZ4 regression fix because object-level XZ and bounded streaming behavior are already tested, and the logical matcher delegates to the same decompression API.
614
-
615
-## External Review Rerun Closure - 2026-05-09
616
-
617
-Why updated:
618
-
619
-- The user asked to run the external reviewers again after the first follow-up fixes.
620
-- The rerun found no evidence that the live RHEL 8.10 static facet failure remained, but it did identify two local hardening issues in the same code path.
621
-
622
-Confirmed findings:
623
-
624
-- `src/crates/journal-core/src/file/file.rs:629` still used `expect("utf8 data")` for FIELD names in the same `load_fields()` path where DATA payload parsing had already been converted to `JournalError::InvalidField`.
625
-- `src/collectors/systemd-journal.plugin/provider/rust_provider.h:13` exposed unique enumeration through a foreach macro that stops on `<= 0`; when `rsd_journal_enumerate_available_unique()` returned a negative decompression error, `src/collectors/systemd-journal.plugin/systemd-journal.c:583` could silently stop without incrementing `failures`, so the fallback at `systemd-journal.c:621` would not run.
626
-- `src/crates/journal-index/src/file_index.rs:493` manually decompressed regex payloads even though `DataObject::logical_payload()` now exists.
627
-
628
-Reviewed and rejected as non-blocking for this SOW:
629
-
630
-- `src/crates/journal-index/src/file_indexer.rs:39-43` documents that compressed values are skipped by the bitmap indexer; `file_indexer.rs:296-309` implements that existing index-size policy. Changing it would require a separate product/performance decision because it would decompress and index every unique compressed value. This PR keeps the documented indexing limit unchanged.
631
-- A reviewer claimed the Zstd object flag should be `1 << 3`. That finding was false. Current systemd source has `OBJECT_COMPRESSED_ZSTD = 1 << 2` and `HEADER_INCOMPATIBLE_COMPRESSED_ZSTD = 1 << 3`; Netdata's object/header constants match that split. Evidence: `https://raw.githubusercontent.com/systemd/systemd/main/src/libsystemd/sd-journal/journal-def.h`, lines 62-64 and 186.
632
-- The stale uncompiled `src/crates/jf/journal_file/src/journal_file.rs` helper still has no runtime callers. It is rejected for this SOW because changing dead code would increase review surface without changing shipped behavior.
633
-- Zstd/XZ `find_data_offset()` integration tests and Zstd object-level tests are useful coverage, but they are rejected for this SOW because the production regression is the RHEL LZ4 path and the shared decompression paths are already covered by object-level LZ4/XZ and bounded-stream tests.
634
-
635
-Actions:
636
-
637
-- Converted the remaining FIELD-name panic in `journal-core` `load_fields()` to `JournalError::InvalidField`.
638
-- Added `nsd_journal_enumerate_available_unique()` to the provider abstraction.
639
-- Replaced the filter builder's `NSD_JOURNAL_FOREACH_UNIQUE()` macro use with an explicit restart/enumerate loop that counts negative `query_unique()` and enumeration returns as setup failures, preserving the existing full-query fallback.
640
-- Replaced `_BOOT_ID` annotation unique enumeration with the same explicit wrapper and logged negative enumeration returns.
641
-- Replaced the manual regex-path decompression in `journal-index` with `DataObject::logical_payload()`.
642
-
643
-Validation:
644
-
645
-- `curl -fsSL https://raw.githubusercontent.com/systemd/systemd/main/src/libsystemd/sd-journal/journal-def.h | rg -n "OBJECT_COMPRESSED_(XZ|LZ4|ZSTD)|HEADER_INCOMPATIBLE_COMPRESSED_ZSTD"`: verified object Zstd is `1 << 2` and header incompatible Zstd is `1 << 3`.
646
-- `cargo fmt -p journal-core -p journal-index` in `src/crates`: passed.
647
-- `git diff --check`: passed.
648
-- `cargo test -q -p journal-core` in `src/crates`: passed; 29 tests passed plus the existing ignored doc tests.
649
-- `cargo test -q -p journal-index` in `src/crates`: passed; 66 tests passed across the package test binaries.
650
-- `cargo test -q --all-targets` in `src/crates/jf`: passed; 11 tests passed.
651
-- `./packaging/makeself/build-static.sh x86_64`: passed; produced `artifacts/netdata-x86_64-latest.gz.run` after compiling `systemd-journal.plugin`, `journal_reader_ffi`, `journal-core`, and `journal-index` in the static musl build.
652
-- `.agents/sow/audit.sh`: SOW status/directory checks passed for this SOW; the audit still exits non-zero on the pre-existing public SSH clone syntax pattern in `.agents/skills/mirror-netdata-repos/SKILL.md:112`, unrelated to this work and not staged by this PR.
653
-
654
-Follow-up mapping:
655
-
656
-- No follow-up remains for the static journal facet filtering regression.
657
-
658
-## Libsystemd Compatibility Follow-up - 2026-05-09
659
-
660
-Why reopened:
661
-
662
-- The user asked whether the new explicit C call could break old `libsystemd` builds if older `SD_JOURNAL_FOREACH_UNIQUE()` macros did not use `sd_journal_enumerate_available_unique()`.
663
-
664
-Evidence:
665
-
666
-- systemd v245 `src/systemd/sd-journal.h` declares `sd_journal_enumerate_unique()` and defines `SD_JOURNAL_FOREACH_UNIQUE()` with `sd_journal_enumerate_unique()`.
667
-- systemd v246 `src/systemd/sd-journal.h` declares `sd_journal_enumerate_available_unique()` and defines `SD_JOURNAL_FOREACH_UNIQUE()` with `sd_journal_enumerate_available_unique()`.
668
-- The current systemd manual records `sd_journal_query_unique()`, `sd_journal_enumerate_unique()`, `sd_journal_restart_unique()`, and `SD_JOURNAL_FOREACH_UNIQUE()` as added in version 195; `sd_journal_enumerate_available_unique()` was added in version 246.
669
-
670
-Conclusion:
671
-
672
-- The user's concern was valid. The previous C wrapper would have broken builds against libsystemd headers older than v246 when `HAVE_SD_JOURNAL_RESTART_FIELDS` was set.
673
-
674
-Actions:
675
-
676
-- Added `HAVE_SD_JOURNAL_ENUMERATE_AVAILABLE_UNIQUE` detection in `packaging/cmake/Modules/NetdataDetectSystemd.cmake`.
677
-- Added the generated config define to `packaging/cmake/config.cmake.h.in`.
678
-- Updated `nsd_journal_enumerate_available_unique()` so:
679
- - Rust provider builds call `rsd_journal_enumerate_available_unique()`.
680
- - Modern libsystemd builds call `sd_journal_enumerate_available_unique()`.
681
- - Older libsystemd builds fall back to `sd_journal_enumerate_unique()`, matching the old `SD_JOURNAL_FOREACH_UNIQUE()` macro behavior.
682
-
683
-Validation:
684
-
685
-- systemd v245 header check: confirmed `SD_JOURNAL_FOREACH_UNIQUE()` uses `sd_journal_enumerate_unique()`.
686
-- systemd v246 header check: confirmed `SD_JOURNAL_FOREACH_UNIQUE()` uses `sd_journal_enumerate_available_unique()`.
687
-- Current systemd manual check: confirmed `sd_journal_enumerate_available_unique()` was added in version 246.
688
-- Compiled `src/collectors/systemd-journal.plugin/provider/netdata_provider.c` with a temporary config where `HAVE_SD_JOURNAL_RESTART_FIELDS` is defined and `HAVE_SD_JOURNAL_ENUMERATE_AVAILABLE_UNIQUE` is not defined: passed, proving the old-libsystemd branch compiles.
689
-- Compiled the same provider file with both `HAVE_SD_JOURNAL_RESTART_FIELDS` and `HAVE_SD_JOURNAL_ENUMERATE_AVAILABLE_UNIQUE` defined: passed, proving the modern-libsystemd branch compiles.
690
-- `git diff --check`: passed.
691
-
692
-Artifact updates:
693
-
694
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
695
-- Runtime project skills: no update needed; this is a compatibility guard in project code.
696
-- Specs: no update needed; this preserves existing libsystemd compatibility.
697
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
698
-- End-user/operator skills: no update needed; public/operator skill behavior was not changed.
699
-- SOW lifecycle: SOW reopened for this compatibility fix and will be moved back to done with `Status: completed` in the same commit.
700
-
701
-Follow-up mapping:
702
-
703
-- No follow-up remains for libsystemd unique-enumeration symbol compatibility.
704
-
705
-## PR Review Cleanup - 2026-05-09
706
-
707
-Why reopened:
708
-
709
-- The user reported that old PR comments/reviews were still unresolved.
710
-- Current PR review state showed three unresolved bot review threads.
711
-
712
-Findings:
713
-
714
-- `PRRT_kwDOAKPxd86AyMvL`, `src/crates/journal-core/src/file/object.rs:1085`: valid. Several `DataObject::decompress()` error paths can return without clearing the caller-provided scratch buffer, including short LZ4 prefix, oversized LZ4 prefix, LZ4 reserve failure, Zstd decoder creation failure, and unknown compression method.
715
-- `PRRT_kwDOAKPxd86AyMwE`, `src/crates/jf/journal_file/src/object.rs:971`: valid. The legacy static reader has the same scratch-buffer error-path issue.
716
-- `PRRT_kwDOAKPxd86Aye5J`, `src/crates/journal-index/src/field_types.rs:242`: valid. `get_timestamp_field()` scans all entry DATA objects for the configured timestamp field. A decompression failure in an unrelated compressed DATA object can abort the search instead of being treated like a non-match and allowing `get_entry_timestamp()` to fall back to the entry realtime timestamp.
717
-
718
-Planned actions:
719
-
720
-- Reset decompression scratch buffers to a new empty `Vec` on every `DataObject::decompress()` error path that occurs before the bounded stream reader or LZ4 decoder already clears it.
721
-- Add regression tests in both readers for short LZ4 prefixes and stale-buffer oversized prefixes.
722
-- Treat compressed-payload decompression failures as timestamp-field non-matches during timestamp parsing, while preserving other journal errors.
723
-- Run focused Rust validation and GitHub review sync before commit/push.
724
-
725
-Actions:
726
-
727
-- Changed `src/crates/journal-core/src/file/object.rs` and `src/crates/jf/journal_file/src/object.rs` so early decompression failures reset the scratch buffer to a new empty `Vec`.
728
-- Added short LZ4-prefix regression tests in both readers.
729
-- Strengthened the oversized LZ4-prefix tests in both readers to start with a stale buffer and assert capacity release.
730
-- Changed `src/crates/journal-index/src/field_types.rs` so `JournalError::DecompressorError` and `JournalError::UnknownCompressionMethod` become `IndexError::InvalidFieldPrefix` for timestamp parsing. Other journal errors still propagate.
731
-- Added a focused timestamp error-classification test.
732
-
733
-Validation:
734
-
735
-- `cargo fmt -p journal-core -p journal-index` in `src/crates`: passed.
736
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
737
-- `cargo test -q -p journal-core` in `src/crates`: passed; 30 tests passed plus existing ignored doc tests.
738
-- `cargo test -q -p journal-index` in `src/crates`: passed; 67 tests passed across package test binaries.
739
-- `cargo test -q --all-targets` in `src/crates/jf`: passed; 12 tests passed.
740
-- `git diff --check`: passed.
741
-- Same-failure scan: checked the affected decompression error returns and confirmed the remaining early returns in both `DataObject::decompress()` implementations clear the scratch buffer before returning; bounded-stream and LZ4 decoder failure paths already clear internally.
742
-- PR sync barrier: `fetch-all.sh 22456` still showed the same three unresolved threads and no newer unresolved threads.
743
-- Sonar sync barrier: `fetch-sonar-findings.sh 22456` reported 0 issues and 0 hotspots.
744
-- CI sync barrier: `ci-status.sh 22456` reported 0 failing checks and 94 running checks before this push.
745
-- `.agents/sow/audit.sh`: SOW status/directory checks passed; audit still exits non-zero on the pre-existing public SSH clone syntax pattern in `.agents/skills/mirror-netdata-repos/SKILL.md:112`, unrelated to this work and not staged by this PR.
746
-
747
-Artifact updates:
748
-
749
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
750
-- Runtime project skills: no update needed; this is a code review cleanup in an existing workflow.
751
-- Specs: no update needed; this preserves intended journal error resilience.
752
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
753
-- End-user/operator skills: no update needed; public/operator skill behavior was not changed.
754
-- SOW lifecycle: SOW reopened for unresolved PR comments and will be moved back to done with `Status: completed` in the same commit.
755
-
756
-Follow-up mapping:
757
-
758
-- No follow-up remains for these unresolved review comments.
759
-
760
-## PR Review Rerun Cleanup - 2026-05-09
761
-
762
-Why reopened:
763
-
764
-- Re-triggered PR review after commit `709857f261` opened three new bot review threads.
765
-
766
-Findings:
767
-
768
-- `PRRT_kwDOAKPxd86Aymui`, `src/crates/journal-core/src/file/file.rs:67`: valid. `DataPayloadMatcher::payload_matches()` propagated decompression failures, so one corrupt compressed DATA object in a hash bucket could stop lookup before a later matching object.
769
-- `PRRT_kwDOAKPxd86Aymuv`, `src/crates/jf/journal_file/src/file.rs:184`: valid. The legacy static reader had the same hash-bucket lookup resilience issue.
770
-- `PRRT_kwDOAKPxd86Aymu0`, `src/collectors/systemd-journal.plugin/systemd-journal-files.c:173`: valid. The log message named `sd_journal_enumerate_available_unique()` even though the call goes through `nsd_journal_enumerate_available_unique()`, which can fall back to older libsystemd APIs.
771
-
772
-Planned actions:
773
-
774
-- Treat DATA-object decompression failures in both payload matchers as non-matches so hash-bucket traversal continues.
775
-- Add bucket-level tests where a bad compressed DATA object precedes a valid matching compressed DATA object in the same bucket.
776
-- Update the C diagnostic to name the provider wrapper.
777
-- Re-run focused tests and PR sync before commit/push.
778
-
779
-Actions:
780
-
781
-- Changed both `DataPayloadMatcher::payload_matches()` implementations to return `Ok(false)` for `JournalError::DecompressorError` and `JournalError::UnknownCompressionMethod`.
782
-- Added bucket-level regression tests in both readers proving `find_data_offset()` skips a bad compressed DATA object and finds a later valid compressed DATA object in the same hash bucket.
783
-- Changed the `_BOOT_ID` annotation diagnostic to name `nsd_journal_enumerate_available_unique()`.
784
-
785
-Validation:
786
-
787
-- `cargo fmt -p journal-core` in `src/crates`: passed.
788
-- `cargo fmt -p journal_file -p journal_reader_ffi` in `src/crates/jf`: passed.
789
-- `cargo test -q -p journal-core` in `src/crates`: passed; 31 tests passed plus existing ignored doc tests.
790
-- `cargo test -q --all-targets` in `src/crates/jf`: passed; 13 tests passed.
791
-- `git diff --check`: passed.
792
-- Same-failure scan: `rg` found no remaining `decompress(&mut self.decompressed_payload)?` direct propagation and confirmed the diagnostic now names the wrapper.
793
-- PR sync barrier: `fetch-all.sh 22456` still showed the same three unresolved rerun threads and no newer unresolved threads.
794
-- Sonar sync barrier: `fetch-sonar-findings.sh 22456` reported 0 issues and 0 hotspots.
795
-- CI sync barrier: `ci-status.sh 22456` reported 0 failing checks and 94 running checks before this push.
796
-- `.agents/sow/audit.sh`: SOW status/directory checks passed; audit still exits non-zero on the pre-existing public SSH clone syntax pattern in `.agents/skills/mirror-netdata-repos/SKILL.md:112`, unrelated to this work and not staged by this PR.
797
-
798
-Artifact updates:
799
-
800
-- AGENTS.md: no update needed; workflow and project guardrails did not change.
801
-- Runtime project skills: no update needed; this is continued PR review cleanup.
802
-- Specs: no update needed; this preserves intended journal lookup resilience.
803
-- End-user/operator docs: no update needed; no user-facing configuration, command, or workflow changed.
804
-- End-user/operator skills: no update needed; public/operator skill behavior was not changed.
805
-- SOW lifecycle: SOW reopened for rerun review comments and moved back to done with `Status: completed` in the same commit.
806
-
807
-Follow-up mapping:
808
-
809
-- No follow-up remains for these rerun review comments.
.agents/sow/done/SOW-0017-20260516-query-duration-below-collection-frequency.md
deleted
-504
@@ -1,504 +0,0 @@
1
-# SOW-0017 - Query duration below collection frequency returns no data
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed on 2026-05-17 after implementation, installation, focused unit validation, and live direct-agent API validation.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Make Netdata metric queries fit for troubleshooting narrow historical windows: when data exists in the requested time area, querying a duration shorter than the metric collection frequency must not silently return an empty result only because the requested window sits between collection-aligned timestamps.
14
-
15
-### User Request
16
-
17
-The user reported a hypothesis:
18
-
19
-- Metrics such as SNMP or go.d charts are often collected every 10 seconds.
20
-- A 10-minute query with 60 points should show the 10-second samples exist.
21
-- A 5-second absolute query around 5 minutes in the past, fully inside one 10-second alignment interval, returns empty data even though nearby data exists.
22
-- The first step must verify the hypothesis before creating a SOW or implementing anything.
23
-
24
-### Assistant Understanding
25
-
26
-Facts:
27
-
28
-- Local Netdata Agent was reachable at `127.0.0.1:19999`.
29
-- Local Netdata Agent version was reported by `/api/v1/info` as `v2.10.0-199-g532b0ceef2`.
30
-- `/api/v3/contexts` does not expose collection `update_every`; it exposes context metadata, labels, dimensions, instances, retention, and liveness depending on options.
31
-- `/api/v3/contexts?contexts=smartctl.device_temperature&options=labels,instances,retention,liveness,minify` confirmed the selected context is collected by `_collect_plugin=go.d` and `_collect_module=smartctl`.
32
-- `/api/v1/context?context=smartctl.device_temperature&options=instances` confirmed instances of `smartctl.device_temperature` have `update_every=10`.
33
-- Direct local `/api/v3/data` in this checkout is query-string driven; POSTing the JSON body documented by the skill was ignored for local direct-agent calls.
34
-
35
-- The verified symptom is in automatic tier selection/planning, not in collector emission or storage execution, because forced `tier=0` and forced `tier=1` return data for the same narrow window.
36
-- The empty result is not caused by retention absence: tier 0 retention covers the tested timestamps.
37
-- The broader hypothesis that any tier-2-only query shorter than tier-2 resolution returns empty was not reproduced on the local agent. Tier-2-only 5-minute windows returned data for both `system.cpu` and `smartctl.device_temperature`.
38
-
39
-Unknowns:
40
-
41
-- Relative-window variants were not exhaustively tested. The traced trigger is based on the effective window after query-window normalization, so any request form that produces the same effective window can hit it.
42
-
43
-### Acceptance Criteria
44
-
45
-- Root cause is traced in code with file:line evidence before patching.
46
-- `/api/v3/data` returns meaningful data for a historical sub-frequency window when a sample exists in the surrounding collection interval, or a deliberate product decision explains why it should not.
47
-- Existing aligned 10-second queries for 10-second go.d metrics continue to return all expected points.
48
-- Tests cover the empty-result regression using a lower-frequency metric or a query-engine fixture.
49
-- Validation records exact commands/results for baseline, failing pre-fix case, and fixed post-fix case.
50
-- Public/end-user skill documentation is changed only if a user/operator workflow changes; internal query-planner findings stay in SOW/spec artifacts.
51
-
52
-## Analysis
53
-
54
-Sources checked:
55
-
56
-- `docs/netdata-ai/skills/query-netdata-agents/SKILL.md`
57
-- `docs/netdata-ai/skills/query-netdata-agents/query-metrics.md`
58
-- `src/web/api/v2/api_v2_contexts.c`
59
-- `src/database/contexts/api_v2_contexts.c`
60
-- `src/web/api/maps/contexts_options.c`
61
-- `src/database/contexts/api_v1_contexts.c`
62
-- `src/web/api/v2/api_v2_data.c`
63
-- `src/web/api/queries/query-window.c`
64
-- `src/web/api/queries/query-plan.c`
65
-- `src/database/contexts/query_target.c`
66
-- `src/database/contexts/rrdcontext.h`
67
-- Local Netdata Agent API at `127.0.0.1:19999`
68
-
69
-Current state:
70
-
71
-- v3 context metadata can identify go.d/SNMP-style collector ownership via labels, but not collection cadence.
72
-- v1 context metadata exposes per-instance `update_every`.
73
-- v3 data responses expose `db.per_tier[].update_every`, which is database tier resolution for the query, not general context metadata.
74
-- Direct local v3 data request parsing reads URL parameters in `api_v23_data_internal()`; the parser does not read the POST JSON body in the inspected path.
75
-
76
-Risks:
77
-
78
-- A query-window fix may affect all `/api/v2/data` and `/api/v3/data` users, including Cloud-proxied metrics queries.
79
-- A naive widening fix could return data outside the user's requested interval and surprise callers expecting strict time boundaries.
80
-- A strict no-widening interpretation preserves mathematical precision but makes sub-frequency historical inspection unreliable for real troubleshooting workflows.
81
-- Tier-selection behavior may differ across tier 0, tier 1, and tier 2, so tests must not cover only tier 0.
82
-
83
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
84
-
85
-Status: diagnosis complete for the verified local reproduction
86
-
87
-Problem / root-cause model:
88
-
89
-- Verified symptom: for a 10-second go.d metric, every tested 5-second absolute query across a full 10-second cadence cycle selects the right target objects but returns no data and performs zero tier queries, while adjacent/wider windows return data from tier 0.
90
-- Corrected phase split: query-target selects the matching context, instances, and dimension; the queryable metric / planner path does not create any tier plan for them.
91
-- Root-cause model: automatic per-metric tier selection gives every overlapping tier a sentinel "unusable" weight when the requested effective duration is shorter than the tier update interval. The same sentinel is also used for non-overlapping tiers. The `>=` tie-break then selects the highest numbered tier among tied sentinel weights.
92
-- Forced-tier controls prove the query engine can answer the exact narrow window when the planner is told to use a covering tier.
93
-- The empty result is only one outcome of the trigger. If the highest tied tier covers the requested window, the query returns data from that higher/coarser tier. If the highest tied tier does not cover the requested window, planning returns false and the response is empty.
94
-
95
-Evidence reviewed:
96
-
97
-- v3 collector identity:
98
- - Request: `/api/v3/contexts?contexts=smartctl.device_temperature&options=labels,instances,retention,liveness,minify`
99
- - Result: selected context has `_collect_plugin=go.d` and `_collect_module=smartctl`.
100
-- v1 update frequency:
101
- - Request: `/api/v1/context?context=smartctl.device_temperature&options=instances`
102
- - Result: `smartctl.device_temperature` instances include `update_every=10`.
103
-- Baseline 10-minute query:
104
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958670&before=1778959270&points=60&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
105
- - Result: `result.data` length was `60`; first timestamp `1778959270`; last timestamp `1778958680`; tier 0 `update_every=10`.
106
-- Failing 5-second sub-frequency query around 5 minutes in the past:
107
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958983&before=1778958988&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
108
- - Result: `result.data` length was `0`; `db.per_tier[0].queries=0`; tier 0 retention covered the window.
109
-- Same failing window with `points=1` and with default points also returned `result.data` length `0`.
110
-- Adjacent wider window:
111
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958980&before=1778958990&points=1&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
112
- - Result: `result.data` length was `1`, with a sample at timestamp `1778958990`.
113
-- Wider 30-second window:
114
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958970&before=1778959000&points=3&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
115
- - Result: `result.data` length was `3`, with timestamps `1778959000`, `1778958990`, and `1778958980`.
116
-- Full 10-second shifted 5-second-window scan:
117
- - Common request shape: `/api/v3/data?contexts=smartctl.device_temperature&after=<after>&before=<before>&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
118
- - `0-5`: `after=1778958980`, `before=1778958985` -> `result.data` length `0`, tier 0 `queries=0`.
119
- - `1-6`: `after=1778958981`, `before=1778958986` -> `result.data` length `0`, tier 0 `queries=0`.
120
- - `2-7`: `after=1778958982`, `before=1778958987` -> `result.data` length `0`, tier 0 `queries=0`.
121
- - `3-8`: `after=1778958983`, `before=1778958988` -> `result.data` length `0`, tier 0 `queries=0`.
122
- - `4-9`: `after=1778958984`, `before=1778958989` -> `result.data` length `0`, tier 0 `queries=0`.
123
- - `5-0`: `after=1778958985`, `before=1778958990` -> `result.data` length `0`, tier 0 `queries=0`.
124
- - `6-1`: `after=1778958986`, `before=1778958991` -> `result.data` length `0`, tier 0 `queries=0`.
125
- - `7-2`: `after=1778958987`, `before=1778958992` -> `result.data` length `0`, tier 0 `queries=0`.
126
- - `8-3`: `after=1778958988`, `before=1778958993` -> `result.data` length `0`, tier 0 `queries=0`.
127
- - `9-4`: `after=1778958989`, `before=1778958994` -> `result.data` length `0`, tier 0 `queries=0`.
128
-- Planner/debug output:
129
- - `options=debug` and `options=plan` both map to `RRDR_OPTION_DEBUG` in `src/web/api/maps/rrdr_options.c`.
130
- - v3 emits request/debug fields with `options=jsonwrap,debug`.
131
- - v3 emits per-metric `plans` only inside `detailed` output with debug, under `detailed.nodes...dimensions.<metric>.plans`.
132
- - The failing 5-second query has no selected dimensions, so there are no per-metric plans to print.
133
- - A nearby non-empty query (`after=1778958980`, `before=1778958990`, `points=1`, `options=jsonwrap,debug,details`) prints per-metric plans such as tier 0 `af=1778958985`, `bf=1778958995`.
134
-- Corrected interpretation from the failing debug response:
135
- - `summary` and `totals` prove the matching target objects were selected.
136
- - The failing response has no query-plan objects and `db.per_tier[0].queries=0`, proving no tier query was initialized.
137
- - Therefore the failure is after target-object selection and before storage execution.
138
-- Tier-2-only hypothesis check, using old data where only tier 2 covered the selected window:
139
- - Baseline tier-retention request: `/api/v3/data?contexts=system.cpu&after=0&before=0&points=1&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,debug&timeout=30000`
140
- - Result: `system.cpu` had tier 0 retention beginning at `1778940602`, tier 1 retention beginning at `1778817840`, and tier 2 retention beginning at `1778086800`.
141
- - One-hour tier-2-only baseline: `/api/v3/data?contexts=system.cpu&after=1778798400&before=1778802000&points=1&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
142
- - Result: `result.data` length was `1`; tier 2 `queries=153`; tier 0 and tier 1 `queries=0`.
143
- - Five-minute tier-2-only query: `/api/v3/data?contexts=system.cpu&after=1778800200&before=1778800500&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
144
- - Result: `result.data` length was `5`; timestamps were `1778800500`, `1778800440`, `1778800380`, `1778800320`, `1778800260`; tier 2 `queries=153`; tier 0 and tier 1 `queries=0`.
145
- - Five-second tier-2-only query: `/api/v3/data?contexts=system.cpu&after=1778800203&before=1778800208&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
146
- - Result: `result.data` length was `5`; timestamps were `1778800208`, `1778800207`, `1778800206`, `1778800205`, `1778800204`; tier 2 `queries=153`; tier 0 and tier 1 `queries=0`.
147
- - Five-minute tier-2-only query for the original go.d context: `/api/v3/data?contexts=smartctl.device_temperature&after=1778800200&before=1778800500&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
148
- - Result: `result.data` length was `5`; tier 2 `queries=18`; tier 0 and tier 1 `queries=0`.
149
- - Five-second tier-2-only query for the original go.d context: `/api/v3/data?contexts=smartctl.device_temperature&after=1778800203&before=1778800208&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
150
- - Result: `result.data` length was `5`; timestamps were `1778800208`, `1778800207`, `1778800206`, `1778800205`, `1778800204`; tier 2 `queries=18`; tier 0 and tier 1 `queries=0`.
151
- - Finding: the broad tier-2 hypothesis was not reproduced. Sub-resolution tier-2-only windows can return data when auto-selection lands on tier 2 and tier 2 covers the window.
152
-- Forced-tier controls for the original failing 5-second query:
153
- - Auto-tier failing request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958983&before=1778958988&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
154
- - Result: `result.data` length was `0`; no per-metric plans; all `db.per_tier[].queries=0`; tier 0 and tier 1 retention covered the window, tier 2 did not.
155
- - Forced tier 0 request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958983&before=1778958988&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&tier=0&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
156
- - Result: `result.data` length was `5`; tier 0 `queries=18`; per-metric plans used tier 0.
157
- - Forced tier 1 request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958983&before=1778958988&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&tier=1&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
158
- - Result: `result.data` length was `5`; tier 1 `queries=18`; per-metric plans used tier 1.
159
- - Forced tier 2 request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958983&before=1778958988&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&tier=2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
160
- - Result: `result.data` length was `0`; no per-metric plans; tier 2 did not cover the window.
161
-- Wrong-tier-but-not-empty control where tier 0, tier 1, and tier 2 all covered the narrow window:
162
- - Auto-tier request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778935003&before=1778935008&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
163
- - Result: `result.data` length was `5`; tier 2 `queries=7`; tier 0 and tier 1 `queries=0`; per-metric plans used tier 2.
164
- - Forced tier 0 request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778935003&before=1778935008&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&tier=0&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
165
- - Result: `result.data` length was `5`; tier 0 `queries=3`; per-metric plans used tier 0.
166
- - Finding: when all tiers overlap but the effective window is shorter than every tier update interval, auto-selection picks the highest tier instead of the most detailed covering tier. This may return data, but it is still the wrong tier choice for a narrow troubleshooting window.
167
-- Code evidence:
168
- - `src/database/contexts/query_target.c:325` to `src/database/contexts/query_target.c:328` uses broad retention matching to decide whether a metric can enter the query target.
169
- - `src/database/contexts/rrdcontext.h:721` to `src/database/contexts/rrdcontext.h:723` allows a two-`update_every` tolerance when checking retention overlap.
170
- - `src/web/api/queries/query-plan.c:30` to `src/web/api/queries/query-plan.c:36` returns `-LONG_MAX` for invalid/no-overlap tiers.
171
- - `src/web/api/queries/query-plan.c:44` to `src/web/api/queries/query-plan.c:52` also returns `-LONG_MAX` when `points_available <= 0`, which happens when the requested window is shorter than the tier update interval.
172
- - `src/web/api/queries/query-plan.c:90` to `src/web/api/queries/query-plan.c:99` marks non-overlapping tiers with the same `-LONG_MAX` value.
173
- - `src/web/api/queries/query-plan.c:108` to `src/web/api/queries/query-plan.c:112` chooses the later tier on equal weights because it uses `>=`.
174
- - `src/web/api/queries/query-plan.c:117` to `src/web/api/queries/query-plan.c:176` has a separate all-invalid fallback to tier 0 for natural update-every selection, but the per-metric selector at `src/web/api/queries/query-plan.c:57` to `src/web/api/queries/query-plan.c:115` does not have the same fallback.
175
- - `src/web/api/queries/query-plan.c:361` to `src/web/api/queries/query-plan.c:365` returns false if the selected tier does not cover the requested window, so no plan reaches `query_planer_initialize_plans()`.
176
-
177
-Trigger conditions:
178
-
179
-1. The request does not explicitly select a tier, so `query_metric_best_tier_for_timeframe()` is used.
180
-2. More than one storage tier exists.
181
-3. After `query_target_calculate_window()`, the effective planner window is shorter than the update interval of every tier that overlaps it:
182
- - for the verified failing request, the API request was `after=1778958983`, `before=1778958988`, `points=5`;
183
- - query-window normalized it to `after=1778958984`, `before=1778958988`, so the planner duration was `4` seconds for `5` output slots;
184
- - tier 0 update interval for the metric was `10` seconds, tier 1 was `600` seconds, and tier 2 was `36000` seconds.
185
-4. Because `points_available = (common_last_t - common_first_t) / db_update_every_s`, each overlapping tier with an update interval larger than the effective duration gets `points_available=0` and weight `-LONG_MAX`.
186
-5. Non-overlapping tiers also get weight `-LONG_MAX`.
187
-6. The tie-break uses `>=`, so the highest-numbered tied tier is selected.
188
-7. Outcome split:
189
- - if the selected highest tier covers the window, the query returns data from that tier;
190
- - if the selected highest tier does not cover the window, `query_plan()` returns false, no tier query is initialized, and the response is empty.
191
-
192
-Tier switching behavior:
193
-
194
-- `query_plan()` can switch tiers in both directions in the same query, but only around the initially selected tier and only to fill coverage gaps.
195
-- If the selected tier starts after the requested `after`, the planner searches higher-numbered tiers (`selected_tier + 1` upward) to fill the older/beginning part of the query.
196
-- If the selected tier ends before the requested `before`, the planner searches lower-numbered tiers (`selected_tier - 1` downward) to fill the newer/end part of the query.
197
-- Both checks are independent, so a middle selected tier can have coarser plans prepended and finer plans appended in the same query.
198
-- The plan entries are sorted by start time before execution.
199
-- Limitation: switching is not a per-segment resolution optimizer. If the selected tier covers the whole requested window, the planner does not split the query just because another tier would provide better point density for a subsection.
200
-- Limitation: explicit `tier=` disables switching because `switch_tiers=false`.
201
-- Limitation relevant to the verified bug: the selected tier must overlap the requested window before switching can help. If automatic selection picks a non-overlapping tier, `query_plan()` returns false before the gap-filling loops run.
202
-- Code evidence:
203
- - `src/web/api/queries/query-plan.c:367` to `src/web/api/queries/query-plan.c:370` starts with one selected-tier plan clipped to that tier's retention.
204
- - `src/web/api/queries/query-plan.c:377` to `src/web/api/queries/query-plan.c:408` fills the beginning by scanning higher-numbered tiers.
205
- - `src/web/api/queries/query-plan.c:411` to `src/web/api/queries/query-plan.c:445` fills the end by scanning lower-numbered tiers.
206
- - `src/web/api/queries/query-plan.c:448` to `src/web/api/queries/query-plan.c:450` sorts multiple plan entries by start time.
207
- - `src/web/api/queries/query-plan.c:349` to `src/web/api/queries/query-plan.c:353` disables switching for explicit selected-tier requests.
208
-
209
-Related tier-selection functions:
210
-
211
-- `query_metric_best_tier_for_timeframe()` is the execution planner selector. It is `static` and is called only by `query_plan()` when no explicit `tier=` is selected.
212
-- This execution selector already partially matches the desired model:
213
- - it excludes zero-overlap tiers before scoring;
214
- - it computes `min_first_time_s` and `max_last_time_s` across all tiers;
215
- - it passes that union range plus each tier's `db_update_every_s` into `query_plan_points_coverage_weight()`;
216
- - therefore each overlapping tier is effectively scored for resolution as if it could cover the full union/requested window.
217
-- The verified bug is in the scoring and tie-break, not in the gap-fill planner:
218
- - sub-resolution tiers collapse to `-LONG_MAX`;
219
- - the `+25000 * tier` bias and `>=` tie-break favor higher/coarser tiers.
220
-- `rrddim_find_best_tier_for_timeframe()` is a separate aggregate helper in the same file. It also uses `query_plan_points_coverage_weight()`, but it is not the execution planner selector.
221
-- The only call to `rrdset_find_natural_update_every_for_timeframe()` is from `query_target_calculate_window()` when `natural_points`, explicit `selected-tier`, and `tier > 0` are all set. Because the same `selected-tier` option is passed through, the aggregate `rrddim_find_best_tier_for_timeframe()` branch is not reached from the current code path.
222
-- Implication: changing the shared `query_plan_points_coverage_weight()` is less surgical because it also changes the aggregate helper if that helper becomes reachable later. Changing `query_metric_best_tier_for_timeframe()` is the narrow execution-planner change.
223
-- API/data paths using automatic execution planner selection:
224
- - `/api/v1/data`, `/api/v2/data`, and `/api/v3/data` when no explicit `tier=` is provided;
225
- - MCP metric queries when no explicit `tier` parameter is provided;
226
- - weights queries and value helper paths that call `rrd2rrdr()` without `RRDR_OPTION_SELECTED_TIER`.
227
-- Paths verified not to depend on automatic tier selection:
228
- - health database lookups pass `points=1` and `RRDR_OPTION_SELECTED_TIER` with tier `0`;
229
- - exporting reads storage tier 0 directly through `storage_engine_query_init(rd->tiers[0]...)`;
230
- - explicit API/MCP `tier=` requests bypass automatic selection and disable tier switching.
231
-
232
-Why the aggregate helper path exists:
233
-
234
-- Historical evidence: commit `3fefd03b94458c9f6ad4164a82b1da6fc4fa435c` (`automatic selection of tier`) introduced automatic tier selection in the old single-chart query engine.
235
-- In that first design, `rrddim_find_best_tier_for_timeframe()` served two real purposes:
236
- - it selected the execution tier for `rrd2rrdr_do_dimension()`;
237
- - it selected the natural-points `update_every` through `rrdset_find_natural_update_every_for_timeframe()` whenever natural points and multiple storage tiers were available.
238
-- Historical evidence: commit `41e14c83e22ed54c8e48a7b637315bbb556c3185` (`natural points should only be offered on tier 0, except a specific tier is selected`) changed the natural-points call site from `natural_points && storage_tiers > 1` to `natural_points && selected-tier && tier > 0 && storage_tiers > 1`.
239
-- That change made the automatic natural-points branch effectively unreachable, because `rrdset_find_natural_update_every_for_timeframe()` is now only called when selected-tier is already set, and therefore it returns the explicit tier instead of calling `rrddim_find_best_tier_for_timeframe()`.
240
-- Commit `00712b351b3c83a54a147ca23365458acbef3105` (`QUERY_TARGET: new query engine for Netdata Agent`) ported the logic into the query-target engine:
241
- - execution-tier selection became `query_metric_best_tier_for_timeframe()`;
242
- - aggregate natural update-every selection remained as `rrddim_find_best_tier_for_timeframe()` behind `rrdset_find_natural_update_every_for_timeframe()`.
243
-- Implication: the other path exists because it is legacy from the original natural-points auto-tier design and was preserved through the query-target port, but the current call site no longer exercises its automatic branch.
244
-- The user's concern is valid: if a future caller exercises the aggregate helper, it does not have the execution planner's gap-fill semantics. It chooses one tier/update_every for the query window, so coverage-sensitive scoring there could produce poor natural-points granularity decisions unless it is reviewed separately.
245
-
246
-Affected contracts and surfaces:
247
-
248
-- `/api/v2/data` and `/api/v3/data` query behavior.
249
-- Cloud-proxied metrics queries, if they use the same agent-side query path.
250
-- Query target preparation, storage tier selection, retention matching, and data grouping semantics.
251
-- Public AI skills under `docs/netdata-ai/skills/` are affected only if the user/operator query workflow changes. Internal planner diagnostics are out of scope for those artifacts.
252
-
253
-Existing patterns to reuse:
254
-
255
-- Existing query target and data API test patterns after they are located.
256
-- Existing `RRDR_OPTION_UNALIGNED` behavior must be preserved.
257
-- Existing v2/v3 data query-string parser in `src/web/api/v2/api_v2_data.c`.
258
-
259
-Risk and blast radius:
260
-
261
-- High enough to require focused tests before patching: data query semantics are user-facing and shared across dashboards, APIs, Cloud, and troubleshooting workflows.
262
-- Security risk is low for the bug itself; evidence must still avoid raw secrets and sensitive label values.
263
-- Performance risk exists if a fix expands storage scans for many narrow queries.
264
-
265
-Sensitive data handling plan:
266
-
267
-- Durable artifacts must not include raw secrets, bearer tokens, SNMP communities, customer identifiers, private endpoints, non-private customer-identifying IPs, personal data, or device serial numbers.
268
-- The verification API responses contained hardware labels. This SOW records only sanitized collector identity and metric names, not raw serial numbers or full hardware-identifying label values.
269
-- Future logs or traces added to this SOW must be summarized or redacted before being written.
270
-
271
-Implementation plan:
272
-
273
-1. Choose the tier-selection fix semantics before patching.
274
-2. Implement the smallest fix in the automatic per-metric tier selector so a covering tier is selected for sub-resolution windows.
275
-3. Add regression tests for sub-frequency historical windows, forced-tier behavior, tier-2-only sub-resolution windows, and unchanged aligned-window behavior.
276
-4. Re-run the exact local API verification commands after the fix.
277
-5. Update query skill/docs if the local direct-agent `/api/v3/data` request contract differs from the current skill text.
278
-
279
-Validation plan:
280
-
281
-- Run targeted query-engine/data API tests found during tracing.
282
-- Add a regression test that fails before the fix.
283
-- Re-run the exact local API verification commands after the fix.
284
-- Search for same-failure risks around tier 1/tier 2, relative windows, absolute windows, `points=0`, `points=1`, and `points>duration`.
285
-
286
-Artifact impact plan:
287
-
288
-- AGENTS.md: likely unaffected unless the fix exposes a durable project-wide query workflow rule.
289
-- Runtime project skills: likely unaffected unless a codebase workflow lesson emerges.
290
-- Specs: likely add or update a query/data API spec if no existing spec covers sub-frequency windows.
291
-- End-user/operator docs: possibly affected if public query semantics are documented.
292
-- End-user/operator skills: no planned update for internal query-planner diagnostics. The user clarified that `docs/netdata-ai/skills/` is for user/operator how-tos, not maintainer implementation notes.
293
-- SOW lifecycle: this SOW moved to `current/` after the user explicitly prioritized it, and will move to `done/` with `Status: completed` when committed.
294
-
295
-Open-source reference evidence:
296
-
297
-- None checked. This is an internal Netdata query-engine/API behavior issue; external observability implementations are not needed until a semantic design fork appears.
298
-
299
-Open decisions:
300
-
301
-- None before implementation.
302
-
303
-## Implications And Decisions
304
-
305
-- User decision already applied: verify the hypothesis before creating this SOW.
306
-- User decision on 2026-05-17: choose option `1A`, cleanup the dormant natural-points aggregate tier-selection path instead of leaving it commented or reactivating it.
307
-- Implication: implementation should remove or simplify the unreachable `rrddim_find_best_tier_for_timeframe()` path so future maintainers do not confuse it with the execution planner selector.
308
-- User decision on 2026-05-17: implement the planner change in addition to cleanup.
309
-- User-approved planner semantics: zero-overlap tiers are non-candidates; among overlapping tiers, score point density as if the tier had full-window coverage; select the sparsest tier that can provide at least 50% of requested point density; if no tier can provide 50%, select the densest overlapping tier.
310
-- Boundary: existing explicit `tier=` semantics remain unchanged.
311
-- User clarification on 2026-05-17: do not put internal/developer query-planner findings in `docs/netdata-ai/skills/`; use `.agents/sow/` specs or project runtime skills for maintainer-facing memory.
312
-
313
-## Plan
314
-
315
-1. Keep unrelated SOWs unchanged and complete the SOW-0017 lifecycle.
316
-2. Remove or simplify the dormant natural-points aggregate tier-selection path.
317
-3. Replace automatic execution-tier scoring with the user-approved point-density semantics.
318
-4. Add focused regression tests or equivalent validation.
319
-5. Re-run local API reproduction URLs after installing/restarting a patched agent, or record the approval blocker.
320
-6. Update internal artifacts before close.
321
-
322
-## Execution Log
323
-
324
-### 2026-05-16
325
-
326
-- Verified the symptom using local direct-agent API requests.
327
-- Created this pending SOW after verification only.
328
-- Verified that tier-2-only sub-resolution windows return data in the tested cases, so the broader tier-2 hypothesis was not reproduced.
329
-- Traced the observed empty result to automatic tier selection choosing a non-covering tier when all tier weights tie at `-LONG_MAX`.
330
-- Verified a second outcome of the same trigger: when all tiers cover a narrow sub-resolution window, automatic selection chooses tier 2 and returns data from tier 2 instead of tier 0.
331
-- No code changes made.
332
-
333
-### 2026-05-17
334
-
335
-- Worked SOW-0017 after the user explicitly prioritized it; unrelated SOWs are outside this SOW's scope.
336
-- Removed the dormant automatic aggregate-tier path:
337
- - deleted `query_plan_points_coverage_weight()`;
338
- - deleted `rrddim_find_best_tier_for_timeframe()`;
339
- - replaced `rrdset_find_natural_update_every_for_timeframe()` with `query_target_min_update_every_for_tier()`, which only computes the minimum update-every for an explicit selected tier.
340
-- Implemented automatic execution-tier selection using the approved density model:
341
- - zero-overlap tiers get sentinel weight and cannot win;
342
- - candidate tiers use fixed-point point-density weights, so sub-resolution windows do not collapse to zero;
343
- - if any tier can provide at least 50% of requested point density, the sparsest acceptable tier wins;
344
- - if none can provide 50%, the densest overlapping tier wins.
345
-- Hardened the automatic selector to treat reversed or zero-duration windows as invalid before adjusting `points_wanted`.
346
-- Split pure plan-entry building from storage query initialization so unit tests can assert tier selection and head/tail gap-filling without mocking storage engines.
347
-- Added `query_plan_unittest()` and `-W queryplantest` coverage for:
348
- - sub-resolution window with a non-overlapping coarser tier;
349
- - sub-resolution window where all tiers overlap;
350
- - 50% tolerance choosing a sparse acceptable tier;
351
- - exact 50% tolerance boundary;
352
- - under-resolution requests choosing the densest tier;
353
- - zero-overlap tiers not being candidates;
354
- - invalid duration returning the first working tier;
355
- - selected tier covering the full window with one plan;
356
- - coarser-tier head gap fill;
357
- - finer-tier tail gap fill;
358
- - simultaneous head and tail gap fill;
359
- - explicit selected tier disabling gap fill;
360
- - no-overlap planning failure;
361
- - selected-tier natural-points update-every cleanup.
362
-- Added internal spec `.agents/sow/specs/query-planner-tier-selection.md`.
363
-- Did not update `docs/netdata-ai/skills/`; those are end-user/operator skills and this work is maintainer/internal query-planner behavior.
364
-- Ran `./install.sh` after explicit user request, and reran it after the final selector guard. Result: install completed and restarted the local `netdata`; non-fatal `git fetch -t` failed due local GitHub SSH permission, but the installer continued and completed.
365
-- Verified the installed local agent reports `v2.10.0-215-ge6e45f29ee` at `/api/v1/info`.
366
-- Ran `/usr/sbin/netdata -W queryplantest`; result: passed all six focused query planner checks.
367
-- Re-ran the live direct-agent `/api/v3/data` baseline and short-window checks against the installed agent. The original failing 5-second automatic-tier query now returns 5 points and uses tier 0.
368
-
369
-## Validation
370
-
371
-Acceptance criteria evidence:
372
-
373
-- Root cause was traced before patching and is recorded in the Pre-Implementation Gate with code evidence.
374
-- The implementation changes `src/web/api/queries/query-plan.c` so sub-resolution overlapping tiers remain candidates with fractional density weights instead of being assigned the same sentinel as non-overlapping tiers.
375
-- Explicit tier behavior remains unchanged: `query_plan()` still disables tier switching for valid `RRDR_OPTION_SELECTED_TIER`.
376
-- Live post-fix API validation against the installed local agent passed. The original failing automatic-tier 5-second query now returns 5 points and initializes tier 0 storage queries.
377
-
378
-Tests or equivalent validation:
379
-
380
-- `git diff --check -- src/web/api/queries/query-plan.c src/web/api/queries/query-window.c src/web/api/queries/query-internal.h src/daemon/main.c`
381
- - Result: passed.
382
-- `cmake --build build-clion --target netdata -j 8`
383
- - Result: configure/build did not reach the changed code. CMake attempted to refresh bundled Sentry crashpad content and failed fetching `mini_chromium` from the external submodule with HTTP 400.
384
-- `cmake -S . -B .local/build-sow17 -G Ninja -DENABLE_SENTRY=OFF -DENABLE_ML=OFF -DCMAKE_BUILD_TYPE=Debug`
385
- - Result: failed because the fresh cache enabled `ENABLE_PLUGIN_XENSTAT=ON` and local `xenstat` dependencies were not available.
386
-- `cmake -S . -B .local/build-sow17 -G Ninja -DENABLE_SENTRY=OFF -DENABLE_ML=OFF -DENABLE_PLUGIN_XENSTAT=OFF -DCMAKE_BUILD_TYPE=Debug`
387
- - Result: passed.
388
-- `cmake --build .local/build-sow17 --target netdata -j 8`
389
- - Result: passed. The changed files `src/daemon/main.c`, `src/web/api/queries/query-window.c`, and `src/web/api/queries/query-plan.c` compiled and linked into `.local/build-sow17/netdata`.
390
-- `.local/build-sow17/netdata -W queryplantest`
391
- - Result: passed all six focused query planner checks.
392
-- `cmake --build .local/build-sow17 --target netdata -j 8` after extracting pure plan-entry building and expanding planner coverage
393
- - Result: passed; `src/web/api/queries/query-plan.c` rebuilt and linked.
394
-- `.local/build-sow17/netdata -W queryplantest` after extracting pure plan-entry building and expanding planner coverage
395
- - Result: passed all fourteen focused query planner checks.
396
-- `cmake --build .local/build-sow17 --target netdata -j 8` after the final selector guard
397
- - Result: passed; `src/web/api/queries/query-plan.c` rebuilt and linked.
398
-- `.local/build-sow17/netdata -W queryplantest` after the final selector guard
399
- - Result: passed all six focused query planner checks.
400
-- `./install.sh`
401
- - Result: passed twice; local `netdata` was restarted. A non-fatal `git fetch -t` step failed due local GitHub SSH permission, but the install completed both times.
402
-- `curl -sS 'http://127.0.0.1:19999/api/v1/info' | jq -r '.version'`
403
- - Result: `v2.10.0-215-ge6e45f29ee`.
404
-- `/usr/sbin/netdata -W queryplantest`
405
- - Result: passed all six focused query planner checks.
406
-- `/usr/sbin/netdata -W queryplantest` after the final reinstall
407
- - Result: passed all six focused query planner checks.
408
-
409
-Real-use evidence:
410
-
411
-- Pre-fix local direct-agent `/api/v3/data` calls reproduced the symptom against `smartctl.device_temperature`.
412
-- Post-fix 10-minute baseline:
413
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778958670&before=1778959270&points=60&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned&timeout=30000`
414
- - Result: `result.data` length `60`; first timestamp `1778959270`; last timestamp `1778958680`; tier 0 `queries=18`, `update_every=10`; tier 1 and tier 2 `queries=0`.
415
-- Post-fix original failing 5-second automatic-tier query:
416
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000&after=1778958983&before=1778958988&points=5`
417
- - Result: `result.data` length `5`; timestamps `1778958988,1778958987,1778958986,1778958985,1778958984`; tier 0 `queries=18`; tier 1 and tier 2 `queries=0`.
418
-- Post-fix forced tier 0 control for the same 5-second window:
419
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000&after=1778958983&before=1778958988&points=5&tier=0`
420
- - Result: `result.data` length `5`; timestamps `1778958988,1778958987,1778958986,1778958985,1778958984`; tier 0 `queries=18`; tier 1 and tier 2 `queries=0`.
421
-- Post-fix shifted 5-second automatic-tier scan across one 10-second cadence interval:
422
- - Common request shape: `/api/v3/data?contexts=smartctl.device_temperature&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000&after=<after>&before=<before>&points=5`
423
- - `0-5`: `after=1778958980`, `before=1778958985` -> `result.data` length `5`, tier 0 `queries=18`.
424
- - `1-6`: `after=1778958981`, `before=1778958986` -> `result.data` length `5`, tier 0 `queries=18`.
425
- - `2-7`: `after=1778958982`, `before=1778958987` -> `result.data` length `5`, tier 0 `queries=18`.
426
- - `3-8`: `after=1778958983`, `before=1778958988` -> `result.data` length `5`, tier 0 `queries=18`.
427
- - `4-9`: `after=1778958984`, `before=1778958989` -> `result.data` length `5`, tier 0 `queries=18`.
428
- - `5-0`: `after=1778958985`, `before=1778958990` -> `result.data` length `5`, tier 0 `queries=18`.
429
- - `6-1`: `after=1778958986`, `before=1778958991` -> `result.data` length `5`, tier 0 `queries=18`.
430
- - `7-2`: `after=1778958987`, `before=1778958992` -> `result.data` length `5`, tier 0 `queries=18`.
431
- - `8-3`: `after=1778958988`, `before=1778958993` -> `result.data` length `5`, tier 0 `queries=18`.
432
- - `9-4`: `after=1778958989`, `before=1778958994` -> `result.data` length `5`, tier 0 `queries=18`.
433
-- Post-fix older two-tier overlap check:
434
- - Request: `/api/v3/data?contexts=smartctl.device_temperature&after=1778935003&before=1778935008&points=5&group_by=dimension&aggregation=average&time_group=average&format=json2&options=jsonwrap,minify,unaligned,debug,details&timeout=30000`
435
- - Result: `result.data` length `5`; tier 0 `queries=0`, `first_entry=1778948390`; tier 1 `queries=18`, `update_every=600`; tier 2 `queries=0`, `last_entry=1778940000`.
436
- - Interpretation: after install/restart, tier 0 and tier 2 no longer overlap on the local database, so the earlier all-three-overlap control is no longer reproducible. This check validates that when only tier 1 and tier 2 overlap a sub-resolution window, the denser overlapping tier 1 wins.
437
-
438
-Reviewer findings:
439
-
440
-- Self-review found one additional hardening point: the automatic selector should treat reversed windows the same as zero-duration windows before adjusting `points_wanted`. This guard was added and revalidated.
441
-- External AI reviewers were not run; the user did not request them for this PR.
442
-
443
-Same-failure scan:
444
-
445
-- Source-code search found no remaining references to `query_plan_points_coverage_weight()`, `rrddim_find_best_tier_for_timeframe()`, or `rrdset_find_natural_update_every_for_timeframe()`.
446
-- Automatic execution-tier selection call sites remain centralized through `query_metric_best_tier_for_timeframe()`.
447
-
448
-Sensitive data gate:
449
-
450
-- This SOW records no raw secrets, bearer tokens, SNMP communities, private endpoints, customer data, personal names, or device serial numbers.
451
-- Hardware-identifying labels observed during verification were not copied into this durable artifact.
452
-
453
-Artifact maintenance gate:
454
-
455
-- AGENTS.md: no update. Existing artifact-boundary instructions already distinguish internal specs/project skills from public end-user/operator skills.
456
-- Runtime project skills: no update. This change adds project behavior, not a new "how to work here" workflow.
457
-- Specs: added `.agents/sow/specs/query-planner-tier-selection.md`.
458
-- End-user/operator docs: no update. The behavior is internal planner selection; no user-facing API parameter or documented workflow changed.
459
-- End-user/operator skills: no update. Public skills under `docs/netdata-ai/skills/` are for user/operator work, not maintainer debugging notes.
460
-- SOW lifecycle: `Status: completed` and file move to `.agents/sow/done/` are part of this commit.
461
-
462
-Specs update:
463
-
464
-- Added `.agents/sow/specs/query-planner-tier-selection.md`.
465
-
466
-Project skills update:
467
-
468
-- No project skill update needed; no durable assistant workflow changed.
469
-
470
-End-user/operator docs update:
471
-
472
-- No end-user/operator docs update needed; the user-visible API contract did not gain a new parameter or required workflow.
473
-
474
-End-user/operator skills update:
475
-
476
-- No end-user/operator skill update needed; this is not a public/operator AI skill workflow.
477
-
478
-Lessons:
479
-
480
-- Internal query-planner diagnostics and maintainer implementation notes belong in the SOW/spec layer, not under `docs/netdata-ai/skills/`.
481
-- The direct-agent public skills may still need separate product-doc review, but that is not part of this internal planner SOW unless a user/operator workflow changes.
482
-
483
-Follow-up mapping:
484
-
485
-- No remaining behavioral follow-up identified.
486
-
487
-## Outcome
488
-
489
-Completed. Automatic tier selection now uses overlapping-tier point density, so sub-resolution query windows pick the densest overlapping tier instead of collapsing overlapping and non-overlapping tiers to the same sentinel score.
490
-
491
-## Lessons Extracted
492
-
493
-- Sub-resolution query windows must be ordered by fractional point density, not by integer point counts, because integer division collapses valid overlapping tiers to the same sentinel value as non-overlapping tiers.
494
-- Internal maintainer findings belong in `.agents/sow/` specs and SOWs; public skills under `docs/netdata-ai/skills/` should only change for user/operator workflows.
495
-
496
-## Followup
497
-
498
-No behavioral follow-up identified.
499
-
500
-## Regression Log
501
-
502
-None yet.
503
-
504
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0020-20260505-network-connections-topology-cloud-errors.md
deleted
-1358
@@ -1,1358 +0,0 @@
1
-# SOW-0020 - Network-connections topology Cloud errors
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed after the compact topology schema migration, producer migrations, validation fixtures, and the targeted network-connections graph-invariant repair for floating endpoint actors.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Make `topology:network-connections` reliable enough for production Cloud topology use in the target Cloud space. The Function must not crash or disconnect mid-response, must report missing/unavailable cases clearly, and must not produce disconnected floating endpoint actors unless the graph semantics explicitly allow them.
14
-
15
-The expanded purpose is to define and implement a lossless compact `detailed` topology payload contract and an `aggregated` topology view path so large topologies can be used by the UI and by Cloud cross-node correlation without exceeding practical payload limits.
16
-
17
-### User Request
18
-
19
-The user reported three Cloud network-connections topology issues:
20
-
21
-- One node returns HTTP 503 while data appears to start arriving; browser console shows `AgentError` with message `The plugin that was servicing this request, exited before responding.`
22
-- Another node returns HTTP 404 for the same `topology:network-connections` Function request.
23
-- A previously observed UI state showed orphan endpoint actors floating in the topology, but the user cannot reproduce it now.
24
-
25
-The user then expanded the required solution:
26
-
27
-- Update the Function schema for the new topology payload contract.
28
-- Add/declare an `aggregated` vs `detailed` request mode for `topology:network-connections`; default behavior is `aggregated`.
29
-- Update all topology Functions to emit the new schema: `topology:network-connections`, `topology:streaming`, and `topology:snmp`.
30
-- Update the Cloud frontend to support deployed compatibility payloads and the new schema. Compatibility aggregation must stay isolated so it is easy to delete after supported Agents emit the new schema.
31
-- Add a separate Cloud topology aggregation microservice. It consumes detailed topology payloads, aggregates topologies, and returns the aggregation requested by the user.
32
-
33
-### Assistant Understanding
34
-
35
-Facts:
36
-
37
-- The failing Function name is `topology:network-connections`.
38
-- The first observed Cloud request returned HTTP 503 and the browser console reported an agent-side plugin exit before response completion.
39
-- The second observed Cloud request returned HTTP 404 for the same Function name and selections.
40
-- `topology:network-connections` is registered by the C network-viewer plugin at `src/collectors/network-viewer.plugin/network-viewer.c`.
41
-- Raw browser cookies and session tokens were present in the chat. They must not be copied into durable artifacts.
42
-
43
-Inferences:
44
-
45
-- The 503 is likely not a pure frontend cancellation because the console includes an explicit agent/plugin error. The early `Request was cancelled` stack may be a frontend request lifecycle side effect, but it is not the strongest failure signal.
46
-- The 404 may mean the target node does not expose the Function, the node is stale/unreachable through Cloud, the function list differs by agent/plugin version, or Cloud maps an agent-side function-missing condition to 404.
47
-- The orphan endpoint symptom may be a graph-construction invariant violation: actors emitted without at least one emitted link, or links filtered/dropped after actor creation.
48
-
49
-Unknowns:
50
-
51
-- Whether the network-viewer plugin process exits due to a crash, fatal(), timeout/cancellation path, memory pressure, malformed topology parameters, or Cloud transport disconnect.
52
-- Whether the 404 node lacks the Function, is stale, runs an older agent, or is behind a Cloud routing/state issue.
53
-- Whether the orphan endpoints were produced by agent data, Cloud/frontend filtering, or a race between partial data and final error handling.
54
-
55
-### Acceptance Criteria
56
-
57
-- The 503 path is reproduced or falsified through token-safe Cloud or direct-agent wrappers, with status code, response class, and sanitized failure evidence recorded.
58
-- The 404 path is explained with node state/function availability/version evidence.
59
-- The serving code path and any crash-prone branches for the reported selections are traced with file:line evidence.
60
-- If a code fix is needed, it has focused tests or an equivalent validation path, and the validation includes a same-failure search for orphan actors.
61
-- The new topology schema preserves the canonical information needed by Cloud aggregation and UI rendering. Compatibility projection for deployed payload parity is test/transition code only and is not part of the production payload.
62
-- The new aggregated topology schema/view is explicitly derived from detailed topology data and is not treated as a correlation source of truth.
63
-- `topology:network-connections`, `topology:streaming`, and `topology:snmp` all emit the new topology schema.
64
-- `topology:network-connections` exposes an `aggregated`/`detailed` request mode in the Function schema, with `aggregated` as the default.
65
-- The Cloud frontend supports deployed compatibility payloads and the new schema, with compatibility aggregation isolated in removable code.
66
-- The Cloud topology aggregation microservice has a defined API contract and validation showing it can consume detailed topology payloads and return requested aggregated views.
67
-- The detailed payload contract supports multiple actor scopes for Cloud aggregation, including node-level, process-name-level, PID-level, container/application-level, and Kubernetes label/workload-level grouping when the required enrichment evidence is available.
68
-- Topology actor drilldown modals must continue to support per-actor tables listing exact dependencies and supporting details. These tables must avoid duplicating the same socket/link evidence under multiple actors where compact references or derived views can provide the same drilldown.
69
-- Topology actor tables may also carry actor-owned custom data that is not relationship evidence and is not generally aggregatable, such as streaming topology `streaming_path` and retention/status tables. The schema must distinguish relationship/evidence tables from actor-detail/custom tables.
70
-- Link direction semantics must be explicit. Some topology link types are direction-significant and must aggregate/render as directed relationships, while others are undirected adjacencies where any `direction` field describes observation completeness or evidence state rather than graph direction.
71
-- Topology links must be able to expose refreshable traffic overlays without recomputing the topology. The schema must support compact link-level telemetry query templates, per-link template parameters, and merge rules for aggregated links.
72
-- Before freezing the schema or updating producers/UI, build an emulation and benchmarking harness that can model required topology use cases at multiple scales, compare alternative encodings, run a hypothetical aggregation service, and produce repeatable payload-size and correctness evidence.
73
-
74
-## Analysis
75
-
76
-Sources checked:
77
-
78
-- `docs/netdata-ai/skills/query-netdata-cloud/SKILL.md`
79
-- `docs/netdata-ai/skills/query-netdata-agents/SKILL.md`
80
-- `.agents/skills/query-agent-events/SKILL.md`
81
-- `.agents/skills/project-writing-collectors/SKILL.md`
82
-- `.agents/sow/pending/SOW-0002-20260501-unified-multi-layered-topology-schema.md`
83
-- `.agents/sow/current/SOW-0012-20260505-streaming-topology-classification-bugs.md`
84
-- `src/collectors/network-viewer.plugin/network-viewer.c`
85
-- `src/plugins.d/FUNCTION_UI_REFERENCE.md`
86
-
87
-Current state:
88
-
89
-- Existing SOW-0002 covers future unified topology merge semantics; it is broader and blocked on design decisions. This incident is narrower and should remain separate unless it turns into schema-level work.
90
-- This SOW has now turned into schema-level work. Before implementation, its relationship with SOW-0002 must be resolved by either merging scope, superseding the relevant part of SOW-0002, or narrowing this SOW back to `topology:network-connections` only. The current user direction points toward this SOW owning the concrete detailed/aggregated schema migration.
91
-- Existing SOW-0012 covers `topology:streaming`; it is also in `.agents/sow/current/` and its current branch state already has the streaming Function split into `src/web/api/functions/function-topology-streaming.c` and `src/web/api/functions/function-netdata-streaming.c`. On 2026-05-06 the user stated SOW-0012 is done. For SOW-0020 sequencing, this unblocks shared schema work against the streaming topology producer, but the SOW-0012 lifecycle file still needs separate close/move handling if that has not already happened elsewhere.
92
-- `project-writing-collectors` classifies topology Functions as live snapshots that must not block collection loops and should be validated with the Function protocol tooling.
93
-- Token-safe Cloud probes using the repository `.env` token path returned HTTP 200 for all checked aliases at investigation time. This means the reported 503 and 404 were not stable, always-reproducible failures from the same API surface.
94
-- Successful `topology:network-connections` responses were very large:
95
- - `node-503`: 134,180,370 bytes, 48 actors, 77,797 links, no orphan actors in the completed response.
96
- - `node-404-original`: 127,847,820 bytes, 47 actors, 73,989 links, no orphan actors in the completed response.
97
- - `node-404-latest`: 110,366,885 bytes, 47 actors, 63,895 links, no orphan actors in the completed response.
98
-- The graph has low visual actor-pair cardinality despite very high link cardinality. For the largest response, 77,797 links collapse to 65 actor pairs, with the hottest actor pair carrying 38,275 separate socket links.
99
-- `pluginsd` has a hard deferred Function response cap: `PLUGINSD_MAX_DEFERRED_SIZE` is `100 * 1024 * 1024` at `src/plugins.d/pluginsd_parser.h:18`, and the parser stops the plugin when a deferred response exceeds it at `src/plugins.d/pluginsd_parser.h:195-207`.
100
-- The exact browser console message `The plugin that was servicing this request, exited before responding.` is emitted when an in-flight Function is deleted with HTTP 503 and no response body at `src/plugins.d/pluginsd_functions.c:96-97`.
101
-- The `network-viewer` topology Function currently receives cancellation and timeout state but marks both unused at `src/collectors/network-viewer.plugin/network-viewer.c:2703-2705`; it builds the full response in memory and sends it only at the end at `src/collectors/network-viewer.plugin/network-viewer.c:2713-2727`.
102
-- The `processes:by_name` option collapses process actors by command name at `src/collectors/network-viewer.plugin/network-viewer.c:1242-1253`, but the link key still includes pid, uid, namespace, local IP, remote IP, protocol, direction, state, local port, and remote endpoint port at `src/collectors/network-viewer.plugin/network-viewer.c:1359-1370`. This is the immediate source of many parallel socket links between the same collapsed actor pairs.
103
-- The latest browser 404 maps to Cloud error key `ErrNodeInstanceNotFound` / message `could not find node instance`. At probe time, the same node alias was present in the room inventory, reachable, running the same nightly version as `node-503`, and advertised `topology:network-connections` through the room Functions endpoint. This points to a Cloud node-instance routing/state race or stale node-instance lookup, not a stable missing Function.
104
-- A sibling `cloud-charts-service` checkout shows the Cloud Function route (`/api/v2/nodes/{nodeID}/function`) is handled by `nodePathProxy` (`cloud-charts-service/http/http.go:145-147`), which calls `DirectNodeRequest` (`cloud-charts-service/http/http.go:264-270`). `DirectNodeRequest` selects a node instance before proxying the request (`cloud-charts-service/internal/service/agent_data.go:565-585`). `ErrNodeInstanceNotFound` is the 404 error key and message in `cloud-charts-service/internal/model/errors.go:15`, and it is returned when node-instance routing filters leave no candidate (`cloud-charts-service/internal/routing/node_instance_filter.go:35-37`).
105
-- A sibling `cloud-frontend` checkout has prior topology TODOs documenting the same class of payload issue: a previous 89 MB response with 42,609 links collapsed to 116 actor-pair tuples. Current source has render-time link aggregation, but the useFetch topology normalizer still synchronously normalizes every source link and then computes aggregated links for graph rendering: `src/domains/functions/useFetch/normalizers/topology/index.js:5-17`.
106
-- The frontend normalizer copies full actor tables into graph node details and stores raw actors/links in table rows: `src/domains/functions/topology/payload.js:276-311,436-460`. It also copies full link `labels`, `metrics`, `src`, and `dst` into every graph link detail object: `src/domains/functions/topology/payload.js:462-487`.
107
-- The topology actor modal uses actor-type table definitions from presentation metadata and renders either link-derived tables or data tables. Data tables read rows from `node.details.tables[tableKey]`, while link tables are derived from the actor's incident graph links: `cloud-frontend/src/domains/functions/components/topology/actorModal/index.js:122-130,284-309` and `cloud-frontend/src/domains/functions/components/topology/actorModal/dataTable.js:21-24`.
108
-- Current `topology:network-connections` presentation defines modal table metadata for actors, including `source: "data"` sockets tables and `source: "links"` connection tables: `src/collectors/network-viewer.plugin/network-viewer.c:1869-1938`.
109
-- Current `topology:network-connections` embeds per-process `tables.sockets` rows directly under each actor: `src/collectors/network-viewer.plugin/network-viewer.c:2209-2226`. This satisfies the modal drilldown UX but contributes to duplication and payload size.
110
-- Current `topology:streaming` defines `streaming_path`, `retention`, `inbound`, and `outbound` actor tables with `source: "data"` even though they have different semantics: some describe streaming relationships, while `streaming_path` is actor-specific path metadata. Evidence: `src/web/api/functions/function-topology-streaming.c:303-318,321-341,360-410`.
111
-- Current `topology:streaming` emits each actor's `streaming_path` table through `rrdhost_stream_path_to_json()`, whose rows contain path metadata such as hostname, host ID, node ID, claim ID, hops, timestamps, capabilities, and flags. Evidence: `src/web/api/functions/function-topology-streaming.c:1583-1584` and `src/streaming/stream-path.c:79-95`.
112
-- The current presentation table schema only distinguishes `source: "data"` from `source: "links"`: `src/plugins.d/FUNCTION_UI_SCHEMA.json:349-357` and `src/go/pkg/topology/types.go:72-78`. This is insufficient to communicate aggregation semantics.
113
-- Current shared `topology_link` has a free-form optional `direction` string, and `topology_presentation_link_type` has visual fields such as label/color/width/dash but no direction semantics or aggregation policy: `src/plugins.d/FUNCTION_UI_SCHEMA.json:304-315,479-497` and `src/go/pkg/topology/types.go:42-55,100-106`.
114
-- Current `topology:network-connections` includes socket direction in the link key and emits it as link `direction` and labels, so direction is part of socket identity and aggregation: `src/collectors/network-viewer.plugin/network-viewer.c:1359-1370,2532-2541,2630-2647`.
115
-- Current `topology:streaming` emits links from child/agent actor to parent/target actor based on stream path, so source/destination order is semantically directed even though the link object does not emit a separate `direction` field for those links: `src/web/api/functions/function-topology-streaming.c:1654-1738`.
116
-- Current SNMP/L2 topology projects discovery adjacencies as `unidirectional` until a reverse pair is merged into `bidirectional`; this is observation/completeness metadata for an L2 adjacency, not application traffic direction: `src/go/pkg/topology/engine/topology_adapter_projection_pairs.go:62-69,230-235,301-315`.
117
-- Current SNMP topology already attaches metric lookup fragments to actors and interface rows. Device actors include `chart_id_prefix`, `chart_context_prefix`, and `device_charts`, while port/status rows include `chart_id_suffix` and `available_metrics`: `src/go/plugin/go.d/collector/snmp_topology/topology_local_actor_attrs.go:61-68` and `src/go/plugin/go.d/collector/snmp_topology/topology_local_actor_charts.go:10-28,65-67,97-115`.
118
-- Current Cloud metric query guidance requires explicit `scope.contexts` to avoid metadata explosion and supports node/context/label/dimension filtering in request scope/selectors. This makes repeated full query payloads on every topology link a payload risk; topology should carry compact query references instead: `docs/netdata-ai/skills/query-netdata-cloud/query-metrics.md:9-13,182-219`.
119
-- Existing function validation tooling already validates Function output against `src/plugins.d/FUNCTION_UI_SCHEMA.json` and provides an E2E pattern that can be extended or mirrored for topology schema tests: `src/go/tools/functions-validation/README.md:1-42`.
120
-- Existing SNMP topology code already uses manifest/golden fixture tests for topology parity and real device scenarios, which is the right pattern for repeatable topology schema experiments: `src/go/pkg/topology/engine/parity/golden_fixture_test.go:1-40` and `src/go/pkg/topology/engine/parity/node_topology_parity_test.go:1-80`.
121
-- Frontend aggregation groups links after normalization by source, target, and link type: `src/domains/functions/topology/graphAggregation.js:58-125`. This reduces render complexity but does not reduce Cloud transfer size, JSON parse cost, or normalization memory cost.
122
-- Payload size analysis of the largest captured response shows the current response is minified but structurally verbose:
123
- - Full response: 134,180,370 bytes.
124
- - `data.links`: 126,858,432 bytes for 77,797 links.
125
- - `data.actors`: 7,316,909 bytes for 48 actors.
126
- - Link `src`/`dst` blocks alone account for 61,303,293 bytes.
127
- - Link `labels` blocks alone account for 32,556,526 bytes.
128
- - Link `metrics` blocks alone account for 11,191,014 bytes.
129
- - Link object key names repeat about 152 bytes per link, about 11,825,144 bytes total.
130
- - Actor `tables` account for 7,291,350 of 7,316,909 actor bytes; one process actor table accounts for 7,286,291 bytes.
131
-- Compact encoding estimates on the same captured response:
132
- - Current core link objects `{src_actor_id,dst_actor_id,link_type,direction,state,protocol,layer}`: 16,050,624 bytes.
133
- - The same core as arrays with string actor IDs: 9,749,067 bytes.
134
- - The same core as arrays with actor indexes: 4,006,272 bytes.
135
- - Actor indexes without repeating the constant layer: 3,617,287 bytes.
136
- - Actor indexes plus direction/protocol/state only: 2,917,072 bytes.
137
- - Backend graph aggregation by current UI dimensions collapses 77,797 raw links to 69 link groups; the grouped array representation is 3,794 bytes before actor data.
138
-- Link cardinality evidence: the largest response has only two protocol values, two state values, three direction values, two link type values, and one layer value, but `labels` and `metrics` objects are effectively per-link unique in the current contract. This means enum dictionary encoding is a clear win, while full per-socket details need a columnar/table form or a separate detail plane to reduce raw uncompressed size.
139
-- A 20,000 actor / 300,000 link target appears realistic under 100 MB only with an array-first, dictionary/columnar raw contract:
140
- - Measured on the largest captured response, core link rows encoded as arrays with enum indexes cost about 17 bytes/link before adjusting for larger actor-index digit width.
141
- - Link rows with core fields plus timestamps cost about 21 bytes/link before larger actor-index adjustment.
142
- - A full columnar/dictionary link row including endpoint, label, and metric fields cost about 222 bytes/link on the captured sample. Projected to 300,000 links this is about 66.7 MB, plus about 1.6 MB extra for wider actor indexes when actor indexes grow from two digits to up to five digits.
143
- - Sample actor rows without embedded tables cost about 180 bytes/actor with dictionary/columnar encoding, but actor richness is the main uncertainty. At 20,000 actors, 500 bytes/actor is about 10 MB, 1,000 bytes/actor is about 20 MB, and 1,500 bytes/actor is about 30 MB.
144
- - Therefore, a full-detail compact raw payload is expected to stay under 100 MB if average compact actor rows remain below roughly 1.5 KB and actor tables are not embedded in the graph actor list.
145
-- Array-first encoding alone is insufficient if it still carries object-shaped per-link endpoint/detail payloads. The captured response dropped from 126.9 MB links to 110.2 MB links when only top-level link keys were removed but `src`, `dst`, `labels`, and `metrics` remained object-shaped. The large reduction appears only when link details are represented as columnar arrays and string/enum dictionaries.
146
-- Socket-count scale evidence:
147
- - Linux sockets are file descriptors from the application point of view. `socket()` returns a file descriptor, and `accept()` creates a new connected socket with a new file descriptor. Therefore established TCP connection count is bounded in practice by per-process file descriptor limits, system-wide file-handle limits, TCP memory, socket buffers, and application architecture.
148
- - Linux kernel documentation defines `ip_local_port_range` as the automatic local port range for TCP/UDP. The documented default is `32768 60999`, which is 28,232 ports. A single client local IP talking to one server IP:port is therefore normally limited to about 28k simultaneous outbound TCP connections before local ephemeral-port exhaustion, unless the range, source IPs, namespaces, or explicit binding strategy differ.
149
- - A listening server is not bounded to 28k total inbound connections on one listening port. Each accepted TCP socket is identified by the full tuple. The practical upper bound becomes roughly client-source-IP fanout times client ephemeral-port availability, then file descriptors and memory. Many client IPs can therefore produce hundreds of thousands or millions of accepted sockets on one server.
150
- - Linux kernel documentation also exposes `tcp_max_tw_buckets`, `tcp_mem`, `tcp_max_orphans`, and TCP hash-bucket controls. These make TIME_WAIT/orphan/memory pressure part of worst-case socket inventory, not just established application-owned sockets.
151
- - The current workstation values at investigation time were `ip_local_port_range=32768 60999`, `tcp_max_tw_buckets=262144`, `somaxconn=4096`, `tcp_max_syn_backlog=4096`, `ulimit -n=524288`, `fs.nr_open=2147483584`, and effectively unlimited `fs.file-max`. These values are local evidence only, not product defaults.
152
- - `network-viewer` configures local-sockets collection with namespaces enabled, inbound/outbound enabled by default, all IPv4/IPv6 TCP/UDP protocols enabled by default, and local/listen graph output disabled by default. The helper reads `/proc/net/tcp`, `/proc/net/udp`, `/proc/net/tcp6`, and `/proc/net/udp6`, and the netlink path requests all socket states. Therefore the topology payload can reflect total socket inventory across host and container network namespaces, not only one process or one namespace.
153
- - Using the largest captured response as a sizing baseline, the old detailed payload costs about 1,725 bytes per emitted link including actor data and about 1,631 bytes per link for `data.links` alone. The 100 MiB Function cap is therefore crossed around 60k-64k links with the old format, matching the observed failure class.
154
- - The local lossless prototype costs about 522 bytes per socket-equivalent row on the captured response and would cross 100 MiB around 200k rows if the same actor/table shape repeats. The target columnar/dictionary estimate of about 222 bytes per full socket row supports about 236k rows in 50 MiB, 378k rows in 80 MiB, and 472k rows in 100 MiB before actor/dictionary overhead.
155
- - Phase 1 should therefore be engineered for hundreds of thousands of socket evidence rows per node in one compact response, while phase 2 chunking is required for honest million-socket scale.
156
-
157
-Risks:
158
-
159
-- Network-viewer is a C plugin; malformed topology building can cause a hard process exit if memory ownership, iterator invalidation, or JSON building is wrong.
160
-- Cloud-proxied Function errors may hide whether the agent, plugin, or Cloud bridge failed. Direct-agent validation may be needed, but only through token-safe bearer handling.
161
-- Raw topology payloads may contain process names, private IPs, hostnames, container identifiers, and other customer- or infrastructure-identifying data. They must stay in `.local/`.
162
-- Raising parser limits alone would move the failure boundary but would not fix the root problem: topology is shipping tens of thousands of near-duplicate graph links and 100+ MB JSON for fewer than 50 actors.
163
-
164
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
165
-
166
-Status at implementation start: open for schema documentation, lab work, validator/helper rails, and first Go producer migration; final closure evidence is in the Execution Log and Validation sections.
167
-
168
-Problem / root-cause model:
169
-
170
-- A Cloud-proxied `topology:network-connections` request can begin producing data but then fail with HTTP 503 and an agent-side `AgentError` saying the plugin exited before responding. Current evidence points to an oversized Function response operational cliff: the reported nodes can produce 110-134 MB topology JSON responses, while the Agent `pluginsd` parser has a 100 MiB deferred Function response cap. The completed responses contain few actors but tens of thousands of socket links because `processes:by_name` and `endpoints:by_ip` collapse actors but the backend still emits per-socket/per-port links. Root cause is not yet closed because the same token-safe API path returned HTTP 200 during investigation, so the failure is intermittent or path-dependent rather than deterministic.
171
-
172
-Evidence reviewed:
173
-
174
-- User-provided browser console evidence: HTTP 503 on `topology:network-connections` plus `AgentError` message `The plugin that was servicing this request, exited before responding.`
175
-- User-provided second request: HTTP 404 for the same Function on a different node.
176
-- Code search evidence: `src/collectors/network-viewer.plugin/network-viewer.c` defines `NETWORK_TOPOLOGY_VIEWER_FUNCTION` as `topology:network-connections`.
177
-- Project skill evidence: topology Functions are live Function snapshots and should not block collection loops.
178
-- Cloud probe evidence: raw bodies stored only under `.local/audits/network-connections-topology/`; durable artifacts record only aliases and aggregate counts.
179
-- Agent Function protocol evidence: `src/plugins.d/pluginsd_parser.h:18,195-207` enforces the 100 MiB deferred response cap, and `src/plugins.d/pluginsd_functions.c:96-97` emits the exact 503 error message observed by the browser when an in-flight Function has no body.
180
-- Network-viewer Function evidence: `src/collectors/network-viewer.plugin/network-viewer.c:1502-1557` prepares dictionaries and calls `local_sockets_process()`, `src/collectors/network-viewer.plugin/network-viewer.c:2703-2727` ignores cancellation state and sends the result only after full JSON generation, `src/collectors/network-viewer.plugin/network-viewer.c:1242-1253` collapses process actors by name, `src/collectors/network-viewer.plugin/network-viewer.c:1359-1370` keeps per-socket/per-port link keys, and the completed response stats show no orphan actors for captured successful payloads.
181
-- Cloud routing evidence: `cloud-charts-service/http/http.go:145-147,264-270`, `cloud-charts-service/internal/service/agent_data.go:565-585`, `cloud-charts-service/internal/model/errors.go:15`, and `cloud-charts-service/internal/routing/node_instance_filter.go:35-37` show the browser 404 is produced before or during Cloud node-instance selection, not by the network-viewer plugin itself.
182
-
183
-Affected contracts and surfaces:
184
-
185
-- `topology:network-connections` Function output.
186
-- `topology:streaming` Function output.
187
-- `topology:snmp` Function output.
188
-- Shared topology detailed/aggregated schema versioning and compatibility contract.
189
-- Network-viewer plugin process stability.
190
-- Cloud `/api/v2/nodes/{node}/function` proxy behavior as observed by users.
191
-- Cloud topology UI rendering and error handling.
192
-- Cloud topology aggregation microservice API and data model.
193
-- Function protocol compliance for topology responses.
194
-
195
-Existing patterns to reuse:
196
-
197
-- Network-viewer's existing `network-connections` table Function and `topology:network-connections` registration path.
198
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json` and Function UI reference for validating envelopes.
199
-- Token-safe query wrappers from `query-netdata-agents/scripts/_lib.sh`.
200
-- `.local/audits/` for raw, gitignored Cloud responses and sanitized summaries.
201
-
202
-Risk and blast radius:
203
-
204
-- Any fix in `network-viewer.c` can affect both topology and table `network-connections` Functions.
205
-- A fix that suppresses crashes by dropping data may hide real topology evidence and cause orphan or missing actors.
206
-- Direct production querying can expose sensitive data. Only sanitized aggregates and aliases are allowed in durable artifacts.
207
-
208
-Sensitive data handling plan:
209
-
210
-- Do not copy browser cookies, session tokens, Cloud tokens, bearer tokens, claim IDs, raw node UUIDs, raw machine GUIDs, raw hostnames, raw process command lines, raw IP addresses, or raw topology payloads into SOWs, specs, docs, skills, code comments, commits, or PR text.
211
-- Store raw API responses only under `.local/audits/network-connections-topology/`, which is gitignored.
212
-- Use aliases such as `node-503`, `node-404`, `endpoint-a`, and `process-a` in durable artifacts.
213
-- If a fixture is needed, derive a sanitized minimal reproducer with placeholder UUIDs and documentation that it is sanitized.
214
-
215
-Implementation plan:
216
-
217
-1. Keep SOW-0002 as the future unified merge/correlation SOW while this SOW owns the immediate topology Function payload migration.
218
-2. Define the new shared topology schema, including `detailed`, `aggregated`, compatibility versioning, extension handling, typed sentinels, actor/link/table columnar sections, actor-scope/grouping metadata, and canonical round-trip rules.
219
-3. Update Function schema/reference artifacts for the new topology payload contract.
220
-4. Add the `aggregated`/`detailed` request mode to `topology:network-connections`, with `aggregated` as the default. The exact parameter name remains to be finalized in the schema, but accepted mode values are fixed by user decision.
221
-5. Update `topology:network-connections`, `topology:streaming`, and `topology:snmp` to emit the new schema.
222
-6. Update the Cloud frontend to support both old and new schemas; isolate old-schema detailed normalization and frontend aggregation in compatibility modules so removal is straightforward.
223
-7. Design and implement a separate Cloud topology aggregation microservice that consumes detailed topology payloads and returns requested aggregated views.
224
-8. Preserve the incident-investigation validation for 503/404/orphan actors as regression coverage.
225
-
226
-Validation plan:
227
-
228
-- Token-safe Cloud Function checks for `node-503` and `node-404`.
229
-- Function discovery check for both nodes.
230
-- Source-level same-failure scan for topology actors emitted without corresponding links.
231
-- Compile or targeted test path for network-viewer/plugin Function code if a patch is made.
232
-- Schema validation of any captured successful topology response when feasible.
233
-- Golden old-detailed <-> new-detailed round-trip tests.
234
-- Synthetic edge-case detailed payload tests for sentinels, dynamic fields, sparse columns, duplicate-looking links, large indexes, and table-heavy actors.
235
-- Fuzz/property tests for lossless detailed conversion.
236
-- Cross-function schema tests for `topology:network-connections`, `topology:streaming`, and `topology:snmp`.
237
-- Cloud frontend compatibility tests for old schema path, new detailed path, and new aggregated path.
238
-- Cloud topology aggregation microservice tests proving requested aggregations are derived from detailed payloads and do not mutate the detailed source of truth.
239
-
240
-Artifact impact plan:
241
-
242
-- AGENTS.md: likely unaffected unless the project-wide topology schema migration workflow needs a new guardrail.
243
-- Runtime project skills: update `project-writing-collectors` with any reusable topology Function schema/validation rules exposed by the migration.
244
-- Specs: add/update a topology detailed/aggregated schema spec and record the lossless detailed invariant.
245
-- End-user/operator docs: update only if the public Function/query contract changes for users or downstream integrations.
246
-- End-user/operator skills: update `query-netdata-cloud` how-to coverage for `topology:network-connections` and the new detailed/aggregated request mode.
247
-- SOW lifecycle: SOW-0002 overlap is resolved by Decision 5; SOW-0012 implementation sequencing is unblocked by the user's 2026-05-06 statement that SOW-0012 is done. The SOW-0012 lifecycle file is still physically under `.agents/sow/current/` in this working tree and should be closed separately if needed.
248
-
249
-Open-source reference evidence:
250
-
251
-- Linux kernel documentation:
252
- - `https://docs.kernel.org/networking/ip-sysctl.html` documents `somaxconn`, `ip_local_port_range`, `tcp_max_tw_buckets`, `tcp_mem`, `tcp_max_orphans`, and TCP socket hash-bucket controls.
253
- - `https://docs.kernel.org/admin-guide/sysctl/fs.html` documents `file-max`, `file-nr`, and `nr_open`.
254
-- Linux man-pages:
255
- - `https://man7.org/linux/man-pages/man2/socket.2.html` documents that `socket()` returns a file descriptor.
256
- - `https://man7.org/linux/man-pages/man2/accept.2.html` documents that `accept()` creates a new connected socket and returns a new file descriptor, and that `EMFILE`, `ENFILE`, and `ENOBUFS`/`ENOMEM` are practical limit failures.
257
-- Apache Arrow official format docs (`https://arrow.apache.org/docs/format/Columnar.html`) describe a language-independent columnar format with data adjacency, random access, typed arrays, nested layouts, and dictionary encoding for repeated values. This supports the columnar/dictionary direction, but Netdata should keep JSON compatibility for Function transport unless a separate binary transport is explicitly chosen.
258
-- open-telemetry/otel-arrow @ 78856dcb2ecd93270265296c7c279cd9ab877e24:
259
- - `docs/otap_basics.md:53-68` splits one semantic telemetry signal across multiple normalized tables with foreign keys so it can be reconstructed.
260
- - `docs/otap_basics.md:136-149` records that dictionary encoding is an encoding choice and can vary by data characteristics.
261
-- grafana/grafana @ 1a416ef1a8724349c2c16b37f64bcf194a3d9b68:
262
- - `public/app/plugins/panel/nodeGraph/utils.ts:65-128` reads node and edge data as typed fields.
263
- - `public/app/plugins/panel/nodeGraph/utils.ts:134-185` transforms node/edge frames into layout objects for rendering.
264
- - `public/app/plugins/datasource/tempo/graphTransform.ts:76-137` builds separate node and edge data frames for a service-map graph.
265
-
266
-Open decisions:
267
-
268
-- Parameter naming for the `aggregated`/`detailed` request mode remains to be finalized.
269
-- Cloud topology aggregation microservice repository, ownership, API route shape, deployment model, and persistence/caching behavior remain to be finalized.
270
-
271
-## Implications And Decisions
272
-
273
-### Decision 1 - Detailed vs Aggregated Contract Terminology
274
-
275
-Date: 2026-05-06
276
-
277
-Decision:
278
-
279
-- Use the existing `network-connections` terminology: `aggregated` and `detailed`.
280
-- `aggregated` is a UI-oriented view only.
281
-- `detailed` is the canonical full-fidelity view for both UI drilldown and cross-node Cloud correlation.
282
-- Any new compact `detailed` topology payload must preserve every piece of information currently available in the old detailed payload.
283
-- A converter must be able to transform old detailed payloads to new detailed payloads and new detailed payloads back to old detailed payloads without information loss.
284
-
285
-Implications:
286
-
287
-- Compact detailed encoding may change representation, but not semantics.
288
-- Array/dictionary/columnar encoding is acceptable only if it preserves field names, field values, missing-vs-null-vs-empty distinctions where meaningful, row order where meaningful, dynamic fields, actor/link tables, endpoint details, labels, metrics, timestamps, and all data needed for future aggregation or correlation.
289
-- Aggregation must be derived from detailed data. It must not become the source of truth for correlation.
290
-- Any backend aggregation for UI must remain separate from detailed raw/correlation payloads.
291
-
292
-Risks:
293
-
294
-- A compact schema with fixed columns only would silently lose future or function-specific fields unless it includes an extension path.
295
-- Treating `aggregated` as suitable for Cloud correlation would destroy per-link/per-socket evidence and reduce future correlation options.
296
-
297
-Validation requirement:
298
-
299
-- Add round-trip compatibility tests when implementing the schema: old detailed -> new detailed -> old detailed must be canonically equivalent, and new detailed -> old detailed -> new detailed must preserve the same information model.
300
-
301
-### Decision 2 - Lossless Detailed Payload Validation Strategy
302
-
303
-Date: 2026-05-06
304
-
305
-Decision:
306
-
307
-- Losslessness must be validated through a canonical information model, not byte-for-byte JSON equality.
308
-- Tests must include both directions:
309
- - old detailed -> canonical -> new detailed -> canonical -> old detailed -> canonical
310
- - new detailed -> canonical -> old detailed -> canonical -> new detailed -> canonical
311
-- Every canonical representation in each chain must compare equal.
312
-
313
-Required test properties:
314
-
315
-- Preserve all known fields.
316
-- Preserve unknown/dynamic fields through extension storage.
317
-- Preserve missing, null, empty string, empty array, empty object, zero, and false distinctly where the old payload can distinguish them.
318
-- Preserve actor IDs, link IDs or reconstructable link identity, endpoints, labels, metrics, timestamps, actor tables, link tables, row ordering where semantically meaningful, and all cross-reference integrity.
319
-- Preserve enough information that all old aggregations and future cross-node correlation can be recomputed from the new detailed payload.
320
-
321
-Required test classes:
322
-
323
-- Golden fixture round-trip tests using sanitized old detailed payloads.
324
-- Synthetic edge-case fixtures for null/missing/empty, unknown fields, sparse columns, duplicate-looking links, large actor indexes, high cardinality labels, and table-heavy actors.
325
-- Property/fuzz tests that generate detailed payloads, round-trip them both ways, and assert canonical equality.
326
-- Field coverage tests that recursively enumerate old detailed JSON paths and fail when a path is not mapped, explicitly preserved as extension data, or intentionally rejected by a recorded decision.
327
-- Differential behavior tests that run the old aggregation/correlation logic on old detailed and on reconstructed old detailed from new detailed, then compare results.
328
-
329
-Implications:
330
-
331
-- The new compact detailed schema needs a typed extension path. Fixed arrays alone are not sufficient unless unknown fields have a lossless place to live.
332
-- The converter must be part of the contract and must be tested in CI alongside topology Functions and UI normalizers.
333
-- Aggregated payloads are not eligible for this lossless guarantee; only detailed payloads are.
334
-
335
-### Decision 3 - Topology Schema Migration Scope
336
-
337
-Date: 2026-05-06
338
-
339
-Decision:
340
-
341
-- The Function schema must be updated for the new topology payload contract.
342
-- `topology:network-connections` must expose an `aggregated`/`detailed` request mode using the existing `network-connections` terminology.
343
-- `aggregated` is the default mode for `topology:network-connections`.
344
-- `detailed` remains the full-fidelity lossless payload used by UI drilldown and Cloud cross-node correlation.
345
-- All topology Functions must emit the new schema:
346
- - `topology:network-connections`
347
- - `topology:streaming`
348
- - `topology:snmp`
349
-- The Cloud frontend must support both old and new schemas.
350
-- Frontend aggregation must be maintained only for the old-schema compatibility path.
351
-- Old-schema detailed normalization and aggregation code must be isolated so it is easy to remove after the compatibility window.
352
-- Cloud topology aggregation must be implemented as a separate microservice. It consumes detailed topology payloads, aggregates topologies, and outputs the aggregation requested by the user.
353
-
354
-Implications:
355
-
356
-- The new schema is not a `topology:network-connections`-only change. It becomes a shared topology contract across network connections, streaming, and SNMP.
357
-- The UI should not pay the old raw detailed payload cost for normal graph rendering once the new aggregated path is available.
358
-- Cloud correlation must use detailed payloads, not aggregated views.
359
-- The Cloud topology aggregation microservice must be able to recompute requested views from detailed topology data without mutating or weakening the detailed source of truth.
360
-- Backward compatibility in the Cloud frontend is required during rollout because old Agents and new Agents will coexist.
361
-
362
-Risks:
363
-
364
-- This is now a cross-repository migration touching Agent Functions, Function schemas, Cloud frontend, and a new Cloud aggregation service.
365
-- Supporting both old and new schemas can add long-lived compatibility code unless the old path is explicitly isolated and scheduled for removal.
366
-- If all topology Functions are not migrated consistently, Cloud aggregation will need per-function adapters and will become harder to reason about.
367
-
368
-Validation requirement:
369
-
370
-- Schema conformance tests must cover all topology Functions.
371
-- Frontend tests must prove old-schema aggregation remains only in the old compatibility path.
372
-- Cloud aggregation service tests must show requested aggregated outputs are reproducible from detailed inputs.
373
-- Rollout tests must include mixed old/new Agent responses.
374
-
375
-### Decision 4 - Cloud Aggregation Location
376
-
377
-Date: 2026-05-06
378
-
379
-Decision:
380
-
381
-- Cloud topology aggregation will be implemented as a separate Cloud microservice.
382
-- It will not be implemented inside charts-service.
383
-
384
-Context:
385
-
386
-- charts-service ownership was considered and recorded earlier on 2026-05-06.
387
-- The backend team changed direction and now prefers a separate service boundary for this capability.
388
-- charts-service still participates in Cloud Function routing for `/api/v2/nodes/{nodeID}/function` and node-instance selection, as recorded in the investigation evidence, so service integration with charts-service remains part of the design.
389
-
390
-Implications:
391
-
392
-- The SOW now includes a new Cloud service boundary, with repository, ownership, deployment, API, authentication/authorization, observability, and scaling decisions.
393
-- The aggregation service API, caching, request fan-out, and error handling must integrate cleanly with existing Cloud services, including charts-service where routing/proxy context is needed.
394
-- charts-service is no longer the implementation home for aggregation logic, but its integration path remains in scope.
395
-
396
-Risks:
397
-
398
-- A new service boundary adds deployment, operational, authentication, authorization, latency, retry, and observability work.
399
-- The service must bound CPU/memory use for detailed payload fetch, decoding, aggregation, and caching.
400
-- The integration path must avoid blocking or degrading existing charts/function proxy traffic.
401
-
402
-Validation requirement:
403
-
404
-- Add service-level tests for requested topology aggregations.
405
-- Add integration tests for charts-service/service interaction where charts-service participates in request routing.
406
-- Add resource-bound tests or benchmarks for large detailed inputs.
407
-- Validate that service errors preserve enough detail to distinguish node-instance routing failures, Agent Function failures, and aggregation failures.
408
-
409
-### Decision 5 - SOW-0002 / SOW-0020 Scope Boundary
410
-
411
-Date: 2026-05-06
412
-
413
-Decision:
414
-
415
-- SOW-0020 owns the immediate topology Function payload migration:
416
- - detailed and aggregated payload contract;
417
- - lossless compact detailed representation;
418
- - old/new schema compatibility;
419
- - `topology:network-connections`, `topology:streaming`, and `topology:snmp` emission requirements;
420
- - Cloud frontend compatibility requirements;
421
- - the separate Cloud topology aggregation microservice contract.
422
-- SOW-0002 remains pending and owns future unified topology merge semantics:
423
- - same-kind and cross-kind merge algorithms;
424
- - identity matching;
425
- - conflict resolution;
426
- - storage/indexing;
427
- - unified cross-layer view behavior.
428
-- SOW-0002 will consume SOW-0020 detailed payloads as source evidence when it eventually implements merge/correlation. It does not block SOW-0020 unless SOW-0020 would remove information needed for SOW-0002.
429
-
430
-Evidence:
431
-
432
-- SOW-0002 is still `open` and explicitly blocked on merge semantics, identity matching, conflict resolution, storage model, scale targets, and L7 process granularity.
433
-- SOW-0020 has concrete incident evidence that current topology payloads can exceed practical Function limits and needs a compact, lossless detailed contract now.
434
-- User decisions already recorded in this SOW require detailed payloads to remain full-fidelity and require aggregation to be derived from detailed data.
435
-
436
-Implications:
437
-
438
-- This SOW can proceed with the payload schema/spec and network-connections incident fixes without solving cross-layer merge semantics.
439
-- The detailed schema must keep enough extension capacity and source fidelity that SOW-0002 can later implement merge/correlation without another immediate payload rewrite.
440
-- SOW-0002 acceptance criteria that mention a unified schema depend on the SOW-0020 schema once this migration ships.
441
-
442
-Risks:
443
-
444
-- If SOW-0020 over-specializes the detailed schema for current UI rendering, SOW-0002 may need a schema vNext before merge work can proceed.
445
-- If SOW-0002 later needs correlation-specific provenance that SOW-0020 did not preserve, detailed payload compatibility tests will pass but future merge quality will suffer.
446
-
447
-Mitigation:
448
-
449
-- Treat `detailed` as the canonical evidence plane, not a render model.
450
-- Preserve unknown/dynamic fields and table data through typed extension storage.
451
-- Keep `aggregated` view-only and explicitly non-authoritative for correlation.
452
-
453
-### Decision 6 - Detailed Payload Socket Evidence And Footprint
454
-
455
-Date: 2026-05-08
456
-
457
-Decision:
458
-
459
-- The most important design goal for the new topology payload is minimizing footprint without losing evidence needed for Cloud-side cross-node matching.
460
-- `detailed` does not mean "one rendered graph edge per socket".
461
-- `detailed` may safely aggregate the graph projection when the aggregation is lossless with respect to reconstructing or correlating the underlying sockets.
462
-- `detailed` must preserve per-socket evidence one by one because Cloud needs socket tuples from multiple nodes to match both sides of a connection.
463
-- Per-socket evidence should move out of repeated object-shaped graph links and into a compact detail plane, such as a columnar `socket_rows` / `socket_evidence` table keyed back to the aggregated graph edge.
464
-- `aggregated` remains the view-oriented payload and does not carry the full per-socket evidence table.
465
-
466
-Evidence:
467
-
468
-- Current `network-viewer` topology link keys include process identity, local endpoint, remote endpoint, protocol, direction, state, and ports. This preserves correlation evidence but explodes link cardinality when actors are grouped.
469
-- Client-side outbound sockets can share the same server endpoint and can be graph-aggregated for rendering, but their local endpoint still matters for matching against server-side inbound observations.
470
-- Server-side inbound sockets see remote endpoint tuples; without the matching client-side local endpoint tuples, Cloud cannot prove which remote process owns each inbound socket.
471
-
472
-Implications:
473
-
474
-- The detailed payload should be structured as:
475
- - compact actor records grouped by selected actor scope;
476
- - compact graph links aggregated by actor pair / endpoint / protocol / direction / state as appropriate;
477
- - compact per-socket evidence rows that retain local endpoint, remote endpoint, protocol, direction, state, owner identity, namespace/container identity when available, and metric summaries needed for correlation.
478
-- The UI can render from the aggregated graph projection without inflating links to one row per socket.
479
-- Cloud correlation can use the per-socket evidence table to match observations from different nodes.
480
-- Future actor scopes such as container and Kubernetes labels require annotations/enrichment, but the payload contract should already allow those scope keys.
481
-
482
-Risks:
483
-
484
-- If `detailed` drops local client endpoint data during outbound aggregation, Cloud cannot reliably match those sockets to server-side inbound rows.
485
-- If per-socket evidence remains embedded in every graph link as repeated JSON objects, the payload-size problem remains.
486
-- If actor scope keys are not explicit, grouping by process name, PID, container, or Kubernetes labels will be hard to compare across agents and Cloud services.
487
-
488
-Validation requirement:
489
-
490
-- Add fixtures where many outbound sockets from one actor share one server endpoint and prove the rendered graph collapses while the socket evidence still preserves every local endpoint tuple.
491
-- Add fixtures where server inbound observations are matched to client outbound observations through the preserved socket evidence.
492
-- Add payload-size checks that fail if detailed output regresses toward one object-shaped graph link per socket.
493
-
494
-### Decision 7 - Phase 1 Payload Budget And Phase 2 Chunking
495
-
496
-Date: 2026-05-08
497
-
498
-Decision:
499
-
500
-- Paged/chunked socket evidence is the correct long-term answer when a lossless detailed topology response still cannot fit safely in one Function response.
501
-- Paged/chunked socket evidence is phase 2, not the immediate phase 1 implementation.
502
-- Phase 1 must do everything practical to prevent reaching the single-response limit:
503
- - keep `aggregated` as the default view;
504
- - keep graph links aggregated for rendering;
505
- - move per-socket evidence into compact columnar/dictionary tables;
506
- - avoid repeated object-shaped per-socket graph links;
507
- - avoid repeated strings where indexes/enums are sufficient;
508
- - keep actor tables and socket-evidence tables separate from the graph projection;
509
- - add payload-size regression tests and size budgets.
510
-- If phase 1 still cannot produce a complete lossless detailed payload within the safe response budget, it must fail explicitly rather than silently truncate.
511
-- Truncating per-socket evidence is not allowed because it breaks Cloud cross-node matching and can create false topology conclusions.
512
-
513
-Implications:
514
-
515
-- Phase 1 remains compatible with the current Function response model while reducing the probability of hitting the hard response cap.
516
-- The phase 1 schema must be designed so phase 2 chunking can add cursors/pages for socket evidence without another semantic rewrite.
517
-- The UI must not depend on receiving all per-socket rows for normal graph rendering; rendering should use the compact graph projection.
518
-- Cloud correlation can consume phase 1 detailed payloads where they fit, and phase 2 chunking will provide the scale-out path for larger estates.
519
-
520
-Risks:
521
-
522
-- Very large servers or very large Cloud selections may still exceed the safe response budget even after compact encoding.
523
-- If phase 1 schema couples graph rows and socket evidence too tightly, phase 2 chunking will require another disruptive schema migration.
524
-
525
-Validation requirement:
526
-
527
-- Add large synthetic payload tests that estimate serialized size for high socket counts and fail if the compact detailed format regresses.
528
-- Add an explicit over-budget behavior test proving the producer fails clearly and does not return partial socket evidence as complete data.
529
-- Add schema compatibility checks that reserve the phase 2 extension path for socket-evidence pagination/chunking.
530
-
531
-### Decision 8 - NAT/LB Matching Boundary
532
-
533
-Date: 2026-05-08
534
-
535
-Decision:
536
-
537
-- NAT, load-balancer, proxy, and masquerade correlation are out of scope for the current phase.
538
-- Cloud-side socket matching must use only exact observable socket evidence available from the endpoints.
539
-- When observable tuples cannot prove both sides of a connection, the topology must leave the edge unresolved rather than infer a process-to-process match.
540
-- NAT/LB/proxy-aware matching requires future evidence sources such as flow data, proxy telemetry, conntrack/NAT metadata, or explicit service annotations.
541
-
542
-Implications:
543
-
544
-- The current implementation should be truthful before it is complete.
545
-- Exact tuple matching can build reliable process-to-process edges where both sides expose compatible local/remote endpoint evidence.
546
-- NAT/LB/proxy paths may remain as endpoint or infrastructure edges until a future SOW adds the required evidence.
547
-
548
-Risks:
549
-
550
-- Users may see incomplete cross-node process maps behind NAT, load balancers, proxies, service meshes, or SNAT-heavy Kubernetes paths.
551
-- Heuristic matching would make the graph look more complete but would introduce false positives during incident analysis, so it is explicitly rejected for this phase.
552
-
553
-Validation requirement:
554
-
555
-- Add matching tests proving exact observable tuple matches are accepted.
556
-- Add NAT/LB-like fixtures proving non-proven edges remain unresolved and are not heuristically matched.
557
-
558
-### Decision 9 - Multi-Level Actor Scopes For Infrastructure Topologies
559
-
560
-Date: 2026-05-09
561
-
562
-Decision:
563
-
564
-- The detailed topology payload must separate evidence from presentation/grouping.
565
-- Cloud aggregation must be able to materialize different actor scopes from the same detailed evidence, without requiring a different Agent payload for each view.
566
-- Required actor scopes include:
567
- - node-level infrastructure dependency maps, for large pet-style fleets where users need to see which nodes depend on which nodes;
568
- - container/application-level dependency maps, for large Kubernetes clusters where users need to see which applications or container names depend on which other applications or container names, not individual container instances;
569
- - process-name-level dependency maps, for large HPC nodes where users need to see which process names depend on which other process names, not individual PIDs;
570
- - PID-level drilldown, where exact process-instance evidence is needed.
571
-- The schema must preserve raw evidence needed to derive those scopes:
572
- - Netdata node identity and host/network identity;
573
- - process name, PID, UID, command line where available, and network namespace identity;
574
- - container identity, container name, image, pod, namespace, workload, and Kubernetes labels when enrichment is available;
575
- - local/remote socket endpoint tuples and direction/state/protocol evidence.
576
-- Actor scopes must be explicit metadata in the payload or aggregation request. They must not be inferred only from actor ID string formats.
577
-- Container and Kubernetes label support is an enrichment requirement. The current network-viewer producer does not yet have enough evidence for those scopes, but the schema must be ready for them.
578
-
579
-Evidence:
580
-
581
-- Current `topology:network-connections` supports process grouping by process name by default and PID when requested.
582
-- Current network-viewer classifies socket namespace type as `system`, `unknown`, or `container` from network namespace identity, but does not yet expose container name, pod, workload, or Kubernetes labels.
583
-- Current shared topology `match` schema already has fields for node, container, pod, and namespace identities, and allows additional match properties.
584
-
585
-Implications:
586
-
587
-- The Agent detailed payload should behave like an evidence plane. It should expose compact rows with stable dimensions and enrichment columns, not only pre-rendered actor/link objects.
588
-- The Cloud aggregator should own materialized views such as `group_by=node`, `group_by=process.name`, `group_by=process.pid`, `group_by=container.name`, `group_by=k8s.workload`, or selected Kubernetes labels.
589
-- Aggregated payloads should state the actor scope used to produce them, so users and downstream systems know what each node in the graph represents.
590
-- The same per-socket evidence can support infrastructure maps, application maps, and process maps when the required identity columns are present.
591
-
592
-Risks:
593
-
594
-- If the schema bakes in only process actors, it will not serve infrastructure-level or Kubernetes-level topology without another migration.
595
-- If grouping scope is inferred from display labels, Cloud can merge unrelated actors that happen to share a name.
596
-- Container-name grouping can intentionally merge many instances, so the payload must preserve instance-level evidence for drilldown and cross-checking.
597
-
598
-Validation requirement:
599
-
600
-- Add schema fixtures proving the same detailed socket evidence can produce node-level, process-name-level, and PID-level aggregations.
601
-- Add synthetic enriched fixtures proving container-name and Kubernetes-label grouping can be derived without changing the detailed evidence contract.
602
-- Add collision tests where two different actor instances share the same display name but must remain distinguishable in detailed evidence.
603
-
604
-### Decision 10 - Actor Drilldown Tables Without Duplicating Evidence
605
-
606
-Date: 2026-05-09
607
-
608
-Decision:
609
-
610
-- Actor drilldown tables are a first-class topology requirement.
611
-- Every materialized actor scope must be able to expose per-actor drilldown tables that list exact dependencies and supporting evidence.
612
-- Drilldown tables must support both:
613
- - aggregated rows, such as one row per peer actor, peer node, peer container, endpoint, protocol, direction, or state with counts and summaries;
614
- - exact rows, such as socket evidence rows, when the user needs to inspect the underlying connections.
615
-- The schema must avoid duplicating the same detailed socket/link evidence inside every actor.
616
-- Actor tables should be materialized views over shared compact evidence, using table definitions, row references, row ranges, filters, or indexes where possible.
617
-- When an aggregated actor table row summarizes many sockets, it must carry enough references or drilldown keys to retrieve or reconstruct the exact rows from the detailed evidence plane.
618
-
619
-Evidence:
620
-
621
-- The current shared topology schema already allows `actor.tables`, so actor-specific drilldown data is part of the existing contract.
622
-- The Cloud frontend actor modal reads presentation table definitions and renders `source: "data"` tables from `node.details.tables[tableKey]` or `source: "links"` tables from incident graph links.
623
-- Current `topology:network-connections` emits per-process `tables.sockets` rows directly under each process actor, which satisfies the drilldown UX but duplicates information already present in link/socket evidence.
624
-
625
-Implications:
626
-
627
-- The new compact schema should keep the modal/table capability, but move repeated row payload out of actor objects.
628
-- Presentation metadata should define table columns and sources, while data sections should provide compact shared table storage and per-actor indexes into it.
629
-- Aggregated topology responses can expose aggregated drilldown rows, but detailed responses must preserve the exact evidence needed to expand them.
630
-- Frontend compatibility code can reconstruct old `actor.details.tables` for old UI paths, but new-schema UI should prefer shared table sections and references.
631
-
632
-Risks:
633
-
634
-- If actor tables remain embedded as full object arrays, large actors will recreate the same payload-size problem even after graph links are compacted.
635
-- If aggregated table rows do not reference the underlying evidence, users will lose trust because they cannot drill from summary to exact dependency rows.
636
-- If table definitions are tied to one actor scope, Cloud will need separate payload shapes for node, container, process, and PID views.
637
-
638
-Validation requirement:
639
-
640
-- Add fixtures where a high-cardinality actor has many socket rows and prove modal drilldown works without duplicating full rows under the actor.
641
-- Add tests proving aggregated table rows preserve counts/summaries and can drill down to exact socket evidence rows.
642
-- Add compatibility tests proving old-schema `actor.tables` can still be normalized while new-schema shared table references render the same user-visible rows.
643
-
644
-### Decision 11 - Actor Custom Tables vs Relationship Evidence Tables
645
-
646
-Date: 2026-05-09
647
-
648
-Decision:
649
-
650
-- The topology schema must differentiate table semantics, not only table rendering source.
651
-- `source: "data"` and `source: "links"` remain presentation/rendering hints, but they are not sufficient as aggregation semantics.
652
-- Every table definition or table section should declare a semantic role, such as:
653
- - `relationship_evidence`: exact rows that describe dependencies or links and can be grouped/rolled up, such as socket evidence;
654
- - `relationship_summary`: aggregated dependency rows derived from evidence, such as peer actor summaries with counts and drilldown keys;
655
- - `actor_detail`: actor-owned custom data that describes the actor and is not generally aggregatable, such as streaming path, retention, status, capabilities, or local inventory;
656
- - `actor_inventory`: actor-owned lists that may be searchable/displayable but should not be treated as dependency evidence unless explicitly referenced by a relationship table.
657
-- Every table should also declare an aggregation policy:
658
- - `none`: preserve per actor, do not merge across aggregated actors unless explicitly requested;
659
- - `derive`: recompute from detailed relationship evidence for the requested actor scope;
660
- - `rollup`: combine rows with declared group keys and measures;
661
- - `reference`: display rows by references/ranges/filters into a shared table section.
662
-- Custom actor tables may still be stored compactly in shared columnar sections with an owner actor column, but their semantic role remains actor-owned detail. Storing them compactly must not imply they are aggregatable relationship evidence.
663
-- When Cloud aggregates actors, relationship tables may be recomputed for the aggregated actor. Actor-detail/custom tables must either remain attached to their original member actors, be exposed as drilldown/member details, or use an explicit table-specific rollup policy.
664
-
665
-Evidence:
666
-
667
-- Streaming topology `streaming_path` is declared as a data table for the actor modal, but its rows describe the actor's path metadata, not an aggregatable dependency list.
668
-- Streaming topology retention/status tables are actor details or operational summaries, not always link evidence.
669
-- Network-connections socket rows are dependency evidence and can support exact drilldown plus aggregated summaries.
670
-- The current schema's `source` enum only says where the frontend reads rows from, not whether the rows are actor-owned custom data or relationship evidence.
671
-
672
-Implications:
673
-
674
-- The new schema needs table metadata beyond `source`, for example `role`, `aggregation_policy`, `owner_scope`, `evidence_ref`, `group_keys`, and `measures`.
675
-- The Cloud aggregator must not blindly merge custom actor tables when materializing node/container/process-level views.
676
-- UI can still render all table roles in the same modal, but Cloud services need the role/policy fields to avoid incorrect aggregation.
677
-- Compatibility adapters can map old `source: "data"` tables to conservative defaults:
678
- - network-connections `sockets` -> `relationship_evidence`;
679
- - streaming `streaming_path` -> `actor_detail`;
680
- - streaming `retention` -> `actor_detail` unless a future explicit rollup policy is added;
681
- - streaming `inbound` / `outbound` -> relationship or operational summary based on their declared policy.
682
-
683
-Risks:
684
-
685
-- If table semantics are not explicit, Cloud may aggregate custom actor metadata incorrectly and present false information.
686
-- If all custom data remains embedded as actor-local object arrays, very large custom tables can still inflate payloads.
687
-- If custom tables are treated as non-aggregatable without a drilldown/member path, users may lose useful details when viewing high-level aggregated actors.
688
-
689
-Validation requirement:
690
-
691
-- Add streaming topology fixtures with `streaming_path` and retention tables proving actor-detail tables survive schema conversion and are not merged as dependency evidence.
692
-- Add network-connections fixtures proving socket tables are relationship evidence and can produce both aggregated dependency summaries and exact drilldown rows.
693
-- Add mixed aggregation tests where multiple actors are grouped and custom actor-detail tables remain accessible as member details rather than incorrectly merged.
694
-
695
-### Decision 12 - Link Direction Semantics And Aggregation Policy
696
-
697
-Date: 2026-05-09
698
-
699
-Decision:
700
-
701
-- The schema must separate the link's endpoint order from the meaning of its `direction` value.
702
-- Each link type must declare direction semantics and aggregation policy. A free-form link-level `direction` string is not enough.
703
-- Required link-type metadata includes:
704
- - `orientation`: whether the graph relationship is `directed`, `undirected`, or `hierarchical`;
705
- - `direction_role`: what the link's direction value means, such as `flow`, `dependency`, `containment`, `observation_completeness`, or `none`;
706
- - `aggregation_direction_policy`: whether Cloud must `preserve` direction in aggregation keys, `ignore` direction and canonicalize endpoint pairs, or `retain_as_attribute` while aggregating independently of direction;
707
- - optional render hints such as whether arrows or curved parallel links are appropriate. The exact UI implementation is out of scope for this SOW, but the schema must carry the information.
708
-- Direction-significant link types, such as network socket flows and streaming parent/child paths, must preserve direction during aggregation.
709
-- Undirected adjacency link types, such as most L2 links, must allow Cloud to aggregate independently of direction while retaining observation metadata such as `unidirectional` or `bidirectional` when useful.
710
-- Backward compatibility may keep the existing `direction` field, but new producers and aggregators must interpret it through the declared link-type semantics.
711
-
712
-Examples:
713
-
714
-- Network socket link type:
715
- - `orientation: directed`
716
- - `direction_role: flow`
717
- - `aggregation_direction_policy: preserve`
718
- - Rationale: inbound and outbound sockets have different source/destination meaning and different cross-node matching behavior.
719
-- Streaming link type:
720
- - `orientation: directed`
721
- - `direction_role: dependency`
722
- - `aggregation_direction_policy: preserve`
723
- - Rationale: child-to-parent streaming direction is the topology relation.
724
-- Ownership/containment link type:
725
- - `orientation: hierarchical`
726
- - `direction_role: containment`
727
- - `aggregation_direction_policy: preserve`
728
- - Rationale: parent contains child; reversing endpoints changes meaning.
729
-- L2 discovery/adjacency link types:
730
- - `orientation: undirected`
731
- - `direction_role: observation_completeness`
732
- - `aggregation_direction_policy: retain_as_attribute`
733
- - Rationale: `unidirectional` and `bidirectional` describe whether one side or both sides reported evidence, not traffic direction. Cloud can aggregate endpoint pairs independently of direction, while preserving the observation status for details.
734
-
735
-Evidence:
736
-
737
-- Shared schema has an optional `direction` string on links but no declared semantics.
738
-- Network-viewer uses socket direction as part of link identity and emits it into links and labels.
739
-- Streaming links are direction-significant through source/destination order even without a separate `direction` field.
740
-- SNMP/L2 projection uses `unidirectional`/`bidirectional` to describe observation completeness and merges reverse evidence into one bidirectional link.
741
-
742
-Implications:
743
-
744
-- The Cloud aggregator must read link-type direction metadata before deciding its aggregation key.
745
-- UI rendering can use the same metadata to decide whether arrows/curves are meaningful, but rendering behavior will be implemented separately.
746
-- Tables and drilldown should expose observation completeness separately from graph direction when `direction_role` is not `flow` or `dependency`.
747
-
748
-Risks:
749
-
750
-- If direction semantics stay implicit, Cloud may incorrectly merge directional dependencies or incorrectly split undirected L2 adjacencies.
751
-- If the UI uses the raw `direction` field alone, it may draw arrows for L2 `unidirectional` evidence even though the user should read it as discovery completeness, not traffic direction.
752
-
753
-Validation requirement:
754
-
755
-- Add schema fixtures for directed socket links proving opposite directions do not collapse unless a requested aggregation explicitly allows it.
756
-- Add streaming fixtures proving child-to-parent direction survives aggregation.
757
-- Add L2 fixtures proving reverse discovery evidence can aggregate to one undirected adjacency while retaining `unidirectional`/`bidirectional` observation metadata.
758
-
759
-### Decision 13 - Refreshable Link Telemetry Overlays
760
-
761
-Date: 2026-05-09
762
-
763
-Decision:
764
-
765
-- The topology schema must support link-level telemetry overlays that can be refreshed without recomputing the topology graph.
766
-- Topology responses should define telemetry overlay templates once per response, view, or link type. Links should carry only a template identifier plus compact parameters.
767
-- Overlay templates must describe:
768
- - the provider kind, such as Cloud time-series metrics, direct Agent metrics, or Function-backed current snapshots;
769
- - the metric families exposed by the overlay, such as traffic, packets, errors, state, utilization, or future plugin-specific measurements;
770
- - required parameter names and their meaning, such as node IDs, host selectors, chart/context prefixes, chart suffixes, interface labels, socket tuple IDs, actor IDs, or compact dictionary references;
771
- - query construction rules, including required contexts, dimensions, labels/selectors, and target nodes;
772
- - merge rules for aggregated links, including whether parameters can be unioned, summed, counted, deduplicated, or must remain as separate query references;
773
- - coverage semantics, such as exact per-link, exact actor-scope, approximate actor-scope, unsupported, or snapshot-only.
774
-- Link records should reference overlay templates through compact telemetry refs rather than embedding full query definitions.
775
-- Aggregated links must merge telemetry refs according to template-defined merge policy. The aggregator must not infer merge behavior by string concatenating query fragments.
776
-- If multiple telemetry refs cannot be merged safely, the aggregated link may carry multiple refs under the same overlay metric, and the UI or Cloud overlay layer can query and combine them according to the template.
777
-- Topology payloads should distinguish static/snapshot metrics already present in the topology response from refreshable overlay definitions.
778
-
779
-Examples:
780
-
781
-- SNMP/L2:
782
- - template kind: time-series metric query;
783
- - parameters: monitored node or vnode, chart/context prefix, local interface chart suffix or interface labels;
784
- - metric families: traffic, packets, errors, operational state;
785
- - merge policy: traffic/packets/errors can usually sum across interfaces; state must use a state-specific rule such as worst-state, count-by-state, or separate member details, not sum;
786
- - coverage: exact for the selected interface rows when the SNMP collector emits the required chart/label references.
787
-- Network-viewer process aggregation:
788
- - template kind: none today, future Function-backed current snapshot;
789
- - parameters could be actor scope plus compact socket/link evidence IDs;
790
- - coverage must be declared as unsupported until the plugin can provide exact per-link current traffic snapshots.
791
-- Network-viewer container aggregation:
792
- - cgroup network metrics may provide container-interface time series, but they do not identify the remote peer link;
793
- - coverage must therefore be exact actor-scope or approximate actor-scope, not exact per-link, unless future evidence adds peer-aware counters.
794
-
795
-Evidence:
796
-
797
-- SNMP topology already exposes chart lookup fragments on devices and interface rows.
798
-- Cloud metric queries must be tightly scoped by context/selector to avoid metadata explosion.
799
-- Current topology links have only a generic `metrics` object and no refreshable query contract.
800
-- Network-viewer currently collects socket identity and TCP summaries, but not per-link traffic time series.
801
-- cgroup network metrics are container/interface scoped, not dependency-link scoped. The cgroups integration documents per-cgroup and per-k8s-cgroup network-device contexts such as `cgroup.net_net`, `cgroup.net_packets`, `cgroup.net_errors`, `k8s.cgroup.net_net`, and `k8s.cgroup.net_packets`: `src/collectors/cgroups.plugin/integrations/containers.md:145-172` and `src/collectors/cgroups.plugin/integrations/kubernetes_containers.md:160-194`.
802
-
803
-Implications:
804
-
805
-- The schema needs a dedicated overlay/telemetry contract, separate from actor tables, relationship evidence tables, and rendered graph links.
806
-- The UI can refresh traffic by issuing targeted metric or Function queries using stable overlay refs, without calling the topology Function again.
807
-- Cloud aggregation can merge telemetry refs at the same time it merges graph links, preserving enough information to calculate aggregate bandwidth for the aggregated edge.
808
-- Overlay refs must be compact and dictionary-friendly because they may appear on many links.
809
-- Overlay refs should carry coverage/confidence so the UI does not present container-level interface traffic as exact peer-to-peer link bandwidth.
810
-
811
-Risks:
812
-
813
-- If the schema stores full query JSON per link, the payload may grow as badly as today's repeated link objects.
814
-- If overlay coverage is not explicit, users may see bandwidth on a link and assume exact peer traffic when the source is actually actor- or interface-level.
815
-- If merge policies are not defined per metric family, Cloud may sum values that should use state-aware or non-additive aggregation.
816
-- If overlay refs are tied to unstable chart IDs without stable labels or repair metadata, topology links may refresh incorrectly after interface rename, container restart, or vnode reassignment.
817
-
818
-Validation requirement:
819
-
820
-- Add fixtures proving many SNMP/L2 links can reference one overlay template with only per-link parameters.
821
-- Add aggregation fixtures proving merged links union member telemetry refs and produce correct additive traffic queries.
822
-- Add state/error fixtures proving non-additive metrics do not use additive merge rules.
823
-- Add network-viewer fixtures proving unsupported overlays are represented explicitly and do not imply exact link traffic.
824
-
825
-### Decision 14 - Schema Emulation Lab Before Producer/UI Migration
826
-
827
-Date: 2026-05-09
828
-
829
-Decision:
830
-
831
-- Do not freeze the topology schema based only on hand-written examples.
832
-- Build a schema emulation and benchmarking harness before updating all topology Functions and the UI.
833
-- The harness must model required use cases at multiple scales, generate candidate payloads, run a prototype aggregator, and produce repeatable evidence for:
834
- - serialized payload size;
835
- - compressed payload size where relevant;
836
- - decode/encode CPU time;
837
- - peak memory where measurable;
838
- - actor/link/socket/relationship evidence counts;
839
- - lossless detailed round-trip correctness;
840
- - aggregation correctness for each required actor scope;
841
- - drilldown table correctness;
842
- - direction-preservation/canonicalization behavior;
843
- - telemetry overlay ref merging behavior.
844
-- The harness output must be durable enough for review and CI regression checks, but raw captured production payloads must stay under `.local/` and must not be committed.
845
-- Synthetic scenarios can be committed when they contain no sensitive data and no customer-identifying identifiers.
846
-- Use read-only captures from internal Netdata-owned Kubernetes infrastructure as a real-world corpus when explicitly authorized for a capture run. Raw payloads, endpoint details, node names, hostnames, process names, IPs, cluster labels, and IDs must stay under `.local/`; only sanitized shape statistics, generated synthetic fixtures, and redacted/canonicalized summaries may be committed.
847
-- For the current real-corpus capture, scope Cloud queries only to the user-authorized space named `Netdata Cloud`. Other spaces visible to the token are out of scope and must not be used for topology payload capture.
848
-
849
-Required modeled scenarios:
850
-
851
-- Real captured topology Function payloads from internal Netdata-owned Kubernetes infrastructure, used as local-only corpus inputs for size and schema-shape analysis.
852
-- Network-viewer process-name aggregation on one node with many sockets to common destination endpoints.
853
-- Network-viewer cross-node matching with client outbound and server inbound evidence preserved one by one.
854
-- Network-viewer node-level infrastructure map across many nodes.
855
-- Network-viewer enriched container/application and Kubernetes-label grouping, using synthetic enrichment until real enrichment exists.
856
-- SNMP/L2 adjacency with unidirectional and bidirectional evidence, many devices, many ports, and metric overlay refs for traffic/packets/errors/state.
857
-- Streaming topology with actor-owned custom tables such as `streaming_path`, plus relationship tables and directed child-to-parent links.
858
-- Telemetry overlay aggregation where additive metrics, state metrics, unsupported overlays, and incompatible refs all behave differently.
859
-
860
-Scale points:
861
-
862
-- Small: developer-readable fixtures for schema review and golden tests.
863
-- Medium: tens to hundreds of actors and thousands of links/evidence rows.
864
-- Large: thousands of actors and hundreds of thousands of socket/interface evidence rows.
865
-- Stress: one million socket evidence rows, expected to demonstrate whether phase 1 fits or must require phase 2 paging/chunking.
866
-
867
-Candidate strategies to compare:
868
-
869
-- Current object-shaped topology payload as the baseline.
870
-- Array-of-objects with dictionaries.
871
-- Columnar tables with dictionaries.
872
-- Split graph projection plus evidence planes.
873
-- Template-based telemetry overlay refs.
874
-- Optional compression measurements, while treating compression as transport relief rather than a schema substitute.
875
-
876
-Aggregator prototype requirements:
877
-
878
-- Consume the same detailed candidate payload the Agent would emit.
879
-- Produce requested views by actor scope: node, process name, PID, container/application, Kubernetes labels/workload.
880
-- Preserve socket evidence for cross-node matching and actor drilldown.
881
-- Apply link-type direction policy and overlay merge policy from schema metadata.
882
-- Produce deterministic output for golden tests.
883
-
884
-Evidence:
885
-
886
-- Existing payload evidence already shows object-shaped per-link JSON is the primary size problem.
887
-- Existing function validation and SNMP topology parity tests provide local patterns for schema validation, E2E checks, and golden fixtures.
888
-- External topology systems such as Kiali, Pixie, and Coroot compute traffic/dependency views from metric or event queries, but their implementation patterns also show that aggregation semantics are source-specific and cannot be inferred from a generic link alone.
889
-- The user has access to internal Netdata-owned Kubernetes infrastructure where existing topology Function payloads can be captured and analyzed locally. This gives the schema lab real-world payload shapes in addition to synthetic scale models, but it requires strict raw-data isolation and sanitization.
890
-
891
-Implications:
892
-
893
-- The schema design work becomes experiment-driven: candidate schema changes must be backed by size and correctness results before producer/UI migration starts.
894
-- The first implementation artifact should be a topology schema lab/test package and prototype aggregator, not changes to all producers.
895
-- The lab should support both synthetic scenario generation and importing raw local Function payloads from `.local/` so the same candidate schema and aggregator can be tested against real and synthetic data.
896
-- Once the schema wins against the scenarios, producer and UI changes can proceed with much lower risk.
897
-
898
-Risks:
899
-
900
-- If the lab becomes too detached from real producer payloads, it may optimize synthetic data and miss real-world fields.
901
-- If the lab only tests size and not semantics, it may choose a compact format that cannot support drilldown, cross-node matching, direction semantics, or overlays.
902
-- If raw captured payloads are committed, sensitive infrastructure data may leak.
903
-- If captures are taken from live infrastructure without clear read-only scope, rate limits, and local-only storage, the schema lab could create operational risk or expose internal topology details.
904
-
905
-Validation requirement:
906
-
907
-- CI-friendly tests must fail on payload-size regressions for representative scenarios.
908
-- Golden tests must prove aggregation output is stable for each modeled use case.
909
-- Round-trip tests must prove detailed schema alternatives preserve the canonical information model.
910
-- Real-corpus import tests must operate on `.local/` payloads when available, but CI must use sanitized synthetic fixtures only.
911
-- Stress tests that are too expensive for default CI may run behind an explicit build tag or local benchmark command, but their command and expected budget must be documented.
912
-
913
-### Decision 15 - Producer Encoding Helper Strategy
914
-
915
-Date: 2026-05-09
916
-
917
-Decision:
918
-
919
-- The user classified the exact helper split as an implementation detail and authorized using the safest path.
920
-- Implement the Go topology v1 model and compact-table helper first.
921
-- Defer the C helper API shape until the first C producer migration, so the C helper is informed by the actual `topology:network-connections` and `topology:streaming` write paths instead of guessed in advance.
922
-
923
-Evidence:
924
-
925
-- Go already has a central topology package, but it currently models the old object-shaped schema in `src/go/pkg/topology/types.go`.
926
-- The SNMP topology Function returns Go topology data directly from `src/go/plugin/go.d/collector/snmp_topology/func_topology_handler.go`.
927
-- The Network Viewer and Streaming producers write JSON manually with `BUFFER` helpers in C, and they have different table and link construction patterns.
928
-
929
-Implications:
930
-
931
-- Go producers and Go-side tooling can share typed compact-table construction and validation immediately.
932
-- The first C producer migration must include the C helper decision instead of duplicating compact-table encoding by hand.
933
-- This avoids freezing a C API before validating it against the high-cardinality Network Viewer socket evidence path and the Streaming custom-table path.
934
-
935
-Risks:
936
-
937
-- Until the C helper is added, C producer migration work remains blocked on a follow-up helper design step.
938
-- If Go helper types drift from the JSON Schema, tests must catch the mismatch before producers rely on the helper.
939
-
940
-Validation requirement:
941
-
942
-- Add Go tests for compact-table row counts, encoding length checks, dictionary index bounds, actor/link reference columns, and JSON round-trip behavior.
943
-- Validate committed topology fixtures against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` and the semantic checks in `src/go/tools/functions-validation/validate`.
944
-
945
-### Decision 16 - Nested Custom Detail Table Cells
946
-
947
-Date: 2026-05-09
948
-
949
-Decision:
950
-
951
-- Add an explicit `json` column type for actor/custom detail table cells that must preserve nested producer-owned data.
952
-- Keep high-cardinality relationship evidence typed as scalar/reference/array columns whenever possible.
953
-- Treat `json` cells as not generally aggregatable unless a table type explicitly defines a safe aggregation policy for that table.
954
-
955
-Evidence:
956
-
957
-- Current SNMP/L2 actor port details can include nested producer data. `src/go/pkg/topology/engine/topology_adapter_device_summary_render.go:35` emits `vlans` from `[]map[string]any`, and `src/go/pkg/topology/engine/topology_adapter_device_summary_render.go:44` builds `neighbors` as `[]map[string]any`.
958
-- The user requirement says actor custom tables, such as streaming paths and topology-specific per-actor data, must remain supported and must be differentiated from aggregatable relationship evidence.
959
-
960
-Implications:
961
-
962
-- The schema can preserve existing custom actor detail tables without flattening away topology-specific structure.
963
-- `json` columns are a compactness escape hatch for low-cardinality custom detail data, not the default representation for sockets, L2 observations, or other high-cardinality evidence.
964
-- Aggregators may append or retain `json` detail rows according to table policy, but should not infer generic merge semantics from arbitrary nested values.
965
-
966
-Risks:
967
-
968
-- Overusing `json` columns in evidence tables would recreate object-shaped payload bloat and weaken aggregation semantics.
969
-- Producers must still prefer typed scalar/reference columns when a value participates in identity, matching, grouping, or metric aggregation.
970
-
971
-Validation requirement:
972
-
973
-- Schema tests must include a typed topology v1 payload with a `json` custom detail column containing nested arrays/objects.
974
-- Producer docs and skills must warn against using `json` for high-cardinality evidence when typed columns are possible.
975
-
976
-## Plan
977
-
978
-1. Keep SOW-0002 as the future unified merge-semantics SOW while this SOW owns the immediate shared topology payload schema migration.
979
-2. Build the topology schema emulation lab and prototype aggregator with representative scenarios, scale profiles, and size/correctness reports.
980
-3. Use the lab to compare candidate schemas and freeze the detailed/aggregated topology contract with measured evidence.
981
-4. Draft the new detailed/aggregated topology schema, including compatibility versioning, typed sentinels, dictionaries, columnar sections, extension preservation, table handling, and refreshable telemetry overlay templates.
982
-5. Promote the local lossless converter prototype into production-grade tests and reusable conversion code.
983
-6. Update Function schema/reference artifacts for topology detailed/aggregated payloads.
984
-7. Update Agent topology Functions:
985
- - `topology:network-connections`
986
- - `topology:streaming`
987
- - `topology:snmp`
988
-8. Update the Cloud frontend compatibility architecture:
989
- - old schema adapter path
990
- - old frontend aggregation isolated in old path only
991
- - new detailed adapter path
992
- - new aggregated adapter path
993
-9. Design and implement the separate Cloud topology aggregation microservice.
994
-10. Validate losslessness, aggregation equivalence, frontend compatibility, mixed-version rollout, and the original 503/404/orphan-actor failure classes.
995
-
996
-## Execution Log
997
-
998
-### 2026-05-05
999
-
1000
-- Created the SOW.
1001
-- Loaded query and collector project skills.
1002
-- Confirmed the reported Function is served by `src/collectors/network-viewer.plugin/network-viewer.c`.
1003
-- Reproduced successful token-safe Cloud calls for all target aliases, saving raw responses under `.local/audits/network-connections-topology/`.
1004
-- Summarized response sizes and graph cardinality without storing sensitive values in the SOW.
1005
-- Traced the Agent-side 503 message and deferred response size cap.
1006
-- Confirmed the latest 404 alias was reachable and advertised `topology:network-connections` at probe time, so the browser 404 is not explained by stable Function absence.
1007
-- Traced the 404 error key through the sibling `cloud-charts-service` checkout and confirmed it is a Cloud node-instance routing/selection error.
1008
-- Measured payload-size waste in the largest captured response and confirmed the main waste is structural repetition: per-link `src`/`dst`, `labels`, `metrics`, repeated object keys, and actor `tables`.
1009
-- Built a local lossless detailed-payload prototype at `.local/audits/network-connections-topology/lossless-detailed-prototype.mjs`.
1010
-- Ran the prototype against the largest captured old detailed payload:
1011
- - old detailed size: 134,180,370 bytes.
1012
- - compact detailed size: 40,639,488 bytes.
1013
- - compact ratio: 30.29% of old detailed.
1014
- - actors: 48.
1015
- - links: 77,797.
1016
- - old canonical paths: 4,279,591.
1017
- - reconstructed canonical paths: 4,279,591.
1018
- - actor columns: 48.
1019
- - actor table sections: 14.
1020
- - maximum actor table columns: 4.
1021
- - link columns: 69.
1022
- - old detailed -> compact detailed -> old detailed canonical comparison: pass.
1023
- - compact detailed -> old detailed -> compact detailed canonical comparison: pass.
1024
-- The first generic prototype pass exposed a real validation bug: internal empty-array/empty-object markers collided with real numeric values. The encoder was corrected to use private internal markers before encoding cells. This reinforces the requirement for explicit sentinel tests in production CI.
1025
-- A second table-aware prototype pass split actor tables into owner-indexed table sections. This reduced the compact detailed size from 76,910,648 bytes to 40,639,488 bytes while preserving canonical equality.
1026
-
1027
-### 2026-05-06
1028
-
1029
-- Recorded the expanded schema migration requirements:
1030
- - Function schema update.
1031
- - `topology:network-connections` `aggregated`/`detailed` request mode with `aggregated` default.
1032
- - New schema for `topology:network-connections`, `topology:streaming`, and `topology:snmp`.
1033
- - Cloud frontend support for both schemas, with old-schema frontend aggregation isolated for easy removal.
1034
- - Separate Cloud topology aggregation microservice that consumes detailed payloads and returns requested aggregated views.
1035
-- Continuation pass: corrected the SOW-0012 relationship. SOW-0012 is also current and already owns active `topology:streaming` implementation work in this branch. SOW-0020 can continue with shared topology payload schema/spec work, but must not edit the streaming topology producer until SOW-0012 is closed or explicitly merged into this SOW.
1036
-- User update: the user stated SOW-0012 is done. For SOW-0020 sequencing, this unblocks shared schema work against the streaming topology producer. The SOW-0012 file still physically lives under `.agents/sow/current/` with `Status: in-progress` in this working tree, so closing or moving that SOW remains separate lifecycle work if needed.
1037
-
1038
-### 2026-05-08
1039
-
1040
-- Renumbered this SOW from `SOW-0013` to `SOW-0020` after rebasing onto `upstream/master`. Upstream already contains completed `SOW-0013` and `SOW-0014`; the user reserved `SOW-0015` and later reported that another worktree added more SOWs, so this work uses `SOW-0020`.
1041
-- Recorded the user's footprint decision: minimize detailed payload size first, while preserving per-socket evidence one by one for Cloud-side cross-node matching. Detailed graph links may be aggregated, but socket evidence must remain lossless in a compact detail plane.
1042
-- Recorded the user's payload-budget and matching decisions: paged/chunked socket evidence is phase 2, phase 1 must minimize footprint aggressively and fail explicitly rather than truncate if still over budget, and NAT/LB/proxy matching is out of scope for the current phase.
1043
-
1044
-### 2026-05-09
1045
-
1046
-- Rebased the worktree onto latest `upstream/master` at `79a23ebd9e`. The rebase completed with no conflicts, the autostash reapplied cleanly, and post-rebase checks `git diff --check` plus `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
1047
-- Recorded the user's multi-level topology requirement: the same detailed evidence must support Cloud aggregation at node, container/application, Kubernetes label/workload, process-name, and PID scopes. Current network-viewer evidence supports node/process/PID and network namespace classification, but container and Kubernetes scopes require enrichment that the schema must be ready to carry.
1048
-- Recorded the user's actor drilldown table requirement: topology actor modals must continue to list exact dependencies, but new schema tables should be compact materialized views over shared evidence rather than duplicated object arrays under every actor.
1049
-- Recorded the user's custom actor table requirement: actor-owned tables such as streaming topology `streaming_path` must remain supported and must be distinguished from aggregatable relationship/evidence tables through explicit table role and aggregation-policy metadata.
1050
-- Recorded the user's direction semantics requirement: schema metadata must tell Cloud and UI whether a link type is directed, undirected, or hierarchical, and whether `direction` is flow/dependency meaning or observation metadata that can be ignored for aggregation identity.
1051
-- Recorded the user's refreshable link-telemetry overlay requirement: topology links need compact template references for bandwidth/packets/errors/state overlays so the UI can refresh traffic without recomputing topology, and Cloud can merge overlay refs when aggregating links.
1052
-- Recorded the user's schema-emulation requirement: before freezing the schema and migrating producers/UI, build a topology schema lab with modeled use cases, scale profiles, candidate encoding comparisons, and a prototype aggregator that produces payload-size and correctness evidence.
1053
-- Recorded the user's real-corpus requirement: the schema lab should be able to import read-only local captures of existing topology Function payloads from internal Netdata-owned Kubernetes infrastructure, with raw payloads kept under `.local/` and only sanitized summaries or generated fixtures committed.
1054
-- Recorded the user's Cloud corpus scope correction: only the `Netdata Cloud` space is in scope for topology payload capture; other visible spaces are irrelevant and must be ignored.
1055
-- Captured the scoped Cloud topology corpus using token-safe wrappers. Raw payloads are local-only under `.local/audits/topology-schema-lab/20260509T053114Z/`. Sanitized shape summary:
1056
- - `topology:network-connections`: 12 successful payloads, 0 failures, 597,525,255 total raw bytes, 15,335,804 total gzip bytes, largest raw payload 151,865,348 bytes, largest graph 331 actors / 87,775 links / 87,894 sockets.
1057
- - `topology:streaming`: 14 successful payloads, 0 failures, 1,732,039 total raw bytes, 101,389 total gzip bytes, largest raw payload 1,202,302 bytes, largest graph 120 actors / 121 links.
1058
-- Built and ran a local schema-lab prototype under `.local/audits/topology-schema-lab/scripts/` against the scoped corpus. The lab compared current object-shaped payloads, recursive string dictionaries, columnar lossless encoding, table-split columnar lossless encoding, graph/evidence-split lossless encoding, table-split graph/evidence lossless encoding, and a lossy aggregated-view estimate. Lossless candidates reconstruct the original payload and are checked with deep equality.
1059
-- Schema-lab result for `topology:network-connections` corpus:
1060
- - Current baseline: 597,525,255 raw bytes / 15,335,804 gzip bytes.
1061
- - Recursive string dictionary only: 319,794,282 raw bytes (53.52%) / 12,728,643 gzip bytes (83.00%).
1062
- - Columnar lossless: 190,662,647 raw bytes (31.91%) / 10,004,844 gzip bytes (65.24%).
1063
- - Table-split columnar lossless: 188,163,449 raw bytes (31.49%) / 9,997,988 gzip bytes (65.19%).
1064
- - Graph/evidence split lossless: 171,733,199 raw bytes (28.74%) / 9,907,243 gzip bytes (64.60%).
1065
- - Table-split graph/evidence split lossless: 168,277,837 raw bytes (28.16%) / 9,888,322 gzip bytes (64.48%).
1066
- - Aggregated-view estimate without per-socket evidence: 25,258,107 raw bytes (4.23%) / 1,503,266 gzip bytes (9.80%).
1067
- - Largest lossless table-split graph/evidence payload: 44,996,216 raw bytes for the largest 87,775-link capture.
1068
- - Largest aggregated-view estimate: 6,284,950 raw bytes for the same largest capture class.
1069
-- Additional lossless scope measurement for `topology:network-connections` corpus:
1070
- - Per-node best raw encoding, choosing the smallest verified-lossless candidate per node: 168,277,025 raw bytes / 9,888,011 gzip bytes summed across 12 node responses.
1071
- - Largest per-node best raw response: 44,996,216 raw bytes / 2,748,857 gzip bytes for the largest 87,894-socket capture.
1072
- - Per-node best gzip encoding, choosing the smallest verified-lossless gzip candidate per node: 170,026,315 raw bytes / 9,878,994 gzip bytes summed across 12 node responses.
1073
- - Single combined current JSON-array response: 597,525,268 raw bytes / 15,342,691 gzip bytes.
1074
- - Best measured single combined raw response: `table-split-graph-evidence-shared-dict-by-node-lossless`, 168,807,003 raw bytes / 9,882,897 gzip bytes.
1075
- - Best measured single combined gzip response: `table-split-graph-evidence-global-table-lossless`, 176,133,862 raw bytes / 9,792,662 gzip bytes.
1076
- - Raw combined lossless encoding is not better than independent per-node lossless encoding in this prototype. The best single combined raw response is 529,978 bytes larger than the per-node best raw sum. The best single combined gzip response saves only 86,332 gzip bytes compared with the per-node best gzip sum, at a 6,107,547 raw-byte cost.
1077
-- Follow-up measurement rejected the earlier "best lossless" interpretation:
1078
- - The earlier 44,996,216-byte largest-node result was only the best among generic table transforms tested at that point; it was not proven best.
1079
- - A per-column codec variant that still reconstructs the current old payload exactly reduced the largest 87,894-socket node to 13,672,282 raw bytes / 2,362,223 gzip bytes.
1080
- - Across the 12-node `topology:network-connections` corpus, this exact-old-payload-lossless codec measured 52,398,087 raw bytes / 8,532,821 gzip bytes.
1081
- - Largest-node section split for the exact-old-payload-lossless codec: string dictionary 3,693,814 bytes, evidence table 8,909,152 bytes, actor/table section 1,062,652 bytes, graph table 1,788 bytes.
1082
- - The largest-node evidence table previously had 30,676,393 bytes of plain value arrays across 40 evidence columns; per-column codecs reduced those same evidence columns to 8,908,635 bytes. This proves the 44,996,216-byte result was not a schema lower bound.
1083
- - Largest-node exact-old-payload-lossless record-size split: 13,672,282 total raw bytes over 87,894 sockets, about 155.6 bytes/socket. Of that, the evidence table is 8,909,152 bytes (101.4 bytes/socket), the global string dictionary is 3,693,814 bytes (42.0 bytes/socket), the actor/table section is 1,062,652 bytes (12.1 bytes/socket), and graph/envelope/metadata are negligible.
1084
- - The largest string-dictionary consumers are legacy display/presentation strings: the shared `labels.display_name` / `metrics.display_name` value set has 72,526 unique strings and about 2,688,446 JSON string bytes before dictionary array overhead; remote `port_name` values add about 708,892 JSON string bytes; local/label port-name strings add about 157,331 JSON string bytes. These are not canonical topology identity requirements.
1085
- - The largest evidence-table column costs are legacy display/port-name fields and current snapshot metrics: `labels.display_name` 972,404 bytes, `dst.attributes.port_name` 706,476 bytes, `labels.port_name` 600,121 bytes, `metrics.rtt_ms_max` 578,083 bytes, `dst.attributes.port` 476,926 bytes, `src.attributes.port` 459,193 bytes, `metrics.recv_rtt_ms_max` 453,373 bytes, plus many low-cardinality legacy label columns that each still cost about 2 bytes/socket as JSON index arrays.
1086
- - A separate canonical socket-tuple experiment, not old-payload-lossless, preserved actor graph identity plus per-socket local bind IP/port, remote IP/port, namespace/address-space/family, direction/protocol/state, and ownership edges. It measured 2,015,773 raw bytes / 443,953 gzip bytes for the largest node, or 3,229,251 raw bytes / 897,960 gzip bytes when carrying current RTT/retransmission/socket-count snapshot metrics.
1087
- - The same canonical socket-tuple experiment across all 12 nodes measured 7,864,151 raw bytes / 1,599,786 gzip bytes without current socket metrics, or 11,861,840 raw bytes / 3,039,411 gzip bytes with current socket metrics.
1088
- - The canonical socket-tuple experiment is not yet a schema decision; it is evidence that the final schema should be purpose-built around socket evidence, not around exact reconstruction of the legacy object-shaped response.
1089
-- User correction on production payload vs test reconstruction:
1090
- - Production payload must be optimized for the Cloud aggregator and UI, not for reconstructing the legacy payload.
1091
- - Reconstruction details for old-payload parity are test harness code/fixtures only. They must not be shipped in production payloads.
1092
- - The new production payload must preserve canonical information needed by aggregator/UI; it should not carry legacy presentation/reconstruction paths, old field names, display-string derivations, or redundant data solely to make old JSON byte/object reconstruction easier.
1093
- - Losslessness for schema design should therefore be measured against a canonical information model, with separate test-side projection code proving that the old payload can be derived where compatibility/parity requires it.
1094
-- Production-only canonical socket payload rerun after separating reconstruction from payload:
1095
- - Corpus scale: 323,077 socket evidence rows, 324,177 reported sockets, 1,839 graph links, and 259 ownership links across 12 captured `topology:network-connections` responses.
1096
- - Current legacy corpus size: 597,525,255 raw bytes / 15,335,804 gzip bytes.
1097
- - Production core payload as independent per-node responses: 7,280,783 raw bytes / 1,568,720 gzip bytes, 22.536 raw bytes per socket evidence row.
1098
- - Production core payload as one combined response with shared string dictionary: 7,250,808 raw bytes / 1,556,744 gzip bytes, 22.443 raw bytes per socket evidence row.
1099
- - Production core plus current RTT/retransmission/socket-count metrics as independent per-node responses: 11,278,208 raw bytes / 3,007,786 gzip bytes, 34.909 raw bytes per socket evidence row.
1100
- - Production core plus current metrics as one combined response with shared string dictionary: 11,248,200 raw bytes / 2,992,573 gzip bytes, 34.816 raw bytes per socket evidence row.
1101
- - Largest captured node under production core: 1,996,265 raw bytes / 443,069 gzip bytes for 87,761 socket evidence rows and 52 graph links.
1102
- - Largest captured node under production core plus current metrics: 3,209,721 raw bytes / 896,764 gzip bytes.
1103
- - Production core column costs across the corpus: graph index 689,510 bytes (2.134 bytes/socket row), local IP 651,101 (2.015), local port 1,359,664 (4.208), remote IP 692,081 (2.142), remote port 1,799,512 (5.570), namespace 646,151 (2.000), protocol family 646,152 (2.000), local address space 96, remote address space 646,305 (2.000). The remaining bytes are graph rows, ownership rows, per-node metadata, actors, JSON separators, and the string dictionary.
1104
- - Current metrics add: `rtt_ms_max` 1,794,102 bytes (5.553 bytes/socket row), `recv_rtt_ms_max` 1,541,595 (4.772), retransmissions 661,560 (2.048), and `socket_count` 84 bytes because it is constant per node in this corpus.
1105
-- New documentation and implementation direction:
1106
- - Document the new topology schema in detail, including the full JSON schema/contract, developer documentation, and an AI skill for creating topology producers.
1107
- - Treat the superseded topology schema as removed from Agent/backend contracts and docs. The only temporary compatibility support should be isolated in Cloud frontend code until all supported Agents emit the new schema; that compatibility path is temporary and should be deleted later.
1108
- - The new schema must be generic across topology types, not network sockets only. It must support network-connections, streaming, SNMP/L2, vSphere topology, and future topology producers.
1109
- - Scope backend and frontend implementation changes after documenting the schema.
1110
- - Build a Cloud topology aggregator as a separate Go service/component after the schema contract is documented and implementation scope is clear.
1111
- - The vSphere topology work in the separate PR worktree must be updated in place, but no edits should be made there before telling the user because another agent is working in that directory.
1112
-- Drafted the production topology schema artifacts:
1113
- - Added `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` as the JSON Schema for `netdata.topology.v1` payloads.
1114
- - Added `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md` documenting actors, links, evidence tables, actor/custom tables, direction semantics, aggregation policy, telemetry overlays, and producer examples for network-connections, streaming, SNMP/L2, and vSphere.
1115
- - Added `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md` scoping backend producer migration, frontend decoder/compatibility migration, and the Go aggregation component.
1116
- - Added the public `docs/netdata-ai/skills/create-topology/` skill and `.agents/skills/create-topology` symlink so future assistants follow this topology contract.
1117
- - Updated Function reference/developer docs and topology query skills to point to the production schema instead of documenting compatibility payload details.
1118
-- Schema-lab result for `topology:streaming` corpus:
1119
- - Current baseline: 1,732,039 raw bytes / 101,389 gzip bytes.
1120
- - Table-split columnar lossless is the best measured lossless candidate: 511,821 raw bytes (29.55%) / 81,484 gzip bytes (80.37%).
1121
- - Graph/evidence split adds no value for streaming where each link already has one evidence row; table-splitting actor custom tables matters more.
1122
-- Prototype aggregator result for `topology:network-connections` corpus:
1123
- - Extracted 323,077 socket evidence rows: 214,546 inbound and 108,531 outbound.
1124
- - Exact reverse-tuple matching found 105,158 matched outbound rows, a 96.89% match ratio among outbound rows in this corpus.
1125
- - After exact matching and unresolved-endpoint aggregation by IP, node-level projection produced 1,093 graph edges from 217,919 dependency evidence rows; process-name projection produced 1,027 graph edges; actor-identity projection produced 1,762 graph edges.
1126
- - The largest node-level edge carried 43,196 evidence rows, proving that graph projection and evidence storage must be separate.
1127
-- User implementation clarification:
1128
- - The Go topology aggregator should be implemented as a Cloud-style microservice similar to `${NETDATA_REPOS_DIR}/cloud-charts-service`, not primarily as an Agent-side helper package.
1129
- - Before coding the aggregator, inspect the Cloud service layout, configuration, HTTP handler, test, and deployment conventions and mirror those patterns.
1130
- - The new microservice name/repository should be `cloud-topology-service`.
1131
- - Hard implementation requirement: `cloud-topology-service` must follow Cloud backend service coding, operational, configuration, testing, deployment, observability, and repository conventions exactly. Implementation must not begin until those conventions are extracted from existing Cloud service repositories with concrete file/line evidence and turned into a compliance checklist.
1132
-- Cloud backend service convention evidence collected so far:
1133
- - `${NETDATA_REPOS_DIR}/cloud-service-builder/README.md:1` describes the skeleton as a microservice based on Netdata standards, but the generated template must be cross-checked against newer services before use.
1134
- - `${NETDATA_REPOS_DIR}/cloud-service-builder/templates/cmd/app/main.go.tmpl:32`, `:41`, `:57`, `:129`, `:155`, and `:192` show the expected generated service-name constant, signal trap, config load, Prometheus/infra server, API server, and instance-name pattern.
1135
- - `${NETDATA_REPOS_DIR}/cloud-service-builder/templates/internal/config/config.go.tmpl:54`, `:63`, and `:136` show the flag groups, log subset, and env parser prefix/set-separator pattern.
1136
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/cmd/service/main.go:62`, `:100`, `:125`, `:177`, `:208`, `:214`, `:245`, `:258`, `:272`, `:280`, `:361`, and `:380` show the current production service-name, logger, config, env prefix, Prometheus, infra server, ADC instrumentation, Pulsar, spaceroom gRPC client, repositories/services, API server, and route registration pattern.
1137
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/http/http.go:86`, `:107`, and `:144` show HTTP route registration, node/room auth wrapping, and the existing Cloud Function execution route pattern.
1138
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/http/auth_middleware.go:64` and `:134` show the Cloud spaceroom authorization pattern for node and room scoped endpoints.
1139
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/http/utils.go:21` and `:57` show the response and error-response helpers, including request id handling and `ckerrors` mapping.
1140
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/internal/model/errors.go:13` shows the service error taxonomy style.
1141
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/internal/service/agent_data.go:565` and `:1233` show direct single-node Function passthrough and concurrent multi-node Function passthrough through ADC after node-instance routing.
1142
- - `${NETDATA_REPOS_DIR}/cloud-agent-data-ctrl-service/internal/config/config.go:21`, `:49`, and `:78` show an ADC-facing service configuration pattern with `ADC_` env prefix.
1143
- - `${NETDATA_REPOS_DIR}/cloud-agent-data-ctrl-service/cmd/service/main.go:138`, `:218`, `:225`, `:235`, and `:242` show errgroup lifecycle, Prometheus, infra server, API server, and OpenTelemetry handler wiring.
1144
- - `${NETDATA_REPOS_DIR}/cloud-agent-data-ctrl-service/transport/http.go:68`, `:95`, `:107`, `:132`, and `:196` show the internal agent API path map, handler construction, middleware, account validation, and internal proxy pattern.
1145
- - `${NETDATA_REPOS_DIR}/cloud-custom-dashboard-service/cmd/customdashboardsvc/main.go:39`, `:71`, `:82`, `:122`, `:136`, `:157`, `:179`, `:185`, and `:192` show another current HTTP service pattern with signal handling, flags, env parser, Prometheus, spaceroom gRPC client, infra/API servers, and OpenTelemetry middleware.
1146
- - `${NETDATA_REPOS_DIR}/cloud-custom-dashboard-service/internal/dashboard/transport_http.go:33` and `:54` show route construction and spaceroom authorization for room-scoped resources.
1147
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/Makefile:4`, `:18`, `:23`, `:28`, `:43`, and `:49` show expected tools, unit, integration, coverage, generate, and lint targets.
1148
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/Dockerfile:2`, `:4`, `:18`, `:29`, and `:42` show the Go base image, service env, build ldflags, Alpine production image, and entrypoint pattern.
1149
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/deployments/helm/values.yaml:2`, `:19`, `:30`, `:38`, `:44`, `:58`, `:64`, `:119`, and `:122` show the microservice anchor, Go memory limit env, probes, resource defaults, service env, security context, Prometheus annotations, service, and ingress-route shape.
1150
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/.github/workflows/main.yml:3`, `:40`, `:103`, `:128`, `:133`, `:137`, `:142`, and `:190` show PR/push triggers, permissions, Go setup, module verification, lint, integration test, coverage, and deployment workflow handoff.
1151
- - `${NETDATA_REPOS_DIR}/cloud-charts-service/.github/CODEOWNERS:1` and `:8` show ownership separation for deployment files and Go service code.
1152
-- Cloud backend service compliance plan:
1153
- - Treat existing Cloud service code as the source of truth. The service-builder template may bootstrap files, but every generated file must be reconciled against the current `cloud-charts-service`, `cloud-agent-data-ctrl-service`, `cloud-custom-dashboard-service`, and `cloud-spaceroom-service` patterns before implementation.
1154
- - Produce a pre-code compliance matrix covering repository layout, Go module/dependency versions, Makefile targets, Dockerfile, GitHub workflows, CODEOWNERS, Helm values, environment variable prefix, configuration flags, logging, signal handling, Prometheus/OpenTelemetry, infra health/readiness, HTTP routing, CORS, authorization, error responses, ADC access, spaceroom access, tests, generated mocks, and deployment annotations.
1155
- - Use `cloud-charts-service` as the primary behavioral reference for Function passthrough, node-instance routing, spaceroom authorization, ADC client instrumentation, request metadata, and Function-specific permissions.
1156
- - Use `cloud-agent-data-ctrl-service` as the primary reference for ADC proxy lifecycle, agent request timeout handling, internal path validation, and service shutdown behavior.
1157
- - Use `cloud-custom-dashboard-service` as the primary reference for compact room-scoped HTTP CRUD-style route construction and spaceroom auth middleware.
1158
- - Use `cloud-service-builder` only for baseline repository shape after checking whether any generated defaults are stale compared to current services.
1159
- - Implementation remains blocked until the compliance matrix identifies the exact file pattern to copy or adapt for each surface and records any gaps that need a user or Cloud-backend decision.
1160
-- Parallel microservice handoff:
1161
- - Created `${NETDATA_REPOS_DIR}/cloud-topology-service/REQUIREMENTS.md` as a standalone handoff contract for a parallel worker to build the Cloud microservice while this SOW continues with schema/frontend/producer tasks.
1162
- - The handoff requires the parallel worker to build the pre-code Cloud service compliance matrix before writing service behavior.
1163
- - The handoff points back to this SOW and the topology schema, developer guide, implementation scope, and `create-topology` skill as the topology contract sources.
1164
- - Reviewed the parallel worker's `${NETDATA_REPOS_DIR}/cloud-topology-service/QUESTIONS1.md` and answered it in `${NETDATA_REPOS_DIR}/cloud-topology-service/ANSWERS1.md`, setting phase-1 defaults for API ownership, route shape, new-schema-only behavior, source scope, partial/error semantics, caching, fixtures, optional metrics, validation location, deployment ownership handling, and compliance matrix location.
1165
- - Kept three true external decisions open in the answer handoff: final service ownership/CODEOWNERS, environment-specific Helm values/deployment targets, and the exact Cloud-approved node-instance routing implementation strategy.
1166
- - User clarified that these three open items are not user decisions; they should be handed to Cloud backend and DevOps once the service is otherwise ready for operational integration. Updated the microservice requirements and answers handoff so the parallel worker proceeds with compliance, schema/codec, aggregation, tests, HTTP scaffolding, and isolated fetcher interfaces without inventing ownership, environment values, or node-routing strategy.
1167
- - User clarified that the phase-1 microservice MVP must support all topology kinds covered by the schema contract. It is not acceptable for the UI to use the service for only some topologies while bypassing it for others. Updated the microservice requirements, answers handoff, and topology implementation scope so `network-connections` remains the high-cardinality benchmark but not the MVP boundary.
1168
- - Aligned the local topology implementation scope, topology schema spec, and `create-topology` skill with the all-topology MVP rule. Replaced stale Cloud aggregator open questions with resolved phase-1 defaults and Cloud backend/DevOps integration gates.
1169
- - Added a current migration inventory to `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`, covering network-viewer, streaming, SNMP/L2, vSphere, and Cloud frontend surfaces with concrete file/line evidence and target migration behavior.
1170
-- Parallel Cloud frontend handoff:
1171
- - Created sibling `cloud-frontend/TODO-topology-schema-new.md` as a standalone handoff contract for a parallel worker to implement frontend support for `netdata.topology.v1`.
1172
- - The handoff records current frontend file/line evidence, requires old-schema support and the old link dedupper to be isolated under a removable legacy adapter, and scopes the new v1 path around compact table decoding, actor/link/evidence indexes, actor-detail vs relationship table separation, explicit direction semantics, overlay refs, worker decoding, and all-topology fixtures.
1173
- - The handoff intentionally keeps Cloud service integration separate from the v1 decoder so per-node Function responses can migrate by schema version before the service route is ready.
1174
- - Removed a later appended open-questions section from the handoff. The useful items were resolved into implementation defaults and concrete decoder clarifications so the frontend worker is not blocked on questions that can be answered from the schema, current frontend code, or the Cloud service handoff.
1175
-- Agent-side validation and fixture rail:
1176
- - Extended `src/go/tools/functions-validation/validate` so the existing Function validation CLI now performs topology v1 semantic checks after JSON Schema validation. The checks validate compact table decoded lengths, inline dictionary indexes, global dictionary references, actor/link references, evidence references when a detail table declares `source_evidence`, and array column values.
1177
- - Added schema-level topology v1 fixtures under `src/go/tools/functions-validation/fixtures/topology-v1/` for `network-connections`, `streaming`, `snmp-l2`, and `vsphere`.
1178
- - The fixtures cover directed socket evidence, streaming actor-detail `stream_path`, SNMP/L2 unordered observation direction plus actor inventory and overlay refs, and vSphere hierarchy/dependency links plus actor detail.
1179
- - Updated `src/go/tools/functions-validation/README.md` with a topology v1 validation command and documented the additional compact-table semantic checks.
1180
-- Producer helper rail:
1181
- - Added `src/go/pkg/topology/v1` as the Go producer-side model and compact-table helper package for `netdata.topology.v1` payloads.
1182
- - The helper provides response/data/type/table structs, compact-table constructors, row-count validation, parallel column/value validation, dictionary-index validation, and a shared decoded-payload semantic validator.
1183
- - Refactored `src/go/tools/functions-validation/validate` to call the shared topology v1 semantic validator instead of keeping topology-specific validation private to the CLI.
1184
- - Updated the topology developer guide, `create-topology` skill, and `project-writing-collectors` skill so future Go topology producers use the helper instead of hand-building compact-table JSON.
1185
-- SNMP/L2 producer migration investigation:
1186
- - Investigated `topology:snmp` as the first Go producer candidate for migration to the helper.
1187
- - Found a schema gap before producer migration: existing SNMP actor port details may include nested custom data such as `vlans` and `neighbors`, which cannot be represented by scalar-only compact table cells without losing information or flattening producer-specific structure.
1188
- - Updated `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` to add a `json` column type for nested custom detail cells.
1189
- - Updated the topology developer guide and `create-topology` skill to restrict `json` columns to actor/custom detail data and warn against using them for high-cardinality relationship evidence.
1190
- - Extended the Go helper schema round-trip test with a nested `json` actor detail column so this requirement stays covered by tests.
1191
-- SNMP/L2 producer migration implementation:
1192
- - Updated `src/go/plugin/go.d/collector/snmp_topology/func_topology_handler.go` so `topology:snmp` returns `netdata.topology.v1` data through a dedicated adapter before sending the Function response.
1193
- - Added `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` to map the current SNMP topology snapshot into compact actor, link, L2 observation evidence, actor metadata, and actor-detail tables using `src/go/pkg/topology/v1`.
1194
- - Preserved nested SNMP custom actor detail cells such as `neighbors` and `vlans` with `json` columns while keeping graph links and L2 evidence typed.
1195
- - Updated SNMP topology Function tests to validate produced payloads against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` and the shared topology v1 semantic validator.
1196
- - Updated the topology implementation scope and spec to record that SNMP now emits v1 through an adapter, while interface metric overlay-template/ref migration remains a refinement.
1197
-- Streaming producer migration implementation:
1198
- - Replaced the superseded object-shaped `topology:streaming` payload in `src/web/api/functions/function-topology-streaming.c` with a direct `netdata.topology.v1` emitter.
1199
- - Preserved streaming agents as compact actor rows, directed streaming/virtual/stale graph links, link evidence rows, and actor/modal tables for `stream_path`, retention, inbound, and outbound data.
1200
- - Classified `stream_path` and retention as actor-detail tables and inbound/outbound rows as relationship summaries, so actor-owned custom data is not conflated with relationship evidence.
1201
- - Kept stale stream-path hops as signed values and split streaming, virtual, and stale link evidence into separate evidence type ids so link-type metadata and evidence metadata agree.
1202
- - Used a local C compact-table emitter for this low-cardinality producer. The shared C helper decision remains tied to the `topology:network-connections` migration, where high-cardinality socket evidence will determine the helper API.
1203
- - Updated the topology implementation scope and topology schema spec to record that streaming now emits v1 directly.
1204
-- Network-viewer producer migration implementation:
1205
- - Replaced the superseded object-shaped `topology:network-connections` payload in `src/collectors/network-viewer.plugin/network-viewer.c` with a direct `netdata.topology.v1` emitter.
1206
- - Added `aggregated` / `mode:aggregated` and `detailed` / `mode:detailed` request handling, with aggregated as the default mode.
1207
- - Preserved process grouping options and current socket filters while moving graph output to compact actor and graph-link tables.
1208
- - Detailed mode emits socket relationship evidence as a shared compact table for exact tuple matching and drilldowns; aggregated mode omits the evidence table.
1209
- - Removed superseded topology presentation emission and actor-nested socket tables from the Agent producer so the old schema is no longer present in this code path.
1210
- - Added automatic string-column encoding for link/evidence columns: the writer uses inline dictionaries only when estimated raw JSON size is smaller than plain values.
1211
- - Updated the topology implementation scope and topology schema spec to record that network-viewer now emits v1 directly.
1212
-- Orphan endpoint actor repair:
1213
- - Investigated the user's reproduced floating endpoint actor with a non-zero
1214
- socket count and no incident graph links.
1215
- - Root cause: `local_sockets_cb_to_topology()` can create a remote endpoint
1216
- actor while scanning sockets before all local IPs are known, but
1217
- `topology_v1_collect_links()` resolves socket destinations later using the
1218
- final local-IP set. If the same IP is learned as local later in the scan,
1219
- link resolution treats it as self while the earlier remote endpoint actor
1220
- remains in the actor table.
1221
- - Fixed `topology_v1_collect_actors()` so endpoint actor emission rechecks
1222
- `topology_ip_belongs_to_self()` against the final local-IP set. This makes
1223
- endpoint actor emission and link destination resolution use the same self
1224
- classification.
1225
-
1226
-## Validation
1227
-
1228
-Acceptance criteria evidence:
1229
-
1230
-- 503 path: not deterministically reproduced through the token-safe path; evidence points to oversized intermittent/path-dependent Function failure. Supporting evidence: 110-134 MB successful bodies, 100 MiB parser cap, and exact 503 message source.
1231
-- 404 path: latest browser 404 was not reproduced through the token-safe path; function discovery and room inventory showed the node alias reachable and exposing `topology:network-connections` at probe time. This supports a Cloud node-instance routing/state race or stale browser/request context rather than a stable missing Function.
1232
-- Orphan endpoint path: reproduced by the user after the initial completed
1233
- captures. Source analysis found an order-dependent actor/link classification
1234
- mismatch in the producer; the targeted repair now aligns endpoint actor
1235
- emission with final link destination classification.
1236
-
1237
-Tests or equivalent validation:
1238
-
1239
-- Schema-lab prototype transforms under `.local/audits/topology-schema-lab/scripts/` were run against the scoped Cloud corpus and produced the sanitized size/correctness measurements recorded in the execution log.
1240
-- `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
1241
-- A minimal `netdata.topology.v1` sample payload was validated with Ajv 2020 against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
1242
-- `git diff --check` passed.
1243
-- Stale-shape scan over the new topology docs/skills found no references to compatibility object fields such as `src_actor_id`, `dst_actor_id`, `data.actors[]`, `actors[]`, or `links[]`. The remaining `src_actor_id`/`dst_actor_id` references are in `src/plugins.d/FUNCTION_UI_SCHEMA.json`, which is the still-deployed Function UI schema and will be handled during implementation migration.
1244
-- Relative link target checks passed for the new query-topology references and the `.agents/skills/create-topology` symlink.
1245
-- Sensitive-path scan over the touched topology docs, skill, spec, and SOW found no per-user filesystem paths.
1246
-- Sensitive-path scan over the sibling Cloud frontend handoff found no per-user filesystem paths, personal names, or raw production identifiers.
1247
-- `jq empty` passed for all `src/go/tools/functions-validation/fixtures/topology-v1/*.json` fixtures.
1248
-- `go test ./tools/functions-validation/validate` passed from `src/go`.
1249
-- The validation CLI passed for all topology v1 fixtures with `--schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
1250
-- `go test ./pkg/topology/v1 ./tools/functions-validation/validate` passed from `src/go`.
1251
-- `go test ./pkg/topology/... ./tools/functions-validation/validate` passed from `src/go`.
1252
-- The topology v1 helper JSON round-trip test marshaled a typed `netdata.topology.v1` response with a nested `json` actor detail column, validated it against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`, and then ran the shared semantic validator.
1253
-- `go test ./plugin/go.d/collector/snmp_topology ./pkg/topology/v1 ./tools/functions-validation/validate` passed from `src/go`.
1254
-- `go test ./pkg/topology/... ./plugin/go.d/collector/snmp_topology ./tools/functions-validation/validate` passed from `src/go`.
1255
-- SNMP topology Function tests now validate actual handler responses against both the production topology JSON Schema and the shared semantic validator.
1256
-- `TestSNMPTopologyToV1_PreservesActorCustomTables` validates that nested SNMP actor detail cells are preserved as `json` columns without weakening the typed graph/evidence path.
1257
-- Local CMake validation build configured with `cmake -S . -B .local/build-topology -DCMAKE_BUILD_TYPE=Debug -DENABLE_PLUGIN_XENSTAT=OFF -DENABLE_PLUGIN_DEBUGFS=OFF`. `ENABLE_PLUGIN_XENSTAT=OFF` avoids a missing local `xenstat` package, and `ENABLE_PLUGIN_DEBUGFS=OFF` avoids an uninitialized optional libsensors subtree in this local worktree.
1258
-- Initialized only the declared `src/aclk/aclk-schemas` submodule so the normal Agent target can generate protobuf sources for the local validation build.
1259
-- `cmake --build .local/build-topology --target netdata -j2` passed after the streaming migration, including compilation of `src/web/api/functions/function-topology-streaming.c` and linking the `netdata` executable.
1260
-- After tightening signed hops and evidence type ids, the incremental `cmake --build .local/build-topology --target netdata -j2` passed again, rebuilding `function-topology-streaming.c` and linking `netdata`.
1261
-- `git diff --check -- src/web/api/functions/function-topology-streaming.c` passed.
1262
-- Stale-shape scan over `src/web/api/functions/function-topology-streaming.c` found no superseded presentation helpers, `schema_version: "2.0"`, or old `src_actor_id` / `dst_actor_id` payload fields.
1263
-- `git diff --check -- src/collectors/network-viewer.plugin/network-viewer.c` passed after the network-viewer migration.
1264
-- `cmake --build .local/build-topology --target network-viewer.plugin -j2` passed after the final network-viewer migration, rebuilding and linking `network-viewer.plugin`.
1265
-- Stale-shape scan over `src/collectors/network-viewer.plugin/network-viewer.c` found no superseded topology presentation helpers, `schema_version: "2.0"`, or old `l7` topology layer values.
1266
-- Local `network-viewer.plugin debug` aggregated sample was captured under `.local/audits/topology-network-viewer-v1/` and validated against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` with the Function validator. Sanitized shape: 74 actors, 86 graph links, 0 socket-evidence rows, 20,132 raw bytes.
1267
-- Local `network-viewer.plugin debug` detailed sample was captured under `.local/audits/topology-network-viewer-v1/` and validated against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` with the Function validator. Sanitized shape: 91 actors, 102 graph links, 173 socket-evidence rows, 39,865 raw bytes.
1268
-- Local network-viewer sample codec check confirmed graph-link string columns use `dict` where beneficial and detailed socket-evidence string columns use a mix of `dict` and `values` based on estimated raw JSON size.
1269
-- Final validation pass after the network-viewer migration: `cmake --build .local/build-topology --target netdata -j2` passed.
1270
-- Final validation pass after the network-viewer migration: `go test ./pkg/topology/... ./plugin/go.d/collector/snmp_topology ./tools/functions-validation/validate` passed from `src/go`.
1271
-- Final validation pass after the network-viewer migration: the Function validator passed for all `src/go/tools/functions-validation/fixtures/topology-v1/*.json` fixtures and for the local aggregated/detailed network-viewer samples.
1272
-- Re-ran `go test ./pkg/topology/... ./plugin/go.d/collector/snmp_topology ./tools/functions-validation/validate` from `src/go`; it passed.
1273
-- Re-ran the Function validator over all `src/go/tools/functions-validation/fixtures/topology-v1/*.json` fixtures with `--schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`; all fixtures passed.
1274
-- Sensitive-data scan over the new topology fixtures and validator changes found no personal paths, personal names, tokens, cookies, API keys, UUID-shaped identifiers, or pushback wording.
1275
-- After the orphan endpoint actor repair:
1276
- - C syntax validation passed for `src/collectors/network-viewer.plugin/network-viewer.c` using the compile command from `build/compile_commands.json` with `-fsyntax-only`.
1277
- - `cmake --build .local/build-topology --target network-viewer.plugin -j2` passed.
1278
- - A local `network-viewer.plugin debug` topology sample was captured under
1279
- `.local/audits/topology-network-viewer-v1/` and validated with
1280
- `go run ./tools/functions-validation/validate --schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
1281
- - The local sample had zero endpoint actors without incident links after
1282
- compact-table decoding.
1283
- - The specific reproduced self-classified endpoint address was not emitted
1284
- as an endpoint actor in the local sample after the repair.
1285
-
1286
-Real-use evidence:
1287
-
1288
-- Cloud API calls through the token-safe path returned HTTP 200 for all checked aliases during investigation. Browser evidence still shows real 503 and 404 failures, so the failures are intermittent/path-dependent.
1289
-
1290
-Reviewer findings:
1291
-
1292
-- No unresolved reviewer findings remain for this SOW. The final orphan-endpoint repair was narrow and was validated with source-path analysis, C syntax validation, a local plugin build, a runtime Function sample, schema validation, and a same-failure orphan scan. The broader topology presentation review was split into SOW-0021 and completed there with multi-reviewer findings and follow-up mapping.
1293
-
1294
-Same-failure scan:
1295
-
1296
-- Completed captured payloads were scanned for actors with no incident links and links referencing missing actors; no orphan actors or broken link references were found in those completed responses.
1297
-
1298
-Sensitive data gate:
1299
-
1300
-- Durable artifacts created so far use aliases and do not include browser cookies, raw tokens, raw node UUIDs, raw machine GUIDs, raw process details, or raw topology payloads.
1301
-
1302
-Artifact maintenance gate:
1303
-
1304
-- AGENTS.md: updated to register the public `create-topology` skill and symlink.
1305
-- Runtime project skills: `.agents/skills/project-writing-collectors/SKILL.md` updated to point topology producers at the new topology schema, guide, and implementation scope.
1306
-- Specs: `.agents/sow/specs/topology-function-schema.md` added as durable project memory for the production topology Function contract and updated with `json` detail-column, SNMP adapter migration notes, direct streaming v1 emission, signed stale-hop handling, and direct network-viewer v1 emission.
1307
-- End-user/operator docs: Function UI developer/reference docs updated to reference the topology schema; no external Learn docs were changed in this pass.
1308
-- End-user/operator skills: `docs/netdata-ai/skills/create-topology/`, `docs/netdata-ai/skills/query-netdata-cloud/`, and `docs/netdata-ai/skills/query-netdata-agents/` updated for topology schema/query guidance.
1309
-- SOW lifecycle: this SOW is marked `completed` and is moved to `.agents/sow/done/` with the implementation commit.
1310
-
1311
-Specs update:
1312
-
1313
-- `.agents/sow/specs/topology-function-schema.md` added and updated with `json` detail-column, SNMP adapter migration notes, direct streaming v1 migration notes, and direct network-viewer v1 migration notes.
1314
-
1315
-Project skills update:
1316
-
1317
-- `.agents/skills/project-writing-collectors/SKILL.md` updated.
1318
-
1319
-End-user/operator docs update:
1320
-
1321
-- `src/plugins.d/FUNCTION_UI_DEVELOPER_GUIDE.md` and `src/plugins.d/FUNCTION_UI_REFERENCE.md` updated.
1322
-
1323
-End-user/operator skills update:
1324
-
1325
-- `docs/netdata-ai/skills/create-topology/`, `docs/netdata-ai/skills/query-netdata-cloud/`, and `docs/netdata-ai/skills/query-netdata-agents/` updated.
1326
-
1327
-Lessons:
1328
-
1329
-- Actor emission and link emission must use the same final self/remote classification inputs. The network-viewer bug came from discovering remote endpoint actors during socket scanning before the final local-IP set was complete, while link emission later used the completed local-IP set.
1330
-- Compact topology validation must include graph invariants, not only schema validity. A payload can be valid JSON and still be unusable if it contains actors that no emitted link can reach.
1331
-- Large topology payload work needs both size tests and semantic round-trip tests. Size reduction alone is not enough if actor/link identity or drilldown evidence is weakened.
1332
-
1333
-Follow-up mapping:
1334
-
1335
-- SOW-0021 completed the topology presentation contract and backend producer presentation updates.
1336
-- SOW-0022 tracks actor modal and table composition.
1337
-- SOW-0023 tracks cross-payload actor identity, reconciliation, and matching strategies.
1338
-- SOW-0024 tracks vSphere topology migration to `netdata.topology.v1`.
1339
-- The Cloud topology service and Cloud frontend implementation handoffs are owned by their respective repositories and workers; this netdata commit does not include those repositories.
1340
-
1341
-## Outcome
1342
-
1343
-`topology:network-connections`, `topology:streaming`, and `topology:snmp` now have a compact `netdata.topology.v1` schema path, shared validation, producer guidance, and realistic fixtures. The network-connections orphan endpoint repair aligns remote actor emission with final link destination classification, so self-classified endpoints are no longer emitted as floating graph actors.
1344
-
1345
-## Lessons Extracted
1346
-
1347
-See `## Validation` lessons above. The durable schema, docs, and `create-topology` skill were updated so future topology producers preserve compactness, graph invariants, evidence, and presentation separation.
1348
-
1349
-## Followup
1350
-
1351
-- SOW-0021: topology presentation contract.
1352
-- SOW-0022: actor modal/table composition.
1353
-- SOW-0023: cross-payload actor matching and reconciliation.
1354
-- SOW-0024: vSphere topology v1 migration.
1355
-
1356
-## Regression Log
1357
-
1358
-None yet.
.agents/sow/done/SOW-0021-20260509-topology-presentation-contract.md
deleted
-947
@@ -1,947 +0,0 @@
1
-# SOW-0021 - Topology presentation contract
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed after inventory, schema/docs/skill updates, producer presentation updates, handoff documents, review rounds, and runtime/user validation of the repaired presentation behavior.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Restore polished, backend-controlled topology visualization while preserving the compact `netdata.topology.v1` payload model. The schema must let topology producers compose visual behavior from UI-supported tokens and profiles, without making the frontend know domain-specific concepts such as parent, child, router, client, server, process, switch, or endpoint.
14
-
15
-### User Request
16
-
17
-The user reported that the new compact topology schema lost important visual behavior from the prior topology payloads:
18
-
19
-- actor fill color, border color, annotation ring color, SVG/icon choice, and popover synthesis;
20
-- link color, line shape, curve direction, width, arrow direction, and popover synthesis;
21
-- streaming highlight-path behavior;
22
-- legend and color coding;
23
-- actor bullets/ports/sockets;
24
-- actor display-name synthesis, including a current failure where one actor label becomes a long concatenation of many MAC addresses;
25
-- Cloud aggregator behavior for conflicting presentation/type definitions when multiple topology payloads are merged;
26
-- Cloud aggregator matching strategies across payloads, for example socket endpoint matching, SNMP port/MAC matching, streaming machine identity matching, and future domain-specific matching.
27
-
28
-The user explicitly split the work:
29
-
30
-- SOW-0021: fix topology presentation.
31
-- SOW-0022: fix table and actor-modal composition.
32
-
33
-### Assistant Understanding
34
-
35
-Facts:
36
-
37
-- `netdata.topology.v1` reduced topology payload size by removing repeated row/object data and by separating graph links from evidence.
38
-- The old topology schema contained presentation metadata for actor types, link types, port types, legends, actor click behavior, actor modal tabs, and table hints.
39
-- The new topology schema currently has actor/link/evidence/table/overlay type registries, but no equivalent visual presentation contract.
40
-- The UI must remain domain-agnostic. It should provide rendering tokens/enums and rendering primitives, while the backend payload chooses how actor/link/table types use them.
41
-- Raw topology payloads and user examples may include hostnames, MAC addresses, IP addresses, interface aliases, private infrastructure details, usernames, masked passwords, and other sensitive or identifying data. Durable artifacts must only contain sanitized summaries.
42
-
43
-Inferences:
44
-
45
-- The current schema is strong for compact aggregation facts but weak for visual semantics.
46
-- Hardcoding domain names and producer-specific behavior in the UI would make the schema generic only on paper.
47
-- Reintroducing the old presentation object verbatim would preserve behavior quickly but would also preserve old schema ambiguity and bloat.
48
-- A compact, enum/token-based presentation plane attached to type registries is the likely correct replacement.
49
-
50
-Unknowns:
51
-
52
-- The full set of UI rendering tokens/enums available or required in `cloud-frontend`.
53
-- The exact conflict policy Cloud aggregation should use when payloads define the same actor/link/table type id with incompatible presentation profiles.
54
-- The complete inventory of old schema fields and frontend consumers that must be preserved, replaced, or deliberately dropped.
55
-
56
-### Acceptance Criteria
57
-
58
-- Inventory all old topology presentation fields, producer emissions, and frontend consumers, including legend, colors, icons, ports/bullets, highlight behavior, popovers, actor labels, link styles, and actor modal/table references.
59
-- Classify every old presentation capability as: preserve in v1, replace with a compact token/profile, move to SOW-0022, derive in UI from explicit backend tokens, or intentionally drop with evidence.
60
-- Run the requested external reviewers after the inventory: Claude, Codex, GLM, MiMo, Kimi, MiniMax, and Qwen. Prompts must be shown before execution, must be read-only, must include the SOW filename, and must ask reviewers to find missing presentation semantics, unwanted side effects, security/privacy issues, and aggregator conflict risks.
61
-- Extend `FUNCTION_TOPOLOGY_SCHEMA.json` with compact presentation/profile contracts that are enum/token based, not raw CSS/layout.
62
-- Extend `FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`, `.agents/sow/specs/topology-function-schema.md`, and `docs/netdata-ai/skills/create-topology/SKILL.md` so future topology producers know how to define presentation profiles.
63
-- Define Cloud aggregation merge/conflict policy for presentation profiles. Cross-payload actor reconciliation is tracked by SOW-0023.
64
-- Create worker handoff documentation for the Cloud frontend and Cloud topology aggregator.
65
-- Update backend producers enough to emit the new presentation contract for existing topology producers covered by this SOW.
66
-- Validate schema, docs, skill, and backend producer output with targeted tests or equivalent checks.
67
-
68
-## Analysis
69
-
70
-Sources checked:
71
-
72
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json`
73
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
74
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
75
-- `.agents/sow/specs/topology-function-schema.md`
76
-- `docs/netdata-ai/skills/create-topology/SKILL.md`
77
-- `.agents/skills/project-writing-collectors/SKILL.md`
78
-- `.agents/sow/current/SOW-0020-20260505-network-connections-topology-cloud-errors.md`
79
-
80
-Current state:
81
-
82
-- The old schema had `topology_presentation_actor_type` with `label`, `color_slot`, `opacity`, `border`, `role`, `size_by_links`, `show_port_bullets`, `icon_svg`, summary fields, tables, and modal tabs.
83
-- The old schema had `topology_presentation_link_type` with `label`, `color_slot`, `opacity`, `width`, and `dash`.
84
-- The old schema had `topology_presentation_port_type`, `topology_presentation_legend`, and `actor_click_behavior` with `highlight_connections` and `highlight_path`.
85
-- The new schema currently defines actors, links, evidence, detail tables, overlays, and aggregation metadata, but it has no presentation/profile plane.
86
-- The topology spec currently says the new network-connections producer no longer emits superseded presentation metadata. This SOW supersedes that statement by replacing old presentation metadata with compact v1 presentation profiles.
87
-
88
-### Inventory - 2026-05-09
89
-
90
-Old schema and shared Go model:
91
-
92
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json:277-324` defines actor, link, and port presentation types. Required actor/link/port fields include labels and `color_slot`; optional fields include opacity, border, role, size-by-links, port bullets, icon SVG, link width, and dashed links.
93
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json:349-363` defines presentation table metadata, including `source: "data" | "links"`, `bullet_source`, and display columns.
94
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json:376-424` defines legend entries, legend sections, and `actor_click_behavior: "highlight_connections" | "highlight_path"`.
95
-- `src/go/pkg/topology/types.go:57-136` mirrors the old presentation model in Go and states that it tells the UI how to render topology. This is useful as the old inventory source, but not the final v1 shape.
96
-- `src/go/pkg/topology/types.go:114-136` includes `port_fields` as Go-only presentation metadata. This is not represented in `FUNCTION_UI_SCHEMA.json`, but the UI consumes it for port bullet tooltips, so it is part of the real contract.
97
-
98
-Old producer emissions:
99
-
100
-- Network-connections old producer emitted:
101
- - actor profiles for `self`, `process`, and `endpoint`;
102
- - `self` and `process` used `size_by_links`;
103
- - `process` used `show_port_bullets`;
104
- - process socket table used `bullet_source`;
105
- - socket and ownership link types had color/width/dash settings;
106
- - port type `topology`;
107
- - actor/link/port legend;
108
- - `actor_click_behavior: "highlight_connections"`.
109
- Evidence: `git show HEAD:src/collectors/network-viewer.plugin/network-viewer.c`, lines 1734-2119 in the checked pre-v1 version.
110
-- Streaming old producer emitted:
111
- - actor profiles for `parent`, `child`, `vnode`, and `stale`;
112
- - parent used `show_port_bullets`, child/vnode/stale disabled it;
113
- - link profiles for `streaming`, `virtual`, and `stale`;
114
- - port profiles for `streaming`, `virtual`, and `stale`;
115
- - actor/link/port legend;
116
- - `actor_click_behavior: "highlight_path"`;
117
- - stream-path/retention/inbound/outbound table display metadata.
118
- Evidence: `git show HEAD:src/web/api/functions/function-topology-streaming.c`, lines 302-414 and 738-936 in the checked pre-v1 version.
119
-- SNMP old producer still has explicit presentation code:
120
- - device-like actor types map labels and color slots for router/switch/firewall/access point/server/storage/load balancer/printer/phone/UPS/camera;
121
- - device actor profiles use border, `size_by_links`, and `show_port_bullets`;
122
- - segment/endpoint profiles use their own colors/roles;
123
- - link types include LLDP/CDP/bridge/FDB/STP/ARP/SNMP/probable;
124
- - port fields and port type profiles drive port bullet popovers and legend.
125
- Evidence: `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation_types.go:7-134`.
126
-- SNMP emits `port_fields` with labels for `type`, `role`, `status`, `mode`, and `sources`, and the Function config attaches the presentation with `WithPresentation()`.
127
- Evidence: `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation_types.go:74-82` and `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation.go:16-50`.
128
-- SNMP old producer also defines curated summary fields and table columns for device, segment, endpoint, ports, and links.
129
- Evidence: `src/go/plugin/go.d/collector/snmp_topology/func_topology_presentation_schema.go:7-96`.
130
-- The vSphere worktree also has an old-schema topology producer that must be migrated after coordination with the worker in that directory. It defines actor profiles for datacenters, clusters, hosts, VMs, datastores, networks, datastore clusters, and resource pools; link profiles for `contains`, `connects`, and `runs`; a legend; and `actor_click_behavior: "highlight_connections"`.
131
- Evidence from the vSphere worktree: `src/go/plugin/go.d/collector/vsphere/func_topology_presentation.go:7-83`.
132
-- The vSphere producer attaches the old presentation to the Function method with `WithPresentation()` and emits inventory actors/links using the old topology package, so it is a real migration consumer, not only dead presentation code.
133
- Evidence from the vSphere worktree: `src/go/plugin/go.d/collector/vsphere/func_topology.go:33-42` and `src/go/plugin/go.d/collector/vsphere/func_topology.go:74-245`.
134
-
135
-Current v1 schema and producers:
136
-
137
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:220-243` has a type registry for actor/link/evidence/table/overlay/aggregation types, but no presentation/profile registry.
138
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:304-399` defines actor and link semantics for identity, layer, direction, and aggregation, but no labels, colors, icons, line styles, bullets, legend, or highlight behavior.
139
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:434-523` defines evidence and table type roles/columns/aggregation, but no modal or table display composition.
140
-- `src/go/pkg/topology/v1/types.go:72-116` mirrors that same v1 gap in producer-side Go types.
141
-- Current v1 network-connections emits a `display_name` actor column and socket evidence columns, but no presentation profile.
142
- Evidence: `src/collectors/network-viewer.plugin/network-viewer.c:2130-2172`.
143
-- Current v1 streaming emits display names, machine GUIDs, link/evidence/table types, and stream-path actor tables, but no presentation profile or highlight-path contract.
144
- Evidence: `src/web/api/functions/function-topology-streaming.c:1043-1272`.
145
-- Current v1 SNMP adapter emits actor metadata as compact rows and JSON table data, but no v1 presentation profile.
146
- Evidence: `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:386-407` and `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:451-483`.
147
-
148
-Current cloud-frontend behavior:
149
-
150
-- Legacy normalizer still has custom display-name synthesis from attributes, labels, and match fields; it avoids using raw match arrays as the first display choice.
151
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/legacy/normalizeLegacyTopology.js:96-121`.
152
-- V1 actor normalizer has a weaker `deriveLabel()` fallback. If `display_name`, name, hostname, address, or id are absent, it can fall back to actor type or generated identity; arrays are joined with commas by `safeString()`. This is consistent with the user-observed long aggregated MAC actor label class.
153
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/v1/buildActors.js:9-18` and `src/domains/functions/topology/v1/buildActors.js:46-56`.
154
-- V1 actor rows and nodes expose raw decoded row values directly as attributes.
155
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/v1/buildActors.js:106-129`.
156
-- The force graph still consumes old presentation keys for click behavior, port bullets, actor visuals, icon SVG, port colors, and legend.
157
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/graph/forceGraph.js:601-645`, `src/domains/functions/components/graph/forceGraph.js:711-754`, and `src/domains/functions/components/graph/forceGraph.js:756-818`.
158
-- The graph legend is entirely driven by `presentation.legend` plus actor/link/port profile maps.
159
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/graph/graphLegend.js:41-104`.
160
-- Color slots are already UI-owned tokens, not backend hex colors. The backend currently chooses slot names and the UI resolves them to theme colors/widths/opacities.
161
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/colorSlots.js:9-65`.
162
-- The current frontend color slot vocabulary is `primary`, `secondary`, `accent`, `self`, `neutral`, `muted`, `dim`, `derived`, `info`, `structural`, and `warning`. Old network-connections, streaming, and SNMP use these tokens, but the vSphere worktree uses hue names such as `blue`, `green`, `orange`, `purple`, `cyan`, `yellow`, `teal`, and `gray`, which are not present in the current frontend slot table.
163
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/colorSlots.js:9-65`. Evidence from the vSphere worktree: `src/go/plugin/go.d/collector/vsphere/func_topology_presentation.go:10-23`.
164
-- The canvas currently draws straight links only. It supports line color, opacity, width, and dash via presentation link types, but it has no generic curve/arrow enum yet.
165
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/graph/forceGraphCanvas.js:63-82` and `src/domains/functions/components/graph/forceGraphCanvas.js:132-205`.
166
-- The current frontend icon token vocabulary is closed and UI-owned: `router`, `switch`, `firewall`, `access_point`, `server`, `storage`, `load_balancer`, `printer`, `phone`, `ups`, `camera`, `process`, `agent`, `netdata-agent`, `parent`, `remote-endpoint`, `local-endpoint`, `segment`, `self`, `ip`, `cloud`, `container`, `vm`, `database`, and `service`.
167
- Evidence from the cloud-frontend worktree: `src/domains/functions/topology/icons.js:5-260`.
168
-- Current v1 actor modal bypasses legacy presentation table tabs and shows a single `V1 data` tab.
169
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/topology/actorModal/index.js:97-107`, `src/domains/functions/components/topology/actorModal/index.js:152-162`, and `src/domains/functions/components/topology/actorModal/index.js:319-364`.
170
-- Current v1 actor modal renderer stringifies objects and overlays directly, which explains raw JSON leaking into final UI.
171
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/topology/actorModal/V1ActorPanel.js:26-30`, `src/domains/functions/components/topology/actorModal/V1ActorPanel.js:130-168`, and `src/domains/functions/components/topology/actorModal/V1ActorPanel.js:257-263`.
172
-- Legacy `summary_fields` are rendered by the frontend actor modal info panel, and legacy `port_fields` are rendered by the graph port bullet tooltip path. This means both are presentation-adjacent contracts even if full table/modal composition remains SOW-0022.
173
- Evidence from the cloud-frontend worktree: `src/domains/functions/components/topology/actorModal/index.js:277-282` and `src/domains/functions/components/graph/forceGraph.js:1015-1044`.
174
-
175
-Current cloud-topology-service behavior:
176
-
177
-- The service schema copy has no presentation fields in `Data`, `TypeRegistry`, `ActorType`, `LinkType`, `EvidenceType`, or `TableType`.
178
- Evidence from the cloud-topology-service repo: `internal/topology/schema/payload.go:29-110`.
179
-- The aggregation core merges type registry definitions by normalized deep equality and returns a hard error for conflicting definitions.
180
- Evidence from the cloud-topology-service repo: `internal/topology/aggregate/aggregate.go:984-1048`.
181
-- The aggregation spec already says type registry entries with the same id must be semantically compatible, and conflicting definitions are aggregation errors.
182
- Evidence from the cloud-topology-service repo: `.agents/sow/specs/cloud-topology-service-contract.md:121-123`.
183
-- The current aggregation core merges actors by actor type plus `merge_identity`, otherwise `identity`, and links by remapped endpoints plus link type and direction policy.
184
- Evidence from the cloud-topology-service repo: `internal/topology/aggregate/aggregate.go:196-246`, `internal/topology/aggregate/aggregate.go:249-300`, and `.agents/sow/specs/cloud-topology-service-contract.md:124-128`.
185
-- The service contract says evidence type match columns preserve exact relationship details, but the current model does not yet define a cross-payload matcher strategy that can replace one topology's endpoint with another topology's actor using domain-specific keys.
186
- Evidence from the cloud-topology-service repo: `.agents/sow/specs/cloud-topology-service-contract.md:103-116`.
187
-
188
-Gap classification:
189
-
190
-- Preserve in v1 as compact presentation/profile metadata:
191
- - actor type label;
192
- - actor fill color token;
193
- - actor border color/style token;
194
- - actor annotation ring token;
195
- - actor role/render role;
196
- - actor icon token;
197
- - actor size-by policy;
198
- - actor label/display-name synthesis policy;
199
- - link type label;
200
- - link color token;
201
- - link line shape token;
202
- - link curve token;
203
- - link width token;
204
- - link direction/arrow token;
205
- - port/bullet type label and color token;
206
- - legend entries/order;
207
- - graph selection/highlight behavior.
208
-- Replace old fields with safer tokens/profiles:
209
- - old `color_slot` stays as token semantics but should become explicit enum/profile vocabulary;
210
- - old raw `icon_svg` should become UI-owned `icon` token unless a controlled signed/allowlisted custom icon registry is explicitly approved;
211
- - old boolean `dash` should become line shape enum such as `solid`, `dotted`, `dashed`;
212
- - old numeric `width` should become width token such as `thin`, `normal`, `thick`, or bounded scalar if the UI team confirms safe limits;
213
- - old `actor_click_behavior` should become a selection/highlight profile with composition rules, not a single global string.
214
-- Move to SOW-0022:
215
- - summary field composition;
216
- - table columns, column formatters, and modal tab composition;
217
- - raw JSON/nested array rendering rules;
218
- - actor/link modal table grouping;
219
- - derived relationship evidence drilldowns;
220
- - safe display of endpoint objects, neighbors, port inventory, and overlays.
221
-- Add because old schema did not cover enough:
222
- - actor `label_policy` / display synthesis, to prevent canonical identity arrays from becoming actor labels;
223
- - explicit note that cross-payload actor/link reconciliation is structural and tracked by SOW-0023;
224
- - presentation conflict policy for Cloud aggregation;
225
- - link curve and arrow tokens for bidirectional/directed rendering;
226
- - popover profile references for actor/link hover summaries;
227
- - annotation ring semantics for status/classification overlays.
228
-- Add to the preserve/replace matrix because reviewers and local verification found missing old behavior:
229
- - `port_fields`, at least as UI label metadata for port bullet tooltip fields;
230
- - actor/link/port opacity semantics, preferably as closed opacity tokens rather than arbitrary floats;
231
- - old `topology_match` display-relevant fields as input to the new `label_policy`, not as a raw object to recreate. Cross-payload identity vocabulary moves to SOW-0023.
232
-- Intentionally keep out of the payload:
233
- - coordinates, force-layout physics, viewport, pan/zoom, z-index, CSS class names, raw theme colors, raw CSS, component names, and user runtime interaction state.
234
-
235
-Required design outputs before implementation:
236
-
237
-1. Presentation profile schema attached to type registry entries and/or a top-level presentation registry.
238
-2. UI token vocabulary for actor/link/port visuals that is stable and documented.
239
-3. Label synthesis policy that separates canonical identity from human display.
240
-4. Highlight profile schema for direct-neighbor, path, and future neighborhood behaviors.
241
-5. Cloud aggregation conflict policy for presentation profile disagreements.
242
-6. SOW-0022 handoff with the modal/table composition inventory above.
243
-7. SOW-0023 handoff for cross-payload actor reconciliation.
244
-
245
-Risks:
246
-
247
-- If presentation stays out of the payload, the UI will need producer/domain-specific hardcoding and future topologies will not render consistently.
248
-- If raw CSS, SVG, layout coordinates, or frontend component names enter the payload, the schema will couple backend producers to frontend implementation details.
249
-- If Cloud aggregation accepts conflicting profiles silently, merged topologies may show inconsistent colors, icons, arrows, legends, or highlight behavior.
250
-- Until SOW-0023 teaches Cloud cross-payload match strategies, Cloud cannot safely replace endpoints from one topology with actors from another.
251
-- If actor labels are synthesized from canonical identity without display rules, the UI can show unusable labels such as long concatenated identity lists.
252
-- If presentation fields allow raw untrusted HTML/SVG, the UI could get a security-sensitive rendering surface.
253
-- If the schema is designed only around the three current in-tree producers, the vSphere topology in the companion worktree will either need a special UI path or will regress during migration.
254
-
255
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
256
-
257
-Status at implementation start: ready-for-implementation (historical snapshot; final closure evidence is in the Validation and Outcome sections).
258
-
259
-Problem / root-cause model:
260
-
261
-- The compact schema removed the old presentation plane together with verbose compatibility data. This improved payload size but also removed backend-controlled visual semantics that the UI needs to render generic topologies without producer-specific hardcoding.
262
-- Actor display names and labels are currently not modeled as a first-class presentation contract. Producers may emit canonical or aggregated identities that are correct for matching but unusable as human labels.
263
-- Cloud aggregation cannot safely merge presentation profiles until the schema declares how profiles are identified, versioned, and resolved.
264
-
265
-Evidence reviewed:
266
-
267
-- `src/plugins.d/FUNCTION_UI_SCHEMA.json` defines the old topology presentation objects.
268
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` lacks equivalent presentation/profile definitions.
269
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md` currently warns against visual layout hints but does not distinguish UI-owned layout from backend-owned visual semantics.
270
-- User-provided UI observation shows raw/aggregated identity and raw data currently leak into final topology presentation. Raw examples are intentionally not copied into this durable artifact.
271
-
272
-Affected contracts and surfaces:
273
-
274
-- `netdata.topology.v1` JSON Schema.
275
-- Topology producer output for network-connections, streaming, SNMP/L2, and future vSphere topology.
276
-- Cloud frontend topology renderer, graph legend, popovers, highlights, actor labels, and icon/color/line token mappings.
277
-- Cloud topology aggregation service type/profile merge logic.
278
-- Public `create-topology` skill and developer guide.
279
-- Topology schema spec under `.agents/sow/specs/`.
280
-
281
-Existing patterns to reuse:
282
-
283
-- Old `FUNCTION_UI_SCHEMA.json` presentation metadata as inventory input, not as a direct replacement.
284
-- New v1 type registry model in `FUNCTION_TOPOLOGY_SCHEMA.json`.
285
-- Existing compact-table and type-registry helpers in `src/go/pkg/topology/v1`.
286
-- Existing topology producer split between graph links, evidence, tables, and overlays.
287
-- UI-token model proposed by the user: UI provides enums/primitives; backend composes profiles using those enums.
288
-
289
-Risk and blast radius:
290
-
291
-- Schema changes affect every producer and both Cloud consumers.
292
-- The frontend must support old and new schemas during rollout.
293
-- Cloud aggregation must not conflate incompatible visual/type definitions.
294
-- Existing large-payload gains must not be lost by adding repeated per-row presentation data.
295
-- Raw topology captures can contain sensitive customer/infrastructure details and must remain out of durable artifacts.
296
-
297
-Sensitive data handling plan:
298
-
299
-- Do not copy raw topology examples, raw MAC/IP addresses, interface aliases, hostnames, usernames, passwords, tokens, node IDs, claim IDs, or customer-identifying data into SOWs, specs, docs, skills, code comments, commits, or PR text.
300
-- Use sanitized summaries and generic examples only.
301
-- Keep raw captured payloads under `.local/` only.
302
-- If fixtures are needed, generate sanitized fixtures with placeholder identifiers and no real infrastructure values.
303
-
304
-Implementation plan:
305
-
306
-1. Inventory old presentation contract, old producer emissions, current v1 schema gaps, and frontend consumption points.
307
-2. Run the requested external read-only reviewers against the inventory and this SOW.
308
-3. Resolve presentation schema decisions, especially profile shape, token sets, label synthesis, highlight composition, and Cloud aggregation conflict policy.
309
-4. Extend schema/docs/spec/skill with compact presentation profiles and SOW-0023 handoff notes for cross-payload matching.
310
-5. Create UI and Cloud aggregator worker handoff documents.
311
-6. Update backend producers to emit the presentation profiles and validate outputs.
312
-
313
-Validation plan:
314
-
315
-- JSON Schema validation for fixtures using presentation profiles.
316
-- Targeted tests for Go topology v1 helpers if helper structs/builders change.
317
-- Function validation for at least one producer output per topology kind touched.
318
-- Same-failure scan for old visual fields and current v1 gaps.
319
-- Reviewer pass before schema freeze and, if material changes are made after reviewer findings, repeat review with the same scope plus fix notes.
320
-
321
-Artifact impact plan:
322
-
323
-- AGENTS.md: no expected update unless workflow rules change.
324
-- Runtime project skills: likely no update except existing `project-writing-collectors` references if topology producer workflow changes.
325
-- Specs: update `.agents/sow/specs/topology-function-schema.md`.
326
-- End-user/operator docs: update `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`; no public user docs expected unless Function schema docs are published.
327
-- End-user/operator skills: update `docs/netdata-ai/skills/create-topology/SKILL.md`.
328
-- SOW lifecycle: SOW-0020 paused; SOW-0021 current; SOW-0022 pending for table/modal composition.
329
-
330
-Open-source reference evidence:
331
-
332
-- `kiali/kiali @ ad210d7fd2a4b819e6ceae5f9a744847c4dcc7b2`, `frontend/src/types/Graph.ts:384-451` models graph nodes and edges with domain-specific fields such as node type, namespace, traffic, health, and source/target. This is useful evidence that mature topology UIs often carry semantic graph data, but it is not a generic presentation-token contract Netdata can copy directly.
333
-- `apache/skywalking @ 4890024b6cc1c222838b5ebd16e10938762cd7f2`, `oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java:27-58` and `oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Call.java:31-143` keep backend topology facts around nodes, calls, components, and detect points.
334
-- `apache/skywalking-booster-ui @ 0dfb65bad317fae75353dc4ff89fae663d5e1dc5`, `src/types/topology.ts:18-74` includes UI-side fields such as positions and lower-arc hints. This supports keeping coordinates, physics, and layout state out of Netdata producer payloads while still allowing backend-owned semantic style tokens.
335
-
336
-Open decisions:
337
-
338
-- Resolved in `## Implications And Decisions` on 2026-05-09. Remaining implementation details are schema design details unless new evidence exposes another product decision.
339
-
340
-## Implications And Decisions
341
-
342
-1. User decision: split the remediation into two SOWs.
343
- - SOW-0021 fixes topology presentation.
344
- - SOW-0022 fixes table and actor-modal composition.
345
- - Reason: presentation profiles and table composition are related but separable contracts, and separating them allows smaller reviewable changes.
346
-
347
-2. User decision: UI must be domain-agnostic.
348
- - The UI provides enums/tokens/primitives.
349
- - Backend payloads compose actor/link/highlight/modal behavior using those tokens.
350
- - The UI must not hardcode meanings such as parent, child, router, client, server, or process.
351
-
352
-3. User decision: new compact topology payloads are all-inclusive for presentation.
353
- - Production `netdata.topology.v1` payloads carry presentation definitions.
354
- - Function `info` must not be the production transport for the new presentation contract.
355
- - Function `info.presentation` may exist only for legacy schema compatibility during rollout.
356
-
357
-4. User decision: actor/link/etc. type definitions are the collision surface, not actor/link rows.
358
- - Payloads define actor types, link types, port types, and presentation profiles once.
359
- - Actor and link rows only reference those definitions.
360
- - Cloud aggregation conflicts are about contradictory type/profile definitions, for example two payloads defining the same `router` type differently.
361
-
362
-5. User decision: raw SVG/custom icon payloads are not allowed in the new compact schema.
363
- - Use UI-owned icon tokens only.
364
- - Raw SVG, raw CSS, and producer-owned rendering code are excluded from `netdata.topology.v1` presentation.
365
-
366
-6. Scope decision: keep SOW-0021 and SOW-0022 separate, but make the boundary explicit.
367
- - SOW-0021 owns graph presentation, safe actor labels, legend, highlight behavior, link style tokens, and port/socket bullets needed on the graph.
368
- - SOW-0022 owns full actor/link modal composition, custom tables, shown/hidden columns, formatters, nested JSON rendering, and curated drilldown views.
369
- - Rationale: merging the SOWs would increase the review surface and risk mixing graph rendering with table composition. The split is safe only if SOW-0021 provides the graph-facing hooks and SOW-0022 consumes them without redefining them.
370
-
371
-7. Scope decision: cross-payload actor reconciliation becomes SOW-0023.
372
- - SOW-0021 documents the requirement but does not implement the matcher.
373
- - SOW-0023 will define cross-payload identity vocabularies, matching strategies, ambiguity handling, endpoint replacement rules, and Cloud aggregator behavior.
374
-
375
-8. User decision: Cloud presentation conflicts should prefer the newest producer/schema definition.
376
- - Producers must expose enough version information for Cloud to compare presentation/type definitions deterministically.
377
- - Cloud should prefer the newer Netdata/function/schema definition when the same presentation/type id has conflicting visual definitions.
378
- - If version comparison cannot break the tie, Cloud may choose one deterministic fallback profile and record diagnostics.
379
- - Structural facts are not considered contradictions only because two producers do not correlate; they either match or remain separate valid observations.
380
- - Superseded by decision 9 for raw aggregation: newest-wins can be used only as a display-preference heuristic after canonicalization, never to drop facts, rows, or distinct definitions.
381
-
382
-9. User decision refinement: type/profile definition collisions should be namespaced, then deduplicated.
383
- - Cloud should treat producer-local type/profile ids as local names until aggregation resolves them.
384
- - Cloud should canonicalize definitions and deduplicate identical definitions by content.
385
- - If two local ids have different definitions, Cloud should keep both by assigning distinct canonical ids instead of failing aggregation or dropping one.
386
- - This creates no data loss and no fatal definition conflicts.
387
- - Version preference can still be used later to choose a preferred display profile when multiple variants are semantically equivalent, but raw aggregation must preserve all variants.
388
-
389
-10. User decision: link width/opacity can be data-scaled, but scaling is keyed per link type.
390
- - Link type definitions declare whether a visual channel is variable and the scale key it belongs to.
391
- - Link rows may carry one raw numeric weight for that declared key.
392
- - The raw number is producer-domain-specific, for example traffic, packet count, socket count, or another unit.
393
- - Producers do not pre-scale to pixels or opacity because each payload does not know the whole aggregated range.
394
- - Cloud/UI scale visual width or opacity per scale key across visible links that share that key.
395
- - Only one variable visual weight is allowed per link row in this SOW.
396
-
397
-11. Link scaling examples accepted for schema design:
398
- - A `bandwidth` link type can declare scale key `traffic`; link rows carry raw traffic such as KiB.
399
- - A `connections` link type can declare scale key `sockets`; link rows carry raw socket counts.
400
- - These two link types can coexist because Cloud/UI scale `traffic` links against other `traffic` links and `sockets` links against other `sockets` links.
401
-
402
-12. User clarification: link rows remain compact table rows, not per-link JSON objects.
403
- - Example object notation in discussion is illustrative only.
404
- - Production link data stays in the compact table shape: columns plus column encodings.
405
- - The link row carries a link type reference and, if the type declares a variable visual channel, the raw numeric value column declared by that link type.
406
-
407
-13. User clarification and schema decision: link type and scale key are separate concepts.
408
- - `type` identifies the semantic link type, such as dependency, bandwidth, connections, LLDP, ownership, or streaming.
409
- - `scale_key` groups raw numeric weights that can be scaled together, such as `traffic` or `sockets`.
410
- - Multiple link types may share one `scale_key`.
411
- - A link type may have no variable scale key.
412
- - Link rows should not repeat `scale_key`; Cloud/UI resolve `row.type -> link_type.presentation.variable.scale_key`.
413
- - Link type values in compact tables should use dictionary/string-ref encoding so repeated type names are stored as compact indexes in row data.
414
- - Scale-key definitions are small type-level metadata. They may use stable ids, but clarity is preferred over numeric-only ids unless measurement shows type-level definitions matter for payload size.
415
-
416
-14. User decision: presentation is attached inside actor, link, and port type definitions.
417
- - Each actor, link, and port type can carry a `presentation` object that defines backend-selected UI tokens for that type.
418
- - This keeps a type as one object that defines both its structural meaning and its visual profile.
419
- - Cloud aggregation still treats producer-local ids as local names, namespaces them, and deduplicates canonical definitions.
420
- - If two payloads use the same local id with different presentation, Cloud keeps both as distinct canonical ids instead of failing or dropping one.
421
- - Structural identity and display presentation remain conceptually separate even though they live in the same type object.
422
-
423
-15. User decision: execution order is SOW-0021, then SOW-0023, then SOW-0022.
424
- - SOW-0021 will finish graph presentation, safe labels, legend, highlighting, type-level presentation, and graph-facing port/socket bullets first.
425
- - SOW-0023 will then solve cross-payload identity, matching, correlation, ambiguity policy, and endpoint replacement before full modal/table composition.
426
- - SOW-0022 will then complete actor/link modal and table composition on top of the presentation and identity foundations.
427
- - Reason: actor identity and naming affect topology shape, but the immediate graph regression still needs the SOW-0021 presentation contract first.
428
-
429
-16. Identity guardrail for SOW-0021:
430
- - Actor identity is stable and canonical; it is never used directly as display text unless explicitly marked safe by the producer.
431
- - Display labels come from explicit presentation/label policy.
432
- - Type/profile ids are producer-local until Cloud namespaces and deduplicates them.
433
- - Cross-payload actor reconciliation is not implemented in SOW-0021, but SOW-0021 must not add presentation rules that block SOW-0023.
434
-
435
-17. vSphere migration posture:
436
- - The vSphere topology in the separate worktree is not migrated by SOW-0021.
437
- - SOW-0021 keeps the vSphere-required color/icon tokens in the schema so the later migration does not need another schema round.
438
- - The frontend handoff explicitly requires fallback and concrete mappings for these tokens before vSphere uses them.
439
-
440
-## Reviewer Findings - 2026-05-09
441
-
442
-Seven requested read-only reviewer agents were run in parallel. Their raw outputs are stored under `.local/audits/topology-presentation-contract/reviews/` and are intentionally not committed.
443
-
444
-Consolidated findings:
445
-
446
-1. Presentation transport is unresolved.
447
- - Old `topology.Presentation` is Function `info` metadata.
448
- - Current cloud-frontend reads `response.presentation` from the Function info response.
449
- - The Cloud topology service aggregates topology payload `data`, not Function info metadata.
450
- - Risk: putting v1 presentation in the wrong place can make either the UI or the aggregator blind to it.
451
-
452
-2. Presentation should not be mixed into structural type definitions without an explicit policy.
453
- - Cloud aggregation currently hard-errors on conflicting type definitions.
454
- - If visual fields are embedded in `actor_types` or `link_types`, a harmless color/label difference can become an aggregation failure.
455
- - A separate presentation registry allows structural semantics and visual semantics to have different merge rules.
456
-
457
-3. Port bullets are graph presentation, not only table composition.
458
- - Old producers used `show_port_bullets`, `port_types`, `port_fields`, and `bullet_source`.
459
- - The frontend uses `port_fields` and `port_types` in the force graph tooltip path.
460
- - SOW-0021 must preserve enough port-bullet metadata for graph polish; SOW-0022 can still own full modal/table composition.
461
-
462
-4. Actor labels need a first-class safe label policy.
463
- - Current v1 frontend fallback joins arrays and can expose long identity lists as labels.
464
- - The policy must define safe source columns, fallback order, max length, array rejection or summarization, and what the UI does when no label source is safe.
465
-
466
-5. Raw SVG must not remain an open-ended producer surface.
467
- - The old schema allowed `icon_svg`.
468
- - Current frontend has regex-based SVG stripping before rendering legacy icons, but this is not a sufficient long-term security boundary.
469
- - Local search found no current topology producer emission of `icon_svg` in network-connections, streaming, SNMP, or the vSphere worktree. This means a closed UI-token icon model can be adopted without preserving active producer SVG output.
470
-
471
-6. Token vocabularies must be explicit and versioned.
472
- - Current cloud-frontend supports color slots `primary`, `secondary`, `accent`, `self`, `neutral`, `muted`, `dim`, `derived`, `info`, `structural`, and `warning`.
473
- - The vSphere worktree uses color slot names not present in that vocabulary.
474
- - Current cloud-frontend supports a closed icon token map, but schema/docs do not list or version the tokens.
475
-
476
-7. Cross-payload matching is structural, not just visual.
477
- - `merge_identity` is per actor type and does not define shared identity classes across producers.
478
- - Evidence `match_columns` preserve exact relationship details but do not declare how to replace one topology's endpoint with another topology's actor.
479
- - Network-connections, SNMP/L2, streaming, and vSphere need different identity keys and ambiguity policies.
480
-
481
-8. Presentation conflict policy must be explicit.
482
- - Options include hard error, first-wins, priority-based merge, or separate profile IDs with deterministic fallback.
483
- - Silent merging is unsafe; hard errors on visual-only differences are operationally fragile.
484
-
485
-9. Existing docs and skills currently contradict the new direction.
486
- - The topology spec says the producer no longer emits superseded presentation metadata.
487
- - The create-topology skill does not describe presentation profiles.
488
- - This SOW must update both so future producers do not repeat the regression.
489
-
490
-10. Cloud service schema/validator drift was found outside pure presentation.
491
- - Agent-side v1 schema allows `json` columns.
492
- - SNMP v1 emits `json` columns for metadata tables.
493
- - The Cloud topology service validator currently does not allow `json` column type and scalar validation rejects objects.
494
- - This must be handed to the Cloud aggregator worker because otherwise valid Agent payloads can be rejected before presentation is considered.
495
-
496
-11. SNMP actor subtype information needs preservation analysis.
497
- - The old presentation registry has distinct profiles for router, switch, firewall, access point, server, storage, load balancer, printer, phone, UPS, camera, generic device, endpoint, and segment.
498
- - Current v1 SNMP type registry exposes only device, endpoint, segment, and custom actor types.
499
- - The backend must preserve enough subtype or classification data for presentation profiles to reproduce the old color/icon distinctions where the old producer actually emitted those actor types.
500
-
501
-12. Curve and arrow tokens require UI support but are still schema-relevant.
502
- - Current canvas draws straight lines and has no arrowhead rendering.
503
- - The user explicitly requires backend-controlled line shape, curve, width, and arrow direction.
504
- - The schema can define closed tokens now, but the UI worker must implement graceful fallback for unsupported tokens.
505
-
506
-## Decision Gate - 2026-05-09
507
-
508
-These decisions are blocking schema and backend implementation. They are written here before implementation so the user can answer by number and option letter.
509
-
510
-Resolution note: this gate is resolved by `## Implications And Decisions`. Decision 14 supersedes the original recommendation in Decision Gate item 2: presentation is attached inside actor, link, and port type definitions, with Cloud namespacing and deduplication preventing visual-only conflicts from becoming data loss or aggregation failure.
511
-
512
-### 1. Where does v1 presentation live?
513
-
514
-A. **Recommended: `data.presentation` in the topology payload, with legacy Function `info.presentation` kept only for old-schema compatibility during rollout.**
515
-
516
-- Pros: Cloud aggregator receives presentation; UI can render from the same production payload; avoids duplication after rollout.
517
-- Cons: cloud-frontend must learn to read v1 presentation from `data`.
518
-- Implications: Function info remains a compatibility path only; schema, docs, producer helpers, frontend, and cloud-topology-service all share one production presentation contract.
519
-- Risks: requires coordinated UI and Cloud service updates before end-to-end validation.
520
-
521
-B. Keep presentation only in Function `info`.
522
-
523
-- Pros: matches old UI transport.
524
-- Cons: Cloud aggregator cannot merge or validate presentation because it is not in topology `data`.
525
-- Implications: cloud-topology-service cannot be the single topology path for all topology kinds.
526
-- Risks: repeats the split-brain old/new contract.
527
-
528
-C. Duplicate presentation in both Function `info` and topology `data`.
529
-
530
-- Pros: easiest frontend transition.
531
-- Cons: larger payloads and possible divergence between two copies.
532
-- Implications: every producer must keep two presentation surfaces synchronized.
533
-- Risks: stale info/data presentation mismatches create hard-to-debug UI behavior.
534
-
535
-### 2. How should presentation attach to schema?
536
-
537
-A. **Recommended: separate top-level `presentation` registry parallel to `types`.**
538
-
539
-- Pros: structural type compatibility remains separate from visual profile compatibility; Cloud can merge profiles with a presentation-specific policy.
540
-- Cons: actor/link/port types need explicit profile references or same-id conventions.
541
-- Implications: schema can evolve visual profiles without redefining topology identity.
542
-- Risks: frontend and service need one more registry to resolve.
543
-
544
-B. Inline presentation fields inside `types.actor_types` and `types.link_types`.
545
-
546
-- Pros: simpler producer shape.
547
-- Cons: visual differences become type definition conflicts unless aggregation is weakened.
548
-- Implications: type registry stops being purely structural.
549
-- Risks: harmless visual changes can break Cloud aggregation.
550
-
551
-C. Put presentation under `extensions`.
552
-
553
-- Pros: fastest schema bypass.
554
-- Cons: no validation, no stable contract, no reliable aggregation policy.
555
-- Implications: future producers will invent incompatible shapes.
556
-- Risks: recreates the current regression with another untyped escape hatch.
557
-
558
-### 3. What is the icon/SVG policy?
559
-
560
-A. **Recommended: closed UI-owned icon tokens only; raw SVG and raw CSS are banned from v1 presentation.**
561
-
562
-- Pros: safest security boundary; compact; frontend can version tokens.
563
-- Cons: producers can only use icons the UI exposes.
564
-- Implications: add missing icon tokens to the UI token catalog as needed.
565
-- Risks: a producer needing a custom icon waits for a UI token addition.
566
-
567
-B. Allow raw SVG with sanitizer.
568
-
569
-- Pros: preserves maximum producer flexibility.
570
-- Cons: regex sanitization is not a strong security boundary; proper SVG sanitization is non-trivial.
571
-- Implications: every topology payload becomes a UI rendering security surface.
572
-- Risks: stored XSS or broken rendering through sanitizer bypasses.
573
-
574
-C. Design a signed/allowlisted custom icon registry now.
575
-
576
-- Pros: flexible and safer than raw inline SVG.
577
-- Cons: substantially larger security, distribution, signing, revocation, and on-prem/offline design.
578
-- Implications: delays this SOW.
579
-- Risks: scope creep and incomplete security model.
580
-
581
-### 4. What belongs to SOW-0021 vs SOW-0022?
582
-
583
-A. **Recommended: SOW-0021 owns graph presentation and safe labels; SOW-0022 owns modal/table composition.**
584
-
585
-- SOW-0021 includes actor/link/port profiles, legend, line shape, width, curve/arrow tokens, highlight profiles, `label_policy`, `show_port_bullets`, `port_types`, and `port_fields` needed for graph port tooltips.
586
-- SOW-0022 includes modal tabs, summary sections, custom tables, table columns, formatters, nested JSON rendering, and raw-data suppression inside actor/link modals.
587
-- Pros: fixes visible graph polish first while preserving the user's two-step split.
588
-- Cons: some legacy fields, especially `summary_fields`, sit near the boundary and need explicit handoff notes.
589
-- Implications: SOW-0021 must not attempt to fully solve actor modals; SOW-0022 must not redefine graph label/profile tokens.
590
-- Risks: port tooltip and hover summary details need careful ownership to avoid gaps.
591
-
592
-B. Move all summaries, popovers, port fields, and modal tabs to SOW-0022.
593
-
594
-- Pros: strict table/modal ownership.
595
-- Cons: SOW-0021 can restore colors but still leave bullets/popovers weak.
596
-- Implications: the graph may remain visibly incomplete until SOW-0022.
597
-- Risks: fails the "actor bullets/ports/sockets" part of the presentation complaint.
598
-
599
-C. Move modal tabs and summary fields into SOW-0021 too.
600
-
601
-- Pros: closer parity with the old `PresentationActorType` bundle.
602
-- Cons: expands SOW-0021 into table/modal composition and violates the requested two-step split.
603
-- Implications: SOW-0022 becomes smaller but SOW-0021 becomes much larger.
604
-- Risks: delays the graph presentation repair.
605
-
606
-### 5. Is cross-payload actor reconciliation in SOW-0021?
607
-
608
-A. **Recommended: SOW-0021 documents the requirement and creates/updates a separate SOW for cross-payload matching; it does not implement the matching algorithm.**
609
-
610
-- Pros: keeps SOW-0021 focused on presentation; acknowledges the issue honestly; avoids mixing visual schema work with structural graph reconciliation.
611
-- Cons: aggregated multi-producer topologies may still show duplicates until the matching SOW is implemented.
612
-- Implications: presentation schema can include only non-invasive identity labels needed by UI; matching strategy, ambiguity handling, and endpoint replacement live in the follow-up.
613
-- Risks: future schema changes may be needed if the matching SOW requires additional producer declarations.
614
-
615
-B. Add producer-declared match strategies to v1 now, but leave Cloud implementation to a later service change.
616
-
617
-- Pros: future-proofs the payload.
618
-- Cons: unimplemented schema fields can drift or be misused.
619
-- Implications: producers must emit strategy metadata before the aggregator consumes it.
620
-- Risks: false confidence that matching works because the payload has declarations.
621
-
622
-C. Include full cross-payload matching schema and Cloud aggregator algorithm in SOW-0021.
623
-
624
-- Pros: solves the structural duplicate-actor problem now.
625
-- Cons: significantly expands scope beyond presentation.
626
-- Implications: SOW-0021 must define typed identity vocabularies, normalization, ambiguity policy, match confidence, and tests across all topology kinds.
627
-- Risks: delays presentation repair and increases review surface.
628
-
629
-### 6. What is the presentation conflict policy for Cloud aggregation?
630
-
631
-A. **Recommended: structural conflicts stay hard errors; presentation conflicts use deterministic profile merge with diagnostics, not topology failure.**
632
-
633
-- Pros: preserves correctness for topology identity while avoiding visual-only aggregation outages.
634
-- Cons: a merged view may pick one visual profile when producers disagree.
635
-- Implications: Cloud service must record conflict counts/details and choose profiles by deterministic priority, such as explicit profile priority then producer/source ordering.
636
-- Risks: users may see a generic or first-selected style when producers disagree.
637
-
638
-B. Hard-error on any presentation conflict.
639
-
640
-- Pros: simplest and maximally strict.
641
-- Cons: visual-only differences can prevent topology rendering.
642
-- Implications: every producer version must agree exactly on profile definitions before Cloud aggregation works.
643
-- Risks: fragile during rolling upgrades.
644
-
645
-C. Last-writer-wins silently.
646
-
647
-- Pros: easy implementation.
648
-- Cons: nondeterministic unless merge order is guaranteed; hides real producer disagreements.
649
-- Implications: UI may change colors/icons depending on aggregation order.
650
-- Risks: confusing and hard to debug.
651
-
652
-### 7. Should link width/opacity be tokens or bounded numbers?
653
-
654
-A. **Recommended: closed tokens for width and opacity in v1.**
655
-
656
-- Pros: compact, theme-owned, predictable, easy to validate.
657
-- Cons: less granular than old numeric values.
658
-- Implications: map old width/opacity to `thin`/`normal`/`emphasis` and `normal`/`muted`/`faded`.
659
-- Risks: some old visual nuance may be approximated.
660
-
661
-B. Bounded numeric values.
662
-
663
-- Pros: closer to old schema and existing canvas math.
664
-- Cons: producers can still tune UI details too closely.
665
-- Implications: schema must enforce min/max and frontend must clamp.
666
-- Risks: style drift between producers.
667
-
668
-C. Keep old raw numeric semantics.
669
-
670
-- Pros: simplest migration from old schema.
671
-- Cons: keeps old ambiguity.
672
-- Implications: backend retains too much control over frontend look.
673
-- Risks: inconsistent and unreviewed visual scaling.
674
-
675
-## Plan
676
-
677
-1. Inventory old and new contracts.
678
-2. Produce a gap matrix with preserve/replace/drop/defer classification.
679
-3. Run requested external reviewers.
680
-4. Extend schema and docs.
681
-5. Write UI and Cloud aggregator handoff docs.
682
-6. Implement backend producer support.
683
-7. Validate and update SOW.
684
-8. Hand execution to SOW-0023 before starting SOW-0022.
685
-
686
-## Execution Log
687
-
688
-### 2026-05-09
689
-
690
-- Opened SOW-0021 from user direction after discovering the compact schema dropped required visual semantics.
691
-- Paused SOW-0020 so presentation repair can be handled as an explicit, reviewable step.
692
-- Created SOW-0022 as the follow-up for actor modal/table composition.
693
-- Ran all seven requested read-only external reviewers and consolidated their findings into `## Reviewer Findings - 2026-05-09`.
694
-- Added `## Decision Gate - 2026-05-09` because the reviewers found blocking design choices that must be answered before schema/backend implementation.
695
-- Recorded the final execution order decision: finish SOW-0021, then run SOW-0023, then run SOW-0022.
696
-- Recorded the SOW-0021 identity guardrail so presentation labels do not misuse canonical identity and do not block the cross-payload matcher.
697
-- Implemented compact presentation in `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`, `src/go/pkg/topology/v1`, network-connections, streaming, and SNMP v1 producers.
698
-- Created worker handoff docs:
699
- - `<cloud-frontend-repo>/TODO-topology-presentation-contract.md`
700
- - `<cloud-topology-service-repo>/REQUIREMENTS-topology-presentation.md`
701
-- Created the backend-worker SOW and expanded the frontend-worker TODO so the
702
- other workers can port the SOW-0021 changes into their codebases:
703
- - `<cloud-topology-service-repo>/.agents/sow/pending/SOW-0005-20260509-port-topology-presentation-contract.md`
704
- - `<cloud-topology-service-repo>/SOW-status.md`
705
- - `<cloud-frontend-repo>/TODO-topology-presentation-contract.md`
706
-- Answered the first Cloud frontend worker question batch by appending
707
- `## Answers - 2026-05-10` to
708
- `<cloud-frontend-repo>/TODO-topology-presentation-contract.md`.
709
- The answers explicitly state that the frontend remains topology-schema
710
- agnostic and must use representative v1 feature fixtures, not
711
- producer-specific UI branches.
712
-- Created `.agents/sow/pending/SOW-0024-20260510-vsphere-topology-v1-migration.md`
713
- to track the later vSphere migration to `netdata.topology.v1` after
714
- SOW-0021, SOW-0023, and SOW-0022 are finished.
715
-- Ran the second requested-style read-only reviewer pass after material implementation changes. Raw outputs are under `.local/audits/topology-presentation-contract/reviews-round2/`.
716
-- Gemini returned no review output in round 2; the output file is zero bytes. The other read-only reviewers completed and their findings were consolidated locally.
717
-- Fixed the real second-round findings: semantic presentation validation, `highlight_path` and metric-size schema conditionals, width/opacity token constraints for link variables, explicit port-bullet sources, token documentation, handoff merge rules, SNMP custom legend entry, and the network-connections ownership metrics leak.
718
-- Ran a third read-only reviewer pass after the second-round fixes, using the same scope plus fix notes. Raw outputs are under `.local/audits/topology-presentation-contract/reviews-round3/`.
719
-- Kimi timed out in the third pass after 30 minutes and produced only progress output, not a final review.
720
-- Fixed the concrete third-round findings:
721
- - streaming port-bullet labels now use `port_name`, not raw `actor_ref` row indexes;
722
- - network-connections only advertises socket-evidence port bullets in detailed mode, where socket evidence is present;
723
- - `label_policy` rejects non-display columns such as `array`, `json`, and raw refs;
724
- - hover fields validate against actor/link table columns;
725
- - highlight-path actor/order columns are type-checked;
726
- - actor-table port sources must reference declared table types or runtime actor tables;
727
- - SNMP always declares the `actor_ports` table type for its port-bullet source and adds `unknown` to the port legend;
728
- - Go `BorderPresentation.Enabled` is now optional so schema defaults are not contradicted by zero-value Go structs.
729
- - topology/v1 tests now verify Go token arrays stay in sync with the JSON Schema token enums.
730
-- Added the public `create-topology` how-to for graph presentation and linked it from the live how-to catalog:
731
- - `docs/netdata-ai/skills/create-topology/how-tos/add-graph-presentation.md`
732
- - `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`
733
-- Investigated runtime SNMP/L2 visual regression reported after frontend
734
- integration: inferred/probable links were rendered like LLDP/CDP links.
735
- Root cause: the SNMP v1 producer collapsed all graph rows into one
736
- `l2_observation` link type, so the frontend had no schema-level signal to
737
- apply the old `probable` presentation.
738
-- Fixed SNMP v1 link typing:
739
- - graph rows now preserve semantic link types for `lldp`, `cdp`, `bridge`,
740
- `fdb`, `stp`, `arp`, `snmp`, `probable`, and fallback `l2_observation`;
741
- - each link type has its own v1 presentation tokens;
742
- - evidence sections are split by matching evidence type so
743
- `evidence_types.<id>.link_type` stays coherent with the graph link type.
744
-- Added the public `create-topology` how-to for preserving semantic link types:
745
- - `docs/netdata-ai/skills/create-topology/how-tos/preserve-semantic-link-types.md`
746
- - `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`
747
-- Investigated runtime streaming highlight-path regression reported after the
748
- frontend integration: clicking a streaming actor highlighted only direct
749
- graph siblings instead of that actor's ordered streaming path.
750
- Root cause: the v1 streaming payload configured `path_actor_column: "actor"`
751
- while the `actor` column is the clicked/owner actor, not the path member; the
752
- frontend v1 adapter also did not materialize v1 path tables into the legacy
753
- `streamingPath` node field consumed by the graph click handler.
754
-- Extended the `highlight_path` contract with optional `path_owner_column`.
755
- `path_actor_column` now means path member; `path_owner_column` means the
756
- clicked actor that owns the path row. Existing global path-table payloads
757
- remain valid when `path_owner_column` is omitted.
758
-- Fixed streaming v1 path rows:
759
- - `stream_path` now carries both owner `actor` and member `path_actor`;
760
- - streaming presentation now points at `path_owner_column: "actor"` and
761
- `path_actor_column: "path_actor"`;
762
- - the v1 stream-path table mirrors the old highlight-path helper by appending
763
- the local agent to stored paths when storage does not already include it.
764
-- Fixed the frontend v1 adapter in the Cloud frontend worktree:
765
- - validates optional `path_owner_column`;
766
- - resolves per-owner highlight paths from compact actor tables;
767
- - materializes `streamingPath` arrays onto graph nodes before `ForceGraph`
768
- handles click selection.
769
-- Updated the Cloud frontend and Cloud topology service handoff documents so
770
- worker agents port `path_owner_column` semantics together with the backend
771
- schema change.
772
-- Added the public `create-topology` how-to for per-actor highlight paths:
773
- - `docs/netdata-ai/skills/create-topology/how-tos/define-per-actor-highlight-paths.md`
774
- - `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`
775
-
776
-## Validation
777
-
778
-Acceptance criteria evidence:
779
-
780
-- Old graph presentation semantics were inventoried from old schema, old producer emissions, and frontend consumers in `## Inventory - 2026-05-09`.
781
-- `netdata.topology.v1` now has:
782
- - type-level actor/link/port presentation;
783
- - graph-level `data.presentation`;
784
- - safe `label_policy`;
785
- - closed icon/color/opacity/width/line/curve/arrow tokens;
786
- - explicit port-bullet sources;
787
- - legend, highlight-path, scale-key, hover, annotation, and variable link scaling contract.
788
-- Backend producers covered by this SOW emit v1 presentation:
789
- - `src/collectors/network-viewer.plugin/network-viewer.c`
790
- - `src/web/api/functions/function-topology-streaming.c`
791
- - `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go`
792
-- UI and Cloud aggregator worker handoff docs exist at the absolute paths recorded in the execution log.
793
-- Backend service handoff is now a real pending SOW in the service repository:
794
- `<cloud-topology-service-repo>/.agents/sow/pending/SOW-0005-20260509-port-topology-presentation-contract.md`.
795
-- Frontend handoff is now an expanded implementation TODO in the frontend
796
- repository: `<cloud-frontend-repo>/TODO-topology-presentation-contract.md`.
797
-
798
-Tests or equivalent validation:
799
-
800
-- `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
801
-- `go test ./pkg/topology/v1 ./plugin/go.d/collector/snmp_topology ./tools/functions-validation/validate` passed from `src/go`.
802
-- C syntax validation passed using the exact compile commands from `build/compile_commands.json` with `-fsyntax-only` for:
803
- - `src/collectors/network-viewer.plugin/network-viewer.c`
804
- - `src/web/api/functions/function-topology-streaming.c`
805
-- After third-round fixes, the same Go test command and both C `-fsyntax-only` checks passed again.
806
-- After the SNMP/L2 inferred-link fix,
807
- `go test ./plugin/go.d/collector/snmp_topology` passed from `src/go`.
808
-- After the streaming highlight-path fix:
809
- - `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
810
- - `go test ./pkg/topology/v1 ./plugin/go.d/collector/snmp_topology ./tools/functions-validation/validate` passed from `src/go`.
811
- - C syntax validation passed for `src/web/api/functions/function-topology-streaming.c` using the exact compile command from `build/compile_commands.json` with `-fsyntax-only`.
812
- - In `<cloud-frontend-repo>`, `yarn test src/domains/functions/topology/v1/normalizeTopologyV1.test.js src/domains/functions/topology/v1/buildRenderPresentation.test.js --runInBand` passed with 83 tests.
813
-- The Go test suite includes a schema-token parity test so future token additions must update the schema and Go validator together.
814
-- The SNMP Go tests now decode `data.links.type` and verify LLDP rows remain
815
- `lldp`, probable/inferred rows become `probable`, presentation tokens differ,
816
- evidence type link references match the graph link type, and legend entries
817
- include both categories.
818
-- Full CMake/Ninja build could not run because the local `build/` directory and Ninja files are owned by `root`; `cmake --build build --target network-viewer.plugin netdata -j 8` fails with `.ninja_lock` permission denied.
819
-- Backend service SOW audit passed after creating
820
- `cloud-topology-service/.agents/sow/pending/SOW-0005-20260509-port-topology-presentation-contract.md`.
821
-
822
-Real-use evidence:
823
-
824
-- The user confirmed the previous SNMP/L2 inferred-link visual fix works.
825
-- The streaming highlight-path repair is locally validated by backend schema,
826
- Go, C syntax, and frontend adapter tests. The user later confirmed the
827
- installed topology view works with the patched producer/UI path.
828
-- Direct Cloud aggregation runtime validation remains owned by the Cloud
829
- topology service worker handoff, not by this netdata repository commit.
830
-- The producer code was validated by Go tests, JSON Schema tests, function-validation fixtures, and C syntax checks. Full Cloud aggregation parity remains dependent on the Cloud topology service handoff.
831
-
832
-Reviewer findings:
833
-
834
-- First review round completed. Raw outputs are under `.local/audits/topology-presentation-contract/reviews/`; consolidated findings are recorded in `## Reviewer Findings - 2026-05-09`.
835
-- Second review round completed for Codex, Claude, Qwen, GLM, MiniMax, Kimi, and MiMo. Raw outputs are under `.local/audits/topology-presentation-contract/reviews-round2/`.
836
-- Second-round Gemini produced no output; `.local/audits/topology-presentation-contract/reviews-round2/gemini.txt` is zero bytes.
837
-- Real findings handled in code/docs/handoffs:
838
- - semantic presentation cross-reference validation in `src/go/pkg/topology/v1/validate.go`;
839
- - schema conditional requirements for `highlight_path` and metric actor sizing;
840
- - link variable `min`/`max` constrained to width/opacity tokens;
841
- - explicit `ports.sources[]` instead of implicit/misleading port type columns;
842
- - token vocabulary and fallback guidance;
843
- - Cloud merge rules for `label_policy`, scale keys, port sources, and column-name preservation;
844
- - sensitive identifier handling note for streaming detail tables;
845
- - SNMP `custom` actor added to legend;
846
- - network-connections ownership link no longer declares socket-only metrics.
847
-- Third review round completed for Codex, Claude, Qwen, GLM, MiniMax, and MiMo. Kimi timed out after 30 minutes without a final review. Raw outputs are under `.local/audits/topology-presentation-contract/reviews-round3/`.
848
-- Third-round concrete findings handled in code/docs:
849
- - streaming port-bullet `name_column` now references a scalar display column;
850
- - network-connections no longer advertises evidence-derived port bullets when evidence is omitted in aggregated mode;
851
- - actor-table port sources no longer silently pass missing table declarations;
852
- - label policy and hover fields now reject non-display columns;
853
- - highlight-path path columns now require `actor_ref` and numeric order types;
854
- - frontend/cloud handoffs now clarify scalar bullet labels, fallback defaults, and evidence id rewriting.
855
-- Third-round findings that remain outside SOW-0021 implementation are tracked by worker handoffs or follow-up SOWs:
856
- - Cloud topology service must accept Agent-valid `json` columns before end-to-end Cloud aggregation is ready;
857
- - frontend worker must implement token fallbacks and port-source rendering tests;
858
- - full CMake build and runtime Function samples require a writable build tree / installed Agent and the worker integrations.
859
-
860
-Same-failure scan:
861
-
862
-- `rg` scan for legacy-only presentation fields over the new v1 producers/docs found no remaining production use of `icon_svg`, `actor_click_behavior`, `summary_fields`, `bullet_source`, or `topology_match`.
863
-- Remaining `show_port_bullets` matches are local variable names in C emitters and the migration note that maps old `show_port_bullets` to v1 `ports.show_bullets`.
864
-- `rg` scan for `path_actor_column` still pointing at owner `actor` found no
865
- remaining production payload definition after the streaming fix; remaining
866
- matches are the SOW root-cause note, schema/spec prose, validator errors, and
867
- the intended `path_actor` emission.
868
-
869
-Sensitive data gate:
870
-
871
-- Raw user-provided examples are not copied into this SOW. This SOW uses sanitized summaries only.
872
-- No raw topology captures were committed. Raw reviewer outputs stay under `.local/`.
873
-
874
-Artifact maintenance gate:
875
-
876
-- AGENTS.md: no update needed; workflow rules did not change.
877
-- Runtime project skills: no generic `.agents/skills/project-*` update was needed; the relevant topology producer workflow is the public `create-topology` skill, updated below.
878
-- Specs: `.agents/sow/specs/topology-function-schema.md` was updated with the presentation contract, token vocabulary, port-bullet sources, and aggregation conflict policy.
879
-- End-user/operator docs: `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md` and `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md` were updated for the new v1 presentation model.
880
-- End-user/operator skills: `docs/netdata-ai/skills/create-topology/SKILL.md` was updated; `docs/netdata-ai/skills/create-topology/how-tos/add-graph-presentation.md` and `docs/netdata-ai/skills/create-topology/how-tos/preserve-semantic-link-types.md` were added and linked from `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`.
881
-- End-user/operator skills: `docs/netdata-ai/skills/create-topology/how-tos/define-per-actor-highlight-paths.md` was added and linked from `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`.
882
-- SOW audit hygiene: `.agents/skills/mirror-netdata-repos/SKILL.md` had an existing SSH clone example that matched the audit email-address heuristic. It was reworded without changing behavior so the sensitive-data audit passes cleanly.
883
-- SOW lifecycle: SOW-0021 is marked `completed` and is moved to `.agents/sow/done/` with the netdata implementation commit. SOW-0020 is closed in the same netdata commit because this work built on its compact topology schema foundation. SOW-0023 and SOW-0022 remain pending follow-ups.
884
-
885
-Specs update:
886
-
887
-- Updated `.agents/sow/specs/topology-function-schema.md`.
888
-
889
-Project skills update:
890
-
891
-- No generic `.agents/skills/project-*` update was needed.
892
-- Updated the public `create-topology` skill because it is the topology producer workflow reference:
893
- - `docs/netdata-ai/skills/create-topology/SKILL.md`
894
- - `docs/netdata-ai/skills/create-topology/how-tos/add-graph-presentation.md`
895
- - `docs/netdata-ai/skills/create-topology/how-tos/preserve-semantic-link-types.md`
896
- - `docs/netdata-ai/skills/create-topology/how-tos/define-per-actor-highlight-paths.md`
897
- - `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`
898
-
899
-End-user/operator docs update:
900
-
901
-- Updated `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`.
902
-- Updated `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`.
903
-
904
-End-user/operator skills update:
905
-
906
-- Updated `docs/netdata-ai/skills/create-topology/SKILL.md`.
907
-- Added `docs/netdata-ai/skills/create-topology/how-tos/add-graph-presentation.md`.
908
-- Added `docs/netdata-ai/skills/create-topology/how-tos/preserve-semantic-link-types.md`.
909
-- Added `docs/netdata-ai/skills/create-topology/how-tos/define-per-actor-highlight-paths.md`.
910
-- Updated `docs/netdata-ai/skills/create-topology/how-tos/INDEX.md`.
911
-
912
-Lessons:
913
-
914
-- Closed token schemas are not enough by themselves. Semantic validation must also check that labels, legends, path references, scale keys, and port-bullet sources point to real tables, columns, and type definitions.
915
-- Port bullets need explicit data-source declarations. A boolean `show_bullets` restores only visibility, not the data contract the UI needs.
916
-- Dynamic actor detail tables, especially SNMP tables, require careful validation: required source columns can be checked, but optional enrichment columns must be allowed to be absent when the table shape varies by device.
917
-- If a type-level presentation references optional runtime data, the type registry still needs a stable declaration for that source. Otherwise validation cannot distinguish an intentionally empty table from a typo.
918
-- Graph link type is the UI's presentation handle. If a producer collapses
919
- visually distinct facts into a generic link type, the frontend cannot remain
920
- topology-agnostic and still recover the old visual meaning.
921
-- Highlight paths that are actor-specific need two actor references: one for
922
- the owner/clicked actor and one for the path member. Reusing one column for
923
- both preserves table shape but loses the selection semantics.
924
-
925
-Follow-up mapping:
926
-
927
-- SOW-0022 tracks actor modal/table composition.
928
-- SOW-0023 tracks cross-payload actor reconciliation and must run before SOW-0022.
929
-
930
-## Outcome
931
-
932
-Backend/schema/docs/handoff work for SOW-0021 is implemented and locally validated in the netdata repository. The schema now carries backend-selected presentation profiles for actor, link, and port types; producer output preserves semantic link types; streaming can define per-actor highlight paths; and the Cloud frontend plus Cloud topology service handoffs are documented for their owning repositories.
933
-
934
-## Lessons Extracted
935
-
936
-See `## Validation` lessons above.
937
-
938
-## Followup
939
-
940
-- SOW-0022 handles modal/table composition.
941
-- SOW-0023 handles cross-payload actor reconciliation and is ordered before SOW-0022.
942
-
943
-## Regression Log
944
-
945
-None yet.
946
-
947
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0023-20260509-topology-cross-payload-matching.md
deleted
-695
@@ -1,695 +0,0 @@
1
-# SOW-0023 - Topology cross-payload matching
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: netdata-side schema, spec, docs, developer skill, validators, fixtures,
8
-network-connections producer changes, and local frontend v1 rendering alignment
9
-are implemented and validated. Cloud service aggregation remains out of scope
10
-for the aggregated local UI fix because the Cloud aggregator receives detailed
11
-payloads.
12
-
13
-## Requirements
14
-
15
-### Purpose
16
-
17
-Define how Cloud reconciles actors and links that arrive from different topology producers and may describe the same real-world entity or relationship with different identity keys.
18
-
19
-### User Request
20
-
21
-The user identified that the Cloud aggregator must eventually know how to match across topology payloads:
22
-
23
-- network-connections by socket endpoint identity such as IP, port, protocol, direction, and address-space context;
24
-- SNMP/L2 by device, MAC, interface, port, chassis, and management identity;
25
-- streaming by machine or node identity;
26
-- vSphere by stable inventory object identity.
27
-
28
-The user accepted splitting this structural problem out of SOW-0021 so SOW-0021 can focus on presentation.
29
-
30
-### Assistant Understanding
31
-
32
-Facts:
33
-
34
-- `netdata.topology.v1` currently has per-actor-type `identity` and `merge_identity`.
35
-- Evidence types have `match_columns`, but those preserve relationship detail inside one payload and do not by themselves define cross-producer actor replacement.
36
-- Different topology producers use different identity vocabularies and may legitimately fail to correlate.
37
-- If two observations do not correlate, both remain valid; this is not a factual contradiction.
38
-
39
-Inferences:
40
-
41
-- Cross-payload matching needs a shared identity vocabulary or strategy registry, normalization rules, ambiguity policy, confidence, and tests.
42
-- This is structural graph reconciliation, not presentation.
43
-- The schema may need additional producer declarations, but those should be designed in this SOW, not hidden in presentation profiles.
44
-
45
-Unknowns:
46
-
47
-- Exact shared identity vocabulary and normalization rules.
48
-- Whether matching is pairwise between producer kinds or generic through typed identity facts.
49
-- How Cloud should handle ambiguous matches, partial matches, and conflicting confidence.
50
-- MVP scope is limited to the netdata-side generic contract plus
51
- network-connections producer emission. Cloud implementation details are
52
- tracked by the service worker SOW.
53
-- Exact producer migration order after the schema contract lands.
54
-
55
-### Acceptance Criteria
56
-
57
-- Inventory identity and match evidence emitted by network-connections, SNMP/L2, streaming, and vSphere.
58
-- Define a compact schema contract for cross-payload identity declarations if needed.
59
-- Define Cloud aggregator matching strategies, normalization rules, ambiguity policy, and diagnostics.
60
-- Define how endpoint actors are replaced, merged, or left separate.
61
-- Create the Cloud service handoff SOW that requires fixtures for successful
62
- match, no match, ambiguous match, partial link match, and conflicting
63
- presentation/type definitions.
64
-- Create the Cloud frontend handoff TODO for correlation rendering and link
65
- layout tokens.
66
-- Implement the network-connections producer changes needed to emit semantic
67
- ownership/resolved/correlation link types and socket correlation rows.
68
-- Update topology schema/spec/docs/skill if producer declarations are added.
69
-
70
-## Analysis
71
-
72
-Sources checked:
73
-
74
-- `.agents/sow/current/SOW-0021-20260509-topology-presentation-contract.md`
75
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
76
-- `src/collectors/network-viewer.plugin/network-viewer.c`
77
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go`
78
-- `src/web/api/functions/function-topology-streaming.c`
79
-- Cloud topology service evidence recorded in SOW-0021.
80
-
81
-Current state:
82
-
83
-- SOW-0021 records that `merge_identity` is per actor type and does not define shared identity classes across producers.
84
-- SOW-0021 records that evidence `match_columns` preserve exact relationship details but do not declare endpoint replacement across topology payloads.
85
-- SOW-0021 records the user decision to split this into SOW-0023.
86
-
87
-Risks:
88
-
89
-- False positive matches could collapse unrelated actors.
90
-- False negative matches could duplicate actors that represent the same entity.
91
-- NAT, load balancers, address reuse, namespaces, and reused MAC/IP identities can make simple exact matching unsafe.
92
-- Matching strategies can leak sensitive infrastructure identities if durable artifacts include raw examples.
93
-
94
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
95
-
96
-Status: unblocked for netdata-side contract and network-connections producer
97
-implementation. Cloud and frontend implementation are delegated through the
98
-handoff artifacts created by this SOW.
99
-
100
-Problem / root-cause model:
101
-
102
-- The compact topology schema has enough identity to aggregate within producer-defined types, but it does not yet define how Cloud should reconcile actors across different producer domains.
103
-- The schema also lacked generic link-layout force/distance tokens, causing dense or weak semantic relationships to use the same graph forces as strong relationships.
104
-- Network-connections currently uses one graph link family for process-to-endpoint socket links, which prevents the UI and aggregator from distinguishing local resolved links from correlation links.
105
-
106
-Evidence reviewed:
107
-
108
-- SOW-0021 reviewer findings and user decision notes.
109
-- Current topology schema identity and evidence match-column fields.
110
-- Current topology schema link presentation fields: color, opacity, line style, width, curve, arrow, variable, and hover, with no layout strength/distance token.
111
-- Current network-viewer socket evidence and link type shape.
112
-- Current SNMP actor identity arrays for chassis, MAC, IP, and sys-name.
113
-- Current streaming actor identity through machine GUID and node id.
114
-
115
-Affected contracts and surfaces:
116
-
117
-- `netdata.topology.v1` schema.
118
-- Cloud topology service aggregation algorithm.
119
-- Topology producers for network-connections, SNMP/L2, streaming, vSphere, and future topology domains.
120
-- Cloud frontend behavior when merged actors replace endpoint actors.
121
-- Cloud frontend force-layout behavior for dense, weak, ownership, and partial-correlation links.
122
-
123
-Existing patterns to reuse:
124
-
125
-- Actor `identity`, `merge_identity`, and `parent_identity`.
126
-- Evidence `match_columns`.
127
-- Link direction and aggregation semantics.
128
-- Type-level presentation tokens and legend entries from SOW-0021.
129
-- Compact tables for high-cardinality rows.
130
-
131
-Risk and blast radius:
132
-
133
-- High: incorrect matching can materially change topology meaning.
134
-- High: Cloud aggregation behavior changes across all topology kinds.
135
-- Medium: producer schemas may need new typed identity declarations.
136
-- Medium: layout tokens influence graph readability but must not leak raw frontend physics into producer payloads.
137
-
138
-Sensitive data handling plan:
139
-
140
-- Do not copy raw IP addresses, MAC addresses, hostnames, machine GUIDs, node IDs, account IDs, customer identifiers, credentials, secrets, API tokens, bearer tokens, session cookies, SNMP communities, or private topology examples into this SOW, specs, docs, skills, code comments, commits, or PR text.
141
-- Use sanitized synthetic fixtures and placeholder identifiers.
142
-- Keep any raw captured payloads under `.local/` only.
143
-
144
-Implementation plan:
145
-
146
-1. Define the netdata-side schema contract for generic correlation rules, pure correlation actors, points, claims, actions, priorities, and output link types.
147
-2. Define generic link layout tokens for all link types.
148
-3. Update the Go topology v1 model and semantic validator.
149
-4. Update the topology spec, developer guide, implementation scope, and
150
- developer `project-create-topology` skill.
151
-5. Update fixtures so validators exercise correlation rules and link-layout tokens.
152
-6. Implement network-connections link taxonomy and correlation rows.
153
-7. Create Cloud aggregator and UI force-layout handoff artifacts for their
154
- owning workers.
155
-
156
-Validation plan:
157
-
158
-- Service-level fixtures for match, no match, ambiguous match, and unsafe match.
159
-- Schema validation for any new identity declarations.
160
-- Payload-size checks for added declarations.
161
-- Same-failure search across topology producers.
162
-- Netdata-side JSON Schema validation for correlation objects and link layout tokens.
163
-- Go semantic validation for correlation rule references, point/claim table shape, rule key columns, and link-layout token parity.
164
-- Go semantic validation must require only the key columns for rules actually
165
- referenced by each correlation table, not every column for every defined rule.
166
-
167
-Artifact impact plan:
168
-
169
-- AGENTS.md: likely unaffected.
170
-- Runtime project skills: likely unaffected unless topology workflow changes.
171
-- Specs: update topology function schema spec.
172
-- End-user/operator docs: update topology developer guide if producer contract changes.
173
-- Runtime project skills: update `project-create-topology` when producer
174
- authoring workflow changes.
175
-- End-user/operator skills: not affected unless public operator workflows
176
- change.
177
-- JSON Schema: update `FUNCTION_TOPOLOGY_SCHEMA.json`.
178
-- Go producer helper: update `src/go/pkg/topology/v1`.
179
-- Function validation fixtures: update topology v1 fixtures.
180
-- SOW lifecycle: this SOW tracks the SOW-0021 split decision.
181
-
182
-Open-source reference evidence:
183
-
184
-- No external open-source implementation was used as normative evidence for
185
- this schema contract. The contract is driven by Netdata producer semantics,
186
- existing topology payload requirements, and the Cloud aggregator handoff.
187
-
188
-Open decisions:
189
-
190
-- None for the Netdata-side SOW-0023 scope. Cross-kind identity policy remains
191
- tracked by SOW-0002, table/modal composition remains tracked by SOW-0022, and
192
- vSphere migration remains tracked by SOW-0024.
193
-
194
-## Correlation Contract
195
-
196
-The production schema must describe producer-visible correlation facts only. It
197
-must not expose aggregator internals. The aggregator may maintain any internal
198
-state needed, but the final emitted topology is a normal `netdata.topology.v1`
199
-payload with actors, links, evidence, tables, overlays, presentation, stats, and
200
-diagnostics as appropriate.
201
-
202
-Correlation points:
203
-
204
-- are pure topology actors;
205
-- have normal actor types, presentation, labels, legend entries, and links;
206
-- remain visible when unmatched, partial, or ambiguous;
207
-- disappear only when an exact unambiguous absorb rule resolves them.
208
-
209
-Correlation rules:
210
-
211
-- are declared by producers under `data.correlation.rules`;
212
-- are generic and topology-agnostic;
213
-- build exact keys from declarative column/literal templates;
214
-- have a `priority` so exact rules can run before broader/partial rules;
215
-- have a `key_space` to avoid accidental matches between unrelated domains;
216
-- have `action: absorb` or `action: link`;
217
-- list pure `point_actor_types`;
218
-- may list `claim_actor_types` that can satisfy a point;
219
-- may list `correlation_link_types` that connect real actors to correlation actors;
220
-- state an `output_link_type` for final rewritten or partial links.
221
-
222
-Correlation tables:
223
-
224
-- `data.correlation.points` contains actor refs for pure correlation actors plus
225
- rule id and key columns.
226
-- `data.correlation.claims` contains actor refs for real actors plus rule id and
227
- key columns.
228
-- A real actor can claim many keys without bloating the actor row itself.
229
-- A correlation actor can have several point rows for aliases such as NAT-derived
230
- additional keys.
231
-
232
-Actions:
233
-
234
-- `absorb`: exact, unambiguous matches remove all matched correlation actors from
235
- the final aggregated output and rewire incident correlation links to the
236
- matched real actors using the rule's `output_link_type`.
237
-- `link`: broader or partial matches keep the correlation actor visible and emit
238
- or preserve a weak semantic link to the matched actor using the rule's
239
- `output_link_type`.
240
-
241
-No match leaves the correlation actor visible. Ambiguous matches must not be
242
-guessed; they remain unresolved and the aggregator records diagnostics.
243
-
244
-The aggregator must stay agnostic. It should not need releases to learn new
245
-strategy names such as `ip_port`, `mac`, or `vsphere_moid`. It should build keys
246
-from declared columns and literals, normalize by column type, respect priority
247
-and key space, and apply the declared action.
248
-
249
-## Link Layout Contract
250
-
251
-Link layout is generic for all link types. It is not special to correlation.
252
-
253
-Every link type may define:
254
-
255
-- `types.link_types.<id>.presentation.layout.strength`
256
-- `types.link_types.<id>.presentation.layout.distance`
257
-
258
-Allowed strength tokens:
259
-
260
-- `weakest`
261
-- `weaker`
262
-- `normal`
263
-- `stronger`
264
-- `strongest`
265
-
266
-Allowed distance tokens:
267
-
268
-- `closest`
269
-- `closer`
270
-- `normal`
271
-- `farther`
272
-- `farthest`
273
-
274
-These are UI-owned relative tokens, not numeric force values. Producers use them
275
-to classify relationship strength and preferred separation:
276
-
277
-- ownership/containment: stronger + closer;
278
-- resolved normal dependency/flow: normal + normal;
279
-- local noise, dense mesh, inferred, or weak evidence: weaker/weakest +
280
- farther/farthest;
281
-- partial or cross-topology correlation links: weaker/weakest + farther.
282
-
283
-The legend must reflect visible semantic differences introduced by these link
284
-types. The UI must not infer forces from topology kind or actor names.
285
-
286
-## Network-Connections Required Shape
287
-
288
-Network-connections should use three graph-link families:
289
-
290
-1. Node-to-process ownership links that keep the graph clustered by node.
291
-2. Local process-to-process links when both endpoints are already resolved.
292
-3. Process-to-correlation-endpoint links for unresolved or cross-node remote
293
- socket endpoints.
294
-
295
-Socket tuple interpretation:
296
-
297
-- outbound: the process claims `protocol + local_ip + local_port`; the
298
- correlation endpoint points at `protocol + remote_ip + remote_port`;
299
-- inbound: the process claims the local destination tuple; the correlation
300
- endpoint points at the remote source tuple;
301
-- local: both process actors are already identified and should use a resolved
302
- local process-to-process link;
303
-- listen: no remote correlation point exists.
304
-
305
-Exact cross-node absorb example:
306
-
307
-1. Node A emits `process-a -> correlation-endpoint(server-ip:server-port)`.
308
-2. Node B emits `process-b` claiming `server-ip:server-port`.
309
-3. Aggregation removes the matched correlation endpoint and rewires the link to
310
- `process-a -> process-b`.
311
-
312
-Partial example:
313
-
314
-1. Node A emits `process-a -> correlation-endpoint(protocol:ip:port)`.
315
-2. Node B does not have a matching process claim, but emits a broader node/IP
316
- claim through a lower-priority `link` rule.
317
-3. Aggregation keeps the correlation endpoint visible and links it to Node B:
318
- `process-a -> correlation-endpoint(protocol:ip:port) -> node-b`.
319
-
320
-## Concrete Change Inventory
321
-
322
-Netdata repository changes required by this SOW:
323
-
324
-- Extend `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` with:
325
- - `data.correlation`;
326
- - `correlation.rules`;
327
- - compact `correlation.points` and `correlation.claims` tables;
328
- - `correlation_rule.action`, `priority`, `key_space`, `key`,
329
- `point_actor_types`, `claim_actor_types`, `correlation_link_types`, and
330
- `output_link_type`;
331
- - declarative `correlation_key_part` column/literal templates;
332
- - `link_type.presentation.layout.strength` and `.distance` tokens.
333
-- Extend `src/go/pkg/topology/v1` structs and semantic validation with:
334
- - correlation data types;
335
- - point/claim table validation;
336
- - rule reference validation against actor/link types;
337
- - rule key column validation;
338
- - link layout token validation and schema-token parity tests.
339
-- Update fixtures under `src/go/tools/functions-validation/fixtures/topology-v1/`
340
- with at least one correlation rule and link layout tokens.
341
-- Update `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`.
342
-- Update `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`.
343
-- Update `.agents/sow/specs/topology-function-schema.md`.
344
-- Update `.agents/skills/project-create-topology/SKILL.md` with topology
345
- correlation and layout authoring guidance.
346
-
347
-Producer changes implemented or scoped after the contract:
348
-
349
-- Network-viewer:
350
- - split link types into ownership, local/resolved socket, and correlation
351
- socket links;
352
- - emit correlation actor rows for remote endpoints;
353
- - emit correlation point rows for unresolved remote endpoint keys;
354
- - emit process claim rows for locally owned socket keys;
355
- - emit legend entries for ownership, resolved socket, and correlation socket;
356
- - assign layout tokens to each link type.
357
-- Streaming:
358
- - likely no correlation actors for the current topology; verify whether
359
- machine/node identity claims should be emitted for future cross-kind
360
- resolution.
361
-- SNMP/L2:
362
- - preserve semantic link types; evaluate whether inferred endpoints should
363
- emit MAC/IP/chassis correlation points or claims.
364
-- vSphere:
365
- - remains tracked by SOW-0024; when migrated, stable object identity can be
366
- expressed as claims or normal actor identity depending on topology scope.
367
-
368
-External repository handoffs required:
369
-
370
-- Cloud topology service:
371
- - implement rule parsing and generic key building;
372
- - apply absorb/link actions;
373
- - keep unmatched/ambiguous correlation actors visible;
374
- - add match/no-match/partial/ambiguous fixtures;
375
- - emit diagnostics without exposing internal states in the output graph.
376
-- Cloud frontend:
377
- - honor link layout strength/distance tokens for all link types;
378
- - show semantic correlation actor/link legend entries;
379
- - avoid topology-name-specific force/layout hardcoding.
380
-
381
-## Implications And Decisions
382
-
383
-1. User decision from SOW-0021: cross-payload actor reconciliation should be solved, but it can be split into SOW-0023.
384
-2. User decision from SOW-0021: execution order is SOW-0021, then SOW-0023, then SOW-0022.
385
-3. User decision from 2026-05-10: correlation points are pure topology actors, not flags on real actors. Exact resolved correlation may remove correlation actors from the aggregated output, but internal aggregator states such as absorbed/candidate/rewrite-plan must not be exposed in the production schema or final UI payload.
386
-4. User decision from 2026-05-10: the schema should define the producer-visible correlation contract for independently produced topology maps of the same kind. The aggregator may choose any internal indexing, matching, rewrite, and diagnostic implementation as long as the final output is a normal topology payload.
387
-5. User decision from 2026-05-10: visible partial or cross-topology correlation links should be weaker in the force layout so related topology clusters stay readable instead of blending into one dense actor soup.
388
-6. User decision from 2026-05-10: link layout force classification must be generic for all link types, not special-cased to correlation links. It should use five-step token scales with `normal` in the middle: strength `weakest`, `weaker`, `normal`, `stronger`, `strongest`; distance `closest`, `closer`, `normal`, `farther`, `farthest`.
389
-7. User decision from 2026-05-10: links between a real actor and a correlation actor must be marked with a distinct semantic link type even in a single unaggregated topology. The UI and aggregator must not infer correlation-link behavior from actor names or topology kind.
390
-8. User decision from 2026-05-10: network-connections graph links should be split into three semantic families: node-to-process ownership links that keep the graph together; local process-to-process links for already resolved local sockets; and process-to-correlation-endpoint links for unresolved/correlatable remote socket endpoints.
391
-
392
-## Plan
393
-
394
-1. Complete netdata-side contract artifacts: schema, Go types/validator, docs,
395
- spec, skill, and fixtures.
396
-2. Validate the contract with JSON Schema, Go tests, function-validation
397
- fixtures, and SOW audit.
398
-3. Implement network-viewer producer migration in SOW-0023.
399
-4. Create or update Cloud service and Cloud frontend handoff documents after the
400
- contract is validated.
401
-5. Implement Cloud aggregator and UI behavior in their owning repositories.
402
-
403
-## Execution Log
404
-
405
-### 2026-05-09
406
-
407
-- Created as follow-up from SOW-0021 decision discussion.
408
-
409
-### 2026-05-10
410
-
411
-- Recorded user decisions that correlation points are pure topology actors, that
412
- aggregator internals must not leak into the schema or final UI payload, and
413
- that the schema should describe producer-visible correlation rules only.
414
-- Recorded user decisions for generic declarative correlation keys, priorities,
415
- `absorb` and `link` actions, exact and partial correlation behavior, NAT/alias
416
- enrichment through additional keys, semantic correlation link types, and
417
- five-step link layout strength/distance tokens.
418
-- Added netdata-side schema/Go/docs/spec/skill work items to this SOW.
419
-- Implemented the network-viewer producer split into `ownership`, `socket`,
420
- `endpoint_socket`, and `correlated_socket` link types.
421
-- Implemented network-viewer `data.correlation.rules`, `points`, and `claims`
422
- emission for socket tuple correlation.
423
-- Created Cloud frontend and Cloud topology service handoff artifacts so their
424
- owning workers can port the correlation and layout contract.
425
-
426
-## Validation
427
-
428
-Acceptance criteria evidence:
429
-
430
-- Identity/evidence inventory is recorded in `## Correlation Contract`,
431
- `## Network-Connections Required Shape`, and `## Concrete Change Inventory`.
432
-- Netdata-side schema contract now defines generic correlation rules, compact
433
- points/claims tables, declarative key parts, absorb/link actions, priorities,
434
- point/claim actor types, correlation link types, output link type, and
435
- five-step link layout strength/distance tokens.
436
-- Cloud aggregator and Cloud frontend implementation requirements are scoped in
437
- `## Concrete Change Inventory` and handed off to their owning workers.
438
-- Network-connections now emits semantic ownership/resolved/correlation link
439
- types, correlation endpoint actors, `data.correlation.points`, and
440
- `data.correlation.claims`.
441
-
442
-Tests or equivalent validation:
443
-
444
-- `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
445
-- `jq empty src/go/tools/functions-validation/fixtures/topology-v1/network-connections.json` passed.
446
-- `go test ./pkg/topology/v1 ./tools/functions-validation/validate` passed from `src/go`.
447
-- Function validator passed for every fixture under
448
- `src/go/tools/functions-validation/fixtures/topology-v1/*.json`.
449
-- C syntax-only validation for
450
- `src/collectors/network-viewer.plugin/network-viewer.c` passed using the
451
- local `build/compile_commands.json` command with `-fsyntax-only`.
452
-- `ninja -C build network-viewer.plugin` could not run because the local build
453
- directory is root-owned and Ninja cannot create `.ninja_lock` or update the
454
- build log.
455
-- `git diff --check` passed.
456
-- `.agents/sow/audit.sh` exited successfully. It still reports the repository's
457
- existing non-project skill classification warning, which is outside this SOW;
458
- `project-create-topology` is classified as a runtime project skill.
459
-
460
-Real-use evidence:
461
-
462
-- A local bearer-protected Agent was queried through the token-safe direct-agent
463
- wrapper. The aggregated network-connections topology response returned
464
- `netdata.topology.v1` with 108 actors, 144 links, 96 correlation points, 179
465
- correlation claims, and 45 ownership links.
466
-- The live payload defined `endpoint_socket` as `strength: weakest` and
467
- `distance: normal`, `correlated_socket` as `strength: weakest` and
468
- `distance: farthest`, `socket` as `strength: stronger` and `distance:
469
- farther`, and `ownership` as dotted/faded normal-distance graph-coherence
470
- links.
471
-- The same live payload confirmed the `socket_exact` rule consumes
472
- `endpoint_socket` and emits `correlated_socket`, matching the intended
473
- single-node versus aggregated-layout split.
474
-- Full `ninja -C build network-viewer.plugin` validation remains blocked by the
475
- root-owned local build directory, but schema validation, Go tests, semantic
476
- fixture validation, C syntax validation, and live Function output validation
477
- all passed.
478
-
479
-Reviewer findings:
480
-
481
-- No external read-only reviewer was run for this close. User live testing found
482
- the remaining force-layout stretch for high-fanout endpoint leaves; analysis
483
- traced that to Cloud frontend physics, not to the Netdata producer contract,
484
- and it is mapped to the frontend polishing work.
485
-
486
-Same-failure scan:
487
-
488
-- Current same-failure class is schema ambiguity rather than a runtime crash.
489
- The netdata-side validator now rejects missing correlation key columns,
490
- unknown rule references, unknown actor/link type references, and invalid link
491
- layout tokens.
492
-- The validator also covers the multi-rule case where a table references only
493
- one of several defined rules, so unrelated rule key columns are not forced
494
- into every correlation table.
495
-
496
-Sensitive data gate:
497
-
498
-- This SOW currently contains only sanitized generic examples.
499
-
500
-Artifact maintenance gate:
501
-
502
-- AGENTS.md: updated the project skills index so topology authoring guidance is
503
- a runtime project skill, not a public operator skill.
504
-- Runtime project skills: `.agents/skills/project-create-topology/SKILL.md`
505
- added as the developer-facing topology authoring workflow.
506
-- Specs: `.agents/sow/specs/topology-function-schema.md` updated with
507
- correlation and link-layout contracts.
508
-- End-user/operator docs: `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`,
509
- `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`, and
510
- `src/plugins.d/FUNCTION_UI_REFERENCE.md` updated.
511
-- End-user/operator skills: `query-netdata-agents` and `query-netdata-cloud`
512
- were audited for developer-contract leakage. The misplaced public
513
- `create-topology` developer skill was removed from the public skill tree, and
514
- public query skills now state that developer validation recipes belong in
515
- project skills.
516
-- SOW lifecycle: SOW moved from pending to current for implementation and is
517
- marked `completed`; it is moved to `.agents/sow/done/` in the same commit as
518
- the implementation.
519
-
520
-Specs update:
521
-
522
-- `.agents/sow/specs/topology-function-schema.md` updated.
523
-
524
-Project skills update:
525
-
526
-- Added `.agents/skills/project-create-topology/SKILL.md`.
527
-- Moved topology developer how-tos from the public skill tree into
528
- `.agents/skills/project-create-topology/how-tos/`.
529
-- Added a developer how-to for verifying local network-connections layout
530
- tokens and correlation rule wiring with token-safe direct-agent wrappers.
531
-
532
-End-user/operator docs update:
533
-
534
-- Updated topology developer guide, topology implementation scope, and Function
535
- UI reference.
536
-
537
-End-user/operator skills update:
538
-
539
-- Removed the developer-facing `docs/netdata-ai/skills/create-topology/` skill
540
- and its `.agents/skills/create-topology` symlink.
541
-- Removed topology producer authoring links from public query-topology guides.
542
-- Removed the collector-author implementation section from the public Cloud
543
- query-functions guide.
544
-- Moved skill-verification seed question lists out of public skill directories
545
- into `.agents/skill-verification/` and updated the pending verification
546
- harness SOW.
547
-- Updated public query skills to keep future how-tos operator-facing.
548
-
549
-Lessons:
550
-
551
-- Correlation actors and correlation rules need a contract separate from actor
552
- identity. Actor `merge_identity` is not enough when one real actor owns many
553
- correlation keys.
554
-- Link force/layout must be type-level and generic. Correlation links exposed
555
- this need, but dense local meshes and ownership links need it too.
556
-
557
-Follow-up mapping:
558
-
559
-- Cloud service implementation is handed off through the service SOW created by
560
- this SOW.
561
-- Cloud frontend force-layout and correlation rendering implementation is handed
562
- off through the frontend TODO created by this SOW.
563
-- Cross-kind identity policy remains tracked by SOW-0002.
564
-- Table/modal composition remains tracked by SOW-0022.
565
-- vSphere migration remains tracked by SOW-0024.
566
-
567
-## Outcome
568
-
569
-Completed. Netdata now has a generic producer-visible topology correlation
570
-contract, generic link layout strength/distance tokens, semantic validation for
571
-the new contract, updated schema/spec/docs/developer skill artifacts, and a
572
-network-connections producer that emits ownership, local socket, unresolved
573
-endpoint socket, and post-correlation socket semantics separately.
574
-
575
-The local Agent runtime payload matches the intended split. Remaining visual
576
-layout stretch around high-fanout endpoint leaves is a Cloud frontend physics
577
-polish issue, not a backend payload issue.
578
-
579
-## 2026-05-10 Aggregated Network-Connections UI Alignment
580
-
581
-User-visible issue:
582
-
583
-- Aggregated network-connections process actors lost resize behavior and port
584
- bullets because the producer only enabled process bullets in detailed mode.
585
-- Endpoint links were rendered as dotted/secondary links even though they are
586
- the main unresolved network dependencies in a single-node view.
587
-- Node-to-process links rendered too prominently even though they only keep the
588
- graph coherent and do not represent network traffic.
589
-
590
-Root cause:
591
-
592
-- Aggregated mode intentionally omits detailed socket evidence, but process port
593
- bullets were still defined only from socket evidence rows.
594
-- The UI treated every bullet source row as one visible bullet and had no
595
- `value_column` to say that one compact row represents multiple sockets.
596
-- Process sizing still relied on graph degree instead of the producer's
597
- `socket_count` metric.
598
-
599
-Implemented contract and producer changes:
600
-
601
-- Added optional numeric `ports.sources[].value_column` to the schema, Go model,
602
- semantic validator, spec, and developer guidance.
603
-- Network-connections now emits an actor-owned `socket_ports` inventory table
604
- with `actor`, `port`, `protocol`, `direction`, and `socket_count`.
605
-- Process actor presentation now uses
606
- `size: {"mode": "metric", "metric_column": "socket_count"}` and
607
- `ports.sources[]` from `socket_ports` with `value_column: "socket_count"`.
608
-- Link presentation was aligned with the intended semantics:
609
- `endpoint_socket` is solid/colored/thin/weakest/normal-distance,
610
- `correlated_socket` is solid/colored/thin/weakest/farthest,
611
- `socket` is gray/thin/stronger/farther and variable by `socket_count`, and
612
- `ownership` is dotted/faded/dim/thin/normal/normal.
613
-
614
-Implemented frontend changes:
615
-
616
-- The v1 port-bullet decoder preserves `value_column` and sums duplicate bullet
617
- keys.
618
-- The v1 renderer adapter passes `size_metric_column` to the graph renderer.
619
-- The graph renderer uses actor metric sizing, weighted port capacity, weighted
620
- visible bullet count, and correct overflow without expanding unbounded data.
621
-- v1-derived socket bullets default to active topology bullets so process rings
622
- are visible even when there is no legacy SNMP-style port status.
623
-
624
-Validation added or rerun:
625
-
626
-- `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
627
-- `jq empty src/go/tools/functions-validation/fixtures/topology-v1/network-connections.json` passed.
628
-- `go test ./pkg/topology/v1 ./tools/functions-validation/validate` passed from
629
- `src/go`.
630
-- Function validator passed for
631
- `src/go/tools/functions-validation/fixtures/topology-v1/network-connections.json`.
632
-- C syntax-only validation passed for
633
- `src/collectors/network-viewer.plugin/network-viewer.c` using
634
- `-Ibuild -include build/config.h`.
635
-- Cloud frontend targeted Jest tests passed:
636
- `portBullets.test.js`, `buildRenderPresentation.test.js`,
637
- `portUtils.test.js`, and `useForceSimulation.test.js`.
638
-
639
-## 2026-05-10 Endpoint vs Correlated Socket Layout Split
640
-
641
-User-visible issue:
642
-
643
-- Unresolved endpoint links were set to `distance: farthest`, which makes
644
- single-node network-connections maps zoom out too much and shrink the useful
645
- process cluster.
646
-- The same link type was also planned as the aggregator's consumed correlation
647
- link, where `farthest` is appropriate after cross-payload absorption because
648
- it keeps independent topology clusters from blending.
649
-
650
-Implemented changes:
651
-
652
-- Split the overloaded network-connections link semantics:
653
- `endpoint_socket` is the visible process-to-endpoint unresolved link, while
654
- `correlated_socket` is the aggregator output link after exact absorption.
655
-- `endpoint_socket` uses solid/colored/thin, `strength: weakest`,
656
- `distance: normal`.
657
-- `correlated_socket` uses solid/colored/thin, `strength: weakest`,
658
- `distance: farthest`, and can vary by `socket_count`.
659
-- The `socket_exact` rule now lists `correlation_link_types:
660
- ["endpoint_socket"]` and `output_link_type: "correlated_socket"`.
661
-- Fixtures, developer docs, and the topology developer skill were updated to
662
- use the split names.
663
-
664
-## Lessons Extracted
665
-
666
-- Pure correlation actors keep the producer contract simple: the aggregator can
667
- absorb, link, or leave them visible without exposing internal matching state.
668
-- Link layout tokens are necessary but not sufficient for final graph polish:
669
- force-directed renderers may still need frontend-side handling for high-fanout
670
- leaves, collision radius, zoom-to-fit, and initial layout seeding.
671
-- Aggregated process port bullets need an actor-owned inventory table with a
672
- numeric value column. Counting compact rows is wrong when one row represents
673
- many sockets.
674
-- Distinguishing unresolved endpoint links from aggregated correlated links is
675
- required. Single-node endpoint leaves should not use the same farthest layout
676
- token as cross-payload correlation output.
677
-
678
-## Followup Mapping
679
-
680
-- Cloud topology service generic rule-based correlation is handed off through
681
- the Cloud service worker SOW.
682
-- Cloud frontend link layout token handling and correlation rendering are handed
683
- off through the Cloud frontend worker TODO.
684
-- The high-fanout endpoint leaf layout stretch observed during live testing is
685
- mapped to the Cloud frontend polishing work; the live Netdata payload already
686
- emits `endpoint_socket` as normal-distance and weakest-strength.
687
-- Cross-kind identity policy remains tracked by SOW-0002.
688
-- Table/modal composition remains tracked by SOW-0022.
689
-- vSphere topology v1 migration remains tracked by SOW-0024.
690
-
691
-## Regression Log
692
-
693
-None yet.
694
-
695
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0025-20260511-network-connections-modal-product-composition.md
deleted
-500
@@ -1,500 +0,0 @@
1
-# SOW-0025 - Network Connections Modal Product Composition
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed after backend producer repair, schema/developer
8
-documentation updates, protocol validation, schema validation, and SOW audit.
9
-
10
-## Requirements
11
-
12
-### Purpose
13
-
14
-Make `topology:network-connections` actor modals useful for sysadmins, DevOps engineers, and SREs who need to understand process/socket dependencies quickly and accurately.
15
-
16
-### User Request
17
-
18
-The user reported that network-connections modals currently show `Connections`, `Ports`, and, in detailed mode, `Socket Evidence`, which creates three confusing views over the same socket facts. The user wants the work handled mechanically and separately from SNMP and streaming.
19
-
20
-### Assistant Understanding
21
-
22
-Facts:
23
-
24
-- The v1 schema can express actor labels, modal sections, projections, owner filters, and table columns without duplicating rows.
25
-- Current network-connections modal recipes are generated in `src/collectors/network-viewer.plugin/network-viewer.c`.
26
-- Current actor modals are declared generically for actor types and add:
27
- - `Connections` from `links`, excluding `ownership`;
28
- - `Ports` from actor table `socket_ports` when port bullets are enabled;
29
- - `Socket Evidence` from `evidence.socket` when detailed evidence exists.
30
-- `network-viewer.c` currently emits actor labels and typed actor columns such as process, username, command line, namespace, local IP, address space, socket count, PID, UID, and netns inode.
31
-
32
-Inferences:
33
-
34
-- The schema contract is not the main problem for this function. The issue is table product design: the current tabs reflect implementation sources instead of the troubleshooting questions users ask.
35
-- `Ports` and `Socket Evidence` are not independently useful as top-level tabs unless their relationship to the connection rows is obvious.
36
-
37
-Unknowns:
38
-
39
-- Whether the frontend can already promote selected actor labels into the modal identification/header area, or whether this requires a frontend contract extension in addition to producer recipe changes.
40
-- Whether all old-network-connections modal columns can be reconstructed from current v1 rows without adding new canonical columns.
41
-
42
-### Acceptance Criteria
43
-
44
-- A complete inventory exists for network-connections actor modal facts: actors, actor labels, links, `socket_ports`, and `evidence.socket`.
45
-- The SOW maps every useful old modal/table field to a v1 source, or records the missing canonical field to add.
46
-- The final modal design avoids three duplicate tabs for the same socket relationship.
47
-- Process, endpoint, and node/self actors each have an explicit useful modal shape.
48
-- The actor identification area has a declared source for important labels, not only the generic `Labels` tab.
49
-- Backend producer changes, schema/doc updates if needed, frontend TODO, and validation plan are completed before this SOW closes.
50
-
51
-## Analysis
52
-
53
-Sources checked:
54
-
55
-- `src/collectors/network-viewer.plugin/network-viewer.c:2505` modal column helper.
56
-- `src/collectors/network-viewer.plugin/network-viewer.c:2543` selected-side socket endpoint projection.
57
-- `src/collectors/network-viewer.plugin/network-viewer.c:2647` actor modal emission.
58
-- `src/collectors/network-viewer.plugin/network-viewer.c:2667` current `Connections` section.
59
-- `src/collectors/network-viewer.plugin/network-viewer.c:2710` current `Ports` section.
60
-- `src/collectors/network-viewer.plugin/network-viewer.c:2738` current `Socket Evidence` section.
61
-- `src/collectors/network-viewer.plugin/network-viewer.c:2867` process actor type enables port bullets from `socket_ports`.
62
-- `src/collectors/network-viewer.plugin/network-viewer.c:2937` `socket_ports` table type.
63
-- `src/collectors/network-viewer.plugin/network-viewer.c:2962` `actor_labels` table type.
64
-- `.agents/sow/specs/topology-function-schema.md:351` network-connections actor/link/socket-port schema notes.
65
-
66
-Current state:
67
-
68
-- The current modal exposes source tables directly:
69
- - graph links for high-level connections;
70
- - actor-owned socket-port inventory;
71
- - detailed socket evidence rows.
72
-- This is technically correct but product-confusing because a user sees multiple tabs that appear to describe the same thing with different missing columns.
73
-- Actor identification facts exist in actor labels and typed actor columns, but the modal header/identification area is not using selected important labels.
74
-
75
-Available facts to inventory:
76
-
77
-- Node/self actor:
78
- - `display_name`, `type`, `hostname`, `machine_guid`, `socket_count`, `local_ip_count`.
79
-- Process actor:
80
- - `display_name`, `type`, `process`, `username`, `cmdline`, `namespace_type`, `local_ip`, `local_address_space`, `socket_count`, optional `pid`, optional `uid`, optional `net_ns_inode`.
81
-- Endpoint actor:
82
- - `display_name`, `type`, `ip`, `address_space`, `socket_count`.
83
-- Graph links:
84
- - opposite actor, link type, protocol, direction, state, socket count, and link-level metrics.
85
-- `socket_ports`:
86
- - actor, port, protocol, direction, socket count.
87
-- `evidence.socket`:
88
- - exact source/destination actors and socket endpoint details, protocol, direction, state, RTT/retransmission fields when available.
89
-
90
-Field inventory and mapping:
91
-
92
-| Useful old/current fact | Existing v1 source | Gap / action |
93
-|---|---|---|
94
-| Actor display name | `actors.display_name`, `actor_labels.display_name` | Available. Use in actor label policy and modal header. |
95
-| Hostname | `actors.hostname`, `actor_labels.hostname` | Available for self actor. Promote to modal header. |
96
-| Machine GUID | `actors.machine_guid`, `actor_labels.machine_guid` | Available. Keep in labels; promote only if useful for debugging. |
97
-| Process name | `actors.process`, `actor_labels.process` | Available. Promote to process modal header. |
98
-| Process user | `actors.username`, `actor_labels.username` | Available. Promote to process modal header. |
99
-| Process command line | `actors.cmdline`, `actor_labels.cmdline` | Available. Promote to process modal header or expanded header; sensitive but allowed by Function classification. |
100
-| Namespace type | `actors.namespace_type`, `actor_labels.namespace_type` | Available. Promote to process modal header. |
101
-| Process local IP/address space | `actors.local_ip`, `actors.local_address_space`, labels | Available. Promote when present. |
102
-| PID/UID/netns inode | typed actor columns and labels when `processes:by_pid` | Available. Header for `by_pid`, labels for `by_name`. |
103
-| Endpoint IP/address space | `actors.ip`, `actors.address_space`, labels | Available. Promote to endpoint modal header. |
104
-| Socket count | actor/link/evidence metrics, labels | Available. Promote to header and tables. |
105
-| Direction/protocol/state | `links`, `evidence.socket`, internal `NV_TOPOLOGY_LINK` | Available. Use badges in connection rows. |
106
-| Local/remote IP and port | `evidence.socket` in detailed mode; internal `NV_TOPOLOGY_LINK` in all modes | Missing from aggregated v1 modal source. Add one relationship-summary table from existing internal rows. |
107
-| Server port/service name | internal `NV_TOPOLOGY_LINK.remote_port` or `local_port`, `port_name` | Missing from emitted v1 tables except exact socket evidence has ports but not `port_name`. Add to relationship summary; add `service_name` to evidence if needed. |
108
-| RTT / receiver RTT / retransmissions | `links` and `evidence.socket` metrics | Available. Relationship summary should expose the same metrics per summary row. |
109
-| Local process ownership / node keeps graph coherent | `ownership` links | Available. Self modal should use this as a process list; process modal should not show it as a network connection. |
110
-
111
-Important evidence:
112
-
113
-- Internal connection summary rows already contain process identity, local/remote IPs, local/remote ports, peer port, protocol, direction, state, address spaces, service name, command line, socket count, and latency/retransmission metrics in `NV_TOPOLOGY_LINK` (`src/collectors/network-viewer.plugin/network-viewer.c:80`).
114
-- Those internal rows are keyed by process, namespace, local IP, remote IP, protocol, direction, state, local port, and endpoint port (`src/collectors/network-viewer.plugin/network-viewer.c:1283`).
115
-- v1 graph links intentionally collapse those rows by source actor, destination actor, link type, protocol, direction, and state only (`src/collectors/network-viewer.plugin/network-viewer.c:2135`), so port pairing and service name cannot be reconstructed from graph links alone.
116
-- Current modal sections expose `links`, `socket_ports`, and `evidence.socket` as peer sections (`src/collectors/network-viewer.plugin/network-viewer.c:2647`). This mirrors storage tables, not the user's troubleshooting workflow.
117
-- The current schema supports `relationship_table` modal sources and `relationship_summary` table roles, so an aggregated connection-summary table fits the existing contract without inventing a network-connections-specific UI path.
118
-
119
-Target audience and questions:
120
-
121
-- Sysadmin/SRE process drilldown:
122
- - What is this process?
123
- - Which user/cmdline/container/namespace is it?
124
- - What is it listening on?
125
- - What remote endpoints or local processes is it connected to?
126
- - How many sockets are involved?
127
- - Which entries are unresolved correlation endpoints?
128
-- Sysadmin/SRE endpoint drilldown:
129
- - Which processes connect to this endpoint?
130
- - Is it inbound, outbound, listening, local, TCP, UDP?
131
- - Is the endpoint local/private/public?
132
-- Sysadmin/SRE node drilldown:
133
- - Which processes on this node participate in network activity?
134
- - Which local IPs are observed?
135
-
136
-Target product model:
137
-
138
-- Self/node actor modal:
139
- - Header: hostname, observed socket count, local IP count.
140
- - Primary table: `Processes` derived from `ownership` links, showing process actor and socket count.
141
- - No raw socket table; self is a scope/root actor, not a socket endpoint.
142
-- Process actor modal:
143
- - Header: process name/display name, user, namespace type, command line, local IP/address space, socket count, and PID/netns when available.
144
- - Primary table: `Connections`, sourced from one relationship-summary table in aggregated mode and from `evidence.socket` in detailed mode.
145
- - Columns should answer "where does this process connect/listen?": peer actor or endpoint, local endpoint, remote endpoint, service/port, protocol, direction, state, sockets, RTT/retransmissions.
146
- - Port bullets remain visual graph affordances. A separate `Ports` modal tab should not be shown unless it provides a distinct, clearly labeled local-port inventory.
147
-- Endpoint actor modal:
148
- - Header: IP, address space, socket count.
149
- - Primary table: `Processes`, sourced from the same connection rows, showing the local process, local endpoint, endpoint port/service, protocol, direction, state, sockets, RTT/retransmissions.
150
- - No `Ports` tab; the endpoint is already the port/IP drilldown target.
151
-
152
-Risks:
153
-
154
-- Leaving `Connections`, `Ports`, and `Socket Evidence` as peers makes users compare inconsistent row sets and assume data is wrong.
155
-- Hiding process identity under `Labels` makes the modal feel empty even when the payload contains the information.
156
-- Detailed evidence can be high-cardinality; it should be expandable or secondary, not a default duplicated top-level list unless the user explicitly asked for detailed sockets.
157
-- If aggregated mode continues to use only graph links plus `socket_ports`, it cannot show accurate local/remote port pairing. `socket_ports` is actor inventory, not a relationship table.
158
-- If the frontend guesses important labels from names such as `process`, `username`, or `cmdline`, the UI becomes topology-specific and violates the v1 contract that producers define presentation.
159
-
160
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
161
-
162
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Execution Log and Validation sections).
163
-
164
-Problem / root-cause model:
165
-
166
-- The schema migration preserved facts but lost product intent. The producer emits table recipes that mirror internal sources instead of user workflows.
167
-- The actor identification area is empty because important labels are only exposed through the `actor_labels` table and not selected for header display.
168
-
169
-Evidence reviewed:
170
-
171
-- Current network-connections modal sections are emitted in `src/collectors/network-viewer.plugin/network-viewer.c:2647-2760`.
172
-- Current network-connections actor labels are emitted in `src/collectors/network-viewer.plugin/network-viewer.c:2029-2102`.
173
-- Current network-connections table-type presentation for `socket_ports` and `actor_labels` is emitted in `src/collectors/network-viewer.plugin/network-viewer.c:2937-2979`.
174
-
175
-Affected contracts and surfaces:
176
-
177
-- Agent Function payload for `topology:network-connections`.
178
-- v1 topology schema only if the identification/header area requires a new contract.
179
-- Cloud frontend actor modal rendering.
180
-- Cloud aggregator table/label preservation.
181
-- Developer guide, durable topology spec, and project topology skill.
182
-
183
-Existing patterns to reuse:
184
-
185
-- `actor_labels` for labels and identity facts.
186
-- Modal sections over existing `links`, `evidence`, and `actor_table` sources.
187
-- `selected_side_endpoint` for socket endpoint rendering.
188
-- `socket_ports` with numeric `socket_count` for aggregated port bullets.
189
-
190
-Risk and blast radius:
191
-
192
-- User-facing modal behavior changes for network-connections only.
193
-- Payload size risk is moderate if detailed evidence is duplicated. This SOW must not duplicate socket rows only for modal display.
194
-- Sensitive data risk is high because process command lines, usernames, and local endpoints are intentionally exposed to authorized users.
195
-
196
-Sensitive data handling plan:
197
-
198
-- Do not copy raw command lines, usernames, bearer tokens, cookies, public IPs from user systems, or customer-identifying endpoint data into durable artifacts.
199
-- Use synthetic examples only.
200
-- Treat all `actor_labels` and socket evidence as topology Function sensitive data.
201
-
202
-Implementation plan:
203
-
204
-1. Inventory the old and current modal fields for node/self, process, and endpoint actors.
205
-2. Define the intended modal composition for each actor type:
206
- - selected identity/header labels;
207
- - one primary connection table;
208
- - optional listening/local-port summary;
209
- - optional detailed evidence expansion/section only when detailed evidence exists and adds information.
210
-3. Add or adjust canonical columns only when the useful field is missing from actors, links, `socket_ports`, or `evidence.socket`.
211
-4. Update `network-viewer.c` modal recipes without duplicating socket evidence.
212
-5. Update docs/spec/skill if the schema contract or recommended network-connections shape changes.
213
-6. Produce a frontend TODO if the header/identification area needs schema support or UI changes.
214
-
215
-Validation plan:
216
-
217
-- Validate generated payload JSON against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
218
-- Run C syntax check for `network-viewer.c`.
219
-- Use local Agent Function output in aggregated and detailed modes.
220
-- Confirm actor modals for self/node, process, and endpoint actors show non-empty identity and a non-duplicative useful connection view.
221
-- Check payload size before/after on a realistic network-connections payload.
222
-
223
-Artifact impact plan:
224
-
225
-- AGENTS.md: likely unaffected.
226
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md` if the recommended network-connections modal shape changes.
227
-- Specs: update `.agents/sow/specs/topology-function-schema.md`.
228
-- End-user/operator docs: likely unaffected unless Function output docs expose examples.
229
-- End-user/operator skills: unaffected.
230
-- SOW lifecycle: close only after integrated local Agent/UI validation or an explicit tracked follow-up SOW exists.
231
-
232
-Open-source reference evidence:
233
-
234
-- Not checked yet. This SOW is about Netdata-specific Function modal semantics; external references may be useful only for general socket table UX and should be recorded if used during analysis.
235
-
236
-Open decisions:
237
-
238
-- Resolved by SOW-0028 and the SOW-0025 decisions below:
239
- - modal identification uses `modal.labels.identification.fields[]`;
240
- - aggregated network-connections uses `data.tables.relationship.connections`;
241
- - detailed network-connections uses exact socket evidence as the primary
242
- modal table.
243
-
244
-## Implications And Decisions
245
-
246
-### Decision 1: Modal Identification/Header Contract
247
-
248
-Evidence:
249
-
250
-- The schema has `presentation.modal.labels` and `presentation.modal.sections`, but no field that selects important labels for the modal header (`src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:1073`).
251
-- The producer already emits the needed labels for self, process, and endpoint actors (`src/collectors/network-viewer.plugin/network-viewer.c:2029`, `src/collectors/network-viewer.plugin/network-viewer.c:2065`, `src/collectors/network-viewer.plugin/network-viewer.c:2101`).
252
-
253
-Options:
254
-
255
-- **A. Extend `presentation.modal.labels` with ordered `summary_keys` / `identity_keys`. Recommended.**
256
- - Pros: small schema change; keeps UI topology-agnostic; reuses `actor_labels`; no row duplication.
257
- - Cons: requires frontend and aggregator to preserve/read the new label presentation field.
258
- - Implication: each actor type can say which label keys appear in the header, while the full `Labels` tab remains complete.
259
-- **B. Add a more general `presentation.modal.identity.fields[]` using modal projections.**
260
- - Pros: most flexible; can pull from actor columns or labels.
261
- - Cons: larger schema/UI work; more ways for producers to create inconsistent header/table semantics.
262
- - Implication: useful later, but heavier than this SOW needs.
263
-- **C. Let the frontend guess from label keys.**
264
- - Pros: no schema change.
265
- - Cons: violates producer-driven presentation; the UI would learn network-connections-specific labels.
266
- - Risk: repeats the old problem in another form.
267
-
268
-Recommendation: **A**. The header is a curated subset of `actor_labels`, not a new data source.
269
-
270
-### Decision 2: Aggregated Mode Connection Detail Source
271
-
272
-Evidence:
273
-
274
-- Internal `NV_TOPOLOGY_LINK` rows preserve aggregated connection detail, including local/remote ports and service name (`src/collectors/network-viewer.plugin/network-viewer.c:80`).
275
-- v1 graph links intentionally remove port pairing from the graph-link key (`src/collectors/network-viewer.plugin/network-viewer.c:2135`).
276
-- `socket_ports` stores only actor-owned port summaries (`src/collectors/network-viewer.plugin/network-viewer.c:2235`), so it cannot answer "which remote endpoint used which port?".
277
-
278
-Options:
279
-
280
-- **A. Add `data.tables.relationship.connections` as a `relationship_summary` table. Recommended.**
281
- - Pros: uses existing internal rows once; restores accurate aggregated modal drilldown; no high-cardinality socket duplication in aggregated mode.
282
- - Cons: increases aggregated payload size by one compact row per internal connection summary.
283
- - Implication: graph links stay small, while modals use a relationship table for precise rows.
284
-- **B. Keep using graph links for aggregated modals.**
285
- - Pros: smallest payload.
286
- - Cons: cannot show local/remote port pairing or service names; leaves the current confusing modal mostly intact.
287
- - Risk: not fit for the requested SRE workflow.
288
-- **C. Use only `socket_ports` as the drilldown.**
289
- - Pros: compact and already emitted.
290
- - Cons: actor inventory is not relationship data; outbound local ephemeral ports and remote server ports get confused.
291
- - Risk: exactly the kind of misleading table the user reported.
292
-
293
-Recommendation: **A**. The summary table is not duplicated presentation data; it is the missing relationship fact plane for aggregated drilldowns.
294
-
295
-### Decision 3: Detailed Mode Modal Shape
296
-
297
-Evidence:
298
-
299
-- Detailed mode already emits `evidence.socket` rows with exact local/remote socket details (`src/collectors/network-viewer.plugin/network-viewer.c:2254`).
300
-- Current detailed modal shows both graph `Connections`, actor `Ports`, and `Socket Evidence` (`src/collectors/network-viewer.plugin/network-viewer.c:2664`, `src/collectors/network-viewer.plugin/network-viewer.c:2706`, `src/collectors/network-viewer.plugin/network-viewer.c:2735`).
301
-
302
-Options:
303
-
304
-- **A. In detailed mode, make exact socket evidence the single primary `Sockets` / `Connections` section. Recommended.**
305
- - Pros: no duplicated tabs; exact rows are shown when the user selected detailed mode; expanded columns can show less-common fields.
306
- - Cons: detailed mode tables can be large, as expected by the mode.
307
- - Implication: hide graph-link and port-summary sections from detailed actor modals unless they add a distinct summary later.
308
-- **B. Keep graph `Connections` and make socket evidence expandable below each row.**
309
- - Pros: best UX if frontend supports row-level grouping by link.
310
- - Cons: more frontend work; current schema cannot directly express nested row groups, only expanded columns.
311
- - Implication: good future improvement, but not the fastest reliable repair.
312
-- **C. Keep the three current tabs with clearer labels.**
313
- - Pros: smallest backend change.
314
- - Cons: still forces users to reconcile three views of the same fact.
315
- - Risk: fails the purpose of this SOW.
316
-
317
-Recommendation: **A**. Detailed mode should show the exact socket rows as the primary drilldown; aggregated mode should show the relationship summary rows.
318
-
319
-### Schema Support Assessment
320
-
321
-Supported by the current schema:
322
-
323
-- Actor modal recipes are already supported under `types.actor_types.<id>.presentation.modal`.
324
-- Modal sections can already read `links`, `evidence`, `actor_table`, and `relationship_table` sources.
325
-- `relationship_summary` table types are already valid.
326
-- Compact table columns can already use `actor_ref`, `link_ref`, `ip`, `uint`, metrics, strings, and dictionary encoding.
327
-- Owner filters can already bind a relationship table to the selected actor when the table has `src_actor` and `dst_actor` columns.
328
-- `selected_side_endpoint` already supports rendering a local/remote endpoint from table columns without topology-specific frontend code.
329
-
330
-Not supported yet:
331
-
332
-- There is no schema field for "show these actor label keys in the modal identification/header area". Current `modal.labels` only tells the UI how to render the full labels table.
333
-
334
-Duplication policy:
335
-
336
-- Do not duplicate raw detailed socket evidence only for modal display.
337
-- Aggregated mode needs one relationship-summary row per existing internal connection summary row because graph links intentionally collapse port pairing and service name. This is not duplicate evidence; it is a distinct compact drilldown grain between graph links and detailed socket evidence.
338
-- Some small scalar values such as protocol, direction, state, socket count, and actor refs will appear both on graph links and relationship-summary rows. This is acceptable because the rows have different grain, are dictionary/number encoded, and are needed for standalone modal filtering/sorting.
339
-- The producer should avoid copying fields that can be projected from actor labels or actor rows. Process identity stays in actors/labels; relationship rows carry relationship facts.
340
-
341
-## Plan
342
-
343
-1. Analyze field inventory and old/current modal parity for `topology:network-connections`.
344
-2. Propose the product-oriented modal tables and header labels.
345
-3. Implement only this function's backend changes after the design is accepted.
346
-4. Hand off required frontend/aggregator behavior if needed.
347
-5. Validate with real local Agent payloads in aggregated and detailed modes.
348
-
349
-## Execution Log
350
-
351
-### 2026-05-11
352
-
353
-- Created SOW from user-reported modal regressions and current code evidence.
354
-- Paused this SOW because SOW-0028 owns the broader cross-repo topology mode,
355
- correlation, aggregation, and actor-identification contract that this SOW now
356
- depends on.
357
-- Resumed after SOW-0028 completed and was committed. The next step is to
358
- validate the installed aggregated/detailed network-connections modal recipes
359
- and repair any remaining backend producer gaps.
360
-- Validated the rebuilt producer output and found one remaining mismatch: the
361
- self/root actor still needed a process summary instead of socket rows.
362
-- Updated `network-viewer.c` so self actors use a `Processes` section over
363
- `ownership` links, process actors use `Connections` or `Sockets` depending on
364
- mode, endpoint actors use `Processes`, and secondary socket metrics are
365
- expanded columns instead of separate duplicate sections.
366
-- Updated the durable topology spec, developer guide, and project topology skill
367
- so future topology producers do not reintroduce `socket_ports` as a normal
368
- network-connections modal tab.
369
-
370
-## Validation
371
-
372
-Acceptance criteria evidence:
373
-
374
-- Complete inventory and mapping are recorded above under `Field inventory and
375
- mapping`.
376
-- Self/node modal shape is explicit: `Processes` from `links`, filtered to
377
- `type == ownership`, with process actor, socket count, and expanded evidence
378
- count.
379
-- Process modal shape is explicit:
380
- - aggregated mode: `Connections` from `tables.relationship.connections`;
381
- - detailed mode: `Sockets` from `evidence.socket`.
382
-- Endpoint modal shape is explicit: `Processes` from the same mode-specific
383
- relationship/evidence source.
384
-- Actor identification source is `modal.labels.identification.fields[]` over
385
- `actor_labels`.
386
-- `socket_ports` remains graph port-bullet inventory only; it is not emitted as
387
- a normal network-connections modal section.
388
-
389
-Tests or equivalent validation:
390
-
391
-- `git diff --check`: passed.
392
-- `sudo -n cmake --build build --target network-viewer.plugin -- -j2`: passed.
393
-- Plugin protocol validation against the rebuilt binary:
394
- - aggregated: status 200, mode `aggregated`, 110 actors, 147 links, 244
395
- relationship rows, 730 actor labels, 192 port rows, 0 socket evidence rows;
396
- - detailed: status 200, mode `detailed`, 111 actors, 147 links, 0 relationship
397
- rows, 735 actor labels, 192 port rows, 244 socket evidence rows.
398
-- `go run ./tools/functions-validation/validate --schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json --input ../../.local/audits/topology-sow-0025/network-connections-aggregated-protocol.json --min-rows 1`: passed.
399
-- `go run ./tools/functions-validation/validate --schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json --input ../../.local/audits/topology-sow-0025/network-connections-detailed-protocol.json --min-rows 1`: passed.
400
-- `go test ./pkg/topology/v1 ./tools/functions-validation/validate`: passed.
401
-
402
-Real-use evidence:
403
-
404
-- Rebuilt and installed only `/usr/libexec/netdata/plugins.d/network-viewer.plugin`
405
- with the same owner/group/mode as the previous installed binary.
406
-- Direct HTTP validation through `localhost:19999` remained blocked by SSO
407
- authorization returning HTTP 412, including through the token-safe minted
408
- bearer helper. The rebuilt plugin was therefore validated through the normal
409
- plugins.d `FUNCTION` stdin protocol, which exercises the same producer code
410
- path without changing Agent authentication settings.
411
-- `.local/audits/topology-sow-0025/network-connections-aggregated-protocol.json`
412
- and `.local/audits/topology-sow-0025/network-connections-detailed-protocol.json`
413
- contain the validation payloads and are intentionally gitignored local
414
- artifacts.
415
-
416
-Reviewer findings:
417
-
418
-- No external AI reviewer was requested for this narrow backend repair. The
419
- broader topology modal/correlation contract was reviewed during the preceding
420
- SOW-0028 work; this SOW validated the concrete producer output with schema and
421
- semantic checks.
422
-
423
-Same-failure scan:
424
-
425
-- Searched the durable topology spec, developer guide, project topology skill,
426
- and this SOW for stale `graph links plus socket_ports`, duplicate modal, and
427
- `Socket Evidence` language. Historical problem statements remain in this SOW;
428
- durable current-contract docs were updated.
429
-
430
-Sensitive data gate:
431
-
432
-- This SOW uses only path/line evidence and synthetic descriptions. No raw sensitive payload data is included.
433
-
434
-Artifact maintenance gate:
435
-
436
-- `AGENTS.md`: unchanged; no workflow or guardrail changed.
437
-- Runtime project skills: updated
438
- `.agents/skills/project-create-topology/SKILL.md` with the
439
- network-connections modal recipe.
440
-- Specs: updated `.agents/sow/specs/topology-function-schema.md` with the
441
- current network-connections modal composition.
442
-- Developer docs: updated `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`.
443
-- End-user/operator docs: unchanged; this changes developer topology payload
444
- composition, not operator instructions.
445
-- End-user/operator skills: unchanged; no public operator workflow changed.
446
-- SOW lifecycle: this SOW is ready to move from `current/` to `done/` with
447
- `Status: completed`.
448
-
449
-Specs update:
450
-
451
-- Updated `.agents/sow/specs/topology-function-schema.md`.
452
-
453
-Project skills update:
454
-
455
-- Updated `.agents/skills/project-create-topology/SKILL.md`.
456
-
457
-End-user/operator docs update:
458
-
459
-- Not affected. The user-facing Function remains the same; only v1 topology
460
- modal presentation metadata changed.
461
-
462
-End-user/operator skills update:
463
-
464
-- Not affected. No public/operator skill behavior or examples changed.
465
-
466
-Lessons:
467
-
468
-- Network-connections modal tabs must describe user tasks, not storage tables.
469
- `socket_ports` is useful for graph bullets but misleading as a peer modal tab.
470
-- The plugin stdin protocol is useful for validating producer output when local
471
- Agent HTTP access is blocked by SSO.
472
-
473
-Follow-up mapping:
474
-
475
-- SOW-0029 tracks the separate detailed loose-side/network-connections model
476
- work. No additional SOW-0025 follow-up remains.
477
-
478
-## Outcome
479
-
480
-Completed. The network-connections producer emits task-oriented actor modal
481
-recipes for self, process, and endpoint actors in both aggregated and detailed
482
-modes, without adding duplicated modal-only socket rows.
483
-
484
-## Lessons Extracted
485
-
486
-- Keep modal composition tied to user troubleshooting questions. Internal tables
487
- can remain in the payload for graph affordances or aggregation, but they should
488
- not automatically become top-level modal tabs.
489
-- Validate topology producer changes through plugins.d protocol output when HTTP
490
- auth prevents local Function calls.
491
-
492
-## Followup
493
-
494
-- SOW-0029 covers the next network-connections detailed loose-side model work.
495
-
496
-## Regression Log
497
-
498
-None yet.
499
-
500
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/done/SOW-0026-20260511-snmp-modal-product-composition.md
deleted
-642
@@ -1,642 +0,0 @@
1
-# SOW-0026 - SNMP Modal Product Composition
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: second regression repair, documentation updates, validation, and
8
-local build completed on 2026-05-11.
9
-
10
-## Requirements
11
-
12
-### Purpose
13
-
14
-Make `topology:snmp` actor modals useful for network engineers who need accurate port, neighbor, and L2 relationship information without contradictory tables.
15
-
16
-### User Request
17
-
18
-The user reported that SNMP modals show `Ports` and `Links`, but the distinction is confusing and appears incorrect. A device cannot have an L2 link that is not associated with a port. The user also reported that local port labels can be wrong and that `Ports` and `Links` must be 100% in sync or the modal is useless.
19
-
20
-### Assistant Understanding
21
-
22
-Facts:
23
-
24
-- Current SNMP device modals include a `Ports` section from actor table `actor_ports` and a `Links` section from graph links.
25
-- Current `actor_ports` modal columns show `name`, statuses, role, VLAN/FDB/link/neighbor counts, and debug `extra`.
26
-- Current `Links` modal columns show remote actor, local port, remote port, protocol, direction, state, and evidence count.
27
-- The current `Links` section derives local/remote ports from selected-side port projections over link endpoint columns.
28
-
29
-Inferences:
30
-
31
-- The issue is likely not just labels. The underlying product model is wrong if ports and links can disagree.
32
-- For a device actor, the primary table should probably be a port-centric table. Links/neighbors should be columns or expandable rows attached to each port, not an independent table with a different interpretation of "local port".
33
-- Inferred endpoint actors may still need a relationship/link modal, but managed SNMP devices need port inventory as the organizing principle.
34
-
35
-Confirmed by code review:
36
-
37
-- Link evidence already carries `src_if_index` and `dst_if_index`, and graph links carry `src_port_name` and `dst_port_name`.
38
-- The port inventory source rows commonly carry `if_index`, `if_name`, `if_descr`, `if_alias`, `mac`, and `speed`, but the current `actor_ports` table does not expose them as typed canonical columns.
39
-- Managed device actors should use a port-centric modal. Endpoint/segment actors, which do not have port inventory, can keep a relationship-oriented `Links` section.
40
-- `topologyV1EndpointPortName()` falls back to `display_name` and `sys_name`; that can turn an actor/device/IP label into a displayed local port name when no real port field is present.
41
-
42
-### Acceptance Criteria
43
-
44
-- A complete inventory exists for SNMP actor modal facts: actor labels, actor typed columns, `actor_ports`, graph links, and evidence sections.
45
-- The desired SNMP device modal is port-centric and shows actual numeric port IDs when known, plus port name.
46
-- `Ports` and link/neighbor information are derived from the same canonical endpoint facts or explicitly cross-checked.
47
-- Any remaining `Links` section has a precise purpose and cannot contradict the port table.
48
-- The SOW identifies every missing canonical field required to make port/link rows 100% aligned.
49
-- The actor identification area exposes important device labels, not only the generic `Labels` tab.
50
-
51
-## Analysis
52
-
53
-Sources checked:
54
-
55
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:158` current SNMP device modal.
56
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:163` current `Ports` section.
57
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:188` current `Links` section.
58
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:212` current selected-side local/remote port columns.
59
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:474` `actor_ports` table type.
60
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:493` current port modal columns.
61
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:534` current `actor_ports` columns.
62
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:699` link/evidence construction entry point.
63
-- `.agents/sow/specs/topology-function-schema.md:380` SNMP modal composition notes.
64
-- `.agents/sow/specs/topology-modes-correlation-aggregation.md:498-556` SNMP detailed/aggregated and port-centric modal requirements.
65
-- `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Models/Port.php:90-138` shows port labels are built from `ifName`, `ifAlias`, `ifDescr`, and sometimes `ifIndex`.
66
-- `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Models/Port.php:401-427` models links as port relationships (`local_port_id`, `remote_port_id`).
67
-- `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Http/Controllers/Device/Tabs/PortsController.php:145-185` organizes neighbors from a port-first data set.
68
-- `netdisco/netdisco @ 7c1bc3e8290fe7b757c6a462de12e4613eca3899 share/views/ajax/device/ports.tt:394-406` renders neighbors from the device port row and remote port.
69
-
70
-Current state:
71
-
72
-- Device modal:
73
- - `Ports` comes from `actor_ports`.
74
- - `Links` comes from graph links.
75
-- `actor_ports` currently has stable columns such as `name`, topology role, admin/oper status, port type, link mode, STP state, VLAN IDs, FDB count, link count, neighbor count, neighbors JSON, VLANs JSON, and debug `extra`.
76
-- `Links` currently uses selected-side projections for `src_port_name` and `dst_port_name`, but the user observed cases where local port looks like a remote actor/IP. This points to a possible endpoint mapping or projection issue.
77
-- `topologyV1EndpointPortName()` currently accepts `display_name` and `sys_name` as final fallbacks. These are actor labels, not port labels, and explain how a remote actor/IP can appear as a port.
78
-
79
-Available facts to inventory:
80
-
81
-- Device actor:
82
- - display name, sysName, sysDescr, model, vendor, management IP, sys location/contact, protocols/capabilities, port counts, VLAN/FDB/LLDP/CDP counts.
83
-- Port table:
84
- - current normalized port name/status/role/VLAN/FDB/neighbor facts.
85
- - likely raw/custom fields in `extra` that may include actual `if_index`, `if_name`, `if_descr`, `if_alias`, MAC, speed, duplex, VLAN details, and neighbor objects.
86
-- Link/evidence:
87
- - remote actor, source/destination port names, source/destination ifIndex when present, protocol, direction/state, confidence/inference/attachment mode, evidence count.
88
-
89
-Target audience and questions:
90
-
91
-- Network engineers need:
92
- - Which physical/logical ports exist?
93
- - What is the actual numeric port ID, ifIndex, interface name, and description/alias?
94
- - Which ports are up/down/admin-down?
95
- - What is connected to each port and by which protocol/evidence?
96
- - Is the link verified by LLDP/CDP or inferred by FDB/ARP/STP?
97
- - Which VLANs, STP role/state, FDB MAC counts, speeds, and neighbors apply to each port?
98
-
99
-Reference pattern:
100
-
101
-- Network inventory tools treat the port as the organizing object. LibreNMS keeps port labels on `ifName`/`ifAlias`/`ifDescr`/`ifIndex`, and its link model joins ports through local and remote port IDs. Netdisco similarly renders neighbor details from the selected device port row and remote port.
102
-- For Netdata topology modals, this means a managed SNMP device should not present graph links as an equal peer of port inventory. It should present port inventory first and neighbor/link evidence as port-aligned details.
103
-
104
-Available backend facts:
105
-
106
-- Device actor labels and typed actor columns already include display name, sysName, vendor, model, management IP, sysDescr, sysLocation, sysContact, capabilities, protocol lists, port counts, VLAN counts, FDB counts, and LLDP/CDP neighbor counts.
107
-- Port actor-owned rows have raw fields such as `if_index`, `if_name`, `if_descr`, `if_alias`, `mac`, `speed`, status, type, mode, topology role, STP state, VLAN IDs/details, FDB count, link count, neighbor count, and neighbor objects.
108
-- Graph links have source/destination actors, link type, protocol, direction, state, source/destination port names, evidence count, discovered time, and last-seen time.
109
-- Evidence rows have source/destination `if_index`, source/destination management IP, confidence, inference, attachment mode, and raw endpoint/metrics JSON for debug.
110
-
111
-Missing or weak canonicalization:
112
-
113
-- `actor_ports` does not expose `if_index`, `if_name`, `if_descr`, `if_alias`, `port_id`, `mac`, or `speed` as typed columns.
114
-- Link modal selected-side port display relies on `src_port_name`/`dst_port_name`, and those values can currently be polluted by actor labels because of the endpoint-port fallback.
115
-- There is no port-aligned relationship table for managed devices. The only link view is generic graph-link-oriented, so it can disagree visually with the port inventory.
116
-
117
-Risks:
118
-
119
-- If port and link tables are inconsistent, users will distrust the topology.
120
-- Showing raw nested neighbor JSON in normal table cells would regress polish and usefulness.
121
-- If numeric port IDs are unavailable, fabricating autoincrement IDs would be worse than showing no numeric ID.
122
-
123
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
124
-
125
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Execution Log and Validation sections).
126
-
127
-Problem / root-cause model:
128
-
129
-- SNMP modals currently expose implementation tables separately rather than presenting a device as a collection of ports with attached neighbor/link evidence.
130
-- The current `Links` section can produce confusing local/remote port labels because it is graph-link-oriented, not port-oriented.
131
-
132
-Evidence reviewed:
133
-
134
-- SNMP device modal currently has both `Ports` and `Links` sections in `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:158-180`.
135
-- `Links` local/remote port columns are selected-side projections in `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:212-213`.
136
-- `actor_ports` modal currently does not expose a clear numeric port ID column in the visible section at `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:493-503`.
137
-- `actor_ports` table columns currently omit `if_index`, `if_name`, `if_descr`, `if_alias`, `port_id`, `mac`, and `speed` at `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:534-550`.
138
-- `topologyV1EndpointPortName()` falls back to `display_name` and `sys_name` at `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:1551-1559`.
139
-
140
-Affected contracts and surfaces:
141
-
142
-- Agent Function payload for `topology:snmp`.
143
-- SNMP topology Go v1 adapter.
144
-- SNMP topology tests and fixtures.
145
-- Cloud frontend modal rendering if the actor identification/header area needs a new selector.
146
-- Cloud aggregator if it merges device port rows and link/evidence rows.
147
-- Developer guide, topology spec, project topology skill.
148
-
149
-Existing patterns to reuse:
150
-
151
-- `actor_ports` actor-owned table for port inventory.
152
-- Structured evidence columns for L2 discovery details.
153
-- `selected_side_endpoint` only where side selection is truly needed.
154
-- Debug `extra` JSON for unknown custom port fields, not normal table display.
155
-- Actor-owned modal detail tables with `owner_filter: actor_column`.
156
-- `modal.labels.identification.fields[]` for modal header facts.
157
-
158
-Risk and blast radius:
159
-
160
-- User-facing SNMP device modal behavior changes.
161
-- Incorrect port/evidence joins can misrepresent physical topology.
162
-- Aggregated cloud topologies may need stricter table merge rules for port identity.
163
-- Sensitive data risk includes device sysContact/sysLocation and management IPs; do not write raw real values to durable artifacts.
164
-
165
-Sensitive data handling plan:
166
-
167
-- Use synthetic device/port/IP examples only.
168
-- Do not store raw SNMP sysContact/sysLocation, management IPs, customer names, credentials, or community strings in SOWs/docs/tests.
169
-- Keep real payload captures under `.local/` only.
170
-
171
-Implementation plan:
172
-
173
-1. Add SNMP modal identification fields:
174
- - managed devices: display name, management IP, vendor, model, port counts, LLDP/CDP counts;
175
- - endpoints/segments: display name and matching network identifiers.
176
-2. Expose typed port inventory columns in `actor_ports`: `if_index`, `port_id`, `if_name`, `if_descr`, `if_alias`, `mac`, and `speed`.
177
-3. Remove actor-label fallbacks from `topologyV1EndpointPortName()` so only real port fields can become link endpoint port labels.
178
-4. Add a compact actor-owned `actor_port_links` detail table:
179
- - one row per selected actor side of each graph link;
180
- - columns include local `if_index`, local port name, remote actor, remote port name, protocol, link type, state, evidence count, confidence, inference, attachment mode, discovered time, and last-seen time;
181
- - this table is derived from graph links/evidence facts and exists to align device modal relationships with port inventory, not to create new topology facts.
182
-5. Change managed device modals to show `Ports` then `Port Neighbors`; keep generic `Links` only for endpoint/segment/custom actors that do not own port inventory.
183
-6. Add tests proving:
184
- - device modal no longer uses the generic `Links` section;
185
- - `actor_ports` exposes typed port identity/status columns;
186
- - `actor_port_links` is side-owned and uses the same local `if_index`/port name as evidence;
187
- - endpoint port names do not fall back to device/IP display names.
188
-7. Update spec, developer guide, and project topology skill.
189
-
190
-Validation plan:
191
-
192
-- Go tests for SNMP topology v1 conversion.
193
-- Payload schema validation.
194
-- Fixture test with a device containing multiple ports, LLDP/CDP verified links, inferred FDB/ARP links, and ports without links.
195
-- Cross-check that every link shown under a device maps to the same local port row shown in the port table when a port identifier exists.
196
-- Manual UI validation on a real or sanitized SNMP topology payload.
197
-
198
-Artifact impact plan:
199
-
200
-- AGENTS.md: likely unaffected.
201
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md` if SNMP modal guidelines change.
202
-- Specs: update `.agents/sow/specs/topology-function-schema.md`.
203
-- End-user/operator docs: likely unaffected unless Function examples are changed.
204
-- End-user/operator skills: unaffected.
205
-- SOW lifecycle: close only after SNMP modal behavior is validated with realistic data.
206
-
207
-Open-source reference evidence:
208
-
209
-- LibreNMS and Netdisco both support the port-first model:
210
- - `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Models/Port.php:90-138`
211
- - `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Models/Port.php:401-427`
212
- - `librenms/librenms @ 1d096602a4ce1197faab084b2523c7dd50419427 app/Http/Controllers/Device/Tabs/PortsController.php:145-185`
213
- - `netdisco/netdisco @ 7c1bc3e8290fe7b757c6a462de12e4613eca3899 share/views/ajax/device/ports.tt:394-406`
214
-
215
-Open decisions:
216
-
217
-- Resolved by user approval to proceed after product analysis:
218
- - Managed SNMP device actors get a port-first modal with `Ports` and a port-aligned `Port Neighbors` section.
219
- - Endpoint/segment/custom actors keep the generic relationship-oriented `Links` section because they do not own port inventory.
220
- - No synthetic numeric port IDs may be generated. Numeric port ID is shown only when a real `if_index` or source port ID is known.
221
-
222
-## Implications And Decisions
223
-
224
-1. Device modal composition:
225
- - Decision: managed devices use `Ports` + `Port Neighbors`; generic graph `Links` is removed from managed device modals.
226
- - Implication: users inspect a device by physical/logical port first, which matches the network engineer workflow.
227
- - Risk: a link with missing local port identity will appear in `Port Neighbors` with an empty local port. That is preferable to fabricating a port or showing an actor label as a port.
228
-
229
-2. Endpoint/segment/custom modal composition:
230
- - Decision: actors without owned port inventory keep the existing `Links` section.
231
- - Implication: endpoint/segment actors can still explain how they connect to the graph.
232
- - Risk: selected-side projections remain there, but the endpoint port fallback fix prevents actor names from being shown as ports.
233
-
234
-3. Port identity:
235
- - Decision: expose actual `if_index` and source `port_id` as typed port columns; never autoincrement.
236
- - Implication: the UI can show numeric port IDs only when the SNMP backend knows them.
237
- - Risk: devices without `if_index` will show blank `Port ID`, which is truthful.
238
-
239
-4. Data duplication:
240
- - Decision: add `actor_port_links` as a compact actor-owned modal index over existing graph links.
241
- - Implication: the table adds small side-specific rows, but does not duplicate raw evidence or nested metadata.
242
- - Risk: Cloud aggregation must treat this as actor-owned modal detail and append/merge consistently.
243
-
244
-## Plan
245
-
246
-1. Move SOW to current and record product decisions.
247
-2. Implement typed port inventory columns and modal identification fields.
248
-3. Implement `actor_port_links` and switch device modal sections.
249
-4. Fix endpoint port fallback.
250
-5. Update tests, spec, developer guide, and project skill.
251
-6. Validate with Go tests and topology schema validation.
252
-
253
-## Execution Log
254
-
255
-### 2026-05-11
256
-
257
-- Created SOW from user-reported SNMP modal regressions and current code evidence.
258
-- Completed product analysis:
259
- - managed SNMP devices must be port-first;
260
- - endpoint/segment/custom actors keep generic links;
261
- - actor labels must not be used as port labels;
262
- - `actor_ports` needs typed port identity columns;
263
- - `actor_port_links` is needed as a compact modal index.
264
-- Implemented managed-device modal identification and port-first sections.
265
-- Added typed `actor_ports` columns for real port identity and status.
266
-- Added `actor_port_links` as a compact port-aligned modal index over graph links.
267
-- Removed actor-label fallbacks from SNMP endpoint port names.
268
-- Updated SNMP topology tests, topology spec, developer guide, and project topology skill.
269
-
270
-## Validation
271
-
272
-Acceptance criteria evidence:
273
-
274
-- Complete inventory: recorded in this SOW under `Analysis`, including actors,
275
- `actor_labels`, `actor_ports`, graph links, evidence, and missing canonical
276
- fields.
277
-- Port-centric device modal: managed device modal now uses `Ports` and
278
- `Port Neighbors`, with `actor_ports` first at
279
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:156-177`.
280
-- Actor identification: device modal label identification fields are declared
281
- at `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:189-204`.
282
-- Numeric port IDs: `actor_ports` exposes `if_index` and source `port_id` as
283
- typed columns at
284
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:658-682`.
285
-- Port/link alignment: `actor_port_links` table type and modal columns are
286
- declared at
287
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:597-628`,
288
- and row construction uses one row per incident actor side at
289
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:1447-1485`.
290
-- Generic graph `Links` remain available only through the endpoint/segment/custom
291
- modal path at
292
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:181-186`.
293
-- Actor labels no longer become port names:
294
- `topologyV1EndpointPortName()` now accepts only real port fields at
295
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go:1830-1837`.
296
-
297
-Tests or equivalent validation:
298
-
299
-- `cd src/go && go test ./plugin/go.d/collector/snmp_topology`
300
-- `cd src/go && go test ./plugin/go.d/collector/snmp_topology ./pkg/topology/v1 ./tools/functions-validation/validate`
301
-- `cd src/go && for fixture in tools/functions-validation/fixtures/topology-v1/*.json; do go run ./tools/functions-validation/validate -schema ../plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json -input "$fixture" -require-rows; done`
302
-- `git diff --check`
303
-- `sudo -n cmake --build build --target go.d.plugin -- -j2`
304
-- Unit-test evidence:
305
- - typed port columns and values:
306
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go:360-384`;
307
- - `actor_port_links` side-owned rows:
308
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go:386-398`;
309
- - managed device modal sections:
310
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go:420-430`;
311
- - endpoint actors keep generic links:
312
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go:432-437`;
313
- - actor label fallback regression:
314
- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go:440-486`.
315
-
316
-Real-use evidence:
317
-
318
-- Built `go.d.plugin` through the local CMake/Ninja build path with the updated
319
- SNMP topology producer. The running local Agent was not modified by this SOW;
320
- no production or customer system was used.
321
-
322
-Reviewer findings:
323
-
324
-- External AI reviewers were not requested for this SOW. Review coverage came
325
- from code inspection, open-source reference comparison, schema validation,
326
- targeted tests, and same-failure search.
327
-
328
-Same-failure scan:
329
-
330
-- Searched for actor-label-to-port-name fallback and selected-side SNMP link
331
- modal risks with:
332
- `rg -n "display_name.*port|sys_name.*port|port.*display_name|port.*sys_name|topologyV1EndpointPortName|selected_side_endpoint" src/go/plugin/go.d/collector/snmp_topology src/go/plugin/go.d/collector src/collectors -S`
333
-- Remaining `selected_side_endpoint` usage in SNMP is intentionally limited to
334
- endpoint/segment/custom actor modals. The SNMP endpoint port-name helper no
335
- longer accepts actor labels.
336
-
337
-Sensitive data gate:
338
-
339
-- This SOW uses only path/line evidence and synthetic descriptions. No raw sensitive payload data is included.
340
-
341
-Artifact maintenance gate:
342
-
343
-- `AGENTS.md`: unchanged. No project-wide workflow rule changed.
344
-- Runtime project skills: updated
345
- `.agents/skills/project-create-topology/SKILL.md:260-283` with SNMP/L2 modal
346
- rules.
347
-- Specs: updated
348
- `.agents/sow/specs/topology-function-schema.md:418-444` with SNMP port-first
349
- modal contract.
350
-- End-user/operator docs: unchanged. This changes developer-facing topology
351
- payload composition, not an operator workflow or public querying procedure.
352
-- End-user/operator skills: unchanged. No operator-facing skill semantics
353
- changed.
354
-- SOW lifecycle: this SOW will move from `current/` to `done/` with status
355
- `completed` in the same commit as the implementation.
356
-
357
-Specs update:
358
-
359
-- Updated `.agents/sow/specs/topology-function-schema.md`.
360
-
361
-Project skills update:
362
-
363
-- Updated `.agents/skills/project-create-topology/SKILL.md`.
364
-
365
-End-user/operator docs update:
366
-
367
-- No update needed. The change is internal developer schema composition for
368
- topology producers.
369
-
370
-End-user/operator skills update:
371
-
372
-- No update needed. Public/operator skills do not describe developer modal
373
- composition.
374
-
375
-Lessons:
376
-
377
-- For SNMP/L2, a generic graph link table is not a good device modal. Port
378
- inventory has to be the organizing object, and relationship rows must align
379
- with that port identity.
380
-- Link endpoint display helpers must never mix actor identity labels with port
381
- labels.
382
-
383
-Follow-up mapping:
384
-
385
-- No deferred implementation work remains in this SOW. Interface overlay work
386
- for traffic/packets/errors/state is a separate topology overlay topic already
387
- documented in the spec and developer guide.
388
-
389
-## Outcome
390
-
391
-Completed. Managed SNMP device modals are now port-first, expose typed port
392
-identity, show port-aligned neighbor rows, and avoid using actor labels as port
393
-names. Endpoint/segment/custom actor modals keep generic graph-link drilldowns.
394
-
395
-## Lessons Extracted
396
-
397
-- SNMP topology presentation needs network-engineer semantics: port identity
398
- first, neighbor/link evidence attached to that identity, and raw JSON only for
399
- debug/expanded diagnostics.
400
-
401
-## Followup
402
-
403
-See regression entries below.
404
-
405
-## Regression Log
406
-
407
-See regression entries below.
408
-
409
-Note: dated regression entries intentionally use `## Regression - YYYY-MM-DD`
410
-headings to match the repository SOW lifecycle contract.
411
-
412
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
413
-
414
-## Regression - 2026-05-11
415
-
416
-### What Broke
417
-
418
-- The SNMP device `Ports` section labels `if_index` as `Port ID`, which looks like
419
- an invented/autoincrement port number. The user requirement is no synthetic
420
- numbering anywhere. Real port numbers may be shown only when sourced from
421
- canonical producer data.
422
-- The `Ports` expanded row does not show the neighboring actor as a clickable
423
- actor link. The separate `Port Neighbors` section has clickable remote actors,
424
- but the port-owned row must also expose the directly related neighbor.
425
-- Actor links in v1 modal tables were not clickable in the frontend because the
426
- producer emits `actor_ref_label` projections with `actor_link` cells, while
427
- the frontend projection engine returned labels instead of actor IDs for that
428
- cell type.
429
-
430
-### Evidence
431
-
432
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` currently
433
- emits the first `Ports` modal column as `if_index` with label `Port ID`.
434
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` currently
435
- defines `actor_ports` without neighbor actor-ref columns, so the expanded
436
- `Ports` view cannot link to the neighbor.
437
-- `cloud-frontend/src/domains/functions/topology/v1/buildModalSections.js`
438
- projected `actor_ref_label` to a label string even when the declared cell type
439
- was `actor_link`.
440
-
441
-### Root Cause
442
-
443
-- The SOW correctly made SNMP modals port-first, but it still reused SNMP
444
- `ifIndex` as the visible port identity. `ifIndex` is useful technical
445
- metadata, but it is not a physical/source port number and should not be
446
- presented as `Port ID`.
447
-- Port inventory and port-neighbor rows were aligned in separate tables, but the
448
- inventory table did not carry a compact derived neighbor actor reference for
449
- the expanded row.
450
-- The frontend projection engine treated projection kind as the only source of
451
- output shape. For `actor_link` cells, the renderer needs the actor ID, not the
452
- already formatted actor label.
453
-
454
-### Repair Plan
455
-
456
-- Add a nullable `port_number` column to `actor_ports`. Populate it only from an
457
- explicit numeric source field such as `port_number` or numeric `port_id`.
458
- Never derive it from row order or `if_index`.
459
-- Rename the expanded `if_index` modal column to `SNMP ifIndex` and remove it
460
- from the visible table columns.
461
-- Add compact nullable `neighbor_actor` and `neighbor_port_name` columns to
462
- `actor_ports`, derived from the same graph-link endpoint facts used by
463
- `actor_port_links`.
464
-- Show `Neighbor` as an expanded `actor_link` cell and `Neighbor Port` as an
465
- expanded text cell in the `Ports` section.
466
-- Fix the frontend modal projection engine so `actor_ref_label` returns actor IDs
467
- when the cell type is `actor_link`.
468
-
469
-### Validation Plan
470
-
471
-- Add/adjust SNMP topology tests proving:
472
- - `port_number` is present and populated only from explicit source data;
473
- - `if_index` remains available as expanded SNMP metadata;
474
- - `neighbor_actor` and `neighbor_port_name` are derived from graph-link facts;
475
- - the main `Ports` modal no longer labels `if_index` as `Port ID`.
476
-- Add frontend projection tests proving `actor_ref_label` + `actor_link` produces
477
- actor IDs.
478
-- Run targeted Go and frontend tests plus schema validation and `git diff --check`.
479
-
480
-### Implementation
481
-
482
-- Added nullable `actor_ports.port_number`, populated only from explicit
483
- `port_number` or numeric source `port_id`.
484
-- Kept SNMP `if_index` as expanded technical metadata labelled `SNMP ifIndex`,
485
- not as visible `Port ID`.
486
-- Added nullable `actor_ports.neighbor_actor` and `neighbor_port_name`, derived
487
- from graph-link endpoint facts so the expanded port row can link to the same
488
- neighbor shown in `actor_port_links` when the port has one unambiguous remote
489
- actor.
490
-- Fixed the frontend v1 modal projection engine so `actor_ref_label` returns
491
- actor IDs for `actor_link` cells while keeping text/badge cells label-based.
492
-- Updated the topology developer guide, topology specs, and project topology
493
- skill with the stricter SNMP port identity and expanded-row neighbor rules.
494
-
495
-### Validation
496
-
497
-Acceptance criteria evidence:
498
-
499
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` now emits
500
- `port_number` as the visible first Ports column and `if_index` as expanded
501
- `SNMP ifIndex`.
502
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` now emits
503
- `neighbor_actor` and `neighbor_port_name` in `actor_ports`.
504
-- `cloud-frontend/src/domains/functions/topology/v1/buildModalSections.js`
505
- now returns actor IDs for `actor_ref_label` projections when the cell is
506
- `actor_link`.
507
-
508
-Tests or equivalent validation:
509
-
510
-- `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp_topology ./pkg/topology/v1 ./tools/functions-validation/validate`
511
-- `yarn test src/domains/functions/topology/v1/buildModalSections.test.js --runInBand`
512
-- `sudo -n cmake --build build --target go.d.plugin -- -j2`
513
-- `git diff --check` in the Agent repository.
514
-- `git diff --check` in the frontend repository.
515
-
516
-Real-use evidence:
517
-
518
-- Built the local `go.d.plugin` target successfully. Browser inspection can be
519
- repeated after the rebuilt Agent and UI are installed; no code change remains
520
- blocked on that check.
521
-
522
-Reviewer findings:
523
-
524
-- External AI reviewers were not requested for this regression repair. The fix
525
- is covered by targeted producer and frontend unit tests plus schema validation.
526
-
527
-Same-failure search:
528
-
529
-- Verified the repaired modal contract by searching for `Port ID`, `if_index`,
530
- `actor_ports`, `neighbor_actor`, and `actor_ref_label` in the affected producer,
531
- specs, skill, and frontend projection files.
532
-
533
-Artifact maintenance gate:
534
-
535
-- `AGENTS.md`: unchanged. No project-wide workflow rule changed.
536
-- Runtime project skills: updated `.agents/skills/project-create-topology/SKILL.md`
537
- with stricter SNMP `port_number`, `if_index`, and expanded neighbor rules.
538
-- Specs: updated `.agents/sow/specs/topology-function-schema.md` and
539
- `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
540
-- End-user/operator docs: unchanged. This is developer-facing topology payload
541
- composition, not an operator workflow.
542
-- End-user/operator skills: unchanged. No public/operator skill semantics
543
- changed.
544
-- SOW lifecycle: reopened regression SOW moved back to `current/`, repaired,
545
- then returned to `done/` with `Status: completed`.
546
-
547
-Follow-up mapping:
548
-
549
-- No new deferred implementation work remains in this SOW. Manual visual
550
- polishing of topology layout remains outside this SNMP modal regression.
551
-
552
-### Regression Outcome
553
-
554
-Completed. SNMP Ports no longer display SNMP `ifIndex` as a synthetic-looking
555
-port number, expanded port rows can expose a clickable neighbor actor when link
556
-facts allow it, and frontend v1 actor-link cells navigate again.
557
-
558
-## Regression - 2026-05-11 - SNMP Port Identity Alignment
559
-
560
-### What Broke
561
-
562
-- The previous regression repair introduced `actor_ports.port_number` and made
563
- it the visible `Ports` table identity.
564
-- The live SNMP payload has the real numeric device port identity in
565
- `actor_ports.if_index` and `actor_port_links.if_index`; it does not
566
- necessarily have a separate numeric `port_number` or numeric `port_id`.
567
-- Result: `Ports` showed empty port IDs while `Port Neighbors` showed correct
568
- non-empty port IDs.
569
-
570
-### Evidence
571
-
572
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` currently
573
- emits `port_number` as the first `Ports` modal column.
574
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` currently
575
- emits `if_index` as the first `Port Neighbors` modal column.
576
-- The user verified in the local UI that the `Ports` values are empty for the
577
- managed SNMP device, while `Port Neighbors` values are correct and non-empty.
578
-
579
-### Root Cause
580
-
581
-- The producer accidentally split one SNMP concept into two modal identities:
582
- `port_number` for `Ports`, and `if_index` for `Port Neighbors`.
583
-- For SNMP/L2, `ifIndex` is the device-provided numeric interface identifier
584
- and is the value the UI should show as the port ID. It is not a row-order
585
- autoincrement invented by Netdata.
586
-
587
-### Repair Plan
588
-
589
-- Remove the `actor_ports.port_number` column and helper logic.
590
-- Use `actor_ports.if_index` as the visible `Ports` `Port ID` column, matching
591
- `actor_port_links.if_index`.
592
-- Keep the no-synthetic-number rule: never derive port IDs from row order.
593
-- Update specs, developer guide, project skill, and tests to describe SNMP
594
- `if_index` as the visible real numeric port identity.
595
-
596
-### Implementation
597
-
598
-- Removed the `actor_ports.port_number` column and helper logic.
599
-- Restored `actor_ports.if_index` as the visible `Ports` `Port ID` column.
600
-- Kept `actor_port_links.if_index` unchanged, so `Ports` and `Port Neighbors`
601
- now use the same real SNMP numeric port identity.
602
-- Updated tests, topology specs, topology developer guide, and project topology
603
- skill to define `if_index` as the visible SNMP port ID when known.
604
-
605
-### Validation
606
-
607
-Acceptance criteria evidence:
608
-
609
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` emits
610
- `if_index` as the first `Ports` modal column and as the first
611
- `Port Neighbors` modal column.
612
-- `src/go/plugin/go.d/collector/snmp_topology/func_topology_test.go` verifies
613
- `actor_ports` has no `port_number` column and still exposes `if_index`.
614
-
615
-Tests or equivalent validation:
616
-
617
-- `cd src/go && go test -count=1 ./plugin/go.d/collector/snmp_topology ./pkg/topology/v1 ./tools/functions-validation/validate`
618
-- `sudo -n cmake --build build --target go.d.plugin -- -j2`
619
-
620
-Artifact maintenance gate:
621
-
622
-- `AGENTS.md`: unchanged. No project-wide workflow rule changed.
623
-- Runtime project skills: updated `.agents/skills/project-create-topology/SKILL.md`
624
- with SNMP `if_index` as the visible port ID and the no-generated-sequence rule.
625
-- Specs: updated `.agents/sow/specs/topology-function-schema.md` and
626
- `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
627
-- End-user/operator docs: unchanged. This remains developer-facing topology
628
- payload composition, not an operator workflow.
629
-- End-user/operator skills: unchanged. No public/operator skill semantics
630
- changed.
631
-- SOW lifecycle: reopened from `done/`, repaired, then returned to `done/` with
632
- `Status: completed`.
633
-
634
-Follow-up mapping:
635
-
636
-- No new deferred implementation work remains in this SOW.
637
-
638
-### Regression Outcome
639
-
640
-Completed. SNMP `Ports` and `Port Neighbors` now use the same real
641
-device-provided `if_index` value for the visible port ID, without any synthetic
642
-numbering.
.agents/sow/done/SOW-0028-20260511-topology-mode-correlation-aggregation.md
deleted
-531
@@ -1,531 +0,0 @@
1
-# SOW-0028 - Topology Mode Correlation Aggregation Contract
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-Sub-state: completed. The cross-repo compatibility contract is implemented and
8
-validated. True one-sided detailed network-connections graph rows are split to
9
-SOW-0029 because they require a separate Agent/UI/aggregator execution pass.
10
-
11
-## Requirements
12
-
13
-### Purpose
14
-
15
-Make topology maps and actor modals fit for SRE, DevOps, sysadmin, and network
16
-engineer workflows by defining one contract for detailed evidence, aggregated
17
-views, cross-payload correlation, table merging, and actor modal identification.
18
-
19
-### User Request
20
-
21
-The user asked to create the full specification first, then create SOW/TODO
22
-handoff artifacts for every affected repo, then implement the Agent, Cloud
23
-frontend, and Cloud topology aggregation service without relying on chat
24
-context.
25
-
26
-The user explicitly requested the spec to cover:
27
-
28
-- Agent detailed and aggregated views;
29
-- aggregator detailed and aggregated views while consuming detailed input;
30
-- UI detailed and aggregated views;
31
-- network-connections;
32
-- SNMP/L2;
33
-- streaming;
34
-- actor identification from selected labels in actor modals.
35
-
36
-### Assistant Understanding
37
-
38
-Facts:
39
-
40
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` already has
41
- `data.view.mode`, actor/link type presentation, modal recipes, actor labels,
42
- and a correlation section.
43
-- Current correlation documentation is centered on pure correlation actors and
44
- absorb/link actions.
45
-- The current network-connections modal analysis showed a broader contract gap:
46
- actor modal identification needs producer-selected labels, and aggregated
47
- network-connections needs relationship-summary rows for useful drilldowns.
48
-- SNMP and streaming do not currently have a meaningful detailed/aggregated
49
- mode split.
50
-
51
-Inferences:
52
-
53
-- The previous pure-correlation-actor model is not enough for
54
- network-connections detailed mode because exact remote `IP:PORT` tuples should
55
- often be loose relationship facts rather than actors.
56
-- SNMP/L2 needs replacement semantics, not loose-side socket resolution.
57
-- Streaming needs merge/enrichment semantics, not replacement.
58
-- The aggregator must consume detailed payloads before returning an aggregated
59
- view, otherwise cross-node correlation loses facts too early.
60
-
61
-Unknowns:
62
-
63
-- Exact final schema field names may need small adjustments during
64
- implementation to fit existing v1 schema style and frontend normalizer
65
- patterns.
66
-- The aggregator service may already implement a subset of SOW-0028 behavior;
67
- this must be verified against its current code before patching.
68
-
69
-### Acceptance Criteria
70
-
71
-- `.agents/sow/specs/topology-modes-correlation-aggregation.md` defines the
72
- full contract and examples for network-connections, SNMP/L2, and streaming.
73
-- This SOW records the cross-repo pending checklist for Agent, UI, and
74
- aggregator work.
75
-- A frontend TODO exists in the Cloud frontend repo and references the new spec.
76
-- A Cloud topology service SOW exists in the aggregator repo and references the
77
- new spec.
78
-- Agent schema and topology producers are updated to the new contract where the
79
- Agent owns the data.
80
-- Cloud frontend is updated to decode/render actor modal identification and the
81
- new mode/correlation semantics it must handle.
82
-- Cloud topology service is updated to aggregate according to the new contract.
83
-- Validation evidence records schema tests, relevant unit tests, and local
84
- Function/API checks that were possible.
85
-
86
-## Analysis
87
-
88
-Sources checked:
89
-
90
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
91
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
92
-- `.agents/sow/specs/topology-function-schema.md`
93
-- `.agents/skills/project-create-topology/SKILL.md`
94
-- `.agents/sow/done/SOW-0025-20260511-network-connections-modal-product-composition.md`
95
-- Cloud frontend `TODO-topology-modal-composition-contract.md`
96
-- Cloud topology service `AGENTS.md`
97
-- Cloud topology service `.agents/sow/specs/cloud-topology-service-contract.md`
98
-
99
-Current state:
100
-
101
-- `data.view.mode` can state `aggregated` or `detailed`, but the durable specs
102
- do not yet define layer-by-layer mode behavior.
103
-- `modal.labels` defines the label table shape, but cannot yet select important
104
- labels for the actor modal identification/header area.
105
-- Existing correlation docs assume pure correlation actors. The new model needs
106
- loose-side resolution for network-connections, replacement for SNMP/L2, and
107
- enrichment/table merging for streaming.
108
-- SOW-0025, SOW-0026, and SOW-0027 are narrower modal composition SOWs that
109
- depend on this broader contract.
110
-
111
-Risks:
112
-
113
-- A weak spec will recreate the same bug in three places: producer emits one
114
- meaning, aggregator merges another, UI renders a third.
115
-- Actor-per-`IP:PORT` detailed network-connections output can explode graph
116
- size and make the map unreadable.
117
-- Random conflict resolution in the aggregator can hide real dependencies.
118
-- Duplicating evidence rows for modal display can recreate the original payload
119
- size problem.
120
-- Actor labels may contain sensitive system metadata, users, command lines, or
121
- endpoint data. Durable artifacts must use synthetic examples only.
122
-
123
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
124
-
125
-Status at implementation start: ready (historical snapshot; final closure evidence is in the Execution Log and Validation sections).
126
-
127
-Problem / root-cause model:
128
-
129
-- The v1 topology contract optimized payload size and presentation tokens, but
130
- mode and correlation semantics were still incomplete. Network-connections,
131
- SNMP/L2, and streaming require different correlation outcomes: loose-side
132
- resolution, actor replacement, and actor enrichment.
133
-- Actor modals lost important identification because the schema exposes labels
134
- as a full table but does not let producers select the labels that belong in
135
- the modal header.
136
-
137
-Evidence reviewed:
138
-
139
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:179` defines `data.view.mode`.
140
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:469` defines current
141
- `data.correlation`.
142
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json:1097` defines current
143
- `modal_labels_presentation` without selected identification fields.
144
-- `.agents/sow/done/SOW-0025-20260511-network-connections-modal-product-composition.md`
145
- records the missing modal header label contract and the need for
146
- relationship-summary rows in aggregated network-connections.
147
-
148
-Affected contracts and surfaces:
149
-
150
-- Agent topology JSON schema and developer guide.
151
-- `topology:network-connections` producer.
152
-- `topology:snmp` producer/adapters, to ensure no false mode selector and to
153
- prepare replacement metadata.
154
-- `topology:streaming` producer, to ensure no false mode selector and to prepare
155
- merge/enrichment metadata.
156
-- Cloud frontend v1 normalizer and actor modal.
157
-- Cloud topology service aggregation core and request fanout.
158
-- Project topology skill and durable specs.
159
-
160
-Existing patterns to reuse:
161
-
162
-- Compact tables with `nullable` actor-ref columns.
163
-- Actor labels through `tables.actor.actor_labels`.
164
-- Type-level presentation under `types.actor_types.<id>.presentation` and
165
- `types.link_types.<id>.presentation`.
166
-- Modal sections over existing facts, not duplicate modal-only row stores.
167
-- Cloud topology service SOW framework and aggregator tests.
168
-- Cloud frontend v1 decoder/normalizer and modal table work.
169
-
170
-Risk and blast radius:
171
-
172
-- High semantic risk: wrong correlation can hide real dependencies or create
173
- false dependencies.
174
-- Medium payload risk: relationship-summary rows add payload size, but avoid
175
- much larger actor-per-socket graphs.
176
-- Medium UI risk: modal identification and loose-side materialization must not
177
- introduce topology-specific frontend code.
178
-- Low operational risk for Agent if schema validation and local Function output
179
- remain valid.
180
-
181
-Sensitive data handling plan:
182
-
183
-- Use only synthetic RFC 5737 IP ranges and synthetic host/process names in
184
- specs, SOWs, TODOs, tests, and examples.
185
-- Do not store raw Function captures, bearer tokens, cookies, usernames,
186
- command lines from local systems, SNMP communities, customer names, or
187
- customer-identifying public endpoints in durable artifacts.
188
-- Treat `actor_labels`, socket tuples, host labels, process command lines, and
189
- SNMP metadata as sensitive Function data.
190
-
191
-Implementation plan:
192
-
193
-1. Create the durable spec and cross-repo work items.
194
-2. Extend the Agent schema for modal label identification and mode/correlation
195
- metadata needed by the spec.
196
-3. Update topology developer docs and project topology skill.
197
-4. Update Agent producers:
198
- - network-connections mode and loose-side/relationship-summary semantics;
199
- - SNMP mode/correlation metadata where Agent owns it;
200
- - streaming merge/table metadata where Agent owns it.
201
-5. Update Cloud frontend to decode/render modal identification and supported
202
- mode/correlation metadata without domain-specific guesses.
203
-6. Update Cloud topology service to consume detailed inputs, correlate/merge by
204
- declared policies, and return detailed/aggregated outputs.
205
-7. Validate all touched repositories.
206
-
207
-Validation plan:
208
-
209
-- Agent: JSON schema validation, narrow C checks/build checks, and local
210
- Function payload checks for network-connections, SNMP, and streaming where
211
- available.
212
-- UI: unit tests for decoder/normalizer/modal identification and local build or
213
- focused test command where available.
214
-- Aggregator: Go tests for request fanout mode rewrite, loose-side resolution,
215
- SNMP replacement, streaming enrichment, and table merge policies.
216
-- Same-failure search for stale pure-correlation-only guidance.
217
-
218
-Artifact impact plan:
219
-
220
-- AGENTS.md: likely unchanged; existing SOW and public-skill boundary rules are
221
- already clear.
222
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md`
223
- so future topology work follows this spec.
224
-- Specs: add
225
- `.agents/sow/specs/topology-modes-correlation-aggregation.md` and update
226
- `.agents/sow/specs/topology-function-schema.md` if needed.
227
-- End-user/operator docs: likely unchanged because this is developer/internal
228
- topology schema work.
229
-- End-user/operator skills: unchanged unless a public querying skill documents
230
- developer validation, which must remain avoided.
231
-- SOW lifecycle: pause SOW-0025, keep SOW-0026 and SOW-0027 pending, create
232
- aggregator SOW and UI TODO, and map all remaining follow-ups before close.
233
-
234
-Open-source reference evidence:
235
-
236
-- Not checked for this gate. This contract is Netdata-specific and is defined
237
- by existing Agent, Cloud frontend, and Cloud topology service behavior.
238
-
239
-Open decisions:
240
-
241
-- No user decision is currently blocking the specification. The user already
242
- selected the key product direction: spec first, then durable repo work items,
243
- then implementation.
244
-
245
-## Implications And Decisions
246
-
247
-### Decision 1: Treat Actor Identification As Part Of This Contract
248
-
249
-Selection: implement in SOW-0028.
250
-
251
-Reasoning:
252
-
253
-- Actor identification affects schema, producer payloads, UI rendering, and
254
- aggregator preservation.
255
-- Keeping it separate would leave actor modals visually incomplete even if mode
256
- and correlation semantics are fixed.
257
-
258
-### Decision 2: Aggregator Consumes Detailed Input
259
-
260
-Selection: Cloud aggregator rewrites `__topology_mode=aggregated` fanout to
261
-`__topology_mode=detailed` only for producers that support the mode.
262
-
263
-Reasoning:
264
-
265
-- Exact cross-node socket correlation needs detailed tuples.
266
-- Aggregating before correlation loses facts that cannot be recovered.
267
-- SNMP/L2 and streaming currently do not have meaningful detailed/aggregated
268
- producer modes, so the aggregator should not send an invented parameter.
269
-
270
-### Decision 3: Network-Connections Detailed Uses Loose Sides
271
-
272
-Selection: known actors remain actors; unknown remote socket peers are
273
-loose-side facts until the aggregator or UI materializes them according to the
274
-schema.
275
-
276
-Reasoning:
277
-
278
-- Actor-per-`IP:PORT` detailed output can explode graph cardinality.
279
-- Exact tuples are still preserved for matching and drilldown.
280
-
281
-### Decision 4: SNMP Uses Replacement, Streaming Uses Enrichment
282
-
283
-Selection:
284
-
285
-- SNMP/L2 aggregation replaces weaker placeholder actors with stronger managed
286
- actors.
287
-- Streaming aggregation merges/enriches actors by `machine_guid` and table
288
- policy.
289
-
290
-Reasoning:
291
-
292
-- SNMP placeholder actors and managed devices represent the same physical
293
- entity at different confidence levels.
294
-- Streaming payloads from multiple parents may contain complementary facts for
295
- the same node; losing either side is wrong.
296
-
297
-## Cross-Repo Pending Checklist
298
-
299
-Agent repository:
300
-
301
-- Add/maintain the spec under `.agents/sow/specs/`.
302
-- Update `FUNCTION_TOPOLOGY_SCHEMA.json`.
303
-- Update `FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`.
304
-- Update `.agents/sow/specs/topology-function-schema.md` as needed.
305
-- Update `.agents/skills/project-create-topology/SKILL.md`.
306
-- Implement producer changes for network-connections, SNMP/L2, and streaming
307
- as far as the Agent owns the emitted payload.
308
-
309
-Cloud frontend repository:
310
-
311
-- Create `TODO-topology-mode-correlation-aggregation.md`.
312
-- Implement modal label identification rendering.
313
-- Implement mode capability behavior and hide no-op mode toggles.
314
-- Preserve and render loose-side/materialization metadata without
315
- domain-specific guesses.
316
-- Keep old-schema support isolated.
317
-
318
-Cloud topology service repository:
319
-
320
-- Create a new current SOW for topology mode, loose-side resolution,
321
- replacement, enrichment, and table merge policy.
322
-- Implement request fanout mode rewrite.
323
-- Implement generic rule classes and table merge policies.
324
-- Add fixtures/tests for network-connections, SNMP/L2, and streaming.
325
-
326
-## Plan
327
-
328
-1. Write the spec and create cross-repo work items.
329
-2. Patch Agent schema/docs/skill.
330
-3. Patch Agent producers and validate local payloads.
331
-4. Patch Cloud frontend and run focused tests.
332
-5. Patch Cloud topology service and run Go tests.
333
-6. Update SOW validation and follow-up mapping.
334
-
335
-## Execution Log
336
-
337
-### 2026-05-11
338
-
339
-- Created the cross-topology mode/correlation/aggregation spec.
340
-- Paused SOW-0025 because it depends on this broader contract.
341
-- Created Cloud frontend handoff TODO:
342
- `<cloud-frontend-repo>/TODO-topology-mode-correlation-aggregation.md`.
343
-- Created and completed Cloud topology service SOW:
344
- `<cloud-topology-service-repo>/.agents/sow/done/SOW-0011-20260511-topology-mode-correlation-aggregation.md`.
345
-- Updated Agent schema, developer guide, project topology skill, Go topology
346
- structs/validation, network-connections producer, SNMP topology tests, and
347
- streaming actor modal identification.
348
-- Implemented UI support for modal label identification and hiding
349
- `__topology_mode` controls when `data.view.supported_modes` does not
350
- advertise a real split.
351
-- Implemented Cloud topology service compatibility for supported modes, modal
352
- label identification, correlation rule classes, fanout mode rewrite helpers,
353
- and detail-table dedupe/merge policies.
354
-
355
-## Validation
356
-
357
-Acceptance criteria evidence:
358
-
359
-- Spec:
360
- - `.agents/sow/specs/topology-modes-correlation-aggregation.md` defines the
361
- layer responsibilities and examples for network-connections, SNMP/L2, and
362
- streaming.
363
-- Agent schema/docs:
364
- - `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` includes
365
- `data.view.supported_modes`, modal label identification, port source
366
- `value_column`, correlation rule `class`, and expanded table aggregation
367
- tokens.
368
- - `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md` documents the new
369
- fields.
370
- - `.agents/skills/project-create-topology/SKILL.md` was updated for the
371
- developer workflow.
372
-- Agent producers:
373
- - `src/collectors/network-viewer.plugin/network-viewer.c` accepts
374
- `__topology_mode`, advertises supported modes, emits selected actor modal
375
- identification labels, emits relationship-summary rows for aggregated
376
- connections, and marks socket correlation rules with
377
- `class: resolve_loose_side`.
378
- - `src/web/api/functions/function-topology-streaming.c` emits selected actor
379
- modal identification labels and remains mode-invariant.
380
- - `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go` remains
381
- mode-invariant and is covered by updated tests.
382
-- Cloud frontend:
383
- - `src/domains/functions/topology/v1/buildModalPresentation.js` decodes
384
- modal label identification.
385
- - `src/domains/functions/components/topology/actorModal/index.js` renders
386
- selected v1 identification labels in the actor modal header.
387
- - `src/domains/functions/useFetch/normalizers/topology/index.js` hides
388
- `__topology_mode` unless `supported_modes` advertises both modes.
389
-- Cloud topology service:
390
- - `internal/topology/schema/payload.go`, `internal/topology/validate/validate.go`,
391
- `internal/topology/mode/mode.go`, and
392
- `internal/topology/aggregate/aggregate.go` implement the service-side
393
- compatibility layer.
394
-
395
-Tests or equivalent validation:
396
-
397
-- Agent:
398
- - `git diff --check` passed.
399
- - `go test ./pkg/topology/v1 ./plugin/go.d/collector/snmp_topology` passed
400
- from `src/go`.
401
- - `sudo -n cmake --build build --target network-viewer.plugin -- -j2`
402
- passed after non-sudo build was blocked by existing build-directory
403
- permissions.
404
- - `.agents/sow/audit.sh` passed with existing non-project-skill
405
- classification warnings.
406
-- UI:
407
- - `git diff --check` passed.
408
- - `yarn test src/domains/functions/topology/v1/buildModalPresentation.test.js src/domains/functions/useFetch/normalizers/topology/index.test.js --runInBand`
409
- passed.
410
-- Cloud topology service:
411
- - `go test ./internal/topology/...` passed.
412
- - `go test ./...` passed.
413
- - `go vet ./internal/topology/...` passed.
414
- - `git diff --check` passed.
415
- - `.agents/sow/audit.sh` passed.
416
-
417
-Real-use evidence:
418
-
419
-- Installed Function checks through token-safe Cloud-proxied Agent calls passed
420
- on 2026-05-11:
421
- - `topology:network-connections ... mode:aggregated` returned status 200,
422
- `view.mode: aggregated`, `view.supported_modes: [aggregated, detailed]`,
423
- actor rows, link rows, `actor_labels`, `socket_ports`, relationship table
424
- `connections`, and `data.correlation`.
425
- - `topology:network-connections ... mode:detailed` returned status 200,
426
- `view.mode: detailed`, the same supported modes, graph rows, socket
427
- evidence rows, and `data.correlation`.
428
- - `topology:streaming` returned status 200, no `supported_modes`, and modal
429
- identification metadata for its actor types.
430
- - `topology:snmp` returned status 200 and no `supported_modes`, so the UI
431
- must not show a fake detailed/aggregated toggle. SNMP-specific modal
432
- identification polish remains tracked by SOW-0026.
433
-
434
-Reviewer findings:
435
-
436
-- No external reviewer loop was run for this SOW pass. The user asked for
437
- implementation continuity after the spec/SOW/TODO artifacts were created.
438
-
439
-Same-failure scan:
440
-
441
-- `rg` checks were used across Agent, UI, and Cloud topology service for the
442
- new contract fields: `supported_modes`, `identification`, `class`,
443
- `value_column`, `merge_metrics`, `set_union`, `__topology_mode`, and
444
- `RewriteFunctionCall`.
445
-
446
-Sensitive data gate:
447
-
448
-- Passed by inspection for this pass. Durable artifacts contain synthetic
449
- examples only; no raw Function captures, bearer tokens, cookies, usernames,
450
- command lines from local systems, SNMP communities, customer names,
451
- customer-identifying public endpoints, private endpoints, raw node IDs, or
452
- raw machine GUIDs were written.
453
-
454
-Artifact maintenance gate:
455
-
456
-- AGENTS.md: unchanged; existing SOW, public-skill boundary, and artifact
457
- maintenance rules already cover this workflow.
458
-- Runtime project skills: updated
459
- `.agents/skills/project-create-topology/SKILL.md`.
460
-- Specs: added
461
- `.agents/sow/specs/topology-modes-correlation-aggregation.md` and updated
462
- `.agents/sow/specs/topology-function-schema.md`.
463
-- End-user/operator docs: unaffected; this is an internal developer topology
464
- schema/producer/service/UI contract.
465
-- End-user/operator skills: unaffected; no operator querying skill was changed.
466
-- SOW lifecycle: SOW-0025 remains paused; SOW-0026 and SOW-0027 remain pending;
467
- SOW-0029 tracks true detailed loose-side graph rows; SOW-0028 is completed
468
- and moved to `done/`.
469
-
470
-Specs update:
471
-
472
-- Added `.agents/sow/specs/topology-modes-correlation-aggregation.md`.
473
-
474
-Project skills update:
475
-
476
-- Updated `.agents/skills/project-create-topology/SKILL.md`.
477
-
478
-End-user/operator docs update:
479
-
480
-- Not affected.
481
-
482
-End-user/operator skills update:
483
-
484
-- Not affected.
485
-
486
-Lessons and follow-up mapping:
487
-
488
-- Completed in the final Lessons Extracted and Followup sections below.
489
-
490
-## Outcome
491
-
492
-Implementation checkpoint is complete across Agent, Cloud frontend, and Cloud
493
-topology service for the documented schema compatibility layer:
494
-
495
-- schema/docs/specs/skill were updated first;
496
-- Agent producers and topology validation were updated;
497
-- UI modal identification and mode-control behavior were updated;
498
-- Cloud topology service schema/validation/mode/table-policy compatibility was
499
- updated;
500
-- focused validation passed in all three repositories.
501
-
502
-The SOW is complete for the implemented cross-repo compatibility contract.
503
-Remaining one-sided detailed network-connections graph rows are explicitly
504
-split to SOW-0029.
505
-
506
-## Lessons Extracted
507
-
508
-- The durable spec/SOW/TODO order worked: after compaction, the repo artifacts
509
- were enough to recover the intended cross-repo work without relying on chat
510
- memory.
511
-- Loose-side rows are the hardest part of the model because they affect schema
512
- shape, UI graph materialization, and aggregation execution together. The
513
- current checkpoint preserves exact socket facts and correlation metadata but
514
- does not yet switch the Agent producer to one-sided detailed graph rows.
515
-
516
-## Followup
517
-
518
-- True detailed one-sided network-connections graph rows and UI
519
- materialization are tracked by
520
- `.agents/sow/pending/SOW-0029-20260511-network-connections-detailed-loose-sides.md`.
521
- The current Agent producer still emits endpoint actors for local direct
522
- views while preserving exact socket evidence for correlation.
523
-- Cloud topology service production fanout/fetch integration remains tracked by
524
- its service handoff gates; SOW-0028 implemented the internal mode rewrite
525
- helper and aggregation compatibility layer.
526
-- SOW-0025, SOW-0026, and SOW-0027 remain the function-specific modal product
527
- polish work for network-connections, SNMP/L2, and streaming.
528
-
529
-## Regression Log
530
-
531
-None yet.
.agents/sow/done/SOW-0031-20260517-topology-v1-zero-heuristic-rendering-contract.md
deleted
-428
@@ -1,428 +0,0 @@
1
-# SOW-0031 - Topology V1 Zero-Heuristic Rendering Contract
2
-
3
-## Status
4
-
5
-Status: completed
6
-
7
-`completed` is the successful terminal status. `done` is a directory name, not a status value. Do not use `Status: done` or `Status: complete`.
8
-
9
-Sub-state: implemented, validated, installed by the user, and committed across the Agent, Cloud frontend, and Cloud topology service repositories.
10
-
11
-## Requirements
12
-
13
-### Purpose
14
-
15
-Make `netdata.topology.v1` fit for polished, topology-agnostic graph rendering. The UI must not hardcode domain words such as self, segment, endpoint, device, SNMP, LLDP, CDP, parent, child, client, server, router, or switch when rendering v1 payloads.
16
-
17
-### User Request
18
-
19
-Create a SOW, analyze the remaining frontend heuristics, propose the missing schema fields, update local schema/spec/docs/skills, create Cloud aggregator and Cloud frontend handoff artifacts, then implement the approved zero-heuristic contract in the Agent producers/shared helpers, Cloud frontend, and Cloud topology service.
20
-
21
-### Assistant Understanding
22
-
23
-Facts:
24
-
25
-- The current v1 schema already carries actor/link/port presentation, modal recipes, legend, highlight behavior, port-bullet sources, and link layout distance/strength.
26
-- The Cloud frontend report identified remaining heuristics outside the v1 decoder/modal path: self detection, segment/endpoint/device/inferred detection, SNMP/LLDP/CDP detection, hardcoded search paths, capability-to-icon fallback, and v1-to-legacy `protocol`/port shims.
27
-- Raw SVG icons are explicitly not allowed by the current topology documentation and should remain disallowed.
28
-
29
-Inferences:
30
-
31
-- The missing contract is not more producer-specific fields. The missing contract is a small set of topology-agnostic type-level policies that let the UI treat v1 as data-driven.
32
-- Link behavior classification belongs on `link_types.<id>.semantic_role`, not under `presentation`, because discovery, ownership, traffic, correlation, and control are graph semantics. Visual appearance remains under `presentation`.
33
-
34
-Unknowns:
35
-
36
-- Exact numeric UI mappings for `size.scale` and `layout.repulsion` are UI-owned and must be tuned visually after implementation.
37
-
38
-### Acceptance Criteria
39
-
40
-- Agent repo has an active SOW with the zero-heuristic contract and implementation boundary.
41
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` defines the proposed optional schema fields without requiring producer migration immediately.
42
-- Local topology spec, developer guide, implementation scope, and project topology skill document the new contract.
43
-- Cloud frontend implements v1 renderer behavior from the TODO while preserving legacy behavior in the legacy path.
44
-- Cloud topology service decodes, preserves, namespaces/deduplicates, and emits the new fields.
45
-- Agent shared topology helpers and producers emit the new fields for network-connections, SNMP/L2, and streaming.
46
-- Validation covers Agent schema/producer fixtures, Cloud frontend tests, Cloud topology service tests, and real local topology payload smoke checks where practical.
47
-
48
-## Analysis
49
-
50
-Sources checked:
51
-
52
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
53
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
54
-- `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`
55
-- `.agents/sow/specs/topology-function-schema.md`
56
-- `.agents/sow/specs/topology-modes-correlation-aggregation.md`
57
-- `.agents/skills/project-create-topology/SKILL.md`
58
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/topology/utils.js`
59
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/components/graph/forceGraph.js`
60
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/components/graph/useForceSimulation.js`
61
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/components/topology/actorModal/portTable.js`
62
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/topology/v1/buildLinks.js`
63
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/topology/v1/buildRenderableLinks.js`
64
-
65
-Current state:
66
-
67
-- Actor presentation exists, but actor size only has `mode` and optional `metric_column`; it lacks a type-level fixed scale token.
68
-- Link presentation has `layout.strength` and `layout.distance`; actor presentation lacks a corresponding repulsion token.
69
-- Link type has `direction_role`, but no generic semantic classification for discovery, ownership, traffic, correlation, or control behavior.
70
-- Actor type has no search contract, so the frontend still indexes hardcoded paths from legacy `details.match`, `details.attributes`, and `details.labels`.
71
-- Icon tokens are closed, which is correct, but producers cannot fully replace capability-based frontend icon inference unless every v1 actor type declares a suitable token.
72
-
73
-Risks:
74
-
75
-- If the UI keeps heuristics in v1, every new topology can regress visually when its actor/link names do not match existing frontend guesses.
76
-- If semantics are placed under `presentation`, the aggregator may need to parse a visual object to make graph-behavior decisions.
77
-- If raw SVG is allowed, topology payloads become a new script/rendering attack surface and every consumer must sanitize untrusted markup.
78
-- If repulsion reuses link strength without a separate field name, producers and UI developers will conflate two different force-graph quantities.
79
-
80
-## Proposed Schema
81
-
82
-### Actor Type Search
83
-
84
-```json
85
-{
86
- "types": {
87
- "actor_types": {
88
- "process": {
89
- "search": {
90
- "enabled": true,
91
- "columns": ["display_name", "process_name"],
92
- "label_keys": ["cmdline", "username"]
93
- }
94
- }
95
- }
96
- }
97
-}
98
-```
99
-
100
-Rules:
101
-
102
-- `search.columns[]` references actor-table scalar columns.
103
-- `search.label_keys[]` references `actor_labels.key` values.
104
-- `search.enabled: false` removes helper actors from graph search.
105
-- UI must not traverse producer-specific `details`, `match`, `attributes`, or label paths for v1 search.
106
-
107
-### Actor Presentation Size And Repulsion
108
-
109
-```json
110
-{
111
- "presentation": {
112
- "size": {
113
- "mode": "metric",
114
- "metric_column": "socket_count",
115
- "scale": "normal"
116
- },
117
- "layout": {
118
- "repulsion": "normal"
119
- }
120
- }
121
-}
122
-```
123
-
124
-Rules:
125
-
126
-- `size.scale` is `compact`, `normal`, or `emphasized`.
127
-- `layout.repulsion` is `weakest`, `weaker`, `normal`, `stronger`, or `strongest`.
128
-- Producers emit tokens only. The UI owns numeric radius and charge mappings.
129
-- Actor repulsion is separate from link strength.
130
-- `size.scale` composes with `size.mode`.
131
-- Missing `size.scale` and missing `layout.repulsion` use `normal`; the UI must
132
- not fall back to self/device/SNMP/endpoint heuristics for v1.
133
-- Initial UI-owned mappings are `compact=0.85`, `normal=1.0`,
134
- `emphasized=1.18` for size scale and `weakest=-200`, `weaker=-300`,
135
- `normal=-450`, `stronger=-700`, `strongest=-1000` for repulsion. These
136
- numbers are not schema.
137
-
138
-### Link Semantic Role
139
-
140
-```json
141
-{
142
- "types": {
143
- "link_types": {
144
- "lldp": {
145
- "orientation": "observed_bidirectional",
146
- "direction_role": "observation",
147
- "semantic_role": "discovery",
148
- "aggregation": {
149
- "direction": "canonicalize_unordered",
150
- "evidence": "append"
151
- }
152
- }
153
- }
154
- }
155
-}
156
-```
157
-
158
-Allowed roles:
159
-
160
-- `normal`
161
-- `discovery`
162
-- `ownership`
163
-- `traffic`
164
-- `correlation`
165
-- `control`
166
-
167
-Rules:
168
-
169
-- `semantic_role` drives behavior such as discovery filtering, ownership/coherence handling, traffic emphasis, and correlation treatment.
170
-- Link appearance still comes from `presentation`.
171
-- UI and aggregator must not infer role from `link.type`, `link.protocol`, LLDP/CDP string checks, or label names.
172
-- Day-1 UI behavior only requires a concrete behavior difference for
173
- `semantic_role: discovery`. Other semantic roles are preserved and render
174
- through presentation until future behavior is specified.
175
-
176
-### Arrow Auto
177
-
178
-`presentation.arrow` remains the authoritative visual signal. When it is
179
-`auto` or omitted, the UI derives arrows from `orientation` and
180
-`direction_role`:
181
-
182
-- `undirected` -> no arrow;
183
-- `observed_bidirectional` -> no arrow;
184
-- `direction_role: none` -> no arrow;
185
-- `direction_role: observation` -> no arrow;
186
-- `directed` with `flow` or `dependency` -> forward from `src_actor` to
187
- `dst_actor`;
188
-- `hierarchical` with `ownership` -> forward from `src_actor` to `dst_actor`;
189
-- all other combinations -> no arrow and a diagnostic if the combination is
190
- schema-valid but semantically unusual.
191
-
192
-`observed_bidirectional` does not mean draw arrows at both ends. Producers must
193
-set `presentation.arrow: "both"` or `"reverse"` explicitly when needed.
194
-
195
-`direction_role` is required by the v1 schema. Missing `direction_role` is
196
-invalid input and must not produce an inferred arrow from `orientation:
197
-"directed"` alone. The UI should render `auto` as no arrow and emit the normal
198
-missing/invalid-field diagnostic.
199
-
200
-For schema-valid values, the semantic diagnostic boundary is explicit:
201
-
202
-- no diagnostic for `directed+flow`, `directed+dependency`,
203
- `hierarchical+ownership`, `undirected+none`, `undirected+observation`,
204
- `observed_bidirectional+none`, or `observed_bidirectional+observation`;
205
-- diagnostic for `directed+none`, `directed+observation`,
206
- `directed+ownership`, `hierarchical+none`, `hierarchical+flow`,
207
- `hierarchical+dependency`, `hierarchical+observation`, `undirected+flow`,
208
- `undirected+dependency`, `undirected+ownership`,
209
- `observed_bidirectional+flow`, `observed_bidirectional+dependency`, or
210
- `observed_bidirectional+ownership`.
211
-
212
-### Closed Icon Tokens Only
213
-
214
-Allowed icons remain schema-owned tokens. This SOW adds generic tokens needed to remove capability inference:
215
-
216
-- `device`
217
-- `endpoint`
218
-- `correlation`
219
-- `interface`
220
-- `group`
221
-- `unknown`
222
-
223
-Rules:
224
-
225
-- Raw SVG remains disallowed.
226
-- Capability-to-icon inference moves to producers, where producer-specific capabilities are already known.
227
-- UI maps icon tokens to safe bundled icons only.
228
-
229
-## Pre-Implementation Gate (Historical Snapshot at Implementation Start)
230
-
231
-Status: approved
232
-
233
-Problem / root-cause model:
234
-
235
-- The v1 payload is mostly schema-driven, but renderer behavior still depends on legacy frontend heuristics. The root cause is missing v1 contract fields for actor search, actor fixed emphasis, actor repulsion, and link semantic behavior.
236
-
237
-Evidence reviewed:
238
-
239
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` already defines `actor_type.presentation`, `link_type.presentation`, and link `presentation.layout`.
240
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` had no actor `search`, no actor `presentation.layout`, no `size.scale`, and no link `semantic_role` before this SOW.
241
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/topology/utils.js` contains actor/link kind heuristics such as self, segment, endpoint/derived, inferred, device, SNMP, LLDP, and CDP detection.
242
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/components/graph/forceGraph.js` contains hardcoded graph search path extraction and icon fallback from capability inference.
243
-- `${CLOUD_FRONTEND_REPO}/src/domains/functions/topology/v1/buildLinks.js` mirrors v1 row fields into legacy `protocol`, `sourcePort`, and `targetPort` properties only for legacy renderer paths.
244
-
245
-Affected contracts and surfaces:
246
-
247
-- Agent topology JSON schema.
248
-- Agent topology developer guide.
249
-- Agent topology durable spec.
250
-- Agent project topology skill.
251
-- Cloud frontend v1 decoder, renderer, search, force simulation, legend, icon mapping, port table, and legacy adapter.
252
-- Cloud topology service type registry decode, aggregation merge, type namespace/dedup, and returned schema preservation.
253
-- Producer migrations for network-connections, SNMP/L2, streaming, and future vSphere v1.
254
-
255
-Existing patterns to reuse:
256
-
257
-- Closed token pattern from color, opacity, width, icon, link layout strength, and link layout distance.
258
-- Existing type registry pattern for actor/link/port type presentation.
259
-- Existing frontend diagnostics pattern for unknown tokens.
260
-- Existing Cloud aggregator namespacing/dedup behavior for type definitions.
261
-
262
-Risk and blast radius:
263
-
264
-- UI renderer refactor has high visual regression risk because it touches the graph hot path.
265
-- Producer migration is broad but optional fields allow incremental rollout.
266
-- Aggregator should preserve and merge the new fields; it must not invent type semantics.
267
-- Raw SVG remains rejected to avoid new untrusted markup risk.
268
-
269
-Sensitive data handling plan:
270
-
271
-- This planning pass uses only schema/docs/code paths and sanitized descriptions. Durable artifacts must not include raw payload captures, credentials, cookies, bearer tokens, SNMP communities, customer names, personal data, non-private customer-identifying IPs, or private endpoints.
272
-
273
-Implementation plan:
274
-
275
-1. Update shared Go topology structs and validators for `actor.search`, `actor.presentation.size.scale`, `actor.presentation.layout.repulsion`, and `link.semantic_role`.
276
-2. Update each Agent producer to emit the new optional fields for every v1 actor/link type it owns.
277
-3. Implement Cloud frontend behavior using the TODO created by this SOW.
278
-4. Implement Cloud topology service behavior using the SOW created by this SOW.
279
-5. Validate with real topology payloads across network-connections, SNMP/L2, and streaming.
280
-
281
-Validation plan:
282
-
283
-- Validate JSON schema syntax with `jq`.
284
-- Validate SOW status/directory consistency with project audit.
285
-- During implementation, add schema validation fixtures and UI/aggregator tests that prove v1 rendering does not call legacy heuristics.
286
-
287
-Artifact impact plan:
288
-
289
-- AGENTS.md: no update expected; this does not change project-wide workflow.
290
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md`.
291
-- Specs: update `.agents/sow/specs/topology-function-schema.md`.
292
-- End-user/operator docs: no update expected; this is developer/schema contract work.
293
-- End-user/operator skills: no update expected; this is not an operator workflow.
294
-- SOW lifecycle: SOW-0031 is the active implementation ledger for this approved cross-repo work.
295
-
296
-Open-source reference evidence:
297
-
298
-- None checked. This is a local Netdata schema/frontend/aggregator contract gap, and the user requested local contract handoff rather than external product research.
299
-
300
-Open decisions:
301
-
302
-- None. The user approved implementation after the frontend TODO follow-up clarifications were recorded.
303
-
304
-## Implications And Decisions
305
-
306
-1. User accepted adding a schema contract to remove frontend v1 heuristics.
307
-2. User accepted keeping raw SVG out of the payload.
308
-3. User accepted adding a dedicated SOW/TODO handoff for Cloud frontend and Cloud aggregator before implementation.
309
-
310
-## Plan
311
-
312
-1. Record the zero-heuristic schema contract in local Agent schema/spec/docs/skill artifacts.
313
-2. Create a Cloud frontend TODO that asks the UI agent to remove v1 renderer heuristics only after schema support lands.
314
-3. Create a Cloud topology service SOW that asks the aggregator agent to decode, preserve, namespace, deduplicate, and emit the new optional fields without inventing producer semantics.
315
-4. Implement the approved contract in Agent, Cloud frontend, and Cloud topology service.
316
-
317
-## Execution Log
318
-
319
-### 2026-05-17
320
-
321
-- Created this SOW.
322
-- Added optional JSON schema fields for actor search, actor size scale, actor layout repulsion, link semantic role, and generic closed icon tokens.
323
-- Updated local topology spec, developer guide, implementation scope, and project topology skill.
324
-- Created Cloud frontend and Cloud topology service handoff artifacts.
325
-- Appended backend answers to frontend follow-up questions: arrow auto mapping,
326
- direction_role UI usage, initial UI-owned numeric mappings, transitional
327
- neutral defaults, and current aggregator unknown-field behavior.
328
-- Appended final `arrow: auto` clarifications covering required
329
- `direction_role`, current producer status, and the semantic diagnostic
330
- boundary.
331
-- User approved implementation after the Cloud frontend TODO review loop.
332
-
333
-## Validation
334
-
335
-Acceptance criteria evidence:
336
-
337
-- SOW completed at `.agents/sow/done/SOW-0031-20260517-topology-v1-zero-heuristic-rendering-contract.md`.
338
-- JSON schema updated at `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
339
-- Spec/docs/skill updates are present in `.agents/sow/specs/topology-function-schema.md`, `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`, `src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md`, and `.agents/skills/project-create-topology/SKILL.md`.
340
-- Cloud frontend TODO created as `${CLOUD_FRONTEND_REPO}/TODO-topology-v1-zero-heuristic-rendering-contract.md`.
341
-- Cloud topology service SOW completed as `${CLOUD_TOPOLOGY_SERVICE_REPO}/.agents/sow/done/SOW-0017-20260517-topology-v1-zero-heuristic-contract.md`.
342
-- Agent topology structs and validators implement actor search, actor size scale, actor layout repulsion, link semantic role, and closed icon tokens in `src/go/pkg/topology/v1/types.go` and `src/go/pkg/topology/v1/validate.go`.
343
-- Agent producers emit the contract fields in `src/collectors/network-viewer.plugin/network-viewer.c`, `src/go/plugin/go.d/collector/snmp_topology/func_topology_v1.go`, and `src/web/api/functions/function-topology-streaming.c`.
344
-- Cloud frontend v1 rendering consumes schema-driven search, size, repulsion, semantic role, and port lookup fields in `${CLOUD_FRONTEND_REPO}`.
345
-- Cloud topology service decodes, validates, preserves, and namespaces conflicting definitions for the new fields in `${CLOUD_TOPOLOGY_SERVICE_REPO}`.
346
-
347
-Tests or equivalent validation:
348
-
349
-- `jq empty src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` passed.
350
-- Agent repo `.agents/sow/audit.sh` accepted SOW-0031 status/directory placement and reported no sensitive-data findings. The audit still reports unrelated pre-existing framework warnings: one older done SOW has a status/directory mismatch, and legacy non-project skill directories need classification.
351
-- Cloud topology service `.agents/sow/audit.sh` passed cleanly after creating SOW-0017.
352
-- `go test ./pkg/topology/v1 ./plugin/go.d/collector/snmp_topology` passed from `src/go`.
353
-- `git diff --check` passed in the Agent repo.
354
-- `git diff --check` passed in the Cloud frontend repo.
355
-- `git diff --check` passed in the Cloud topology service repo.
356
-- Cloud frontend focused tests passed for v1 normalization, presentation adapter, renderable links, color token handling, port utilities, and force simulation helpers.
357
-- Cloud topology service tests passed: `go test ./internal/topology/schema ./internal/topology/validate ./internal/topology/aggregate`.
358
-- Full Cloud frontend test suite passed: 269 suites passed, 2,139 tests passed, 6 skipped, 7 snapshots passed.
359
-- Agent C build validation is blocked by local filesystem permissions: `build`, `build/.ninja_log`, and `build/.ninja_deps` are owned by `root:root`; `ninja -C build` cannot create `build/.ninja_lock` in this worktree.
360
-- User later installed the Agent and UI successfully from the updated worktrees.
361
-
362
-Real-use evidence:
363
-
364
-- User reported the Agent and UI were installed and running after implementation.
365
-
366
-Reviewer findings:
367
-
368
-- Frontend agent report is the input to this SOW. Additional external review can be run after implementation validation if requested.
369
-
370
-Same-failure scan:
371
-
372
-- `rg` verified the new contract terms appear in the schema, developer guide, implementation scope, durable spec, project skill, and SOW.
373
-
374
-Sensitive data gate:
375
-
376
-- Durable artifacts use only schema names, file paths, sanitized repo placeholders, and generic examples. No raw secrets, credentials, bearer tokens, SNMP communities, personal names, customer identifiers, non-private customer-identifying IPs, private endpoints, or proprietary incidents are included.
377
-
378
-Artifact maintenance gate:
379
-
380
-- AGENTS.md: not changed; no workflow or project-wide guardrail changed.
381
-- Runtime project skills: `.agents/skills/project-create-topology/SKILL.md` updated.
382
-- Specs: `.agents/sow/specs/topology-function-schema.md` updated.
383
-- End-user/operator docs: not affected; this is developer schema work.
384
-- End-user/operator skills: not affected; no operator workflow changed.
385
-- SOW lifecycle: SOW is `completed` in `done/`; implementation and lifecycle closure are committed in follow-up commits because the first implementation commit accidentally left the SOWs in progress.
386
-
387
-Specs update:
388
-
389
-- `.agents/sow/specs/topology-function-schema.md` updated.
390
-
391
-Project skills update:
392
-
393
-- `.agents/skills/project-create-topology/SKILL.md` updated.
394
-
395
-End-user/operator docs update:
396
-
397
-- Not affected. The change is internal topology producer/UI/aggregator contract behavior.
398
-
399
-End-user/operator skills update:
400
-
401
-- Not affected. Public query skills do not teach topology producer development or frontend renderer internals.
402
-
403
-Lessons:
404
-
405
-- v1 schemas need behavior contracts for renderer decisions, not only visual tokens. Otherwise legacy UI heuristics leak into new topology types.
406
-
407
-Follow-up mapping:
408
-
409
-- Cloud frontend implementation is committed in `${CLOUD_FRONTEND_REPO}`.
410
-- Cloud topology service implementation is committed in `${CLOUD_TOPOLOGY_SERVICE_REPO}`.
411
-- No remaining SOW follow-up is open for this contract.
412
-
413
-## Outcome
414
-
415
-Completed. Core Agent, Cloud frontend, and Cloud topology service code paths are implemented, validated, installed by the user, and committed.
416
-
417
-## Lessons Extracted
418
-
419
-- The v1 renderer cannot become topology-agnostic from presentation colors alone. It needs producer-owned behavior tokens for search, discovery behavior, actor sizing, and graph-layout repulsion.
420
-- Aggregators must namespace contradictory type definitions instead of inventing precedence rules for producer-owned presentation or behavior metadata.
421
-
422
-## Followup
423
-
424
-- None.
425
-
426
-## Regression Log
427
-
428
-None yet.
.agents/sow/pending/.gitkeep
.agents/sow/pending/SOW-0002-20260501-unified-multi-layered-topology-schema.md
deleted
-349
@@ -1,349 +0,0 @@
1
-# SOW-0002 - Unified multi-layered topology schema and merge engine
2
-
3
-## Status
4
-
5
-Status: open
6
-
7
-Sub-state: scope captured, awaiting user decisions on merge semantics,
8
-cross-layer identity-matching policy, conflict resolution, storage model, and
9
-scale targets before implementation planning advances. Immediate
10
-detailed/aggregated topology Function payload migration is owned by SOW-0020.
11
-Same-kind producer-visible correlation points/claims are owned by SOW-0023.
12
-This SOW remains the future unified cross-layer merge/correlation work.
13
-
14
-## Requirements
15
-
16
-### Purpose
17
-
18
-Make Netdata produce a single coherent topology view that unifies every source contributing topology evidence — at the same layer (e.g. multiple L2-speaking switches) and across layers (L2 ↔ L3 ↔ L7) — so users see one map of their network/infrastructure rather than four disconnected maps. This is foundational work: topology in Netdata is new, and the choice of merge semantics will shape every downstream consumer (UI, alerts, automation, AI agents).
19
-
20
-The work is fit-for-purpose for production observability on large enterprise networks where a single physical device routinely shows up under multiple identities at different layers (e.g. a server has a hostname, a Netdata machine GUID, several MAC addresses, multiple IP addresses, container IDs, Kubernetes pod/namespace identifiers, SNMP-discovered LLDP chassis ID).
21
-
22
-### User Request
23
-
24
-Verbatim user request: *"create a pending SOW for the Unified multi-layered topology schema, which should allow merging topologies of the same kind (L2 + L2), but also merge topologies of different kinds (L2 + L3, or L2 + L3 + L7)"*.
25
-
26
-This SOW captures the problem space, references existing prior planning (`TODO-UNIFIED-TOPOLOGY-SCHEMA.md`), and surfaces the decisions that must be locked before any implementation begins. It does not commit to a specific merge algorithm, identity-matching policy, or storage model — those are user decisions captured in `## Implications And Decisions` below.
27
-
28
-### Assistant Understanding
29
-
30
-Facts (verified when this SOW was opened, updated 2026-05-10):
31
-
32
-- Historical note: this SOW originally referenced the legacy
33
- `src/go/pkg/topology/types.go` `Actor`/`Link`/`Match` model. That model is no
34
- longer the target contract for new topology Function payloads.
35
-- The current producer-facing contract is `netdata.topology.v1` in
36
- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`, with durable semantics recorded
37
- in `.agents/sow/specs/topology-function-schema.md`.
38
-- SOW-0023 adds producer-visible `data.correlation` rules, points, and claims
39
- for same-kind correlation. Future unified cross-layer work must reuse that
40
- plane where it fits instead of resurrecting a parallel legacy `Match` surface.
41
-- Topology data is produced today by four distinct sources (paths verified in repo):
42
- 1. **SNMP L2** — Go, in `src/go/pkg/topology/engine/` and `src/go/plugin/go.d/collector/snmp_topology/`. Produces actors of type `device` and `endpoint` with chassis IDs, MACs, IPs, sysName/sysObjectID. LLDP, CDP, FDB, ARP, STP.
43
- 2. **NetFlow / IPFIX / sFlow L3** — Rust, in `src/crates/netdata-netflow/`, plus Go plumbing in `src/go/`. Produces flow records with src/dst IP, ports, protocol, AS info, geolocation.
44
- 3. **Network-viewer L7** — C, in `src/collectors/network-viewer.plugin/`. eBPF-derived process-to-process connection observations with PIDs, container IDs, hostnames.
45
- 4. **Netdata streaming** — C, in `src/database/contexts/` and streaming subsystem. Produces parent ↔ child agent relationships with `NetdataNodeID`, `NetdataMachineGUID`.
46
-- A 6797-line prior-planning artifact exists at `TODO-UNIFIED-TOPOLOGY-SCHEMA.md` with extensive notes on the design space, source coverage, IP tracking policy, ASN/geo enrichment options, and several decision items already taken or pending. This SOW supersedes the unstructured TODO once committed; the TODO stays in place as historical reference until the SOW closes.
47
-- The codebase has helper code at `src/go/tools/topology-flow-merge/` and a per-poll merge implementation in `src/go/plugin/go.d/collector/snmp_topology/topology_output_merge.go` (sub-second scope: merging successive snapshots of the same SNMP topology). Cross-source / cross-layer merge does NOT exist today.
48
-- The underlying identity principle remains valid: each source contributes the
49
- identity facts it knows, and a downstream merge step can correlate actors when
50
- compatible identity facts overlap. In v1, those facts are represented through
51
- actor identity columns, evidence rows, and SOW-0023 correlation point/claim
52
- rows.
53
-
54
-Inferences:
55
-
56
-- "Same-kind merge" (L2 + L2) is the easier case: multiple SNMP-managed switches each produce their own L2 view; merging them means deduplicating actors by identity match and unioning their links. Most algorithmic complexity is in identity normalization (MAC formatting, hostname canonicalization, FQDN vs short-name).
57
-- "Cross-kind merge" (L2 + L3, or L2 + L3 + L7) is the harder case: actors are at different abstraction levels (a physical switch port at L2, an IP address at L3, a process at L7). Naive identity match works for some pairs (server's IP appears in both L3 flows and L2 ARP) but breaks for others (a process at L7 has no MAC; a switch at L2 has no PID). The schema must allow a parent-child or "lives-on" relationship rather than forcing every merge to a strict equality.
58
-- Conflict resolution policy is a real design problem: two sources legitimately disagree (e.g. an LLDP neighbor advertises a chassis ID that disagrees with what the device's own SNMP poll reports because of stale cache). Whichever policy is chosen affects every downstream consumer.
59
-- Performance and storage model interact: a memory-only merged view is simpler but limits historical queries and re-merge after configuration change; a persisted index enables more, at significant complexity cost.
60
-
61
-Unknowns (real, blocking design decisions):
62
-
63
-- Where does merge happen — agent-side per-source, agent-side cross-source, parent-side aggregating from multiple agents, or Cloud-side?
64
-- What's the identity-matching algorithm — strict any-overlap equality, canonicalized equality, probabilistic scoring with thresholds, or learned per-deployment?
65
-- What's the conflict-resolution policy when two sources disagree — newest wins, source-priority order, evidence-set union, or human-resolvable flag?
66
-- What scale targets must the design meet — actors-per-deployment, links-per-deployment, sources-per-actor, merge latency budget?
67
-- What's the storage and re-merge story — recompute on each request, persist a merged snapshot, persist an index of identities, or stream-update?
68
-- How does L7 process-level granularity surface in the merged graph without exploding actor count? (TODO-UNIFIED-TOPOLOGY-SCHEMA.md:1028 notes "L7 Detailed vs Aggregated Views" as an open question.)
69
-- How are stale entries aged out across sources whose freshness windows differ (LLDP cache ~120s, FDB ~5 min, NetFlow flow record duration, network-viewer eBPF connection lifetime)?
70
-
71
-### Acceptance Criteria
72
-
73
-(Final acceptance criteria are gated on user decisions in `## Implications And Decisions`. Until those decisions land, the criteria below are the high-level shape; they will be sharpened to specific verifiable outcomes once decisions are recorded.)
74
-
75
-- A single, documented unified topology schema is in production use by all four
76
- current sources (SNMP L2, NetFlow L3, network-viewer L7, Netdata streaming),
77
- using `netdata.topology.v1` or a clearly evolved version of it as the
78
- canonical contract.
79
-- A merge engine exists that takes N topology graphs (same-kind or cross-kind) and produces one merged graph following the chosen identity-match and conflict-resolution policies. Same-kind (L2 + L2) and cross-kind (L2 + L3, L2 + L3 + L7) cases each have explicit test coverage with expected merged output.
80
-- Identity-matching and conflict-resolution behavior is unit-tested with targeted fixtures covering: actors that should merge, actors that should NOT merge despite an incidental field overlap, sources that legitimately disagree, and stale entries aged across sources with different freshness windows.
81
-- A scale benchmark exists that exercises the merge engine at the chosen actors/links/sources targets and reports merge latency.
82
-- The merged graph can be served as a topology function output without breaking existing UI consumers; if a schema migration is needed, the migration plan is documented.
83
-- No customer-identifying data, community-member names, SNMP communities, bearer tokens, or PII appear in any committed artifact (SOW, code, code comments, tests, fixtures, commit messages, PR body).
84
-
85
-## Analysis
86
-
87
-Sources checked:
88
-
89
-- `src/go/pkg/topology/types.go` — legacy `Match`, `Actor`, `Link` schema
90
- reviewed when this SOW was opened; superseded for new Function payload work
91
- by `netdata.topology.v1`.
92
-- `src/go/pkg/topology/engine/` — current SNMP L2 merge logic (within-source).
93
-- `src/go/plugin/go.d/collector/snmp_topology/topology_output_merge.go` — per-poll snapshot merge (within-source).
94
-- `src/go/tools/topology-flow-merge/` — standalone helper, not currently wired into the runtime path (see TODO-UNIFIED-TOPOLOGY-SCHEMA.md branch-cleanup audit).
95
-- `src/crates/netdata-netflow/` — L3 source.
96
-- `src/collectors/network-viewer.plugin/` — L7 source.
97
-- Streaming subsystem (parent/child agent topology) — under `src/database/contexts/` and adjacent.
98
-- `TODO-UNIFIED-TOPOLOGY-SCHEMA.md` — 6797 lines of prior thinking and partial decisions.
99
-- Adjacent TODOs not yet absorbed: `TODO-streaming-topology.md`, `TODO-TOPOLOGY-ENRICHMENT.md`, `TODO-TOPOLOGY-FLOWS-INCOMPLETE-INTEGRATIONS.md`, `TODO-topology-flows-sync.md`, `TODO-topology-library.md`, `TODO-topology-library-phase2-direct-port.md`, `TODO-topology-netflow-metadata.md`.
100
-
101
-Current state:
102
-
103
-- The old extended-match principle remains useful, but the implementation path
104
- now needs to use v1 actor identity, evidence, and SOW-0023 correlation facts.
105
- What's missing is the merge layer that uses these identity facts across
106
- sources and layers.
107
-- Each source today produces its own topology output as a separate function. There is no merged endpoint.
108
-- Where merge does exist (same-source, per-poll snapshot fusion in the SNMP topology engine), it is implementation-internal — no shared abstraction, no reuse path for cross-source.
109
-- The codebase has hooks for layered presentation in the topology UI (the `PresentationActorType`, `PresentationLinkType` and friends in `types.go`), suggesting prior thinking about layered views, but no live data wiring.
110
-- Scope boundary recorded 2026-05-06: SOW-0020 owns the immediate shared topology Function payload migration, including lossless detailed encoding and view-oriented aggregation contracts.
111
-- Scope boundary recorded 2026-05-10: SOW-0023 owns generic same-kind
112
- `netdata.topology.v1` correlation rules, pure correlation actors,
113
- point/claim tables, and link layout tokens. This SOW owns the later unified
114
- cross-layer merge/correlation layer that consumes those detailed payloads and
115
- SOW-0023 correlation facts.
116
-
117
-Risks (cross-cutting):
118
-
119
-- **Schema lock-in**: any merge engine that consumes the wrong identity surface
120
- freezes that surface for downstream Cloud consumers. Future work must avoid
121
- reintroducing the legacy `Match` shape as a second public contract.
122
-- **Identity correlation false positives**: matching on "any overlapping field" is dangerous if a field is non-unique (e.g. private RFC1918 IP that recurs across customer subnets, hostname `localhost`). Without canonicalization and field-quality weighting, merge can fuse unrelated actors.
123
-- **Scale**: cross-layer merge expands actor count; L7 process granularity especially. If the design persists everything, storage grows; if it doesn't, historical queries degrade.
124
-- **Conflict noise**: two sources legitimately disagreeing produces user-visible warnings unless conflict resolution is automatic. Policy choice affects perceived data quality.
125
-- **Cross-version compatibility**: agents at different versions producing different schema versions. The merge engine must tolerate version drift gracefully.
126
-- **Sensitive data exposure**: merged topology surfaces hostnames, IPs, container/pod names, sysName, sysDescr — all potentially customer-identifying. The merge engine output is a public topology surface that the user sees; the produced data must not leak across tenants in any multi-tenant deployment scenario.
127
-
128
-## Pre-Implementation Gate
129
-
130
-Status: needs-user-decision
131
-
132
-Problem / root-cause model:
133
-
134
-- Topology evidence is produced by four distinct sources at three distinct
135
- layers (L2/L3/L7) plus a streaming hierarchy. Each source has its own view,
136
- none is reconciled with the others, and there is no merge engine that takes
137
- evidence from multiple sources and produces a single coherent graph.
138
- `netdata.topology.v1` now has actor identity and SOW-0023 same-kind
139
- correlation facts, but the cross-layer runtime layer that performs unified
140
- correlation does not exist. Implementation cannot start until merge semantics,
141
- identity-matching policy, conflict resolution, and storage model are locked.
142
-
143
-Evidence reviewed:
144
-
145
-- See "Sources checked" above. Direct evidence of the gap: no production code path correlates an SNMP-discovered L2 device with a NetFlow-observed L3 endpoint or a network-viewer-observed L7 process; the four topology functions are independent and produce four independent graphs.
146
-
147
-Affected contracts and surfaces:
148
-
149
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json` — schema may evolve; SOW-0023
150
- correlation plane is the current v1 payload contract to reuse instead of
151
- inventing a parallel matching surface.
152
-- All four source producers (SNMP topology Go path, netflow Rust+Go, network-viewer C, streaming C) — output schema may need conformance changes.
153
-- Topology functions exposed to the UI — possibly a new "merged" function or the existing per-source functions extended.
154
-- Cloud-side consumers — the merged output schema becomes a Cloud contract.
155
-- Test fixtures — new cross-source fixtures required.
156
-- Documentation: `profile-format.md` is unrelated; topology UI documentation will need a section once merge ships.
157
-
158
-Existing patterns to reuse:
159
-
160
-- Reuse the v1 identity and correlation contract rather than adding another
161
- public identity surface.
162
-- Per-source within-source merge logic in `topology_output_merge.go` and the SNMP topology engine — the same shape (deduplicate by identity, union links) likely generalizes; a candidate to lift to a shared package.
163
-- `src/go/tools/topology-flow-merge/` exists but is not wired in. Decide whether it's the seed of the cross-source merge engine or whether it gets retired.
164
-- The Netdata streaming hierarchy already establishes a parent/child actor relation — useful as a precedent for the "lives-on" cross-layer relationship.
165
-
166
-Risk and blast radius:
167
-
168
-- Schema evolution touches every consumer; needs a versioning story.
169
-- Cross-source merge is a new public-facing function; UI breakage risk if rolled out without a feature flag or staged rollout.
170
-- Identity-matching false positives are user-visible and erode trust faster than missing data; conservative defaults are safer.
171
-- Performance regressions on large deployments are likely if storage model isn't sized correctly.
172
-
173
-Sensitive data plan:
174
-
175
-- Same baseline as SOW-0001's plan: no community member names, no customer names, no SNMP communities, no bearer tokens, no SNMPv3 credentials, no customer-identifying IPs, hostnames, sysName/sysDescr/ifAlias/ifDescr, LLDP remote names, port descriptions, chassis IDs, management addresses, container/pod/namespace names.
176
-- Topology fixtures derived from real environments must be sanitized: replace customer-pointing strings with neutral labels (`switch-a`, `endpoint-1`, `service-x`).
177
-- Pre-commit checklist: same `rg -P` rules adopted for SOW-0001 (TBD after SOW-0001 cleanup pass), extended for the additional layer-3 / layer-7 surface (process names, container IDs, k8s pod/namespace names, ASN/geo data).
178
-- Cross-tenant data isolation in the merge engine itself is a runtime concern: the merge function output must respect Cloud tenant boundaries; this is an explicit acceptance check.
179
-
180
-Implementation plan:
181
-
182
-To be filled after the user decisions below are recorded. The plan will likely have these phases (illustrative, not committed):
183
-
184
-1. Reuse the SOW-0023 `data.correlation` contract where it fits cross-layer
185
- correlation; extend only when cross-layer semantics require additional facts.
186
-2. Build a shared `topology/merge` package implementing the chosen identity-match + conflict-resolution policy. Unit-test thoroughly with same-kind and cross-kind fixtures.
187
-3. Wire the merge engine behind a new topology function (`topology:unified` or similar). Keep per-source functions intact; the merged function is additive.
188
-4. Add Cloud-side consumer integration plus migration notes for existing UI surfaces.
189
-5. Add scale benchmark + observability counters (rows merged, conflicts detected, identity-match hit/miss rates).
190
-6. Iterate on real-deployment validation: at least two distinct deployments (different vendor mix, different layer mix) should produce coherent merged graphs.
191
-
192
-Validation plan:
193
-
194
-- Unit tests on the merge package with fixture matrices covering same-kind and cross-kind, identity overlaps and disagreements, age-out scenarios.
195
-- Integration test that wires a synthetic L2 + L3 + L7 input set and verifies the merged graph matches a hand-curated expected output.
196
-- Scale benchmark at the chosen targets.
197
-- Real-deployment validation in at least two environments.
198
-
199
-Artifact impact plan:
200
-
201
-- `AGENTS.md`: no expected change.
202
-- Runtime project skills: no expected change.
203
-- Specs under `.agents/sow/specs/`: a new spec documenting the unified-topology contract and identity-match algorithm is likely warranted on close (not at open).
204
-- End-user/operator docs: topology UI section(s) updated when the new function ships.
205
-- End-user/operator skills: none.
206
-- SOW lifecycle: this SOW absorbs `TODO-UNIFIED-TOPOLOGY-SCHEMA.md` once user decisions are recorded; the TODO stays in place as historical reference until this SOW closes.
207
-
208
-Open decisions: see `## Implications And Decisions` below — six decisions are outstanding.
209
-
210
-## Implications And Decisions
211
-
212
-The decisions below are unresolved and block implementation. Each is presented with options, pros/cons/implications/risks, and a recommendation that the user can accept or override.
213
-
214
-### Decision 1 — Where does the merge happen?
215
-
216
-**Options:**
217
-- **A. Agent-side, per-source.** Each source produces an already-merged-within-itself graph. No cross-source merge.
218
-- **B. Agent-side, cross-source on the same node.** The local agent merges all sources it produces. Cross-agent merge happens elsewhere (parent or Cloud).
219
-- **C. Parent-side aggregation.** Streaming parents merge children's contributions. Cloud receives an aggregated view.
220
-- **D. Cloud-side.** All raw evidence flows up, Cloud merges. Maximum flexibility, maximum bandwidth and storage cost.
221
-- **E. Hybrid.** Local same-source merge (A) + parent same-kind merge (C) + Cloud cross-kind merge (D).
222
-
223
-**Implications/Risks:** D leaks raw evidence to Cloud (cardinality concerns), C requires every parent to know all children's sources, A produces nothing useful for cross-source queries, B is the cleanest for single-host deployments but can't unify a fleet. E is the most flexible but has the largest blast radius.
224
-
225
-**Recommendation: B for now, plan E as the long-term shape.** Start by merging on the local agent for sources the local agent produces; defer cross-agent merge until single-agent merge is solid. This minimizes Cloud schema lock-in and lets the merge engine evolve based on real local-agent feedback.
226
-
227
-### Decision 2 — Identity-matching algorithm
228
-
229
-**Options:**
230
-- **A. Strict any-overlap equality on raw identity facts.** Two actors merge if
231
- any one of their identity fact sets intersects.
232
-- **B. Canonicalized equality.** MACs lowercased / colon-stripped, hostnames lowercased / FQDN-normalized, IPs canonicalized, before equality check.
233
-- **C. Field-quality-weighted scoring.** Each identity field has a
234
- discriminator weight (high for chassis ID, low for hostname `localhost`).
235
- Merge above a threshold.
236
-- **D. Probabilistic / learned per deployment.** A model decides; tunable per environment.
237
-
238
-**Implications/Risks:** A produces false positives on ambient identifiers (`localhost`, RFC1918 reuse). B handles 90% of false positives at low complexity. C handles ambiguity well but introduces a tuning knob users must understand. D is overkill for opening this work.
239
-
240
-**Recommendation: B as MVP; C as a follow-up SOW once real deployments produce telemetry on false-positive rates.** B is implementable, debuggable, and predictable.
241
-
242
-### Decision 3 — Conflict-resolution policy when sources disagree
243
-
244
-**Options:**
245
-- **A. Newest wins (timestamp-based).** Last evidence supersedes earlier.
246
-- **B. Source-priority order.** A defined priority — e.g. SNMP managed > LLDP-derived inferred > NetFlow-implied > network-viewer-implied. Highest priority wins.
247
-- **C. Evidence-set union, no resolution.** Both values are kept, exposed to the consumer as a multi-valued field.
248
-- **D. Human-resolvable flag.** Conflicts produce a UI warning, user picks.
249
-
250
-**Implications/Risks:** A loses provenance; B requires getting the priority right and is hard to change later; C grows attribute payloads without bound but never silently drops information; D produces UX friction.
251
-
252
-**Recommendation: C for identity facts, B for `Attributes`/`Derived` fields.**
253
-Identity should be inclusive (preserve all evidence so future merges can still
254
-match); attributes should be resolvable (one displayed value at a time).
255
-Provenance in either case is preserved as a per-value `source` tag.
256
-
257
-### Decision 4 — Storage model
258
-
259
-**Options:**
260
-- **A. Memory-only, recompute every request.** Simplest. No persistence.
261
-- **B. Persisted merged snapshot, recomputed on schedule.** One canonical view, served from cache.
262
-- **C. Persisted identity index.** Identity → actor lookup is persisted; the merged graph is computed on demand from raw source evidence using the index.
263
-- **D. Stream-updated.** Each source emits deltas; the merge engine maintains a live merged graph.
264
-
265
-**Implications/Risks:** A doesn't scale to large fleets; B trades freshness for cost; C is the right shape for query-heavy workloads but is a bigger build; D is a real-time pipeline with all the operational complexity that implies.
266
-
267
-**Recommendation: A for MVP within a single agent (minimal complexity, fast to ship), with the design ensuring the merge engine is deterministic so future migration to B or C is straightforward.**
268
-
269
-### Decision 5 — Scale targets
270
-
271
-**Options (illustrative):**
272
-- **A. Modest.** ≤ 1000 actors, ≤ 5000 links, ≤ 4 sources per actor.
273
-- **B. Mid.** ≤ 10000 actors, ≤ 50000 links, ≤ 8 sources per actor.
274
-- **C. High.** ≤ 100000 actors, ≤ 1M links, ≤ 16 sources per actor.
275
-
276
-**Implications/Risks:** Picking too small understates real enterprise networks; picking too large bloats the design. The right pick depends on whether L7 processes are first-class actors (then C) or aggregated to host level (then B).
277
-
278
-**Recommendation: B as initial target, with the design able to scale to C without re-architecture.** Validates against real enterprise mid-tier deployments and leaves room for L7-detail growth.
279
-
280
-### Decision 6 — L7 process granularity
281
-
282
-**Options:**
283
-- **A. L7 process is a first-class actor.** Each process gets its own actor; potentially explodes count.
284
-- **B. L7 aggregates to host actor with process-level attributes / sub-table.** One actor per host; processes are children or attributes.
285
-- **C. Two views: detailed and aggregated.** UI toggles between per-process and per-host.
286
-
287
-**Implications/Risks:** A is most informative but punishing at scale; B loses detail; C is the most flexible but requires two code paths. (TODO-UNIFIED-TOPOLOGY-SCHEMA.md:1028 notes this as an open item.)
288
-
289
-**Recommendation: B for MVP, C as a follow-up once UI and scale targets are validated.**
290
-
291
-## Plan
292
-
293
-Filled after Decisions 1-6 are recorded. Default skeleton (assuming the recommendations above):
294
-
295
-1. Lock and document the v1-based unified schema. Extend
296
- `netdata.topology.v1` only if the recorded decisions require it. Add a
297
- per-value `source` provenance tag mechanism to support Decision 3 if still
298
- needed after reviewing existing v1 evidence/correlation provenance.
299
-2. Implement a `topology/merge` package on the agent side with the chosen identity-match (Decision 2) and conflict-resolution (Decision 3) policies. Unit tests cover same-kind and cross-kind matrices.
300
-3. Wire the merge engine behind a new topology function on the local agent. Per-source functions remain available unchanged.
301
-4. Sanitized fixture set covering at least: a 2-switch L2 same-kind merge, an L2 + L3 cross-kind merge, an L2 + L3 + L7 cross-kind merge with a host-level L7 aggregation per Decision 6.
302
-5. Scale benchmark at the chosen target (Decision 5).
303
-6. Documentation: add a "Topology unified view" section to user-facing docs; add a project-level spec under `.agents/sow/specs/` describing the unified contract and the merge algorithm.
304
-7. Real-deployment validation across two distinct environments before declaring done.
305
-
306
-## Execution Log
307
-
308
-### 2026-05-01
309
-
310
-- SOW opened in `pending/` per user request following completion of the third review round on SOW-0001.
311
-- Existing prior planning artifact (`TODO-UNIFIED-TOPOLOGY-SCHEMA.md`, 6797 lines) noted as historical context; this SOW is the canonical replacement once user decisions are locked.
312
-- Six open decisions recorded; no implementation work begins until they are resolved.
313
-
314
-### 2026-05-06
315
-
316
-- Scope boundary recorded from SOW-0020: detailed/aggregated topology Function payload migration is owned by SOW-0020. This SOW remains pending for unified merge semantics, identity matching, conflict resolution, storage/indexing, scale targets, and cross-layer view behavior.
317
-
318
-### 2026-05-10
319
-
320
-- Updated scope boundary after SOW-0023 added the v1 correlation plane. This
321
- SOW must not reintroduce the legacy `Match` schema as a second public
322
- contract; future work starts from `netdata.topology.v1`, actor/evidence
323
- identity facts, and SOW-0023 correlation points/claims.
324
-
325
-## Validation
326
-
327
-Pending — gated on locked decisions and implementation.
328
-
329
-## Outcome
330
-
331
-Pending.
332
-
333
-## Lessons Extracted
334
-
335
-Pending until validation.
336
-
337
-## Followup
338
-
339
-Adjacent TODOs to absorb or supersede when this SOW progresses:
340
-
341
-- `TODO-streaming-topology.md` — streaming-source contributions to the merged graph.
342
-- `TODO-TOPOLOGY-ENRICHMENT.md` — geo / ASN / vendor enrichment as cross-cutting attributes.
343
-- `TODO-TOPOLOGY-FLOWS-INCOMPLETE-INTEGRATIONS.md` — flows-side integration gaps surfaced during the original split-PR cleanup.
344
-- `TODO-topology-flows-sync.md` — flows-side sync semantics.
345
-- `TODO-topology-library.md` — shared-library packaging considerations.
346
-- `TODO-topology-library-phase2-direct-port.md` — phase-2 port plan (likely a separate SOW once this one progresses).
347
-- `TODO-topology-netflow-metadata.md` — netflow metadata fields for cross-layer correlation.
348
-- `src/go/tools/topology-flow-merge/` — decide retirement vs absorb-as-merge-package-seed.
349
-- Possible child SOWs after this one progresses: identity-match scoring (Decision 2C as follow-up), persisted merge index (Decision 4C as follow-up), L7 detailed view (Decision 6C as follow-up), Cloud-side merge (Decision 1D/E as follow-up).
.agents/sow/pending/SOW-0006-20260503-skill-verification-harness.md
deleted
-206
@@ -1,206 +0,0 @@
1
-# SOW-0006 - Skill verification harness
2
-
3
-## Status
4
-
5
-Status: open
6
-
7
-Sub-state: stub. Created 2026-05-03 evening when the user split the verification piece out of SOW-0010 ("the evaluation does not need to be done now"). Depends on SOW-0010 closing first because the harness consumes the seed question lists and how-tos that SOW-0010 lays down.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Build a reusable **skill verification harness** that grades
14
-whether an AI assistant can answer concrete questions about a
15
-running Netdata environment using ONLY the skill's
16
-SKILL.md + per-domain guides + how-tos. The harness is the
17
-acceptance gate for any querying skill in this repo.
18
-
19
-The user's framing (verbatim from the 2026-05-03 expansion
20
-message): "use sonnet (not the best model) to query the APIs to
21
-find out if the instructions work. Sonnet should be responding
22
-what it did to find the answer and we should maintain a directory
23
-with how-tos and an index in SKILL.md to help the model figure
24
-out its way through it."
25
-
26
-The harness validates the two SOW-0010 skills first, then is
27
-re-used by SOW-0003 (`query-agent-events`), SOW-0004
28
-(`learn-site-structure`), SOW-0005 (`mirror-netdata-repos`), and
29
-SOW-0007 (`integrations-lifecycle`) as their own acceptance gates.
30
-
31
-### User Request
32
-
33
-> "We need to build a verification system: spawn an agent and
34
-> ask it a question:
35
-> - Node `costa-desktop` (on netdata-cloud in space Netdata People)
36
-> - find its hardware specs
37
-> - which operating system it runs?
38
-> - is it a parent? of how many and which nodes?
39
-> - is it a child and where it stream?
40
-> - does it have any vnodes and which?
41
-> - are there any failed data collection jobs?
42
-> - does it monitor nvidia DCGM? What is the data collection
43
-> frequency?
44
-> - which is the PID with the biggest memory consumption and in
45
-> which app group/category I can see in on the dashboard?
46
-> - find the last netdata status file log in the logs
47
-> - and many more
48
->
49
-> The idea is to use sonnet (not the best model) to query the
50
-> APIs to find out if the instructions work. Sonnet should be
51
-> responding what it did to find the answer and we should
52
-> maintain a directory with how-tos and an index in SKILL.md to
53
-> help the model figure out its way through it.
54
->
55
-> The list of how-tos should be supposed to be live. Every time
56
-> the assistant is asked to do something and it is not documented
57
-> and it is forced to do an analysis to answer, it should create
58
-> a how-to to help future assistants work faster.
59
->
60
-> The agent skill and documents and scripts should ensure the
61
-> assistant never sees the cloud api token and agent bearer
62
-> token."
63
->
64
-> Follow-up: "The evaluation does not need to be done now. But
65
-> the live how-tos inventory and everything else must be done."
66
-> -- so the inventory + token-safety + structural scope stay in
67
-> SOW-0010; the harness itself (this SOW) is deferred.
68
-
69
-### Acceptance Criteria
70
-
71
-- A verification runner under `.agents/skill-verification/<skill>/`, e.g.
72
- `<repo>/.agents/skill-verification/query-netdata-cloud/run.sh`
73
- -- that:
74
- 1. Reads `.agents/skill-verification/<skill>/questions.md` (the seed list
75
- shipped by the upstream SOW).
76
- 2. For each question, spawns a Sonnet-class assistant with a
77
- minimal system prompt that points at the skill (SKILL.md +
78
- `how-tos/INDEX.md` + canonical reference docs).
79
- 3. Captures the assistant's full transcript, the tool calls
80
- it made, and the final answer.
81
- 4. Records results under
82
- `<repo>/.local/audits/<skill>/verify/<timestamp>/`.
83
- 5. Grades each answer against `.agents/skill-verification/<skill>/grader.md`.
84
- 6. Reports pass / fail / unanswered counts.
85
-- `.agents/skill-verification/<skill>/grader.md` per skill: rubric covering (a) correctness,
86
- (b) evidence shown (file:line refs, response keys), (c) no
87
- exposed tokens / bearers / claim ids in the transcript or
88
- output, (d) how-to authored when missing.
89
-- An "unanswered question" prompt loop: when Sonnet cannot
90
- answer, the runner asks (interactively, or via a follow-up
91
- agent run) for a draft how-to and stages it under
92
- `<skill>/how-tos/<slug>.md` for human review.
93
-- A self-test mode: the runner can be pointed at any of the
94
- four SOW-0010+ skills and produces consistent reports. This
95
- is what makes it reusable across SOWs.
96
-- Sensitive-data gate: every committed file passes the
97
- pre-commit grep from
98
- `<repo>/.agents/sow/specs/sensitive-data-discipline.md`. The
99
- harness MUST scrub captured transcripts of any token / bearer
100
- / claim_id before persisting them under `.local/audits/`.
101
-
102
-## Analysis
103
-
104
-Sources to consult during stage 2:
105
-
106
-- `<repo>/.agents/sow/done/SOW-0010-20260503-netdata-query-skills-infrastructure.md`
107
- -- the SKILL.md and how-tos format this SOW must validate.
108
-- the user-global Claude Code config and `claude` CLI invocation patterns (the user
109
- has documented them in `the user-global Claude Code config (gitignored)` under "Run other
110
- AI assistants for a second opinion").
111
-- The pattern of the legacy private skills (`coverity-audit`,
112
- `pr-reviews`) for their `_lib.sh` shape and `_run` wrappers.
113
-
114
-Risks:
115
-
116
-- Token leakage in transcripts is the highest risk. The harness
117
- must scrub captured stdout/stderr before writing.
118
-- Cost: running Sonnet over many questions has real cost. The
119
- harness should default to a small seed-question subset and
120
- let the operator opt into the full run.
121
-- Flakiness: live API answers are time-dependent (e.g.
122
- "currently active alerts" varies). The grader rubric must
123
- accept structural correctness + presence of evidence rather
124
- than exact byte equality.
125
-
126
-## Pre-Implementation Gate
127
-
128
-Status: blocked-on-prereq
129
-
130
-Depends on SOW-0010 closing (it provides the SKILL.md, the
131
-per-domain guides, the `how-tos/INDEX.md`, the
132
-`.agents/skill-verification/<skill>/questions.md` seed list, and the
133
-token-safe wrappers that the harness must invoke).
134
-
135
-Sensitive data handling plan:
136
-
137
-- Follows `<repo>/.agents/sow/specs/sensitive-data-discipline.md`.
138
-- Captured transcripts and response bodies live under
139
- `<repo>/.local/audits/<skill>/verify/<timestamp>/` (gitignored).
140
-- Before persisting any transcript, the harness scrubs anything
141
- matching the discipline grep (UUIDs, IPv4 literals to specific
142
- hosts, bearer/cloud-token shaped strings, forbidden absolute
143
- paths). The scrubbed transcript is what gets stored; the
144
- unscrubbed in-memory copy is dropped.
145
-- No `.env` keys are required beyond what SOW-0010 already
146
- defines.
147
-
148
-Holding-pattern decisions to record now:
149
-
150
-- The harness invokes `claude -p` (Sonnet) per the user's
151
- documented pattern in `the user-global Claude Code config (gitignored)`. Model selection
152
- is a runner flag with a default; not a per-question
153
- hardcode.
154
-- The how-to generation prompt is a separate template under
155
- `.agents/skill-verification/<skill>/howto-template.md`. Stage 2
156
- implementation defines it.
157
-- Verification seed questions are harness inputs, not public/operator skill
158
- content. They live under `.agents/skill-verification/<skill>/questions.md`,
159
- not under `docs/netdata-ai/skills/`.
160
-
161
-## Implications And Decisions
162
-
163
-No new decisions at this stub stage. Will be added when
164
-SOW-0010 closes and stage 2 begins.
165
-
166
-## Plan
167
-
168
-1. **Wait for SOW-0010 to close.**
169
-2. Stage 2a: read SOW-0010 final deliverables (SKILL.md
170
- structure, how-tos shape, wrappers).
171
-3. Stage 2b: implement `.agents/skill-verification/query-netdata-cloud/run.sh`;
172
- prove end-to-end on the seed questions.
173
-4. Stage 2c: parameterize `run.sh` so it can target any of the
174
- SOW-0010+ skills.
175
-5. Stage 2d: implement the how-to generation prompt loop.
176
-6. Stage 2e: implement the transcript scrubber + unit test
177
- that asserts no token bytes are persisted.
178
-7. Stage 2f: validate end-to-end. Close.
179
-
180
-## Execution Log
181
-
182
-### 2026-05-03
183
-
184
-- Created as a stub during the second SOW-0010 scope expansion.
185
-
186
-## Validation
187
-
188
-Pending. All sub-fields will be filled at close.
189
-
190
-## Outcome
191
-
192
-Pending.
193
-
194
-## Lessons Extracted
195
-
196
-Pending.
197
-
198
-## Followup
199
-
200
-None yet.
201
-
202
-## Regression Log
203
-
204
-None yet.
205
-
206
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/pending/SOW-0014-20260507-snmp-licensing-unsupported-table-cache.md
deleted
-225
@@ -1,225 +0,0 @@
1
-# SOW-0014 - SNMP typed unsupported table cache
2
-
3
-## Status
4
-
5
-Status: open
6
-
7
-Sub-state: follow-up created from SOW-0013 close-out review; scope expanded by user on 2026-05-08 to cover both typed licensing and typed BGP table roots.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Avoid repeated typed SNMP table-root walks on Cisco and other devices that explicitly report unsupported typed licensing or typed BGP tables, without reintroducing broad missing-OID poisoning for ordinary metric tables or empty-but-valid tables.
14
-
15
-### User Request
16
-
17
-User accepted broad Cisco licensing coverage through dedicated mixins and agreed that nonexistent licensing table OID caching should be tracked as the solution for unsupported devices.
18
-
19
-User later accepted broad Catalyst BGP attachment and clarified that the unsupported table-root cache should apply to both typed licensing and typed BGP table roots.
20
-
21
-### Assistant Understanding
22
-
23
-Facts:
24
-
25
-- `cisco.yaml` extends `_cisco-licensing-traditional.yaml` and `_cisco-licensing-smart.yaml`.
26
-- Those licensing mixins add two table roots: `clmgmtLicenseInfoTable` and `ciscoSlaEntitlementInfoTable`.
27
-- `cisco-catalyst.yaml` extends `_cisco-bgp4-mib.yaml` in the BGP typed projection SOW, so Catalyst devices that do not expose Cisco BGP tables can hit unsupported typed BGP table roots.
28
-- SOW-0013 intentionally removed the old empty-walk table missing cache write because empty tables and unsupported tables must not be conflated.
29
-- Explicit `NoSuchObject` / `NoSuchInstance` table-root responses are still a reasonable candidate for a narrowly scoped typed-row unsupported-table cache.
30
-
31
-Inferences:
32
-
33
-- The follow-up should distinguish unsupported typed table roots from empty typed tables with zero rows.
34
-- The cache should be scoped to typed licensing and typed BGP table roots and must not suppress ordinary metric table OIDs.
35
-
36
-Unknowns:
37
-
38
-- Exact `gosnmp` walk response shapes for explicit table-root `NoSuchObject` / `NoSuchInstance` need fixture or mock confirmation before implementation.
39
-
40
-### Acceptance Criteria
41
-
42
-- Typed licensing and typed BGP table walks that receive explicit no-such table-root responses are skipped on later collection cycles.
43
-- Empty valid tables with zero rows are not cached as unsupported.
44
-- Ordinary metrics and topology table collection are unaffected.
45
-- Tests cover unsupported typed licensing table root, unsupported typed BGP table root, empty table, and ordinary metric table behavior.
46
-
47
-## Analysis
48
-
49
-Sources checked:
50
-
51
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_licensing.go`
52
-- `src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_table.go`
53
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/cisco.yaml`
54
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_cisco-licensing-traditional.yaml`
55
-- `src/go/plugin/go.d/config/go.d/snmp.profiles/default/_cisco-licensing-smart.yaml`
56
-- `.agents/sow/current/SOW-0013-20260507-snmp-licensing-projection.md`
57
-
58
-Current state:
59
-
60
-- Missing scalar OIDs are cached exactly.
61
-- Licensing table roots with no returned PDUs are not cached, by design, because zero rows may mean an empty valid table.
62
-- Broad Cisco licensing and broad Catalyst BGP can therefore continue walking unsupported typed table roots each cycle until this follow-up is implemented.
63
-
64
-Risks:
65
-
66
-- Over-broad caching would hide licensing data that appears later or suppress unrelated regular metrics.
67
-- Under-scoped caching leaves avoidable per-cycle Cisco probe cost.
68
-
69
-## Pre-Implementation Gate
70
-
71
-Status: blocked
72
-
73
-Problem / root-cause model:
74
-
75
-- Broad Cisco licensing and Catalyst BGP coverage intentionally trade initial probe breadth for profile simplicity. Unsupported table roots need a precise negative-cache path keyed to explicit no-such table-root responses, not to empty walk results.
76
-
77
-Evidence reviewed:
78
-
79
-- `collector_licensing.go` walks licensing table roots on cache misses.
80
-- `collector_table.go` no longer marks zero-PDU walks missing.
81
-- SOW-0013 execution log records this as a follow-up requirement.
82
-
83
-Affected contracts and surfaces:
84
-
85
-- ddsnmpcollector typed licensing table collection.
86
-- ddsnmpcollector typed BGP table collection.
87
-- Collection stats for missing OIDs and table walks.
88
-- Cisco SNMP collection overhead on unsupported licensing and BGP devices.
89
-
90
-Existing patterns to reuse:
91
-
92
-- Shared exact `missingOIDs` scalar cache.
93
-- Existing table cache and SNMP walk helpers.
94
-- Existing mock SNMP tests under `ddsnmpcollector`.
95
-
96
-Risk and blast radius:
97
-
98
-- Narrow runtime collector behavior change.
99
-- Must not alter profile schema.
100
-- Must not change ordinary table metric missing behavior.
101
-
102
-Sensitive data handling plan:
103
-
104
-- Use synthetic SNMP mock PDUs only. Do not add real device walks or raw MIB content.
105
-
106
-Implementation plan:
107
-
108
-1. Model explicit table-root no-such responses in tests.
109
-2. Add a typed-row unsupported table-root cache or equivalent filter for licensing and BGP table roots.
110
-3. Prove empty valid tables are still retried.
111
-
112
-Validation plan:
113
-
114
-- `go test -count=1 ./collector/snmp/ddsnmp/ddsnmpcollector`
115
-- `go test -count=1 ./collector/snmp/ddsnmp/...`
116
-- `go test -count=1 ./collector/snmp/...`
117
-
118
-Artifact impact plan:
119
-
120
-- AGENTS.md: no expected update.
121
-- Runtime project skills: no expected update unless a new authoring rule emerges.
122
-- Specs: update `.agents/sow/specs/snmp-profile-projection.md` if the cache becomes a durable projection behavior.
123
-- End-user/operator docs: no expected update.
124
-- End-user/operator skills: no expected update.
125
-- SOW lifecycle: track this as the SOW-0013 follow-up.
126
-
127
-Open-source reference evidence:
128
-
129
-- None. This is collector runtime behavior with synthetic tests.
130
-
131
-Open decisions:
132
-
133
-- None for planning. Implementation details should be validated against actual `gosnmp` no-such walk behavior before patching.
134
-
135
-## Implications And Decisions
136
-
137
-- Follow-up created to satisfy SOW-0013 follow-up discipline; no implementation is part of SOW-0013.
138
-- Scope expanded on 2026-05-08: the same unsupported table-root handling must apply to typed licensing and typed BGP table roots.
139
-
140
-## Plan
141
-
142
-1. Add tests for unsupported typed licensing and typed BGP table-root no-such handling.
143
-2. Implement narrowly scoped typed-row cache/filter behavior.
144
-3. Validate narrow and shared SNMP suites.
145
-
146
-## Execution Log
147
-
148
-### 2026-05-07
149
-
150
-- Created as follow-up from SOW-0013 final review.
151
-
152
-## Validation
153
-
154
-Acceptance criteria evidence:
155
-
156
-- Pending implementation.
157
-
158
-Tests or equivalent validation:
159
-
160
-- Pending implementation.
161
-
162
-Real-use evidence:
163
-
164
-- Pending implementation.
165
-
166
-Reviewer findings:
167
-
168
-- Pending implementation.
169
-
170
-Same-failure scan:
171
-
172
-- Pending implementation.
173
-
174
-Sensitive data gate:
175
-
176
-- Pending implementation.
177
-
178
-Artifact maintenance gate:
179
-
180
-- AGENTS.md: pending implementation outcome.
181
-- Runtime project skills: pending implementation outcome.
182
-- Specs: pending implementation outcome.
183
-- End-user/operator docs: pending implementation outcome.
184
-- End-user/operator skills: pending implementation outcome.
185
-- SOW lifecycle: pending implementation outcome.
186
-
187
-Specs update:
188
-
189
-- Pending implementation.
190
-
191
-Project skills update:
192
-
193
-- Pending implementation.
194
-
195
-End-user/operator docs update:
196
-
197
-- Pending implementation.
198
-
199
-End-user/operator skills update:
200
-
201
-- Pending implementation.
202
-
203
-Lessons:
204
-
205
-- Pending implementation.
206
-
207
-Follow-up mapping:
208
-
209
-- Pending implementation.
210
-
211
-## Outcome
212
-
213
-Pending.
214
-
215
-## Lessons Extracted
216
-
217
-Pending.
218
-
219
-## Followup
220
-
221
-None yet.
222
-
223
-## Regression Log
224
-
225
-None yet.
.agents/sow/pending/SOW-0024-20260510-vsphere-topology-v1-migration.md
deleted
-274
@@ -1,274 +0,0 @@
1
-# SOW-0024 - vSphere topology v1 migration
2
-
3
-## Status
4
-
5
-Status: open
6
-
7
-`completed` is the successful terminal status. `done` is a directory name, not a status value. Do not use `Status: done` or `Status: complete`.
8
-
9
-Sub-state: pending until SOW-0021 graph presentation, SOW-0023 cross-payload
10
-matching/link layout, and SOW-0022 table/modal composition are finished.
11
-
12
-## Requirements
13
-
14
-### Purpose
15
-
16
-Migrate the vSphere topology producer from the superseded topology schema to `netdata.topology.v1`, preserving vSphere inventory semantics, presentation, and drilldown data while keeping the Cloud frontend producer-agnostic.
17
-
18
-### User Request
19
-
20
-The user asked to add a later SOW to transform vSphere to the new schema because the vSphere topology is still legacy.
21
-
22
-### Assistant Understanding
23
-
24
-Facts:
25
-
26
-- The vSphere topology producer lives in the separate vSphere feature worktree
27
- `<vsphere-worktree>`.
28
-- The vSphere producer currently uses the legacy Go topology package and `WithPresentation()`.
29
-- Agent SOW-0021 explicitly did not migrate vSphere, but added color/icon tokens so the later vSphere migration should not need another graph-presentation schema redesign.
30
-- Agent SOW-0023 added generic correlation rules and link layout tokens that
31
- vSphere must respect when migrated.
32
-- The frontend must remain topology-schema agnostic; it must not branch on vSphere as a special UI domain.
33
-
34
-Inferences:
35
-
36
-- This migration should happen only after the schema, Cloud aggregator, and UI can consume the full v1 graph-presentation/table contract.
37
-- The migration should produce v1 topology facts and presentation profiles directly, not old-schema compatibility payloads.
38
-
39
-Unknowns:
40
-
41
-- Whether the vSphere worktree will still be clean and owned by the same worker when this SOW starts.
42
-- Whether SOW-0022 table/modal composition will require additional vSphere-specific actor detail table fields by the time this migration starts.
43
-
44
-### Acceptance Criteria
45
-
46
-- vSphere `topology:vsphere` emits `schema_version: "netdata.topology.v1"`.
47
-- The producer no longer uses the superseded topology package for the production payload.
48
-- Actor types cover datacenter, cluster, host, VM, datastore, network, datastore cluster, and resource pool.
49
-- Link types cover contains, connects, runs, and any other vSphere relationship emitted by the producer.
50
-- Actor identities use stable vSphere managed object identifiers where available, with display labels kept separate through `presentation.label_policy`.
51
-- vSphere graph presentation is expressed through `types.actor_types.<id>.presentation`, `types.link_types.<id>.presentation`, optional `types.port_types`, and `data.presentation`.
52
-- vSphere link types use SOW-0023 `presentation.layout.strength` and
53
- `presentation.layout.distance` tokens where ownership, dependency, inferred,
54
- or weak relationships need different graph forces.
55
-- vSphere stable object identifiers are evaluated for SOW-0023
56
- `data.correlation.claims` only when they can help cross-payload matching;
57
- producer output must not expose aggregator internal states.
58
-- vSphere actor detail/inventory tables use the SOW-0022 table/modal composition contract if that contract is complete before this SOW starts.
59
-- JSON Schema validation and Go semantic validation pass.
60
-- Payload size is measured on realistic or synthetic vSphere inventory shapes.
61
-- The vSphere worktree owner is informed before edits start.
62
-
63
-## Analysis
64
-
65
-Sources checked:
66
-
67
-- `.agents/skills/project-create-topology/SKILL.md`
68
-- `.agents/sow/current/SOW-0021-20260509-topology-presentation-contract.md`
69
-- `src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md`
70
-- `.agents/sow/specs/topology-function-schema.md`
71
-- `<vsphere-worktree>/src/go/plugin/go.d/collector/vsphere/func_topology.go`
72
-- `<vsphere-worktree>/src/go/plugin/go.d/collector/vsphere/func_topology_presentation.go`
73
-
74
-Current state:
75
-
76
-- `func_topology.go` in the vSphere worktree declares `vsphereTopologySchemaVersion = "2.0"` and returns `topology.Data`.
77
-- `vsphereTopologyMethodConfig()` attaches legacy presentation with `WithPresentation(vsphereTopologyPresentation())`.
78
-- `func_topology_presentation.go` defines legacy actor/link presentation for vSphere actor and link types.
79
-- The vSphere worktree was checked when this SOW was created and had no modified files under the vSphere collector.
80
-
81
-Risks:
82
-
83
-- Starting this before SOW-0021/SOW-0023/SOW-0022 integration settles may force duplicate migration work.
84
-- Editing the vSphere worktree without coordination may collide with another worker.
85
-- Mixing old-schema reconstruction metadata into the v1 payload would violate the topology schema contract.
86
-- Treating vSphere display names as identities would make aggregation and cross-payload matching fragile.
87
-
88
-## Pre-Implementation Gate
89
-
90
-Status: blocked
91
-
92
-Problem / root-cause model:
93
-
94
-- vSphere remains on the superseded topology schema while the rest of the topology work is moving to `netdata.topology.v1`.
95
-- SOW-0021 intentionally left vSphere unmigrated and only preserved the token vocabulary needed for a later migration.
96
-- This SOW is blocked until the core v1 contract, frontend rendering, Cloud aggregation, cross-payload matching, and table/modal composition are complete enough to avoid rework.
97
-
98
-Evidence reviewed:
99
-
100
-- SOW-0021 vSphere migration posture records that vSphere is not migrated by SOW-0021 and should be coordinated separately.
101
-- SOW-0023 records the final correlation and link layout contract that vSphere
102
- must use when migrated.
103
-- The vSphere worktree producer still uses the legacy topology package and `WithPresentation()`.
104
-- The project topology skill states the vSphere worktree must not be edited
105
- before telling the user because another agent may be working there.
106
-
107
-Affected contracts and surfaces:
108
-
109
-- vSphere Function output schema.
110
-- vSphere topology producer implementation in the separate worktree.
111
-- Topology JSON Schema and semantic validator compatibility.
112
-- Cloud frontend graph presentation and actor modal/table rendering.
113
-- Cloud topology service aggregation and cross-payload matching.
114
-- Tests/fixtures for vSphere inventory topology.
115
-
116
-Existing patterns to reuse:
117
-
118
-- `src/go/pkg/topology/v1` compact-table helpers from this worktree.
119
-- Network-connections, streaming, and SNMP v1 producers after SOW-0021.
120
-- SOW-0022 table/modal composition contract once complete.
121
-- SOW-0023 correlation rules, pure correlation actors, point/claim tables, and
122
- link layout tokens once complete.
123
-
124
-Risk and blast radius:
125
-
126
-- Medium: vSphere topology is a producer-specific migration, but it touches public Function payload shape.
127
-- Compatibility: Cloud frontend must continue old-schema support until the Agent rollout is complete.
128
-- Performance: large vSphere inventories may produce many actors/links and require compact tables.
129
-- Security: vSphere managed object ids, inventory paths, hostnames, datastores, networks, and labels can identify private infrastructure and must not be copied raw into durable artifacts.
130
-
131
-Sensitive data handling plan:
132
-
133
-- Do not commit raw vSphere payload captures.
134
-- Use synthetic or sanitized fixtures.
135
-- Redact private inventory names, managed object ids, hostnames, datastore names, network names, private endpoints, credentials, tokens, and customer data from SOWs, docs, skills, tests, and review artifacts.
136
-
137
-Implementation plan:
138
-
139
-1. Reconfirm with the user before editing `<vsphere-worktree>`.
140
-2. Read completed SOW-0021, SOW-0023, and SOW-0022 outcomes.
141
-3. Inventory current vSphere actors, links, attributes, labels, and presentation.
142
-4. Design the v1 actor/link/evidence/table types for vSphere.
143
-5. Implement v1 payload generation using compact tables.
144
-6. Preserve graph presentation through v1 type-level and graph-level presentation.
145
-7. Add fixtures/tests and validate against the schema and semantic validator,
146
- including link layout tokens and any vSphere correlation claims.
147
-8. Coordinate Cloud frontend and Cloud aggregator validation with generic topology fixtures.
148
-
149
-Validation plan:
150
-
151
-- Go unit tests for vSphere v1 payload shape.
152
-- JSON Schema validation against `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
153
-- Semantic validation using `src/go/pkg/topology/v1`.
154
-- Payload size measurement on synthetic or sanitized vSphere inventory shapes.
155
-- Frontend manual check through generic topology rendering after UI support lands.
156
-- Cloud aggregator fixture once the service supports the final v1 presentation/table/matching contracts.
157
-
158
-Artifact impact plan:
159
-
160
-- AGENTS.md: likely unaffected.
161
-- Runtime project skills: update `.agents/skills/project-create-topology/SKILL.md`
162
- only if migration reveals new reusable vSphere topology guidance.
163
-- Specs: update `.agents/sow/specs/topology-function-schema.md` if this migration changes durable topology semantics.
164
-- End-user/operator docs: update only if public vSphere topology behavior or Function docs exist and change.
165
-- End-user/operator skills: update only if public topology skill guidance changes.
166
-- SOW lifecycle: keep this SOW pending until SOW-0021, SOW-0023, and SOW-0022 are done.
167
-
168
-Open-source reference evidence:
169
-
170
-- None checked. This is an internal producer migration from the legacy Netdata topology contract to the new Netdata topology contract.
171
-
172
-Open decisions:
173
-
174
-- None for creating this pending SOW.
175
-- Before implementation starts, confirm the vSphere worktree ownership and whether SOW-0022 table/modal composition is mandatory for the first vSphere v1 payload.
176
-
177
-## Implications And Decisions
178
-
179
-- User decision: vSphere migration is tracked as a later SOW after the current topology schema/UI/Cloud work.
180
-- This SOW must not be started before the other topology SOWs are finished unless the user explicitly changes the order.
181
-
182
-## Plan
183
-
184
-1. Wait for SOW-0021, SOW-0023, and SOW-0022 completion.
185
-2. Confirm vSphere worktree ownership with the user.
186
-3. Move this SOW to current and fill any newly discovered implementation specifics.
187
-4. Implement and validate the vSphere migration to `netdata.topology.v1`.
188
-
189
-## Execution Log
190
-
191
-### 2026-05-10
192
-
193
-- Created pending SOW after the user noted that vSphere remains legacy and should be migrated after the other topology SOWs.
194
-
195
-- Updated prerequisites after SOW-0023 added declarative correlation rules and
196
- link layout tokens. vSphere migration must use the final v1 contract rather
197
- than the older presentation-only contract.
198
-
199
-## Validation
200
-
201
-Acceptance criteria evidence:
202
-
203
-- Pending; this SOW has not started implementation.
204
-
205
-Tests or equivalent validation:
206
-
207
-- Pending.
208
-
209
-Real-use evidence:
210
-
211
-- Pending.
212
-
213
-Reviewer findings:
214
-
215
-- Pending.
216
-
217
-Same-failure scan:
218
-
219
-- Pending.
220
-
221
-Sensitive data gate:
222
-
223
-- This SOW contains only sanitized file references and no raw vSphere inventory payloads or credentials.
224
-
225
-Artifact maintenance gate:
226
-
227
-- AGENTS.md: not updated; workflow rules did not change.
228
-- Runtime project skills: not updated; this SOW only records future work.
229
-- Specs: not updated; behavior has not changed yet.
230
-- End-user/operator docs: not updated; behavior has not changed yet.
231
-- End-user/operator skills: not updated; behavior has not changed yet.
232
-- SOW lifecycle: open in `.agents/sow/pending/` and blocked on earlier topology SOWs.
233
-
234
-Specs update:
235
-
236
-- No spec update yet because no behavior changed.
237
-
238
-Project skills update:
239
-
240
-- No project skill update yet because no workflow changed.
241
-
242
-End-user/operator docs update:
243
-
244
-- No docs update yet because no behavior changed.
245
-
246
-End-user/operator skills update:
247
-
248
-- No skill update yet because no behavior changed.
249
-
250
-Lessons:
251
-
252
-- vSphere must be tracked explicitly because SOW-0021 prepared the schema for it but did not migrate its producer.
253
-
254
-Follow-up mapping:
255
-
256
-- Implemented by this pending SOW after SOW-0021, SOW-0023, and SOW-0022 finish.
257
-
258
-## Outcome
259
-
260
-Pending.
261
-
262
-## Lessons Extracted
263
-
264
-Pending until implementation.
265
-
266
-## Followup
267
-
268
-- Start this after SOW-0021, SOW-0023, and SOW-0022 are complete.
269
-
270
-## Regression Log
271
-
272
-None yet.
273
-
274
-Append regression entries here only after this SOW was completed or closed and later testing or use found broken behavior. Use a dated `## Regression - YYYY-MM-DD` heading at the end of the file. Never prepend regression content above the original SOW narrative.
.agents/sow/pending/SOW-0029-20260511-network-connections-detailed-loose-sides.md
deleted
-268
@@ -1,268 +0,0 @@
1
-# SOW-0029 - Network Connections Detailed Loose Sides
2
-
3
-## Status
4
-
5
-Status: open
6
-
7
-Sub-state: pending follow-up split from SOW-0028.
8
-
9
-## Requirements
10
-
11
-### Purpose
12
-
13
-Make detailed `topology:network-connections` preserve exact socket endpoints
14
-without forcing every remote `IP:PORT` tuple to become a graph actor, while
15
-keeping direct-Agent views readable and Cloud aggregation correlation-safe.
16
-
17
-### User Request
18
-
19
-The user asked to complete SOW-0028. The remaining true loose-side detailed
20
-graph work was split here because it requires a separate Agent/UI/aggregator
21
-execution pass.
22
-
23
-### Assistant Understanding
24
-
25
-Facts:
26
-
27
-- SOW-0028 implemented the schema compatibility layer, modal identification,
28
- mode metadata, correlation rule classes, and relationship-summary rows.
29
-- The current Agent producer still emits endpoint actors in detailed
30
- network-connections views.
31
-- The SOW-0028 spec describes a stronger target: detailed rows may have one
32
- known actor side and one loose endpoint side, with exact tuple facts preserved
33
- for correlation and UI materialization.
34
-
35
-Inferences:
36
-
37
-- Implementing true loose sides is not a small closure task. It affects compact
38
- table shape, schema validation, UI materialization, and aggregator
39
- correlation execution together.
40
-- Keeping it separate reduces risk of breaking current working direct-Agent
41
- topology views.
42
-
43
-Unknowns:
44
-
45
-- The exact compact schema shape for one-sided graph/evidence rows must be
46
- finalized against the existing `netdata.topology.v1` table model before code
47
- changes.
48
-
49
-### Acceptance Criteria
50
-
51
-- The schema/docs/specs define the exact compact representation for one-sided
52
- detailed network-connections rows and materialization policy.
53
-- The Agent detailed network-connections producer emits known actors plus
54
- one-sided loose endpoint facts where appropriate, without actor-per-ephemeral
55
- `IP:PORT` graph explosion.
56
-- The UI materializes loose sides only for direct-Agent detailed views and only
57
- according to producer-declared policy.
58
-- The Cloud topology service resolves exact loose-side matches before returning
59
- aggregated output and keeps unresolved/partial cases visible without exposing
60
- aggregator-internal state.
61
-- Installed Function checks and focused tests prove detailed and aggregated
62
- network-connections still render and correlate correctly.
63
-
64
-## Analysis
65
-
66
-Sources checked:
67
-
68
-- `.agents/sow/specs/topology-modes-correlation-aggregation.md`
69
-- `.agents/sow/done/SOW-0028-20260511-topology-mode-correlation-aggregation.md`
70
-- `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`
71
-- `src/collectors/network-viewer.plugin/network-viewer.c`
72
-- Cloud frontend `TODO-topology-mode-correlation-aggregation.md`
73
-- Cloud topology service SOW-0011
74
-
75
-Current state:
76
-
77
-- Detailed network-connections preserves exact socket evidence and correlation
78
- metadata.
79
-- Detailed network-connections still uses endpoint actors in the renderable
80
- graph.
81
-- UI loose-side materialization is documented in the frontend TODO but not yet
82
- implemented because producers do not emit one-sided detailed rows.
83
-
84
-Risks:
85
-
86
-- Actor-per-`IP:PORT` remains a graph cardinality risk if detailed views grow.
87
-- A careless loose-side implementation can become a hidden frontend aggregator
88
- or hide unresolved dependencies.
89
-- Changing detailed graph rows can break existing modal recipes if source
90
- columns and owner filters are not updated together.
91
-
92
-## Pre-Implementation Gate
93
-
94
-Status: blocked
95
-
96
-Problem / root-cause model:
97
-
98
-- The current v1 payload supports exact socket evidence, but graph links still
99
- require materialized endpoint actors for remote peers. The desired detailed
100
- model needs one-sided rows and a declared materialization policy, so exact
101
- evidence can be preserved without graph actor explosion.
102
-
103
-Evidence reviewed:
104
-
105
-- SOW-0028 installed checks show detailed network-connections returns socket
106
- evidence and correlation metadata, but still returns endpoint actors and
107
- normal graph links.
108
-- `src/collectors/network-viewer.plugin/network-viewer.c` currently emits
109
- non-null `src_actor` and `dst_actor` columns for graph links and socket
110
- evidence table declarations.
111
-
112
-Affected contracts and surfaces:
113
-
114
-- Agent topology JSON schema, developer guide, and network-viewer producer.
115
-- Cloud frontend v1 normalizer, graph model, actor modal source selection, and
116
- diagnostics.
117
-- Cloud topology service correlation and aggregation core.
118
-- Topology specs and project topology skill.
119
-
120
-Existing patterns to reuse:
121
-
122
-- Compact tables with nullable reference columns.
123
-- Existing modal `selected_side_endpoint` projection.
124
-- Existing correlation `resolve_loose_side` rule class.
125
-- Existing UI diagnostics for unsupported metadata.
126
-
127
-Risk and blast radius:
128
-
129
-- High semantic risk for network dependency correctness.
130
-- Medium UI risk because one-sided rows need deterministic render-only actors.
131
-- Medium aggregator risk because exact match, partial match, and no-match cases
132
- must remain visible and truthful.
133
-
134
-Sensitive data handling plan:
135
-
136
-- Use synthetic socket fixtures and summaries only.
137
-- Keep any live Function captures under `.local/`; do not commit raw process
138
- names, command lines, bearer tokens, private endpoints, raw node IDs, raw
139
- machine GUIDs, or customer-identifying public endpoints.
140
-
141
-Implementation plan:
142
-
143
-1. Finalize the schema shape and materialization policy with synthetic
144
- examples.
145
-2. Update Agent producer and schema validation.
146
-3. Update UI normalizer/materialization and modal behavior.
147
-4. Update Cloud topology service correlation handling for one-sided rows.
148
-5. Validate with synthetic tests and installed Function checks.
149
-
150
-Validation plan:
151
-
152
-- Agent schema validation and network-viewer build.
153
-- Focused UI tests for direct-Agent detailed loose-side materialization.
154
-- Cloud topology service tests for exact, partial, ambiguous, and no-match
155
- loose-side cases.
156
-- Installed `topology:network-connections` detailed/aggregated checks.
157
-
158
-Artifact impact plan:
159
-
160
-- AGENTS.md: not expected unless workflow changes.
161
-- Runtime project skills: update `project-create-topology` if the final
162
- materialization policy adds a recurring rule.
163
-- Specs: update topology schema specs.
164
-- End-user/operator docs: not expected.
165
-- End-user/operator skills: not expected.
166
-- SOW lifecycle: start only after SOW-0028 is completed and the user approves
167
- this follow-up priority.
168
-
169
-Open-source reference evidence:
170
-
171
-- Not checked. This is a Netdata topology schema contract, not an external
172
- protocol behavior.
173
-
174
-Open decisions:
175
-
176
-- Blocked until this SOW is selected as the next active topology task.
177
-
178
-## Implications And Decisions
179
-
180
-No user decision has been requested yet.
181
-
182
-## Plan
183
-
184
-1. Re-open the detailed network-connections schema section and decide the exact
185
- compact table representation.
186
-2. Patch Agent, UI, and Cloud topology service in one coordinated pass.
187
-3. Validate with installed Function output and focused tests.
188
-
189
-## Execution Log
190
-
191
-### 2026-05-11
192
-
193
-- Created as a pending follow-up split from SOW-0028 closure.
194
-
195
-## Validation
196
-
197
-Acceptance criteria evidence:
198
-
199
-- Pending.
200
-
201
-Tests or equivalent validation:
202
-
203
-- Pending.
204
-
205
-Real-use evidence:
206
-
207
-- Pending.
208
-
209
-Reviewer findings:
210
-
211
-- Pending.
212
-
213
-Same-failure scan:
214
-
215
-- Pending.
216
-
217
-Sensitive data gate:
218
-
219
-- Pending.
220
-
221
-Artifact maintenance gate:
222
-
223
-- AGENTS.md: not changed.
224
-- Runtime project skills: pending if materialization policy changes.
225
-- Specs: pending.
226
-- End-user/operator docs: not expected.
227
-- End-user/operator skills: not expected.
228
-- SOW lifecycle: pending/open.
229
-
230
-Specs update:
231
-
232
-- Pending.
233
-
234
-Project skills update:
235
-
236
-- Pending.
237
-
238
-End-user/operator docs update:
239
-
240
-- Not expected.
241
-
242
-End-user/operator skills update:
243
-
244
-- Not expected.
245
-
246
-Lessons:
247
-
248
-- Pending.
249
-
250
-Follow-up mapping:
251
-
252
-- Pending.
253
-
254
-## Outcome
255
-
256
-Pending.
257
-
258
-## Lessons Extracted
259
-
260
-Pending.
261
-
262
-## Followup
263
-
264
-None yet.
265
-
266
-## Regression Log
267
-
268
-None yet.
.agents/sow/scan-sensitive.sh
new
+108
@@ -0,0 +1,108 @@
1
+#!/usr/bin/env bash
2
+# Scan durable SOW-related artifacts for sensitive values.
3
+# Usage: .agents/sow/scan-sensitive.sh FILE...
4
+
5
+set -uo pipefail
6
+
7
+if [ "$#" -eq 0 ]; then
8
+ echo "usage: $0 FILE..." >&2
9
+ exit 2
10
+fi
11
+
12
+failures=0
13
+
14
+if ! command -v perl >/dev/null 2>&1; then
15
+ echo "perl is required for sensitive-data scanning" >&2
16
+ exit 2
17
+fi
18
+
19
+scan_sensitive_file() {
20
+ local file="$1"
21
+ perl -ne '
22
+ chomp;
23
+ my $line = $_;
24
+ my @hits;
25
+ next if $ARGV =~ m{(^|/)\.agents/sow/scan-sensitive\.sh$}
26
+ && $line =~ /^\s*(?:push \@hits, "[^"]+"\s+(?:if|unless)\s+\$line =~|if \(\$line =~)/;
27
+
28
+ sub is_public_customer_ip {
29
+ my ($ip) = @_;
30
+ my @o = split(/\./, $ip);
31
+ return 0 unless @o == 4;
32
+ return 0 if grep { $_ !~ /^\d+$/ || $_ < 0 || $_ > 255 } @o;
33
+ return 0 if $o[0] == 10;
34
+ return 0 if $o[0] == 172 && $o[1] >= 16 && $o[1] <= 31;
35
+ return 0 if $o[0] == 192 && $o[1] == 168;
36
+ return 0 if $o[0] == 127;
37
+ return 0 if $o[0] == 169 && $o[1] == 254;
38
+ return 0 if $o[0] == 100 && $o[1] >= 64 && $o[1] <= 127;
39
+ return 0 if $o[0] == 0;
40
+ return 0 if $o[0] >= 224;
41
+ return 0 if $o[0] == 192 && $o[1] == 0 && $o[2] == 2;
42
+ return 0 if $o[0] == 198 && $o[1] == 51 && $o[2] == 100;
43
+ return 0 if $o[0] == 203 && $o[1] == 0 && $o[2] == 113;
44
+ return 1;
45
+ }
46
+
47
+ push @hits, "private-key-material" if $line =~ /-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----/;
48
+ push @hits, "aws-access-key" if $line =~ /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/;
49
+ push @hits, "github-token" if $line =~ /\b(?:github_pat_[A-Za-z0-9_]{20,}|gh[pousr]_[A-Za-z0-9_]{20,})\b/;
50
+ push @hits, "slack-token" if $line =~ /\bxox[baprs]-[A-Za-z0-9-]{20,}\b/;
51
+ push @hits, "openai-key" if $line =~ /\bsk-(?:proj-)?[A-Za-z0-9_-]{20,}\b/;
52
+ push @hits, "google-api-key" if $line =~ /\bAIza[0-9A-Za-z_-]{20,}\b/;
53
+ push @hits, "jwt" if $line =~ /\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/;
54
+ push @hits, "credentialed-url" if $line =~ m{[a-z][a-z0-9+.-]*://[^/\s:@]+:[^/\s:@]+@}i && $line !~ m{x-access-token:(?:TOK|TOKEN|REDACTED|EXAMPLE|PLACEHOLDER|YOUR[_-]?TOKEN)\@github\.com}i;
55
+ push @hits, "bearer-token" if $line =~ /\bBearer\s+[A-Za-z0-9._~+\/=-]{16,}\b/i && $line !~ /\b(REDACTED|EXAMPLE|PLACEHOLDER|YOUR[_-]?(?:TOKEN|ACCESS[_-]?TOKEN|BEARER[_-]?TOKEN))\b/i;
56
+
57
+ if ($line =~ /\b(?:pass(?:word)?|passwd|pwd|api[_-]?key|secret|token|client[_-]?secret|private[_-]?key|access[_-]?key)\b\s*[:=]\s*["'\''`]?([^"'\''`\s<>{}\[\]&,]{8,})/i) {
58
+ my $value = lc $1;
59
+ push @hits, "credential-assignment" unless $value =~ /^(redacted|example|placeholder|changeme|change-me|xxx|xxxx|null|none|your[_-]?|dummy|sample|fake|test)/ || $value =~ /^\$/ || $value =~ /^(config|settings|options|opts|env|process\.env|os\.environ)\./ || $value =~ /^[a-z_][a-z0-9_.]*(token|secret|key|password)[a-z0-9_.]*$/ || $value =~ /^(tok|token)\@github\.com\b/;
60
+ }
61
+
62
+ if ($line =~ /\b(?:snmp[_-]?)?(?:community|community[_-]?string|rocommunity|rwcommunity)\b\s*[:=]\s*["'\''`]?([^"'\''`\s<>{}\[\]]{3,})/i) {
63
+ my $value = lc $1;
64
+ push @hits, "snmp-community" unless $value =~ /^(redacted|example|placeholder|changeme|change-me|xxx|xxxx|null|none)$/;
65
+ }
66
+
67
+ if ($line =~ /\b(?:customer|client|tenant|account|organization|org|community[ _-]?member)[ _-](?:name|id|identifier)\b\s*[:=]\s*["'\''`]?([^"'\''`<>\[\]{}][^"'\''`<>\[\]{}]{2,})/i) {
68
+ my $value = $1;
69
+ $value =~ s/^\s+|\s+$//g;
70
+ push @hits, "customer-or-private-identifier" unless $value =~ /^(redacted|example|placeholder|customer-|client-|tenant-|account-|org-|user|none|null)/i;
71
+ }
72
+
73
+ if ($line =~ /\b[A-Z0-9._%+\-]+@[A-Z0-9.\-]+\.[A-Z]{2,}\b/i) {
74
+ push @hits, "email-address" unless $line =~ /\b(example\.com|example\.org|example\.net|localhost)\b/i || $line =~ /\bgit\@github\.com[:\/]/i || $line =~ /x-access-token:(?:TOK|TOKEN|REDACTED|EXAMPLE|PLACEHOLDER|YOUR[_-]?TOKEN)\@github\.com/i;
75
+ }
76
+
77
+ if ($line =~ /\b(customer|client|tenant|account|community member|support|production|prod|log|trace|request|source ip|remote ip|x-forwarded-for|host ip)\b/i) {
78
+ while ($line =~ /\b((?:\d{1,3}\.){3}\d{1,3})\b/g) {
79
+ push @hits, "public-ip-address" if is_public_customer_ip($1);
80
+ }
81
+ }
82
+
83
+ for my $hit (@hits) {
84
+ print "$ARGV:$.:$hit\n";
85
+ }
86
+ ' "$file"
87
+}
88
+
89
+for file in "$@"; do
90
+ if [ ! -f "$file" ]; then
91
+ echo "missing file: $file" >&2
92
+ failures=$((failures + 1))
93
+ continue
94
+ fi
95
+
96
+ if ! hits=$(scan_sensitive_file "$file"); then
97
+ echo "failed to scan file: $file" >&2
98
+ failures=$((failures + 1))
99
+ continue
100
+ fi
101
+
102
+ if [ -n "$hits" ]; then
103
+ printf '%s\n' "$hits"
104
+ failures=$((failures + 1))
105
+ fi
106
+done
107
+
108
+exit "$failures"
.agents/sow/specs/README.md
new
+38
@@ -0,0 +1,38 @@
1
+# SOW Specs Index
2
+
3
+Specs are durable project memory: current contracts, cross-cutting behavioral
4
+rules, and area-specific guarantees extracted from completed work.
5
+
6
+Keep this directory flat until scale proves hierarchy is needed. Add one row to
7
+this index in the same change that adds, renames, or removes a spec.
8
+
9
+## Specs
10
+
11
+| Spec | Scope | Main consumers |
12
+|---|---|---|
13
+| [go-v2-host-scope.md](go-v2-host-scope.md) | Go framework V2 host-scope and vnode metric routing contract. | Go V2 collector work, `metrix`, `jobruntime`, `chartengine`, go.d collector docs and skills. |
14
+| [query-planner-tier-selection.md](query-planner-tier-selection.md) | Automatic storage-tier selection for metric query planning. | Query planner code, API data paths, MCP metric query behavior. |
15
+| [sensitive-data-discipline.md](sensitive-data-discipline.md) | Sensitive-data rules for committed artifacts, scripts, skills, specs, SOWs, commit messages, and PR text. | All repository work, SOW audits, private/public skills, token-safe scripts. |
16
+| [snmp-profile-projection.md](snmp-profile-projection.md) | SNMP profile projection contract across metrics, topology, licensing, and BGP consumers. | SNMP profile authoring, ddsnmp loader/projection code, SNMP project skill. |
17
+| [taxonomy.md](taxonomy.md) | Collector chart taxonomy source, validation, and generated artifact contract. | Collector taxonomy files, integrations generators/checkers, Cloud dashboard taxonomy consumption. |
18
+| [topology-function-schema.md](topology-function-schema.md) | Production `netdata.topology.v1` Function payload, presentation, modal, and producer contract. | Topology producers, Function schemas, Cloud aggregation, UI rendering, topology project skill. |
19
+| [topology-modes-correlation-aggregation.md](topology-modes-correlation-aggregation.md) | Topology detailed/aggregated modes, correlation, aggregation, and actor identification contract. | Topology producers, Cloud aggregator, UI modal/table behavior. |
20
+| [vsphere-parity-matrix.md](vsphere-parity-matrix.md) | vSphere collector parity classification and implementation policy. | vSphere collector work, Go V2 migration decisions, reviewer parity checks. |
21
+| [vsphere-v1-compatibility-manifest.md](vsphere-v1-compatibility-manifest.md) | Historical vSphere V1 compatibility baseline retained for migration traceability. | vSphere collector compatibility review and historical reference. |
22
+
23
+## Adding A Spec
24
+
25
+- Use a flat path: `.agents/sow/specs/<domain>-<topic>.md`.
26
+- Capture current reality, not aspiration.
27
+- Prefer one durable contract or cross-cutting rule per file.
28
+- Do not restate implementation details that are already clear from code.
29
+- Do not split specs by repository path; choose the contract audience.
30
+- Update this index in the same change.
31
+- Run `.agents/sow/audit.sh` before commit.
32
+
33
+## Reconsidering Hierarchy
34
+
35
+Do not add subdirectories only for aesthetics. Revisit hierarchy when the specs
36
+set grows beyond roughly 25 files, or when one domain accumulates roughly 6 or
37
+more specs. If hierarchy becomes necessary, use domain/contract ownership rather
38
+than repository layout.
.agents/sow/specs/query-planner-tier-selection.md
+1
-2
@@ -2,7 +2,7 @@
2
3
## Status
4
5
-Active. Added by SOW-0017.
5
+Active. Added by PR #22495 / commit `2c44acdef`.
6
7
## Scope
8
@@ -75,4 +75,3 @@ target metrics.
75
selection and planner gap-fill.
76
- `src/web/api/queries/query-window.c` - selected-tier natural-points
77
update-every calculation.
78
-
.agents/sow/specs/sensitive-data-discipline.md
+9
-10
@@ -82,11 +82,10 @@ For every reference to a value covered above, use ONE of:
82
83
## Required env keys
84
85
-These keys MUST be defined in `<repo>/.env` (gitignored) for the
86
-SOW family from SOW-0010 onward to function. If a SOW or script
87
-references one and the key is unset, the script must error loudly
88
-and exit non-zero. Values live ONLY in `.env`; this spec lists
89
-names and roles only.
85
+These keys MUST be defined in `<repo>/.env` (gitignored) when a SOW,
86
+skill, or script references them. If a SOW or script references one and
87
+the key is unset, the script must error loudly and exit non-zero. Values
88
+live ONLY in `.env`; this spec lists names and roles only.
89
90
| Key | Role |
91
|---|---|
@@ -189,8 +188,8 @@ project reports its own data).
188
189
## Failure mode
190
192
-If a verification grep returns a hit on a committed file, the SOW
193
-that introduced it has failed its Sensitive Data Gate and must be
194
-treated as a regression. Re-open the SOW, redact, force-push only
195
-with explicit user approval (otherwise create a follow-up commit
196
-that scrubs).
191
+If verification returns a hit on a committed file, the work that introduced it
192
+failed its Sensitive Data Gate and must be treated as a regression. Create a new
193
+branch-local SOW that links the prior PR/commit, redact the durable artifact,
194
+and force-push only with explicit user approval. Otherwise, create a follow-up
195
+commit that scrubs the leak.
.agents/sow/specs/topology-modes-correlation-aggregation.md
+6
-5
@@ -2,13 +2,14 @@
2
3
## Status
4
5
-Implementation contract introduced by SOW-0028.
5
+Implementation contract introduced by topology v1 producer work in PR #22496 /
6
+commit `8afe52d8b`.
7
7
-SOW-0028 completed the cross-repo compatibility layer for modes, modal
8
+PR #22496 completed the cross-repo compatibility layer for modes, modal
9
identification, correlation classes, and table merge policy. The stronger
9
-detailed network-connections loose-side graph model remains the target
10
-behavior and is tracked by SOW-0029 because it requires a separate
11
-Agent/UI/aggregator execution pass.
10
+detailed network-connections loose-side graph model remains target behavior,
11
+but requires a separate Agent/UI/aggregator execution pass before it becomes
12
+current contract.
13
14
## Purpose
15
.agents/sow/specs/vsphere-parity-matrix.md
+2
-1
@@ -1,6 +1,7 @@
1
# vSphere Collector Parity Matrix
2
3
-Status: draft baseline for `SOW-0015`.
3
+Status: draft baseline from the vSphere V2 migration in PR #22458 /
4
+commit `18991fd4e`.
5
6
Purpose: normalize LogicMonitor, Datadog, and mirrored open-source vSphere
7
coverage into Netdata implementation groups. Every row is classified; there are
.agents/sow/specs/vsphere-v1-compatibility-manifest.md
+3
-2
@@ -1,6 +1,7 @@
1
# vSphere Collector V1 Compatibility Manifest
2
3
-Status: superseded historical baseline for `SOW-0015`.
3
+Status: superseded historical baseline from the vSphere V2 migration in
4
+PR #22458 / commit `18991fd4e`.
5
6
The executable V1 golden fixture
7
`src/go/plugin/go.d/collector/vsphere/testdata/v1_compat_manifest.json` and
@@ -92,7 +93,7 @@ Removed executable golden scope:
93
94
This table records the migration baseline before later cleanup decisions. It is
95
not the authoritative current accepted surface after the 2026-05-20 and
95
-2026-05-22 removals documented in `SOW-0015`.
96
+2026-05-22 removals documented in PR #22458 / commit `18991fd4e`.
97
98
| YAML key | JSON key | Required by schema | Notes |
99
|---|---|---:|---|
.github/workflows/sow.yml
new
+72
@@ -0,0 +1,72 @@
1
+name: SOW
2
+
3
+on:
4
+ pull_request:
5
+ branches:
6
+ - master
7
+ paths:
8
+ - ".agents/sow/**"
9
+ - ".agents/skills/**"
10
+ - ".agents/skill-verification/**"
11
+ - ".agents/ENV.md"
12
+ - "AGENTS.md"
13
+ - "CLAUDE.md"
14
+ - "GEMINI.md"
15
+
16
+permissions:
17
+ contents: read
18
+
19
+jobs:
20
+ no-working-files:
21
+ runs-on: ubuntu-latest
22
+ steps:
23
+ - uses: actions/checkout@v4
24
+
25
+ - name: Reject committed SOW working files
26
+ run: |
27
+ set -euo pipefail
28
+
29
+ found=0
30
+
31
+ while IFS= read -r file; do
32
+ [ -n "$file" ] || continue
33
+ echo "::error file=${file}::SOW working files are branch-local and must be deleted before merge."
34
+ found=1
35
+ done < <(
36
+ {
37
+ find .agents/sow/active -maxdepth 1 -type f -name 'SOW-*.md' 2>/dev/null || true
38
+ find .agents/sow/pending .agents/sow/current .agents/sow/done -type f -name 'SOW-*.md' 2>/dev/null || true
39
+ } | sort
40
+ )
41
+
42
+ exit "$found"
43
+
44
+ sensitive-data:
45
+ runs-on: ubuntu-latest
46
+ steps:
47
+ - uses: actions/checkout@v4
48
+ with:
49
+ fetch-depth: 0
50
+
51
+ - name: Scan changed durable artifacts for sensitive data
52
+ run: |
53
+ set -euo pipefail
54
+
55
+ base="${{ github.event.pull_request.base.sha }}"
56
+ head="${{ github.event.pull_request.head.sha }}"
57
+
58
+ mapfile -t files < <(
59
+ git diff --name-only --diff-filter=ACMR "$base...$head" -- \
60
+ .agents/sow .agents/skills .agents/skill-verification .agents/ENV.md \
61
+ AGENTS.md CLAUDE.md GEMINI.md \
62
+ | while IFS= read -r file; do
63
+ [ -f "$file" ] && printf '%s\n' "$file"
64
+ done
65
+ )
66
+
67
+ if [ "${#files[@]}" -eq 0 ]; then
68
+ echo "No changed durable artifacts to scan."
69
+ exit 0
70
+ fi
71
+
72
+ bash .agents/sow/scan-sensitive.sh "${files[@]}"
AGENTS.md
+97
-36
@@ -65,7 +65,13 @@ Project SOW status: initialized
65
66
This project uses a local Statement of Work system.
67
68
-The SOW system is self-contained in this repository. Normal SOW work must not depend on `~/.agents`, `~/.AGENTS.md`, global skills, global templates, or global scripts. Use this `AGENTS.md`, project-local SOW files, project-local specs, project-local skills, and the active SOW.
68
+SOWs are branch-local working memory, not product artifacts. A SOW lives on the
69
+feature branch for the duration of the work so it preserves the root-cause
70
+model, decisions, evidence, and validation for PR takeover. It is removed before
71
+the branch merges. `master` MUST contain no SOW working files; durable memory
72
+belongs in `.agents/sow/specs/`, project skills, docs, code, and tests.
73
+
74
+The SOW system is self-contained in this repository. Normal SOW work must not depend on `~/.agents`, `~/.AGENTS.md`, global skills, global templates, or global scripts. Use this `AGENTS.md`, the branch-local SOW, project-local specs, and project-local skills.
75
76
### Roles
77
@@ -76,7 +82,7 @@ The SOW system is self-contained in this repository. Normal SOW work must not de
82
83
Before non-trivial work:
84
79
-1. Read pending/current SOWs for overlap, contradictions, and existing decisions.
85
+1. Read the current branch's SOW under `.agents/sow/active/` if one exists. Since SOWs are branch-local, discover other in-flight work through open PRs and issues, not through `master`.
86
2. Read relevant specs under `.agents/sow/specs/`.
87
3. Inspect `.agents/skills/*/SKILL.md` if any exist, and load every runtime project skill whose trigger matches the work.
88
4. Inspect legacy runtime skills listed below when the user request matches their frontmatter trigger.
@@ -117,7 +123,7 @@ Resolve `owner/repo` from the repository remote, record the checked commit, and
123
124
### Pre-Implementation Gate
125
120
-Implementation must not begin until the active SOW contains a concrete `## Pre-Implementation Gate` section. Before moving a SOW from `pending/open` to `current/in-progress`, or before continuing implementation in an existing current SOW that lacks this section, fill the gate.
126
+Implementation must not begin until the branch-local SOW contains a concrete `## Pre-Implementation Gate` section with `Status: ready` or `Status: in-progress`. Before changing implementation files, or before continuing implementation in an existing SOW that lacks this section, fill the gate.
127
128
The gate must record the problem/root-cause model, evidence reviewed, affected contracts and surfaces, existing patterns to reuse, risk and blast radius, sensitive data handling plan, implementation plan, validation plan, artifact impact plan, and open decisions. The sensitive data plan must cover SOWs, specs, documentation, project skills, agent instructions, and code comments. Generic placeholders such as `TBD`, `N/A`, or "to be checked later" are invalid unless the SOW explains why the item truly does not apply. If the gate exposes an unknown that cannot be resolved by investigation, stop and ask the user before implementation.
129
@@ -149,45 +155,92 @@ Trivial work does not need a SOW:
155
156
When unsure, treat the work as non-trivial.
157
152
-### SOW Locations
158
+### SOW Locations And Naming
159
154
-- Pending: `.agents/sow/pending/`
155
-- Current: `.agents/sow/current/`
156
-- Done: `.agents/sow/done/`
160
+- Active branch-local SOWs: `.agents/sow/active/`
161
- Specs: `.agents/sow/specs/`
162
- Template for new SOWs: `.agents/sow/SOW.template.md`
163
- Local audit: `.agents/sow/audit.sh`
164
165
+There is no `done/` directory and no committed pending queue. On `master`,
166
+`.agents/sow/active/` is empty except for `.gitkeep`; real SOW files exist only
167
+on feature branches and are deleted before merge.
168
+
169
Create new SOW files from `.agents/sow/SOW.template.md`. The template is project-local and may be customized for this repository.
170
171
Empty SOW directories must contain `.gitkeep` or `.keep` so the committed repository preserves the full SOW layout after clone/checkout.
172
173
+### Local SOW Parking
174
+
175
+Users may keep private paused, abandoned, or not-yet-public SOW drafts under
176
+`<repo-root>/.local/sow/`. This directory is gitignored and outside the project
177
+SOW lifecycle.
178
+
179
+Use `<repo-root>/.local/sow/` when the user wants to preserve work locally
180
+without creating a public or team-visible GitHub issue yet.
181
+
182
+Local parked SOWs are private memory only:
183
+
184
+- they are not durable project memory;
185
+- they are not visible to other contributors;
186
+- they are not acceptable as the only tracking for work that must coordinate a
187
+ team, block a merge, or survive across machines.
188
+
189
+Deferred work has two valid tracking paths:
190
+
191
+- public or team-visible follow-up: GitHub issue;
192
+- private or local follow-up: `<repo-root>/.local/sow/`.
193
+
194
+Active implementation work still MUST use `.agents/sow/active/`, and active SOW
195
+files still MUST be deleted before merge.
196
+
197
Filename:
198
199
```text
168
-SOW-NNNN-YYYYMMDD-{slug}.md
200
+SOW-YYYYMMDD-{slug}.md
201
```
202
171
-Status and directory must agree:
203
+Use the creation date plus a descriptive slug. There is no sequential `NNNN`
204
+counter because it cannot be allocated safely across parallel branches.
205
173
-- `open` lives in `pending/`
174
-- `in-progress` lives in `current/`
175
-- `paused` lives in `current/`
176
-- `completed` lives in `done/`
177
-- `closed` lives in `done/`
206
+SOW state lives in the file's `Status:` field:
207
179
-### SOW Completion And Commit
208
+- `planning` - analysis or decisions are incomplete; implementation is blocked.
209
+- `ready` - the Pre-Implementation Gate is complete and implementation can start.
210
+- `in-progress` - implementation is underway.
211
+- `paused` - work is intentionally stopped but may resume on the branch.
212
+- `completed` - work is validated and durable memory has been transferred; this is a transient state before deleting the SOW file.
213
181
-The successful terminal SOW status is `completed`. `done` is a directory name, not a status value. Never write `Status: done` or `Status: complete`.
214
+### SOW Completion And Merge
215
183
-When a SOW's work is ready to close:
216
+The successful terminal SOW status is `completed`.
217
+
218
+When a SOW's work is ready to merge:
219
220
1. Finish implementation, docs, specs, skills, validation, and follow-up mapping.
186
-2. Update the SOW to `Status: completed`.
187
-3. Move the SOW file to `.agents/sow/done/`.
188
-4. Commit the work, artifact updates, SOW status change, and SOW move together as one commit, unless the user explicitly requested a different commit split.
221
+2. Transfer all durable knowledge into `.agents/sow/specs/`, project skills, docs, code, and tests. After this step, the SOW body MUST hold nothing durable that is not captured elsewhere.
222
+3. Update the SOW to `Status: completed`.
223
+4. Delete the SOW working file before merge.
224
+
225
+The branch HEAD that merges MUST contain no `.agents/sow/active/SOW-*.md` file. CI enforces this.
226
190
-Do not create a separate commit just to mark or move the SOW. Do not claim a SOW is completed while the implementation and the SOW lifecycle change live in separate uncommitted or separately committed states.
227
+### Enforcement
228
+
229
+The SOW system is enforced by local audit tooling and CI:
230
+
231
+- `.agents/sow/audit.sh` is the local consistency audit for SOW rules, specs,
232
+ references, and sensitive-data scanning.
233
+- `.agents/sow/scan-sensitive.sh` is the shared sensitive-data scanner used by
234
+ local audit and CI.
235
+- `.github/workflows/sow.yml` rejects pull requests that contain branch-local
236
+ SOW working files under `.agents/sow/active/SOW-*.md` or legacy SOW working
237
+ files under `.agents/sow/{pending,current,done}/SOW-*.md`.
238
+- The same workflow scans changed SOW, spec, instruction, and cross-tool
239
+ bridge files for raw sensitive data.
240
+
241
+These checks are guards, not substitutes for the SOW Validation Gate. The
242
+assistant still owns transferring durable knowledge out of the SOW before
243
+merge.
244
245
### One SOW At A Time
246
@@ -195,7 +248,8 @@ Never execute multiple SOWs as one batch.
248
249
If work overlaps:
250
198
-- merge or consolidate before implementation; or
251
+- coordinate through the relevant open PRs and issues;
252
+- merge or consolidate branches before implementation; or
253
- split into separate SOWs and complete one before starting the next.
254
255
Progress reports are not stop points. Once a SOW is in progress, continue until it is delivered, failed with evidence, blocked on a real user decision/approval, or superseded by newer user instructions.
@@ -218,7 +272,7 @@ Before a SOW can close, every valid deferred item must be:
272
273
- implemented in the current SOW; or
274
- explicitly rejected as not worth doing, with evidence; or
221
-- represented by a real pending/current SOW file.
275
+- represented by a GitHub issue linked from the current SOW or PR.
276
277
Pre-close, search the SOW for:
278
@@ -230,19 +284,20 @@ Map every remaining item to implemented, rejected, or tracked.
284
285
### Regressions
286
233
-A regression is discovered after a SOW was considered completed or closed, later testing or use finds broken behavior, and the original SOW's claimed outcome is no longer true.
287
+A regression is broken behavior discovered after a SOW's work merged, where the
288
+original claimed outcome is no longer true.
289
235
-When behavior that a completed SOW claimed working stops working:
290
+Because completed SOWs are not retained on `master`, a regression is handled as
291
+new work:
292
237
-1. Find the original SOW in `done/`.
238
-2. Move it back to `current/`.
239
-3. Mark it `in-progress` with a regression note in `## Status`.
240
-4. Append a new dated `## Regression - YYYY-MM-DD` section at the end of the file, after the original outcome, lessons, and follow-up content.
241
-5. In that appended section, record what broke, evidence, why previous validation missed it, the repair plan, validation, and updates needed to specs, skills, docs, audits, or follow-up SOWs.
242
-6. Fix and validate there.
293
+1. Open a new branch-local SOW under `.agents/sow/active/`.
294
+2. In `## Requirements`, link the prior work: `Regresses: PR #NNNNN` and cite
295
+ any known commit, spec, issue, or test evidence.
296
+3. Run the normal Pre-Implementation Gate and Validation for the new SOW.
297
+4. Update the relevant spec, skill, doc, code, or test so durable memory reflects
298
+ current reality.
299
244
-Never prepend regression content above the original SOW narrative. The original requirements, analysis, plan, validation, outcome, lessons, and follow-up must remain readable first.
245
-Do not create a new SOW for a true regression.
300
+Do not attempt to resurrect or mutate a prior SOW.
301
302
### Validation Gate
303
@@ -254,7 +309,7 @@ A SOW cannot be completed until Validation records:
309
- reviewer findings and how they were handled;
310
- same-failure search results;
311
- artifact maintenance gate for `AGENTS.md`, runtime project skills, specs, end-user/operator docs, end-user/operator skills, and SOW lifecycle;
257
-- SOW status/directory consistency;
312
+- SOW working file removed before merge;
313
- spec update or specific reason no spec update was needed;
314
- project skill update or specific reason no skill update was needed;
315
- end-user/operator docs update or evidence-backed reason none were affected;
@@ -273,7 +328,7 @@ Every SOW close must explicitly record whether each durable artifact class was u
328
- Specs - `.agents/sow/specs/` for WHAT the project does.
329
- End-user/operator docs - README, docs site, runbooks, published guides, help text, or other human-facing documentation.
330
- End-user/operator skills - output/reference skills copied or consumed outside normal repo work.
276
-- SOW lifecycle - split, merge, status, directory, deferred work, regression reopening, and follow-up mapping.
331
+- SOW lifecycle - branch-local active SOW, durable memory transfer, SOW deletion before merge, deferred work tracked as GitHub issues, and regressions handled as new linked SOWs.
332
333
This is an assistant responsibility. If a SOW changes behavior, docs, specs, commands, schemas, defaults, workflows, examples, or operating procedure, the assistant must update every affected artifact in the same SOW, or record the evidence-backed reason an artifact is unaffected.
334
@@ -283,6 +338,12 @@ Specs are memory of WHAT this project does.
338
339
This repository is bootstrapped incrementally. The existing source tree and public documentation remain the primary ground truth. SOW specs under `.agents/sow/specs/` should capture durable project decisions, cross-cutting behavioral rules, and area-specific contracts as they are worked.
340
341
+`.agents/sow/specs/` stays flat until scale proves hierarchy is needed. Use
342
+`<domain>-<topic>.md` names, one durable contract or cross-cutting rule per file,
343
+and update `.agents/sow/specs/README.md` in the same change. Do not split specs
344
+by repository path; specs are organized by contract ownership, not source-file
345
+location.
346
+
347
Update specs when shipped work changes:
348
349
- product behavior;
@@ -400,7 +461,7 @@ Runtime input skills:
461
462
- `.agents/skills/codacy-audit/`
463
Trigger: Codacy Cloud workflow for this repository -- pre-push local analysis (`codacy-analysis-cli` via docker or local binary) and read-only PR-issue fetching via the v3 API.
403
- Status: live. SKILL.md plus `scripts/_lib.sh` (token-safe wrappers + sentinel no-leak self-test), `scripts/analyze-local.sh`, `scripts/pr-issues.sh`, and a live `how-tos/INDEX.md` catalog. Read-only by design; write actions deferred to a future SOW.
464
+ Status: live. SKILL.md plus `scripts/_lib.sh` (token-safe wrappers + sentinel no-leak self-test), `scripts/analyze-local.sh`, `scripts/pr-issues.sh`, and a live `how-tos/INDEX.md` catalog. Read-only by design; write actions require a GitHub issue or branch-local SOW.
465
466
Public skills (canonical under `docs/netdata-ai/skills/<name>/`; relative symlinks at `.agents/skills/<name>`):
467
src/plugins.d/FUNCTION_TOPOLOGY_IMPLEMENTATION_SCOPE.md
+1
-1
@@ -220,7 +220,7 @@ Current state:
220
not as actor-owned duplicated modal data;
221
- link and evidence string columns choose dictionary encoding only when it
222
reduces raw payload size;
223
-- SOW-0023 semantic-link split and correlation endpoint/point/claim emission
223
+- PR #22496 semantic-link split and correlation endpoint/point/claim emission
224
are implemented in the Agent producer;
225
- remaining network-connections work is corpus-scale validation with captured
226
Cloud payloads and Cloud/frontend integration once the parallel workers are
src/streaming/STREAM_PATH.md
+14
-12
@@ -23,8 +23,9 @@ Helper for read access by index: `rrdhost_stream_path_get_host_ids(host,
23
from, host_ids, max)` (returns just the UUIDs starting at `from`).
24
25
For full per-entry data with a callback: `rrdhost_stream_path_visit(host,
26
-from, cb, userdata)` (added by SOW-0012; mirrors the get_host_ids locking
27
-pattern but exposes hostname, hops, since, capabilities, flags, etc.).
26
+from, cb, userdata)` (added by PR #22432 / commit `cc50307bc`; mirrors the
27
+get_host_ids locking pattern but exposes hostname, hops, since, capabilities,
28
+flags, etc.).
29
30
## 3. Storage rule (received vs emitted)
31
@@ -132,17 +133,18 @@ Known consumers of `host->stream.path.array` in this tree:
133
- `src/web/api/functions/function-topology-streaming.c` — the
134
`topology:streaming` Function. Uses both `streaming_topology_get_path_ids`
135
(paths for actor/link emission) and `rrdhost_stream_path_visit` (Bug C
135
- synthesis from SOW-0012). Reads stored data, falls back to the
136
+ synthesis from PR #22432). Reads stored data, falls back to the
137
rrdhost-derived live data via `rrdhost_status()` for the localhost
137
- classification (Bug A in SOW-0012).
138
+ classification (Bug A in PR #22432).
139
- `src/database/contexts/api_v2_contexts.c:425, :510` — emits
140
`streaming_path` per host via `rrdhost_stream_path_to_json` (so the
141
emit-time self-append fires). No direct array access.
142
- `src/streaming/stream-path.c:145-158` — `rrdhost_stream_path_total_reboot_time_ms`
143
walks the stored array looking for localhost's own entry. **This shares
144
Bug A's blind assumption**: on the apex parent, the localhost entry is
144
- not in storage, so this function returns 0 silently. Tracked as a
145
- same-failure follow-up in SOW-0012.
145
+ not in storage, so this function returns 0 silently. This is the same
146
+ failure class as the PR #22432 streaming topology fix and needs a separate
147
+ GitHub issue before implementation starts.
148
149
## 8. vnode special case
150
@@ -160,7 +162,7 @@ visits via `rrdhost_stream_path_visit` return zero entries.
162
163
The `topology:streaming` Function classifies each entry in
164
`rrdhost_root_index` as `parent`, `child`, `vnode`, or `stale`. The intended
163
-rule (project-owner confirmed in SOW-0012):
165
+rule was confirmed during PR #22432:
166
167
1. **Source of truth = `rrdhost_root_index`.** Every actor on the
168
topology graph corresponds to either an entry in this index OR a
@@ -177,7 +179,7 @@ rule (project-owner confirmed in SOW-0012):
179
For the localhost itself, classification reads live state via
180
`rrdhost_status()` (count of non-virtual children with `s.ingest.type ==
181
CHILD` and `s.ingest.status` ∈ `{ONLINE, REPLICATING}`). This was Bug A in
180
-SOW-0012 — the path-based check was unreliable on apex agents because
182
+PR #22432 — the path-based check was unreliable on apex agents because
183
storage hadn't converged yet.
184
185
For the cross-agent merge (Cloud combines topology responses from multiple
@@ -187,7 +189,7 @@ actor_id form (`netdata-machine-guid:<agent_id>`) and uses that to identify
189
which actor in `actors[]` is the response's "self" — that response is the
190
authoritative source for that actor's attributes.
191
190
-For the bug history, decisions, and design rationale, see
191
-[`SOW-0012`](../../.agents/sow/done/SOW-0012-20260505-streaming-topology-classification-bugs.md)
192
-once it lands in `done/`. While the SOW is in progress it lives at
193
-`.agents/sow/current/`.
192
+For bug history, decisions, and design rationale, use PR #22432 / commit
193
+`cc50307bc6ca180285a0e0cda16d73ab7a42cd86`. Completed SOW working files are
194
+not retained on `master`; durable behavior belongs in this maintenance
195
+reference.