@cryptotaxi247 / netdata
Code Commits Issues Pulls Sessions CI/CD
master
master
md 427 lines 18.3 KB
Rendered Raw
1 # How to Write a go.d Collector (V2)
2
3 This is the canonical starting point for new go.d collectors. New collectors
4 MUST use framework V2. V1 collectors remain in the tree for compatibility and
5 maintenance only.
6
7 For migrating an existing V1 collector, use
8 `src/go/plugin/go.d/docs/migrate-v1-to-v2.md` instead. Migration is
9 compatibility work and has different rules from new collector authoring.
10
11 Use `src/go/plugin/go.d/collector/cato_networks/` as the primary modern example.
12 It is large, so copy the pattern, not the whole shape. The useful references are
13 called out below by responsibility.
14
15 ## Before Writing Code
16
17 Do the design work first:
18
19 1. Read the upstream API or protocol docs. Do not infer current behavior from
20 memory or from generated SDK types alone.
21 2. Check existing helper packages before implementing parser, HTTP, selector,
22 command-execution, SQL, ping, log-reading, or log-limiting plumbing. Start
23 with `src/go/plugin/go.d/docs/helper-packages.md`.
24 3. You MUST aim for the clean end state, not the smallest initial diff. If the
25 clean collector design requires a framework improvement, surface that as a
26 design decision and follow
27 `src/go/plugin/framework/docs/changing-framework-code.md` instead of hiding
28 it behind collector-local glue.
29 4. Decide the monitored entities and cardinality bounds. If one job collects
30 remote resources that SHOULD be separate Netdata nodes, design V2 host scopes
31 from the start.
32 5. Decide the minimal public config surface. Public config is a compatibility
33 contract. Use constants for internal tuning such as page limits, scan cadence,
34 retry limits, fan-out concurrency, and cache TTLs unless the operator has a
35 real decision to make. A proposed config option MUST name that concrete
36 operator decision; "operators may want to tune it" is not enough.
37 6. Decide whether the collector needs Functions or topology. Functions are
38 interactive live/snapshot views; metrics are time series. New topology
39 producers MUST use `src/go/pkg/topology/v1` and validate against
40 `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`.
41 7. Plan collector consistency using
42 `.agents/skills/integrations-lifecycle/consistency.md`. Generated
43 integration pages and README symlinks are outputs, not hand-authored sources.
44 8. Plan the first coherent batch and its boundaries. At each boundary, you MUST
45 re-check whether new work has drifted out of scope; defer it or land it
46 independently before continuing.
47
48 ## Source References
49
50 Primary V2 reference:
51
52 - `src/go/plugin/go.d/collector/cato_networks/`
53
54 Read these files by responsibility:
55
56 - `collector.go`: registration, defaults, public lifecycle methods,
57 `MetricStore()`, `ChartTemplateYAML()`, and Function wiring.
58 - `config.go`: config defaults, normalization, validation, and intentionally
59 small public config.
60 - `collect.go`, `collect_metrics.go`, `collect_bgp.go`: collection
61 orchestration and split domain operations.
62 - `metrix.go`, `write_metrics.go`, `charts.yaml`: typed instruments, metric
63 writes, chart template, `StateSet`, `instances.by_labels`, and
64 `label_promotion`. Audit every `instances.by_labels` identity choice instead
65 of copying labels from the example blindly.
66 - `host_scope.go`: deterministic per-site V2 host scopes/vnodes.
67 - `func_deps.go`, `catofunc/`: Function subpackage boundary behind a narrow
68 dependency interface.
69 - `topology_store.go`, `topology.go`, `topology_test.go`: immutable topology
70 snapshot publishing and topology v1 schema validation.
71 - `config_test.go`, `collector_lifecycle_test.go`, `collector_collect_test.go`,
72 `charts_test.go`: table-driven V2 tests and fixture validation.
73
74 Framework/API references:
75
76 - `src/go/plugin/framework/collectorapi/collector.go`
77 - `src/go/plugin/framework/docs/changing-framework-code.md`
78 - `src/go/plugin/go.d/docs/helper-packages.md`
79 - `src/go/pkg/metrix/README.md`
80 - `src/go/plugin/framework/charttpl/README.md`
81 - `src/go/plugin/framework/chartengine/README.md`
82 - `src/go/plugin/framework/functions/README.md`
83 - `src/go/tools/functions-validation/README.md`
84 - `.agents/sow/specs/go-v2-host-scope.md`
85 - `.agents/skills/integrations-lifecycle/consistency.md`
86
87 ## File Layout
88
89 Start with this layout and add focused files only when a responsibility needs
90 its own boundary:
91
92 ```text
93 src/go/plugin/go.d/collector/<name>/
94 |-- collector.go # registration, New, public lifecycle, store/template
95 |-- init.go # Init helper methods for clients/matchers/state
96 |-- config.go # Config, defaults, validation
97 |-- collect.go # Collect orchestration
98 |-- metrix.go # typed metrix instruments built once in New
99 |-- write_metrics.go # normalized state -> metrix observations
100 |-- models.go # collector-local state/DTOs
101 |-- client.go # API/client boundary
102 |-- charts.yaml # V2 chart template
103 |-- config_schema.json # DYNCFG schema
104 |-- metadata.yaml # integration metadata source
105 |-- taxonomy.yaml # dashboard TOC placement source
106 |-- integrations/ # generated integration page
107 |-- README.md # symlink to generated integration page
108 |-- testdata/ # fixtures and config serialization files
109 `-- *_test.go # table-driven tests
110 ```
111
112 Common optional splits:
113
114 Most collectors need none of these optional files. Add one only when the
115 collector has the corresponding product surface or state boundary; do not create
116 empty `host_scope.go`, `topology.go`, or `<name>func/` files just because Cato
117 has them.
118
119 - `init.go` when `Init()` needs helper setup for clients, matchers, caches, or
120 other persistent state. Keep the public `Init()` method itself in
121 `collector.go`; let it call focused helpers such as `initClient()` or
122 `initSiteSelector()`.
123 - `collect_<operation>.go` when the collector has multiple distinct collection
124 operations, such as discovery, account metrics, BGP, or inventory.
125 - `normalize_<operation>.go` when API payload normalization would otherwise
126 dominate `collect.go`.
127 - `host_scope.go` when the collector emits generated vnodes.
128 - `<name>func/` plus `func_deps.go` when the collector exposes Functions.
129 - `topology.go` and `topology_store.go` when the collector emits topology.
130
131 Avoid files whose names hide their responsibility. For example, a file named
132 `diagnostics.go` SHOULD NOT contain only error classification.
133
134 ## Registration And Lifecycle
135
136 New collectors MUST implement `collectorapi.CollectorV2` from
137 `src/go/plugin/framework/collectorapi/collector.go` and register via `CreateV2`.
138 In practice, `collector.go` should:
139
140 - embed `config_schema.json` for `JobConfigSchema`;
141 - embed `charts.yaml` for `ChartTemplateYAML()`;
142 - expose `Config: func() any { return &Config{} }`;
143 - return a new collector from `CreateV2`;
144 - add `Methods` and `MethodHandler` only when the collector has Functions.
145
146 `New()` SHOULD own defaults and test seams:
147
148 - create `metrix.NewCollectorStore()`;
149 - build typed instruments once from that store;
150 - set default config values;
151 - set injected seams such as client factories or clocks;
152 - create the Function router when Functions exist.
153
154 Public lifecycle and framework-contract methods MUST stay in `collector.go`:
155
156 - `Configuration() any`
157 - `Init(context.Context) error`
158 - `Check(context.Context) error`
159 - `Collect(context.Context) error`
160 - `Cleanup(context.Context)`
161 - `MetricStore() metrix.CollectorStore`
162 - `ChartTemplateYAML() string`
163
164 `Init()` validates config, prepares matchers/clients, and initializes persistent
165 state. Explicit setup details SHOULD live in helper methods, preferably in
166 `init.go`, so the public method reads as the lifecycle sequence. `Check()` MUST
167 be a cheap auth/connectivity probe, not a full collection. `Collect()` MUST run
168 the real write path through `metrix`. `Cleanup()` closes idle connections and
169 forwards Function cleanup.
170
171 ## Config
172
173 Config SHOULD stay small and operator-oriented:
174
175 - connection identity and credentials;
176 - endpoint and standard HTTP/TLS/proxy fields when applicable;
177 - `update_every`, `timeout`, and `vnode` when relevant;
178 - selectors that let users intentionally scope cardinality.
179
180 Implementation tuning SHOULD use constants:
181
182 - discovery refresh cadence;
183 - page sizes and maximum pages;
184 - per-cycle fan-out concurrency;
185 - cache TTLs;
186 - retry/backoff internals;
187 - API batching constraints.
188
189 You MUST NOT add a config option just because it is easy to expose. Once
190 shipped, it is hard to remove and MUST stay synchronized across `Config`,
191 `config_schema.json`, stock `.conf`, metadata, generated docs, and tests.
192 A proposed config option MUST name the concrete operator decision it enables;
193 "operators may want to tune it" is not enough.
194
195 For SaaS/API credentials, examples SHOULD prefer secret indirection such as
196 `${env:COLLECTOR_API_KEY}` or `${file:/run/secrets/collector_api_key}` instead
197 of realistic-looking inline credentials. Schema fields that carry secrets MUST
198 be marked sensitive and use password-style UI handling where the schema
199 supports it.
200
201 Selectors SHOULD use existing matcher packages such as `src/go/pkg/matcher`
202 unless the upstream API forces a different grammar. Document the exact matching
203 input, for example "site name when present, otherwise site ID."
204
205 ## Collect Flow
206
207 `Collect()` SHOULD stay orchestration, not a large parser. A typical flow is:
208
209 1. ensure the client is initialized;
210 2. refresh stable discovery only when needed;
211 3. fetch the current snapshot/state needed for this cycle;
212 4. enrich with optional or slower data;
213 5. normalize API payloads into collector-local state;
214 6. publish any immutable Function/topology snapshot;
215 7. write metrics to `metrix`.
216
217 When the collector performs several upstream calls or collection operations,
218 those operations SHOULD be split into focused files named by operation, for
219 example `collect_metrics.go` or `collect_bgp.go`. `collect.go` SHOULD explain
220 the cycle; the operation files SHOULD own the operation-specific API calls,
221 fail-soft behavior, and merge rules.
222
223 Fail-soft behavior MUST be used only when partial data is still truthful. If one
224 optional operation fails, log a rate-limited warning and omit or preserve only
225 values that remain honest. If the core operation fails, return an error with
226 context.
227
228 `Collect()` MUST preserve context cancellation. If the context is canceled
229 during a partial path, `Collect()` MUST return the context error so the runtime
230 aborts the cycle instead of committing a stale or partial frame.
231
232 ## Metrics And Charts
233
234 Metric instruments SHOULD be built once in `New()` when the metric surface is
235 known. Use a typed collector metrics struct so write code is a value mapping,
236 not repeated dynamic instrument lookup.
237
238 Use the right instrument:
239
240 - `SnapshotGaugeVec` for labeled current values; use scalar
241 `SnapshotMeter.Gauge` only when the metric is intentionally unlabeled;
242 - `Counter.ObserveTotal()` for source counters;
243 - `StateSet` SHOULD be used for fixed mutually exclusive states, such as
244 connected vs disconnected or up vs down.
245
246 `charts.yaml` is the chart contract. Every template MUST define:
247
248 - `version: v1`;
249 - `context_namespace`;
250
251 Templates SHOULD also group charts by operational area, use
252 `instances.by_labels` for stable instance identity when charts are
253 entity-scoped, use `label_promotion` for descriptive labels that should not
254 define uniqueness, and keep the default lifecycle unless a concrete reason
255 exists to override it.
256
257 Metric labels and chart instance labels MUST be bounded and stable. Use IDs for
258 identity. Mutable display names SHOULD be promoted with `label_promotion`.
259 Do not blindly copy `instances.by_labels` from Cato or any other example; audit
260 every label used for chart identity and record why it is stable enough for that
261 collector.
262
263 ## Host Scopes And Vnodes
264
265 Use `metrix.HostScope` when one job emits data for remote entities that SHOULD
266 appear as separate Netdata nodes. Labels alone are not enough for that product
267 semantics.
268
269 Rules:
270
271 - `ScopeKey` and `GUID` are deterministic and based on stable IDs.
272 - Hostname may use a human-readable name when safe, with stable fallback.
273 - Add `_vnode_type=<source>` and useful source labels.
274 - Route every metric for that remote entity through the same host scope.
275 - Keep the default host scope empty unless the metric truly belongs to the
276 agent/job host.
277
278 Use `.agents/sow/specs/go-v2-host-scope.md` for the framework contract.
279
280 ## Functions
281
282 Functions MUST NOT freely access collector internals. Put Function code in a
283 dedicated subpackage, for example `<name>func/`, with a narrow `Deps` interface
284 declared by that subpackage.
285
286 Non-topology Function responses MUST conform to
287 `src/plugins.d/FUNCTION_UI_SCHEMA.json`; topology Function responses MUST
288 conform to `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`. Function payloads
289 MUST be validated with `src/go/tools/functions-validation/` or an equivalent
290 schema-validation test, and the validation method must be recorded.
291
292 Pattern:
293
294 - collector package owns state and implements a small adapter in `func_deps.go`;
295 - Function package owns method IDs, router, handlers, presentation, and tests;
296 - Function package MUST import only framework/function types and other allowed
297 dependencies, not the collector package;
298 - the Function package `Deps` interface MUST expose only the methods the
299 Function needs and MUST NOT expose, return, or embed `*Collector`;
300 - `Collector.Cleanup()` forwards cleanup to the Function router.
301
302 Use `catofunc/` as the primary example. Test the Function package with fake
303 deps so the boundary is compile-enforced.
304
305 ## Topology
306
307 New topology producers MUST use `src/go/pkg/topology/v1`. The non-v1 root
308 `src/go/pkg/topology` payload model has been retired and MUST NOT be
309 reintroduced for topology payloads.
310
311 Rules:
312
313 - build topology from normalized collector state, not directly from raw API
314 payloads;
315 - publish immutable snapshots for Function readers;
316 - MUST NOT mutate a published topology value;
317 - use `src/go/plugin/go.d/collector/cato_networks/topology.go` as the concrete
318 construction reference for actors, links, detail tables, and telemetry fields;
319 - MUST validate topology payloads in tests with both
320 `topologyv1.ValidateDecodedData` and
321 `src/plugins.d/FUNCTION_TOPOLOGY_SCHEMA.json`; see
322 `src/go/plugin/go.d/collector/cato_networks/topology_test.go`
323 `validateCatoTopologyV1Data` for the full marshal/decode/schema check shape;
324 - follow `.agents/skills/project-create-topology/SKILL.md` for actor/link/table
325 design.
326
327 ## Repository Wiring
328
329 For a new collector `<name>`:
330
331 1. Add the collector package under `src/go/plugin/go.d/collector/<name>/`.
332 2. Import it in `src/go/plugin/go.d/collector/init.go`.
333 3. Add the default stock config:
334 `src/go/plugin/go.d/config/go.d/<name>.conf`.
335 4. Add the module toggle in `src/go/plugin/go.d/config/go.d.conf`.
336 5. Add or update `src/go/plugin/go.d/README.md`.
337 6. Add health alerts under `src/health/health.d/<name>.conf` only when alerts
338 are useful and backed by emitted chart contexts.
339 7. If adding or changing service-discovery rules under
340 `src/go/plugin/go.d/config/go.d/sd/` or `sdext`, update generated
341 service-discovery documentation through the integrations lifecycle recipe.
342 8. Generate `integrations/<slug>.md` and the README symlink from
343 `metadata.yaml`.
344 Single-integration collector directories normally use the symlinked README.
345 Multi-integration plugin directories may keep a hand-authored umbrella
346 README; follow `.agents/skills/integrations-lifecycle/consistency.md`.
347
348 Use `.agents/skills/integrations-lifecycle/recipes/add-go-collector.md` for the
349 integration-generation commands and taxonomy pipeline details.
350
351 The PR description or design note MUST enumerate the relevant collector
352 consistency artifacts and justify every artifact that did not need a matching
353 change. Most of this is not CI-enforced; it must be reviewer-visible.
354
355 ## Tests
356
357 Tests SHOULD be table-driven with `map[string]struct{}` when cases share setup and
358 assertion shape.
359
360 Recommended test coverage:
361
362 - config JSON/YAML serialization with `collecttest.TestConfigurationSerialize`;
363 - config validation, including required credentials and unsafe URLs;
364 - `Init`, cheap `Check`, `Collect`, `Cleanup`, `MetricStore`;
365 - hard failure, partial failure, context cancellation, and recovery behavior;
366 - chart-template schema validation with `collecttest.AssertChartTemplateSchema`
367 and chart-template compile validation through the chartengine path used by
368 nearby V2 collectors;
369 - post-collect chart coverage with `collecttest.AssertChartCoverage`;
370 - state-set values for every known state and unknown fallback;
371 - host-scope routing when scopes/vnodes are used;
372 - Function handler tests with fake deps when Functions exist;
373 - topology schema validation when topology exists;
374 - fixture validity and attribution when fixtures come from public third-party
375 projects.
376
377 Do not let tests depend on real credentials or live services unless the test is
378 explicitly an integration test gated outside the default unit-test path.
379
380 ## Validate Locally
381
382 From `src/go`, run the narrow collector tests:
383
384 ```bash
385 go test -count=1 ./plugin/go.d/collector/<name>/...
386 ```
387
388 Verify that go.d can load the module:
389
390 ```bash
391 timeout 15s go run ./cmd/godplugin -m <name> -d
392 ```
393
394 Success means the module is registered, a job starts, and the command keeps
395 running until the timeout stops it. Treat `unknown module`, `no jobs started`,
396 config-load errors, or an immediate exit before the timeout as failures. Use
397 `-c <config-dir>` when the test config lives outside the normal go.d config
398 search path.
399
400 When the collector uses concurrency or Functions, also run:
401
402 ```bash
403 go test -race -count=1 ./plugin/go.d/collector/<name>/...
404 ```
405
406 When integration metadata, generated pages, taxonomy, or health alerts change,
407 run the relevant integrations pipeline checks from
408 `.agents/skills/integrations-lifecycle/`.
409
410 Do not claim full-project validation from a narrow collector command. State
411 exactly what was run.
412
413 ## Anti-Patterns
414
415 - New collector using `Collect() map[string]int64`.
416 - Full live collection from `Check()`.
417 - Public config knobs for internal implementation details.
418 - Custom selector or retry framework when existing package/framework behavior is
419 enough.
420 - Collector-local singleton, adapter, or glue code that substitutes for a
421 missing shared framework capability.
422 - Per-cycle warning/error logs for recoverable partial failures.
423 - Metric charts for collector internals when logs are enough.
424 - Mutable names used as chart or vnode identity.
425 - Function package holding `*Collector`.
426 - New topology producer using legacy topology payloads.
427 - Hand-written `README.md` when the integration page should be generated.