Artifacts and banners
Every file the integrations pipeline reads or writes, with banner conventions and edit rules.
Tracked vs runtime artifacts
| Path | Producer | Tracked in git? | Auto-regenerated by |
|---|---|---|---|
integrations/integrations.js |
gen_integrations.py:1311-1325 |
NO -- gitignored (.gitignore:162) |
Every CI run; rm-ed by the workflow before opening the PR |
integrations/integrations.json |
gen_integrations.py:1311-1325 (clean variant) |
NO -- gitignored (.gitignore:163) |
as above |
<plugin-dir>/integrations/<slug>.md (per-integration) |
gen_docs_integrations.py |
YES (committed) | generate-integrations.yml auto-PR; verified in check-markdown.yml |
<plugin-dir>/README.md (symlink) |
gen_docs_integrations.py:make_symlinks (:527-544) |
YES (the symlink itself) | as above |
src/health/notifications/<dir>/README.md (DIRECT, not a symlink) |
gen_docs_integrations.py:agent_notification branch (:488-496) |
YES | as above |
src/collectors/COLLECTORS.md |
gen_doc_collector_page.py |
YES | as above |
src/collectors/SECRETS.md |
gen_doc_secrets_page.py |
YES | as above |
src/collectors/SERVICE-DISCOVERY.md |
gen_doc_service_discovery_page.py |
YES | NOT in generate-integrations.yml -- manual or cmake render-docs only -- see gotchas.md |
src/go/plugin/ibm.d/modules/<m>/metadata.yaml |
ibm.d docgen |
YES | manual go generate ./... -- never CI |
src/go/plugin/ibm.d/modules/<m>/README.md |
ibm.d docgen |
YES | as above |
src/go/plugin/ibm.d/modules/<m>/config_schema.json |
ibm.d docgen |
YES | as above |
src/go/plugin/ibm.d/modules/<m>/contexts/zz_generated_contexts.go |
ibm.d metricgen |
YES | as above |
Hand-written metadata.yaml (non-ibm.d), config_schema.json, stock .conf, health.d/<...>.conf, hand-written README.md |
collector author | YES | none -- author edits + commits manually |
<collector-dir>/taxonomy.yaml |
collector author / gen_taxonomy_seed.py starter output |
YES | validated by check-markdown.yml; not auto-authored |
integrations/taxonomy/sections.yaml |
taxonomy framework author | YES | validated by gen_taxonomy.py |
integrations/taxonomy/icons.yaml |
taxonomy framework author | YES | validated by gen_taxonomy.py |
integrations/taxonomy.json |
gen_taxonomy.py |
NO -- gitignored (.gitignore:164) |
generated locally/CI; downstream cloud-frontend contract |
Banner conventions per file kind
integrations/integrations.js
Top of file (integrations/templates/integrations.js:1-2):
// DO NOT EDIT THIS FILE DIRECTLY
// It gets generated by integrations/gen_integrations.py in the Netdata repo
Body shape:
export const categories = [/* JSON array of category trees */];
export const integrations = [/* JSON array of integration objects */];
The body uses the rendered-with-folding/{% details %} markers
(the clean=False variant). Consumed by the cloud-frontend
dashboard which has its own renderer that interprets these
markers.
integrations/integrations.json
Pure JSON, no banner. Top-level shape:
{ "categories": [...], "integrations": [...] }
Uses the clean=True variant (markers stripped). Consumed by
scripts/checkIntegrations.js in the dashboard repo (validation
fetch from https://raw.githubusercontent.com/netdata/netdata/master/integrations/integrations.json)
and by anything that needs GitHub-renderable markdown.
Per-integration .md (collector, exporter, secretstore, etc.)
Every per-integration .md opens with a <!--startmeta
HTML-comment block. This block is consumed by the Learn ingest
pipeline (<repo>/docs/.map/map.yaml row matching is by
custom_edit_url).
Schema:
<!--startmeta
custom_edit_url: "https://github.com/netdata/netdata/edit/master/<plugin-dir>/integrations/<slug>.md"
meta_yaml: "https://github.com/netdata/netdata/edit/master/<plugin-dir>/metadata.yaml"
sidebar_label: "<display name>"
learn_status: "Published"
learn_rel_path: "<Learn category path>"
keywords: ['k1', 'k2', ...] # only when metadata has keywords
most_popular: true # only when metadata sets it
message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE"
endmeta-->
The exact wording of message: varies by type. From
gen_docs_integrations.py:
| Integration type | message: text |
|---|---|
collector |
DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE |
exporter |
... GENERATED BY THE EXPORTER'S metadata.yaml FILE |
agent_notification, cloud_notification |
... GENERATED BY THE NOTIFICATION'S metadata.yaml FILE |
logs |
... GENERATED BY THE LOGS' metadata.yaml FILE |
authentication |
... GENERATED BY THE AUTHENTICATION'S metadata.yaml FILE |
secretstore |
... GENERATED BY THE SECRETSTORE'S metadata.yaml FILE |
service_discovery |
... GENERATED BY THE SERVICE DISCOVERY DISCOVERER'S metadata.yaml FILE |
After the banner, the script injects a community-or-Netdata
badge image before the first ## (:422-437). Badge color
depends on meta.community boolean.
learn_rel_path mapping
The first category in meta.monitored_instance.categories is
remapped to a Learn navigation path (gen_docs_integrations.py
per-type branches):
| Integration type | Mapping |
|---|---|
collector |
data-collection.<x> -> Collecting Metrics/Collectors/<x> |
exporter |
hardcoded Exporting Metrics/Connectors |
agent_notification |
<x> with notifications -> Alerts & Notifications/Notifications |
cloud_notification |
same as above |
logs |
<x> with logs -> Logs |
authentication |
<x> with authentication -> Netdata Cloud/Authentication & Authorization/Cloud Authentication & Authorization Integrations |
secretstore |
hardcoded Collecting Metrics/Secrets Management/Secret Stores |
service_discovery |
hardcoded Collecting Metrics/Service Discovery |
<plugin-dir>/README.md (symlink case)
When the symlink rule fires (single-integration directory), the
README is a symlink to integrations/<slug>.md. The symlink
target's banner is the per-integration banner above, with two
adjustments:
- custom_edit_url points at the README path, not the
integration file path -- so editing through Learn opens the
README.
- The body has {element}/{symlinks[element]} replaced with
{element}/README.md (:542-544) so internal anchor refs
don't break.
src/health/notifications/<dir>/README.md (direct case)
Same banner as above. Written DIRECTLY, not as a symlink. Keep in mind for the collector consistency rule -- the README.md is the generated artifact.
src/collectors/COLLECTORS.md, SECRETS.md, SERVICE-DISCOVERY.md
No generated-file banner. COLLECTORS.md currently opens
with <!-- markdownlint-disable-file --> and then its marketing
header (# Monitor anything with Netdata). SECRETS.md and
SERVICE-DISCOVERY.md open with their marketing headers
(# Secrets Management, # Service Discovery). This is a
known oversight -- there is no DO-NOT-EDIT marker on these
files even though they are generated. Editing them directly
will be silently overwritten on the next CI run (for
COLLECTORS.md and SECRETS.md). For SERVICE-DISCOVERY.md the
absence of CI wiring means manual edits stick until someone
runs the script.
COLLECTORS.md is the Learn page titled "Monitor anything with
Netdata". It is generated by integrations/gen_doc_collector_page.py
from integrations/integrations.js. The generator groups normal
collector entries by children of data-collection and also treats
the top-level flows category as a first-class section named
Network Flows; without that special case, NetFlow / IPFIX / sFlow
and flow enrichment integrations either appear in the wrong generic
section or are easy to miss.
To update the page, do not edit the generated table rows in
src/collectors/COLLECTORS.md. Edit the source
metadata.yaml / integrations/categories.yaml or the
generator, then run:
python3 integrations/gen_integrations.py
python3 integrations/gen_doc_collector_page.py
ibm.d-generated files
| File | Banner |
|---|---|
metadata.yaml |
First line: # Generated metadata.yaml for <module> module (template at src/go/plugin/ibm.d/docgen/main.go:562) |
zz_generated_contexts.go |
Filename prefix zz_generated_. Package doc at <module>/contexts/doc.go:2-3 declares DO NOT EDIT zz_generated_contexts.go - it is auto-generated from contexts.yaml. |
README.md |
Banner depends on docgen's readme template. Includes module info, metric tables, config tables. |
config_schema.json |
Pure JSON, no banner. |
See ibm-d.md for the full chain.
Maintainer rules
- Per-integration
.mdfiles: NEVER edit by hand. Edit the sourcemetadata.yaml, regenerate, commit. - Symlinked
README.mdfiles: NEVER edit by hand. Same reason -- they point at the generated.md. src/health/notifications/<dir>/README.md: NEVER edit by hand. Direct generated file, looks identical to a normal README -- the<!--startmetabanner is the giveaway.src/collectors/COLLECTORS.md/SECRETS.md: NEVER edit the dynamic tables. Edit the sourcemetadata.yaml. The static prose ("# Monitor anything with Netdata", the secret-store onboarding text) is baked into the generator scripts -- to change static prose, editgen_doc_collector_page.py/gen_doc_secrets_page.pyand commit.src/collectors/SERVICE-DISCOVERY.md: same rule. Caveat: the script is not in CI, so manual regen is required after editing source metadata until that gap is fixed.- ibm.d generated files: NEVER edit. Edit
contexts.yaml,config.go, ormodule.yamlinstead and re-rungo generate.