| 1 | # In-app dashboard contract |
| 2 | |
| 3 | The Netdata cloud-frontend dashboard (the React app that powers |
| 4 | `app.netdata.cloud`) renders the Integrations page from the |
| 5 | `integrations.js` artifact this repo produces. Collector dashboard |
| 6 | taxonomy is published separately as `integrations/taxonomy.json`. |
| 7 | This guide |
| 8 | documents the contract between the two repositories so |
| 9 | maintainers know what is and is NOT in scope when working on |
| 10 | integrations-lifecycle changes. |
| 11 | |
| 12 | The cloud-frontend repo lives at |
| 13 | `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/` (private, |
| 14 | Netdata-org). React component internals are explicitly OUT OF |
| 15 | SCOPE for this skill; only the artifact contract matters. |
| 16 | |
| 17 | ## What gets shipped |
| 18 | |
| 19 | **This repo produces:** `integrations/integrations.js` (and |
| 20 | `integrations/integrations.json`) on every CI run of |
| 21 | `generate-integrations.yml` (or local run of |
| 22 | `gen_integrations.py`). It also produces |
| 23 | `integrations/taxonomy.json` from `gen_taxonomy.py`. All three |
| 24 | files are gitignored in this repo. |
| 25 | |
| 26 | **The cloud-frontend repo consumes:** |
| 27 | `integrations/integrations.js` -- specifically, it copies the |
| 28 | file into `src/domains/integrations/data/integrations.js` in |
| 29 | its own source tree. |
| 30 | |
| 31 | `integrations/taxonomy.json` is a new opt-in downstream contract. |
| 32 | This repo validates and emits it, but cloud-frontend consumption is |
| 33 | owned by the dashboard team and may land independently. |
| 34 | |
| 35 | ## How the consumption works |
| 36 | |
| 37 | Confirmed at |
| 38 | `${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/.github/workflows/sync-to-s3.yaml:48-66`: |
| 39 | |
| 40 | 1. Cloud-frontend's CI checks out `netdata/netdata` (this repo). |
| 41 | 2. Runs `python3 integrations/gen_integrations.py` against |
| 42 | the freshly checked-out master. |
| 43 | 3. `cp ./integrations/integrations.js ../src/domains/integrations/data/integrations.js` |
| 44 | into the dashboard source tree. |
| 45 | 4. The dashboard builds with the just-copied artifact baked |
| 46 | in. |
| 47 | |
| 48 | A second script in the dashboard repo, |
| 49 | `scripts/checkIntegrations.js`, fetches |
| 50 | `https://raw.githubusercontent.com/netdata/netdata/master/integrations/integrations.json` |
| 51 | and validates against the in-tree copy as a drift detector |
| 52 | (`scripts/checkIntegrations.js:13`). |
| 53 | |
| 54 | A third script, `scripts/checkLinks.js`, validates that links |
| 55 | in `src/domains/integrations/data/integrations.js` and |
| 56 | `src/domains/integrations/utils/integrations.js` resolve. |
| 57 | |
| 58 | At the time this skill was updated, cloud-frontend had not yet |
| 59 | consumed `taxonomy.json`; its CI needs an explicit follow-up change |
| 60 | to run `python3 integrations/gen_taxonomy.py` and copy the JSON |
| 61 | artifact if/when the dashboard switches chart TOC ownership. |
| 62 | |
| 63 | ## The artifact shape |
| 64 | |
| 65 | `integrations/integrations.js`: |
| 66 | |
| 67 | ```js |
| 68 | // DO NOT EDIT THIS FILE DIRECTLY |
| 69 | // It gets generated by integrations/gen_integrations.py in the Netdata repo |
| 70 | |
| 71 | export const categories = [ |
| 72 | /* recursive tree of category objects, each: |
| 73 | { id, name, description, children: [...], collector_default?: boolean } |
| 74 | */ |
| 75 | ]; |
| 76 | |
| 77 | export const integrations = [ |
| 78 | /* flat array of integration objects, each carries |
| 79 | integration_type, id, meta, keywords, plus the rendered |
| 80 | section keys per type (e.g. setup, troubleshooting, alerts, |
| 81 | metrics, functions, overview, related_resources for |
| 82 | collectors) and their `clean_*` siblings (e.g. clean_setup, |
| 83 | clean_alerts) */ |
| 84 | ]; |
| 85 | ``` |
| 86 | |
| 87 | `integrations/taxonomy.json`: |
| 88 | |
| 89 | ```json |
| 90 | { |
| 91 | "taxonomy_schema_version": 1, |
| 92 | "source": { |
| 93 | "netdata_commit": "...", |
| 94 | "generated_at": "..." |
| 95 | }, |
| 96 | "sections": [], |
| 97 | "placements": [], |
| 98 | "opted_out_collectors": [] |
| 99 | } |
| 100 | ``` |
| 101 | |
| 102 | Each taxonomy placement keeps the ordered recursive `items:` tree and |
| 103 | snapshot fields generated from current metadata for CI/review |
| 104 | diffing. `resolved_contexts` contains owned contexts; |
| 105 | `referenced_contexts` contains display/widget references, and |
| 106 | `unresolved_references` carries explicit unresolved-reference escape |
| 107 | hatches for downstream consumers. The schema lives at |
| 108 | `integrations/schemas/taxonomy_output.json`. |
| 109 | |
| 110 | FE adapters must discriminate these v1 taxonomy node kinds: |
| 111 | |
| 112 | - `owned_context` -- structural leaf that owns one literal context. |
| 113 | - `group` -- structural container. |
| 114 | - `flatten` -- structural container whose children flatten into the |
| 115 | parent menu level. |
| 116 | - `selector` -- structural dynamic owner from `context_prefix` or |
| 117 | `collect_plugin`. |
| 118 | - `context` -- display widget that references contexts. |
| 119 | - `grid` -- display container with positioned child widgets. |
| 120 | - `first_available` -- ordered display alternatives. |
| 121 | - `view_switch` -- whole-body replacement for multi-node vs |
| 122 | single-node rendering. |
| 123 | - string shorthand appears only in authoring; generated output |
| 124 | normalizes it to `owned_context`. |
| 125 | |
| 126 | All public content sections consumed by downstream renderers must be |
| 127 | markdown strings in the generated artifacts, even when the source |
| 128 | `metadata.yaml` stores them as structured YAML objects or arrays. |
| 129 | Examples: collector-like `metrics`, `alerts`, `functions`, `overview`, |
| 130 | `setup`, `troubleshooting`, and `related_resources` must pass through |
| 131 | the renderer before reaching `integrations.js` / `integrations.json`. |
| 132 | Leaving raw objects or arrays in these fields breaks the website Hugo |
| 133 | renderer and produces blank tabs or link-check failures in |
| 134 | cloud-frontend. |
| 135 | |
| 136 | The dashboard's renderer interprets the `{% details %}` / |
| 137 | `{% /details %}` markers embedded in the rendered text (the |
| 138 | `clean=False` variant is the one written into the `.js` file). |
| 139 | This is why the dashboard receives rich-text content with |
| 140 | collapsible sections, while GitHub-rendered `.md` files use |
| 141 | the `clean=True` variant where these markers are stripped. |
| 142 | |
| 143 | ## Special case: `deploy` integrations |
| 144 | |
| 145 | `deploy` entries are NOT written to disk as per-integration |
| 146 | `.md` files (see `per-type-matrix.md`). They live ONLY inside |
| 147 | `integrations.js`, sorted by `quick_start` integer. The |
| 148 | dashboard's "Add Nodes" dialog is the consumer. |
| 149 | |
| 150 | `quick_start: -1` (or any negative) hides the entry from the |
| 151 | dialog. Positive values define the sort order. |
| 152 | |
| 153 | ## Drift detection |
| 154 | |
| 155 | The cloud-frontend's `checkIntegrations.js` is the only |
| 156 | end-to-end drift check between the two repos. It runs in the |
| 157 | dashboard's CI and surfaces a failure if the local copy |
| 158 | diverges from `netdata/master`. There is no symmetric check |
| 159 | in this repo (this repo doesn't know what version of |
| 160 | `integrations.js` the dashboard currently has baked in). |
| 161 | |
| 162 | In practice this means: |
| 163 | |
| 164 | - A PR in this repo that changes `metadata.yaml` does NOT |
| 165 | break the dashboard immediately. The dashboard rebuilds on |
| 166 | its own schedule (or when its developers re-run their |
| 167 | sync-to-s3 workflow). |
| 168 | - A breaking change to the `integrations.js` shape (e.g. a |
| 169 | removed top-level field) WILL break the dashboard on the |
| 170 | next sync. There is no shape-versioning today; both repos |
| 171 | assume the JS export shape is stable. |
| 172 | - A breaking change to `taxonomy.json` must bump |
| 173 | `taxonomy_schema_version` and coordinate with downstream |
| 174 | consumers. Additive fields are acceptable only when old |
| 175 | consumers can ignore them safely. |
| 176 | |
| 177 | ## What is OUT of scope for integrations-lifecycle |
| 178 | |
| 179 | - **The React renderer** in cloud-frontend that turns |
| 180 | `integrations.js` into UI. Not documented here. |
| 181 | - **The Integrations page UX, search behavior, filtering, |
| 182 | navigation.** Cloud-frontend territory. |
| 183 | - **The "Add Nodes" dialog flow** beyond the `quick_start` |
| 184 | sort contract. |
| 185 | - **Per-platform install commands rendering** (the `deploy` |
| 186 | entries' `methods[].commands[]`). The dashboard renders |
| 187 | them; the metadata produces them. |
| 188 | |
| 189 | ## Maintainer rules |
| 190 | |
| 191 | 1. **Treat `integrations.js` as a published artifact**. Its |
| 192 | shape (the two named exports, the per-integration object |
| 193 | keys) is a contract. Avoid breaking changes; coordinate |
| 194 | with the cloud-frontend team if a key must be renamed or |
| 195 | removed. |
| 196 | 2. **Treat `taxonomy.json` as a versioned published artifact |
| 197 | once consumed.** Keep v1 authoring closed: `section_id:`, |
| 198 | ordered `items:`, explicit item `type` values, selector keys |
| 199 | (`context_prefix:`, `context_prefix_exclude:`, |
| 200 | `collect_plugin:`), widget `contexts:`, and sparse |
| 201 | `single_node:` overrides. |
| 202 | 3. **Render structured metadata before publication**. A new |
| 203 | integration type that reuses collector-style sections must |
| 204 | include every structured content key in its render-key list. |
| 205 | Do not publish raw `metrics` objects, `alerts` arrays, or |
| 206 | similar YAML structures under the public markdown section |
| 207 | names. |
| 208 | 4. **Custom Jinja markers in metadata** (`{% details %}`, |
| 209 | `{% relatedResource %}`, `{% if %}`) are part of the |
| 210 | contract. The dashboard's renderer interprets them. Test |
| 211 | any new marker against both surfaces before relying on it. |
| 212 | 4. **Do not commit `integrations.js` to this repo**. It is |
| 213 | gitignored on purpose; the dashboard pulls fresh on each |
| 214 | build. |