| 1 | To generate a copy of `integrations.js` and validate collector |
| 2 | taxonomy locally, you will need: |
| 3 | |
| 4 | - Python 3.6 or newer (only tested on Python 3.10 currently, should work |
| 5 | on any version of Python newer than 3.6). |
| 6 | - The following third-party Python modules: |
| 7 | - `jsonschema` |
| 8 | - `referencing` |
| 9 | - `jinja2` |
| 10 | - `ruamel.yaml` |
| 11 | - A local checkout of https://github.com/netdata/netdata |
| 12 | - A local checkout of https://github.com/netdata/go.d.plugin. The script |
| 13 | expects this to be checked out in a directory called `go.d.plugin` |
| 14 | in the root directory of the Agent repo, though a symlink with that |
| 15 | name pointing at the actual location of the repo will work as well. |
| 16 | |
| 17 | The first two parts can be easily covered in a Linux environment, such |
| 18 | as a VM or Docker container: |
| 19 | |
| 20 | - On Debian or Ubuntu: `apt-get install python3-jsonschema python3-referencing python3-jinja2 python3-ruamel.yaml` |
| 21 | - On Alpine: `apk add py3-jsonschema py3-referencing py3-jinja2 py3-ruamel.yaml` |
| 22 | - On Fedora or RHEL (EPEL is required on RHEL systems): `dnf install python3-jsonschema python3-referencing python3-jinja2 python3-ruamel-yaml` |
| 23 | |
| 24 | Once the environment is set up, run the documentation generators from |
| 25 | the Agent repo root: |
| 26 | |
| 27 | - `integrations/gen_integrations.py` |
| 28 | - `integrations/gen_taxonomy.py --check-only` |
| 29 | - `integrations/check_collector_taxonomy.py` |
| 30 | - `integrations/gen_docs_integrations.py` |
| 31 | - `integrations/gen_doc_collector_page.py` |
| 32 | - `integrations/gen_doc_secrets_page.py` |
| 33 | |
| 34 | These scripts must be run _from this specific location_, as they use |
| 35 | their own path to figure out where all the files they need are. |
| 36 | |
| 37 | Collector dashboard taxonomy is authored in sibling `taxonomy.yaml` |
| 38 | files next to collector `metadata.yaml` files. Static collectors use |
| 39 | ordered `items:` trees; a plain context string in `items:` owns that |
| 40 | chart context and normalizes to `type: owned_context`. Display widgets |
| 41 | use `type: context` with `contexts:` and `chart_library`, and every |
| 42 | referenced literal context must be owned somewhere in the structural |
| 43 | tree. Dynamic collectors use `type: selector` with `context_prefix:` |
| 44 | or `collect_plugin:` and must opt in from `metadata.yaml` with |
| 45 | `metrics.dynamic_context_prefixes:` or |
| 46 | `metrics.dynamic_collect_plugins:`; a taxonomy `context_prefix:` may |
| 47 | narrow a declared metadata namespace. The generated |
| 48 | `integrations/taxonomy.json` artifact is gitignored like |
| 49 | `integrations/integrations.js`. |
| 50 | |
| 51 | To seed a static collector taxonomy from existing metadata contexts: |
| 52 | |
| 53 | ```bash |
| 54 | python3 integrations/gen_taxonomy_seed.py src/go/plugin/go.d/collector/apache/metadata.yaml --module-name apache --section-id applications.apache --placement-id apache --icon apache |
| 55 | ``` |
| 56 | |
| 57 | Pull requests run `integrations/check_collector_taxonomy.py` from |
| 58 | `.github/workflows/check-markdown.yml`. The gate validates committed |
| 59 | taxonomy files and fails when a collector `metadata.yaml` metrics block |
| 60 | or `taxonomy.yaml` changes without matching taxonomy coverage. |