| 1 | --- |
| 2 | name: learn-site-structure |
| 3 | description: Authoritative reference for how docs in this repo (and 5 other Netdata-org repos) become published pages on `learn.netdata.cloud`. Covers the `<repo>/docs/.map/map.yaml` source-of-truth (the actual lever -- filesystem path is irrelevant for routing), the live `ingest/ingest.py` orchestrator in the learn repo (NOT the legacy `ingest.js`), frontmatter injection, slug rules, sidebar autogeneration, MDX escape rules, versioning, the 4-mechanism redirect stack, the 6 source repositories, the every-3-hours CI ingest, Netlify deploy, and the `part_of_learn=True` opt-in for files hand-authored in the learn repo. Use when adding/moving/renaming/deleting a docs page; when a page on Learn looks wrong; when wondering whether to edit a doc here or in the learn repo; when reading `ingest.py`, `sidebars.js`, `docusaurus.config.js`, `static.toml`, `LegacyLearnCorrelateLinksWithGHURLs.json`, `netlify.toml`, the `<!--startmeta` blocks in `.mdx` files, or the workflows `ingest.yml` and `daily-learn-link-check.yml`. |
| 4 | --- |
| 5 | |
| 6 | # learn-site-structure |
| 7 | |
| 8 | This skill is the **single place** to learn how content in this |
| 9 | repo (and adjacent Netdata-org repos) becomes published pages on |
| 10 | the Netdata learn site (`learn.netdata.cloud`). It documents: |
| 11 | |
| 12 | - the `<repo>/docs/.map/map.yaml` source-of-truth that every |
| 13 | published page must appear in; |
| 14 | - the source-path-to-Learn-URL computation (frontmatter is |
| 15 | injected from `map.yaml`; the source filesystem path is |
| 16 | irrelevant for routing); |
| 17 | - the 16-step `ingest/ingest.py` orchestrator in the learn repo |
| 18 | (the live one -- NOT the legacy `ingest.js`); |
| 19 | - the 6 source repositories the pipeline pulls from; |
| 20 | - sidebar autogeneration via Docusaurus `autogenerated` mode; |
| 21 | - the MDX escape rules every transformation enforces; |
| 22 | - the 4-mechanism redirect stack and how moves auto-redirect; |
| 23 | - the every-3-hours CI ingest cadence and Netlify deploy |
| 24 | contract; |
| 25 | - the `part_of_learn: True` opt-in for files hand-authored in |
| 26 | the learn repo; |
| 27 | - every surprise, dead artifact, and gotcha worth knowing. |
| 28 | |
| 29 | After reading SKILL.md plus the per-domain guides linked below, |
| 30 | an assistant should never need to ask "how does a page get |
| 31 | added to learn?", "how is the URL computed?", "should I edit |
| 32 | this here or in the learn repo?", or "what runs in CI?". |
| 33 | |
| 34 | ## Key concepts (read first) |
| 35 | |
| 36 | 1. **`<repo>/docs/.map/map.yaml` is the lever.** Every page |
| 37 | published on Learn appears in this file. To add a page, |
| 38 | add a node. To move a page, move its node. To unpublish a |
| 39 | page, remove its node and (manually) update the redirect |
| 40 | catalog. The schema lives at |
| 41 | `<repo>/docs/.map/map.schema.json`; the authoring guide |
| 42 | lives at `<repo>/docs/.map/README.md`. |
| 43 | |
| 44 | 2. **Source filesystem path is IRRELEVANT for routing.** A |
| 45 | doc's destination URL on Learn is computed from |
| 46 | `sidebar_label` + `learn_rel_path` injected from `map.yaml`, |
| 47 | NOT from where the file lives in this repo. Renaming a |
| 48 | source file does not break the URL; renaming the |
| 49 | `meta.label` in `map.yaml` DOES (and triggers an |
| 50 | auto-redirect). |
| 51 | |
| 52 | 3. **The live orchestrator is `ingest/ingest.py`. Ignore |
| 53 | `ingest.js` and `ingest.md`.** Both are legacy. The README |
| 54 | at the learn repo root and `docs/.map/README.md` document |
| 55 | the live Python flow. Don't edit `ingest.js`; don't trust |
| 56 | `ingest.md`. |
| 57 | |
| 58 | 4. **Sidebar is autogenerated.** `${NETDATA_REPOS_DIR}/learn/sidebars.js` |
| 59 | is one line: `{type: "autogenerated", dirName: "."}`. |
| 60 | Order is driven by `sidebar_position` frontmatter that |
| 61 | ingest assigns from `map.yaml` traversal order. To reorder |
| 62 | the sidebar, reorder rows in `map.yaml`. |
| 63 | |
| 64 | 5. **`docs/` (in the learn repo) is OWNED by ingest.** Anything |
| 65 | under `${NETDATA_REPOS_DIR}/learn/docs/` is cleaned and |
| 66 | regenerated each ingest run, EXCEPT files marked |
| 67 | `part_of_learn: True` (currently only `docs/ask-nedi.mdx`). |
| 68 | Do NOT hand-edit `docs/` in the learn repo unless you set |
| 69 | that flag and accept that this is a hand-authored |
| 70 | exception. |
| 71 | |
| 72 | 6. **6 source repositories feed Learn.** This repo |
| 73 | (`netdata/netdata`) is the bulk; also pulled: |
| 74 | `netdata-cloud-onprem`, `.github`, `agent-service-discovery`, |
| 75 | `netdata-grafana-datasource-plugin`, `helmchart`. Each can |
| 76 | be overridden locally with `--repos` for testing. |
| 77 | |
| 78 | 7. **Auto-redirects on move/rename, manual on delete.** |
| 79 | Moving or renaming a page in `map.yaml` produces an |
| 80 | automatic Netlify redirect from the old URL to the new |
| 81 | target. Deleting a page requires manual surgery on |
| 82 | `${NETDATA_REPOS_DIR}/learn/LegacyLearnCorrelateLinksWithGHURLs.json` |
| 83 | (documented in `<repo>/docs/.map/README.md`). |
| 84 | |
| 85 | ## Table of contents |
| 86 | |
| 87 | | Guide | Purpose | |
| 88 | |---|---| |
| 89 | | `mapping.md` | The map.yaml schema; frontmatter injection; source-path-to-URL computation; slug rules; edge cases. | |
| 90 | | `pipeline.md` | The 16-step `ingest.py` flow; source repos; CI workflow; Netlify deploy. | |
| 91 | | `sidebars.md` | Docusaurus autogenerated sidebar; ordering rules; category overview pages; auto-grid pages. | |
| 92 | | `mdx-rules.md` | Every transformation in `_escape_mdx_braces` and `sanitize_page`; what breaks MDX 3 parsing. | |
| 93 | | `redirects.md` | The 4-mechanism redirect stack; auto-redirect on move/rename; manual unpublish surgery. | |
| 94 | | `pitfalls-and-gotchas.md` | Every surprise, dead artifact, edge case, silent failure mode, undocumented behavior. | |
| 95 | | `authoring-boundary.md` | What is owned by ingest (do not edit) vs hand-authored in learn repo; `part_of_learn: True`; cross-repo authoring. | |
| 96 | | `recipes/INDEX.md` | Step-by-step recipes for add / move / rename / delete a doc page. | |
| 97 | | `how-tos/INDEX.md` | Live catalog: every analysis question becomes a how-to entry. | |
| 98 | |
| 99 | ## Live how-to rule (mandatory) |
| 100 | |
| 101 | If an assistant is asked a concrete question about how a Learn |
| 102 | page is produced that is NOT already documented under |
| 103 | `how-tos/` or one of the per-domain guides above, AND answering |
| 104 | it requires non-trivial analysis (reading `ingest.py` or |
| 105 | related scripts, running ingest locally, checking |
| 106 | cross-references between this repo's `map.yaml` and the |
| 107 | learn-repo output), the assistant MUST author a new how-to |
| 108 | under `how-tos/<slug>.md` and add a one-line entry to |
| 109 | `how-tos/INDEX.md` BEFORE completing the task. This rule is |
| 110 | durable. |
| 111 | |
| 112 | ## Path discipline |
| 113 | |
| 114 | This skill follows |
| 115 | `<repo>/.agents/sow/specs/sensitive-data-discipline.md`: |
| 116 | |
| 117 | - This repo's files: repo-relative |
| 118 | (`<repo>/docs/.map/map.yaml`, `<repo>/docs/...`). |
| 119 | - Learn repo files: `${NETDATA_REPOS_DIR}/learn/...` (env-key |
| 120 | from `.env`). |
| 121 | - Other Netdata-org sibling repos: `${NETDATA_REPOS_DIR}/<repo>/...`. |
| 122 | - No literal home-directory or workstation-root paths anywhere. |
| 123 | |
| 124 | ## Sources of truth referenced by this skill |
| 125 | |
| 126 | - `<repo>/docs/.map/map.yaml` -- the publication source of truth. |
| 127 | - `<repo>/docs/.map/map.schema.json` -- the schema. |
| 128 | - `<repo>/docs/.map/README.md` -- the authoring guide. |
| 129 | - `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py` -- live orchestrator. |
| 130 | - `${NETDATA_REPOS_DIR}/learn/ingest/autogenerateRedirects.py` -- redirect generator. |
| 131 | - `${NETDATA_REPOS_DIR}/learn/sidebars.js` -- sidebar config. |
| 132 | - `${NETDATA_REPOS_DIR}/learn/docusaurus.config.js` -- site config. |
| 133 | - `${NETDATA_REPOS_DIR}/learn/static.toml` -- hand-curated static redirects. |
| 134 | - `${NETDATA_REPOS_DIR}/learn/test_escape_mdx_braces.py` -- MDX escape test suite. |
| 135 | - `${NETDATA_REPOS_DIR}/learn/.github/workflows/ingest.yml` -- ingest CI workflow. |
| 136 | - `${NETDATA_REPOS_DIR}/learn/.github/workflows/daily-learn-link-check.yml` -- daily link checker. |
| 137 | |
| 138 | ## Related skills |
| 139 | |
| 140 | - `integrations-lifecycle` -- the integrations side |
| 141 | (`metadata.yaml` -> per-integration `.md`). Integration pages |
| 142 | are produced before ingest runs and inserted into `map.yaml` |
| 143 | by ingest's `populate_integrations` step (see `mapping.md`). |
| 144 | - `project-writing-collectors` -- the broader collector |
| 145 | authoring context. |