| 1 | # MDX rules |
| 2 | |
| 3 | Docusaurus uses MDX 3 for `.mdx` files. MDX is markdown plus |
| 4 | JSX, which means certain markdown text that is harmless in |
| 5 | plain GitHub-rendered markdown will break MDX parsing. Ingest |
| 6 | runs a battery of escape transformations to make source `.md` |
| 7 | content survive the conversion to `.mdx`. |
| 8 | |
| 9 | Live in `${NETDATA_REPOS_DIR}/learn/ingest/ingest.py:1721-1799` |
| 10 | (the `_escape_mdx_braces` function and adjacent transforms), |
| 11 | exhaustively tested in |
| 12 | `${NETDATA_REPOS_DIR}/learn/test_escape_mdx_braces.py`. Every |
| 13 | transform is applied to every published file via |
| 14 | `sanitize_page` (`ingest.py:1765-1829`), in this order: |
| 15 | |
| 16 | ## 1. Frontmatter conversion |
| 17 | |
| 18 | - First `<!--` -> `---`, first `-->` -> `---` (`ingest.py:1779-1780`). |
| 19 | - Turns the injected hidden metadata block into real YAML |
| 20 | frontmatter. |
| 21 | |
| 22 | ## 2. Strip unhideme markers |
| 23 | |
| 24 | `<!--unhideme` and `unhideme-->` markers are stripped |
| 25 | (`ingest.py:1783-1784`). |
| 26 | |
| 27 | ## 3. `<details><summary>` newline fix |
| 28 | |
| 29 | `<details><summary>` -> `<details>\n<summary>` (and the |
| 30 | `<details open>` variant) for MDX 3 compatibility |
| 31 | (`ingest.py:1787-1788`). Without this fix, the inline form |
| 32 | breaks the parser. |
| 33 | |
| 34 | **Caveat**: only the two literal forms it knows are fixed. Any |
| 35 | other variant (e.g. `<details class="x"><summary>`) breaks |
| 36 | MDX rendering silently. |
| 37 | |
| 38 | ## 4. `_escape_mdx_braces` |
| 39 | |
| 40 | Escape every bare `{` outside fenced/inline code: |
| 41 | |
| 42 | - **Preserve** fenced ```` ``` ... ``` ``` blocks (DOTALL |
| 43 | match). |
| 44 | - **Preserve** inline `` `...` `` (no newlines, no nested |
| 45 | backticks). |
| 46 | - **Preserve** `^import ...$` lines (MDX ESM destructuring |
| 47 | syntax). |
| 48 | - **Preserve** `^export (default|function|const|let|var|{) ...$` |
| 49 | lines (regex |
| 50 | `export\s+(?:default|function|const|let|var|\{)`). |
| 51 | - Replace every bare `{` (not already preceded by `\`) with |
| 52 | `\{`. |
| 53 | - Restore `style=\{\{` back to `style={{` because that's valid |
| 54 | JSX. |
| 55 | |
| 56 | **Bash `export VAR=...` lines are NOT preserved as ESM**. The |
| 57 | `export` regex requires `default`, `function`, `const`, `let`, |
| 58 | `var`, or `{` after `export`. Bash-style `export NETDATA_FOO=bar` |
| 59 | passes through escape unchanged because it has no `{`. If a |
| 60 | bash export contained `{`, it would get escaped (probably what |
| 61 | you want). |
| 62 | |
| 63 | ## 5. Specific operator escapes |
| 64 | |
| 65 | - `<=` -> `\<=` |
| 66 | - `%<` -> `%\<` |
| 67 | - `<->` -> `\<->` |
| 68 | |
| 69 | (`ingest.py:1792-1794`). MDX would otherwise try to parse |
| 70 | these as JSX tags. Note these are **exact-substring** rules; |
| 71 | near-variants like `< =` (with space) or `<---->` (multi-dash) |
| 72 | are NOT covered. See `pitfalls-and-gotchas.md`. |
| 73 | |
| 74 | ### Patterns that the escape battery does NOT cover |
| 75 | |
| 76 | - **`<word>` placeholders in prose** (e.g. `<service-name>`, |
| 77 | `<scope>`, `<app>`). MDX 3 parses anything that lexically |
| 78 | looks like an open tag and then expects a matching close tag. |
| 79 | Without one, the build fails with |
| 80 | `Expected a closing tag for \`<word>\` ... before the end of \`paragraph\``. |
| 81 | - **`<` followed by a digit** (e.g. `<100 minutes`, |
| 82 | `<5 seconds`). MDX rejects this with |
| 83 | `Unexpected character '1' (U+0031) before name, expected a |
| 84 | character that can start a name, such as a letter, $, or _`. |
| 85 | This pattern is common in lists describing thresholds. |
| 86 | - **`Type<param>` Rust/C++/Java generic syntax** (e.g. |
| 87 | `Vec<u32>`, `HashMap<String, Vec<u8>>`, `unique_ptr<T>`). |
| 88 | Same JSX-tag issue. |
| 89 | |
| 90 | The safe options, in order of preference: |
| 91 | |
| 92 | 1. **Wrap in inline code with backticks** -- rule 4 |
| 93 | preserves inline code, so `\`<service-name>\``, |
| 94 | `\`Vec<u32>\``, `\`<APP>\`` all survive intact. This is |
| 95 | the standard fix for our Netdata integration content. |
| 96 | 2. **Rephrase the sentence** -- e.g. `< 100 minutes` |
| 97 | becomes `under 100 minutes`. Often clearer than the |
| 98 | original anyway. |
| 99 | 3. **Backslash-escape the `<`** -- `\<word>`. Works but |
| 100 | uglier than backticks. Use only when the `<` must remain |
| 101 | visibly a less-than operator, not a placeholder. |
| 102 | |
| 103 | These three were exercised in the netflow-plugin docs |
| 104 | (2026-05-07 ingest preview deploy failure): |
| 105 | `docs/network-flows/retention-querying.md` had `<100 minutes`, |
| 106 | fixed by rephrasing; `aws_ip_ranges.md`, `gcp_ip_ranges.md`, |
| 107 | and `generic_json-over-http_ipam.md` (generated from |
| 108 | `metadata.yaml`) had `<service-name>`, `<scope>`, `<app>`, |
| 109 | fixed by wrapping the placeholders in backticks at the |
| 110 | `metadata.yaml` source. |
| 111 | |
| 112 | ## 6. Bare URL angle-bracket links |
| 113 | |
| 114 | Converted to markdown links (`ingest.py:1797-1799`): |
| 115 | |
| 116 | - `<https://...>` -> `[https://...](https://...)` |
| 117 | - `<http://...>` -> `[http://...](http://...)` |
| 118 | - `<email@x.y>` -> `[email@x.y](mailto:email@x.y)` |
| 119 | |
| 120 | So `<...>` used as a "I want this rendered as a link" markdown |
| 121 | shortcut works correctly through ingest. |
| 122 | |
| 123 | ## 7. `meta_yaml` rewrite |
| 124 | |
| 125 | If `meta_yaml: "<url>"` is present in the file, the file is |
| 126 | treated as an integration. `meta_yaml:` is removed and |
| 127 | `custom_edit_url` is rewritten to that URL |
| 128 | (`ingest.py:1801-1808`). Silent rewrite -- any file with that |
| 129 | key triggers it. |
| 130 | |
| 131 | ## 8. Integration logo annotation |
| 132 | |
| 133 | Integration files (`INTEGRATION_MARKER` present) get |
| 134 | `<img src="https://(www.)?netdata.cloud/img/...">` annotated |
| 135 | with `data-integration-logo`, `data-logo-contrast-light`, |
| 136 | `data-logo-contrast-dark`, `data-logo-contrast-confidence` |
| 137 | after a fetch+luminance analysis |
| 138 | (`_annotate_integration_logo_tags` and `_analyze_remote_logo`, |
| 139 | `ingest.py:1647-1718`). Used by `Grid_integrations` and the |
| 140 | dashboard theme to add a subtle glow on low-contrast logos. |
| 141 | |
| 142 | ## 9. Drop analytics pixel lines |
| 143 | |
| 144 | Lines starting with `[![analytics]` are dropped |
| 145 | (`ingest.py:1816-1817`). These are leftover GitHub |
| 146 | README-style tracking pixels that don't make sense on Learn. |
| 147 | |
| 148 | ## What survives, what doesn't |
| 149 | |
| 150 | The escape rules cover the most common breakage patterns: |
| 151 | |
| 152 | | Pattern | Survives? | Why | |
| 153 | |---|---|---| |
| 154 | | `{word}` outside code | escaped to `\{word}` | rule 4 | |
| 155 | | `${expr}` | escaped to `\${expr}` | rule 4 | |
| 156 | | `style={{ }}` JSX | yes (un-escaped) | rule 4 restoration | |
| 157 | | Already-escaped braces | unchanged | rule 4 idempotent | |
| 158 | | Fenced or inline code | unchanged | rule 4 preservation | |
| 159 | | MDX `import`/`export` ESM | unchanged | rule 4 preservation | |
| 160 | | Tables with `{` cells | escaped per cell | rule 4 | |
| 161 | | `<=`, `%<`, `<->` exact | escaped | rule 5 | |
| 162 | | `< =` (with space) | NOT covered | rule 5 is exact-substring | |
| 163 | | `<---->` (long arrow) | NOT covered | rule 5 is exact-substring | |
| 164 | | `<htmltag>` body content | NOT escaped | breaks MDX unless wrapped in code | |
| 165 | | `<word>` placeholders in prose | NOT escaped | breaks MDX; wrap in backticks or rephrase | |
| 166 | | `<` + digit (`<100`, `<5s`) | NOT escaped | breaks MDX; rephrase as "under N" or escape | |
| 167 | | `Vec<u32>`, `HashMap<K,V>` generics | NOT escaped | breaks MDX; wrap in backticks | |
| 168 | | `<details>` inline summary | fixed | rule 3 | |
| 169 | | `<details class="x">` | NOT fixed | rule 3 only knows two forms | |
| 170 | | `}` closing brace alone | NOT escaped | rule 4 only escapes `{` | |
| 171 | | Bare URL in `<...>` | converted to MD link | rule 6 | |
| 172 | |
| 173 | ## Test suite |
| 174 | |
| 175 | `${NETDATA_REPOS_DIR}/learn/test_escape_mdx_braces.py:74-377` |
| 176 | exercises: |
| 177 | |
| 178 | - simple `{word}`, `${expr}` templates; |
| 179 | - `{{double}}` braces; |
| 180 | - `style={{ }}` JSX (preserved); |
| 181 | - already-escaped braces (idempotent); |
| 182 | - fenced and inline code preservation (multiple code blocks |
| 183 | per file, mixed inline and fenced); |
| 184 | - table rows with mixed bare and code-fenced braces; |
| 185 | - real Zabbix and Nagios integration content; |
| 186 | - MDX `import`/`export` ESM lines; |
| 187 | - bash `export` lines (currently pass through unchanged |
| 188 | because they have no braces); |
| 189 | - empty input; |
| 190 | - only-`{}` content; |
| 191 | - nested `{outer{inner}}`; |
| 192 | - unclosed `{`; |
| 193 | - lone `}`; |
| 194 | - newline after `{`; |
| 195 | - the original hardcoded `{attribute_name}` patterns; |
| 196 | - a full-document simulation. |
| 197 | |
| 198 | ## Mermaid diagrams |
| 199 | |
| 200 | Mermaid diagrams are enabled at the markdown level in |
| 201 | `docusaurus.config.js:26-30`. The escape rules preserve |
| 202 | fenced code blocks, so ` ```mermaid ... ``` ` blocks survive |
| 203 | intact. |
| 204 | |
| 205 | ## What you can put in source `.md` files safely |
| 206 | |
| 207 | - Any plain markdown. |
| 208 | - Code blocks (fenced and inline) -- whatever's inside them is |
| 209 | preserved. |
| 210 | - MDX import/export ESM at the top of file (works in `.mdx`, |
| 211 | not in `.md`; ingest converts to `.mdx`). |
| 212 | - `style={{ ... }}` JSX (preserved). |
| 213 | - Mermaid in fenced ```` ```mermaid ``` ```. |
| 214 | - `<details>` and `<summary>` with newlines between them. |
| 215 | |
| 216 | ## What you should escape yourself in source |
| 217 | |
| 218 | - `<htmltag>` in body content -- always wrap in code (fenced |
| 219 | or inline). |
| 220 | - Multi-character operators like `<->` with extra dashes |
| 221 | (`<-->`, `<--->`). |
| 222 | - `< ` or ` <` with spaces around the `<`. |
| 223 | - Closing-brace-only sequences if they're standalone. |
| 224 | |
| 225 | ## Onbroken-links policy |
| 226 | |
| 227 | `docusaurus.config.js:22`: `onBrokenLinks: 'warn'`. Broken |
| 228 | links never fail the Docusaurus build at the parse level; they |
| 229 | only get caught by the ingest's pre-build link checker |
| 230 | (`--fail-links`) and the daily 404 sweep |
| 231 | (`daily-learn-link-check.yml`). So a broken link in your `.md` |
| 232 | will silently make it to production unless caught by one of |
| 233 | those two gates. |