@cryptotaxi247 / netdata-1 / commits / 45450d643

docs(integrations): add generated secretstore docs and SECRETS page (#22028)

* Add secretstore integration docs pipeline * Add generated secrets documentation * Drive secrets backend summary from metadata * docs(integrations): refine generated secrets docs * docs(map): point secrets management to SECRETS page

Ilya Mashchenko committed Mar 24, 2026 at 14:26 UTC 45450d6436f98d0fd58845f65ec804e54ca628bc

34 files changed +2732 -186

.github/workflows/check-markdown.yml

@@ -56,6 +56,11 @@ jobs:

56 source ./venv/bin/activate

57 cd netdata && python3 integrations/gen_doc_collector_page.py

59 + - name: Generate src/collectors/SECRETS.md

60 + run: |

61 + source ./venv/bin/activate

62 + cd netdata && python3 integrations/gen_doc_secrets_page.py

63 +

64 - name: Run Ingest

65 working-directory: learn

66 run: |

.github/workflows/generate-integrations.yml

@@ -12,11 +12,15 @@ on:

12 - 'src/exporting/**/metadata.yaml'

13 - 'src/health/notifications/**/metadata.yaml'

14 - 'integrations/templates/**'

15 + - 'integrations/schemas/**'

16 - 'integrations/categories.yaml'

17 - 'integrations/deploy.yaml'

18 - 'integrations/cloud-notifications/metadata.yaml'

19 - 'integrations/cloud-authentication/metadata.yaml'

20 - 'integrations/gen_integrations.py'

21 + - 'integrations/gen_docs_integrations.py'

22 + - 'integrations/gen_doc_collector_page.py'

23 + - 'integrations/gen_doc_secrets_page.py'

24 workflow_dispatch: null

25 concurrency: # This keeps multiple instances of the job from running concurrently for the same ref.

26 group: integrations-${{ github.ref }}

@@ -53,6 +57,10 @@ jobs:

57 id: generate-collectors-md

58 run: |

59 python3 integrations/gen_doc_collector_page.py

60 + - name: Generate src/collectors/SECRETS.md

61 + id: generate-secrets-md

62 + run: |

63 + python3 integrations/gen_doc_secrets_page.py

64 - name: Clean Up Temporary Data

65 id: clean

66 run: rm -rf go.d.plugin virtualenv integrations/integrations.js integrations/integrations.json

@@ -86,6 +94,7 @@ jobs:

94 Generate Integrations: ${{ steps.generate.outcome }}

95 Generate Integrations Documentation: ${{ steps.generate-integrations-documentation.outcome }}

96 Generate src/collectors/COLLECTORS.md: ${{ steps.generate-collectors-md.outcome }}

97 + Generate src/collectors/SECRETS.md: ${{ steps.generate-secrets-md.outcome }}

98 Clean Up Temporary Data: ${{ steps.clean.outcome }}

99 Create PR: ${{ steps.create-pr.outcome }}

100 SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

docs/.map/map.yaml

+2 -2

@@ -227,8 +227,8 @@ sidebar:

227 edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md

228 - meta:

229 label: Secrets Management

230 - edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/secrets-management.md

231 - description: Use secret references to avoid storing credentials in plain text. Supports environment variables, files, and external vaults.

230 + edit_url: https://github.com/netdata/netdata/edit/master/src/collectors/SECRETS.md

231 + description: Use secret references to avoid storing credentials in plain text. Supports environment variables, files, commands, and secretstore backends.

232 - meta:

233 label: Performance Optimization

234 edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md

docs/netdata-agent/configuration/secrets-management.md deleted

-143

@@ -1,143 +0,0 @@

1 -# Secrets Management

2 -

3 -Netdata supports secrets management for collector configurations, so you don't need to store plain-text credentials in configuration files. Instead, you use secret references that are resolved when a collector starts.

4 -

5 -| Reference Type | Syntax | Use Case |

6 -|:---------------------|:-----------------------------------|:-----------------------------------------------------------|

7 -| Environment variable | `${env:VAR_NAME}` | Secrets available as environment variables |

8 -| File | `${file:/path/to/secret}` | Secrets stored in files on disk |

9 -| Command | `${cmd:/path/to/command args}` | Secrets retrieved by running a command |

10 -| Secretstore | `${store:<kind>:<name>:<operand>}` | Secrets stored in remote backends (Vault, AWS, Azure, GCP) |

11 -

12 -## Environment Variables

13 -

14 -Use `${env:VARIABLE_NAME}` to reference an environment variable.

15 -

16 -```yaml

17 -jobs:

18 - - name: local

19 - dsn: "${env:MYSQL_USER}:${env:MYSQL_PASSWORD}@tcp(127.0.0.1:3306)/"

20 -```

21 -

22 -## Files

23 -

24 -Use `${file:/absolute/path}` to read a secret from a file. Leading and trailing whitespace is trimmed automatically.

25 -

26 -```yaml

27 -jobs:

28 - - name: myapp

29 - password: "${file:/run/secrets/myapp_password}"

30 -```

31 -

32 -## Commands

33 -

34 -Use `${cmd:/absolute/path/to/command args}` to execute a command and use its stdout as the secret value.

35 -

36 -```yaml

37 -jobs:

38 - - name: prod

39 - password: "${cmd:/usr/bin/op read op://vault/netdata/mysql/password}"

40 -```

41 -

42 -:::warning

43 -

44 -- Command paths must be absolute.

45 -- Commands have a 10-second timeout.

46 -- Arguments are split on whitespace. Quoting, pipes, redirects, and variable expansion are not interpreted unless you run a shell explicitly (e.g., `${cmd:/bin/sh -c "your command here"}`).

47 -

48 -:::

49 -

50 -## Secretstores

51 -

52 -For remote secret backends (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager), you configure a **secretstore** and then reference it from collector configurations.

53 -

54 -### Supported Providers

55 -

57 -|:-----------|:--------------------|:---------------------------------------------|:-------------------------------------|

62 -

63 -### Reference Format

64 -

65 -```text

66 -${store:<kind>:<name>:<operand>}

67 -```

68 -

69 -| Part | Description |

70 -|:----------|:---------------------------------------------------------|

71 -| `kind` | Provider kind from the table above (e.g., `vault`) |

72 -| `name` | The name you gave the secretstore when you configured it |

73 -| `operand` | Provider-specific path to the secret (see table above) |

74 -

75 -### Examples

76 -

77 -```yaml

78 -jobs:

79 - - name: mysql_prod

80 - password: "${store:vault:vault_prod:secret/data/netdata/mysql#password}"

81 -

82 - - name: redis_prod

83 - password: "${store:aws-sm:aws_prod:netdata/redis#password}"

84 -

85 - - name: api_prod

86 - token: "${store:azure-kv:azure_prod:my-vault/api-token}"

87 -

88 - - name: app_prod

89 - password: "${store:gcp-sm:gcp_prod:my-project/mysql-password}"

90 -```

91 -

92 -### Configuring a Secretstore

93 -

94 -#### Option 1: Dynamic Configuration (UI)

95 -

96 -1. Open the Netdata Dynamic Configuration UI.

97 -2. Choose a provider kind and give your secretstore a name.

98 -3. Fill in the provider-specific settings (address, credentials, etc.).

99 -4. Use the reference syntax `${store:<kind>:<name>:<operand>}` in your collector configs.

100 -

101 -#### Option 2: Configuration Files

102 -

103 -You can define secretstores in configuration files. Each provider has its own file:

104 -

105 -| File | Provider |

106 -|:-------------------------------------|:--------------------|

107 -| `/etc/netdata/go.d/ss/vault.conf` | HashiCorp Vault |

108 -| `/etc/netdata/go.d/ss/aws-sm.conf` | AWS Secrets Manager |

109 -| `/etc/netdata/go.d/ss/azure-kv.conf` | Azure Key Vault |

110 -| `/etc/netdata/go.d/ss/gcp-sm.conf` | GCP Secret Manager |

111 -

112 -Each file contains a `jobs` array. The provider kind is determined by the filename.

113 -

114 -Example (`/etc/netdata/go.d/ss/vault.conf`):

115 -

116 -```yaml

117 -jobs:

118 - - name: vault_prod

119 - mode: token

120 - mode_token:

121 - token: your-vault-token

122 - addr: https://vault.example.com

123 -```

124 -

125 -:::note

126 -

127 -File-based secretstores are loaded at agent startup. If you edit these files, restart the Netdata Agent to apply the changes.

128 -

129 -:::

130 -

131 -## How It Works

132 -

133 -- Secrets are resolved each time a collector job starts or restarts.

134 -- If a secret cannot be resolved, the collector job will fail to start and log an error.

135 -- Updating a secretstore automatically restarts running and failed collector jobs that use it, so they pick up the new credentials.

136 -- Accepted or disabled jobs keep their state and use the updated secretstore the next time they start.

137 -- If a secretstore change applies successfully but some dependent collector restarts fail, the command reports those restart failures.

138 -

139 -:::tip

140 -

141 -Avoid storing plain-text credentials in collector configurations. Use environment variables, files, commands, or secretstores instead.

142 -

143 -:::

integrations/README.md

+10 -4

@@ -20,7 +20,13 @@ as a VM or Docker container:

20 - On Alpine: `apk add py3-jsonschema py3-referencing py3-jinja2 py3-ruamel.yaml`

21 - On Fedora or RHEL (EPEL is required on RHEL systems): `dnf install python3-jsonschema python3-referencing python3-jinja2 python3-ruamel-yaml`

23 -Once the environment is set up, simply run

24 -`integrations/gen_integrations.py` from the Agent repo. Note that the

25 -script must be run _from this specific location_, as it uses it’s own

26 -path to figure out where all the files it needs are.

23 +Once the environment is set up, run the documentation generators from

24 +the Agent repo root:

25 +

26 +- `integrations/gen_integrations.py`

27 +- `integrations/gen_docs_integrations.py`

28 +- `integrations/gen_doc_collector_page.py`

29 +- `integrations/gen_doc_secrets_page.py`

30 +

31 +These scripts must be run _from this specific location_, as they use

32 +their own path to figure out where all the files they need are.

integrations/gen_doc_secrets_page.py new

+330

@@ -0,0 +1,330 @@

1 +"""

2 +Generate src/collectors/SECRETS.md from integrations/integrations.js.

3 +

4 +This script:

5 +- reads discovered secretstore integrations from integrations.js;

6 +- renders a shared Secrets Management entry page;

7 +- keeps shared resolver documentation in one dedicated configuration block.

8 +"""

9 +

10 +from __future__ import annotations

11 +

12 +import json

13 +import pathlib

14 +import re

15 +from typing import Any, Dict, List

16 +

17 +GITHUB_BLOB_PREFIX = "https://github.com/netdata/netdata/blob/master"

18 +TEMPLATE_PATH = pathlib.Path(__file__).resolve().parent / "templates"

19 +

20 +SECRETS_PAGE = {

21 + "title": "# Secrets Management",

22 + "intro": [

23 + "Keep collector credentials out of plain-text configuration files.",

24 + "Netdata lets you reference secret values in collector configs instead of storing them directly in YAML. "

25 + "Depending on where the secret lives, you can resolve it from environment variables, local files, local "

26 + "commands, or remote secretstore backends.",

27 + ],

28 + "jump_to": [

29 + {"label": "Resolver Quick Reference", "anchor": "resolver-quick-reference"},

30 + {"label": "Environment Variables", "anchor": "environment-variables"},

31 + {"label": "Files", "anchor": "files"},

32 + {"label": "Commands", "anchor": "commands"},

33 + {"label": "Secretstores", "anchor": "secretstores"},

34 + {"label": "Supported Secretstore Backends", "anchor": "supported-secretstore-backends"},

35 + {"label": "How It Works", "anchor": "how-it-works"},

36 + {"label": "Troubleshooting", "anchor": "troubleshooting"},

37 + ],

38 + "quick_reference": [

39 + {

40 + "resolver": "Environment variable",

41 + "syntax": "`${env:VAR_NAME}`",

42 + "best_for": "Secrets already injected into the Netdata service environment",

43 + "notes": "Value is trimmed. The variable must exist.",

44 + },

45 + {

46 + "resolver": "File",

47 + "syntax": "`${file:/absolute/path}`",

48 + "best_for": "Secrets stored in local files on disk",

49 + "notes": "The path must be absolute. File contents are trimmed.",

50 + },

51 + {

52 + "resolver": "Command",

53 + "syntax": "`${cmd:/absolute/path/to/command args}`",

54 + "best_for": "Secrets returned by a trusted local command",

55 + "notes": "The command path must be absolute. Netdata uses a 10-second timeout.",

56 + },

57 + {

58 + "resolver": "Secretstore",

59 + "syntax": "`${store:<kind>:<name>:<operand>}`",

60 + "best_for": "Secrets stored in remote backends such as Vault, AWS, Azure, or GCP",

61 + "notes": "Configure the secretstore first, then reference it from collector configs.",

62 + },

63 + ],

64 + "sections": {

65 + "env": {

66 + "heading": "## Environment Variables",

67 + "body": "Use `${env:VARIABLE_NAME}` to read a secret from the Netdata process environment.",

68 + "example": """```yaml

69 +jobs:

70 + - name: mysql_prod

71 + password: "${env:MYSQL_PASSWORD}"

72 +```""",

73 + "notes": [

74 + "Netdata trims leading and trailing whitespace from the environment variable value.",

75 + "The variable must be set in the environment of the Netdata service or process that runs the collector.",

76 + ],

77 + },

78 + "file": {

79 + "heading": "## Files",

80 + "body": "Use `${file:/absolute/path}` to read a secret from a local file on disk.",

81 + "example": """```yaml

82 +jobs:

83 + - name: mysql_prod

84 + password: "${file:/run/secrets/mysql_password}"

85 +```""",

86 + "notes": [

87 + "The file path must be absolute.",

88 + "Netdata trims leading and trailing whitespace from the file contents.",

89 + "The file must exist on the Netdata host and be readable by the `netdata` user.",

90 + ],

91 + },

92 + "cmd": {

93 + "heading": "## Commands",

94 + "body": "Use `${cmd:/absolute/path/to/command args}` to execute a trusted local command and use its stdout as the secret value.",

95 + "example": """```yaml

96 +jobs:

97 + - name: mysql_prod

98 + password: "${cmd:/usr/bin/op read op://vault/netdata/mysql/password}"

99 +```""",

100 + "notes": [

101 + "The command path must be absolute.",

102 + "Arguments are split on whitespace. Netdata does not interpret shell quoting, pipes, redirects, or variable expansion unless you explicitly run a shell such as `/bin/sh -c`.",

103 + "Netdata uses a 10-second timeout for command resolvers.",

104 + "Netdata trims leading and trailing whitespace from stdout and ignores stderr.",

105 + ],

106 + },

107 + },

108 + "store": {

109 + "heading": "## Secretstores",

110 + "body": "Use secretstores when you want Netdata collectors to fetch secrets from remote backends at runtime instead of storing them locally in collector configs.",

111 + "reference_intro": "Configure a secretstore first, then reference it from collector configs with:",

112 + "reference_syntax": "${store:<kind>:<name>:<operand>}",

113 + "reference_parts": [

114 + {"name": "`kind`", "description": "Secretstore backend kind, such as `vault` or `aws-sm`."},

115 + {"name": "`name`", "description": "The store name you configured in Netdata, such as `vault_prod`."},

116 + {"name": "`operand`", "description": "Backend-specific identifier for the secret you want to read."},

117 + ],

118 + "example": """```yaml

119 +jobs:

120 + - name: mysql_prod

121 + password: "${store:vault:vault_prod:secret/data/netdata/mysql#password}"

122 +```""",

123 + "ui_steps": [

124 + "Open the Netdata Dynamic Configuration UI.",

125 + "Go to `Collectors -> go.d -> SecretStores`.",

126 + "Choose the backend kind you want to configure.",

127 + "Give the secretstore a name.",

128 + "Fill in the backend-specific settings.",

129 + "Save the secretstore and use its `${store:<kind>:<name>:<operand>}` reference in collector configs.",

130 + ],

131 + "file_intro": "Each secretstore backend has its own file under `/etc/netdata/go.d/ss/`:",

132 + "file_note": "File-based secretstores are loaded at agent startup. If you edit these files, restart the Netdata Agent to apply the changes.",

133 + },

134 + "secretstores": {

135 + "heading": "## Supported Secretstore Backends",

136 + "intro": "Use the backend README for provider-specific authentication, operand rules, configuration examples, and troubleshooting.",

137 + },

138 + "how_it_works": [

139 + "Secrets are resolved each time a collector job starts or restarts.",

140 + "If a secret cannot be resolved, the collector job will fail to start and log an error.",

141 + "Updating a secretstore automatically restarts running and failed collector jobs that use it so they pick up the new credentials.",

142 + "Accepted or disabled jobs keep their state and use the updated secretstore the next time they start.",

143 + "If a secretstore change applies successfully but some dependent collector restarts fail, Netdata reports those restart failures.",

144 + ],

145 + "security_notes": [

146 + "Prefer secret references over plain-text credentials in collector configs.",

147 + "Prefer platform-native identity modes for production when a backend supports them, such as instance roles, managed identities, or metadata-based credentials.",

148 + "Keep local secret material readable only by the `netdata` user, including token files, service account files, and any files used with `${file:...}`.",

149 + "Use `${cmd:...}` only with trusted local commands and absolute paths.",

150 + ],

151 + "troubleshooting": {

152 + "intro": [

153 + "Secret resolution failures appear in agent logs and usually surface as collector jobs failing to start.",

154 + "Start by checking the resolver syntax you used in the collector config.",

155 + "For `${env:...}`, make sure the variable exists in the Netdata process environment.",

156 + "For `${file:...}`, make sure the path is absolute and the file is readable by `netdata`.",

157 + "For `${cmd:...}`, make sure the command path is absolute and the command completes within 10 seconds.",

158 + "For `${store:...}`, check the backend README for provider-specific operand rules, authentication requirements, and troubleshooting.",

159 + ],

160 + "errors": [

161 + {"syntax": "`${env:VAR_NAME}`", "message": "environment variable is not set"},

162 + {"syntax": "`${file:relative/path}`", "message": "file path must be absolute"},

163 + {"syntax": "`${cmd:echo hello}`", "message": "command path must be absolute"},

164 + {"syntax": "`${cmd:/path/to/slow-command}`", "message": "command timed out after 10s"},

165 + ],

166 + },

167 +}

168 +

169 +

170 +def _extract_integrations_json(js_text: str) -> str:

171 + """Extract integrations JSON from integrations.js."""

172 + after_categories = js_text.split("export const categories = ", 1)[1]

173 + _, after_integrations = after_categories.split("export const integrations = ", 1)

174 + return re.split(r"\n\s*export const|\Z", after_integrations, maxsplit=1)[0].strip().rstrip(';').strip()

175 +

176 +

177 +def load_integrations(js_path: str = "integrations/integrations.js") -> Any:

178 + """Load integrations catalog from JavaScript."""

179 + with open(js_path, "r", encoding="utf-8") as f:

180 + js_data = f.read()

181 + return json.loads(_extract_integrations_json(js_data))

182 +

183 +

184 +def iterate_integrations(integrations: Any):

185 + """Yield integration objects from dict or list."""

186 + if isinstance(integrations, dict):

187 + for integ in integrations.values():

188 + if isinstance(integ, dict):

189 + yield integ

190 + elif isinstance(integrations, list):

191 + for integ in integrations:

192 + if isinstance(integ, dict):

193 + yield integ

194 +

195 +

196 +def collect_secretstore_integrations(integrations: Any) -> List[Dict[str, Any]]:

197 + """Collect secretstore integration entries."""

198 + items = []

199 +

200 + for integ in iterate_integrations(integrations):

201 + if integ.get("integration_type") != "secretstore":

202 + continue

203 + meta = integ.get("meta", {})

204 + if not isinstance(meta, dict):

205 + continue

206 + if not isinstance(meta.get("name"), str) or not isinstance(meta.get("kind"), str):

207 + continue

208 + items.append(integ)

209 +

210 + items.sort(key=lambda item: item["meta"]["name"].lower())

211 + return items

212 +

213 +

214 +def get_repo_path_from_blob_url(url: str) -> str:

215 + """Convert a GitHub blob URL to a repo-root markdown path when possible."""

216 + if url.startswith(GITHUB_BLOB_PREFIX):

217 + return url[len(GITHUB_BLOB_PREFIX):]

218 + return url

219 +

220 +

221 +def get_secretstore_readme_link(integ: Dict[str, Any]) -> str:

222 + """Generate the repo-local README link for a secretstore backend."""

223 + edit_link = integ.get("edit_link", "") if isinstance(integ, dict) else ""

224 + repo_path = get_repo_path_from_blob_url(edit_link)

225 + if repo_path.endswith("/metadata.yaml"):

226 + return repo_path[: -len("metadata.yaml")] + "README.md"

227 + return ""

228 +

229 +

230 +_jinja_env = None

231 +

232 +

233 +def get_jinja_env():

234 + """Return the shared Jinja environment used by docs templates."""

235 + global _jinja_env

236 +

237 + if _jinja_env is None:

238 + from jinja2 import Environment, FileSystemLoader, select_autoescape

239 +

240 + _jinja_env = Environment(

241 + loader=FileSystemLoader(TEMPLATE_PATH),

242 + autoescape=select_autoescape(),

243 + block_start_string='[%',

244 + block_end_string='%]',

245 + variable_start_string='[[',

246 + variable_end_string=']]',

247 + comment_start_string='[#',

248 + comment_end_string='#]',

249 + trim_blocks=True,

250 + lstrip_blocks=True,

251 + )

252 +

253 + return _jinja_env

254 +

255 +

256 +def build_secretstores_context(integrations: Any) -> List[Dict[str, str]]:

257 + """Build template context for discovered secretstore backends."""

258 + items = []

259 +

260 + for integ in collect_secretstore_integrations(integrations):

261 + meta = integ.get("meta", {})

262 + summary = integ.get("collector_configs_summary", {})

263 + if not isinstance(summary, dict):

264 + summary = {}

265 + kind = meta.get("kind", "")

266 + readme_link = get_secretstore_readme_link(integ)

267 + name = meta.get("name", "")

268 + items.append(

269 + {

270 + "name": name,

271 + "kind": kind,

272 + "config_file": f"/etc/netdata/go.d/ss/{kind}.conf",

273 + "name_link": f'[{name}]({readme_link})',

274 + "operand_format": summary.get("operand_format", "See backend README"),

275 + "example_operand": summary.get("example_operand", "See backend README"),

276 + }

277 + )

278 +

279 + return items

280 +

281 +

282 +def build_page_context() -> Dict[str, Any]:

283 + """Build template-friendly page context."""

284 + return {

285 + "title": SECRETS_PAGE["title"],

286 + "intro": SECRETS_PAGE["intro"],

287 + "jump_to_line": " • ".join(

288 + f'[{jump["label"]}](#{jump["anchor"]})' for jump in SECRETS_PAGE["jump_to"]

289 + ),

290 + "quick_reference": SECRETS_PAGE["quick_reference"],

291 + "sections": [

292 + SECRETS_PAGE["sections"]["env"],

293 + SECRETS_PAGE["sections"]["file"],

294 + SECRETS_PAGE["sections"]["cmd"],

295 + ],

296 + "store": SECRETS_PAGE["store"],

297 + "secretstores": {

298 + "heading": SECRETS_PAGE["secretstores"]["heading"],

299 + "intro": SECRETS_PAGE["secretstores"]["intro"],

300 + },

301 + "how_it_works": SECRETS_PAGE["how_it_works"],

302 + "security_notes": SECRETS_PAGE["security_notes"],

303 + "troubleshooting": SECRETS_PAGE["troubleshooting"],

304 + }

305 +

306 +

307 +def render_secrets_md(integrations: Any) -> str:

308 + """Render the shared Secrets Management entry page."""

309 + template = get_jinja_env().get_template("secrets.md")

310 + return template.render(

311 + page=build_page_context(),

312 + secretstores=build_secretstores_context(integrations),

313 + )

314 +

315 +

316 +def generate_secrets_md() -> None:

317 + """Generate SECRETS.md from integrations.js and shared resolver content."""

318 + integrations = load_integrations()

319 + content = render_secrets_md(integrations)

320 +

321 + outfile = pathlib.Path("./src/collectors/SECRETS.md")

322 + outfile.parent.mkdir(parents=True, exist_ok=True)

323 +

324 + tmp = outfile.with_suffix(outfile.suffix + ".tmp")

325 + tmp.write_text(content.rstrip("\n") + "\n", encoding="utf-8")

326 + tmp.replace(outfile)

327 +

328 +

329 +if __name__ == "__main__":

330 + generate_secrets_md()

integrations/gen_docs_integrations.py

+55 -8

@@ -32,6 +32,7 @@ def cleanup(only_base_paths=None):

32 "integrations/cloud-notifications",

33 "integrations/logs",

34 "integrations/cloud-authentication",

35 + "src/go/plugin/agent/secrets/secretstore/backends",

36 ]

37 bases = only_base_paths if only_base_paths else targets

38 for base in bases:

@@ -92,19 +93,21 @@ def build_path(meta_yaml_link: str) -> str:

93 # Content builders

94 # -----------------------------

95 def add_custom_edit_url(markdown_string: str, meta_yaml_link: str, sidebar_label_string: str,

95 - mode: str = "default") -> str:

96 + mode: str = "default", output_slug: str = None) -> str:

97 """

98 Inject custom_edit_url into the metadata header.

99 """

100 + slug = output_slug or clean_string(sidebar_label_string)

101 +

102 if mode == "default":

100 - path_to_md_file = f"{meta_yaml_link.replace('/metadata.yaml', '')}/integrations/{clean_string(sidebar_label_string)}"

103 + path_to_md_file = f"{meta_yaml_link.replace('/metadata.yaml', '')}/integrations/{slug}"

104 elif mode in ("cloud-notification", "logs", "cloud-authentication"):

102 - path_to_md_file = meta_yaml_link.replace("metadata.yaml", f"integrations/{clean_string(sidebar_label_string)}")

105 + path_to_md_file = meta_yaml_link.replace("metadata.yaml", f"integrations/{slug}")

106 elif mode == "agent-notification":

107 path_to_md_file = meta_yaml_link.replace("metadata.yaml", "README")

108 else:

109 # safe fallback

107 - path_to_md_file = f"{meta_yaml_link.replace('/metadata.yaml', '')}/integrations/{clean_string(sidebar_label_string)}"

110 + path_to_md_file = f"{meta_yaml_link.replace('/metadata.yaml', '')}/integrations/{slug}"

111

112 return markdown_string.replace(

113 "<!--startmeta", f"<!--startmeta\ncustom_edit_url: \"{path_to_md_file}.md\""

@@ -354,6 +357,34 @@ endmeta-->

357 if integration.get("troubleshooting"):

358 md += f"\n{integration['troubleshooting']}\n"

359

360 + elif mode == "secretstore":

361 + meta_yaml = integration["edit_link"].replace("blob", "edit")

362 + sidebar_label = integration["meta"]["name"]

363 + learn_rel_path = "Collecting Metrics/Secret Stores"

364 + keywords = integration["keywords"] if "keywords" in integration else None

365 +

366 + md = f"""<!--startmeta

367 +meta_yaml: "{meta_yaml}"

368 +sidebar_label: "{sidebar_label}"

369 +learn_status: "Published"

370 +learn_rel_path: "{learn_rel_path}"

371 +"""

372 + if keywords:

373 + md += f"keywords: {keywords}\n"

374 +

375 + md += """message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE SECRETSTORE'S metadata.yaml FILE"

376 +endmeta-->

377 +

378 +"""

379 + md += create_overview(integration, integration['meta']['icon_filename'])

380 +

381 + if integration.get("setup"):

382 + md += f"\n{integration['setup']}\n"

383 + if integration.get("collector_configs"):

384 + md += f"\n{integration['collector_configs']}\n"

385 + if integration.get("troubleshooting"):

386 + md += f"\n{integration['troubleshooting']}\n"

387 +

388 except Exception as e:

389 print("Exception building md", e, integration.get("id"))

390

@@ -376,7 +407,7 @@ def create_overview_banner(md: str, community_badge: str) -> str:

407

408

409 def write_to_file(path: str, md: str, meta_yaml: str, sidebar_label: str, community: str, integration=None,

379 - mode: str = "default", integration_id: str = None):

410 + mode: str = "default", integration_id: str = None, output_slug: str = None):

411 """

412 Write the generated markdown into an `integrations/` subdirectory located alongside the `metadata.yaml` file.

413 This mirrors the original behavior of placing docs next to their source metadata.

@@ -389,10 +420,11 @@ def write_to_file(path: str, md: str, meta_yaml: str, sidebar_label: str, commun

420 if base.exists():

421 integrations_dir = base / "integrations"

422 integrations_dir.mkdir(exist_ok=True)

423 + slug = output_slug or clean_string(sidebar_label)

424

425 try:

394 - md2 = add_custom_edit_url(md, meta_yaml, sidebar_label)

395 - outfile = integrations_dir / f"{clean_string(sidebar_label)}.md"

426 + md2 = add_custom_edit_url(md, meta_yaml, sidebar_label, output_slug=slug)

427 + outfile = integrations_dir / f"{slug}.md"

428 clean_and_write(md2, outfile)

429 if integration_id:

430 id_to_path[integration_id] = str(outfile)

@@ -401,7 +433,7 @@ def write_to_file(path: str, md: str, meta_yaml: str, sidebar_label: str, commun

433

434 # If there's only one file inside the directory, register it for README symlink

435 if len(list(integrations_dir.iterdir())) == 1:

404 - symlink_dict.update({path: f"integrations/{clean_string(sidebar_label)}.md"})

436 + symlink_dict.update({path: f"integrations/{slug}.md"})

437 else:

438 try:

439 symlink_dict.pop(path)

@@ -562,6 +594,21 @@ def main():

594 path = build_path(meta_yaml)

595 write_to_file(path, md, meta_yaml, sidebar_label, community, integration_id=iid)

596

597 + elif itype == "secretstore" and not args.collector:

598 + meta_yaml, sidebar_label, learn_rel_path, md, community = build_readme_from_integration(

599 + integration, categories, mode="secretstore"

600 + )

601 + path = build_path(meta_yaml)

602 + write_to_file(

603 + path,

604 + md,

605 + meta_yaml,

606 + sidebar_label,

607 + community,

608 + integration_id=iid,

609 + output_slug=clean_string(integration["meta"]["kind"]),

610 + )

611 +

612 elif itype == "agent_notification" and not args.collector:

613 meta_yaml, sidebar_label, learn_rel_path, md, community = build_readme_from_integration(

614 integration, categories, mode="agent-notification"

integrations/gen_integrations.py

+137 -8

@@ -60,6 +60,10 @@ AUTHENTICATION_SOURCES = [

60 (AGENT_REPO, INTEGRATIONS_PATH / 'cloud-authentication' / 'metadata.yaml', False),

61 ]

63 +SECRETSTORE_SOURCES = [

64 + (AGENT_REPO, REPO_PATH / 'src' / 'go' / 'plugin' / 'agent' / 'secrets' / 'secretstore' / 'backends', True),

65 +]

66 +

67 COLLECTOR_RENDER_KEYS = [

68 'alerts',

69 'metrics',

@@ -98,6 +102,13 @@ AUTHENTICATION_RENDER_KEYS = [

102 'troubleshooting',

103 ]

104

105 +SECRETSTORE_RENDER_KEYS = [

106 + 'overview',

107 + 'setup',

108 + 'collector_configs',

109 + 'troubleshooting',

110 +]

111 +

112 CUSTOM_TAG_PATTERN = re.compile('\\{% if .*?%\\}.*?\\{% /if %\\}|\\{%.*?%\\}', flags=re.DOTALL)

113 FIXUP_BLANK_PATTERN = re.compile('\\\\\\n *\\n')

114

@@ -185,6 +196,11 @@ COLLECTOR_VALIDATOR = Draft7Validator(

196 registry=registry,

197 )

198

199 +SECRETSTORE_VALIDATOR = Draft7Validator(

200 + {'$ref': './secretstore.json#'},

201 + registry=registry,

202 +)

203 +

204 _jinja_env = False

205

206

@@ -231,6 +247,18 @@ def anchorfy(value):

247 return anchor

248

249

250 +def get_section_template_name(item, key):

251 + if key != 'setup':

252 + return f'{key}.md'

253 +

254 + integration_type = item.get('integration_type')

255 + if integration_type == 'secretstore':

256 + return 'setup-secretstore.md'

257 + if integration_type == 'logs':

258 + return 'setup-logs.md'

259 + return 'setup-generic.md'

260 +

261 +

262 def get_category_sets(categories):

263 default = set()

264 valid = set()

@@ -613,6 +641,54 @@ def load_authentications():

641 return ret

642

643

644 +def _load_secretstore_file(file, repo):

645 + debug(f'Loading {file}.')

646 + data = load_yaml(file)

647 +

648 + if not data:

649 + return []

650 +

651 + try:

652 + SECRETSTORE_VALIDATOR.validate(data)

653 + except ValidationError as e:

654 + warn(

655 + f'Failed to validate {file} against the schema: {e.message} (path: {"/".join(str(p) for p in e.absolute_path)})',

656 + file)

657 + return []

658 +

659 + if 'id' in data:

660 + data['integration_type'] = 'secretstore'

661 + data['_src_path'] = file

662 + data['_repo'] = repo

663 + data['_index'] = 0

664 +

665 + return [data]

666 + else:

667 + ret = []

668 +

669 + for idx, item in enumerate(data):

670 + item['integration_type'] = 'secretstore'

671 + item['_src_path'] = file

672 + item['_repo'] = repo

673 + item['_index'] = idx

674 + ret.append(item)

675 +

676 + return ret

677 +

678 +

679 +def load_secretstores():

680 + ret = []

681 +

682 + for repo, path, match in SECRETSTORE_SOURCES:

683 + if match and path.exists() and path.is_dir():

684 + for file in path.glob(METADATA_PATTERN):

685 + ret.extend(_load_secretstore_file(file, repo))

686 + elif not match and path.exists() and path.is_file():

687 + ret.extend(_load_secretstore_file(path, repo))

688 +

689 + return ret

690 +

691 +

692 def make_id(meta):

693 if 'monitored_instance' in meta:

694 instance_name = meta['monitored_instance']['name'].replace(' ', '_')

@@ -773,7 +849,7 @@ def render_collectors(categories, collectors, ids):

849

850 for key in COLLECTOR_RENDER_KEYS:

851 if key in item.keys():

776 - template = get_jinja_env().get_template(f'{key}.md')

852 + template = get_jinja_env().get_template(get_section_template_name(item, key))

853 data = template.render(entry=item, related=related, clean=False)

854 clean_data = template.render(entry=item, related=related, clean=True)

855

@@ -867,7 +943,7 @@ def render_exporters(categories, exporters, ids):

943

944 for key in EXPORTER_RENDER_KEYS:

945 if key in item.keys():

870 - template = get_jinja_env().get_template(f'{key}.md')

946 + template = get_jinja_env().get_template(get_section_template_name(item, key))

947 data = template.render(entry=item, clean=False)

948 clean_data = template.render(entry=item, clean=True)

949

@@ -909,7 +985,7 @@ def render_agent_notifications(categories, notifications, ids):

985

986 for key in AGENT_NOTIFICATION_RENDER_KEYS:

987 if key in item.keys():

912 - template = get_jinja_env().get_template(f'{key}.md')

988 + template = get_jinja_env().get_template(get_section_template_name(item, key))

989 data = template.render(entry=item, clean=False)

990

991 clean_data = template.render(entry=item, clean=True)

@@ -952,7 +1028,7 @@ def render_cloud_notifications(categories, notifications, ids):

1028

1029 for key in CLOUD_NOTIFICATION_RENDER_KEYS:

1030 if key in item.keys():

955 - template = get_jinja_env().get_template(f'{key}.md')

1031 + template = get_jinja_env().get_template(get_section_template_name(item, key))

1032 data = template.render(entry=item, clean=False)

1033 clean_data = template.render(entry=item, clean=True)

1034

@@ -994,7 +1070,7 @@ def render_logs(categories, logs, ids):

1070

1071 for key in LOGS_RENDER_KEYS:

1072 if key in item.keys():

997 - template = get_jinja_env().get_template(f'{key}.md')

1073 + template = get_jinja_env().get_template(get_section_template_name(item, key))

1074 data = template.render(entry=item, clean=False)

1075 clean_data = template.render(entry=item, clean=True)

1076

@@ -1037,7 +1113,7 @@ def render_authentications(categories, authentications, ids):

1113 for key in AUTHENTICATION_RENDER_KEYS:

1114

1115 if key in item.keys():

1040 - template = get_jinja_env().get_template(f'{key}.md')

1116 + template = get_jinja_env().get_template(get_section_template_name(item, key))

1117 data = template.render(entry=item, clean=False)

1118 clean_data = template.render(entry=item, clean=True)

1119

@@ -1061,6 +1137,57 @@ def render_authentications(categories, authentications, ids):

1137 return authentications, clean_authentications, ids

1138

1139

1140 +def render_secretstores(categories, secretstores, ids):

1141 + debug('Sorting secretstores.')

1142 +

1143 + sort_integrations(secretstores)

1144 +

1145 + debug('Checking secretstore ids.')

1146 +

1147 + secretstores, ids = dedupe_integrations(secretstores, ids)

1148 +

1149 + clean_secretstores = []

1150 +

1151 + for item in secretstores:

1152 + item['edit_link'] = make_edit_link(item)

1153 +

1154 + clean_item = deepcopy(item)

1155 + collector_configs = item.get('collector_configs', {})

1156 + collector_configs_summary = {}

1157 + if isinstance(collector_configs, dict):

1158 + summary = collector_configs.get('summary', {})

1159 + if isinstance(summary, dict):

1160 + collector_configs_summary = deepcopy(summary)

1161 +

1162 + item['collector_configs_summary'] = deepcopy(collector_configs_summary)

1163 + clean_item['collector_configs_summary'] = deepcopy(collector_configs_summary)

1164 +

1165 + for key in SECRETSTORE_RENDER_KEYS:

1166 + if key in item.keys():

1167 + template = get_jinja_env().get_template(get_section_template_name(item, key))

1168 + data = template.render(entry=item, clean=False)

1169 + clean_data = template.render(entry=item, clean=True)

1170 +

1171 + if 'variables' in item['meta']:

1172 + template = get_jinja_env().from_string(data)

1173 + data = template.render(variables=item['meta']['variables'], clean=False)

1174 + template = get_jinja_env().from_string(clean_data)

1175 + clean_data = template.render(variables=item['meta']['variables'], clean=True)

1176 + else:

1177 + data = ''

1178 + clean_data = ''

1179 +

1180 + item[key] = data

1181 + clean_item[key] = clean_data

1182 +

1183 + for k in ['_src_path', '_repo', '_index']:

1184 + del item[k], clean_item[k]

1185 +

1186 + clean_secretstores.append(clean_item)

1187 +

1188 + return secretstores, clean_secretstores, ids

1189 +

1190 +

1191 def convert_local_links(text, prefix):

1192 return text.replace("](/", f"]({prefix}/")

1193

@@ -1092,6 +1219,7 @@ def main():

1219 cloud_notifications = load_cloud_notifications()

1220 logs = load_logs()

1221 authentications = load_authentications()

1222 + secretstores = load_secretstores()

1223

1224 collectors, clean_collectors, ids = render_collectors(categories, collectors, dict())

1225 deploy, clean_deploy, ids = render_deploy(distros, categories, deploy, ids)

@@ -1102,11 +1230,12 @@ def main():

1230 ids)

1231 logs, clean_logs, ids = render_logs(categories, logs, ids)

1232 authentications, clean_authentications, ids = render_authentications(categories, authentications, ids)

1233 + secretstores, clean_secretstores, ids = render_secretstores(categories, secretstores, ids)

1234

1106 - integrations = collectors + deploy + exporters + agent_notifications + cloud_notifications + logs + authentications

1235 + integrations = collectors + deploy + exporters + agent_notifications + cloud_notifications + logs + authentications + secretstores

1236 render_integrations(categories, integrations)

1237

1109 - clean_integrations = clean_collectors + clean_deploy + clean_exporters + clean_agent_notifications + clean_cloud_notifications + clean_logs + clean_authentications

1238 + clean_integrations = clean_collectors + clean_deploy + clean_exporters + clean_agent_notifications + clean_cloud_notifications + clean_logs + clean_authentications + clean_secretstores

1239 render_json(categories, clean_integrations)

1240

1241 return fail_on_warnings()

integrations/logs/integrations/opentelemetry_logs.md

+2 -3

@@ -40,13 +40,12 @@ You can start exploring OpenTelemetry logs on the "Logs" tab of the Netdata UI.

41 ## Setup

43 -## Prerequisites

43 +### Prerequisites

45 - A Netdata Cloud account

46 - The `otel.plugin` configured to ingest OpenTelemetry logs

49 -## Configuration

49 +### Configuration

51 There is no configuration needed for this integration.

52 -

integrations/logs/integrations/systemd_journal_logs.md

+2 -3

@@ -42,12 +42,11 @@ You can start exploring `systemd` journal logs on the "Logs" tab of the Netdata

43 ## Setup

45 -## Prerequisites

45 +### Prerequisites

47 - A Netdata Cloud account

50 -## Configuration

50 +### Configuration

52 There is no configuration needed for this integration.

53 -

integrations/logs/integrations/windows_event_logs.md

+2 -3

@@ -42,12 +42,11 @@ You can start exploring Windows event logs on the "Logs" tab of the Netdata UI.

43 ## Setup

45 -## Prerequisites

45 +### Prerequisites

47 - Netdata Cloud paid subscription

50 -## Configuration

50 +### Configuration

52 There is no configuration needed for this integration.

53 -

integrations/schemas/secretstore.json new

+225

@@ -0,0 +1,225 @@

1 +{

2 + "$schema": "http://json-schema.org/draft-07/schema#",

3 + "title": "Netdata secretstore backend metadata.",

4 + "oneOf": [

5 + {

6 + "$ref": "#/$defs/entry"

7 + },

8 + {

9 + "type": "array",

10 + "minItems": 1,

11 + "items": {

12 + "$ref": "#/$defs/entry"

13 + }

14 + }

15 + ],

16 + "$defs": {

17 + "meta": {

18 + "type": "object",

19 + "description": "Information about the secretstore backend.",

20 + "properties": {

21 + "kind": {

22 + "type": "string",

23 + "description": "Runtime secretstore kind used in ${store:<kind>:...} references and UI paths."

24 + },

25 + "name": {

26 + "type": "string",

27 + "description": "Secretstore display name."

28 + },

29 + "link": {

30 + "type": "string",

31 + "description": "Official provider or product page."

32 + },

33 + "icon_filename": {

34 + "type": "string",

35 + "description": "The filename of the integration icon."

36 + }

37 + },

38 + "required": [

39 + "kind",

40 + "name",

41 + "link",

42 + "icon_filename"

43 + ]

44 + },

45 + "overview": {

46 + "type": "object",

47 + "description": "General information about the secretstore backend.",

48 + "properties": {

49 + "description": {

50 + "type": "string",

51 + "description": "General description of what the secretstore backend does."

52 + },

53 + "limitations": {

54 + "type": "string",

55 + "description": "Optional explanation of notable limitations or behavior."

56 + }

57 + },

58 + "required": [

59 + "description"

60 + ]

61 + },

62 + "collector_configs_part": {

63 + "type": "object",

64 + "properties": {

65 + "name": {

66 + "type": "string",

67 + "description": "Part of the secret reference syntax."

68 + },

69 + "description": {

70 + "type": "string",

71 + "description": "What this part means."

72 + }

73 + },

74 + "required": [

75 + "name",

76 + "description"

77 + ]

78 + },

79 + "collector_configs_example": {

80 + "type": "object",

81 + "properties": {

82 + "name": {

83 + "type": "string",

84 + "description": "Example name."

85 + },

86 + "description": {

87 + "type": "string",

88 + "description": "Example description."

89 + },

90 + "language": {

91 + "type": "string",

92 + "description": "Code fence language. Defaults to text if omitted."

93 + },

94 + "content": {

95 + "type": "string",

96 + "description": "Reference or collector configuration snippet."

97 + }

98 + },

99 + "required": [

100 + "name",

101 + "description",

102 + "content"

103 + ]

104 + },

105 + "collector_configs_summary": {

106 + "type": "object",

107 + "description": "Short backend summary fields used by the shared SECRETS.md page.",

108 + "properties": {

109 + "operand_format": {

110 + "type": "string",

111 + "description": "Short backend operand format summary."

112 + },

113 + "example_operand": {

114 + "type": "string",

115 + "description": "Short backend operand example."

116 + }

117 + },

118 + "required": [

119 + "operand_format",

120 + "example_operand"

121 + ]

122 + },

123 + "collector_configs": {

124 + "type": "object",

125 + "description": "How to use the secretstore backend from collector configurations.",

126 + "properties": {

127 + "description": {

128 + "type": "string",

129 + "description": "Introductory text for collector usage."

130 + },

131 + "summary": {

132 + "$ref": "#/$defs/collector_configs_summary"

133 + },

134 + "format": {

135 + "type": "object",

136 + "properties": {

137 + "description": {

138 + "type": "string",

139 + "description": "Optional text explaining backend-specific operand behavior."

140 + },

141 + "syntax": {

142 + "type": "string",

143 + "description": "Secret reference syntax."

144 + },

145 + "parts": {

146 + "type": "object",

147 + "properties": {

148 + "list": {

149 + "type": "array",

150 + "items": {

151 + "$ref": "#/$defs/collector_configs_part"

152 + }

153 + }

154 + },

155 + "required": [

156 + "list"

157 + ]

158 + }

159 + },

160 + "required": [

161 + "syntax",

162 + "parts"

163 + ]

164 + },

165 + "examples": {

166 + "type": "object",

167 + "properties": {

168 + "list": {

169 + "type": "array",

170 + "minLength": 1,

171 + "items": {

172 + "$ref": "#/$defs/collector_configs_example"

173 + }

174 + }

175 + },

176 + "required": [

177 + "list"

178 + ]

179 + }

180 + },

181 + "required": [

182 + "description",

183 + "summary",

184 + "format",

185 + "examples"

186 + ]

187 + },

188 + "entry": {

189 + "type": "object",

190 + "description": "Data for a single secretstore backend.",

191 + "properties": {

192 + "id": {

193 + "$ref": "./shared.json#/$defs/id"

194 + },

195 + "meta": {

196 + "$ref": "#/$defs/meta"

197 + },

198 + "keywords": {

199 + "$ref": "./shared.json#/$defs/keywords"

200 + },

201 + "overview": {

202 + "$ref": "#/$defs/overview"

203 + },

204 + "setup": {

205 + "$ref": "./shared.json#/$defs/full_setup"

206 + },

207 + "collector_configs": {

208 + "$ref": "#/$defs/collector_configs"

209 + },

210 + "troubleshooting": {

211 + "$ref": "./shared.json#/$defs/troubleshooting"

212 + }

213 + },

214 + "required": [

215 + "id",

216 + "meta",

217 + "keywords",

218 + "overview",

219 + "setup",

220 + "collector_configs",

221 + "troubleshooting"

222 + ]

223 + }

224 + }

225 +}

integrations/templates/README.md

+8 -1

@@ -17,11 +17,18 @@ differences from the defaults:

17 the first newline after a block will be _removed_, as will any leading

18 whitespace on the same line as a block.

20 -Each markdown template corresponds to the key of the same name in the

20 +Most markdown templates correspond to the key of the same name in the

21 integrations objects in that file. Those templates get passed the

22 integration data using the name `entry`, plus the composed related

23 resource data using the name `rel_res`.

25 +The `setup` section is a special case and is selected by integration

26 +type in `gen_integrations.py`:

27 +

28 +- `setup-generic.md` for the generic integration path

29 +- `setup-logs.md` for logs integrations

30 +- `setup-secretstore.md` for secretstore integrations

31 +

32 The `integrations.js` template is used to compose the final file. It gets

33 passed the JSON-formatted category and integration data using the names

34 `categories` and `integrations` respectively.

integrations/templates/collector_configs.md new

+26

@@ -0,0 +1,26 @@

1 +## Use in collector configs

2 +

3 +[[ entry.collector_configs.description ]]

4 +

5 +[% if entry.collector_configs.format.description %]

6 +[[ entry.collector_configs.format.description ]]

7 +[% endif %]

8 +

9 +```text

10 +[[ entry.collector_configs.format.syntax ]]

11 +```

12 +

13 +[% for part in entry.collector_configs.format.parts.list %]

14 +- `[[ part.name ]]`: [[ part.description ]]

15 +[% endfor %]

16 +

17 +### Examples

18 +[% for example in entry.collector_configs.examples.list %]

19 +#### [[ example.name ]]

20 +

21 +[[ example.description ]]

22 +

23 +```[[ example.language or 'text' ]]

24 +[[ example.content ]]

25 +```

26 +[% endfor %]

integrations/templates/overview.md

@@ -1,5 +1,7 @@

1 [% if entry.integration_type == 'collector' %]

2 [% include 'overview/collector.md' %]

3 +[% elif entry.integration_type == 'secretstore' %]

4 +[% include 'overview/secretstore.md' %]

5 [% elif entry.integration_type == 'exporter' %]

6 [% include 'overview/exporter.md' %]

7 [% elif entry.integration_type == 'agent_notification' %]

integrations/templates/overview/secretstore.md new

+13

@@ -0,0 +1,13 @@

1 +# [[ entry.meta.name ]]

2 +

3 +Kind: [[ entry.meta.kind ]]

4 +

5 +## Overview

6 +

7 +[[ entry.overview.description ]]

8 +

9 +[% if entry.overview.limitations %]

10 +### Limitations

11 +

12 +[[ entry.overview.limitations ]]

13 +[% endif %]

integrations/templates/secrets.md new

+110

@@ -0,0 +1,110 @@

1 +[[ page.title ]]

2 +

3 +[% for paragraph in page.intro %]

4 +[[ paragraph ]]

5 +

6 +[% endfor %]

7 +### Jump To

8 +

9 +[[ page.jump_to_line ]]

10 +

11 +

12 +## Resolver Quick Reference

13 +

15 +|:---------|:-------|:---------|:------|

16 +[% for item in page.quick_reference %]

18 +[% endfor %]

19 +

20 +[% for section in page.sections %]

21 +[[ section.heading ]]

22 +

23 +[[ section.body ]]

24 +

25 +[[ section.example ]]

26 +

27 +[% for note in section.notes %]

28 +- [[ note ]]

29 +[% endfor %]

30 +

31 +[% endfor %]

32 +[[ page.store.heading ]]

33 +

34 +[[ page.store.body ]]

35 +

36 +[[ page.store.reference_intro ]]

37 +

38 +```text

39 +[[ page.store.reference_syntax ]]

40 +```

41 +

42 +| Part | Description |

43 +|:-----|:------------|

44 +[% for part in page.store.reference_parts %]

45 +| [[ part.name ]] | [[ part.description ]] |

46 +[% endfor %]

47 +

48 +Example:

49 +

50 +[[ page.store.example ]]

51 +

52 +### Configuration Methods

53 +

54 +#### Dynamic Configuration UI

55 +

56 +[% for step in page.store.ui_steps %]

57 +[[ loop.index ]]. [[ step ]]

58 +[% endfor %]

59 +

60 +#### Configuration Files

61 +

62 +[[ page.store.file_intro ]]

63 +

64 +| File | Backend |

65 +|:-----|:--------|

66 +[% for backend in secretstores %]

67 +| `[[ backend.config_file ]]` | [[ backend.name ]] |

68 +[% endfor %]

69 +

70 +Each file contains a `jobs` array. The backend kind is determined by the filename.

71 +

72 +:::note

73 +

74 +[[ page.store.file_note ]]

75 +

76 +:::

77 +

78 +[[ page.secretstores.heading ]]

79 +

80 +[[ page.secretstores.intro ]]

81 +

83 +|:--------|:-----|:---------------|:----------------|

84 +[% for backend in secretstores %]

86 +[% endfor %]

87 +

88 +## How It Works

89 +

90 +[% for item in page.how_it_works %]

91 +- [[ item ]]

92 +[% endfor %]

93 +

94 +## Security Notes

95 +

96 +[% for item in page.security_notes %]

97 +- [[ item ]]

98 +[% endfor %]

99 +

100 +## Troubleshooting

101 +

102 +[% for item in page.troubleshooting.intro %]

103 +- [[ item ]]

104 +[% endfor %]

105 +

106 +Representative error patterns:

107 +

108 +[% for err in page.troubleshooting.errors %]

109 +- [[ err.syntax ]]: [[ err.message ]]

110 +[% endfor %]

integrations/templates/setup-generic.md renamed

-11

@@ -1,14 +1,4 @@

1 ## Setup

2 -[% if entry.integration_type == 'logs' %]

3 -

4 -## Prerequisites

5 -

6 -[[ entry.setup.prerequisites.description]]

7 -

8 -## Configuration

9 -

10 -There is no configuration needed for this integration.

11 -[% else %]

3 [% if entry.meta.plugin_name == 'go.d.plugin' %]

@@ -171,4 +161,3 @@ There are no configuration examples.

161

162 [% endif %]

163 [% endif %]

174 -[% endif %]

integrations/templates/setup-logs.md new

@@ -0,0 +1,9 @@

1 +## Setup

2 +

3 +### Prerequisites

4 +

5 +[[ entry.setup.prerequisites.description]]

6 +

7 +### Configuration

8 +

9 +There is no configuration needed for this integration.

integrations/templates/setup-secretstore.md new

+113

@@ -0,0 +1,113 @@

1 +## Setup

2 +

3 +You can configure the `[[ entry.meta.kind ]]` secretstore in two ways:

4 +

5 +| Method | Best for | How to |

6 +|:--|:--|:--|

7 +| [**UI**](#via-ui) | Fast setup without editing files | Go to `Collectors -> go.d -> SecretStores -> [[ entry.meta.kind ]]`, then add a secretstore. |

8 +| [**File**](#via-file) | File-based configuration or automation | Edit `/etc/netdata/[[ entry.setup.configuration.file.name ]]` and add a `jobs` entry. |

9 +

10 +### Prerequisites

11 +[% if entry.setup.prerequisites.list %]

12 +

13 +[% for prereq in entry.setup.prerequisites.list %]

14 +#### [[ prereq.title ]]

15 +

16 +[[ prereq.description ]]

17 +

18 +[% endfor %]

19 +[% else %]

20 +

21 +No action required.

22 +

23 +[% endif %]

24 +### Configuration

25 +

26 +#### Options

27 +

28 +[[ entry.setup.configuration.options.description ]]

29 +

30 +[% if entry.setup.configuration.options.list %]

31 +[% if entry.setup.configuration.options.folding.enabled and not clean %]

32 +{% details open=true summary="[[ entry.setup.configuration.options.folding.title or 'Config options' ]]" %}

33 +[% endif %]

34 +

35 +[% set has_groups = entry.setup.configuration.options.list | selectattr("group","defined") | list | length > 0 %]

36 +

37 +[% if has_groups %]

39 +|:------|:-----|:------------|:--------|:---------:|

40 +[% set ns = namespace(last_group=None) %]

41 +[% for item in entry.setup.configuration.options.list %]

42 +[% set anchor_source = (item.group ~ "-" ~ item.name) if (item.group is defined and item.group) else item.name %]

43 +[% set item_anchor = "option-" ~ anchorfy(anchor_source) %]

44 +| [[ ("**" ~ item.group ~ "**") if (item.group is defined and item.group != ns.last_group) else "" ]] | [[ ("[" ~ strfy(item.name) ~ "](#" ~ item_anchor ~ ")") if ('detailed_description' in item) else strfy(item.name) ]] | [[ strfy(item.description) ]] | [[ strfy(item.default_value) ]] | [[ strfy(item.required) ]] |

45 +[% set ns.last_group = item.group if item.group is defined else ns.last_group %]

46 +[% endfor %]

47 +[% else %]

49 +|:-----|:------------|:--------|:---------:|

50 +[% for item in entry.setup.configuration.options.list %]

51 +[% set anchor_source = (item.group ~ "-" ~ item.name) if (item.group is defined and item.group) else item.name %]

52 +[% set item_anchor = "option-" ~ anchorfy(anchor_source) %]

53 +| [[ ("[" ~ strfy(item.name) ~ "](#" ~ item_anchor ~ ")") if ('detailed_description' in item) else strfy(item.name) ]] | [[ strfy(item.description) ]] | [[ strfy(item.default_value) ]] | [[ strfy(item.required) ]] |

54 +[% endfor %]

55 +[% endif %]

56 +

57 +[% for item in entry.setup.configuration.options.list %]

58 +[% if 'detailed_description' in item %]

59 +[% set anchor_source = (item.group ~ "-" ~ item.name) if (item.group is defined and item.group) else item.name %]

60 +<a id="[[ "option-" ~ anchorfy(anchor_source) ]]"></a>

61 +##### [[ item.name ]]

62 +

63 +[[ item.detailed_description ]]

64 +

65 +[% endif %]

66 +[% endfor %]

67 +

68 +[% if entry.setup.configuration.options.folding.enabled and not clean %]

69 +{% /details %}

70 +[% endif %]

71 +[% else %]

72 +There are no configuration options.

73 +

74 +[% endif %]

75 +

76 +#### via UI

77 +

78 +1. Open the Netdata Dynamic Configuration UI.

79 +2. Go to `Collectors -> go.d -> SecretStores -> [[ entry.meta.kind ]]`.

80 +3. Add a new secretstore and give it a store name.

81 +4. Fill in the backend-specific settings.

82 +5. Save the secretstore.

83 +

84 +#### via File

85 +

86 +Define the secretstore in `/etc/netdata/[[ entry.setup.configuration.file.name ]]`.

87 +

88 +Each file contains a `jobs` array, and the secretstore kind is determined by the filename.

89 +

90 +After editing the file, restart the Netdata Agent to load the updated secretstore definition.

91 +

92 +##### Examples

93 +[% if entry.setup.configuration.examples.list %]

94 +

95 +[% for example in entry.setup.configuration.examples.list %]

96 +###### [[ example.name ]]

97 +

98 +[[ example.description ]]

99 +

100 +[% if example.folding is defined and example.folding.enabled and not clean %]

101 +{% details open=true summary="[[ entry.setup.configuration.examples.folding.title or 'Example configuration' ]]" %}

102 +[% endif %]

103 +```yaml

104 +[[ example.config ]]

105 +```

106 +[% if example.folding is defined and example.folding.enabled and not clean %]

107 +{% /details %}

108 +[% endif %]

109 +[% endfor %]

110 +[% else %]

111 +There are no configuration examples.

112 +

113 +[% endif %]

integrations/templates/troubleshooting.md

@@ -123,6 +123,11 @@ Note that this will test _all_ alert mechanisms for the selected role.

123 [% if entry.troubleshooting.problems.list %]

124 ## Troubleshooting

125

126 +[% endif %]

127 +[% elif entry.integration_type == 'secretstore' %]

128 +[% if entry.troubleshooting.problems.list %]

129 +## Troubleshooting

130 +

131 [% endif %]

132 [% endif %]

133 [% for item in entry.troubleshooting.problems.list %]

src/collectors/SECRETS.md new

+157

@@ -0,0 +1,157 @@

1 +# Secrets Management

2 +

3 +Keep collector credentials out of plain-text configuration files.

4 +

5 +Netdata lets you reference secret values in collector configs instead of storing them directly in YAML. Depending on where the secret lives, you can resolve it from environment variables, local files, local commands, or remote secretstore backends.

6 +

7 +### Jump To

8 +

9 +[Resolver Quick Reference](#resolver-quick-reference) • [Environment Variables](#environment-variables) • [Files](#files) • [Commands](#commands) • [Secretstores](#secretstores) • [Supported Secretstore Backends](#supported-secretstore-backends) • [How It Works](#how-it-works) • [Troubleshooting](#troubleshooting)

10 +

11 +

12 +## Resolver Quick Reference

13 +

15 +|:---------|:-------|:---------|:------|

16 +| Environment variable | `${env:VAR_NAME}` | Secrets already injected into the Netdata service environment | Value is trimmed. The variable must exist. |

17 +| File | `${file:/absolute/path}` | Secrets stored in local files on disk | The path must be absolute. File contents are trimmed. |

18 +| Command | `${cmd:/absolute/path/to/command args}` | Secrets returned by a trusted local command | The command path must be absolute. Netdata uses a 10-second timeout. |

19 +| Secretstore | `${store:<kind>:<name>:<operand>}` | Secrets stored in remote backends such as Vault, AWS, Azure, or GCP | Configure the secretstore first, then reference it from collector configs. |

20 +

21 +## Environment Variables

22 +

23 +Use `${env:VARIABLE_NAME}` to read a secret from the Netdata process environment.

24 +

25 +```yaml

26 +jobs:

27 + - name: mysql_prod

28 + password: "${env:MYSQL_PASSWORD}"

29 +```

30 +

31 +- Netdata trims leading and trailing whitespace from the environment variable value.

32 +- The variable must be set in the environment of the Netdata service or process that runs the collector.

33 +

34 +## Files

35 +

36 +Use `${file:/absolute/path}` to read a secret from a local file on disk.

37 +

38 +```yaml

39 +jobs:

40 + - name: mysql_prod

41 + password: "${file:/run/secrets/mysql_password}"

42 +```

43 +

44 +- The file path must be absolute.

45 +- Netdata trims leading and trailing whitespace from the file contents.

46 +- The file must exist on the Netdata host and be readable by the `netdata` user.

47 +

48 +## Commands

49 +

50 +Use `${cmd:/absolute/path/to/command args}` to execute a trusted local command and use its stdout as the secret value.

51 +

52 +```yaml

53 +jobs:

54 + - name: mysql_prod

55 + password: "${cmd:/usr/bin/op read op://vault/netdata/mysql/password}"

56 +```

57 +

58 +- The command path must be absolute.

59 +- Arguments are split on whitespace. Netdata does not interpret shell quoting, pipes, redirects, or variable expansion unless you explicitly run a shell such as `/bin/sh -c`.

60 +- Netdata uses a 10-second timeout for command resolvers.

61 +- Netdata trims leading and trailing whitespace from stdout and ignores stderr.

62 +

63 +## Secretstores

64 +

65 +Use secretstores when you want Netdata collectors to fetch secrets from remote backends at runtime instead of storing them locally in collector configs.

66 +

67 +Configure a secretstore first, then reference it from collector configs with:

68 +

69 +```text

70 +${store:<kind>:<name>:<operand>}

71 +```

72 +

73 +| Part | Description |

74 +|:-----|:------------|

75 +| `kind` | Secretstore backend kind, such as `vault` or `aws-sm`. |

76 +| `name` | The store name you configured in Netdata, such as `vault_prod`. |

77 +| `operand` | Backend-specific identifier for the secret you want to read. |

78 +

79 +Example:

80 +

81 +```yaml

82 +jobs:

83 + - name: mysql_prod

84 + password: "${store:vault:vault_prod:secret/data/netdata/mysql#password}"

85 +```

86 +

87 +### Configuration Methods

88 +

89 +#### Dynamic Configuration UI

90 +

91 +1. Open the Netdata Dynamic Configuration UI.

92 +2. Go to `Collectors -> go.d -> SecretStores`.

93 +3. Choose the backend kind you want to configure.

94 +4. Give the secretstore a name.

95 +5. Fill in the backend-specific settings.

96 +6. Save the secretstore and use its `${store:<kind>:<name>:<operand>}` reference in collector configs.

97 +

98 +#### Configuration Files

99 +

100 +Each secretstore backend has its own file under `/etc/netdata/go.d/ss/`:

101 +

102 +| File | Backend |

103 +|:-----|:--------|

104 +| `/etc/netdata/go.d/ss/aws-sm.conf` | AWS Secrets Manager |

105 +| `/etc/netdata/go.d/ss/azure-kv.conf` | Azure Key Vault |

106 +| `/etc/netdata/go.d/ss/gcp-sm.conf` | Google Secret Manager |

107 +| `/etc/netdata/go.d/ss/vault.conf` | Vault |

108 +

109 +Each file contains a `jobs` array. The backend kind is determined by the filename.

110 +

111 +:::note

112 +

113 +File-based secretstores are loaded at agent startup. If you edit these files, restart the Netdata Agent to apply the changes.

114 +

115 +:::

116 +

117 +## Supported Secretstore Backends

118 +

119 +Use the backend README for provider-specific authentication, operand rules, configuration examples, and troubleshooting.

120 +

122 +|:--------|:-----|:---------------|:----------------|

127 +

128 +## How It Works

129 +

130 +- Secrets are resolved each time a collector job starts or restarts.

131 +- If a secret cannot be resolved, the collector job will fail to start and log an error.

132 +- Updating a secretstore automatically restarts running and failed collector jobs that use it so they pick up the new credentials.

133 +- Accepted or disabled jobs keep their state and use the updated secretstore the next time they start.

134 +- If a secretstore change applies successfully but some dependent collector restarts fail, Netdata reports those restart failures.

135 +

136 +## Security Notes

137 +

138 +- Prefer secret references over plain-text credentials in collector configs.

139 +- Prefer platform-native identity modes for production when a backend supports them, such as instance roles, managed identities, or metadata-based credentials.

140 +- Keep local secret material readable only by the `netdata` user, including token files, service account files, and any files used with `${file:...}`.

141 +- Use `${cmd:...}` only with trusted local commands and absolute paths.

142 +

143 +## Troubleshooting

144 +

145 +- Secret resolution failures appear in agent logs and usually surface as collector jobs failing to start.

146 +- Start by checking the resolver syntax you used in the collector config.

147 +- For `${env:...}`, make sure the variable exists in the Netdata process environment.

148 +- For `${file:...}`, make sure the path is absolute and the file is readable by `netdata`.

149 +- For `${cmd:...}`, make sure the command path is absolute and the command completes within 10 seconds.

150 +- For `${store:...}`, check the backend README for provider-specific operand rules, authentication requirements, and troubleshooting.

151 +

152 +Representative error patterns:

153 +

154 +- `${env:VAR_NAME}`: environment variable is not set

155 +- `${file:relative/path}`: file path must be absolute

156 +- `${cmd:echo hello}`: command path must be absolute

157 +- `${cmd:/path/to/slow-command}`: command timed out after 10s

src/go/plugin/agent/secrets/secretstore/backends/aws/README.md new

@@ -0,0 +1 @@

1 +integrations/aws-sm.md

\ No newline at end of file

src/go/plugin/agent/secrets/secretstore/backends/aws/integrations/aws-sm.md new

+223

@@ -0,0 +1,223 @@

1 +<!--startmeta

2 +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/aws/README.md"

3 +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/aws/metadata.yaml"

4 +sidebar_label: "AWS Secrets Manager"

5 +learn_status: "Published"

6 +learn_rel_path: "Collecting Metrics/Secret Stores"

7 +keywords: ['secretstore', 'secrets', 'aws', 'aws-sm', 'aws secrets manager']

8 +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE SECRETSTORE'S metadata.yaml FILE"

9 +endmeta-->

10 +

11 +# AWS Secrets Manager

12 +

13 +

14 +<img src="https://netdata.cloud/img/aws.svg" width="150"/>

15 +

16 +

17 +Kind: aws-sm

18 +

19 +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />

20 +

21 +## Overview

22 +

23 +Use AWS Secrets Manager as a secretstore backend when you want Netdata collectors to read secrets from AWS at runtime instead of storing them in plain text in collector configuration files.

24 +

25 +This page covers AWS Secrets Manager specific setup. For the shared resolver workflow and syntax, see [Secrets Management](https://github.com/netdata/netdata/blob/master/src/collectors/SECRETS.md).

26 +

27 +

28 +### Limitations

29 +

30 +Netdata reads existing secrets from AWS Secrets Manager. It does not create, rotate, or manage those secrets. If you use `secret-name#key`, the secret value must be stored as a JSON `SecretString`.

31 +

32 +

33 +## Setup

34 +

35 +You can configure the `aws-sm` secretstore in two ways:

36 +

37 +| Method | Best for | How to |

38 +|:--|:--|:--|

39 +| [**UI**](#via-ui) | Fast setup without editing files | Go to `Collectors -> go.d -> SecretStores -> aws-sm`, then add a secretstore. |

40 +| [**File**](#via-file) | File-based configuration or automation | Edit `/etc/netdata/go.d/ss/aws-sm.conf` and add a `jobs` entry. |

41 +

42 +### Prerequisites

43 +

44 +#### Provide AWS credentials

45 +

46 +Choose one supported authentication mode and make sure the Netdata Agent can obtain credentials for it:

47 +

48 +- `env`: set `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for the Netdata service. Set `AWS_SESSION_TOKEN` too if you use temporary credentials.

49 +- `ecs`: run Netdata in ECS with a task role so `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is available.

50 +- `imds`: run Netdata on EC2 with an instance profile and access to IMDSv2.

51 +

52 +For production on AWS, prefer `ecs` or `imds` over `env` so credentials are supplied by the platform instead of being stored in the Netdata service environment.

53 +

54 +

55 +#### Allow access to Secrets Manager

56 +

57 +The AWS identity used by this secretstore must be allowed to read the secrets you reference in collector configs in the configured `region`.

58 +

59 +

60 +#### Plan for file-based changes

61 +

62 +If you edit `/etc/netdata/go.d/ss/aws-sm.conf`, restart the Netdata Agent to load the updated secretstore definition.

63 +

64 +

65 +### Configuration

66 +

67 +#### Options

68 +

69 +The following options can be defined for this secretstore backend.

70 +

71 +<details open><summary>Config options</summary>

72 +

73 +

74 +

76 +|:-----|:------------|:--------|:---------:|

77 +| [auth_mode](#option-auth-mode) | How Netdata obtains AWS credentials. | env | yes |

78 +| region | AWS region used for Secrets Manager requests. | | yes |

79 +

80 +<a id="option-auth-mode"></a>

81 +##### auth_mode

82 +

83 +Supported values:

84 +

85 +- `env`: read credentials from the Netdata process environment.

86 +- `ecs`: read credentials from the ECS task credentials endpoint.

87 +- `imds`: read credentials from the EC2 Instance Metadata Service.

88 +

89 +For production on AWS, prefer `ecs` or `imds` when Netdata runs on ECS or EC2. Use `env` when you intentionally manage credentials in the Netdata service environment.

90 +

91 +

92 +

93 +</details>

94 +

95 +

96 +#### via UI

97 +

98 +1. Open the Netdata Dynamic Configuration UI.

99 +2. Go to `Collectors -> go.d -> SecretStores -> aws-sm`.

100 +3. Add a new secretstore and give it a store name.

101 +4. Fill in the backend-specific settings.

102 +5. Save the secretstore.

103 +

104 +#### via File

105 +

106 +Define the secretstore in `/etc/netdata/go.d/ss/aws-sm.conf`.

107 +

108 +Each file contains a `jobs` array, and the secretstore kind is determined by the filename.

109 +

110 +After editing the file, restart the Netdata Agent to load the updated secretstore definition.

111 +

112 +##### Examples

113 +

114 +###### Environment credentials

115 +

116 +Use environment-provided AWS credentials for the Netdata service.

117 +

118 +```yaml

119 +jobs:

120 + - name: aws_prod

121 + auth_mode: env

122 + region: us-east-1

123 +

124 +```

125 +###### ECS task role

126 +

127 +Use credentials provided to a Netdata task running in ECS.

128 +

129 +```yaml

130 +jobs:

131 + - name: aws_ecs

132 + auth_mode: ecs

133 + region: us-east-1

134 +

135 +```

136 +###### EC2 instance profile

137 +

138 +Use the instance profile attached to the EC2 instance running Netdata.

139 +

140 +```yaml

141 +jobs:

142 + - name: aws_imds

143 + auth_mode: imds

144 + region: us-east-1

145 +

146 +```

147 +

148 +

149 +## Use in collector configs

150 +

151 +Reference AWS Secrets Manager secrets from collector configs with the `aws-sm` secretstore kind.

152 +

153 +

154 +The operand is `secret-name` or `secret-name#key`.

155 +

156 +- Use `secret-name` to return the whole `SecretString`.

157 +- Use `secret-name#key` to read one top-level field from a JSON `SecretString`.

158 +- If you use `#key`, Netdata parses the secret value as JSON. Secret resolution fails if the value is not valid JSON or if the key does not exist.

159 +- Nested paths such as `parent.child` are not interpreted as nested JSON lookups.

160 +

161 +

162 +```text

163 +${store:aws-sm:<store-name>:<secret-name[#key]>}

164 +```

165 +

166 +- `aws-sm`: The secretstore backend kind.

167 +- `<store-name>`: The name of the configured secretstore, for example `aws_prod`.

168 +- `<secret-name[#key]>`: The AWS Secrets Manager secret name, optionally followed by `#key` to read one field from a JSON `SecretString`.

169 +

170 +### Examples

171 +#### Whole secret value

172 +

173 +Return the full `SecretString` stored under the `netdata/mysql/password` secret.

174 +

175 +```text

176 +${store:aws-sm:aws_prod:netdata/mysql/password}

177 +```

178 +#### JSON field from SecretString

179 +

180 +Read the `password` field from a JSON `SecretString`.

181 +

182 +```text

183 +${store:aws-sm:aws_prod:netdata/mysql#password}

184 +```

185 +#### Collector config example

186 +

187 +Use an AWS-stored password in a collector DSN.

188 +

189 +```yaml

190 +jobs:

191 + - name: mysql_prod

192 + dsn: "netdata:${store:aws-sm:aws_prod:netdata/mysql#password}@tcp(127.0.0.1:3306)/"

193 +

194 +```

195 +

196 +

197 +## Troubleshooting

198 +

199 +### Find the exact error

200 +

201 +Check the Netdata Agent logs when the collector starts or restarts. AWS resolver errors include messages such as `AWS_ACCESS_KEY_ID is not set`, `parsing SecretString as JSON`, or `key 'password' not found in SecretString JSON`.

202 +

203 +

204 +### AWS credentials are not found

205 +

206 +Check the selected `auth_mode`.

207 +

208 +- For `env`, make sure the Netdata service has `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.

209 +- For `ecs`, make sure Netdata runs in ECS and `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is available.

210 +- For `imds`, make sure the EC2 instance profile is attached and IMDSv2 is reachable.

211 +

212 +

213 +### Access denied or wrong region

214 +

215 +Confirm the configured `region` and make sure the AWS identity used by Netdata can read the referenced secret in that region.

216 +

217 +

218 +### JSON key lookup fails

219 +

220 +If you use `secret-name#key`, the secret must be stored as a JSON `SecretString`, and the requested key must exist as a top-level field in that JSON object.

221 +

222 +

223 +

src/go/plugin/agent/secrets/secretstore/backends/aws/metadata.yaml new

+149

@@ -0,0 +1,149 @@

1 +# yamllint disable rule:line-length

2 +---

3 +id: 'secretstore-aws-sm'

4 +meta:

5 + kind: 'aws-sm'

6 + name: 'AWS Secrets Manager'

7 + link: 'https://aws.amazon.com/secrets-manager/'

8 + icon_filename: 'aws.svg'

9 +keywords:

10 + - 'secretstore'

11 + - 'secrets'

12 + - 'aws'

13 + - 'aws-sm'

14 + - 'aws secrets manager'

15 +overview:

16 + description: |

17 + Use AWS Secrets Manager as a secretstore backend when you want Netdata collectors to read secrets from AWS at runtime instead of storing them in plain text in collector configuration files.

18 +

19 + This page covers AWS Secrets Manager specific setup. For the shared resolver workflow and syntax, see [Secrets Management](/src/collectors/SECRETS.md).

20 + limitations: |

21 + Netdata reads existing secrets from AWS Secrets Manager. It does not create, rotate, or manage those secrets. If you use `secret-name#key`, the secret value must be stored as a JSON `SecretString`.

22 +setup:

23 + prerequisites:

24 + list:

25 + - title: 'Provide AWS credentials'

26 + description: |

27 + Choose one supported authentication mode and make sure the Netdata Agent can obtain credentials for it:

28 +

29 + - `env`: set `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for the Netdata service. Set `AWS_SESSION_TOKEN` too if you use temporary credentials.

30 + - `ecs`: run Netdata in ECS with a task role so `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is available.

31 + - `imds`: run Netdata on EC2 with an instance profile and access to IMDSv2.

32 +

33 + For production on AWS, prefer `ecs` or `imds` over `env` so credentials are supplied by the platform instead of being stored in the Netdata service environment.

34 + - title: 'Allow access to Secrets Manager'

35 + description: |

36 + The AWS identity used by this secretstore must be allowed to read the secrets you reference in collector configs in the configured `region`.

37 + - title: 'Plan for file-based changes'

38 + description: |

39 + If you edit `/etc/netdata/go.d/ss/aws-sm.conf`, restart the Netdata Agent to load the updated secretstore definition.

40 + configuration:

41 + file:

42 + name: 'go.d/ss/aws-sm.conf'

43 + options:

44 + description: 'The following options can be defined for this secretstore backend.'

45 + folding:

46 + title: 'Config options'

47 + enabled: true

48 + list:

49 + - name: 'auth_mode'

50 + description: 'How Netdata obtains AWS credentials.'

51 + default_value: 'env'

52 + required: true

53 + detailed_description: |

54 + Supported values:

55 +

56 + - `env`: read credentials from the Netdata process environment.

57 + - `ecs`: read credentials from the ECS task credentials endpoint.

58 + - `imds`: read credentials from the EC2 Instance Metadata Service.

59 +

60 + For production on AWS, prefer `ecs` or `imds` when Netdata runs on ECS or EC2. Use `env` when you intentionally manage credentials in the Netdata service environment.

61 + - name: 'region'

62 + description: 'AWS region used for Secrets Manager requests.'

63 + default_value: ''

64 + required: true

65 + examples:

66 + folding:

67 + title: 'Example configuration'

68 + enabled: true

69 + list:

70 + - name: 'Environment credentials'

71 + description: 'Use environment-provided AWS credentials for the Netdata service.'

72 + config: |

73 + jobs:

74 + - name: aws_prod

75 + auth_mode: env

76 + region: us-east-1

77 + - name: 'ECS task role'

78 + description: 'Use credentials provided to a Netdata task running in ECS.'

79 + config: |

80 + jobs:

81 + - name: aws_ecs

82 + auth_mode: ecs

83 + region: us-east-1

84 + - name: 'EC2 instance profile'

85 + description: 'Use the instance profile attached to the EC2 instance running Netdata.'

86 + config: |

87 + jobs:

88 + - name: aws_imds

89 + auth_mode: imds

90 + region: us-east-1

91 +collector_configs:

92 + description: |

93 + Reference AWS Secrets Manager secrets from collector configs with the `aws-sm` secretstore kind.

94 + summary:

95 + operand_format: 'secret-name[#key]'

96 + example_operand: 'netdata/mysql#password'

97 + format:

98 + description: |

99 + The operand is `secret-name` or `secret-name#key`.

100 +

101 + - Use `secret-name` to return the whole `SecretString`.

102 + - Use `secret-name#key` to read one top-level field from a JSON `SecretString`.

103 + - If you use `#key`, Netdata parses the secret value as JSON. Secret resolution fails if the value is not valid JSON or if the key does not exist.

104 + - Nested paths such as `parent.child` are not interpreted as nested JSON lookups.

105 + syntax: '${store:aws-sm:<store-name>:<secret-name[#key]>}'

106 + parts:

107 + list:

108 + - name: 'aws-sm'

109 + description: 'The secretstore backend kind.'

110 + - name: '<store-name>'

111 + description: 'The name of the configured secretstore, for example `aws_prod`.'

112 + - name: '<secret-name[#key]>'

113 + description: 'The AWS Secrets Manager secret name, optionally followed by `#key` to read one field from a JSON `SecretString`.'

114 + examples:

115 + list:

116 + - name: 'Whole secret value'

117 + description: 'Return the full `SecretString` stored under the `netdata/mysql/password` secret.'

118 + language: 'text'

119 + content: '${store:aws-sm:aws_prod:netdata/mysql/password}'

120 + - name: 'JSON field from SecretString'

121 + description: 'Read the `password` field from a JSON `SecretString`.'

122 + language: 'text'

123 + content: '${store:aws-sm:aws_prod:netdata/mysql#password}'

124 + - name: 'Collector config example'

125 + description: 'Use an AWS-stored password in a collector DSN.'

126 + language: 'yaml'

127 + content: |

128 + jobs:

129 + - name: mysql_prod

130 + dsn: "netdata:${store:aws-sm:aws_prod:netdata/mysql#password}@tcp(127.0.0.1:3306)/"

131 +troubleshooting:

132 + problems:

133 + list:

134 + - name: 'Find the exact error'

135 + description: |

136 + Check the Netdata Agent logs when the collector starts or restarts. AWS resolver errors include messages such as `AWS_ACCESS_KEY_ID is not set`, `parsing SecretString as JSON`, or `key 'password' not found in SecretString JSON`.

137 + - name: 'AWS credentials are not found'

138 + description: |

139 + Check the selected `auth_mode`.

140 +

141 + - For `env`, make sure the Netdata service has `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.

142 + - For `ecs`, make sure Netdata runs in ECS and `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is available.

143 + - For `imds`, make sure the EC2 instance profile is attached and IMDSv2 is reachable.

144 + - name: 'Access denied or wrong region'

145 + description: |

146 + Confirm the configured `region` and make sure the AWS identity used by Netdata can read the referenced secret in that region.

147 + - name: 'JSON key lookup fails'

148 + description: |

149 + If you use `secret-name#key`, the secret must be stored as a JSON `SecretString`, and the requested key must exist as a top-level field in that JSON object.

src/go/plugin/agent/secrets/secretstore/backends/azure/README.md new

@@ -0,0 +1 @@

1 +integrations/azure-kv.md

\ No newline at end of file

src/go/plugin/agent/secrets/secretstore/backends/azure/integrations/azure-kv.md new

+220

@@ -0,0 +1,220 @@

1 +<!--startmeta

2 +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/azure/README.md"

3 +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/azure/metadata.yaml"

4 +sidebar_label: "Azure Key Vault"

5 +learn_status: "Published"

6 +learn_rel_path: "Collecting Metrics/Secret Stores"

7 +keywords: ['secretstore', 'secrets', 'azure', 'azure-kv', 'azure key vault']

8 +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE SECRETSTORE'S metadata.yaml FILE"

9 +endmeta-->

10 +

11 +# Azure Key Vault

12 +

13 +

14 +<img src="https://netdata.cloud/img/azure.svg" width="150"/>

15 +

16 +

17 +Kind: azure-kv

18 +

19 +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />

20 +

21 +## Overview

22 +

23 +Use Azure Key Vault as a secretstore backend when you want Netdata collectors to read secrets from Azure at runtime instead of storing them in plain text in collector configuration files.

24 +

25 +This page covers Azure Key Vault specific setup. For the shared resolver workflow and syntax, see [Secrets Management](https://github.com/netdata/netdata/blob/master/src/collectors/SECRETS.md).

26 +

27 +

28 +### Limitations

29 +

30 +Netdata reads the latest version of a secret value from Azure Key Vault. The operand format does not select a specific secret version.

31 +

32 +

33 +## Setup

34 +

35 +You can configure the `azure-kv` secretstore in two ways:

36 +

37 +| Method | Best for | How to |

38 +|:--|:--|:--|

39 +| [**UI**](#via-ui) | Fast setup without editing files | Go to `Collectors -> go.d -> SecretStores -> azure-kv`, then add a secretstore. |

40 +| [**File**](#via-file) | File-based configuration or automation | Edit `/etc/netdata/go.d/ss/azure-kv.conf` and add a `jobs` entry. |

41 +

42 +### Prerequisites

43 +

44 +#### Choose an Azure authentication mode

45 +

46 +Choose one supported authentication mode and make sure the Netdata Agent can use it:

47 +

48 +- `service_principal`: provide `tenant_id`, `client_id`, and `client_secret`.

49 +- `managed_identity`: run Netdata on an Azure resource with a managed identity.

50 +- `default`: use the Azure SDK `DefaultAzureCredential` chain, which automatically tries available Azure credential sources such as environment-based credentials, managed identity, and local developer credentials.

51 +

52 +Prefer `managed_identity` for production on Azure when Netdata runs on an Azure resource with an attached identity. Use `service_principal` for explicit application credentials. Use `default` for Azure SDK auto-discovery or local development convenience.

53 +

54 +

55 +#### Allow secret read access

56 +

57 +The Azure identity used by this secretstore must be allowed to read secret values from the target vaults.

58 +

59 +

60 +#### Plan for file-based changes

61 +

62 +If you edit `/etc/netdata/go.d/ss/azure-kv.conf`, restart the Netdata Agent to load the updated secretstore definition.

63 +

64 +

65 +### Configuration

66 +

67 +#### Options

68 +

69 +The following options can be defined for this secretstore backend.

70 +

71 +<details open><summary>Config options</summary>

72 +

73 +

74 +

76 +|:------|:-----|:------------|:--------|:---------:|

79 +| | mode_service_principal.client_id | Azure application / service principal client ID. Required when `mode` is `service_principal`. | | yes |

80 +| | mode_service_principal.client_secret | Azure application / service principal client secret. Required when `mode` is `service_principal`. | | yes |

82 +

83 +<a id="option-mode"></a>

84 +##### mode

85 +

86 +Supported values:

87 +

88 +- `service_principal`: use an Azure app / service principal.

89 +- `managed_identity`: use the managed identity attached to the Azure resource running Netdata.

90 +- `default`: use the Azure SDK `DefaultAzureCredential` chain. It automatically tries available Azure credential sources such as environment-based credentials, managed identity, and local developer credentials.

91 +

92 +Prefer `managed_identity` for production on Azure. Use `service_principal` for explicit app credentials. Use `default` when you want Azure SDK auto-discovery or local development convenience.

93 +

94 +

95 +

96 +</details>

97 +

98 +

99 +#### via UI

100 +

101 +1. Open the Netdata Dynamic Configuration UI.

102 +2. Go to `Collectors -> go.d -> SecretStores -> azure-kv`.

103 +3. Add a new secretstore and give it a store name.

104 +4. Fill in the backend-specific settings.

105 +5. Save the secretstore.

106 +

107 +#### via File

108 +

109 +Define the secretstore in `/etc/netdata/go.d/ss/azure-kv.conf`.

110 +

111 +Each file contains a `jobs` array, and the secretstore kind is determined by the filename.

112 +

113 +After editing the file, restart the Netdata Agent to load the updated secretstore definition.

114 +

115 +##### Examples

116 +

117 +###### Service principal

118 +

119 +Use explicit Azure app credentials.

120 +

121 +```yaml

122 +jobs:

123 + - name: azure_prod

124 + mode: service_principal

125 + mode_service_principal:

126 + tenant_id: 00000000-0000-0000-0000-000000000000

127 + client_id: 00000000-0000-0000-0000-000000000000

128 + client_secret: your-client-secret

129 +

130 +```

131 +###### Managed identity

132 +

133 +Use the managed identity attached to the Azure resource running Netdata.

134 +

135 +```yaml

136 +jobs:

137 + - name: azure_vm

138 + mode: managed_identity

139 + mode_managed_identity:

140 + client_id: 00000000-0000-0000-0000-000000000000

141 +

142 +```

143 +###### Default credential chain

144 +

145 +Use the Azure SDK default credential chain.

146 +

147 +```yaml

148 +jobs:

149 + - name: azure_default

150 + mode: default

151 +

152 +```

153 +

154 +

155 +## Use in collector configs

156 +

157 +Reference Azure Key Vault secrets from collector configs with the `azure-kv` secretstore kind.

158 +

159 +

160 +The operand is `vault-name/secret-name`.

161 +

162 +Netdata requests the latest secret value from `https://<vault-name>.vault.azure.net/secrets/<secret-name>?api-version=7.4`.

163 +Both `vault-name` and `secret-name` must use only letters, numbers, and hyphens.

164 +

165 +

166 +```text

167 +${store:azure-kv:<store-name>:<vault-name/secret-name>}

168 +```

169 +

170 +- `azure-kv`: The secretstore backend kind.

171 +- `<store-name>`: The name of the configured secretstore, for example `azure_prod`.

172 +- `<vault-name/secret-name>`: The Azure Key Vault name and the secret name, separated by `/`.

173 +

174 +### Examples

175 +#### Secret reference

176 +

177 +Read the latest value of the `mysql-password` secret from the `my-keyvault` vault.

178 +

179 +```text

180 +${store:azure-kv:azure_prod:my-keyvault/mysql-password}

181 +```

182 +#### Collector config example

183 +

184 +Use an Azure Key Vault secret in a collector DSN.

185 +

186 +```yaml

187 +jobs:

188 + - name: mysql_prod

189 + dsn: "netdata:${store:azure-kv:azure_prod:my-keyvault/mysql-password}@tcp(127.0.0.1:3306)/"

190 +

191 +```

192 +

193 +

194 +## Troubleshooting

195 +

196 +### Find the exact error

197 +

198 +Check the Netdata Agent logs when the collector starts or restarts. Azure resolver errors include messages such as `invalid vault name`, `invalid secret name`, or `Azure Key Vault returned HTTP 403`.

199 +

200 +

201 +### Azure authentication fails

202 +

203 +Check the selected `mode` and the credentials it requires.

204 +

205 +- For `service_principal`, verify `tenant_id`, `client_id`, and `client_secret`.

206 +- For `managed_identity`, make sure Netdata runs on an Azure resource with an attached identity.

207 +- For `default`, confirm that one of the Azure SDK credential sources is available to the Netdata process.

208 +

209 +

210 +### Secret lookup fails

211 +

212 +Check the operand format. It must be `vault-name/secret-name`, and both names must use only letters, numbers, and hyphens.

213 +

214 +

215 +### Access denied

216 +

217 +Make sure the Azure identity used by Netdata can read secret values from the target vault.

218 +

219 +

220 +

src/go/plugin/agent/secrets/secretstore/backends/azure/metadata.yaml new

+162

@@ -0,0 +1,162 @@

1 +# yamllint disable rule:line-length

2 +---

3 +id: 'secretstore-azure-kv'

4 +meta:

5 + kind: 'azure-kv'

6 + name: 'Azure Key Vault'

7 + link: 'https://azure.microsoft.com/en-us/products/key-vault'

8 + icon_filename: 'azure.svg'

9 +keywords:

10 + - 'secretstore'

11 + - 'secrets'

12 + - 'azure'

13 + - 'azure-kv'

14 + - 'azure key vault'

15 +overview:

16 + description: |

17 + Use Azure Key Vault as a secretstore backend when you want Netdata collectors to read secrets from Azure at runtime instead of storing them in plain text in collector configuration files.

18 +

19 + This page covers Azure Key Vault specific setup. For the shared resolver workflow and syntax, see [Secrets Management](/src/collectors/SECRETS.md).

20 + limitations: |

21 + Netdata reads the latest version of a secret value from Azure Key Vault. The operand format does not select a specific secret version.

22 +setup:

23 + prerequisites:

24 + list:

25 + - title: 'Choose an Azure authentication mode'

26 + description: |

27 + Choose one supported authentication mode and make sure the Netdata Agent can use it:

28 +

29 + - `service_principal`: provide `tenant_id`, `client_id`, and `client_secret`.

30 + - `managed_identity`: run Netdata on an Azure resource with a managed identity.

31 + - `default`: use the Azure SDK `DefaultAzureCredential` chain, which automatically tries available Azure credential sources such as environment-based credentials, managed identity, and local developer credentials.

32 +

33 + Prefer `managed_identity` for production on Azure when Netdata runs on an Azure resource with an attached identity. Use `service_principal` for explicit application credentials. Use `default` for Azure SDK auto-discovery or local development convenience.

34 + - title: 'Allow secret read access'

35 + description: |

36 + The Azure identity used by this secretstore must be allowed to read secret values from the target vaults.

37 + - title: 'Plan for file-based changes'

38 + description: |

39 + If you edit `/etc/netdata/go.d/ss/azure-kv.conf`, restart the Netdata Agent to load the updated secretstore definition.

40 + configuration:

41 + file:

42 + name: 'go.d/ss/azure-kv.conf'

43 + options:

44 + description: 'The following options can be defined for this secretstore backend.'

45 + folding:

46 + title: 'Config options'

47 + enabled: true

48 + list:

49 + - name: 'mode'

50 + description: 'Azure authentication mode.'

51 + default_value: 'default'

52 + required: true

53 + detailed_description: |

54 + Supported values:

55 +

56 + - `service_principal`: use an Azure app / service principal.

57 + - `managed_identity`: use the managed identity attached to the Azure resource running Netdata.

58 + - `default`: use the Azure SDK `DefaultAzureCredential` chain. It automatically tries available Azure credential sources such as environment-based credentials, managed identity, and local developer credentials.

59 +

60 + Prefer `managed_identity` for production on Azure. Use `service_principal` for explicit app credentials. Use `default` when you want Azure SDK auto-discovery or local development convenience.

61 + - name: 'mode_service_principal.tenant_id'

62 + group: 'Service Principal'

63 + description: 'Azure tenant ID. Required when `mode` is `service_principal`.'

64 + default_value: ''

65 + required: true

66 + - name: 'mode_service_principal.client_id'

67 + group: 'Service Principal'

68 + description: 'Azure application / service principal client ID. Required when `mode` is `service_principal`.'

69 + default_value: ''

70 + required: true

71 + - name: 'mode_service_principal.client_secret'

72 + group: 'Service Principal'

73 + description: 'Azure application / service principal client secret. Required when `mode` is `service_principal`.'

74 + default_value: ''

75 + required: true

76 + - name: 'mode_managed_identity.client_id'

77 + group: 'Managed Identity'

78 + description: 'Optional client ID of a user-assigned managed identity when `mode` is `managed_identity`. Leave it empty for the system-assigned identity.'

79 + default_value: ''

80 + required: false

81 + examples:

82 + folding:

83 + title: 'Example configuration'

84 + enabled: true

85 + list:

86 + - name: 'Service principal'

87 + description: 'Use explicit Azure app credentials.'

88 + config: |

89 + jobs:

90 + - name: azure_prod

91 + mode: service_principal

92 + mode_service_principal:

93 + tenant_id: 00000000-0000-0000-0000-000000000000

94 + client_id: 00000000-0000-0000-0000-000000000000

95 + client_secret: your-client-secret

96 + - name: 'Managed identity'

97 + description: 'Use the managed identity attached to the Azure resource running Netdata.'

98 + config: |

99 + jobs:

100 + - name: azure_vm

101 + mode: managed_identity

102 + mode_managed_identity:

103 + client_id: 00000000-0000-0000-0000-000000000000

104 + - name: 'Default credential chain'

105 + description: 'Use the Azure SDK default credential chain.'

106 + config: |

107 + jobs:

108 + - name: azure_default

109 + mode: default

110 +collector_configs:

111 + description: |

112 + Reference Azure Key Vault secrets from collector configs with the `azure-kv` secretstore kind.

113 + summary:

114 + operand_format: 'vault-name/secret-name'

115 + example_operand: 'my-keyvault/mysql-password'

116 + format:

117 + description: |

118 + The operand is `vault-name/secret-name`.

119 +

120 + Netdata requests the latest secret value from `https://<vault-name>.vault.azure.net/secrets/<secret-name>?api-version=7.4`.

121 + Both `vault-name` and `secret-name` must use only letters, numbers, and hyphens.

122 + syntax: '${store:azure-kv:<store-name>:<vault-name/secret-name>}'

123 + parts:

124 + list:

125 + - name: 'azure-kv'

126 + description: 'The secretstore backend kind.'

127 + - name: '<store-name>'

128 + description: 'The name of the configured secretstore, for example `azure_prod`.'

129 + - name: '<vault-name/secret-name>'

130 + description: 'The Azure Key Vault name and the secret name, separated by `/`.'

131 + examples:

132 + list:

133 + - name: 'Secret reference'

134 + description: 'Read the latest value of the `mysql-password` secret from the `my-keyvault` vault.'

135 + language: 'text'

136 + content: '${store:azure-kv:azure_prod:my-keyvault/mysql-password}'

137 + - name: 'Collector config example'

138 + description: 'Use an Azure Key Vault secret in a collector DSN.'

139 + language: 'yaml'

140 + content: |

141 + jobs:

142 + - name: mysql_prod

143 + dsn: "netdata:${store:azure-kv:azure_prod:my-keyvault/mysql-password}@tcp(127.0.0.1:3306)/"

144 +troubleshooting:

145 + problems:

146 + list:

147 + - name: 'Find the exact error'

148 + description: |

149 + Check the Netdata Agent logs when the collector starts or restarts. Azure resolver errors include messages such as `invalid vault name`, `invalid secret name`, or `Azure Key Vault returned HTTP 403`.

150 + - name: 'Azure authentication fails'

151 + description: |

152 + Check the selected `mode` and the credentials it requires.

153 +

154 + - For `service_principal`, verify `tenant_id`, `client_id`, and `client_secret`.

155 + - For `managed_identity`, make sure Netdata runs on an Azure resource with an attached identity.

156 + - For `default`, confirm that one of the Azure SDK credential sources is available to the Netdata process.

157 + - name: 'Secret lookup fails'

158 + description: |

159 + Check the operand format. It must be `vault-name/secret-name`, and both names must use only letters, numbers, and hyphens.

160 + - name: 'Access denied'

161 + description: |

162 + Make sure the Azure identity used by Netdata can read secret values from the target vault.

src/go/plugin/agent/secrets/secretstore/backends/gcp/README.md new

@@ -0,0 +1 @@

1 +integrations/gcp-sm.md

\ No newline at end of file

src/go/plugin/agent/secrets/secretstore/backends/gcp/integrations/gcp-sm.md new

+210

@@ -0,0 +1,210 @@

1 +<!--startmeta

2 +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/gcp/README.md"

3 +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/gcp/metadata.yaml"

4 +sidebar_label: "Google Secret Manager"

5 +learn_status: "Published"

6 +learn_rel_path: "Collecting Metrics/Secret Stores"

7 +keywords: ['secretstore', 'secrets', 'gcp', 'gcp-sm', 'google secret manager']

8 +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE SECRETSTORE'S metadata.yaml FILE"

9 +endmeta-->

10 +

11 +# Google Secret Manager

12 +

13 +

14 +<img src="https://netdata.cloud/img/google.svg" width="150"/>

15 +

16 +

17 +Kind: gcp-sm

18 +

19 +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />

20 +

21 +## Overview

22 +

23 +Use Google Secret Manager as a secretstore backend when you want Netdata collectors to read secrets from GCP at runtime instead of storing them in plain text in collector configuration files.

24 +

25 +This page covers Google Secret Manager specific setup. For the shared resolver workflow and syntax, see [Secrets Management](https://github.com/netdata/netdata/blob/master/src/collectors/SECRETS.md).

26 +

27 +

28 +### Limitations

29 +

30 +If you omit the version in the operand, Netdata reads the `latest` secret version automatically.

31 +

32 +

33 +## Setup

34 +

35 +You can configure the `gcp-sm` secretstore in two ways:

36 +

37 +| Method | Best for | How to |

38 +|:--|:--|:--|

39 +| [**UI**](#via-ui) | Fast setup without editing files | Go to `Collectors -> go.d -> SecretStores -> gcp-sm`, then add a secretstore. |

40 +| [**File**](#via-file) | File-based configuration or automation | Edit `/etc/netdata/go.d/ss/gcp-sm.conf` and add a `jobs` entry. |

41 +

42 +### Prerequisites

43 +

44 +#### Choose a GCP authentication mode

45 +

46 +Choose one supported authentication mode and make sure the Netdata Agent can use it:

47 +

48 +- `metadata`: run Netdata in a Google Cloud environment where the metadata server is reachable.

49 +- `service_account_file`: provide a service account JSON file on the Netdata host.

50 +

51 +Prefer `metadata` for production when Netdata runs in a supported Google Cloud environment. Use `service_account_file` when Netdata runs outside Google Cloud or when you need explicit credentials.

52 +

53 +

54 +#### Protect the service account file

55 +

56 +If you use `service_account_file`, the JSON file contains a private key. Keep it on the Netdata host, make it readable by the `netdata` user, and restrict access as tightly as possible. A common setup is `chmod 0600` with ownership that allows the `netdata` user to read the file.

57 +

58 +

59 +#### Allow Secret Manager access

60 +

61 +The Google identity used by this secretstore must be allowed to access the referenced secrets in Google Secret Manager.

62 +

63 +

64 +#### Plan for file-based changes

65 +

66 +If you edit `/etc/netdata/go.d/ss/gcp-sm.conf`, restart the Netdata Agent to load the updated secretstore definition.

67 +

68 +

69 +### Configuration

70 +

71 +#### Options

72 +

73 +The following options can be defined for this secretstore backend.

74 +

75 +<details open><summary>Config options</summary>

76 +

77 +

78 +

80 +|:------|:-----|:------------|:--------|:---------:|

82 +| **Service Account File** | mode_service_account_file.path | Path to a service account JSON file. Required when `mode` is `service_account_file`. The file contains a private key and should be readable only by the `netdata` user or another tightly scoped owner. | | yes |

83 +

84 +<a id="option-mode"></a>

85 +##### mode

86 +

87 +Supported values:

88 +

89 +- `metadata`: get an access token from the Google metadata server.

90 +- `service_account_file`: use a local service account JSON file.

91 +

92 +Prefer `metadata` for production when Netdata runs in a supported Google Cloud environment. Use `service_account_file` when you need explicit credentials or when the metadata server is not available.

93 +

94 +

95 +

96 +</details>

97 +

98 +

99 +#### via UI

100 +

101 +1. Open the Netdata Dynamic Configuration UI.

102 +2. Go to `Collectors -> go.d -> SecretStores -> gcp-sm`.

103 +3. Add a new secretstore and give it a store name.

104 +4. Fill in the backend-specific settings.

105 +5. Save the secretstore.

106 +

107 +#### via File

108 +

109 +Define the secretstore in `/etc/netdata/go.d/ss/gcp-sm.conf`.

110 +

111 +Each file contains a `jobs` array, and the secretstore kind is determined by the filename.

112 +

113 +After editing the file, restart the Netdata Agent to load the updated secretstore definition.

114 +

115 +##### Examples

116 +

117 +###### Metadata server

118 +

119 +Use credentials from the Google metadata server.

120 +

121 +```yaml

122 +jobs:

123 + - name: gcp_metadata

124 + mode: metadata

125 +

126 +```

127 +###### Service account file

128 +

129 +Use a service account JSON file stored on the Netdata host.

130 +

131 +```yaml

132 +jobs:

133 + - name: gcp_service_account

134 + mode: service_account_file

135 + mode_service_account_file:

136 + path: /etc/netdata/gcp-service-account.json

137 +

138 +```

139 +

140 +

141 +## Use in collector configs

142 +

143 +Reference Google Secret Manager secrets from collector configs with the `gcp-sm` secretstore kind.

144 +

145 +

146 +The operand is `project/secret` or `project/secret/version`.

147 +

148 +If you omit the version, Netdata uses `latest`.

149 +Project IDs may use letters, numbers, `.`, `_`, `:`, or `-`. Secret names and versions may use letters, numbers, `_`, or `-`.

150 +When you specify a version, use the version name accepted by Secret Manager, such as `3`.

151 +

152 +

153 +```text

154 +${store:gcp-sm:<store-name>:<project/secret[/version]>}

155 +```

156 +

157 +- `gcp-sm`: The secretstore backend kind.

158 +- `<store-name>`: The name of the configured secretstore, for example `gcp_prod`.

159 +- `<project/secret[/version]>`: The Google Cloud project ID, secret name, and optional version.

160 +

161 +### Examples

162 +#### Latest version

163 +

164 +Read the latest version of the `mysql-password` secret from the `my-project` project.

165 +

166 +```text

167 +${store:gcp-sm:gcp_prod:my-project/mysql-password}

168 +```

169 +#### Specific version

170 +

171 +Read version `3` of the `mysql-password` secret.

172 +

173 +```text

174 +${store:gcp-sm:gcp_prod:my-project/mysql-password/3}

175 +```

176 +#### Collector config example

177 +

178 +Use a Google Secret Manager secret in a collector DSN.

179 +

180 +```yaml

181 +jobs:

182 + - name: mysql_prod

183 + dsn: "netdata:${store:gcp-sm:gcp_prod:my-project/mysql-password}@tcp(127.0.0.1:3306)/"

184 +

185 +```

186 +

187 +

188 +## Troubleshooting

189 +

190 +### Find the exact error

191 +

192 +Check the Netdata Agent logs when the collector starts or restarts. GCP resolver errors include messages such as `metadata token request returned HTTP 404`, `invalid project ID`, `invalid version`, or `reading service account file`.

193 +

194 +

195 +### Metadata mode does not work

196 +

197 +`mode: metadata` requires the Google metadata server. If Netdata is not running in a supported Google Cloud environment, switch to `service_account_file`.

198 +

199 +

200 +### Service account file cannot be read

201 +

202 +Check the file path, the JSON contents, and that the `netdata` user can read the file. Because the file contains a private key, keep its permissions as tight as possible.

203 +

204 +

205 +### Permission denied or secret not found

206 +

207 +Make sure the Google identity used by Netdata can access the referenced secret, and confirm that the operand uses the correct `project/secret` or `project/secret/version` format.

208 +

209 +

210 +

src/go/plugin/agent/secrets/secretstore/backends/gcp/metadata.yaml new

+139

@@ -0,0 +1,139 @@

1 +# yamllint disable rule:line-length

2 +---

3 +id: 'secretstore-gcp-sm'

4 +meta:

5 + kind: 'gcp-sm'

6 + name: 'Google Secret Manager'

7 + link: 'https://cloud.google.com/secret-manager'

8 + icon_filename: 'google.svg'

9 +keywords:

10 + - 'secretstore'

11 + - 'secrets'

12 + - 'gcp'

13 + - 'gcp-sm'

14 + - 'google secret manager'

15 +overview:

16 + description: |

17 + Use Google Secret Manager as a secretstore backend when you want Netdata collectors to read secrets from GCP at runtime instead of storing them in plain text in collector configuration files.

18 +

19 + This page covers Google Secret Manager specific setup. For the shared resolver workflow and syntax, see [Secrets Management](/src/collectors/SECRETS.md).

20 + limitations: |

21 + If you omit the version in the operand, Netdata reads the `latest` secret version automatically.

22 +setup:

23 + prerequisites:

24 + list:

25 + - title: 'Choose a GCP authentication mode'

26 + description: |

27 + Choose one supported authentication mode and make sure the Netdata Agent can use it:

28 +

29 + - `metadata`: run Netdata in a Google Cloud environment where the metadata server is reachable.

30 + - `service_account_file`: provide a service account JSON file on the Netdata host.

31 +

32 + Prefer `metadata` for production when Netdata runs in a supported Google Cloud environment. Use `service_account_file` when Netdata runs outside Google Cloud or when you need explicit credentials.

33 + - title: 'Protect the service account file'

34 + description: |

35 + If you use `service_account_file`, the JSON file contains a private key. Keep it on the Netdata host, make it readable by the `netdata` user, and restrict access as tightly as possible. A common setup is `chmod 0600` with ownership that allows the `netdata` user to read the file.

36 + - title: 'Allow Secret Manager access'

37 + description: |

38 + The Google identity used by this secretstore must be allowed to access the referenced secrets in Google Secret Manager.

39 + - title: 'Plan for file-based changes'

40 + description: |

41 + If you edit `/etc/netdata/go.d/ss/gcp-sm.conf`, restart the Netdata Agent to load the updated secretstore definition.

42 + configuration:

43 + file:

44 + name: 'go.d/ss/gcp-sm.conf'

45 + options:

46 + description: 'The following options can be defined for this secretstore backend.'

47 + folding:

48 + title: 'Config options'

49 + enabled: true

50 + list:

51 + - name: 'mode'

52 + description: 'GCP authentication mode.'

53 + default_value: 'metadata'

54 + required: true

55 + detailed_description: |

56 + Supported values:

57 +

58 + - `metadata`: get an access token from the Google metadata server.

59 + - `service_account_file`: use a local service account JSON file.

60 +

61 + Prefer `metadata` for production when Netdata runs in a supported Google Cloud environment. Use `service_account_file` when you need explicit credentials or when the metadata server is not available.

62 + - name: 'mode_service_account_file.path'

63 + group: 'Service Account File'

64 + description: 'Path to a service account JSON file. Required when `mode` is `service_account_file`. The file contains a private key and should be readable only by the `netdata` user or another tightly scoped owner.'

65 + default_value: ''

66 + required: true

67 + examples:

68 + folding:

69 + title: 'Example configuration'

70 + enabled: true

71 + list:

72 + - name: 'Metadata server'

73 + description: 'Use credentials from the Google metadata server.'

74 + config: |

75 + jobs:

76 + - name: gcp_metadata

77 + mode: metadata

78 + - name: 'Service account file'

79 + description: 'Use a service account JSON file stored on the Netdata host.'

80 + config: |

81 + jobs:

82 + - name: gcp_service_account

83 + mode: service_account_file

84 + mode_service_account_file:

85 + path: /etc/netdata/gcp-service-account.json

86 +collector_configs:

87 + description: |

88 + Reference Google Secret Manager secrets from collector configs with the `gcp-sm` secretstore kind.

89 + summary:

90 + operand_format: 'project/secret[/version]'

91 + example_operand: 'my-project/mysql-password'

92 + format:

93 + description: |

94 + The operand is `project/secret` or `project/secret/version`.

95 +

96 + If you omit the version, Netdata uses `latest`.

97 + Project IDs may use letters, numbers, `.`, `_`, `:`, or `-`. Secret names and versions may use letters, numbers, `_`, or `-`.

98 + When you specify a version, use the version name accepted by Secret Manager, such as `3`.

99 + syntax: '${store:gcp-sm:<store-name>:<project/secret[/version]>}'

100 + parts:

101 + list:

102 + - name: 'gcp-sm'

103 + description: 'The secretstore backend kind.'

104 + - name: '<store-name>'

105 + description: 'The name of the configured secretstore, for example `gcp_prod`.'

106 + - name: '<project/secret[/version]>'

107 + description: 'The Google Cloud project ID, secret name, and optional version.'

108 + examples:

109 + list:

110 + - name: 'Latest version'

111 + description: 'Read the latest version of the `mysql-password` secret from the `my-project` project.'

112 + language: 'text'

113 + content: '${store:gcp-sm:gcp_prod:my-project/mysql-password}'

114 + - name: 'Specific version'

115 + description: 'Read version `3` of the `mysql-password` secret.'

116 + language: 'text'

117 + content: '${store:gcp-sm:gcp_prod:my-project/mysql-password/3}'

118 + - name: 'Collector config example'

119 + description: 'Use a Google Secret Manager secret in a collector DSN.'

120 + language: 'yaml'

121 + content: |

122 + jobs:

123 + - name: mysql_prod

124 + dsn: "netdata:${store:gcp-sm:gcp_prod:my-project/mysql-password}@tcp(127.0.0.1:3306)/"

125 +troubleshooting:

126 + problems:

127 + list:

128 + - name: 'Find the exact error'

129 + description: |

130 + Check the Netdata Agent logs when the collector starts or restarts. GCP resolver errors include messages such as `metadata token request returned HTTP 404`, `invalid project ID`, `invalid version`, or `reading service account file`.

131 + - name: 'Metadata mode does not work'

132 + description: |

133 + `mode: metadata` requires the Google metadata server. If Netdata is not running in a supported Google Cloud environment, switch to `service_account_file`.

134 + - name: 'Service account file cannot be read'

135 + description: |

136 + Check the file path, the JSON contents, and that the `netdata` user can read the file. Because the file contains a private key, keep its permissions as tight as possible.

137 + - name: 'Permission denied or secret not found'

138 + description: |

139 + Make sure the Google identity used by Netdata can access the referenced secret, and confirm that the operand uses the correct `project/secret` or `project/secret/version` format.

src/go/plugin/agent/secrets/secretstore/backends/vault/README.md new

@@ -0,0 +1 @@

1 +integrations/vault.md

\ No newline at end of file

src/go/plugin/agent/secrets/secretstore/backends/vault/integrations/vault.md new

+234

@@ -0,0 +1,234 @@

1 +<!--startmeta

2 +custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/vault/README.md"

3 +meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/agent/secrets/secretstore/backends/vault/metadata.yaml"

4 +sidebar_label: "Vault"

5 +learn_status: "Published"

6 +learn_rel_path: "Collecting Metrics/Secret Stores"

7 +keywords: ['secretstore', 'secrets', 'vault', 'hashicorp vault']

8 +message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE SECRETSTORE'S metadata.yaml FILE"

9 +endmeta-->

10 +

11 +# Vault

12 +

13 +

14 +<img src="https://netdata.cloud/img/vault.svg" width="150"/>

15 +

16 +

17 +Kind: vault

18 +

19 +<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />

20 +

21 +## Overview

22 +

23 +Use Vault as a secretstore backend when you want Netdata collectors to read secrets from HashiCorp Vault at runtime instead of storing them in plain text in collector configuration files.

24 +

25 +This page covers Vault specific setup. For the shared resolver workflow and syntax, see [Secrets Management](https://github.com/netdata/netdata/blob/master/src/collectors/SECRETS.md).

26 +

27 +

28 +### Limitations

29 +

30 +Netdata reads existing secrets from Vault. It does not create or renew Vault tokens. If the configured token expires or becomes invalid, secret resolution fails until Netdata can read a valid token again. For KV v2 secrets, Netdata does not add `/data/` to the path automatically.

31 +

32 +

33 +## Setup

34 +

35 +You can configure the `vault` secretstore in two ways:

36 +

37 +| Method | Best for | How to |

38 +|:--|:--|:--|

39 +| [**UI**](#via-ui) | Fast setup without editing files | Go to `Collectors -> go.d -> SecretStores -> vault`, then add a secretstore. |

40 +| [**File**](#via-file) | File-based configuration or automation | Edit `/etc/netdata/go.d/ss/vault.conf` and add a `jobs` entry. |

41 +

42 +### Prerequisites

43 +

44 +#### Make Vault reachable

45 +

46 +Netdata must be able to reach your Vault server at the address you configure in `addr`.

47 +

48 +

49 +#### Provide a Vault token

50 +

51 +Choose one supported authentication mode and make sure Netdata can use it:

52 +

53 +- `token`: store the Vault token directly in the secretstore configuration.

54 +- `token_file`: store the Vault token in a local file on the Netdata host that is readable by the `netdata` user.

55 +

56 +Prefer `token_file` for production so the Vault token is not embedded directly in the secretstore configuration.

57 +

58 +

59 +#### Allow access to the referenced secret paths

60 +

61 +The Vault token used by this secretstore must be allowed to read the paths you reference from collector configs.

62 +

63 +

64 +#### Plan for file-based changes

65 +

66 +If you edit `/etc/netdata/go.d/ss/vault.conf`, restart the Netdata Agent to load the updated secretstore definition.

67 +

68 +

69 +### Configuration

70 +

71 +#### Options

72 +

73 +The following options can be defined for this secretstore backend.

74 +

75 +<details open><summary>Config options</summary>

76 +

77 +

78 +

80 +|:------|:-----|:------------|:--------|:---------:|

82 +| | addr | Vault server address / base URL. | | yes |

83 +| | namespace | Optional Vault Enterprise namespace. Leave it empty for open-source Vault or when your Vault deployment does not use namespaces. | | no |

84 +| | [tls_skip_verify](#option-tls-skip-verify) | Disable TLS certificate verification for Vault requests. | no | no |

87 +

88 +<a id="option-mode"></a>

89 +##### mode

90 +

91 +Supported values:

92 +

93 +- `token`: store the Vault token directly in the secretstore configuration.

94 +- `token_file`: read the Vault token from a local file on the Netdata host.

95 +

96 +Prefer `token_file` for production so the token is stored separately from the secretstore configuration.

97 +

98 +

99 +<a id="option-tls-skip-verify"></a>

100 +##### tls_skip_verify

101 +

102 +This is insecure. Use it only as a temporary workaround or in a non-production environment.

103 +

104 +

105 +

106 +</details>

107 +

108 +

109 +#### via UI

110 +

111 +1. Open the Netdata Dynamic Configuration UI.

112 +2. Go to `Collectors -> go.d -> SecretStores -> vault`.

113 +3. Add a new secretstore and give it a store name.

114 +4. Fill in the backend-specific settings.

115 +5. Save the secretstore.

116 +

117 +#### via File

118 +

119 +Define the secretstore in `/etc/netdata/go.d/ss/vault.conf`.

120 +

121 +Each file contains a `jobs` array, and the secretstore kind is determined by the filename.

122 +

123 +After editing the file, restart the Netdata Agent to load the updated secretstore definition.

124 +

125 +##### Examples

126 +

127 +###### Token

128 +

129 +Store the Vault token directly in the secretstore definition.

130 +

131 +```yaml

132 +jobs:

133 + - name: vault_prod

134 + mode: token

135 + mode_token:

136 + token: your-vault-token

137 + addr: https://vault.example

138 + namespace: admin

139 + tls_skip_verify: false

140 +

141 +```

142 +###### Token file

143 +

144 +Read the Vault token from a local file on the Netdata host.

145 +

146 +```yaml

147 +jobs:

148 + - name: vault_prod_file_token

149 + mode: token_file

150 + mode_token_file:

151 + path: /var/lib/netdata/vault.token

152 + addr: https://vault.example

153 +

154 +```

155 +

156 +

157 +## Use in collector configs

158 +

159 +Reference Vault secrets from collector configs with the `vault` secretstore kind.

160 +

161 +

162 +The operand is `path#key`.

163 +

164 +Netdata sends the path to Vault as `/v1/<path>` exactly as you provide it. For KV v2 secrets, include `/data/` in the path yourself.

165 +The `path` must not be empty and must not contain `..`, `?`, or `#`.

166 +

167 +

168 +```text

169 +${store:vault:<store-name>:<path#key>}

170 +```

171 +

172 +- `vault`: The secretstore backend kind.

173 +- `<store-name>`: The name of the configured secretstore, for example `vault_prod`.

174 +- `<path#key>`: The Vault API path and the secret field name, separated by `#`.

175 +

176 +### Examples

177 +#### KV v1 secret

178 +

179 +Read the `password` field from a KV v1 secret.

180 +

181 +```text

182 +${store:vault:vault_prod:secret/netdata/mysql#password}

183 +```

184 +#### KV v2 secret

185 +

186 +Read the `password` field from a KV v2 secret. Include `/data/` in the path.

187 +

188 +```text

189 +${store:vault:vault_prod:secret/data/netdata/mysql#password}

190 +```

191 +#### Collector config example

192 +

193 +Use a Vault secret in a collector DSN.

194 +

195 +```yaml

196 +jobs:

197 + - name: mysql_prod

198 + dsn: "netdata:${store:vault:vault_prod:secret/data/netdata/mysql#password}@tcp(127.0.0.1:3306)/"

199 +

200 +```

201 +

202 +

203 +## Troubleshooting

204 +

205 +### Find the exact error

206 +

207 +Check the Netdata Agent logs when the collector starts or restarts. Vault resolver errors include messages such as `vault returned HTTP 403`, `vault path contains invalid characters`, `operand must be in format 'path#key'`, or `key 'password' not found in vault response`.

208 +

209 +

210 +### Vault returns permission denied or the token has expired

211 +

212 +Check the Vault token policy and, if you use Vault Enterprise namespaces, confirm that `namespace` is correct. If you use a short-lived token, make sure the token is renewed or replaced before it expires.

213 +

214 +

215 +### Secret or key is not found

216 +

217 +Check the operand carefully:

218 +

219 +- Make sure the path is the Vault API path.

220 +- For KV v2, make sure the path includes `/data/`.

221 +- Make sure the `key` exists in the returned secret payload.

222 +

223 +

224 +### TLS verification fails

225 +

226 +Make sure the Netdata host trusts the CA that signed the Vault certificate. Use `tls_skip_verify: true` only as an insecure workaround.

227 +

228 +

229 +### Token file cannot be read

230 +

231 +Check the file path, file contents, and that the `netdata` user can read the file.

232 +

233 +

234 +

src/go/plugin/agent/secrets/secretstore/backends/vault/metadata.yaml new

+169

@@ -0,0 +1,169 @@

1 +# yamllint disable rule:line-length

2 +---

3 +id: 'secretstore-vault'

4 +meta:

5 + kind: 'vault'

6 + name: 'Vault'

7 + link: 'https://developer.hashicorp.com/vault'

8 + icon_filename: 'vault.svg'

9 +keywords:

10 + - 'secretstore'

11 + - 'secrets'

12 + - 'vault'

13 + - 'hashicorp vault'

14 +overview:

15 + description: |

16 + Use Vault as a secretstore backend when you want Netdata collectors to read secrets from HashiCorp Vault at runtime instead of storing them in plain text in collector configuration files.

17 +

18 + This page covers Vault specific setup. For the shared resolver workflow and syntax, see [Secrets Management](/src/collectors/SECRETS.md).

19 + limitations: |

20 + Netdata reads existing secrets from Vault. It does not create or renew Vault tokens. If the configured token expires or becomes invalid, secret resolution fails until Netdata can read a valid token again. For KV v2 secrets, Netdata does not add `/data/` to the path automatically.

21 +setup:

22 + prerequisites:

23 + list:

24 + - title: 'Make Vault reachable'

25 + description: |

26 + Netdata must be able to reach your Vault server at the address you configure in `addr`.

27 + - title: 'Provide a Vault token'

28 + description: |

29 + Choose one supported authentication mode and make sure Netdata can use it:

30 +

31 + - `token`: store the Vault token directly in the secretstore configuration.

32 + - `token_file`: store the Vault token in a local file on the Netdata host that is readable by the `netdata` user.

33 +

34 + Prefer `token_file` for production so the Vault token is not embedded directly in the secretstore configuration.

35 + - title: 'Allow access to the referenced secret paths'

36 + description: |

37 + The Vault token used by this secretstore must be allowed to read the paths you reference from collector configs.

38 + - title: 'Plan for file-based changes'

39 + description: |

40 + If you edit `/etc/netdata/go.d/ss/vault.conf`, restart the Netdata Agent to load the updated secretstore definition.

41 + configuration:

42 + file:

43 + name: 'go.d/ss/vault.conf'

44 + options:

45 + description: 'The following options can be defined for this secretstore backend.'

46 + folding:

47 + title: 'Config options'

48 + enabled: true

49 + list:

50 + - name: 'mode'

51 + description: 'How Vault authentication is provided.'

52 + default_value: 'token'

53 + required: true

54 + detailed_description: |

55 + Supported values:

56 +

57 + - `token`: store the Vault token directly in the secretstore configuration.

58 + - `token_file`: read the Vault token from a local file on the Netdata host.

59 +

60 + Prefer `token_file` for production so the token is stored separately from the secretstore configuration.

61 + - name: 'addr'

62 + description: 'Vault server address / base URL.'

63 + default_value: ''

64 + required: true

65 + - name: 'namespace'

66 + description: 'Optional Vault Enterprise namespace. Leave it empty for open-source Vault or when your Vault deployment does not use namespaces.'

67 + default_value: ''

68 + required: false

69 + - name: 'tls_skip_verify'

70 + description: 'Disable TLS certificate verification for Vault requests.'

71 + default_value: false

72 + required: false

73 + detailed_description: |

74 + This is insecure. Use it only as a temporary workaround or in a non-production environment.

75 + - name: 'mode_token.token'

76 + group: 'Token'

77 + description: 'Vault token value. Required when `mode` is `token`.'

78 + default_value: ''

79 + required: true

80 + - name: 'mode_token_file.path'

81 + group: 'Token File'

82 + description: 'Path to a file containing the Vault token. Required when `mode` is `token_file`.'

83 + default_value: ''

84 + required: true

85 + examples:

86 + folding:

87 + title: 'Example configuration'

88 + enabled: true

89 + list:

90 + - name: 'Token'

91 + description: 'Store the Vault token directly in the secretstore definition.'

92 + config: |

93 + jobs:

94 + - name: vault_prod

95 + mode: token

96 + mode_token:

97 + token: your-vault-token

98 + addr: https://vault.example

99 + namespace: admin

100 + tls_skip_verify: false

101 + - name: 'Token file'

102 + description: 'Read the Vault token from a local file on the Netdata host.'

103 + config: |

104 + jobs:

105 + - name: vault_prod_file_token

106 + mode: token_file

107 + mode_token_file:

108 + path: /var/lib/netdata/vault.token

109 + addr: https://vault.example

110 +collector_configs:

111 + description: |

112 + Reference Vault secrets from collector configs with the `vault` secretstore kind.

113 + summary:

114 + operand_format: 'path#key'

115 + example_operand: 'secret/data/netdata/mysql#password'

116 + format:

117 + description: |

118 + The operand is `path#key`.

119 +

120 + Netdata sends the path to Vault as `/v1/<path>` exactly as you provide it. For KV v2 secrets, include `/data/` in the path yourself.

121 + The `path` must not be empty and must not contain `..`, `?`, or `#`.

122 + syntax: '${store:vault:<store-name>:<path#key>}'

123 + parts:

124 + list:

125 + - name: 'vault'

126 + description: 'The secretstore backend kind.'

127 + - name: '<store-name>'

128 + description: 'The name of the configured secretstore, for example `vault_prod`.'

129 + - name: '<path#key>'

130 + description: 'The Vault API path and the secret field name, separated by `#`.'

131 + examples:

132 + list:

133 + - name: 'KV v1 secret'

134 + description: 'Read the `password` field from a KV v1 secret.'

135 + language: 'text'

136 + content: '${store:vault:vault_prod:secret/netdata/mysql#password}'

137 + - name: 'KV v2 secret'

138 + description: 'Read the `password` field from a KV v2 secret. Include `/data/` in the path.'

139 + language: 'text'

140 + content: '${store:vault:vault_prod:secret/data/netdata/mysql#password}'

141 + - name: 'Collector config example'

142 + description: 'Use a Vault secret in a collector DSN.'

143 + language: 'yaml'

144 + content: |

145 + jobs:

146 + - name: mysql_prod

147 + dsn: "netdata:${store:vault:vault_prod:secret/data/netdata/mysql#password}@tcp(127.0.0.1:3306)/"

148 +troubleshooting:

149 + problems:

150 + list:

151 + - name: 'Find the exact error'

152 + description: |

153 + Check the Netdata Agent logs when the collector starts or restarts. Vault resolver errors include messages such as `vault returned HTTP 403`, `vault path contains invalid characters`, `operand must be in format 'path#key'`, or `key 'password' not found in vault response`.

154 + - name: 'Vault returns permission denied or the token has expired'

155 + description: |

156 + Check the Vault token policy and, if you use Vault Enterprise namespaces, confirm that `namespace` is correct. If you use a short-lived token, make sure the token is renewed or replaced before it expires.

157 + - name: 'Secret or key is not found'

158 + description: |

159 + Check the operand carefully:

160 +

161 + - Make sure the path is the Vault API path.

162 + - For KV v2, make sure the path includes `/data/`.

163 + - Make sure the `key` exists in the returned secret payload.

164 + - name: 'TLS verification fails'

165 + description: |

166 + Make sure the Netdata host trusts the CA that signed the Vault certificate. Use `tls_skip_verify: true` only as an insecure workaround.

167 + - name: 'Token file cannot be read'

168 + description: |

169 + Check the file path, file contents, and that the `netdata` user can read the file.