| 1 | # yamllint disable rule:line-length |
| 2 | --- |
| 3 | id: 'secretstore-vault' |
| 4 | meta: |
| 5 | kind: 'vault' |
| 6 | name: 'Vault' |
| 7 | link: 'https://developer.hashicorp.com/vault' |
| 8 | icon_filename: 'vault.svg' |
| 9 | keywords: |
| 10 | - 'secretstore' |
| 11 | - 'secrets' |
| 12 | - 'vault' |
| 13 | - 'hashicorp vault' |
| 14 | overview: |
| 15 | description: | |
| 16 | Netdata can pull collector credentials directly from HashiCorp Vault at runtime, so you never store passwords or tokens in plain-text configuration files. |
| 17 | |
| 18 | This page covers Vault specific setup. For the full resolver overview and syntax reference, including simpler alternatives like `${env:...}`, `${file:...}`, and `${cmd:...}`, see [Secrets Management](/src/collectors/SECRETS.md). |
| 19 | limitations: | |
| 20 | Netdata reads existing secrets from Vault. It does not create or renew Vault tokens. If the configured token expires or becomes invalid, secret resolution fails until Netdata can read a valid token again. If you use `token_file` mode, Netdata re-reads the file on every secret resolution, so an external process (e.g. Vault Agent, a cron job) can renew the token by writing to the file. For KV v2 secrets, Netdata does not add `/data/` to the path automatically. |
| 21 | setup: |
| 22 | prerequisites: |
| 23 | list: |
| 24 | - title: 'Make Vault reachable' |
| 25 | description: | |
| 26 | Netdata must be able to reach your Vault server at the address you configure in `addr`. |
| 27 | - title: 'Provide a Vault token' |
| 28 | description: | |
| 29 | Choose one supported authentication mode and make sure Netdata can use it: |
| 30 | |
| 31 | - `token`: store the Vault token directly in the secretstore configuration. |
| 32 | - `token_file`: store the Vault token in a local file on the Netdata host that is readable by the `netdata` user. |
| 33 | |
| 34 | Prefer `token_file` for production so the Vault token is not embedded directly in the secretstore configuration. |
| 35 | - title: 'Allow access to the referenced secret paths' |
| 36 | description: | |
| 37 | The Vault token used by this secretstore must have a policy that grants `read` capability on the paths you reference from collector configs. Scope the policy to only the paths Netdata needs. |
| 38 | - title: 'Plan for file-based changes' |
| 39 | description: | |
| 40 | If you edit `/etc/netdata/go.d/ss/vault.conf`, restart the Netdata Agent to load the updated secretstore definition. |
| 41 | configuration: |
| 42 | file: |
| 43 | name: 'go.d/ss/vault.conf' |
| 44 | options: |
| 45 | description: 'The following options can be defined for this secretstore backend.' |
| 46 | folding: |
| 47 | title: 'Config options' |
| 48 | enabled: true |
| 49 | list: |
| 50 | - name: 'mode' |
| 51 | description: 'How Vault authentication is provided.' |
| 52 | default_value: 'token' |
| 53 | required: true |
| 54 | detailed_description: | |
| 55 | Supported values: |
| 56 | |
| 57 | - `token`: store the Vault token directly in the secretstore configuration. |
| 58 | - `token_file`: read the Vault token from a local file on the Netdata host. |
| 59 | |
| 60 | Prefer `token_file` for production so the token is stored separately from the secretstore configuration. |
| 61 | - name: 'addr' |
| 62 | description: 'Vault server address / base URL.' |
| 63 | default_value: '' |
| 64 | required: true |
| 65 | - name: 'namespace' |
| 66 | description: 'Optional Vault Enterprise namespace. Leave it empty for open-source Vault or when your Vault deployment does not use namespaces.' |
| 67 | default_value: '' |
| 68 | required: false |
| 69 | - name: 'tls_skip_verify' |
| 70 | description: 'Disable TLS certificate verification for Vault requests.' |
| 71 | default_value: false |
| 72 | required: false |
| 73 | detailed_description: | |
| 74 | This is insecure. Use it only as a temporary workaround or in a non-production environment. |
| 75 | - name: 'timeout' |
| 76 | description: 'Timeout in seconds for HTTP requests made by this secretstore backend.' |
| 77 | default_value: 3 |
| 78 | required: false |
| 79 | - name: 'mode_token.token' |
| 80 | group: 'Token' |
| 81 | description: 'Vault token value. Required when `mode` is `token`.' |
| 82 | default_value: '' |
| 83 | required: true |
| 84 | - name: 'mode_token_file.path' |
| 85 | group: 'Token File' |
| 86 | description: 'Path to a file containing the Vault token. Required when `mode` is `token_file`.' |
| 87 | default_value: '' |
| 88 | required: true |
| 89 | examples: |
| 90 | folding: |
| 91 | title: 'Example configuration' |
| 92 | enabled: true |
| 93 | list: |
| 94 | - name: 'Token' |
| 95 | description: 'Store the Vault token directly in the secretstore definition.' |
| 96 | config: | |
| 97 | jobs: |
| 98 | - name: vault_prod |
| 99 | mode: token |
| 100 | mode_token: |
| 101 | token: your-vault-token |
| 102 | addr: https://vault.example |
| 103 | - name: 'Token from environment variable' |
| 104 | description: 'Use a `${env:...}` resolver for the Vault token to avoid storing it in plain text in the secretstore config file.' |
| 105 | config: | |
| 106 | jobs: |
| 107 | - name: vault_prod |
| 108 | mode: token |
| 109 | mode_token: |
| 110 | token: "${env:VAULT_TOKEN}" |
| 111 | addr: https://vault.example |
| 112 | - name: 'Token file' |
| 113 | description: 'Read the Vault token from a local file on the Netdata host. Netdata re-reads the file on every secret resolution, so an external process can renew the token by writing to this file.' |
| 114 | config: | |
| 115 | jobs: |
| 116 | - name: vault_prod_file_token |
| 117 | mode: token_file |
| 118 | mode_token_file: |
| 119 | path: /var/lib/netdata/vault.token |
| 120 | addr: https://vault.example |
| 121 | - name: 'Vault Enterprise with namespace' |
| 122 | description: 'Connect to a Vault Enterprise server using a specific namespace.' |
| 123 | config: | |
| 124 | jobs: |
| 125 | - name: vault_enterprise |
| 126 | mode: token_file |
| 127 | mode_token_file: |
| 128 | path: /var/lib/netdata/vault.token |
| 129 | addr: https://vault.example |
| 130 | namespace: admin |
| 131 | collector_configs: |
| 132 | description: | |
| 133 | Use the `${store:vault:...}` syntax to reference Vault secrets in any string field of a collector configuration file. |
| 134 | summary: |
| 135 | operand_format: 'path#key' |
| 136 | example_operand: 'secret/data/netdata/mysql#password' |
| 137 | format: |
| 138 | description: | |
| 139 | The operand is `path#key`. |
| 140 | |
| 141 | Netdata sends the path to Vault as `/v1/<path>` exactly as you provide it. For KV v2 secrets, include `/data/` in the path yourself. |
| 142 | The `path` must not be empty and must not contain `..`, `?`, or `#`. |
| 143 | |
| 144 | - KV v1 example: `${store:vault:vault_prod:secret/netdata/mysql#password}`. |
| 145 | - KV v2 example: `${store:vault:vault_prod:secret/data/netdata/mysql#password}` — note the `/data/` segment, Netdata does not add it automatically. |
| 146 | syntax: '${store:vault:<store-name>:<path#key>}' |
| 147 | parts: |
| 148 | list: |
| 149 | - name: 'vault' |
| 150 | description: 'The secretstore backend kind.' |
| 151 | - name: '<store-name>' |
| 152 | description: 'The name of the configured secretstore, for example `vault_prod`.' |
| 153 | - name: '<path#key>' |
| 154 | description: 'The Vault API path and the secret field name, separated by `#`.' |
| 155 | examples: |
| 156 | list: |
| 157 | - name: 'MySQL collector with password from Vault' |
| 158 | description: | |
| 159 | This example configures a MySQL collector job in `/etc/netdata/go.d/mysql.conf`. |
| 160 | The password in the DSN connection string is not stored in plain text. Instead, |
| 161 | `${store:vault:vault_prod:secret/data/netdata/mysql#password}` tells Netdata to |
| 162 | read the KV v2 secret at `secret/data/netdata/mysql` from the `vault_prod` store, |
| 163 | extract the `password` field from the response, and substitute it into the DSN at runtime. |
| 164 | content: | |
| 165 | # /etc/netdata/go.d/mysql.conf |
| 166 | jobs: |
| 167 | - name: mysql_prod |
| 168 | dsn: "netdata:${store:vault:vault_prod:secret/data/netdata/mysql#password}@tcp(127.0.0.1:3306)/" |
| 169 | - name: 'Elasticsearch collector with HTTP basic auth from Vault' |
| 170 | description: | |
| 171 | This example configures an Elasticsearch collector job in `/etc/netdata/go.d/elasticsearch.conf`. |
| 172 | The `password` field uses a secret reference instead of a plain-text value. Netdata reads the |
| 173 | KV v2 secret at `secret/data/netdata/elasticsearch` from the `vault_prod` store, extracts the |
| 174 | `password` field, and substitutes it at runtime. |
| 175 | content: | |
| 176 | # /etc/netdata/go.d/elasticsearch.conf |
| 177 | jobs: |
| 178 | - name: es_prod |
| 179 | url: https://elasticsearch.example.com:9200 |
| 180 | username: netdata |
| 181 | password: "${store:vault:vault_prod:secret/data/netdata/elasticsearch#password}" |
| 182 | troubleshooting: |
| 183 | problems: |
| 184 | list: |
| 185 | - name: 'Find the exact error' |
| 186 | description: | |
| 187 | Check the Netdata Agent logs when the collector starts or restarts. Vault resolver errors include messages such as `vault returned HTTP 403`, `vault path contains invalid characters`, `operand must be in format 'path#key'`, or `key 'password' not found in vault response`. |
| 188 | - name: 'Vault returns permission denied or the token has expired' |
| 189 | description: | |
| 190 | Check the Vault token policy and, if you use Vault Enterprise namespaces, confirm that `namespace` is correct. If you use a short-lived token, make sure the token is renewed or replaced before it expires. |
| 191 | - name: 'Secret or key is not found' |
| 192 | description: | |
| 193 | Check the operand carefully: |
| 194 | |
| 195 | - Make sure the path is the Vault API path. |
| 196 | - For KV v2, make sure the path includes `/data/`. |
| 197 | - Make sure the `key` exists in the returned secret payload. |
| 198 | - name: 'TLS verification fails' |
| 199 | description: | |
| 200 | Make sure the Netdata host trusts the CA that signed the Vault certificate. Use `tls_skip_verify: true` only as an insecure workaround. |
| 201 | - name: 'Token file cannot be read' |
| 202 | description: | |
| 203 | Check the file path, file contents, and that the `netdata` user can read the file. |