@cryptotaxi247 / netdata-1 / commits / 879468086

feat(go.d): implement secretstore-based secrets management (#21951)

Ilya Mashchenko committed Mar 18, 2026 at 23:11 UTC 87946808680b0248c6c19314e2924e7fa30c0136

110 files changed +14302 -2232

docs/.map/map.yaml

@@ -225,6 +225,10 @@ sidebar:

225 - meta:

226 label: H2O

227 edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-h2o.md

228 + - meta:

229 + label: Secrets Management

230 + edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/secrets-management.md

231 + description: Use secret references to avoid storing credentials in plain text. Supports environment variables, files, and external vaults.

232 - meta:

233 label: Performance Optimization

234 edit_url: https://github.com/netdata/netdata/edit/master/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md

docs/netdata-agent/configuration/secrets-management.md new

+143

@@ -0,0 +1,143 @@

1 +# Secrets Management

2 +

3 +Netdata supports secrets management for collector configurations, so you don't need to store plain-text credentials in configuration files. Instead, you use secret references that are resolved when a collector starts.

4 +

5 +| Reference Type | Syntax | Use Case |

6 +|:---------------------|:-----------------------------------|:-----------------------------------------------------------|

7 +| Environment variable | `${env:VAR_NAME}` | Secrets available as environment variables |

8 +| File | `${file:/path/to/secret}` | Secrets stored in files on disk |

9 +| Command | `${cmd:/path/to/command args}` | Secrets retrieved by running a command |

10 +| Secretstore | `${store:<kind>:<name>:<operand>}` | Secrets stored in remote backends (Vault, AWS, Azure, GCP) |

11 +

12 +## Environment Variables

13 +

14 +Use `${env:VARIABLE_NAME}` to reference an environment variable.

15 +

16 +```yaml

17 +jobs:

18 + - name: local

19 + dsn: "${env:MYSQL_USER}:${env:MYSQL_PASSWORD}@tcp(127.0.0.1:3306)/"

20 +```

21 +

22 +## Files

23 +

24 +Use `${file:/absolute/path}` to read a secret from a file. Leading and trailing whitespace is trimmed automatically.

25 +

26 +```yaml

27 +jobs:

28 + - name: myapp

29 + password: "${file:/run/secrets/myapp_password}"

30 +```

31 +

32 +## Commands

33 +

34 +Use `${cmd:/absolute/path/to/command args}` to execute a command and use its stdout as the secret value.

35 +

36 +```yaml

37 +jobs:

38 + - name: prod

39 + password: "${cmd:/usr/bin/op read op://vault/netdata/mysql/password}"

40 +```

41 +

42 +:::warning

43 +

44 +- Command paths must be absolute.

45 +- Commands have a 10-second timeout.

46 +- Arguments are split on whitespace. Quoting, pipes, redirects, and variable expansion are not interpreted unless you run a shell explicitly (e.g., `${cmd:/bin/sh -c "your command here"}`).

47 +

48 +:::

49 +

50 +## Secretstores

51 +

52 +For remote secret backends (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager), you configure a **secretstore** and then reference it from collector configurations.

53 +

54 +### Supported Providers

55 +

57 +|:-----------|:--------------------|:---------------------------------------------|:-------------------------------------|

62 +

63 +### Reference Format

64 +

65 +```text

66 +${store:<kind>:<name>:<operand>}

67 +```

68 +

69 +| Part | Description |

70 +|:----------|:---------------------------------------------------------|

71 +| `kind` | Provider kind from the table above (e.g., `vault`) |

72 +| `name` | The name you gave the secretstore when you configured it |

73 +| `operand` | Provider-specific path to the secret (see table above) |

74 +

75 +### Examples

76 +

77 +```yaml

78 +jobs:

79 + - name: mysql_prod

80 + password: "${store:vault:vault_prod:secret/data/netdata/mysql#password}"

81 +

82 + - name: redis_prod

83 + password: "${store:aws-sm:aws_prod:netdata/redis#password}"

84 +

85 + - name: api_prod

86 + token: "${store:azure-kv:azure_prod:my-vault/api-token}"

87 +

88 + - name: app_prod

89 + password: "${store:gcp-sm:gcp_prod:my-project/mysql-password}"

90 +```

91 +

92 +### Configuring a Secretstore

93 +

94 +#### Option 1: Dynamic Configuration (UI)

95 +

96 +1. Open the Netdata Dynamic Configuration UI.

97 +2. Choose a provider kind and give your secretstore a name.

98 +3. Fill in the provider-specific settings (address, credentials, etc.).

99 +4. Use the reference syntax `${store:<kind>:<name>:<operand>}` in your collector configs.

100 +

101 +#### Option 2: Configuration Files

102 +

103 +You can define secretstores in configuration files. Each provider has its own file:

104 +

105 +| File | Provider |

106 +|:-------------------------------------|:--------------------|

107 +| `/etc/netdata/go.d/ss/vault.conf` | HashiCorp Vault |

108 +| `/etc/netdata/go.d/ss/aws-sm.conf` | AWS Secrets Manager |

109 +| `/etc/netdata/go.d/ss/azure-kv.conf` | Azure Key Vault |

110 +| `/etc/netdata/go.d/ss/gcp-sm.conf` | GCP Secret Manager |

111 +

112 +Each file contains a `jobs` array. The provider kind is determined by the filename.

113 +

114 +Example (`/etc/netdata/go.d/ss/vault.conf`):

115 +

116 +```yaml

117 +jobs:

118 + - name: vault_prod

119 + mode: token

120 + mode_token:

121 + token: your-vault-token

122 + addr: https://vault.example.com

123 +```

124 +

125 +:::note

126 +

127 +File-based secretstores are loaded at agent startup. If you edit these files, restart the Netdata Agent to apply the changes.

128 +

129 +:::

130 +

131 +## How It Works

132 +

133 +- Secrets are resolved each time a collector job starts or restarts.

134 +- If a secret cannot be resolved, the collector job will fail to start and log an error.

135 +- Updating a secretstore automatically restarts running and failed collector jobs that use it, so they pick up the new credentials.

136 +- Accepted or disabled jobs keep their state and use the updated secretstore the next time they start.

137 +- If a secretstore change applies successfully but some dependent collector restarts fail, the command reports those restart failures.

138 +

139 +:::tip

140 +

141 +Avoid storing plain-text credentials in collector configurations. Use environment variables, files, commands, or secretstores instead.

142 +

143 +:::

src/go/plugin/agent/agent.go

@@ -247,6 +247,7 @@ func (a *Agent) run(ctx context.Context) {

247 VarLibDir: a.VarLibDir,

248 FnReg: fnMgr,

249 Vnodes: a.setupVnodeRegistry(),

250 + SecretStores: a.setupSecretStoreConfigs(),

251 AuditMode: a.auditDuration > 0,

252 AuditAnalyzer: a.auditAnalyzer,

253 AuditDataDir: a.auditDataDir,

src/go/plugin/agent/jobmgr/cache.go

@@ -41,6 +41,8 @@ type (

41 }

43 retryingTasks struct {

44 + // retryingTasks is intentionally lock-free. All access must remain serialized

45 + // through manager-owned command flow and synchronous dyncfg handler callbacks.

46 // [cfg.UID()]

47 items map[string]*retryTask

48 }

src/go/plugin/agent/jobmgr/config_apply.go new

+88

@@ -0,0 +1,88 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "context"

7 + "fmt"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/resolver"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

13 + "gopkg.in/yaml.v2"

14 +)

15 +

16 +func newConfigModule(creator collectorapi.Creator) (configModule, error) {

17 + if creator.CreateV2 != nil {

18 + mod := creator.CreateV2()

19 + if mod == nil {

20 + return nil, fmt.Errorf("CreateV2 returned nil")

21 + }

22 + return mod, nil

23 + }

24 + if creator.Create == nil {

25 + return nil, fmt.Errorf("no module creator is defined")

26 + }

27 + mod := creator.Create()

28 + if mod == nil {

29 + return nil, fmt.Errorf("Create returned nil")

30 + }

31 + return mod, nil

32 +}

33 +

34 +func applyConfig(

35 + ctx context.Context,

36 + cfg confgroup.Config,

37 + module any,

38 + resolver *secretresolver.Resolver,

39 + storeService secretstore.Service,

40 + storeSnapshot *secretstore.Snapshot,

41 +) error {

42 + if resolver == nil {

43 + return fmt.Errorf("secret resolver is nil")

44 + }

45 + cfgResolved, err := cfg.Clone()

46 + if err != nil {

47 + return fmt.Errorf("cloning config: %w", err)

48 + }

49 + storeResolver := secretresolver.StoreRefResolver(nil)

50 + if storeService != nil {

51 + storeResolver = func(resolveCtx context.Context, ref, original string) (string, error) {

52 + if resolveCtx == nil {

53 + resolveCtx = ctx

54 + }

55 + return storeService.Resolve(resolveCtx, storeSnapshot, ref, original)

56 + }

57 + }

58 + if err := resolver.ResolveWithStoreResolver(ctx, cfgResolved, storeResolver); err != nil {

59 + return fmt.Errorf("resolving secrets: %w", err)

60 + }

61 + bs, err := yaml.Marshal(cfgResolved)

62 + if err != nil {

63 + return err

64 + }

65 + return yaml.Unmarshal(bs, module)

66 +}

67 +

68 +// applyConfigRaw applies config without resolving secrets.

69 +// Used by dyncfg get to avoid exposing resolved secret values.

70 +func applyConfigRaw(cfg confgroup.Config, module any) error {

71 + bs, err := yaml.Marshal(cfg)

72 + if err != nil {

73 + return err

74 + }

75 + return yaml.Unmarshal(bs, module)

76 +}

77 +

78 +func makeLabels(cfg confgroup.Config) map[string]string {

79 + labels := make(map[string]string)

80 + for name, value := range cfg.Labels() {

81 + n, ok1 := name.(string)

82 + v, ok2 := value.(string)

83 + if ok1 && ok2 {

84 + labels[n] = v

85 + }

86 + }

87 + return labels

88 +}

src/go/plugin/agent/jobmgr/dyncfg.go

+12 -12

@@ -11,13 +11,13 @@ import (

11 func (m *Manager) dyncfgConfig(fn dyncfg.Function) {

12 if err := fn.ValidateArgs(2); err != nil {

13 m.Warningf("dyncfg: %v", err)

14 - m.dyncfgApi.SendCodef(fn, 400, "%v", err)

14 + m.dyncfgResponder.SendCodef(fn, 400, "%v", err)

15 return

16 }

18 select {

19 case <-m.ctx.Done():

20 - m.dyncfgApi.SendCodef(fn, 503, "Job manager is shutting down.")

20 + m.dyncfgResponder.SendCodef(fn, 503, "Job manager is shutting down.")

21 return

22 default:

23 }

@@ -26,27 +26,27 @@ func (m *Manager) dyncfgConfig(fn dyncfg.Function) {

26 }

28 func (m *Manager) dyncfgQueuedExec(fn dyncfg.Function) {

29 - id := fn.ID()

30 -

29 switch {

32 - case strings.HasPrefix(id, m.dyncfgCollectorPrefixValue()):

30 + case strings.HasPrefix(fn.ID(), m.dyncfgSecretStorePrefixValue()):

31 + m.dyncfgSecretStoreExec(fn)

32 + case strings.HasPrefix(fn.ID(), m.dyncfgCollectorPrefixValue()):

33 m.dyncfgCollectorExec(fn)

34 - case strings.HasPrefix(id, m.dyncfgVnodePrefixValue()):

34 + case strings.HasPrefix(fn.ID(), m.dyncfgVnodePrefixValue()):

35 m.dyncfgVnodeExec(fn)

36 default:

37 - m.dyncfgApi.SendCodef(fn, 503, "unknown function '%s' (%s).", fn.Fn().Name, id)

37 + m.dyncfgResponder.SendCodef(fn, 503, "unknown function '%s' (%s).", fn.Fn().Name, fn.ID())

38 }

39 }

41 func (m *Manager) dyncfgSeqExec(fn dyncfg.Function) {

42 - id := fn.ID()

43 -

42 switch {

45 - case strings.HasPrefix(id, m.dyncfgCollectorPrefixValue()):

43 + case strings.HasPrefix(fn.ID(), m.dyncfgSecretStorePrefixValue()):

44 + m.dyncfgSecretStoreSeqExec(fn)

45 + case strings.HasPrefix(fn.ID(), m.dyncfgCollectorPrefixValue()):

46 m.dyncfgCollectorSeqExec(fn)

47 - case strings.HasPrefix(id, m.dyncfgVnodePrefixValue()):

47 + case strings.HasPrefix(fn.ID(), m.dyncfgVnodePrefixValue()):

48 m.dyncfgVnodeSeqExec(fn)

49 default:

50 - m.dyncfgApi.SendCodef(fn, 503, "unknown function '%s' (%s).", fn.Fn().Name, id)

50 + m.dyncfgResponder.SendCodef(fn, 503, "unknown function '%s' (%s).", fn.Fn().Name, fn.ID())

51 }

52 }

src/go/plugin/agent/jobmgr/dyncfg_collector.go

+13 -484

@@ -3,19 +3,9 @@

3 package jobmgr

5 import (

6 - "context"

7 - "encoding/json"

6 "fmt"

9 - "log/slog"

10 - "slices"

11 - "strings"

12 - "time"

14 - "gopkg.in/yaml.v2"

15 -

16 - "github.com/netdata/netdata/go/plugins/logger"

8 "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

18 - "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

9 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

10 "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 )

@@ -48,7 +38,7 @@ func dyncfgCollectorModCmds() string {

38 }

40 func (m *Manager) dyncfgCollectorModuleCreate(name string) {

51 - m.dyncfgApi.ConfigCreate(netdataapi.ConfigOpts{

41 + m.dyncfgResponder.ConfigCreate(netdataapi.ConfigOpts{

42 ID: m.dyncfgModID(name),

43 Status: dyncfg.StatusAccepted.String(),

44 ConfigType: dyncfg.ConfigTypeTemplate.String(),

@@ -65,15 +55,7 @@ func (m *Manager) exposedLookupByName(module, job string) (*dyncfg.Entry[confgro

55 if module == job {

56 key = job

57 }

68 - return m.exposed.LookupByKey(key)

69 -}

70 -

71 -type dyncfgCmdTestTask struct {

72 - fn dyncfg.Function

73 - moduleName string

74 - creator collectorapi.Creator

75 - cfg confgroup.Config

76 - timeout time.Duration

58 + return m.collectorExposed.LookupByKey(key)

59 }

61 func (m *Manager) dyncfgCollectorExec(fn dyncfg.Function) {

@@ -90,21 +72,24 @@ func (m *Manager) dyncfgCollectorExec(fn dyncfg.Function) {

73 func (m *Manager) dyncfgCollectorSeqExec(fn dyncfg.Function) {

74 cmd := fn.Command()

93 - m.handler.SyncDecision(fn)

75 + m.collectorHandler.SyncDecision(fn)

77 switch cmd {

78 case dyncfg.CommandAdd:

97 - m.handler.CmdAdd(fn)

79 + m.collectorHandler.CmdAdd(fn)

80 + m.syncSecretStoreDepsByFunction(fn)

81 case dyncfg.CommandUpdate:

99 - m.handler.CmdUpdate(fn)

82 + m.collectorHandler.CmdUpdate(fn)

83 + m.syncSecretStoreDepsByFunction(fn)

84 case dyncfg.CommandEnable:

101 - m.handler.CmdEnable(fn)

85 + m.collectorHandler.CmdEnable(fn)

86 case dyncfg.CommandDisable:

103 - m.handler.CmdDisable(fn)

87 + m.collectorHandler.CmdDisable(fn)

88 case dyncfg.CommandRemove:

105 - m.handler.CmdRemove(fn)

89 + m.collectorHandler.CmdRemove(fn)

90 + m.syncSecretStoreDepsByFunction(fn)

91 case dyncfg.CommandRestart:

107 - m.handler.CmdRestart(fn)

92 + m.collectorHandler.CmdRestart(fn)

93 case dyncfg.CommandTest:

94 m.dyncfgCmdTest(fn)

95 case dyncfg.CommandSchema:

@@ -113,462 +98,6 @@ func (m *Manager) dyncfgCollectorSeqExec(fn dyncfg.Function) {

98 m.dyncfgCmdGet(fn)

99 default:

100 m.Warningf("dyncfg: function '%s' command '%s' not implemented", fn.Fn().Name, cmd)

116 - m.dyncfgApi.SendCodef(fn, 501, "Function '%s' command '%s' is not implemented.", fn.Fn().Name, cmd)

117 - }

118 -}

119 -

120 -func (m *Manager) dyncfgCmdUserconfig(fn dyncfg.Function) {

121 - cmd := fn.Command()

122 -

123 - id := fn.ID()

124 - jn := fn.JobName()

125 - if jn == "" {

126 - jn = "test"

127 - }

128 -

129 - mn, ok := m.extractModuleName(id)

130 - if !ok {

131 - m.Warningf("dyncfg: %s: could not extract module and job from id (%s)", cmd, id)

132 - m.dyncfgApi.SendCodef(fn, 400, "Invalid ID format. Could not extract module and job name from ID. Provided ID: %s.", id)

133 - return

134 - }

135 -

136 - creator, ok := m.modules.Lookup(mn)

137 - if !ok {

138 - m.Warningf("dyncfg: %s: module %s not found", cmd, mn)

139 - m.dyncfgApi.SendCodef(fn, 404, "The specified module '%s' is not registered.", mn)

140 - return

141 - }

142 -

143 - if creator.Config == nil || creator.Config() == nil {

144 - m.Warningf("dyncfg: %s: module %s: configuration not found", cmd, mn)

145 - m.dyncfgApi.SendCodef(fn, 500, "Module %s does not provide configuration.", mn)

146 - return

147 - }

148 -

149 - bs, err := userConfigFromPayload(creator.Config(), jn, fn)

150 - if err != nil {

151 - m.Warningf("dyncfg: %s: module %s: failed to create config from payload: %v", cmd, mn, err)

152 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

153 - return

154 - }

155 -

156 - m.dyncfgApi.SendYAML(fn, string(bs))

157 -}

158 -

159 -func (m *Manager) dyncfgCmdTest(fn dyncfg.Function) {

160 - cmd := fn.Command()

161 -

162 - id := fn.ID()

163 - mn, ok := m.extractModuleName(id)

164 - if !ok {

165 - m.Warningf("dyncfg: %s: could not extract module and job from id (%s)", cmd, id)

166 - m.dyncfgApi.SendCodef(fn, 400, "Invalid ID format. Could not extract module and job name from ID. Provided ID: %s.", id)

167 - return

168 - }

169 -

170 - jn := fn.JobName()

171 - if jn == "" {

172 - jn = "test"

173 - }

174 -

175 - m.Infof("dyncfg: %s: %s/%s job by user '%s'", cmd, mn, jn, fn.User())

176 -

177 - if err := dyncfg.ValidateJobName(jn); err != nil {

178 - m.Warningf("dyncfg: %s: module %s: unacceptable job name '%s': %v", cmd, mn, jn, err)

179 - m.dyncfgApi.SendCodef(fn, 400, "Unacceptable job name '%s': %v.", jn, err)

180 - return

181 - }

182 -

183 - creator, ok := m.modules.Lookup(mn)

184 - if !ok {

185 - m.Warningf("dyncfg: %s: module %s not found", cmd, mn)

186 - m.dyncfgApi.SendCodef(fn, 404, "The specified module '%s' is not registered.", mn)

187 - return

188 - }

189 - cfg, err := configFromPayload(fn)

190 - if err != nil {

191 - m.Warningf("dyncfg: %s: module %s: failed to create config from payload: %v", cmd, mn, err)

192 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

193 - return

194 - }

195 -

196 - if cfg.Vnode() != "" {

197 - if _, ok := m.vnodes.Lookup(cfg.Vnode()); !ok {

198 - m.Warningf("dyncfg: %s: module %s: vnode %s not found", cmd, mn, cfg.Vnode())

199 - m.dyncfgApi.SendCodef(fn, 400, "The specified vnode '%s' is not registered.", cfg.Vnode())

200 - return

201 - }

202 - }

203 -

204 - cfg.SetModule(mn)

205 - cfg.SetName(jn)

206 -

207 - if err := m.baseContext().Err(); err != nil {

208 - m.dyncfgApi.SendCodef(fn, 503, "Job manager is shutting down.")

209 - return

210 - }

211 -

212 - select {

213 - case m.cmdTestSem <- struct{}{}:

214 - task := dyncfgCmdTestTask{

215 - fn: fn,

216 - moduleName: mn,

217 - creator: creator,

218 - cfg: cfg,

219 - timeout: m.dyncfgCmdTestTimeout(fn),

220 - }

221 - m.cmdTestWG.Go(func() {

222 - m.runDyncfgCmdTest(task)

223 - })

224 - default:

225 - m.Warningf("dyncfg: %s: module %s: too many concurrent test requests", cmd, mn)

226 - m.dyncfgApi.SendCodef(fn, 503, "Too many concurrent test requests, try again later.")

227 - }

228 -}

229 -

230 -func (m *Manager) runDyncfgCmdTest(task dyncfgCmdTestTask) {

231 - defer func() { <-m.cmdTestSem }()

232 -

233 - job, err := newConfigModule(task.creator)

234 - if err != nil {

235 - m.Warningf("dyncfg: test: module %s: failed to create module: %v", task.moduleName, err)

236 - m.dyncfgApi.SendCodef(task.fn, 500, "Module %s instantiation failed: %v.", task.moduleName, err)

237 - return

238 - }

239 -

240 - cleanupCtx, cleanupCancel := context.WithTimeout(m.baseContext(), cmdTestWorkerDrainWait)

241 - defer cleanupCancel()

242 - defer job.Cleanup(cleanupCtx)

243 -

244 - if err := applyConfig(task.cfg, job); err != nil {

245 - m.Warningf("dyncfg: test: module %s: failed to apply config: %v", task.moduleName, err)

246 - m.dyncfgApi.SendCodef(task.fn, 400, "Invalid configuration. Failed to apply configuration: %v.", err)

247 - return

248 - }

249 -

250 - job.GetBase().Logger = logger.New().With(

251 - slog.String("collector", task.cfg.Module()),

252 - slog.String("job", task.cfg.Name()),

253 - )

254 -

255 - ctx, cancel := context.WithTimeout(m.baseContext(), task.timeout)

256 - defer cancel()

257 -

258 - if err := job.Init(ctx); err != nil {

259 - m.dyncfgApi.SendCodef(task.fn, 422, "Job initialization failed: %v", err)

260 - return

261 - }

262 - if err := job.Check(ctx); err != nil {

263 - m.dyncfgApi.SendCodef(task.fn, 422, "Job check failed: %v", err)

264 - return

265 - }

266 -

267 - m.dyncfgApi.SendCodef(task.fn, 200, "")

268 -}

269 -

270 -func (m *Manager) dyncfgCmdTestTimeout(fn dyncfg.Function) time.Duration {

271 - if timeout := fn.Fn().Timeout; timeout > 0 {

272 - return timeout

273 - }

274 - return cmdTestDefaultTimeout

275 -}

276 -

277 -func (m *Manager) dyncfgCmdSchema(fn dyncfg.Function) {

278 - cmd := fn.Command()

279 -

280 - id := fn.ID()

281 - mn, ok := m.extractModuleName(id)

282 - if !ok {

283 - m.Warningf("dyncfg: %s: could not extract module from id (%s)", cmd, id)

284 - m.dyncfgApi.SendCodef(fn, 400, "Invalid ID format. Could not extract module name from ID. Provided ID: %s.", id)

285 - return

286 - }

287 -

288 - mod, ok := m.modules.Lookup(mn)

289 - if !ok {

290 - m.Warningf("dyncfg: %s: module %s not found", cmd, mn)

291 - m.dyncfgApi.SendCodef(fn, 404, "The specified module '%s' is not registered.", mn)

292 - return

293 - }

294 -

295 - m.Infof("dyncfg: %s: %s module by user '%s'", cmd, mn, fn.User())

296 -

297 - if mod.JobConfigSchema == "" {

298 - m.Warningf("dyncfg: schema: module %s: schema not found", mn)

299 - m.dyncfgApi.SendCodef(fn, 500, "Module %s configuration schema not found.", mn)

300 - return

301 - }

302 -

303 - m.dyncfgApi.SendJSON(fn, mod.JobConfigSchema)

304 -}

305 -

306 -func (m *Manager) dyncfgCmdGet(fn dyncfg.Function) {

307 - cmd := fn.Command()

308 -

309 - id := fn.ID()

310 - mn, jn, ok := m.extractModuleJobName(id)

311 - if !ok {

312 - m.Warningf("dyncfg: %s: could not extract module and job from id (%s)", cmd, id)

313 - m.dyncfgApi.SendCodef(fn, 400, "Invalid ID format. Could not extract module and job name from ID. Provided ID: %s.", id)

314 - return

315 - }

316 -

317 - creator, ok := m.modules.Lookup(mn)

318 - if !ok {

319 - m.Warningf("dyncfg: %s: module %s not found", cmd, mn)

320 - m.dyncfgApi.SendCodef(fn, 404, "The specified module '%s' is not registered.", mn)

321 - return

322 - }

323 -

324 - m.Infof("dyncfg: %s: %s/%s job by user '%s'", cmd, mn, jn, fn.User())

325 -

326 - entry, ok := m.exposedLookupByName(mn, jn)

327 - if !ok {

328 - m.Warningf("dyncfg: %s: module %s job %s not found", cmd, mn, jn)

329 - m.dyncfgApi.SendCodef(fn, 404, "The specified module '%s' job '%s' is not registered.", mn, jn)

330 - return

101 + m.dyncfgResponder.SendCodef(fn, 501, "Function '%s' command '%s' is not implemented.", fn.Fn().Name, cmd)

102 }

332 -

333 - mod, err := newConfigModule(creator)

334 - if err != nil {

335 - m.Warningf("dyncfg: %s: module %s job %s failed to create module: %v", cmd, mn, jn, err)

336 - m.dyncfgApi.SendCodef(fn, 500, "Module %s instantiation failed: %v.", mn, err)

337 - return

338 - }

339 -

340 - if err := applyConfig(entry.Cfg, mod); err != nil {

341 - m.Warningf("dyncfg: %s: module %s job %s failed to apply config: %v", cmd, mn, jn, err)

342 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration. Failed to apply configuration: %v.", err)

343 - return

344 - }

345 -

346 - conf := mod.Configuration()

347 - if conf == nil {

348 - m.Warningf("dyncfg: %s: module %s: configuration not found", cmd, mn)

349 - m.dyncfgApi.SendCodef(fn, 500, "Module %s does not provide configuration.", mn)

350 - return

351 - }

352 -

353 - bs, err := json.Marshal(conf)

354 - if err != nil {

355 - m.Warningf("dyncfg: %s: module %s job %s failed to json marshal config: %v", cmd, mn, jn, err)

356 - m.dyncfgApi.SendCodef(fn, 500, "Failed to convert configuration into JSON: %v.", err)

357 - return

358 - }

359 -

360 - m.dyncfgApi.SendJSON(fn, string(bs))

361 -}

362 -

363 -func (m *Manager) dyncfgSetConfigMeta(cfg confgroup.Config, module, name string, fn dyncfg.Function) {

364 - cfg.SetProvider("dyncfg")

365 - cfg.SetSource(fn.Source())

366 - cfg.SetSourceType("dyncfg")

367 - cfg.SetModule(module)

368 - cfg.SetName(name)

369 - if def, ok := m.configDefaults.Lookup(module); ok {

370 - cfg.ApplyDefaults(def)

371 - }

372 -}

373 -

374 -// scheduleRetryTask schedules a retry if the job supports auto-detection retry.

375 -func (m *Manager) scheduleRetryTask(cfg confgroup.Config, job runtimeJob) {

376 - if !job.RetryAutoDetection() {

377 - return

378 - }

379 - m.Infof("%s[%s] job detection failed, will retry in %d seconds",

380 - cfg.Module(), cfg.Name(), job.AutoDetectionEvery())

381 -

382 - ctx, cancel := context.WithCancel(m.ctx)

383 - m.retryingTasks.add(cfg, &retryTask{cancel: cancel})

384 -

385 - go runRetryTask(ctx, m.addCh, cfg)

386 -}

387 -

388 -func userConfigFromPayload(cfg any, jobName string, fn dyncfg.Function) ([]byte, error) {

389 - if err := fn.UnmarshalPayload(cfg); err != nil {

390 - return nil, err

391 - }

392 -

393 - bs, err := yaml.Marshal(cfg)

394 - if err != nil {

395 - return nil, err

396 - }

397 -

398 - var yms yaml.MapSlice

399 - if err := yaml.Unmarshal(bs, &yms); err != nil {

400 - return nil, err

401 - }

402 -

403 - yms = slices.DeleteFunc(yms, func(item yaml.MapItem) bool { return item.Key == "name" })

404 -

405 - yms = append([]yaml.MapItem{{Key: "name", Value: jobName}}, yms...)

406 -

407 - v := map[string]any{

408 - "jobs": []any{yms},

409 - }

410 -

411 - return yaml.Marshal(v)

412 -}

413 -

414 -func configFromPayload(fn dyncfg.Function) (confgroup.Config, error) {

415 - var cfg confgroup.Config

416 -

417 - if fn.IsContentTypeJSON() {

418 - if err := json.Unmarshal(fn.Payload(), &cfg); err != nil {

419 - return nil, err

420 - }

421 -

422 - return cfg.Clone()

423 - }

424 -

425 - if err := yaml.Unmarshal(fn.Payload(), &cfg); err != nil {

426 - return nil, err

427 - }

428 -

429 - return cfg, nil

430 -}

431 -

432 -func (m *Manager) extractModuleJobName(id string) (mn string, jn string, ok bool) {

433 - if mn, ok = m.extractModuleName(id); !ok {

434 - return "", "", false

435 - }

436 - if jn, ok = extractJobName(id); !ok {

437 - return "", "", false

438 - }

439 - return mn, jn, true

440 -}

441 -

442 -func (m *Manager) extractModuleName(id string) (string, bool) {

443 - id = strings.TrimPrefix(id, m.dyncfgCollectorPrefixValue())

444 - i := strings.IndexByte(id, ':')

445 - if i == -1 {

446 - return id, id != ""

447 - }

448 - return id[:i], true

449 -}

450 -

451 -func extractJobName(id string) (string, bool) {

452 - i := strings.LastIndexByte(id, ':')

453 - if i == -1 {

454 - return "", false

455 - }

456 - return id[i+1:], true

457 -}

458 -

459 -// --- collectorCallbacks implements dyncfg.Callbacks[confgroup.Config] ---

460 -

461 -type collectorCallbacks struct {

462 - mgr *Manager

463 -}

464 -

465 -func (cb *collectorCallbacks) ExtractKey(fn dyncfg.Function) (key, name string, ok bool) {

466 - var mn, jn string

467 -

468 - if fn.Command() == dyncfg.CommandAdd {

469 - // For add: ID is module template, job name is in Args[2].

470 - mn, ok = cb.mgr.extractModuleName(fn.ID())

471 - if !ok {

472 - return "", "", false

473 - }

474 - jn = fn.JobName()

475 - if jn == "" {

476 - return "", "", false

477 - }

478 - } else {

479 - // For other commands: ID contains module:job.

480 - mn, jn, ok = cb.mgr.extractModuleJobName(fn.ID())

481 - if !ok {

482 - return "", "", false

483 - }

484 - }

485 -

486 - key = mn + "_" + jn

487 - if mn == jn {

488 - key = jn

489 - }

490 - return key, jn, true

491 -}

492 -

493 -func (cb *collectorCallbacks) ParseAndValidate(fn dyncfg.Function, name string) (confgroup.Config, error) {

494 - mn, ok := cb.mgr.extractModuleName(fn.ID())

495 - if !ok {

496 - return nil, fmt.Errorf("could not extract module name from ID: %s", fn.ID())

497 - }

498 -

499 - cfg, err := configFromPayload(fn)

500 - if err != nil {

501 - return nil, fmt.Errorf("invalid configuration format: failed to create configuration from payload: %v", err)

502 - }

503 -

504 - cb.mgr.dyncfgSetConfigMeta(cfg, mn, name, fn)

505 -

506 - if _, err := cb.mgr.createCollectorJob(cfg); err != nil {

507 - return nil, fmt.Errorf("invalid configuration: failed to apply configuration: %v", err)

508 - }

509 -

510 - return cfg, nil

103 }

512 -

513 -func (cb *collectorCallbacks) Start(cfg confgroup.Config) error {

514 - cb.mgr.retryingTasks.remove(cfg)

515 -

516 - job, err := cb.mgr.createCollectorJob(cfg)

517 - if err != nil {

518 - return &codedError{err: fmt.Errorf("invalid configuration: failed to apply configuration: %v", err), code: 400}

519 - }

520 -

521 - if err := job.AutoDetection(); err != nil {

522 - job.Cleanup()

523 - cb.mgr.scheduleRetryTask(cfg, job)

524 - return fmt.Errorf("job enable failed: %v", err)

525 - }

526 -

527 - cb.mgr.startRunningJob(job)

528 - return nil

529 -}

530 -

531 -func (cb *collectorCallbacks) Update(oldCfg, newCfg confgroup.Config) error {

532 - cb.mgr.retryingTasks.remove(oldCfg)

533 - cb.mgr.stopRunningJob(oldCfg.FullName())

534 - cb.mgr.fileStatus.remove(oldCfg)

535 -

536 - job, err := cb.mgr.createCollectorJob(newCfg)

537 - if err != nil {

538 - return fmt.Errorf("job update failed: %v", err)

539 - }

540 -

541 - if err := job.AutoDetection(); err != nil {

542 - job.Cleanup()

543 - cb.mgr.scheduleRetryTask(newCfg, job)

544 - return fmt.Errorf("job update failed: %v", err)

545 - }

546 -

547 - cb.mgr.startRunningJob(job)

548 - return nil

549 -}

550 -

551 -func (cb *collectorCallbacks) Stop(cfg confgroup.Config) {

552 - cb.mgr.retryingTasks.remove(cfg)

553 - cb.mgr.stopRunningJob(cfg.FullName())

554 - cb.mgr.fileStatus.remove(cfg)

555 -}

556 -

557 -func (cb *collectorCallbacks) OnStatusChange(entry *dyncfg.Entry[confgroup.Config], _ dyncfg.Status, _ dyncfg.Function) {

558 - if entry.Status == dyncfg.StatusRunning && isDyncfg(entry.Cfg) {

559 - cb.mgr.fileStatus.add(entry.Cfg, entry.Status.String())

560 - }

561 -}

562 -

563 -func (cb *collectorCallbacks) ConfigID(cfg confgroup.Config) string {

564 - return cb.mgr.dyncfgJobID(cfg)

565 -}

566 -

567 -// codedError wraps an error with an HTTP status code for the handler.

568 -type codedError struct {

569 - err error

570 - code int

571 -}

572 -

573 -func (e *codedError) Error() string { return e.err.Error() }

574 -func (e *codedError) Code() int { return e.code }

src/go/plugin/agent/jobmgr/dyncfg_collector_callbacks.go new

+143

@@ -0,0 +1,143 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "context"

7 + "fmt"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 +)

12 +

13 +// scheduleRetryTask schedules a retry if the job supports auto-detection retry.

14 +func (m *Manager) scheduleRetryTask(cfg confgroup.Config, job runtimeJob) {

15 + if !job.RetryAutoDetection() {

16 + return

17 + }

18 + m.Infof("%s[%s] job detection failed, will retry in %d seconds",

19 + cfg.Module(), cfg.Name(), job.AutoDetectionEvery())

20 +

21 + ctx, cancel := context.WithCancel(m.ctx)

22 + m.retryingTasks.add(cfg, &retryTask{cancel: cancel})

23 +

24 + go runRetryTask(ctx, m.addCh, cfg)

25 +}

26 +

27 +// --- collectorCallbacks implements dyncfg.Callbacks[confgroup.Config] ---

28 +

29 +type collectorCallbacks struct {

30 + mgr *Manager

31 +}

32 +

33 +func (cb *collectorCallbacks) ExtractKey(fn dyncfg.Function) (key, name string, ok bool) {

34 + var mn, jn string

35 +

36 + if fn.Command() == dyncfg.CommandAdd {

37 + // For add: ID is module template, job name is in Args[2].

38 + mn, ok = cb.mgr.extractModuleName(fn.ID())

39 + if !ok {

40 + return "", "", false

41 + }

42 + jn = fn.JobName()

43 + if jn == "" {

44 + return "", "", false

45 + }

46 + } else {

47 + // For other commands: ID contains module:job.

48 + mn, jn, ok = cb.mgr.extractModuleJobName(fn.ID())

49 + if !ok {

50 + return "", "", false

51 + }

52 + }

53 +

54 + key = mn + "_" + jn

55 + if mn == jn {

56 + key = jn

57 + }

58 + return key, jn, true

59 +}

60 +

61 +func (cb *collectorCallbacks) ParseAndValidate(fn dyncfg.Function, name string) (confgroup.Config, error) {

62 + mn, ok := cb.mgr.extractModuleName(fn.ID())

63 + if !ok {

64 + return nil, fmt.Errorf("could not extract module name from ID: %s", fn.ID())

65 + }

66 +

67 + cfg, err := configFromPayload(fn)

68 + if err != nil {

69 + return nil, fmt.Errorf("invalid configuration format: failed to create configuration from payload: %v", err)

70 + }

71 +

72 + cb.mgr.dyncfgSetConfigMeta(cfg, mn, name, fn)

73 +

74 + if err := cb.mgr.validateCollectorJob(cfg); err != nil {

75 + return nil, fmt.Errorf("invalid configuration: failed to apply configuration: %v", err)

76 + }

77 +

78 + return cfg, nil

79 +}

80 +

81 +func (cb *collectorCallbacks) Start(cfg confgroup.Config) error {

82 + cb.mgr.retryingTasks.remove(cfg)

83 +

84 + job, err := cb.mgr.createCollectorJob(cfg)

85 + if err != nil {

86 + return &codedError{err: fmt.Errorf("invalid configuration: failed to apply configuration: %v", err), code: 400}

87 + }

88 +

89 + if err := job.AutoDetection(); err != nil {

90 + job.Cleanup()

91 + cb.mgr.scheduleRetryTask(cfg, job)

92 + return fmt.Errorf("job enable failed: %v", err)

93 + }

94 +

95 + cb.mgr.startRunningJob(job)

96 + return nil

97 +}

98 +

99 +func (cb *collectorCallbacks) Update(oldCfg, newCfg confgroup.Config) error {

100 + cb.mgr.retryingTasks.remove(oldCfg)

101 + cb.mgr.stopRunningJob(oldCfg.FullName())

102 + cb.mgr.fileStatus.remove(oldCfg)

103 +

104 + job, err := cb.mgr.createCollectorJob(newCfg)

105 + if err != nil {

106 + return fmt.Errorf("job update failed: %v", err)

107 + }

108 +

109 + if err := job.AutoDetection(); err != nil {

110 + job.Cleanup()

111 + cb.mgr.scheduleRetryTask(newCfg, job)

112 + return fmt.Errorf("job update failed: %v", err)

113 + }

114 +

115 + cb.mgr.startRunningJob(job)

116 + return nil

117 +}

118 +

119 +func (cb *collectorCallbacks) Stop(cfg confgroup.Config) {

120 + cb.mgr.retryingTasks.remove(cfg)

121 + cb.mgr.stopRunningJob(cfg.FullName())

122 + cb.mgr.fileStatus.remove(cfg)

123 +}

124 +

125 +func (cb *collectorCallbacks) OnStatusChange(entry *dyncfg.Entry[confgroup.Config], _ dyncfg.Status, _ dyncfg.Function) {

126 + if entry.Status == dyncfg.StatusRunning && isDyncfg(entry.Cfg) {

127 + cb.mgr.fileStatus.add(entry.Cfg, entry.Status.String())

128 + }

129 +}

130 +

131 +func (cb *collectorCallbacks) ConfigID(cfg confgroup.Config) string {

132 + return cb.mgr.dyncfgJobID(cfg)

133 +}

134 +

135 +// codedError wraps an error with an HTTP status code for the handler.

136 +type codedError struct {

137 + err error

138 + code int

139 +}

140 +

141 +func (e *codedError) Error() string { return e.err.Error() }

142 +func (e *codedError) Unwrap() error { return e.err }

143 +func (e *codedError) Code() int { return e.code }

src/go/plugin/agent/jobmgr/dyncfg_collector_cmds.go new

+285

@@ -0,0 +1,285 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "context"

7 + "encoding/json"

8 + "log/slog"

9 + "time"

10 +

11 + "github.com/netdata/netdata/go/plugins/logger"

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

14 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

15 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

16 +)

17 +

18 +type dyncfgCmdTestTask struct {

19 + fn dyncfg.Function

20 + moduleName string

21 + creator collectorapi.Creator

22 + cfg confgroup.Config

23 + timeout time.Duration

24 +}

25 +

26 +func (m *Manager) requireModuleFromID(fn dyncfg.Function, target string) (string, collectorapi.Creator, bool) {

27 + cmd := fn.Command()

28 + id := fn.ID()

29 +

30 + moduleName, ok := m.extractModuleName(id)

31 + if !ok {

32 + m.Warningf("dyncfg: %s: could not extract %s from id (%s)", cmd, target, id)

33 + m.dyncfgResponder.SendCodef(fn, 400, "Invalid ID format. Could not extract %s from ID. Provided ID: %s.", target, id)

34 + return "", collectorapi.Creator{}, false

35 + }

36 +

37 + creator, ok := m.modules.Lookup(moduleName)

38 + if !ok {

39 + m.Warningf("dyncfg: %s: module %s not found", cmd, moduleName)

40 + m.dyncfgResponder.SendCodef(fn, 404, "The specified module '%s' is not registered.", moduleName)

41 + return "", collectorapi.Creator{}, false

42 + }

43 +

44 + return moduleName, creator, true

45 +}

46 +

47 +func (m *Manager) requireTemplateModule(fn dyncfg.Function) (string, collectorapi.Creator, bool) {

48 + return m.requireModuleFromID(fn, "module and job name")

49 +}

50 +

51 +func (m *Manager) requireModule(fn dyncfg.Function) (string, collectorapi.Creator, bool) {

52 + return m.requireModuleFromID(fn, "module name")

53 +}

54 +

55 +func (m *Manager) requireModuleJob(fn dyncfg.Function) (string, string, collectorapi.Creator, bool) {

56 + cmd := fn.Command()

57 + id := fn.ID()

58 +

59 + moduleName, jobName, ok := m.extractModuleJobName(id)

60 + if !ok {

61 + m.Warningf("dyncfg: %s: could not extract module and job from id (%s)", cmd, id)

62 + m.dyncfgResponder.SendCodef(fn, 400, "Invalid ID format. Could not extract module and job name from ID. Provided ID: %s.", id)

63 + return "", "", collectorapi.Creator{}, false

64 + }

65 +

66 + creator, ok := m.modules.Lookup(moduleName)

67 + if !ok {

68 + m.Warningf("dyncfg: %s: module %s not found", cmd, moduleName)

69 + m.dyncfgResponder.SendCodef(fn, 404, "The specified module '%s' is not registered.", moduleName)

70 + return "", "", collectorapi.Creator{}, false

71 + }

72 +

73 + return moduleName, jobName, creator, true

74 +}

75 +

76 +func (m *Manager) dyncfgCmdUserconfig(fn dyncfg.Function) {

77 + jn := fn.JobName()

78 + if jn == "" {

79 + jn = "test"

80 + }

81 +

82 + mn, creator, ok := m.requireTemplateModule(fn)

83 + if !ok {

84 + return

85 + }

86 +

87 + if creator.Config == nil || creator.Config() == nil {

88 + m.Warningf("dyncfg: %s: module %s: configuration not found", fn.Command(), mn)

89 + m.dyncfgResponder.SendCodef(fn, 500, "Module %s does not provide configuration.", mn)

90 + return

91 + }

92 +

93 + bs, err := userConfigFromPayload(creator.Config(), jn, fn)

94 + if err != nil {

95 + m.Warningf("dyncfg: %s: module %s: failed to create config from payload: %v", fn.Command(), mn, err)

96 + m.dyncfgResponder.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

97 + return

98 + }

99 +

100 + m.dyncfgResponder.SendYAML(fn, string(bs))

101 +}

102 +

103 +func (m *Manager) dyncfgCmdTest(fn dyncfg.Function) {

104 + cmd := fn.Command()

105 +

106 + mn, creator, ok := m.requireTemplateModule(fn)

107 + if !ok {

108 + return

109 + }

110 +

111 + jn := fn.JobName()

112 + if jn == "" {

113 + jn = "test"

114 + }

115 +

116 + m.Infof("dyncfg: %s: %s/%s job by user '%s'", cmd, mn, jn, fn.User())

117 +

118 + if err := dyncfg.ValidateJobName(jn); err != nil {

119 + m.Warningf("dyncfg: %s: module %s: unacceptable job name '%s': %v", cmd, mn, jn, err)

120 + m.dyncfgResponder.SendCodef(fn, 400, "Unacceptable job name '%s': %v.", jn, err)

121 + return

122 + }

123 + if !fn.HasPayload() {

124 + m.Warningf("dyncfg: %s: module %s: missing configuration payload", cmd, mn)

125 + m.dyncfgResponder.SendCodef(fn, 400, "Missing configuration payload.")

126 + return

127 + }

128 +

129 + cfg, err := configFromPayload(fn)

130 + if err != nil {

131 + m.Warningf("dyncfg: %s: module %s: failed to create config from payload: %v", cmd, mn, err)

132 + m.dyncfgResponder.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

133 + return

134 + }

135 +

136 + if cfg.Vnode() != "" {

137 + if _, ok := m.vnodesCtl.Lookup(cfg.Vnode()); !ok {

138 + m.Warningf("dyncfg: %s: module %s: vnode %s not found", cmd, mn, cfg.Vnode())

139 + m.dyncfgResponder.SendCodef(fn, 400, "The specified vnode '%s' is not registered.", cfg.Vnode())

140 + return

141 + }

142 + }

143 +

144 + cfg.SetModule(mn)

145 + cfg.SetName(jn)

146 +

147 + if err := m.baseContext().Err(); err != nil {

148 + m.dyncfgResponder.SendCodef(fn, 503, "Job manager is shutting down.")

149 + return

150 + }

151 +

152 + select {

153 + case m.cmdTestSem <- struct{}{}:

154 + task := dyncfgCmdTestTask{

155 + fn: fn,

156 + moduleName: mn,

157 + creator: creator,

158 + cfg: cfg,

159 + timeout: m.dyncfgCmdTestTimeout(fn),

160 + }

161 + m.cmdTestWG.Go(func() {

162 + m.runDyncfgCmdTest(task)

163 + })

164 + default:

165 + m.Warningf("dyncfg: %s: module %s: too many concurrent test requests", cmd, mn)

166 + m.dyncfgResponder.SendCodef(fn, 503, "Too many concurrent test requests, try again later.")

167 + }

168 +}

169 +

170 +func (m *Manager) runDyncfgCmdTest(task dyncfgCmdTestTask) {

171 + defer func() { <-m.cmdTestSem }()

172 +

173 + job, err := newConfigModule(task.creator)

174 + if err != nil {

175 + m.Warningf("dyncfg: test: module %s: failed to create module: %v", task.moduleName, err)

176 + m.dyncfgResponder.SendCodef(task.fn, 500, "Module %s instantiation failed: %v.", task.moduleName, err)

177 + return

178 + }

179 +

180 + cleanupCtx, cleanupCancel := context.WithTimeout(m.baseContext(), cmdTestWorkerDrainWait)

181 + defer cleanupCancel()

182 + defer job.Cleanup(cleanupCtx)

183 +

184 + ctx, cancel := context.WithTimeout(m.baseContext(), task.timeout)

185 + defer cancel()

186 +

187 + secretStoreSvc := m.secretsCtl.Service()

188 +

189 + storeSnapshot := (*secretstore.Snapshot)(nil)

190 + if secretStoreSvc != nil {

191 + storeSnapshot = secretStoreSvc.Capture()

192 + }

193 + if err := applyConfig(ctx, task.cfg, job, m.secretResolver, secretStoreSvc, storeSnapshot); err != nil {

194 + m.Warningf("dyncfg: test: module %s: failed to apply config: %v", task.moduleName, err)

195 + m.dyncfgResponder.SendCodef(task.fn, 400, "Invalid configuration. Failed to apply configuration: %v.", err)

196 + return

197 + }

198 +

199 + job.GetBase().Logger = logger.New().With(

200 + slog.String("collector", task.cfg.Module()),

201 + slog.String("job", task.cfg.Name()),

202 + )

203 +

204 + if err := job.Init(ctx); err != nil {

205 + m.dyncfgResponder.SendCodef(task.fn, 422, "Job initialization failed: %v", err)

206 + return

207 + }

208 + if err := job.Check(ctx); err != nil {

209 + m.dyncfgResponder.SendCodef(task.fn, 422, "Job check failed: %v", err)

210 + return

211 + }

212 +

213 + m.dyncfgResponder.SendCodef(task.fn, 200, "")

214 +}

215 +

216 +func (m *Manager) dyncfgCmdTestTimeout(fn dyncfg.Function) time.Duration {

217 + if timeout := fn.Fn().Timeout; timeout > 0 {

218 + return timeout

219 + }

220 + return cmdTestDefaultTimeout

221 +}

222 +

223 +func (m *Manager) dyncfgCmdSchema(fn dyncfg.Function) {

224 + mn, mod, ok := m.requireModule(fn)

225 + if !ok {

226 + return

227 + }

228 +

229 + m.Infof("dyncfg: %s: %s module by user '%s'", fn.Command(), mn, fn.User())

230 +

231 + if mod.JobConfigSchema == "" {

232 + m.Warningf("dyncfg: schema: module %s: schema not found", mn)

233 + m.dyncfgResponder.SendCodef(fn, 500, "Module %s configuration schema not found.", mn)

234 + return

235 + }

236 +

237 + m.dyncfgResponder.SendJSON(fn, mod.JobConfigSchema)

238 +}

239 +

240 +func (m *Manager) dyncfgCmdGet(fn dyncfg.Function) {

241 + cmd := fn.Command()

242 +

243 + mn, jn, creator, ok := m.requireModuleJob(fn)

244 + if !ok {

245 + return

246 + }

247 +

248 + m.Infof("dyncfg: %s: %s/%s job by user '%s'", fn.Command(), mn, jn, fn.User())

249 +

250 + entry, ok := m.exposedLookupByName(mn, jn)

251 + if !ok {

252 + m.Warningf("dyncfg: %s: module %s job %s not found", cmd, mn, jn)

253 + m.dyncfgResponder.SendCodef(fn, 404, "The specified module '%s' job '%s' is not registered.", mn, jn)

254 + return

255 + }

256 +

257 + mod, err := newConfigModule(creator)

258 + if err != nil {

259 + m.Warningf("dyncfg: %s: module %s job %s failed to create module: %v", cmd, mn, jn, err)

260 + m.dyncfgResponder.SendCodef(fn, 500, "Module %s instantiation failed: %v.", mn, err)

261 + return

262 + }

263 +

264 + if err := applyConfigRaw(entry.Cfg, mod); err != nil {

265 + m.Warningf("dyncfg: %s: module %s job %s failed to apply config: %v", cmd, mn, jn, err)

266 + m.dyncfgResponder.SendCodef(fn, 400, "Invalid configuration. Failed to apply configuration: %v.", err)

267 + return

268 + }

269 +

270 + conf := mod.Configuration()

271 + if conf == nil {

272 + m.Warningf("dyncfg: %s: module %s: configuration not found", cmd, mn)

273 + m.dyncfgResponder.SendCodef(fn, 500, "Module %s does not provide configuration.", mn)

274 + return

275 + }

276 +

277 + bs, err := json.Marshal(conf)

278 + if err != nil {

279 + m.Warningf("dyncfg: %s: module %s job %s failed to json marshal config: %v", cmd, mn, jn, err)

280 + m.dyncfgResponder.SendCodef(fn, 500, "Failed to convert configuration into JSON: %v.", err)

281 + return

282 + }

283 +

284 + m.dyncfgResponder.SendJSON(fn, string(bs))

285 +}

src/go/plugin/agent/jobmgr/dyncfg_collector_helpers.go new

+95

@@ -0,0 +1,95 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "encoding/json"

7 + "slices"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

12 + "gopkg.in/yaml.v2"

13 +)

14 +

15 +func (m *Manager) dyncfgSetConfigMeta(cfg confgroup.Config, module, name string, fn dyncfg.Function) {

16 + cfg.SetProvider("dyncfg")

17 + cfg.SetSource(fn.Source())

18 + cfg.SetSourceType("dyncfg")

19 + cfg.SetModule(module)

20 + cfg.SetName(name)

21 + if def, ok := m.configDefaults.Lookup(module); ok {

22 + cfg.ApplyDefaults(def)

23 + }

24 +}

25 +

26 +func userConfigFromPayload(cfg any, jobName string, fn dyncfg.Function) ([]byte, error) {

27 + if err := fn.UnmarshalPayload(cfg); err != nil {

28 + return nil, err

29 + }

30 +

31 + bs, err := yaml.Marshal(cfg)

32 + if err != nil {

33 + return nil, err

34 + }

35 +

36 + var yms yaml.MapSlice

37 + if err := yaml.Unmarshal(bs, &yms); err != nil {

38 + return nil, err

39 + }

40 +

41 + yms = slices.DeleteFunc(yms, func(item yaml.MapItem) bool { return item.Key == "name" })

42 +

43 + yms = append([]yaml.MapItem{{Key: "name", Value: jobName}}, yms...)

44 +

45 + v := map[string]any{

46 + "jobs": []any{yms},

47 + }

48 +

49 + return yaml.Marshal(v)

50 +}

51 +

52 +func configFromPayload(fn dyncfg.Function) (confgroup.Config, error) {

53 + var cfg confgroup.Config

54 +

55 + if fn.IsContentTypeJSON() {

56 + if err := json.Unmarshal(fn.Payload(), &cfg); err != nil {

57 + return nil, err

58 + }

59 +

60 + return cfg.Clone()

61 + }

62 +

63 + if err := yaml.Unmarshal(fn.Payload(), &cfg); err != nil {

64 + return nil, err

65 + }

66 +

67 + return cfg, nil

68 +}

69 +

70 +func (m *Manager) extractModuleJobName(id string) (mn string, jn string, ok bool) {

71 + if mn, ok = m.extractModuleName(id); !ok {

72 + return "", "", false

73 + }

74 + if jn, ok = extractJobName(id); !ok {

75 + return "", "", false

76 + }

77 + return mn, jn, true

78 +}

79 +

80 +func (m *Manager) extractModuleName(id string) (string, bool) {

81 + id = strings.TrimPrefix(id, m.dyncfgCollectorPrefixValue())

82 + i := strings.IndexByte(id, ':')

83 + if i == -1 {

84 + return id, id != ""

85 + }

86 + return id[:i], true

87 +}

88 +

89 +func extractJobName(id string) (string, bool) {

90 + i := strings.LastIndexByte(id, ':')

91 + if i == -1 {

92 + return "", false

93 + }

94 + return id[i+1:], true

95 +}

src/go/plugin/agent/jobmgr/dyncfg_collector_test.go

+800

@@ -6,6 +6,8 @@ import (

6 "bytes"

7 "context"

8 "encoding/json"

9 + "errors"

10 + "os"

11 "strings"

12 "testing"

13 "time"

@@ -15,8 +17,12 @@ import (

18 "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

19 "github.com/netdata/netdata/go/plugins/pkg/safewriter"

20 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

21 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

22 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

23 "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

24 "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

25 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

26 )

28 func TestDyncfgConfigUserconfig_InvalidPayload_Returns400Only(t *testing.T) {

@@ -123,3 +129,797 @@ func TestDyncfgCmdTestTimeout_RequestTimeoutOverridesDefault(t *testing.T) {

129 withoutTimeout := dyncfg.NewFunction(functions.Function{})

130 assert.Equal(t, cmdTestDefaultTimeout, mgr.dyncfgCmdTestTimeout(withoutTimeout))

131 }

132 +

133 +func TestDyncfgCollectorSeqExec_SyncsSecretStoreDepsForMutatingCommands(t *testing.T) {

134 + tests := map[string]struct {

135 + command dyncfg.Command

136 + oldCfg confgroup.Config

137 + newCfg confgroup.Config

138 + args []string

139 + wantOldExposed int

140 + wantNewExposed int

141 + wantOldRunning int

142 + wantNewRunning int

143 + }{

144 + "add syncs active deps from newly exposed config": {

145 + command: dyncfg.CommandAdd,

146 + newCfg: prepareDyncfgCfg("success", "job").Set("password", "${store:vault:vault_prod:secret/data/mysql#password}"),

147 + args: []string{"success", string(dyncfg.CommandAdd), "job"},

148 + wantOldExposed: 0,

149 + wantNewExposed: 1,

150 + wantOldRunning: 0,

151 + wantNewRunning: 0,

152 + },

153 + "update replaces active deps with updated config": {

154 + command: dyncfg.CommandUpdate,

155 + oldCfg: prepareDyncfgCfg("success", "job").Set("password", "${store:vault:vault_old:secret/data/mysql#password}"),

156 + newCfg: prepareDyncfgCfg("success", "job").Set("password", "${store:vault:vault_new:secret/data/mysql#password}"),

157 + args: []string{"success:job", string(dyncfg.CommandUpdate)},

158 + wantOldExposed: 0,

159 + wantNewExposed: 1,

160 + wantOldRunning: 0,

161 + wantNewRunning: 0,

162 + },

163 + "remove clears active deps when config disappears": {

164 + command: dyncfg.CommandRemove,

165 + oldCfg: prepareDyncfgCfg("success", "job").Set("password", "${store:vault:vault_old:secret/data/mysql#password}"),

166 + args: []string{"success:job", string(dyncfg.CommandRemove)},

167 + wantOldExposed: 0,

168 + wantNewExposed: 0,

169 + wantOldRunning: 0,

170 + wantNewRunning: 0,

171 + },

172 + }

173 +

174 + for name, tc := range tests {

175 + t.Run(name, func(t *testing.T) {

176 + mgr := newCollectorTestManager()

177 + cb := &collectorSeqTestCallbacks{mgr: mgr, parsed: map[dyncfg.Command]confgroup.Config{}}

178 + if tc.newCfg != nil {

179 + cb.parsed[tc.command] = tc.newCfg

180 + }

181 + mgr.collectorHandler = newCollectorTestHandler(mgr, cb)

182 +

183 + if tc.oldCfg != nil {

184 + seedCollectorEntry(mgr, tc.oldCfg, dyncfg.StatusDisabled)

185 + mgr.syncSecretStoreDepsForConfig(tc.oldCfg)

186 + }

187 +

188 + var payload []byte

189 + if tc.command == dyncfg.CommandAdd || tc.command == dyncfg.CommandUpdate {

190 + payload = mustMarshalCollectorConfigPayload(t, tc.newCfg)

191 + }

192 +

193 + fn := dyncfg.NewFunction(functions.Function{

194 + UID: name,

195 + ContentType: "application/json",

196 + Payload: payload,

197 + Args: collectorTestArgs(mgr, tc.args...),

198 + })

199 +

200 + mgr.dyncfgCollectorSeqExec(fn)

201 +

202 + oldExposed, oldRunning := mgr.secretStoreDeps.Impacted("vault:vault_old")

203 + newExposed, newRunning := mgr.secretStoreDeps.Impacted("vault:vault_prod")

204 + if tc.command == dyncfg.CommandUpdate {

205 + newExposed, newRunning = mgr.secretStoreDeps.Impacted("vault:vault_new")

206 + }

207 +

208 + assert.Len(t, oldExposed, tc.wantOldExposed)

209 + assert.Len(t, oldRunning, tc.wantOldRunning)

210 + assert.Len(t, newExposed, tc.wantNewExposed)

211 + assert.Len(t, newRunning, tc.wantNewRunning)

212 + })

213 + }

214 +}

215 +

216 +func TestDyncfgCollectorSeqExec_DoesNotSyncSecretStoreDepsForNonMutatingCommands(t *testing.T) {

217 + tests := map[string]struct {

218 + command dyncfg.Command

219 + args []string

220 + payload confgroup.Config

221 + status dyncfg.Status

222 + }{

223 + "restart leaves deps unchanged": {

224 + command: dyncfg.CommandRestart,

225 + args: []string{"success:job", string(dyncfg.CommandRestart)},

226 + status: dyncfg.StatusDisabled,

227 + },

228 + "test leaves deps unchanged": {

229 + command: dyncfg.CommandTest,

230 + args: []string{"success", string(dyncfg.CommandTest), "job"},

231 + payload: prepareDyncfgCfg("success", "job"),

232 + status: dyncfg.StatusDisabled,

233 + },

234 + "schema leaves deps unchanged": {

235 + command: dyncfg.CommandSchema,

236 + args: []string{"success", string(dyncfg.CommandSchema)},

237 + status: dyncfg.StatusDisabled,

238 + },

239 + "get leaves deps unchanged": {

240 + command: dyncfg.CommandGet,

241 + args: []string{"success:job", string(dyncfg.CommandGet)},

242 + status: dyncfg.StatusDisabled,

243 + },

244 + }

245 +

246 + for name, tc := range tests {

247 + t.Run(name, func(t *testing.T) {

248 + mgr := newCollectorTestManager()

249 + cfg := prepareDyncfgCfg("success", "job").Set("password", "${store:vault:vault_prod:secret/data/mysql#password}")

250 + seedCollectorEntry(mgr, cfg, tc.status)

251 + mgr.syncSecretStoreDepsForConfig(cfg)

252 +

253 + beforeExposed, beforeRunning := mgr.secretStoreDeps.Impacted("vault:vault_prod")

254 + require.Len(t, beforeExposed, 1)

255 +

256 + var payload []byte

257 + if tc.payload != nil {

258 + payload = mustMarshalCollectorConfigPayload(t, tc.payload)

259 + }

260 +

261 + fn := dyncfg.NewFunction(functions.Function{

262 + UID: name,

263 + ContentType: "application/json",

264 + Payload: payload,

265 + Args: collectorTestArgs(mgr, tc.args...),

266 + })

267 +

268 + mgr.dyncfgCollectorSeqExec(fn)

269 + if tc.command == dyncfg.CommandTest {

270 + mgr.cmdTestWG.Wait()

271 + }

272 +

273 + afterExposed, afterRunning := mgr.secretStoreDeps.Impacted("vault:vault_prod")

274 + assert.Equal(t, beforeExposed, afterExposed)

275 + assert.Equal(t, beforeRunning, afterRunning)

276 + })

277 + }

278 +}

279 +

280 +func TestCollectorCallbacks_ParseAndValidate(t *testing.T) {

281 + tests := map[string]struct {

282 + args []string

283 + cfg confgroup.Config

284 + payload []byte

285 + wantErr string

286 + wantModule string

287 + wantName string

288 + wantProvider string

289 + wantSourceTyp string

290 + }{

291 + "invalid id is rejected": {

292 + args: []string{"", string(dyncfg.CommandAdd), "validated"},

293 + cfg: prepareDyncfgCfg("success", "validated"),

294 + wantErr: "could not extract module name from ID",

295 + },

296 + "invalid payload is rejected": {

297 + args: []string{"success", string(dyncfg.CommandAdd), "validated"},

298 + payload: []byte("{"),

299 + wantErr: "invalid configuration format",

300 + },

301 + "valid payload is accepted and metadata is set": {

302 + args: []string{"success", string(dyncfg.CommandAdd), "validated"},

303 + cfg: prepareDyncfgCfg("success", "payload-name").Set("option_str", "one").Set("option_int", 2),

304 + wantModule: "success",

305 + wantName: "validated",

306 + wantProvider: "dyncfg",

307 + wantSourceTyp: confgroup.TypeDyncfg,

308 + },

309 + }

310 +

311 + for name, tc := range tests {

312 + t.Run(name, func(t *testing.T) {

313 + mgr := newCollectorTestManager()

314 + cb := &collectorCallbacks{mgr: mgr}

315 + payload := tc.payload

316 + if payload == nil && tc.cfg != nil {

317 + payload = mustMarshalCollectorConfigPayload(t, tc.cfg)

318 + }

319 + fn := dyncfg.NewFunction(functions.Function{

320 + UID: name,

321 + ContentType: "application/json",

322 + Payload: payload,

323 + Args: collectorTestArgs(mgr, tc.args...),

324 + })

325 +

326 + cfg, err := cb.ParseAndValidate(fn, "validated")

327 + if tc.wantErr != "" {

328 + require.Error(t, err)

329 + assert.Contains(t, err.Error(), tc.wantErr)

330 + return

331 + }

332 +

333 + require.NoError(t, err)

334 + assert.Equal(t, tc.wantModule, cfg.Module())

335 + assert.Equal(t, tc.wantName, cfg.Name())

336 + assert.Equal(t, tc.wantProvider, cfg.Provider())

337 + assert.Equal(t, tc.wantSourceTyp, cfg.SourceType())

338 + })

339 + }

340 +}

341 +

342 +func TestCollectorCallbacks_ParseAndValidate_SuppressesAuditSideEffects(t *testing.T) {

343 + tempDir := t.TempDir()

344 + analyzer := &auditAnalyzerSpy{}

345 + mgr := newCollectorTestManager()

346 + mgr.auditAnalyzer = analyzer

347 + mgr.auditDataDir = tempDir

348 + cb := &collectorCallbacks{mgr: mgr}

349 +

350 + cfg := prepareDyncfgCfg("success", "payload-name").Set("option_str", "one").Set("option_int", 2)

351 + fn := dyncfg.NewFunction(functions.Function{

352 + UID: "validation-audit-side-effects",

353 + ContentType: "application/json",

354 + Payload: mustMarshalCollectorConfigPayload(t, cfg),

355 + Args: collectorTestArgs(mgr, "success", string(dyncfg.CommandAdd), "validated"),

356 + })

357 +

358 + _, err := cb.ParseAndValidate(fn, "validated")

359 + require.NoError(t, err)

360 +

361 + entries, err := os.ReadDir(tempDir)

362 + require.NoError(t, err)

363 + assert.Empty(t, entries)

364 + assert.Empty(t, analyzer.registered)

365 +}

366 +

367 +func TestCollectorCallbacks_Start(t *testing.T) {

368 + tests := map[string]struct {

369 + cfg confgroup.Config

370 + wantErr string

371 + wantCode int

372 + wantRunning bool

373 + wantRetryPending bool

374 + }{

375 + "success starts the job": {

376 + cfg: prepareDyncfgCfg("success", "job"),

377 + wantRunning: true,

378 + },

379 + "autodetection failure schedules retry": {

380 + cfg: prepareDyncfgCfg("retrycheck", "job").Set("autodetection_retry", 1),

381 + wantErr: "job enable failed",

382 + wantRetryPending: true,

383 + },

384 + "invalid config returns coded validation error": {

385 + cfg: prepareDyncfgCfg("missing", "job"),

386 + wantErr: "invalid configuration",

387 + wantCode: 400,

388 + },

389 + }

390 +

391 + for name, tc := range tests {

392 + t.Run(name, func(t *testing.T) {

393 + mgr := newCollectorTestManager()

394 + cb := &collectorCallbacks{mgr: mgr}

395 +

396 + err := cb.Start(tc.cfg)

397 + if tc.wantErr != "" {

398 + require.Error(t, err)

399 + assert.Contains(t, err.Error(), tc.wantErr)

400 + if tc.wantCode != 0 {

401 + var coded interface{ Code() int }

402 + require.ErrorAs(t, err, &coded)

403 + assert.Equal(t, tc.wantCode, coded.Code())

404 + }

405 + } else {

406 + require.NoError(t, err)

407 + }

408 +

409 + _, running := mgr.runningJobs.lookup(tc.cfg.FullName())

410 + assert.Equal(t, tc.wantRunning, running)

411 +

412 + _, retryPending := mgr.retryingTasks.lookup(tc.cfg)

413 + assert.Equal(t, tc.wantRetryPending, retryPending)

414 +

415 + if tc.wantRunning {

416 + mgr.stopRunningJob(tc.cfg.FullName())

417 + }

418 + if tc.wantRetryPending {

419 + mgr.retryingTasks.remove(tc.cfg)

420 + }

421 + })

422 + }

423 +}

424 +

425 +func TestCollectorCallbacks_Update(t *testing.T) {

426 + tests := map[string]struct {

427 + oldCfg confgroup.Config

428 + newCfg confgroup.Config

429 + wantErr string

430 + wantRunning bool

431 + wantRetryPending bool

432 + }{

433 + "success restarts with the new config and clears old state": {

434 + oldCfg: prepareDyncfgCfg("success", "job"),

435 + newCfg: prepareDyncfgCfg("success", "job").Set("option_str", "changed"),

436 + wantRunning: true,

437 + },

438 + "autodetection failure clears old state and schedules retry": {

439 + oldCfg: prepareDyncfgCfg("retrycheck", "job"),

440 + newCfg: prepareDyncfgCfg("retrycheck", "job").Set("autodetection_retry", 1),

441 + wantErr: "job update failed",

442 + wantRetryPending: true,

443 + },

444 + }

445 +

446 + for name, tc := range tests {

447 + t.Run(name, func(t *testing.T) {

448 + mgr := newCollectorTestManager()

449 + cb := &collectorCallbacks{mgr: mgr}

450 + oldJob := &collectorProbeJob{

451 + fullName: tc.oldCfg.FullName(),

452 + moduleName: tc.oldCfg.Module(),

453 + name: tc.oldCfg.Name(),

454 + }

455 +

456 + mgr.runningJobs.lock()

457 + mgr.runningJobs.add(oldJob.FullName(), oldJob)

458 + mgr.runningJobs.unlock()

459 + mgr.fileStatus.add(tc.oldCfg, dyncfg.StatusRunning.String())

460 +

461 + _, cancel := context.WithCancel(context.Background())

462 + defer cancel()

463 + mgr.retryingTasks.add(tc.oldCfg, &retryTask{cancel: cancel})

464 +

465 + err := cb.Update(tc.oldCfg, tc.newCfg)

466 + if tc.wantErr != "" {

467 + require.Error(t, err)

468 + assert.Contains(t, err.Error(), tc.wantErr)

469 + } else {

470 + require.NoError(t, err)

471 + }

472 +

473 + assert.True(t, oldJob.stopped)

474 +

475 + _, oldRetryPending := mgr.retryingTasks.lookup(tc.oldCfg)

476 + assert.False(t, oldRetryPending)

477 +

478 + _, oldFileStatus := mgr.fileStatus.lookup(tc.oldCfg)

479 + assert.False(t, oldFileStatus)

480 +

481 + _, running := mgr.runningJobs.lookup(tc.newCfg.FullName())

482 + assert.Equal(t, tc.wantRunning, running)

483 +

484 + _, retryPending := mgr.retryingTasks.lookup(tc.newCfg)

485 + assert.Equal(t, tc.wantRetryPending, retryPending)

486 +

487 + if tc.wantRunning {

488 + mgr.stopRunningJob(tc.newCfg.FullName())

489 + }

490 + if tc.wantRetryPending {

491 + mgr.retryingTasks.remove(tc.newCfg)

492 + }

493 + })

494 + }

495 +}

496 +

497 +func TestCollectorCallbacks_Stop(t *testing.T) {

498 + tests := map[string]struct{}{

499 + "stop removes retry task, running job, and file status": {},

500 + }

501 +

502 + for name := range tests {

503 + t.Run(name, func(t *testing.T) {

504 + mgr := newCollectorTestManager()

505 + cb := &collectorCallbacks{mgr: mgr}

506 + cfg := prepareDyncfgCfg("success", "job")

507 + job := &collectorProbeJob{

508 + fullName: cfg.FullName(),

509 + moduleName: cfg.Module(),

510 + name: cfg.Name(),

511 + }

512 +

513 + mgr.runningJobs.lock()

514 + mgr.runningJobs.add(job.FullName(), job)

515 + mgr.runningJobs.unlock()

516 + mgr.fileStatus.add(cfg, dyncfg.StatusRunning.String())

517 +

518 + _, cancel := context.WithCancel(context.Background())

519 + defer cancel()

520 + mgr.retryingTasks.add(cfg, &retryTask{cancel: cancel})

521 +

522 + cb.Stop(cfg)

523 +

524 + assert.True(t, job.stopped)

525 + _, running := mgr.runningJobs.lookup(cfg.FullName())

526 + assert.False(t, running)

527 + _, retryPending := mgr.retryingTasks.lookup(cfg)

528 + assert.False(t, retryPending)

529 + _, fileStatus := mgr.fileStatus.lookup(cfg)

530 + assert.False(t, fileStatus)

531 + })

532 + }

533 +}

534 +

535 +func TestCollectorCallbacks_OnStatusChange(t *testing.T) {

536 + tests := map[string]struct {

537 + cfg confgroup.Config

538 + status dyncfg.Status

539 + wantSeen bool

540 + }{

541 + "running dyncfg config is persisted to file status": {

542 + cfg: prepareDyncfgCfg("success", "job"),

543 + status: dyncfg.StatusRunning,

544 + wantSeen: true,

545 + },

546 + "failed dyncfg config is ignored": {

547 + cfg: prepareDyncfgCfg("success", "job"),

548 + status: dyncfg.StatusFailed,

549 + },

550 + "running non-dyncfg config is ignored": {

551 + cfg: prepareUserCfg("success", "job"),

552 + status: dyncfg.StatusRunning,

553 + },

554 + }

555 +

556 + for name, tc := range tests {

557 + t.Run(name, func(t *testing.T) {

558 + mgr := newCollectorTestManager()

559 + cb := &collectorCallbacks{mgr: mgr}

560 + entry := &dyncfg.Entry[confgroup.Config]{

561 + Cfg: tc.cfg,

562 + Status: tc.status,

563 + }

564 +

565 + cb.OnStatusChange(entry, dyncfg.StatusAccepted, dyncfg.NewFunction(functions.Function{}))

566 +

567 + _, ok := mgr.fileStatus.lookup(tc.cfg)

568 + assert.Equal(t, tc.wantSeen, ok)

569 + })

570 + }

571 +}

572 +

573 +func TestRunDyncfgCmdTest_CleanupIsDeferred(t *testing.T) {

574 + tests := map[string]struct {

575 + initErr string

576 + checkErr string

577 + wantStatus float64

578 + }{

579 + "cleanup runs after init failure": {

580 + initErr: "init failed",

581 + wantStatus: 422,

582 + },

583 + "cleanup runs after check failure": {

584 + checkErr: "check failed",

585 + wantStatus: 422,

586 + },

587 + "cleanup runs after success": {

588 + wantStatus: 200,

589 + },

590 + }

591 +

592 + for name, tc := range tests {

593 + t.Run(name, func(t *testing.T) {

594 + var buf bytes.Buffer

595 + mgr := newCollectorTestManager()

596 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

597 +

598 + module := &collectorapi.MockCollectorV1{}

599 + if tc.initErr != "" {

600 + module.InitFunc = func(context.Context) error { return errors.New(tc.initErr) }

601 + }

602 + if tc.checkErr != "" {

603 + module.CheckFunc = func(context.Context) error { return errors.New(tc.checkErr) }

604 + }

605 +

606 + task := dyncfgCmdTestTask{

607 + fn: dyncfg.NewFunction(functions.Function{

608 + UID: name,

609 + }),

610 + moduleName: "success",

611 + creator: collectorapi.Creator{

612 + Create: func() collectorapi.CollectorV1 {

613 + return module

614 + },

615 + },

616 + cfg: prepareDyncfgCfg("success", "job"),

617 + timeout: time.Second,

618 + }

619 +

620 + mgr.cmdTestSem <- struct{}{}

621 + mgr.runDyncfgCmdTest(task)

622 +

623 + assert.True(t, module.CleanupDone)

624 +

625 + var resp map[string]any

626 + mustDecodeFunctionPayload(t, buf.String(), name, &resp)

627 + assert.Equal(t, tc.wantStatus, resp["status"])

628 + })

629 + }

630 +}

631 +

632 +func TestRunDyncfgCmdTest_ApplyConfigUsesRequestTimeout(t *testing.T) {

633 + tests := map[string]struct {

634 + timeout time.Duration

635 + }{

636 + "secret resolution sees request deadline": {

637 + timeout: 20 * time.Millisecond,

638 + },

639 + }

640 +

641 + for name, tc := range tests {

642 + t.Run(name, func(t *testing.T) {

643 + var buf bytes.Buffer

644 + blockingSvc := &blockingSecretStoreService{}

645 + mgr := newCollectorTestManagerWithService(blockingSvc)

646 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

647 +

648 + task := dyncfgCmdTestTask{

649 + fn: dyncfg.NewFunction(functions.Function{

650 + UID: name,

651 + }),

652 + moduleName: "success",

653 + creator: collectorapi.Creator{

654 + Create: func() collectorapi.CollectorV1 {

655 + return &collectorapi.MockCollectorV1{}

656 + },

657 + },

658 + cfg: prepareDyncfgCfg("success", "job").

659 + Set("password", "${store:vault:vault_prod:secret/data/mysql#password}"),

660 + timeout: tc.timeout,

661 + }

662 +

663 + mgr.cmdTestSem <- struct{}{}

664 + start := time.Now()

665 + mgr.runDyncfgCmdTest(task)

666 + elapsed := time.Since(start)

667 +

668 + var resp map[string]any

669 + mustDecodeFunctionPayload(t, buf.String(), name, &resp)

670 + assert.Equal(t, float64(400), resp["status"])

671 + assert.Contains(t, resp["errorMessage"], context.DeadlineExceeded.Error())

672 + assert.True(t, blockingSvc.sawDeadline)

673 + assert.Less(t, elapsed, tc.timeout+250*time.Millisecond)

674 + })

675 + }

676 +}

677 +

678 +func TestDyncfgCmdTest_ShutdownBeforeWorker_Returns503(t *testing.T) {

679 + tests := map[string]struct{}{

680 + "shutdown manager returns 503 before scheduling worker": {},

681 + }

682 +

683 + for name := range tests {

684 + t.Run(name, func(t *testing.T) {

685 + var buf bytes.Buffer

686 + mgr := newCollectorTestManager()

687 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

688 +

689 + ctx, cancel := context.WithCancel(context.Background())

690 + cancel()

691 + mgr.ctx = ctx

692 +

693 + fn := dyncfg.NewFunction(functions.Function{

694 + UID: name,

695 + ContentType: "application/json",

696 + Payload: mustMarshalCollectorConfigPayload(t, prepareDyncfgCfg("success", "job")),

697 + Args: []string{mgr.dyncfgModID("success"), string(dyncfg.CommandTest), "job"},

698 + })

699 +

700 + mgr.dyncfgCmdTest(fn)

701 +

702 + var resp map[string]any

703 + mustDecodeFunctionPayload(t, buf.String(), name, &resp)

704 + assert.Equal(t, float64(503), resp["status"])

705 + })

706 + }

707 +}

708 +

709 +func TestDyncfgCmdTest_MissingPayload_Returns400(t *testing.T) {

710 + tests := map[string]struct{}{

711 + "missing payload is rejected before config parsing": {},

712 + }

713 +

714 + for name := range tests {

715 + t.Run(name, func(t *testing.T) {

716 + var buf bytes.Buffer

717 + mgr := newCollectorTestManager()

718 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

719 +

720 + fn := dyncfg.NewFunction(functions.Function{

721 + UID: name,

722 + Args: []string{mgr.dyncfgModID("success"), string(dyncfg.CommandTest), "job"},

723 + })

724 +

725 + mgr.dyncfgCmdTest(fn)

726 +

727 + var resp map[string]any

728 + mustDecodeFunctionPayload(t, buf.String(), name, &resp)

729 + assert.Equal(t, float64(400), resp["status"])

730 + assert.Contains(t, resp["errorMessage"], "Missing configuration payload.")

731 + })

732 + }

733 +}

734 +

735 +func newCollectorTestManager() *Manager {

736 + return newCollectorTestManagerWithService(nil)

737 +}

738 +

739 +func newCollectorTestManagerWithService(secretStoreSvc secretstore.Service) *Manager {

740 + mgr := New(Config{

741 + PluginName: testPluginName,

742 + SecretStoreService: secretStoreSvc,

743 + })

744 + mgr.ctx = context.Background()

745 + mgr.modules = prepareMockRegistry()

746 + mgr.modules.Register("retrycheck", collectorapi.Creator{

747 + Create: func() collectorapi.CollectorV1 {

748 + return &collectorapi.MockCollectorV1{

749 + CheckFunc: func(context.Context) error { return errors.New("mock failed check") },

750 + }

751 + },

752 + })

753 + mgr.fileStatus = newFileStatus()

754 + return mgr

755 +}

756 +

757 +func newCollectorTestHandler(mgr *Manager, cb dyncfg.Callbacks[confgroup.Config]) *dyncfg.Handler[confgroup.Config] {

758 + return dyncfg.NewHandler(dyncfg.HandlerOpts[confgroup.Config]{

759 + Logger: mgr.Logger,

760 + API: mgr.dyncfgResponder,

761 + Seen: mgr.collectorSeen,

762 + Exposed: mgr.collectorExposed,

763 + Callbacks: cb,

764 + WaitKey: func(cfg confgroup.Config) string {

765 + return cfg.FullName()

766 + },

767 + WaitTimeout: waitDecisionTimeout,

768 + Path: "/collectors/test/Jobs",

769 + EnableFailCode: 200,

770 + RemoveStockOnEnableFail: true,

771 + JobCommands: []dyncfg.Command{

772 + dyncfg.CommandSchema,

773 + dyncfg.CommandGet,

774 + dyncfg.CommandEnable,

775 + dyncfg.CommandDisable,

776 + dyncfg.CommandUpdate,

777 + dyncfg.CommandRestart,

778 + dyncfg.CommandTest,

779 + dyncfg.CommandUserconfig,

780 + },

781 + })

782 +}

783 +

784 +func seedCollectorEntry(mgr *Manager, cfg confgroup.Config, status dyncfg.Status) {

785 + mgr.collectorSeen.Add(cfg)

786 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

787 + Cfg: cfg,

788 + Status: status,

789 + })

790 +}

791 +

792 +func collectorTestArgs(mgr *Manager, args ...string) []string {

793 + out := make([]string, len(args))

794 + copy(out, args)

795 + if len(out) == 0 {

796 + return out

797 + }

798 +

799 + switch {

800 + case strings.Contains(out[0], ":"):

801 + out[0] = mgr.dyncfgCollectorPrefixValue() + out[0]

802 + case out[0] != "":

803 + out[0] = mgr.dyncfgModID(out[0])

804 + }

805 +

806 + return out

807 +}

808 +

809 +func mustMarshalCollectorConfigPayload(t *testing.T, cfg confgroup.Config) []byte {

810 + t.Helper()

811 +

812 + payload, err := json.Marshal(cfg)

813 + require.NoError(t, err)

814 + return payload

815 +}

816 +

817 +type collectorSeqTestCallbacks struct {

818 + mgr *Manager

819 + parsed map[dyncfg.Command]confgroup.Config

820 +}

821 +

822 +func (cb *collectorSeqTestCallbacks) ExtractKey(fn dyncfg.Function) (key, name string, ok bool) {

823 + return cb.mgr.collectorCallbacks.ExtractKey(fn)

824 +}

825 +

826 +func (cb *collectorSeqTestCallbacks) ParseAndValidate(fn dyncfg.Function, _ string) (confgroup.Config, error) {

827 + cfg, ok := cb.parsed[fn.Command()]

828 + if !ok {

829 + return nil, errors.New("unexpected parse request")

830 + }

831 + return cfg, nil

832 +}

833 +

834 +func (cb *collectorSeqTestCallbacks) Start(confgroup.Config) error { return nil }

835 +

836 +func (cb *collectorSeqTestCallbacks) Update(_, _ confgroup.Config) error { return nil }

837 +

838 +func (cb *collectorSeqTestCallbacks) Stop(confgroup.Config) {}

839 +

840 +func (cb *collectorSeqTestCallbacks) OnStatusChange(*dyncfg.Entry[confgroup.Config], dyncfg.Status, dyncfg.Function) {

841 +}

842 +

843 +func (cb *collectorSeqTestCallbacks) ConfigID(cfg confgroup.Config) string {

844 + return cb.mgr.dyncfgJobID(cfg)

845 +}

846 +

847 +type collectorProbeJob struct {

848 + fullName string

849 + moduleName string

850 + name string

851 + stopped bool

852 +}

853 +

854 +func (j *collectorProbeJob) FullName() string { return j.fullName }

855 +func (j *collectorProbeJob) ModuleName() string { return j.moduleName }

856 +func (j *collectorProbeJob) Name() string { return j.name }

857 +func (j *collectorProbeJob) Collector() any { return nil }

858 +func (j *collectorProbeJob) Start() {}

859 +func (j *collectorProbeJob) Stop() { j.stopped = true }

860 +func (j *collectorProbeJob) Tick(int) {}

861 +func (j *collectorProbeJob) AutoDetection() error {

862 + return nil

863 +}

864 +func (j *collectorProbeJob) AutoDetectionEvery() int { return 0 }

865 +func (j *collectorProbeJob) RetryAutoDetection() bool {

866 + return false

867 +}

868 +

869 +type blockingSecretStoreService struct {

870 + sawDeadline bool

871 +}

872 +

873 +func (s *blockingSecretStoreService) Capture() *secretstore.Snapshot { return nil }

874 +

875 +func (s *blockingSecretStoreService) Resolve(ctx context.Context, _ *secretstore.Snapshot, _, _ string) (string, error) {

876 + _, s.sawDeadline = ctx.Deadline()

877 + if !s.sawDeadline {

878 + return "", errors.New("missing request deadline")

879 + }

880 + <-ctx.Done()

881 + return "", ctx.Err()

882 +}

883 +

884 +func (*blockingSecretStoreService) Kinds() []secretstore.StoreKind { return nil }

885 +

886 +func (*blockingSecretStoreService) DisplayName(secretstore.StoreKind) (string, bool) {

887 + return "", false

888 +}

889 +

890 +func (*blockingSecretStoreService) Schema(secretstore.StoreKind) (string, bool) { return "", false }

891 +

892 +func (*blockingSecretStoreService) New(secretstore.StoreKind) (secretstore.Store, bool) {

893 + return nil, false

894 +}

895 +

896 +func (*blockingSecretStoreService) GetStatus(string) (secretstore.StoreStatus, bool) {

897 + return secretstore.StoreStatus{}, false

898 +}

899 +

900 +func (*blockingSecretStoreService) Validate(secretstore.Config) error { return nil }

901 +

902 +func (*blockingSecretStoreService) ValidateStored(string) error { return nil }

903 +

904 +func (*blockingSecretStoreService) Add(secretstore.Config) error { return nil }

905 +

906 +func (*blockingSecretStoreService) Update(string, secretstore.Config) error { return nil }

907 +

908 +func (*blockingSecretStoreService) Remove(string) error { return nil }

909 +func (j *collectorProbeJob) Cleanup() {}

910 +func (j *collectorProbeJob) IsRunning() bool { return true }

911 +func (j *collectorProbeJob) Panicked() bool { return false }

912 +func (j *collectorProbeJob) Vnode() vnodes.VirtualNode { return vnodes.VirtualNode{} }

913 +func (j *collectorProbeJob) UpdateVnode(*vnodes.VirtualNode) {}

914 +

915 +type auditAnalyzerSpy struct {

916 + registered []string

917 +}

918 +

919 +func (a *auditAnalyzerSpy) RegisterJob(jobName, moduleName, dir string) {

920 + a.registered = append(a.registered, moduleName+":"+jobName+":"+dir)

921 +}

922 +

923 +func (*auditAnalyzerSpy) RecordJobStructure(string, string, *collectorapi.Charts) {}

924 +func (*auditAnalyzerSpy) UpdateJobStructure(string, string, *collectorapi.Charts) {}

925 +func (*auditAnalyzerSpy) RecordCollection(string, string, map[string]int64) {}

src/go/plugin/agent/jobmgr/dyncfg_handoff.go

+3 -3

@@ -22,12 +22,12 @@ func (m *Manager) enqueueDyncfgFunction(fn dyncfg.Function) {

22 return

23 case dyncfg.BoundedSendContextDone:

24 if m.baseContext().Err() != nil {

25 - m.dyncfgApi.SendCodef(fn, 503, dyncfgShuttingDownMsg)

25 + m.dyncfgResponder.SendCodef(fn, 503, dyncfgShuttingDownMsg)

26 return

27 }

28 - m.dyncfgApi.SendCodef(fn, 503, dyncfgBusyMsg)

28 + m.dyncfgResponder.SendCodef(fn, 503, dyncfgBusyMsg)

29 case dyncfg.BoundedSendTimeout:

30 - m.dyncfgApi.SendCodef(fn, 503, dyncfgBusyMsg)

30 + m.dyncfgResponder.SendCodef(fn, 503, dyncfgBusyMsg)

31 }

32 }

src/go/plugin/agent/jobmgr/dyncfg_secretstore.go new

+64

@@ -0,0 +1,64 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "fmt"

7 + "strings"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

12 +)

13 +

14 +func (m *Manager) dyncfgSecretStorePrefixValue() string {

15 + return m.secretsCtl.Prefix()

16 +}

17 +

18 +func (m *Manager) dyncfgSecretStoreExec(fn dyncfg.Function) {

19 + if fn.Command() == dyncfg.CommandSchema {

20 + m.dyncfgSecretStoreSeqExec(fn)

21 + return

22 + }

23 + m.enqueueDyncfgFunction(fn)

24 +}

25 +

26 +func (m *Manager) dyncfgSecretStoreSeqExec(fn dyncfg.Function) {

27 + m.secretsCtl.SeqExec(fn)

28 +}

29 +

30 +func (m *Manager) restartDependentCollectorJob(fullName string) error {

31 + entry, ok := m.lookupExposedByFullName(fullName)

32 + if !ok {

33 + return fmt.Errorf("job '%s' is not exposed", fullName)

34 + }

35 +

36 + oldStatus := entry.Status

37 + switch oldStatus {

38 + case dyncfg.StatusRunning, dyncfg.StatusFailed:

39 + default:

40 + return fmt.Errorf("job '%s' restart is not allowed in '%s' state", fullName, oldStatus)

41 + }

42 +

43 + m.collectorCallbacks.Stop(entry.Cfg)

44 +

45 + if err := m.collectorCallbacks.Start(entry.Cfg); err != nil {

46 + entry.Status = dyncfg.StatusFailed

47 + m.collectorHandler.NotifyJobStatus(entry.Cfg, dyncfg.StatusFailed)

48 + m.collectorCallbacks.OnStatusChange(entry, oldStatus, dyncfg.NewFunction(functions.Function{}))

49 + return fmt.Errorf("job '%s' restart failed: %w", fullName, err)

50 + }

51 +

52 + entry.Status = dyncfg.StatusRunning

53 + m.collectorHandler.NotifyJobStatus(entry.Cfg, dyncfg.StatusRunning)

54 + m.collectorCallbacks.OnStatusChange(entry, oldStatus, dyncfg.NewFunction(functions.Function{}))

55 + return nil

56 +}

57 +

58 +func (m *Manager) lookupExposedByFullName(fullName string) (*dyncfg.Entry[confgroup.Config], bool) {

59 + fullName = strings.TrimSpace(fullName)

60 + if fullName == "" {

61 + return nil, false

62 + }

63 + return m.collectorExposed.LookupByKey(fullName)

64 +}

src/go/plugin/agent/jobmgr/dyncfg_secretstore_cache.go new

+78

@@ -0,0 +1,78 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "fmt"

7 + "strings"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 +)

12 +

13 +func (m *Manager) affectedJobs(key string) []secretstore.JobRef {

14 + if m == nil || m.secretStoreDeps == nil {

15 + return nil

16 + }

17 +

18 + exposed, _ := m.secretStoreDeps.Impacted(key)

19 + return exposed

20 +}

21 +

22 +func (m *Manager) restartableAffectedJobs(key string) []secretstore.JobRef {

23 + if m == nil || m.secretStoreDeps == nil {

24 + return nil

25 + }

26 +

27 + exposed, _ := m.secretStoreDeps.Impacted(key)

28 + refs := make([]secretstore.JobRef, 0, len(exposed))

29 + for _, job := range exposed {

30 + entry, ok := m.lookupExposedByFullName(job.ID)

31 + if !ok {

32 + continue

33 + }

34 + switch entry.Status {

35 + case dyncfg.StatusRunning, dyncfg.StatusFailed:

36 + refs = append(refs, job)

37 + }

38 + }

39 + return refs

40 +}

41 +

42 +type secretStoreRestartFailure struct {

43 + ref secretstore.JobRef

44 + err error

45 +}

46 +

47 +func (m *Manager) restartDependentJobs(key string) string {

48 + failures := m.restartDependentJobsBestEffort(key)

49 + if len(failures) == 0 {

50 + return ""

51 + }

52 +

53 + parts := make([]string, 0, len(failures))

54 + for _, failure := range failures {

55 + name := failure.ref.Display

56 + if name == "" {

57 + name = failure.ref.ID

58 + }

59 + parts = append(parts, fmt.Sprintf("%s (%v)", name, failure.err))

60 + }

61 +

62 + return fmt.Sprintf("Secretstore change applied, but dependent collector restarts failed: %s.", strings.Join(parts, "; "))

63 +}

64 +

65 +func (m *Manager) restartDependentJobsBestEffort(key string) []secretStoreRestartFailure {

66 + if m == nil {

67 + return nil

68 + }

69 +

70 + var failures []secretStoreRestartFailure

71 + for _, job := range m.restartableAffectedJobs(key) {

72 + if err := m.restartDependentCollectorJob(job.ID); err != nil {

73 + m.Warningf("dyncfg: secretstore: failed to restart dependent job '%s' after store '%s' change: %v", job.ID, key, err)

74 + failures = append(failures, secretStoreRestartFailure{ref: job, err: err})

75 + }

76 + }

77 + return failures

78 +}

src/go/plugin/agent/jobmgr/dyncfg_secretstore_test.go new

+811

@@ -0,0 +1,811 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "bytes"

7 + "context"

8 + "encoding/json"

9 + "regexp"

10 + "testing"

11 +

12 + "github.com/stretchr/testify/assert"

13 + "github.com/stretchr/testify/require"

14 + "gopkg.in/yaml.v2"

15 +

16 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

17 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

18 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

19 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

20 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

21 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

22 +)

23 +

24 +func TestDyncfgSecretStoreSeqExec(t *testing.T) {

25 + tests := map[string]struct {

26 + run func(t *testing.T, mgr *Manager, out *bytes.Buffer)

27 + }{

28 + "add and get": {

29 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

30 + addFn := dyncfg.NewFunction(functions.Function{

31 + UID: "ss-add",

32 + ContentType: "application/json",

33 + Payload: mustJSON(t, testVaultConfig()),

34 + Args: []string{

35 + mgr.dyncfgSecretStoreTemplateID(secretstore.KindVault),

36 + string(dyncfg.CommandAdd),

37 + "vault_prod",

38 + },

39 + })

40 + mgr.dyncfgSecretStoreSeqExec(addFn)

41 +

42 + var addResp map[string]any

43 + mustDecodeFunctionPayload(t, out.String(), "ss-add", &addResp)

44 + assert.Equal(t, float64(200), addResp["status"])

45 + assert.Equal(t, "", addResp["message"])

46 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

47 + require.True(t, ok)

48 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

49 + _, ok = mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

50 + assert.True(t, ok)

51 + assert.Contains(t, out.String(), "schema get update test remove")

52 + assert.NotContains(t, out.String(), "enable")

53 + assert.NotContains(t, out.String(), "disable")

54 +

55 + getFn := dyncfg.NewFunction(functions.Function{

56 + UID: "ss-get",

57 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

58 + })

59 + mgr.dyncfgSecretStoreSeqExec(getFn)

60 +

61 + var cfg map[string]any

62 + mustDecodeFunctionPayload(t, out.String(), "ss-get", &cfg)

63 + _, ok = cfg["name"]

64 + assert.False(t, ok)

65 + _, ok = cfg["kind"]

66 + assert.False(t, ok)

67 + assert.Equal(t, "token", cfg["mode"])

68 + },

69 + },

70 + "add and get from yaml payload": {

71 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

72 + addFn := dyncfg.NewFunction(functions.Function{

73 + UID: "ss-add-yaml",

74 + Payload: []byte(`

75 +mode: token

76 +mode_token:

77 + token: vault-token

78 +addr: https://vault.example

79 +`),

80 + Args: []string{

81 + mgr.dyncfgSecretStoreTemplateID(secretstore.KindVault),

82 + string(dyncfg.CommandAdd),

83 + "vault_prod",

84 + },

85 + })

86 + mgr.dyncfgSecretStoreSeqExec(addFn)

87 +

88 + var addResp map[string]any

89 + mustDecodeFunctionPayload(t, out.String(), "ss-add-yaml", &addResp)

90 + assert.Equal(t, float64(200), addResp["status"])

91 +

92 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

93 + require.True(t, ok)

94 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

95 +

96 + getFn := dyncfg.NewFunction(functions.Function{

97 + UID: "ss-get-yaml",

98 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

99 + })

100 + mgr.dyncfgSecretStoreSeqExec(getFn)

101 +

102 + var cfg map[string]any

103 + mustDecodeFunctionPayload(t, out.String(), "ss-get-yaml", &cfg)

104 + _, ok = cfg["name"]

105 + assert.False(t, ok)

106 + assert.Equal(t, "token", cfg["mode"])

107 +

108 + modeToken, ok := cfg["mode_token"].(map[string]any)

109 + require.True(t, ok)

110 + assert.Equal(t, "vault-token", modeToken["token"])

111 + },

112 + },

113 + "add activation failure publishes failed store": {

114 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

115 + addFn := dyncfg.NewFunction(functions.Function{

116 + UID: "ss-add-failed",

117 + ContentType: "application/json",

118 + Payload: mustJSON(t, map[string]any{"mode": "token"}),

119 + Args: []string{

120 + mgr.dyncfgSecretStoreTemplateID(secretstore.KindVault),

121 + string(dyncfg.CommandAdd),

122 + "vault_prod",

123 + },

124 + })

125 + mgr.dyncfgSecretStoreSeqExec(addFn)

126 +

127 + var addResp map[string]any

128 + mustDecodeFunctionPayload(t, out.String(), "ss-add-failed", &addResp)

129 + assert.Equal(t, float64(400), addResp["status"])

130 + assert.Contains(t, addResp["errorMessage"], "mode_token")

131 +

132 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

133 + require.True(t, ok)

134 + assert.Equal(t, dyncfg.StatusFailed, entry.Status)

135 + assert.Equal(t, confgroup.TypeDyncfg, entry.Cfg.SourceType())

136 + _, ok = mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

137 + assert.False(t, ok)

138 + },

139 + },

140 + "duplicate add is rejected": {

141 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

142 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

143 +

144 + cfg := prepareDyncfgCfg("success", "mysql")

145 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

146 + Cfg: cfg,

147 + Status: dyncfg.StatusRunning,

148 + })

149 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

150 + mgr.secretStoreDeps.setRunning(cfg.FullName(), true)

151 +

152 + addFn := dyncfg.NewFunction(functions.Function{

153 + UID: "ss-add-duplicate",

154 + ContentType: "application/json",

155 + Payload: mustJSON(t, testVaultConfigTokenFile()),

156 + Args: []string{

157 + mgr.dyncfgSecretStoreTemplateID(secretstore.KindVault),

158 + string(dyncfg.CommandAdd),

159 + "vault_prod",

160 + },

161 + })

162 + mgr.dyncfgSecretStoreSeqExec(addFn)

163 +

164 + var addResp map[string]any

165 + mustDecodeFunctionPayload(t, out.String(), "ss-add-duplicate", &addResp)

166 + assert.Equal(t, float64(409), addResp["status"])

167 + assert.Contains(t, addResp["errorMessage"], "already exists")

168 + assert.NotContains(t, out.String(), "CONFIG test:collector:success:mysql status running")

169 +

170 + getFn := dyncfg.NewFunction(functions.Function{

171 + UID: "ss-get-after-duplicate",

172 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

173 + })

174 + mgr.dyncfgSecretStoreSeqExec(getFn)

175 +

176 + var got map[string]any

177 + mustDecodeFunctionPayload(t, out.String(), "ss-get-after-duplicate", &got)

178 + assert.Equal(t, "token", got["mode"])

179 +

180 + _, ok := mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

181 + assert.True(t, ok)

182 + },

183 + },

184 + "runtime-affecting update succeeds for running store": {

185 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

186 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

187 +

188 + cfg := prepareDyncfgCfg("success", "mysql")

189 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

190 + Cfg: cfg,

191 + Status: dyncfg.StatusRunning,

192 + })

193 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

194 + mgr.secretStoreDeps.setRunning(cfg.FullName(), true)

195 +

196 + updateFn := dyncfg.NewFunction(functions.Function{

197 + UID: "ss-update",

198 + ContentType: "application/json",

199 + Payload: mustJSON(t, testVaultConfigTokenFile()),

200 + Args: []string{

201 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

202 + string(dyncfg.CommandUpdate),

203 + },

204 + })

205 + mgr.dyncfgSecretStoreSeqExec(updateFn)

206 +

207 + var resp map[string]any

208 + mustDecodeFunctionPayload(t, out.String(), "ss-update", &resp)

209 + assert.Equal(t, float64(200), resp["status"])

210 + assert.Equal(t, "", resp["message"])

211 +

212 + getFn := dyncfg.NewFunction(functions.Function{

213 + UID: "ss-get-updated",

214 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

215 + })

216 + mgr.dyncfgSecretStoreSeqExec(getFn)

217 +

218 + var got map[string]any

219 + mustDecodeFunctionPayload(t, out.String(), "ss-get-updated", &got)

220 + assert.Equal(t, "token_file", got["mode"])

221 + },

222 + },

223 + "unknown-field update is preserved in raw config but hidden from get": {

224 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

225 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

226 +

227 + cfg := prepareDyncfgCfg("success", "mysql")

228 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

229 + Cfg: cfg,

230 + Status: dyncfg.StatusRunning,

231 + })

232 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

233 + mgr.secretStoreDeps.setRunning(cfg.FullName(), true)

234 +

235 + updateCfg := testVaultConfig()

236 + updateCfg["ui_note"] = "updated description"

237 +

238 + updateFn := dyncfg.NewFunction(functions.Function{

239 + UID: "ss-update-metadata",

240 + ContentType: "application/json",

241 + Payload: mustJSON(t, updateCfg),

242 + Args: []string{

243 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

244 + string(dyncfg.CommandUpdate),

245 + },

246 + })

247 + mgr.dyncfgSecretStoreSeqExec(updateFn)

248 +

249 + var resp map[string]any

250 + mustDecodeFunctionPayload(t, out.String(), "ss-update-metadata", &resp)

251 + assert.Equal(t, float64(200), resp["status"])

252 +

253 + getFn := dyncfg.NewFunction(functions.Function{

254 + UID: "ss-get-metadata",

255 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

256 + })

257 + mgr.dyncfgSecretStoreSeqExec(getFn)

258 +

259 + var got map[string]any

260 + mustDecodeFunctionPayload(t, out.String(), "ss-get-metadata", &got)

261 + _, ok := got["ui_note"]

262 + assert.False(t, ok)

263 +

264 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

265 + require.True(t, ok)

266 + assert.Equal(t, "updated description", entry.Cfg["ui_note"])

267 + },

268 + },

269 + "test command reports affected jobs": {

270 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

271 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

272 +

273 + cfg := prepareDyncfgCfg("success", "mysql")

274 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

275 + Cfg: cfg,

276 + Status: dyncfg.StatusRunning,

277 + })

278 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

279 + testFn := dyncfg.NewFunction(functions.Function{

280 + UID: "ss-test-affected",

281 + ContentType: "application/json",

282 + Payload: mustJSON(t, testVaultConfigTokenFile()),

283 + Args: []string{

284 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

285 + string(dyncfg.CommandTest),

286 + },

287 + })

288 + mgr.dyncfgSecretStoreSeqExec(testFn)

289 +

290 + var resp map[string]any

291 + mustDecodeFunctionPayload(t, out.String(), "ss-test-affected", &resp)

292 + assert.Equal(t, float64(202), resp["status"])

293 + assert.Equal(t, "Updated configuration is used by jobs: success:mysql. Running or failed jobs that would be restarted automatically: success:mysql.", resp["message"])

294 + },

295 + },

296 + "test command reports no-op for unchanged payload": {

297 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

298 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

299 +

300 + cfg := prepareDyncfgCfg("success", "mysql")

301 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

302 + Cfg: cfg,

303 + Status: dyncfg.StatusRunning,

304 + })

305 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

306 +

307 + testFn := dyncfg.NewFunction(functions.Function{

308 + UID: "ss-test-noop",

309 + ContentType: "application/json",

310 + Payload: mustJSON(t, testVaultConfig()),

311 + Args: []string{

312 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

313 + string(dyncfg.CommandTest),

314 + },

315 + })

316 + mgr.dyncfgSecretStoreSeqExec(testFn)

317 +

318 + var resp map[string]any

319 + mustDecodeFunctionPayload(t, out.String(), "ss-test-noop", &resp)

320 + assert.Equal(t, float64(202), resp["status"])

321 + assert.Equal(t, "Submitted configuration does not change the active secretstore.", resp["message"])

322 + assert.NotContains(t, out.String(), "CONFIG test:collector:success:mysql status running")

323 + },

324 + },

325 + "test command does not mutate generation": {

326 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

327 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

328 + before := mustSecretStoreService(t, mgr).Capture().Generation()

329 +

330 + testCfg := testVaultConfig()

331 + testCfg["mode_token"].(map[string]any)["extra"] = "ignored"

332 +

333 + testFn := dyncfg.NewFunction(functions.Function{

334 + UID: "ss-test",

335 + ContentType: "application/json",

336 + Payload: mustJSON(t, testCfg),

337 + Args: []string{

338 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

339 + string(dyncfg.CommandTest),

340 + },

341 + })

342 + mgr.dyncfgSecretStoreSeqExec(testFn)

343 +

344 + var resp map[string]any

345 + mustDecodeFunctionPayload(t, out.String(), "ss-test", &resp)

346 + assert.Equal(t, float64(202), resp["status"])

347 + assert.Equal(t, before, mustSecretStoreService(t, mgr).Capture().Generation())

348 + },

349 + },

350 + "test command with empty payload validates stored config": {

351 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

352 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

353 + before := mustSecretStoreService(t, mgr).Capture().Generation()

354 +

355 + testFn := dyncfg.NewFunction(functions.Function{

356 + UID: "ss-test-empty",

357 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandTest)},

358 + })

359 + mgr.dyncfgSecretStoreSeqExec(testFn)

360 +

361 + var resp map[string]any

362 + mustDecodeFunctionPayload(t, out.String(), "ss-test-empty", &resp)

363 + assert.Equal(t, float64(202), resp["status"])

364 + assert.Equal(t, "Stored configuration is valid. No jobs are currently using this secretstore.", resp["message"])

365 +

366 + status, ok := mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

367 + require.True(t, ok)

368 + require.NotNil(t, status.LastValidation)

369 + assert.True(t, status.LastValidation.OK)

370 + assert.Equal(t, before, mustSecretStoreService(t, mgr).Capture().Generation())

371 + },

372 + },

373 + "test command with empty payload reports affected jobs": {

374 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

375 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

376 +

377 + cfg := prepareDyncfgCfg("success", "mysql")

378 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

379 + Cfg: cfg,

380 + Status: dyncfg.StatusRunning,

381 + })

382 + mgr.secretStoreDeps.SetActiveJobStores(cfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

383 +

384 + testFn := dyncfg.NewFunction(functions.Function{

385 + UID: "ss-test-empty-affected",

386 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandTest)},

387 + })

388 + mgr.dyncfgSecretStoreSeqExec(testFn)

389 +

390 + var resp map[string]any

391 + mustDecodeFunctionPayload(t, out.String(), "ss-test-empty-affected", &resp)

392 + assert.Equal(t, float64(202), resp["status"])

393 + assert.Equal(t, "Stored configuration is valid. This secretstore is used by jobs: success:mysql. Running or failed jobs that would be restarted automatically by a change: success:mysql.", resp["message"])

394 + },

395 + },

396 + "test command reports all dependent jobs and restartable subset": {

397 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

398 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

399 +

400 + runningCfg := prepareDyncfgCfg("success", "mysql")

401 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

402 + Cfg: runningCfg,

403 + Status: dyncfg.StatusRunning,

404 + })

405 + mgr.secretStoreDeps.SetActiveJobStores(runningCfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

406 +

407 + acceptedCfg := prepareDyncfgCfg("success", "nginx")

408 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

409 + Cfg: acceptedCfg,

410 + Status: dyncfg.StatusAccepted,

411 + })

412 + mgr.secretStoreDeps.SetActiveJobStores(acceptedCfg.FullName(), "success:nginx", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

413 +

414 + testFn := dyncfg.NewFunction(functions.Function{

415 + UID: "ss-test-all-deps",

416 + ContentType: "application/json",

417 + Payload: mustJSON(t, testVaultConfigTokenFile()),

418 + Args: []string{

419 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

420 + string(dyncfg.CommandTest),

421 + },

422 + })

423 + mgr.dyncfgSecretStoreSeqExec(testFn)

424 +

425 + var resp map[string]any

426 + mustDecodeFunctionPayload(t, out.String(), "ss-test-all-deps", &resp)

427 + assert.Equal(t, float64(202), resp["status"])

428 + assert.Equal(t, "Updated configuration is used by jobs: success:mysql, success:nginx. Running or failed jobs that would be restarted automatically: success:mysql.", resp["message"])

429 + },

430 + },

431 + "test command reports no affected jobs for changed payload": {

432 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

433 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

434 +

435 + testFn := dyncfg.NewFunction(functions.Function{

436 + UID: "ss-test-no-affected",

437 + ContentType: "application/json",

438 + Payload: mustJSON(t, testVaultConfigTokenFile()),

439 + Args: []string{

440 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

441 + string(dyncfg.CommandTest),

442 + },

443 + })

444 + mgr.dyncfgSecretStoreSeqExec(testFn)

445 +

446 + var resp map[string]any

447 + mustDecodeFunctionPayload(t, out.String(), "ss-test-no-affected", &resp)

448 + assert.Equal(t, float64(202), resp["status"])

449 + assert.Equal(t, "No jobs currently use this secretstore.", resp["message"])

450 + },

451 + },

452 + "enable is unsupported": {

453 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

454 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

455 +

456 + enableFn := dyncfg.NewFunction(functions.Function{

457 + UID: "ss-enable-unsupported",

458 + Args: []string{

459 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

460 + string(dyncfg.CommandEnable),

461 + },

462 + })

463 + mgr.dyncfgSecretStoreSeqExec(enableFn)

464 +

465 + var resp map[string]any

466 + mustDecodeFunctionPayload(t, out.String(), "ss-enable-unsupported", &resp)

467 + assert.Equal(t, float64(501), resp["status"])

468 +

469 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

470 + require.True(t, ok)

471 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

472 + },

473 + },

474 + "disable is unsupported": {

475 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

476 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

477 +

478 + disableFn := dyncfg.NewFunction(functions.Function{

479 + UID: "ss-disable-unsupported",

480 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandDisable)},

481 + })

482 + mgr.dyncfgSecretStoreSeqExec(disableFn)

483 +

484 + var resp map[string]any

485 + mustDecodeFunctionPayload(t, out.String(), "ss-disable-unsupported", &resp)

486 + assert.Equal(t, float64(501), resp["status"])

487 +

488 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

489 + require.True(t, ok)

490 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

491 + },

492 + },

493 + "remove deletes store": {

494 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

495 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

496 +

497 + removeFn := dyncfg.NewFunction(functions.Function{

498 + UID: "ss-remove",

499 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandRemove)},

500 + })

501 + mgr.dyncfgSecretStoreSeqExec(removeFn)

502 +

503 + var resp map[string]any

504 + mustDecodeFunctionPayload(t, out.String(), "ss-remove", &resp)

505 + assert.Equal(t, float64(200), resp["status"])

506 + _, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

507 + assert.False(t, ok)

508 + _, ok = mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

509 + assert.False(t, ok)

510 + },

511 + },

512 + "remove blocks when dependent jobs use the store": {

513 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

514 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusRunning)

515 +

516 + runningCfg := prepareDyncfgCfg("success", "mysql")

517 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

518 + Cfg: runningCfg,

519 + Status: dyncfg.StatusRunning,

520 + })

521 + mgr.secretStoreDeps.SetActiveJobStores(runningCfg.FullName(), "success:mysql", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

522 +

523 + disabledCfg := prepareDyncfgCfg("success", "nginx")

524 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

525 + Cfg: disabledCfg,

526 + Status: dyncfg.StatusDisabled,

527 + })

528 + mgr.secretStoreDeps.SetActiveJobStores(disabledCfg.FullName(), "success:nginx", []string{secretstore.StoreKey(secretstore.KindVault, "vault_prod")})

529 +

530 + removeFn := dyncfg.NewFunction(functions.Function{

531 + UID: "ss-remove-blocked",

532 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandRemove)},

533 + })

534 + mgr.dyncfgSecretStoreSeqExec(removeFn)

535 +

536 + var resp map[string]any

537 + mustDecodeFunctionPayload(t, out.String(), "ss-remove-blocked", &resp)

538 + assert.Equal(t, float64(409), resp["status"])

539 + assert.Equal(t, "The specified secretstore 'vault:vault_prod' is used by jobs (success:mysql, success:nginx).", resp["errorMessage"])

540 +

541 + _, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

542 + assert.True(t, ok)

543 + _, ok = mustSecretStoreService(t, mgr).GetStatus(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

544 + assert.True(t, ok)

545 + },

546 + },

547 + "userconfig returns yaml from payload": {

548 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

549 + userconfigFn := dyncfg.NewFunction(functions.Function{

550 + UID: "ss-userconfig",

551 + ContentType: "application/json",

552 + Payload: mustJSON(t, testVaultConfig()),

553 + Args: []string{

554 + mgr.dyncfgSecretStoreTemplateID(secretstore.KindVault),

555 + string(dyncfg.CommandUserconfig),

556 + },

557 + })

558 + mgr.dyncfgSecretStoreSeqExec(userconfigFn)

559 +

560 + re := regexp.MustCompile(`(?s)FUNCTION_RESULT_BEGIN ss-userconfig [^\n]+\n(.*?)\nFUNCTION_RESULT_END`)

561 + match := re.FindStringSubmatch(out.String())

562 + require.Len(t, match, 2)

563 + var parsed map[string]any

564 + require.NoError(t, yaml.Unmarshal([]byte(match[1]), &parsed))

565 + _, ok := parsed["name"]

566 + assert.False(t, ok)

567 + _, ok = parsed["kind"]

568 + assert.False(t, ok)

569 + },

570 + },

571 + "test rejects wrapped config payload": {

572 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

573 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", testVaultConfig(), dyncfg.StatusAccepted)

574 +

575 + testFn := dyncfg.NewFunction(functions.Function{

576 + UID: "ss-test-wrapped-config-payload",

577 + ContentType: "application/json",

578 + Payload: mustJSON(t, map[string]any{

579 + "config": testVaultConfigTokenFile(),

580 + }),

581 + Args: []string{

582 + mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")),

583 + string(dyncfg.CommandTest),

584 + },

585 + })

586 + mgr.dyncfgSecretStoreSeqExec(testFn)

587 +

588 + var resp map[string]any

589 + mustDecodeFunctionPayload(t, out.String(), "ss-test-wrapped-config-payload", &resp)

590 + assert.Equal(t, float64(400), resp["status"])

591 + assert.Contains(t, resp["errorMessage"], "mode")

592 + },

593 + },

594 + }

595 +

596 + for name, tc := range tests {

597 + t.Run(name, func(t *testing.T) {

598 + mgr, out := newDyncfgSecretStoreTestManager()

599 + tc.run(t, mgr, out)

600 + })

601 + }

602 +}

603 +

604 +func TestSecretStoreConfigFromPayload(t *testing.T) {

605 + tests := map[string]struct {

606 + fn functions.Function

607 + name string

608 + kind secretstore.StoreKind

609 + wantErrContains string

610 + assertConfig func(t *testing.T, cfg secretstore.Config)

611 + }{

612 + "json direct config payload": {

613 + fn: functions.Function{

614 + ContentType: "application/json",

615 + Payload: mustJSON(t, testVaultConfig()),

616 + },

617 + name: "vault_prod",

618 + kind: secretstore.KindVault,

619 + assertConfig: func(t *testing.T, cfg secretstore.Config) {

620 + require.NotNil(t, cfg)

621 + assert.Equal(t, "vault_prod", cfg.Name())

622 + assert.Equal(t, secretstore.KindVault, cfg.Kind())

623 + },

624 + },

625 + "yaml direct config payload": {

626 + fn: functions.Function{

627 + Payload: []byte("mode: token\nmode_token:\n token: vault-token\naddr: https://vault.example\n"),

628 + },

629 + name: "vault_prod",

630 + kind: secretstore.KindVault,

631 + assertConfig: func(t *testing.T, cfg secretstore.Config) {

632 + require.NotNil(t, cfg)

633 + assert.Equal(t, "vault_prod", cfg.Name())

634 + assert.Equal(t, secretstore.KindVault, cfg.Kind())

635 + },

636 + },

637 + "missing payload": {

638 + fn: functions.Function{

639 + ContentType: "application/json",

640 + },

641 + name: "vault_prod",

642 + kind: secretstore.KindVault,

643 + wantErrContains: "missing configuration payload",

644 + },

645 + "wrapped config payload becomes invalid raw config": {

646 + fn: functions.Function{

647 + ContentType: "application/json",

648 + Payload: mustJSON(t, map[string]any{

649 + "config": testVaultConfig(),

650 + }),

651 + },

652 + name: "vault_prod",

653 + kind: secretstore.KindVault,

654 + assertConfig: func(t *testing.T, cfg secretstore.Config) {

655 + require.NotNil(t, cfg)

656 + assert.Equal(t, "vault_prod", cfg.Name())

657 + assert.Equal(t, secretstore.KindVault, cfg.Kind())

658 + },

659 + },

660 + }

661 +

662 + for name, tc := range tests {

663 + t.Run(name, func(t *testing.T) {

664 + mgr, _ := newDyncfgSecretStoreTestManager()

665 + cfg, err := mgr.secretStoreConfigFromPayload(dyncfg.NewFunction(tc.fn), tc.name, tc.kind)

666 + if tc.wantErrContains != "" {

667 + require.Error(t, err)

668 + assert.Contains(t, err.Error(), tc.wantErrContains)

669 + return

670 + }

671 +

672 + require.NoError(t, err)

673 + if tc.assertConfig != nil {

674 + tc.assertConfig(t, cfg)

675 + }

676 + })

677 + }

678 +}

679 +

680 +func TestNew_InitializesSecretStoreController(t *testing.T) {

681 + tests := map[string]struct{}{

682 + "new manager initializes secretstore controller and service": {},

683 + }

684 +

685 + for name := range tests {

686 + t.Run(name, func(t *testing.T) {

687 + mgr := New(Config{PluginName: testPluginName})

688 +

689 + require.NotNil(t, mgr.secretsCtl)

690 + require.NotNil(t, mgr.secretsCtl.Service())

691 + assert.Equal(t, mgr.secretsCtl.Prefix(), mgr.dyncfgSecretStorePrefixValue())

692 + })

693 + }

694 +}

695 +

696 +func newDyncfgSecretStoreTestManager() (*Manager, *bytes.Buffer) {

697 + return newDyncfgSecretStoreTestManagerWithService(nil)

698 +}

699 +

700 +func newDyncfgSecretStoreTestManagerWithService(secretStoreSvc secretstore.Service) (*Manager, *bytes.Buffer) {

701 + var out bytes.Buffer

702 +

703 + mgr := New(Config{

704 + PluginName: testPluginName,

705 + SecretStoreService: secretStoreSvc,

706 + })

707 + mgr.ctx = context.Background()

708 + mgr.modules = prepareMockRegistry()

709 + mgr.fileStatus = newFileStatus()

710 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&out))))

711 +

712 + return mgr, &out

713 +}

714 +

715 +func mustSecretStoreService(t *testing.T, mgr *Manager) secretstore.Service {

716 + t.Helper()

717 + require.NotNil(t, mgr.secretsCtl)

718 + svc := mgr.secretsCtl.Service()

719 + require.NotNil(t, svc)

720 + return svc

721 +}

722 +

723 +func testVaultConfig() map[string]any {

724 + return map[string]any{

725 + "mode": "token",

726 + "mode_token": map[string]any{

727 + "token": "vault-token",

728 + },

729 + "addr": "https://vault.example",

730 + }

731 +}

732 +

733 +func testVaultConfigTokenFile() map[string]any {

734 + return map[string]any{

735 + "mode": "token_file",

736 + "mode_token_file": map[string]any{

737 + "path": "/var/lib/netdata/vault.token",

738 + },

739 + "addr": "https://vault.example",

740 + }

741 +}

742 +

743 +func newSecretStoreFromConfig(t *testing.T, svc secretstore.Service, kind secretstore.StoreKind, name string, cfg map[string]any) secretstore.Config {

744 + t.Helper()

745 + _ = svc

746 + return newSecretStoreConfigWithSource(t, kind, name, cfg, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

747 +}

748 +

749 +func newSecretStoreConfigWithSource(t *testing.T, kind secretstore.StoreKind, name string, cfg map[string]any, source, sourceType string) secretstore.Config {

750 + t.Helper()

751 + bs, err := json.Marshal(cfg)

752 + require.NoError(t, err)

753 + var payload map[string]any

754 + require.NoError(t, json.Unmarshal(bs, &payload))

755 + out := secretstore.Config(payload)

756 + out.SetName(name)

757 + out.SetKind(kind)

758 + out.SetSource(source)

759 + out.SetSourceType(sourceType)

760 + return out

761 +}

762 +

763 +func seedSecretStore(t *testing.T, mgr *Manager, kind secretstore.StoreKind, name string, cfg map[string]any, status dyncfg.Status) secretstore.Config {

764 + t.Helper()

765 +

766 + switch status {

767 + case dyncfg.StatusAccepted:

768 + raw := newSecretStoreFromConfig(t, mustSecretStoreService(t, mgr), kind, name, cfg)

769 + entry, changed, err := mgr.rememberSecretStoreConfig(raw)

770 + require.NoError(t, err)

771 + require.True(t, changed)

772 + require.NotNil(t, entry)

773 + return entry.Cfg

774 + case dyncfg.StatusRunning, dyncfg.StatusFailed:

775 + fn := dyncfg.NewFunction(functions.Function{

776 + UID: "seed-" + string(kind) + "-" + name + "-" + status.String(),

777 + ContentType: "application/json",

778 + Payload: mustJSON(t, cfg),

779 + Args: []string{

780 + mgr.dyncfgSecretStoreTemplateID(kind),

781 + string(dyncfg.CommandAdd),

782 + name,

783 + },

784 + })

785 + mgr.dyncfgSecretStoreSeqExec(fn)

786 +

787 + entry, ok := mgr.lookupSecretStoreEntry(secretstore.StoreKey(kind, name))

788 + require.True(t, ok)

789 + require.Equal(t, status, entry.Status)

790 + return entry.Cfg

791 + default:

792 + t.Fatalf("unsupported secretstore seed status %q", status)

793 + return nil

794 + }

795 +}

796 +

797 +func mustJSON(t *testing.T, v any) []byte {

798 + t.Helper()

799 + bs, err := json.Marshal(v)

800 + require.NoError(t, err)

801 + return bs

802 +}

803 +

804 +func mustDecodeFunctionPayload(t *testing.T, output, uid string, dst any) {

805 + t.Helper()

806 +

807 + re := regexp.MustCompile(`(?s)FUNCTION_RESULT_BEGIN ` + regexp.QuoteMeta(uid) + ` [^\n]+\n(.*?)\nFUNCTION_RESULT_END`)

808 + match := re.FindStringSubmatch(output)

809 + require.Len(t, match, 2, "function result for uid '%s' not found in output:\n%s", uid, output)

810 + require.NoError(t, json.Unmarshal([]byte(match[1]), dst))

811 +}

src/go/plugin/agent/jobmgr/dyncfg_secretstore_test_helpers_test.go new

+68

@@ -0,0 +1,68 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "fmt"

7 +

8 + "github.com/netdata/netdata/go/plugins/plugin/agent/jobmgr/secretsctl"

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

12 +)

13 +

14 +func (m *Manager) dyncfgSecretStoreTemplateID(kind secretstore.StoreKind) string {

15 + return fmt.Sprintf("%s%s", m.dyncfgSecretStorePrefixValue(), kind)

16 +}

17 +

18 +func (m *Manager) dyncfgSecretStoreID(id string) string {

19 + return fmt.Sprintf("%s%s", m.dyncfgSecretStorePrefixValue(), id)

20 +}

21 +

22 +func (m *Manager) secretStoreConfigFromPayload(fn dyncfg.Function, name string, kind secretstore.StoreKind) (secretstore.Config, error) {

23 + if err := fn.ValidateHasPayload(); err != nil {

24 + return nil, err

25 + }

26 +

27 + var payload secretstore.Config

28 + if err := fn.UnmarshalPayload(&payload); err != nil {

29 + return nil, fmt.Errorf("invalid configuration format: %w", err)

30 + }

31 + if payload == nil {

32 + payload = secretstore.Config{}

33 + }

34 + payload.SetName(name)

35 + payload.SetKind(kind)

36 + payload.SetSource(confgroup.TypeDyncfg)

37 + payload.SetSourceType(confgroup.TypeDyncfg)

38 + return payload, nil

39 +}

40 +

41 +func (m *Manager) lookupSecretStoreEntry(key string) (*dyncfg.Entry[secretstore.Config], bool) {

42 + entry, ok := m.secretsCtl.Lookup(key)

43 + return dyncfgSecretStoreTestEntry(entry, ok)

44 +}

45 +

46 +func (m *Manager) rememberSecretStoreConfig(cfg secretstore.Config) (*dyncfg.Entry[secretstore.Config], bool, error) {

47 + entry, changed, err := m.secretsCtl.RememberDiscoveredConfig(cfg)

48 + if err != nil || !changed {

49 + return nil, changed, err

50 + }

51 + got, _ := dyncfgSecretStoreTestEntry(entry, true)

52 + return got, true, nil

53 +}

54 +

55 +func (m *Manager) removeSecretStoreConfig(cfg secretstore.Config) (*dyncfg.Entry[secretstore.Config], bool) {

56 + entry, ok := m.secretsCtl.RemoveDiscoveredConfig(cfg)

57 + return dyncfgSecretStoreTestEntry(entry, ok)

58 +}

59 +

60 +func dyncfgSecretStoreTestEntry(entry secretsctl.Entry, ok bool) (*dyncfg.Entry[secretstore.Config], bool) {

61 + if !ok {

62 + return nil, false

63 + }

64 + return &dyncfg.Entry[secretstore.Config]{

65 + Cfg: entry.Cfg,

66 + Status: entry.Status,

67 + }, true

68 +}

src/go/plugin/agent/jobmgr/dyncfg_vnode.go

+15 -363

@@ -3,393 +3,45 @@

3 package jobmgr

5 import (

6 - "encoding/json"

6 "fmt"

8 - "strings"

10 - "github.com/google/uuid"

11 - "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

12 - "gopkg.in/yaml.v2"

13 -

14 - "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

8 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

9 "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

17 -)

18 -

19 -const (

20 - dyncfgVnodeIDf = "%s:vnode"

21 - dyncfgVnodePath = "/collectors/%s/Vnodes"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

11 )

13 func (m *Manager) dyncfgVnodePrefixValue() string {

25 - return fmt.Sprintf(dyncfgVnodeIDf, m.pluginName)

26 -}

27 -

28 -func dyncfgVnodeModCmds() string {

29 - return dyncfg.JoinCommands(

30 - dyncfg.CommandAdd,

31 - dyncfg.CommandSchema,

32 - dyncfg.CommandUserconfig,

33 - dyncfg.CommandTest,

34 - )

35 -}

36 -func dyncfgVnodeJobCmds(isDyncfgJob bool) string {

37 - cmds := []dyncfg.Command{

38 - dyncfg.CommandUserconfig,

39 - dyncfg.CommandSchema,

40 - dyncfg.CommandGet,

41 - dyncfg.CommandUpdate,

42 - dyncfg.CommandTest,

43 - }

44 - if isDyncfgJob {

45 - cmds = append(cmds, dyncfg.CommandRemove)

46 - }

47 - return dyncfg.JoinCommands(cmds...)

48 -}

49 -

50 -func (m *Manager) dyncfgVnodeModuleCreate() {

51 - m.dyncfgApi.ConfigCreate(netdataapi.ConfigOpts{

52 - ID: m.dyncfgVnodePrefixValue(),

53 - Status: dyncfg.StatusAccepted.String(),

54 - ConfigType: dyncfg.ConfigTypeTemplate.String(),

55 - Path: fmt.Sprintf(dyncfgVnodePath, m.pluginName),

56 - SourceType: "internal",

57 - Source: "internal",

58 - SupportedCommands: dyncfgVnodeModCmds(),

59 - })

60 -}

61 -

62 -func (m *Manager) dyncfgVnodeJobCreate(cfg *vnodes.VirtualNode, status dyncfg.Status) {

63 - m.dyncfgApi.ConfigCreate(netdataapi.ConfigOpts{

64 - ID: fmt.Sprintf("%s:%s", m.dyncfgVnodePrefixValue(), cfg.Name),

65 - Status: status.String(),

66 - ConfigType: dyncfg.ConfigTypeJob.String(),

67 - Path: fmt.Sprintf(dyncfgVnodePath, m.pluginName),

68 - SourceType: cfg.SourceType,

69 - Source: cfg.Source,

70 - SupportedCommands: dyncfgVnodeJobCmds(cfg.SourceType == confgroup.TypeDyncfg),

71 - })

14 + return m.vnodesCtl.Prefix()

15 }

17 func (m *Manager) dyncfgVnodeExec(fn dyncfg.Function) {

75 - cmd := fn.Command()

76 -

77 - switch cmd {

78 - case dyncfg.CommandUserconfig:

79 - m.dyncfgVnodeUserconfig(fn)

80 - return

81 - case dyncfg.CommandSchema:

82 - m.dyncfgApi.SendJSON(fn, vnodes.ConfigSchema)

18 + switch fn.Command() {

19 + case dyncfg.CommandSchema, dyncfg.CommandUserconfig:

20 + m.dyncfgVnodeSeqExec(fn)

21 return

22 }

23 m.enqueueDyncfgFunction(fn)

24 }

26 func (m *Manager) dyncfgVnodeSeqExec(fn dyncfg.Function) {

89 - cmd := fn.Command()

90 -

91 - switch cmd {

92 - case dyncfg.CommandTest:

93 - m.dyncfgVnodeTest(fn)

94 - case dyncfg.CommandGet:

95 - m.dyncfgVnodeGet(fn)

96 - case dyncfg.CommandAdd:

97 - m.dyncfgVnodeAdd(fn)

98 - case dyncfg.CommandUpdate:

99 - m.dyncfgVnodeUpdate(fn)

100 - case dyncfg.CommandRemove:

101 - m.dyncfgVnodeRemove(fn)

102 - default:

103 - m.Warningf("dyncfg: function '%s' command '%s' not implemented", fn.Fn().Name, cmd)

104 - m.dyncfgApi.SendCodef(fn, 501, "Function '%s' command '%s' is not implemented.", fn.Fn().Name, cmd)

105 - }

106 -}

107 -

108 -func (m *Manager) dyncfgVnodeGet(fn dyncfg.Function) {

109 - cmd := dyncfg.CommandGet

110 -

111 - id := fn.ID()

112 - name := strings.TrimPrefix(id, m.dyncfgVnodePrefixValue()+":")

113 -

114 - cfg, ok := m.vnodes.Lookup(name)

115 - if !ok {

116 - m.Warningf("dyncfg: %s: vnode %s not found", cmd, name)

117 - m.dyncfgApi.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

118 - return

119 - }

120 -

121 - bs, err := json.Marshal(cfg)

122 - if err != nil {

123 - m.Warningf("dyncfg: %s: vnode job %s failed to json marshal config: %v", cmd, name, err)

124 - m.dyncfgApi.SendCodef(fn, 500, "Failed to convert configuration into JSON: %v.", err)

125 - return

126 - }

127 -

128 - m.dyncfgApi.SendJSON(fn, string(bs))

129 -}

130 -

131 -func (m *Manager) dyncfgVnodeAdd(fn dyncfg.Function) {

132 - cmd := dyncfg.CommandAdd

133 -

134 - if err := fn.ValidateArgs(3); err != nil {

135 - m.Warningf("dyncfg: %s: %v", cmd, err)

136 - m.dyncfgApi.SendCodef(fn, 400, "%v", err)

137 - return

138 - }

139 -

140 - name := fn.JobName()

141 -

142 - if !fn.HasPayload() {

143 - m.Warningf("dyncfg: %s: vnode job %s missing configuration payload.", cmd, name)

144 - m.dyncfgApi.SendCodef(fn, 400, "Missing configuration payload.")

145 - return

146 - }

147 -

148 - cfg, err := vnodeConfigFromPayload(fn)

149 - if err != nil {

150 - m.Warningf("dyncfg: %s: vnode job %s: failed to create config from payload: %v", cmd, name, err)

151 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid configuration format: %v.", err)

152 - return

153 - }

154 -

155 - if err := uuid.Validate(cfg.GUID); err != nil {

156 - m.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", cmd, name, err)

157 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

158 - return

159 - }

160 -

161 - dyncfgUpdateVnodeConfig(cfg, name, fn)

162 -

163 - if err := m.verifyVnodeUnique(cfg); err != nil {

164 - m.Warningf("dyncfg: %s: vnode job %s: %v", cmd, name, err)

165 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload: %v.", err)

166 - return

167 - }

168 -

169 - if orig, ok := m.vnodes.Lookup(name); ok && orig.Equal(cfg) {

170 - m.dyncfgApi.SendCodef(fn, 202, "")

171 - m.dyncfgVnodeJobCreate(cfg, dyncfg.StatusRunning)

172 - return

173 - }

174 -

175 - _, _, err = m.vnodes.Upsert(cfg)

176 - if err != nil {

177 - m.Warningf("dyncfg: %s: vnode job %s: %v", cmd, name, err)

178 - m.dyncfgApi.SendCodef(fn, 400, "Failed to update vnode configuration: %v.", err)

179 - return

180 - }

181 -

182 - for _, job := range m.runningJobs.snapshot() {

183 - if job.Vnode().Name == name {

184 - job.UpdateVnode(cfg)

185 - }

186 - }

187 -

188 - m.dyncfgApi.SendCodef(fn, 202, "")

189 - m.dyncfgVnodeJobCreate(cfg, dyncfg.StatusRunning)

190 -}

191 -

192 -func (m *Manager) dyncfgVnodeRemove(fn dyncfg.Function) {

193 - cmd := dyncfg.CommandRemove

194 -

195 - id := fn.ID()

196 - name := strings.TrimPrefix(id, m.dyncfgVnodePrefixValue()+":")

197 -

198 - vnode, ok := m.vnodes.Lookup(name)

199 - if !ok {

200 - m.Warningf("dyncfg: %s: vnode %s not found", cmd, name)

201 - m.dyncfgApi.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

202 - return

203 - }

204 - if vnode.SourceType != confgroup.TypeDyncfg {

205 - m.Warningf("dyncfg: %s: module vnode %s: can not remove vnode of type %s", cmd, vnode.Name, vnode.SourceType)

206 - m.dyncfgApi.SendCodef(fn, 405, "Removing vnode of type '%s' is not supported. Only 'dyncfg' vnodes can be removed.", vnode.SourceType)

207 - return

208 - }

209 -

210 - if s := m.dyncfgVnodeAffectedJobs(vnode.Name); s != "" {

211 - m.Warningf("dyncfg: %s: vnode %s has running jobs (%s)", cmd, name, s)

212 - m.dyncfgApi.SendCodef(fn, 404, "The specified vnode '%s' has running jobs (%s).", name, s)

213 - return

214 - }

215 -

216 - _, _ = m.vnodes.Remove(name)

217 -

218 - m.dyncfgApi.ConfigDelete(id)

219 - m.dyncfgApi.SendCodef(fn, 200, "")

220 -}

221 -

222 -func (m *Manager) dyncfgVnodeTest(fn dyncfg.Function) {

223 - cmd := dyncfg.CommandTest

224 -

225 - if err := fn.ValidateArgs(3); err != nil {

226 - m.Warningf("dyncfg: %s: %v", cmd, err)

227 - m.dyncfgApi.SendCodef(fn, 400, "%v", err)

228 - return

229 - }

230 -

231 - name := fn.JobName()

232 -

233 - cfg, err := vnodeConfigFromPayload(fn)

234 - if err != nil {

235 - m.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", cmd, err)

236 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

237 - return

238 - }

239 -

240 - if err := uuid.Validate(cfg.GUID); err != nil {

241 - m.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", cmd, name, err)

242 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

243 - return

244 - }

245 -

246 - dyncfgUpdateVnodeConfig(cfg, name, fn)

247 -

248 - if err := m.verifyVnodeUnique(cfg); err != nil {

249 - m.Warningf("dyncfg: %s: vnode job %s: %v", cmd, name, err)

250 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload: %v.", err)

251 - return

252 - }

253 -

254 - if s := m.dyncfgVnodeAffectedJobs(cfg.Name); s != "" {

255 - m.dyncfgApi.SendCodef(fn, 202, "Updated configuration will affect: %s.", s)

256 - } else {

257 - m.dyncfgApi.SendCodef(fn, 202, "No jobs will be affected by this change.")

258 - }

259 -}

260 -

261 -func (m *Manager) dyncfgVnodeUpdate(fn dyncfg.Function) {

262 - cmd := dyncfg.CommandUpdate

263 -

264 - id := fn.ID()

265 - name := strings.TrimPrefix(id, m.dyncfgVnodePrefixValue()+":")

266 -

267 - orig, ok := m.vnodes.Lookup(name)

268 - if !ok {

269 - m.Warningf("dyncfg: %s: vnode %s not found", cmd, name)

270 - m.dyncfgApi.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

271 - return

272 - }

273 -

274 - cfg, err := vnodeConfigFromPayload(fn)

275 - if err != nil {

276 - m.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", cmd, err)

277 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

278 - return

279 - }

280 -

281 - if err := uuid.Validate(cfg.GUID); err != nil {

282 - m.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", cmd, name, err)

283 - m.dyncfgApi.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

284 - return

285 - }

286 -

287 - dyncfgUpdateVnodeConfig(cfg, name, fn)

288 -

289 - if orig.Equal(cfg) {

290 - m.dyncfgApi.SendCodef(fn, 202, "")

291 - return

292 - }

293 -

294 - _, _, err = m.vnodes.Upsert(cfg)

295 - if err != nil {

296 - m.Warningf("dyncfg: %s: vnode job %s: %v", cmd, name, err)

297 - m.dyncfgApi.SendCodef(fn, 400, "Failed to update vnode configuration: %v.", err)

298 - return

299 - }

300 -

301 - for _, job := range m.runningJobs.snapshot() {

302 - if job.Vnode().Name == name {

303 - job.UpdateVnode(cfg)

304 - }

305 - }

306 -

307 - m.dyncfgApi.SendCodef(fn, 202, "")

308 - m.dyncfgVnodeJobCreate(cfg, dyncfg.StatusRunning)

27 + m.vnodesCtl.SeqExec(fn)

28 }

311 -func (m *Manager) dyncfgVnodeUserconfig(fn dyncfg.Function) {

312 - cmd := dyncfg.CommandUserconfig

313 -

314 - bs, err := vnodeUserconfigFromPayload(fn)

315 - if err != nil {

316 - m.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", cmd, err)

317 - m.dyncfgApi.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

318 - return

319 - }

320 -

321 - m.dyncfgApi.SendYAML(fn, string(bs))

322 -}

323 -

324 -func (m *Manager) dyncfgVnodeAffectedJobs(vnode string) string {

325 - var s strings.Builder

326 - m.exposed.ForEach(func(_ string, entry *dyncfg.Entry[confgroup.Config]) bool {

30 +func (m *Manager) affectedVnodeJobs(vnode string) []string {

31 + var jobs []string

32 + m.collectorExposed.ForEach(func(_ string, entry *dyncfg.Entry[confgroup.Config]) bool {

33 if entry.Cfg.Vnode() == vnode {

328 - if s.Len() > 0 {

329 - s.WriteString(", ")

330 - }

331 - s.WriteString(fmt.Sprintf("%s:%s", entry.Cfg.Module(), entry.Cfg.Name()))

34 + jobs = append(jobs, fmt.Sprintf("%s:%s", entry.Cfg.Module(), entry.Cfg.Name()))

35 }

36 return true

37 })

335 - return s.String()

38 + return jobs

39 }

338 -func (m *Manager) verifyVnodeUnique(newCfg *vnodes.VirtualNode) error {

339 - var err error

340 - m.vnodes.ForEach(func(cfg *vnodes.VirtualNode) bool {

341 - if cfg.Name == newCfg.Name {

342 - return true

343 - }

344 - if cfg.Hostname == newCfg.Hostname {

345 - err = fmt.Errorf("duplicate virtual node name detected (job '%s')", cfg.Name)

346 - return false

347 - }

348 - if cfg.GUID == newCfg.GUID {

349 - err = fmt.Errorf("duplicate virtual node guid detected (job '%s')", cfg.Name)

350 - return false

41 +func (m *Manager) applyVnodeUpdate(name string, cfg *vnodes.VirtualNode) {

42 + for _, job := range m.runningJobs.snapshot() {

43 + if job.Vnode().Name == name {

44 + job.UpdateVnode(cfg)

45 }

352 - return true

353 - })

354 - return err

355 -}

356 -

357 -func dyncfgUpdateVnodeConfig(cfg *vnodes.VirtualNode, name string, fn dyncfg.Function) {

358 - cfg.SourceType = confgroup.TypeDyncfg

359 - cfg.Source = fn.Source()

360 - cfg.Name = name

361 - if cfg.Hostname == "" {

362 - cfg.Hostname = name

363 - }

364 -}

365 -

366 -func vnodeConfigFromPayload(fn dyncfg.Function) (*vnodes.VirtualNode, error) {

367 - var cfg vnodes.VirtualNode

368 -

369 - if err := fn.UnmarshalPayload(&cfg); err != nil {

370 - return nil, err

371 - }

372 -

373 - return &cfg, nil

374 -}

375 -

376 -func vnodeUserconfigFromPayload(fn dyncfg.Function) ([]byte, error) {

377 - cfg, err := vnodeConfigFromPayload(fn)

378 - if err != nil {

379 - return nil, err

380 - }

381 -

382 - name := fn.JobName()

383 - if name == "" {

384 - name = "test"

46 }

386 -

387 - dyncfgUpdateVnodeConfig(cfg, name, fn)

388 -

389 - bs, err := yaml.Marshal([]any{cfg})

390 - if err != nil {

391 - return nil, err

392 - }

393 -

394 - return bs, nil

47 }

src/go/plugin/agent/jobmgr/dyncfg_vnode_test.go new

+338

@@ -0,0 +1,338 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "bytes"

7 + "context"

8 + "encoding/json"

9 + "strings"

10 + "testing"

11 + "time"

12 +

13 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

14 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

15 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

18 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

19 + "github.com/stretchr/testify/assert"

20 + "github.com/stretchr/testify/require"

21 +)

22 +

23 +func TestDyncfgVnodeExec_Dispatch(t *testing.T) {

24 + tests := map[string]struct {

25 + fn func(t *testing.T, mgr *Manager) dyncfg.Function

26 + wantQueuedCmd dyncfg.Command

27 + assertDirect func(t *testing.T, out string)

28 + }{

29 + "schema stays direct": {

30 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

31 + return dyncfg.NewFunction(functions.Function{UID: "vn-schema", Args: []string{mgr.dyncfgVnodePrefixValue(), string(dyncfg.CommandSchema)}})

32 + },

33 + assertDirect: func(t *testing.T, out string) {

34 + assert.Contains(t, out, "FUNCTION_RESULT_BEGIN vn-schema 200 application/json")

35 + },

36 + },

37 + "userconfig stays direct": {

38 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

39 + return dyncfg.NewFunction(functions.Function{

40 + UID: "vn-userconfig",

41 + ContentType: "application/json",

42 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

43 + Args: []string{mgr.dyncfgVnodePrefixValue(), string(dyncfg.CommandUserconfig), "db"},

44 + })

45 + },

46 + assertDirect: func(t *testing.T, out string) {

47 + assert.Contains(t, out, "FUNCTION_RESULT_BEGIN vn-userconfig 200 application/yaml")

48 + assert.Contains(t, out, "name: db")

49 + },

50 + },

51 + "add is queued": {

52 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

53 + return dyncfg.NewFunction(functions.Function{

54 + UID: "vn-add",

55 + ContentType: "application/json",

56 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

57 + Args: []string{mgr.dyncfgVnodePrefixValue(), string(dyncfg.CommandAdd), "db"},

58 + })

59 + },

60 + wantQueuedCmd: dyncfg.CommandAdd,

61 + },

62 + "update is queued": {

63 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

64 + return dyncfg.NewFunction(functions.Function{

65 + UID: "vn-update",

66 + ContentType: "application/json",

67 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

68 + Args: []string{mgr.dyncfgVnodePrefixValue() + ":db", string(dyncfg.CommandUpdate)},

69 + })

70 + },

71 + wantQueuedCmd: dyncfg.CommandUpdate,

72 + },

73 + "remove is queued": {

74 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

75 + return dyncfg.NewFunction(functions.Function{UID: "vn-remove", Args: []string{mgr.dyncfgVnodePrefixValue() + ":db", string(dyncfg.CommandRemove)}})

76 + },

77 + wantQueuedCmd: dyncfg.CommandRemove,

78 + },

79 + "test is queued": {

80 + fn: func(t *testing.T, mgr *Manager) dyncfg.Function {

81 + return dyncfg.NewFunction(functions.Function{

82 + UID: "vn-test",

83 + ContentType: "application/json",

84 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

85 + Args: []string{mgr.dyncfgVnodePrefixValue(), string(dyncfg.CommandTest), "db"},

86 + })

87 + },

88 + wantQueuedCmd: dyncfg.CommandTest,

89 + },

90 + }

91 +

92 + for name, tc := range tests {

93 + t.Run(name, func(t *testing.T) {

94 + var buf bytes.Buffer

95 + mgr := New(Config{PluginName: testPluginName})

96 + mgr.ctx = context.Background()

97 + mgr.dyncfgCh = make(chan dyncfg.Function, 1)

98 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

99 +

100 + fn := tc.fn(t, mgr)

101 + mgr.dyncfgVnodeExec(fn)

102 +

103 + if tc.wantQueuedCmd != "" {

104 + select {

105 + case queued := <-mgr.dyncfgCh:

106 + assert.Equal(t, tc.wantQueuedCmd, queued.Command())

107 + assert.Equal(t, fn.UID(), queued.UID())

108 + assert.Equal(t, "", strings.TrimSpace(buf.String()))

109 + case <-time.After(time.Second):

110 + t.Fatal("vnode command was not queued")

111 + }

112 + return

113 + }

114 +

115 + select {

116 + case queued := <-mgr.dyncfgCh:

117 + t.Fatalf("unexpected queued vnode command: %s", queued.Command())

118 + default:

119 + }

120 + tc.assertDirect(t, buf.String())

121 + })

122 + }

123 +}

124 +

125 +func TestRun_PublishesExistingVnodesThroughController(t *testing.T) {

126 + tests := map[string]struct {

127 + run func(t *testing.T, out *bytes.Buffer, fnReg *recordingFunctionRegistry)

128 + }{

129 + "startup publishes vnode module and existing job": {

130 + run: func(t *testing.T, out *bytes.Buffer, fnReg *recordingFunctionRegistry) {

131 + assert.Contains(t, out.String(), "CONFIG test:vnode create accepted template /collectors/test/Vnodes")

132 + assert.Contains(t, out.String(), "CONFIG test:vnode:db create running job /collectors/test/Vnodes")

133 + prefixes := fnReg.registeredPrefixes()

134 + assert.Contains(t, prefixes, registeredPrefix{name: "config", prefix: "test:vnode"})

135 + },

136 + },

137 + }

138 +

139 + for name, tc := range tests {

140 + t.Run(name, func(t *testing.T) {

141 + var buf bytes.Buffer

142 + fnReg := &recordingFunctionRegistry{}

143 + mgr := New(Config{

144 + PluginName: testPluginName,

145 + Out: &buf,

146 + FnReg: fnReg,

147 + Vnodes: map[string]*vnodes.VirtualNode{

148 + "db": {

149 + Name: "db",

150 + Hostname: "db",

151 + GUID: "11111111-1111-1111-1111-111111111111",

152 + SourceType: confgroup.TypeDyncfg,

153 + Source: confgroup.TypeDyncfg,

154 + },

155 + },

156 + })

157 +

158 + ctx, cancel := context.WithCancel(context.Background())

159 + in := make(chan []*confgroup.Group)

160 + done := make(chan struct{})

161 + go func() {

162 + mgr.Run(ctx, in)

163 + close(done)

164 + }()

165 +

166 + waitCtx, waitCancel := context.WithTimeout(context.Background(), time.Second)

167 + defer waitCancel()

168 + require.True(t, mgr.WaitStarted(waitCtx))

169 +

170 + cancel()

171 + close(in)

172 + select {

173 + case <-done:

174 + case <-time.After(2 * time.Second):

175 + t.Fatal("manager did not stop after cancel")

176 + }

177 +

178 + tc.run(t, &buf, fnReg)

179 + })

180 + }

181 +}

182 +

183 +func TestCreateCollectorJob_UsesVnodeControllerLookup(t *testing.T) {

184 + tests := map[string]struct {

185 + cfg confgroup.Config

186 + vnodes map[string]*vnodes.VirtualNode

187 + wantErr string

188 + run func(t *testing.T, job runtimeJob)

189 + }{

190 + "existing vnode is copied into created job": {

191 + cfg: prepareDyncfgCfg("success", "mysql").Set("vnode", "db"),

192 + vnodes: map[string]*vnodes.VirtualNode{

193 + "db": {

194 + Name: "db",

195 + Hostname: "db",

196 + GUID: "11111111-1111-1111-1111-111111111111",

197 + SourceType: confgroup.TypeDyncfg,

198 + Source: confgroup.TypeDyncfg,

199 + },

200 + },

201 + run: func(t *testing.T, job runtimeJob) {

202 + assert.Equal(t, "db", job.Vnode().Name)

203 + },

204 + },

205 + "missing vnode returns an error": {

206 + cfg: prepareDyncfgCfg("success", "mysql").Set("vnode", "db"),

207 + wantErr: "vnode 'db' is not found",

208 + },

209 + }

210 +

211 + for name, tc := range tests {

212 + t.Run(name, func(t *testing.T) {

213 + mgr := New(Config{PluginName: testPluginName, Vnodes: tc.vnodes})

214 + mgr.modules = prepareMockRegistry()

215 +

216 + job, err := mgr.createCollectorJob(tc.cfg)

217 + if tc.wantErr != "" {

218 + require.Error(t, err)

219 + assert.Contains(t, err.Error(), tc.wantErr)

220 + return

221 + }

222 +

223 + require.NoError(t, err)

224 + tc.run(t, job)

225 + })

226 + }

227 +}

228 +

229 +func TestDyncfgCmdTest_ValidatesVnodeThroughControllerLookup(t *testing.T) {

230 + tests := map[string]struct {

231 + run func(t *testing.T, mgr *Manager, out *bytes.Buffer)

232 + }{

233 + "missing vnode returns 400 before worker execution": {

234 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

235 + cfg := prepareDyncfgCfg("success", "job").Set("vnode", "missing")

236 + payload, err := json.Marshal(cfg)

237 + require.NoError(t, err)

238 +

239 + fn := dyncfg.NewFunction(functions.Function{

240 + UID: "collector-vnode-missing",

241 + ContentType: "application/json",

242 + Payload: payload,

243 + Args: []string{mgr.dyncfgModID("success"), string(dyncfg.CommandTest), "job"},

244 + })

245 +

246 + mgr.dyncfgCmdTest(fn)

247 +

248 + var resp map[string]any

249 + mustDecodeFunctionPayload(t, out.String(), "collector-vnode-missing", &resp)

250 + assert.Equal(t, float64(400), resp["status"])

251 + assert.Contains(t, resp["errorMessage"], "missing")

252 + },

253 + },

254 + }

255 +

256 + for name, tc := range tests {

257 + t.Run(name, func(t *testing.T) {

258 + var buf bytes.Buffer

259 + mgr := New(Config{PluginName: testPluginName})

260 + mgr.modules = prepareMockRegistry()

261 + mgr.ctx = context.Background()

262 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

263 + tc.run(t, mgr, &buf)

264 + })

265 + }

266 +}

267 +

268 +func TestApplyVnodeUpdate_UpdatesMatchingRunningJobs(t *testing.T) {

269 + mgr := New(Config{PluginName: testPluginName})

270 +

271 + dbJob := &vnodeUpdateProbeJob{

272 + fullName: "success_db",

273 + module: "success",

274 + name: "db",

275 + vnode: vnodes.VirtualNode{Name: "db"},

276 + }

277 + otherJob := &vnodeUpdateProbeJob{

278 + fullName: "success_other",

279 + module: "success",

280 + name: "other",

281 + vnode: vnodes.VirtualNode{Name: "other"},

282 + }

283 +

284 + mgr.runningJobs.lock()

285 + mgr.runningJobs.add(dbJob.FullName(), dbJob)

286 + mgr.runningJobs.add(otherJob.FullName(), otherJob)

287 + mgr.runningJobs.unlock()

288 +

289 + next := &vnodes.VirtualNode{

290 + Name: "db",

291 + Hostname: "db-new",

292 + GUID: "11111111-1111-1111-1111-111111111111",

293 + SourceType: confgroup.TypeDyncfg,

294 + Source: confgroup.TypeDyncfg,

295 + }

296 +

297 + mgr.applyVnodeUpdate("db", next)

298 +

299 + require.NotNil(t, dbJob.updated)

300 + assert.Equal(t, "db-new", dbJob.updated.Hostname)

301 + assert.Nil(t, otherJob.updated)

302 +}

303 +

304 +type vnodeUpdateProbeJob struct {

305 + fullName string

306 + module string

307 + name string

308 + vnode vnodes.VirtualNode

309 + updated *vnodes.VirtualNode

310 +}

311 +

312 +func (j *vnodeUpdateProbeJob) FullName() string { return j.fullName }

313 +func (j *vnodeUpdateProbeJob) ModuleName() string { return j.module }

314 +func (j *vnodeUpdateProbeJob) Name() string { return j.name }

315 +func (j *vnodeUpdateProbeJob) Collector() any { return nil }

316 +func (j *vnodeUpdateProbeJob) Start() {}

317 +func (j *vnodeUpdateProbeJob) Stop() {}

318 +func (j *vnodeUpdateProbeJob) Tick(int) {}

319 +func (j *vnodeUpdateProbeJob) AutoDetection() error {

320 + return nil

321 +}

322 +func (j *vnodeUpdateProbeJob) AutoDetectionEvery() int { return 0 }

323 +func (j *vnodeUpdateProbeJob) RetryAutoDetection() bool {

324 + return false

325 +}

326 +func (j *vnodeUpdateProbeJob) Cleanup() {}

327 +func (j *vnodeUpdateProbeJob) IsRunning() bool { return true }

328 +func (j *vnodeUpdateProbeJob) Panicked() bool { return false }

329 +func (j *vnodeUpdateProbeJob) Vnode() vnodes.VirtualNode { return j.vnode }

330 +func (j *vnodeUpdateProbeJob) UpdateVnode(vnode *vnodes.VirtualNode) {

331 + if vnode == nil {

332 + j.updated = nil

333 + return

334 + }

335 + copy := *vnode

336 + j.updated = &copy

337 + j.vnode = copy

338 +}

src/go/plugin/agent/jobmgr/funcctl/controller.go new

+268

@@ -0,0 +1,268 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package funcctl

4 +

5 +import (

6 + "context"

7 + "fmt"

8 +

9 + "github.com/netdata/netdata/go/plugins/logger"

10 + "github.com/netdata/netdata/go/plugins/pkg/funcapi"

11 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

14 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

15 +)

16 +

17 +type Options struct {

18 + Logger *logger.Logger

19 + FnReg functions.Registry

20 + API *dyncfg.Responder

21 + JSONWriter func([]byte, int)

22 +}

23 +

24 +type Controller struct {

25 + *logger.Logger

26 +

27 + api *dyncfg.Responder

28 + jsonWriter func([]byte, int)

29 + fnReg functions.Registry

30 + ctx context.Context

31 +

32 + registry *moduleFuncRegistry

33 + staticMethodsSeen map[string]struct{}

34 +}

35 +

36 +func New(opts Options) *Controller {

37 + log := opts.Logger

38 + if log == nil {

39 + log = logger.New()

40 + }

41 + reg := opts.FnReg

42 + if reg == nil {

43 + reg = noopRegistry{}

44 + }

45 +

46 + return &Controller{

47 + Logger: log,

48 + api: opts.API,

49 + jsonWriter: opts.JSONWriter,

50 + fnReg: reg,

51 + registry: newModuleFuncRegistry(),

52 + staticMethodsSeen: make(map[string]struct{}),

53 + }

54 +}

55 +

56 +func (c *Controller) Init(ctx context.Context) {

57 + c.ctx = ctx

58 +}

59 +

60 +func (c *Controller) SetAPI(api *dyncfg.Responder) {

61 + if api == nil {

62 + // Nil means "keep the current responder" rather than clearing output wiring.

63 + return

64 + }

65 + c.api = api

66 +}

67 +

68 +func (c *Controller) RegisterModules(modules collectorapi.Registry) {

69 + for name, creator := range modules {

70 + if creator.Methods == nil && creator.JobMethods == nil {

71 + continue

72 + }

73 + c.registry.registerModule(name, creator)

74 + }

75 +}

76 +

77 +func (c *Controller) GetJobNames(moduleName string) []string {

78 + return c.registry.getJobNames(moduleName)

79 +}

80 +

81 +func (c *Controller) OnJobStart(job collectorapi.RuntimeJob) {

82 + if job == nil {

83 + return

84 + }

85 +

86 + c.registry.addJob(job.ModuleName(), job.Name(), job)

87 + c.registerModuleMethodsOnFirstJobStart(job.ModuleName())

88 +

89 + creator, ok := c.registry.getCreator(job.ModuleName())

90 + if !ok || creator.JobMethods == nil {

91 + return

92 + }

93 +

94 + methods := creator.JobMethods(job)

95 + if len(methods) > 0 {

96 + c.registerJobMethods(job, methods)

97 + }

98 +}

99 +

100 +func (c *Controller) OnJobStop(job collectorapi.RuntimeJob) {

101 + if job == nil {

102 + return

103 + }

104 +

105 + c.unregisterJobMethods(job)

106 + c.registry.removeJob(job.ModuleName(), job.Name())

107 +}

108 +

109 +func (c *Controller) Cleanup() {

110 + for name, creator := range c.registry.snapshotCreators() {

111 + if creator.Methods == nil {

112 + continue

113 + }

114 + for _, method := range creator.Methods() {

115 + if method.ID == "" {

116 + continue

117 + }

118 + funcName := fmt.Sprintf("%s:%s", name, method.ID)

119 + c.fnReg.Unregister(funcName)

120 + if c.api != nil {

121 + c.api.FunctionRemove(funcName)

122 + }

123 + }

124 + }

125 +}

126 +

127 +func (c *Controller) registerModuleMethodsOnFirstJobStart(moduleName string) {

128 + if _, ok := c.staticMethodsSeen[moduleName]; ok {

129 + return

130 + }

131 +

132 + creator, ok := c.registry.getCreator(moduleName)

133 + if !ok || creator.Methods == nil {

134 + return

135 + }

136 +

137 + for _, method := range creator.Methods() {

138 + if method.ID == "" {

139 + c.Warningf("skipping function registration for module '%s': empty method ID", moduleName)

140 + continue

141 + }

142 +

143 + funcName := fmt.Sprintf("%s:%s", moduleName, method.ID)

144 + c.fnReg.Register(funcName, c.makeMethodFuncHandler(moduleName, method.ID))

145 +

146 + if c.api != nil {

147 + help := method.Help

148 + if help == "" {

149 + help = fmt.Sprintf("%s %s data function", moduleName, method.ID)

150 + }

151 +

152 + const cloudAccess = "0x0013"

153 + access := "0x0000"

154 + if method.RequireCloud {

155 + access = cloudAccess

156 + }

157 +

158 + c.api.FunctionGlobal(netdataapi.FunctionGlobalOpts{

159 + Name: funcName,

160 + Timeout: 60,

161 + Help: help,

162 + Tags: "top",

163 + Access: access,

164 + Priority: 100,

165 + Version: 3,

166 + })

167 + }

168 + }

169 +

170 + c.staticMethodsSeen[moduleName] = struct{}{}

171 +}

172 +

173 +func (c *Controller) registerJobMethods(job collectorapi.RuntimeJob, methods []funcapi.MethodConfig) {

174 + planned := make(map[string]struct{}, len(methods))

175 +

176 + for _, method := range methods {

177 + if method.ID == "" {

178 + c.Warningf("skipping job method registration for %s[%s]: empty method ID", job.ModuleName(), job.Name())

179 + continue

180 + }

181 +

182 + funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

183 + if _, exists := planned[method.ID]; exists {

184 + c.Errorf("job method registration aborted for %s[%s]: duplicate method ID in batch ('%s')", job.ModuleName(), job.Name(), funcName)

185 + return

186 + }

187 + planned[method.ID] = struct{}{}

188 +

189 + if collision, exists := c.registry.findMethodCollision(job.ModuleName(), job.Name(), method.ID); exists {

190 + c.Errorf("job method registration aborted for %s[%s]: collision on '%s' (%s)", job.ModuleName(), job.Name(), funcName, collision)

191 + return

192 + }

193 + }

194 +

195 + // Record methods before publishing handlers so startup-time calls do not race a false 404.

196 + c.registry.registerJobMethods(job.ModuleName(), job.Name(), methods)

197 +

198 + for _, method := range methods {

199 + if method.ID == "" {

200 + continue

201 + }

202 +

203 + funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

204 + c.fnReg.Register(funcName, c.makeJobMethodFuncHandler(job.ModuleName(), job.Name(), method.ID))

205 +

206 + if c.api != nil {

207 + help := method.Help

208 + if help == "" {

209 + help = fmt.Sprintf("%s %s data function", job.ModuleName(), method.ID)

210 + }

211 +

212 + const cloudAccess = "0x0013"

213 + access := "0x0000"

214 + if method.RequireCloud {

215 + access = cloudAccess

216 + }

217 +

218 + c.api.FunctionGlobal(netdataapi.FunctionGlobalOpts{

219 + Name: funcName,

220 + Timeout: 60,

221 + Help: help,

222 + Tags: "top",

223 + Access: access,

224 + Priority: 100,

225 + Version: 3,

226 + })

227 + }

228 +

229 + c.Debugf("registered job method: %s for job %s[%s]", funcName, job.ModuleName(), job.Name())

230 + }

231 +}

232 +

233 +func (c *Controller) unregisterJobMethods(job collectorapi.RuntimeJob) {

234 + methods := c.registry.getJobMethods(job.ModuleName(), job.Name())

235 + if len(methods) == 0 {

236 + return

237 + }

238 +

239 + for _, method := range methods {

240 + if method.ID == "" {

241 + continue

242 + }

243 +

244 + funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

245 + c.fnReg.Unregister(funcName)

246 + if c.api != nil {

247 + c.api.FunctionRemove(funcName)

248 + }

249 + c.Debugf("unregistered job method: %s for job %s[%s]", funcName, job.ModuleName(), job.Name())

250 + }

251 +

252 + c.registry.unregisterJobMethods(job.ModuleName(), job.Name())

253 +}

254 +

255 +func (c *Controller) baseContext() context.Context {

256 + if c.ctx != nil {

257 + return c.ctx

258 + }

259 + return context.Background()

260 +}

261 +

262 +type noopRegistry struct{}

263 +

264 +func (noopRegistry) Register(string, func(functions.Function)) {}

265 +func (noopRegistry) Unregister(string) {}

266 +func (noopRegistry) RegisterPrefix(string, string, func(functions.Function)) {

267 +}

268 +func (noopRegistry) UnregisterPrefix(string, string) {}

src/go/plugin/agent/jobmgr/funcctl/controller_test.go new

+576

@@ -0,0 +1,576 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package funcctl

4 +

5 +import (

6 + "bytes"

7 + "encoding/json"

8 + "testing"

9 +

10 + "github.com/stretchr/testify/assert"

11 + "github.com/stretchr/testify/require"

12 +

13 + "github.com/netdata/netdata/go/plugins/pkg/funcapi"

14 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

15 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

18 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

19 +)

20 +

21 +func TestModuleFuncRegistry_RegisterModule(t *testing.T) {

22 + tests := map[string]struct {

23 + modules []string

24 + expected []string

25 + }{

26 + "single module": {

27 + modules: []string{"postgres"},

28 + expected: []string{"postgres"},

29 + },

30 + "multiple modules": {

31 + modules: []string{"postgres", "mysql", "mssql"},

32 + expected: []string{"mysql", "mssql", "postgres"},

33 + },

34 + "duplicate registration overwrites": {

35 + modules: []string{"postgres", "postgres"},

36 + expected: []string{"postgres"},

37 + },

38 + }

39 +

40 + for name, tc := range tests {

41 + t.Run(name, func(t *testing.T) {

42 + r := newModuleFuncRegistry()

43 +

44 + for _, module := range tc.modules {

45 + r.registerModule(module, collectorapi.Creator{

46 + Methods: func() []funcapi.MethodConfig {

47 + return []funcapi.MethodConfig{{ID: "test"}}

48 + },

49 + })

50 + }

51 +

52 + assert.Equal(t, len(tc.expected), len(r.modules))

53 + for _, module := range tc.expected {

54 + assert.True(t, r.isModuleRegistered(module))

55 + }

56 + })

57 + }

58 +}

59 +

60 +func TestModuleFuncRegistry_Operations(t *testing.T) {

61 + tests := map[string]struct{}{

62 + "add remove job": {},

63 + "job replacement increments generation": {},

64 + "job re-add after removal increments generation": {},

65 + "generation verification fails on wrong generation and missing job": {},

66 + "get methods": {},

67 + "get job names sorted": {},

68 + "operations on unregistered module are no op": {},

69 + "get creator": {},

70 + }

71 +

72 + for name := range tests {

73 + t.Run(name, func(t *testing.T) {

74 + r := newModuleFuncRegistry()

75 +

76 + switch name {

77 + case "add remove job":

78 + r.registerModule("postgres", collectorapi.Creator{})

79 +

80 + job1 := newTestRuntimeJob("postgres", "job1", true)

81 + job2 := newTestRuntimeJob("postgres", "job2", true)

82 +

83 + r.addJob("postgres", "job1", job1)

84 + r.addJob("postgres", "job2", job2)

85 +

86 + names := r.getJobNames("postgres")

87 + assert.ElementsMatch(t, []string{"job1", "job2"}, names)

88 +

89 + got1, ok := r.getJob("postgres", "job1")

90 + assert.True(t, ok)

91 + assert.Equal(t, job1, got1)

92 +

93 + r.removeJob("postgres", "job1")

94 +

95 + names = r.getJobNames("postgres")

96 + assert.ElementsMatch(t, []string{"job2"}, names)

97 +

98 + _, ok = r.getJob("postgres", "job1")

99 + assert.False(t, ok)

100 +

101 + case "job replacement increments generation":

102 + r.registerModule("postgres", collectorapi.Creator{})

103 +

104 + job1 := newTestRuntimeJob("postgres", "master", true)

105 + job2 := newTestRuntimeJob("postgres", "master", true)

106 +

107 + r.addJob("postgres", "master", job1)

108 + _, gen1 := r.getJobWithGeneration("postgres", "master")

109 + assert.Equal(t, uint64(1), gen1)

110 +

111 + r.addJob("postgres", "master", job2)

112 + got, gen2 := r.getJobWithGeneration("postgres", "master")

113 + assert.Equal(t, uint64(2), gen2)

114 + assert.Equal(t, job2, got)

115 +

116 + case "job re-add after removal increments generation":

117 + r.registerModule("postgres", collectorapi.Creator{})

118 +

119 + job1 := newTestRuntimeJob("postgres", "master", true)

120 + job2 := newTestRuntimeJob("postgres", "master", true)

121 +

122 + r.addJob("postgres", "master", job1)

123 + _, gen1 := r.getJobWithGeneration("postgres", "master")

124 + assert.Equal(t, uint64(1), gen1)

125 +

126 + r.removeJob("postgres", "master")

127 + r.addJob("postgres", "master", job2)

128 + got, gen2 := r.getJobWithGeneration("postgres", "master")

129 + assert.Equal(t, uint64(2), gen2)

130 + assert.Equal(t, job2, got)

131 +

132 + case "generation verification fails on wrong generation and missing job":

133 + r.registerModule("postgres", collectorapi.Creator{})

134 +

135 + job := newTestRuntimeJob("postgres", "master", true)

136 + r.addJob("postgres", "master", job)

137 + _, gen := r.getJobWithGeneration("postgres", "master")

138 +

139 + assert.False(t, r.verifyJobGeneration("postgres", "master", gen+1))

140 + r.removeJob("postgres", "master")

141 + assert.False(t, r.verifyJobGeneration("postgres", "master", gen))

142 +

143 + case "get methods":

144 + expectedMethods := []funcapi.MethodConfig{{ID: "top-queries", Name: "Top Queries"}}

145 + r.registerModule("postgres", collectorapi.Creator{

146 + Methods: func() []funcapi.MethodConfig { return expectedMethods },

147 + })

148 +

149 + assert.Equal(t, expectedMethods, r.getMethods("postgres"))

150 + assert.Nil(t, r.getMethods("nonexistent"))

151 +

152 + case "get job names sorted":

153 + r.registerModule("postgres", collectorapi.Creator{})

154 +

155 + r.addJob("postgres", "zebra-db", newTestRuntimeJob("postgres", "zebra-db", true))

156 + r.addJob("postgres", "alpha-db", newTestRuntimeJob("postgres", "alpha-db", true))

157 + r.addJob("postgres", "middle-db", newTestRuntimeJob("postgres", "middle-db", true))

158 +

159 + assert.Equal(t, []string{"alpha-db", "middle-db", "zebra-db"}, r.getJobNames("postgres"))

160 +

161 + case "operations on unregistered module are no op":

162 + r.addJob("nonexistent", "job1", newTestRuntimeJob("nonexistent", "job1", true))

163 + r.removeJob("nonexistent", "job1")

164 +

165 + assert.False(t, r.isModuleRegistered("nonexistent"))

166 + assert.Nil(t, r.getJobNames("nonexistent"))

167 + assert.Nil(t, r.getMethods("nonexistent"))

168 +

169 + _, ok := r.getJob("nonexistent", "job1")

170 + assert.False(t, ok)

171 +

172 + case "get creator":

173 + creator := collectorapi.Creator{JobConfigSchema: "test-schema"}

174 + r.registerModule("postgres", creator)

175 +

176 + got, ok := r.getCreator("postgres")

177 + require.True(t, ok)

178 + assert.Equal(t, "test-schema", got.JobConfigSchema)

179 +

180 + _, ok = r.getCreator("nonexistent")

181 + assert.False(t, ok)

182 + }

183 + })

184 + }

185 +}

186 +

187 +func TestModuleFuncRegistry_Concurrency(t *testing.T) {

188 + r := newModuleFuncRegistry()

189 + r.registerModule("postgres", collectorapi.Creator{

190 + Methods: func() []funcapi.MethodConfig {

191 + return []funcapi.MethodConfig{{ID: "test"}}

192 + },

193 + })

194 +

195 + done := make(chan bool)

196 +

197 + go func() {

198 + for i := 0; i < 100; i++ {

199 + job := newTestRuntimeJob("postgres", "job", true)

200 + r.addJob("postgres", "job", job)

201 + r.removeJob("postgres", "job")

202 + }

203 + done <- true

204 + }()

205 +

206 + go func() {

207 + for i := 0; i < 100; i++ {

208 + _ = r.getJobNames("postgres")

209 + _ = r.getMethods("postgres")

210 + _, _ = r.getJob("postgres", "job")

211 + }

212 + done <- true

213 + }()

214 +

215 + <-done

216 + <-done

217 +}

218 +

219 +func TestModuleFuncRegistry_VerifyJobGeneration_JobStopped(t *testing.T) {

220 + r := newModuleFuncRegistry()

221 + r.registerModule("postgres", collectorapi.Creator{})

222 +

223 + job := newTestRuntimeJob("postgres", "master", false)

224 + r.addJob("postgres", "master", job)

225 + _, gen := r.getJobWithGeneration("postgres", "master")

226 +

227 + assert.False(t, r.verifyJobGeneration("postgres", "master", gen))

228 +}

229 +

230 +func TestDispatchHelpers(t *testing.T) {

231 + tests := map[string]struct{}{

232 + "extract param values": {},

233 + "build params": {},

234 + }

235 +

236 + for name := range tests {

237 + t.Run(name, func(t *testing.T) {

238 + switch name {

239 + case "extract param values":

240 + cases := map[string]struct {

241 + payload map[string]any

242 + key string

243 + expected []string

244 + }{

245 + "string value": {

246 + payload: map[string]any{"__job": "local"},

247 + key: "__job",

248 + expected: []string{"local"},

249 + },

250 + "array value": {

251 + payload: map[string]any{"__sort": []any{"calls", "total_time"}},

252 + key: "__sort",

253 + expected: []string{"calls", "total_time"},

254 + },

255 + "string array": {

256 + payload: map[string]any{"__job": []string{"local"}},

257 + key: "__job",

258 + expected: []string{"local"},

259 + },

260 + "missing key": {

261 + payload: map[string]any{"__job": "local"},

262 + key: "__sort",

263 + expected: nil,

264 + },

265 + "prefers selections": {

266 + payload: map[string]any{

267 + "__job": "root",

268 + "selections": map[string]any{

269 + "__job": []any{"selected"},

270 + },

271 + },

272 + key: "__job",

273 + expected: []string{"selected"},

274 + },

275 + }

276 +

277 + for caseName, tc := range cases {

278 + t.Run(caseName, func(t *testing.T) {

279 + assert.Equal(t, tc.expected, extractParamValues(tc.payload, tc.key))

280 + })

281 + }

282 +

283 + case "build params":

284 + cases := map[string]struct{}{

285 + "build accepted params": {},

286 + "build required params uses select type": {},

287 + }

288 +

289 + for caseName := range cases {

290 + t.Run(caseName, func(t *testing.T) {

291 + switch caseName {

292 + case "build accepted params":

293 + sortDir := funcapi.FieldSortDescending

294 + methodParams := []funcapi.ParamConfig{

295 + {ID: "__sort", Selection: funcapi.ParamSelect, Options: []funcapi.ParamOption{{ID: "calls", Name: "Calls", Sort: &sortDir}}},

296 + {ID: "db"},

297 + {ID: "extra"},

298 + }

299 +

300 + assert.Equal(t, []string{"__job", "__sort", "db", "extra"}, buildAcceptedParams(methodParams))

301 +

302 + case "build required params uses select type":

303 + controller := New(Options{})

304 + controller.RegisterModules(collectorapi.Registry{

305 + "postgres": collectorapi.Creator{

306 + Methods: func() []funcapi.MethodConfig {

307 + return []funcapi.MethodConfig{{ID: "top-queries", Name: "Top Queries"}}

308 + },

309 + },

310 + })

311 + controller.registry.addJob("postgres", "master-db", newTestRuntimeJob("postgres", "master-db", true))

312 +

313 + methodParams := []funcapi.ParamConfig{{

314 + ID: "__sort",

315 + Name: "Filter By",

316 + Selection: funcapi.ParamSelect,

317 + UniqueView: true,

318 + Options: []funcapi.ParamOption{

319 + {ID: "total_time", Name: "By Total Time", Default: true},

320 + },

321 + }}

322 + params := controller.buildRequiredParams("postgres", methodParams)

323 +

324 + assert.Len(t, params, 2)

325 + for _, param := range params {

326 + paramType, ok := param["type"]

327 + assert.True(t, ok)

328 + assert.Equal(t, "select", paramType)

329 + assert.Contains(t, param, "id")

330 + assert.Contains(t, param, "name")

331 + assert.Contains(t, param, "options")

332 + assert.Contains(t, param, "unique_view")

333 + uniqueView, _ := param["unique_view"].(bool)

334 + assert.True(t, uniqueView)

335 + }

336 +

337 + assert.Equal(t, "__job", params[0]["id"])

338 + assert.Equal(t, "__sort", params[1]["id"])

339 + }

340 + })

341 + }

342 + }

343 + })

344 + }

345 +}

346 +

347 +func TestControllerLifecycleHooks(t *testing.T) {

348 + tests := map[string]struct{}{

349 + "register modules does not register static methods yet": {},

350 + "first job start registers static methods once": {},

351 + "job stop unregisters job methods": {},

352 + "cleanup unregisters static methods": {},

353 + "cleanup with api configured still unregisters static methods": {},

354 + }

355 +

356 + for name := range tests {

357 + t.Run(name, func(t *testing.T) {

358 + reg := newTestFunctionRegistry()

359 + controller := New(Options{FnReg: reg})

360 +

361 + switch name {

362 + case "register modules does not register static methods yet":

363 + controller.RegisterModules(collectorapi.Registry{

364 + "mod": collectorapi.Creator{

365 + Methods: func() []funcapi.MethodConfig { return []funcapi.MethodConfig{{ID: "a"}} },

366 + },

367 + })

368 +

369 + assert.Empty(t, reg.registeredNames())

370 +

371 + case "first job start registers static methods once":

372 + controller.RegisterModules(collectorapi.Registry{

373 + "mod": collectorapi.Creator{

374 + Methods: func() []funcapi.MethodConfig { return []funcapi.MethodConfig{{ID: "a"}} },

375 + },

376 + })

377 +

378 + controller.OnJobStart(newTestRuntimeJob("mod", "job1", true))

379 + controller.OnJobStart(newTestRuntimeJob("mod", "job2", true))

380 +

381 + assert.Equal(t, []string{"mod:a"}, reg.registeredNames())

382 +

383 + case "job stop unregisters job methods":

384 + controller.RegisterModules(collectorapi.Registry{

385 + "mod": collectorapi.Creator{

386 + JobMethods: func(_ collectorapi.RuntimeJob) []funcapi.MethodConfig {

387 + return []funcapi.MethodConfig{{ID: "job-method"}}

388 + },

389 + },

390 + })

391 +

392 + job := newTestRuntimeJob("mod", "job1", true)

393 + controller.OnJobStart(job)

394 + controller.OnJobStop(job)

395 +

396 + assert.Contains(t, reg.unregisteredNames(), "mod:job-method")

397 +

398 + case "cleanup unregisters static methods":

399 + controller.RegisterModules(collectorapi.Registry{

400 + "mod": collectorapi.Creator{

401 + Methods: func() []funcapi.MethodConfig { return []funcapi.MethodConfig{{ID: "a"}} },

402 + },

403 + })

404 +

405 + controller.OnJobStart(newTestRuntimeJob("mod", "job1", true))

406 + controller.Cleanup()

407 +

408 + assert.Contains(t, reg.unregisteredNames(), "mod:a")

409 +

410 + case "cleanup with api configured still unregisters static methods":

411 + var buf bytes.Buffer

412 + controller = New(Options{

413 + FnReg: reg,

414 + API: dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))),

415 + })

416 + controller.RegisterModules(collectorapi.Registry{

417 + "mod": collectorapi.Creator{

418 + Methods: func() []funcapi.MethodConfig { return []funcapi.MethodConfig{{ID: "a"}} },

419 + },

420 + })

421 +

422 + controller.OnJobStart(newTestRuntimeJob("mod", "job1", true))

423 + assert.Contains(t, buf.String(), "FUNCTION GLOBAL \"mod:a\"")

424 +

425 + controller.Cleanup()

426 +

427 + assert.Contains(t, reg.unregisteredNames(), "mod:a")

428 + }

429 + })

430 + }

431 +}

432 +

433 +func TestControllerRegisterJobMethods(t *testing.T) {

434 + tests := map[string]struct{}{

435 + "fail fast on collision with static method": {},

436 + "fail fast on collision with other job": {},

437 + "fail fast on duplicate within batch": {},

438 + "registry is populated before handlers are callable": {},

439 + "success commits all methods": {},

440 + }

441 +

442 + for name := range tests {

443 + t.Run(name, func(t *testing.T) {

444 + reg := newTestFunctionRegistry()

445 + controller := New(Options{FnReg: reg})

446 +

447 + switch name {

448 + case "fail fast on collision with static method":

449 + controller.RegisterModules(collectorapi.Registry{

450 + "mod": collectorapi.Creator{

451 + Methods: func() []funcapi.MethodConfig { return []funcapi.MethodConfig{{ID: "dup"}} },

452 + },

453 + })

454 +

455 + controller.registerJobMethods(newTestRuntimeJob("mod", "job1", true), []funcapi.MethodConfig{{ID: "dup"}})

456 +

457 + assert.Empty(t, reg.registeredNames())

458 + assert.Empty(t, controller.registry.getJobMethods("mod", "job1"))

459 +

460 + case "fail fast on collision with other job":

461 + controller.registry.registerModule("mod", collectorapi.Creator{})

462 + controller.registry.registerJobMethods("mod", "jobA", []funcapi.MethodConfig{{ID: "dup"}})

463 +

464 + controller.registerJobMethods(newTestRuntimeJob("mod", "jobB", true), []funcapi.MethodConfig{{ID: "dup"}})

465 +

466 + assert.Empty(t, reg.registeredNames())

467 + assert.Empty(t, controller.registry.getJobMethods("mod", "jobB"))

468 +

469 + case "fail fast on duplicate within batch":

470 + controller.registry.registerModule("mod", collectorapi.Creator{})

471 +

472 + controller.registerJobMethods(newTestRuntimeJob("mod", "job1", true), []funcapi.MethodConfig{{ID: "dup"}, {ID: "dup"}})

473 +

474 + assert.Empty(t, reg.registeredNames())

475 + assert.Empty(t, controller.registry.getJobMethods("mod", "job1"))

476 +

477 + case "registry is populated before handlers are callable":

478 + var gotCode int

479 + var gotResp map[string]any

480 +

481 + reg.onRegister = func(_ string, fn func(functions.Function)) {

482 + fn(functions.Function{

483 + UID: "during-register",

484 + Args: []string{"info"},

485 + })

486 + }

487 + controller = New(Options{

488 + FnReg: reg,

489 + JSONWriter: func(data []byte, code int) {

490 + gotCode = code

491 + require.NoError(t, json.Unmarshal(data, &gotResp))

492 + },

493 + })

494 + controller.registry.registerModule("mod", collectorapi.Creator{})

495 +

496 + controller.registerJobMethods(newTestRuntimeJob("mod", "job1", true), []funcapi.MethodConfig{{ID: "a", Help: "job method help"}})

497 +

498 + assert.Equal(t, 200, gotCode)

499 + assert.Equal(t, float64(200), gotResp["status"])

500 + assert.Equal(t, "job method help", gotResp["help"])

501 + assert.Len(t, controller.registry.getJobMethods("mod", "job1"), 1)

502 +

503 + case "success commits all methods":

504 + controller.registry.registerModule("mod", collectorapi.Creator{})

505 +

506 + controller.registerJobMethods(newTestRuntimeJob("mod", "job1", true), []funcapi.MethodConfig{{ID: "a"}, {ID: "b"}})

507 +

508 + assert.ElementsMatch(t, []string{"mod:a", "mod:b"}, reg.registeredNames())

509 + assert.Len(t, controller.registry.getJobMethods("mod", "job1"), 2)

510 + }

511 + })

512 + }

513 +}

514 +

515 +type testRuntimeJob struct {

516 + fullName string

517 + moduleName string

518 + name string

519 + running bool

520 +}

521 +

522 +func newTestRuntimeJob(moduleName, name string, running bool) *testRuntimeJob {

523 + return &testRuntimeJob{

524 + fullName: moduleName + "_" + name,

525 + moduleName: moduleName,

526 + name: name,

527 + running: running,

528 + }

529 +}

530 +

531 +func (j *testRuntimeJob) FullName() string { return j.fullName }

532 +func (j *testRuntimeJob) ModuleName() string { return j.moduleName }

533 +func (j *testRuntimeJob) Name() string { return j.name }

534 +func (j *testRuntimeJob) IsRunning() bool { return j.running }

535 +func (j *testRuntimeJob) Collector() any { return nil }

536 +

537 +type testFunctionRegistry struct {

538 + handlers map[string]func(functions.Function)

539 + registered []string

540 + unregistered []string

541 + onRegister func(string, func(functions.Function))

542 +}

543 +

544 +func newTestFunctionRegistry() *testFunctionRegistry {

545 + return &testFunctionRegistry{

546 + handlers: make(map[string]func(functions.Function)),

547 + }

548 +}

549 +

550 +func (r *testFunctionRegistry) Register(name string, fn func(functions.Function)) {

551 + r.handlers[name] = fn

552 + r.registered = append(r.registered, name)

553 + if r.onRegister != nil {

554 + r.onRegister(name, fn)

555 + }

556 +}

557 +

558 +func (r *testFunctionRegistry) Unregister(name string) {

559 + r.unregistered = append(r.unregistered, name)

560 + delete(r.handlers, name)

561 +}

562 +

563 +func (r *testFunctionRegistry) RegisterPrefix(string, string, func(functions.Function)) {}

564 +func (r *testFunctionRegistry) UnregisterPrefix(string, string) {}

565 +

566 +func (r *testFunctionRegistry) registeredNames() []string {

567 + out := make([]string, len(r.registered))

568 + copy(out, r.registered)

569 + return out

570 +}

571 +

572 +func (r *testFunctionRegistry) unregisteredNames() []string {

573 + out := make([]string, len(r.unregistered))

574 + copy(out, r.unregistered)

575 + return out

576 +}

src/go/plugin/agent/jobmgr/funcctl/dispatch.go renamed

+101 -230

@@ -1,6 +1,6 @@

1 // SPDX-License-Identifier: GPL-3.0-or-later

3 -package jobmgr

3 +package funcctl

5 import (

6 "context"

@@ -11,7 +11,6 @@ import (

12 "github.com/netdata/netdata/go/plugins/pkg/funcapi"

13 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

14 - "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

14 "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

15 )

@@ -21,8 +20,6 @@ const (

21 type methodParamResolver func(ctx context.Context, methodCfg *funcapi.MethodConfig, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error)

24 -type methodResponseWriter func(dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int)

25 -

23 type methodExecutionInput struct {

24 fn functions.Function

25 moduleName string

@@ -39,38 +36,46 @@ type methodExecutionInput struct {

36 respond methodResponseWriter

37 }

42 -// executeMethodRequest runs the common method execution pipeline for both

43 -// module-level and job-bound method handlers.

44 -func (m *Manager) executeMethodRequest(in methodExecutionInput) {

45 - ctx, cancel := context.WithTimeout(m.baseContext(), in.fn.Timeout)

39 +func (c *Controller) ExecuteFunction(functionName string, fn functions.Function) {

40 + moduleName, methodID, err := functions.SplitFunctionName(functionName)

41 + if err != nil {

42 + c.respondError(fn, 400, "%v", err)

43 + return

44 + }

45 +

46 + c.makeMethodFuncHandler(moduleName, methodID)(fn)

47 +}

48 +

49 +func (c *Controller) executeMethodRequest(in methodExecutionInput) {

50 + ctx, cancel := context.WithTimeout(c.baseContext(), in.fn.Timeout)

51 defer cancel()

53 if !in.job.IsRunning() {

49 - m.respondError(in.fn, 503, "job '%s' is no longer running", in.jobLabel)

54 + c.respondError(in.fn, 503, "job '%s' is no longer running", in.jobLabel)

55 return

56 }

53 - creator, ok := m.moduleFuncs.getCreator(in.moduleName)

58 + creator, ok := c.registry.getCreator(in.moduleName)

59 if !ok || creator.MethodHandler == nil {

55 - m.respondError(in.fn, 500, "module '%s' does not implement MethodHandler", in.moduleName)

60 + c.respondError(in.fn, 500, "module '%s' does not implement MethodHandler", in.moduleName)

61 return

62 }

64 handler := creator.MethodHandler(in.job)

65 if handler == nil {

61 - m.respondError(in.fn, 500, "module '%s' returned nil handler for job '%s'", in.moduleName, in.jobName)

66 + c.respondError(in.fn, 500, "module '%s' returned nil handler for job '%s'", in.moduleName, in.jobName)

67 return

68 }

70 methodParams, paramsFromJob, err := in.resolveParams(ctx, in.methodCfg, handler, in.methodID)

71 if err != nil {

67 - m.respondError(in.fn, 503, "job '%s' cannot provide parameters: %v", in.jobLabel, err)

72 + c.respondError(in.fn, 503, "job '%s' cannot provide parameters: %v", in.jobLabel, err)

73 return

74 }

76 if paramsFromJob {

77 if err := validateParamValues(methodParams, in.argValues, in.payload, in.jobName); err != nil {

73 - m.respondError(in.fn, 400, "%v", err)

78 + c.respondError(in.fn, 400, "%v", err)

79 return

80 }

81 }

@@ -86,8 +91,8 @@ func (m *Manager) executeMethodRequest(in methodExecutionInput) {

92 dataResp := handler.Handle(ctx, in.methodID, resolvedParams)

89 - if !m.moduleFuncs.verifyJobGeneration(in.moduleName, in.jobName, in.jobGen) {

90 - m.respondError(in.fn, 503, "job '%s' was replaced during request, please retry", in.jobLabel)

94 + if !c.registry.verifyJobGeneration(in.moduleName, in.jobName, in.jobGen) {

95 + c.respondError(in.fn, 503, "job '%s' was replaced during request, please retry", in.jobLabel)

96 return

97 }

@@ -99,47 +104,51 @@ func (m *Manager) executeMethodRequest(in methodExecutionInput) {

104 in.respond(dataResp, methodParams, updateEvery)

105 }

106

102 -// makeMethodFuncHandler creates a function handler for a module+method function (module:method).

103 -func (m *Manager) makeMethodFuncHandler(moduleName, methodID string) func(functions.Function) {

107 +func (c *Controller) makeMethodFuncHandler(moduleName, methodID string) func(functions.Function) {

108 return func(fn functions.Function) {

105 - // Check for "info" request

109 if slices.Contains(fn.Args, "info") {

107 - m.handleMethodFuncInfo(moduleName, methodID, fn)

110 + c.handleMethodFuncInfo(moduleName, methodID, fn)

111 return

112 }

113

111 - methodCfg, ok := m.moduleFuncs.getMethod(moduleName, methodID)

114 + methodCfg, ok := c.registry.getMethod(moduleName, methodID)

115 if !ok {

113 - m.respondError(fn, 404, "unknown method '%s' for module '%s'", methodID, moduleName)

116 + c.respondError(fn, 404, "unknown method '%s' for module '%s'", methodID, moduleName)

117 return

118 }

119

120 payload := parsePayload(fn.Payload)

121 argValues := parseArgsParams(fn.Args)

122

120 - jobs := m.moduleFuncs.getJobNames(moduleName)

123 + jobs := c.registry.getJobNames(moduleName)

124 if len(jobs) == 0 {

122 - m.respondError(fn, 422, "no %s instances configured", moduleName)

125 + c.respondError(fn, 422, "no %s instances configured", moduleName)

126 return

127 }

128 jobParam := buildJobParamConfig(jobs)

129 jobValues := paramValues(argValues, payload, paramJob)

130 + if len(jobValues) > 1 {

131 + c.respondError(fn, 400, "parameter '%s' expects a single value", paramJob)

132 + return

133 + }

134 resolvedJob := funcapi.ResolveParam(jobParam, jobValues)

135 jobName := resolvedJob.GetOne()

136 + if len(jobValues) > 0 && jobValues[0] != jobName {

137 + c.respondError(fn, 404, "unknown job '%s', available: %v", jobValues[0], jobs)

138 + return

139 + }

140 if jobName == "" {

130 - m.respondError(fn, 404, "no %s instances configured", moduleName)

141 + c.respondError(fn, 404, "no %s instances configured", moduleName)

142 return

143 }

144

134 - // Get job WITH generation for race condition detection

135 - // The generation increments when a job is replaced (config reload)

136 - job, jobGen := m.moduleFuncs.getJobWithGeneration(moduleName, jobName)

145 + job, jobGen := c.registry.getJobWithGeneration(moduleName, jobName)

146 if job == nil {

138 - m.respondError(fn, 404, "unknown job '%s', available: %v", jobName, jobs)

147 + c.respondError(fn, 404, "unknown job '%s', available: %v", jobName, jobs)

148 return

149 }

150

142 - m.executeMethodRequest(methodExecutionInput{

151 + c.executeMethodRequest(methodExecutionInput{

152 fn: fn,

153 moduleName: moduleName,

154 jobName: jobName,

@@ -151,27 +160,25 @@ func (m *Manager) makeMethodFuncHandler(moduleName, methodID string) func(functi

160 payload: payload,

161 argValues: argValues,

162 resolveParams: func(ctx context.Context, methodCfg *funcapi.MethodConfig, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error) {

154 - return m.resolveMethodParamsForJob(ctx, moduleName, methodID, methodCfg, job, handler)

163 + return c.resolveMethodParamsForJob(ctx, methodCfg, job, handler, methodID)

164 },

165 augmentParams: func(resolvedParams funcapi.ResolvedParams) {

166 resolvedParams[paramJob] = resolvedJob

167 },

168 respond: func(dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

160 - m.respondWithParams(fn, moduleName, dataResp, methodParams, updateEvery)

169 + c.respondWithParams(fn, moduleName, dataResp, methodParams, updateEvery)

170 },

171 })

172 }

173 }

174

166 -// handleMethodFuncInfo handles "info" requests for a module:method function

167 -func (m *Manager) handleMethodFuncInfo(moduleName, methodID string, fn functions.Function) {

168 - methodCfg, ok := m.moduleFuncs.getMethod(moduleName, methodID)

175 +func (c *Controller) handleMethodFuncInfo(moduleName, methodID string, fn functions.Function) {

176 + methodCfg, ok := c.registry.getMethod(moduleName, methodID)

177 if !ok {

170 - m.respondError(fn, 404, "unknown method '%s' for module '%s'", methodID, moduleName)

178 + c.respondError(fn, 404, "unknown method '%s' for module '%s'", methodID, moduleName)

179 return

180 }

181

174 - // Use static params for info. Actual requests return job-specific params in the response.

182 methodParams := methodCfg.RequiredParams

183 help := methodCfg.Help

184 if help == "" {

@@ -183,7 +190,7 @@ func (m *Manager) handleMethodFuncInfo(moduleName, methodID string, fn functions

190 updateEvery = methodCfg.UpdateEvery

191 }

192

186 - resp := map[string]any{

193 + c.respondJSON(fn, map[string]any{

194 "v": 3,

195 "update_every": updateEvery,

196 "status": 200,

@@ -191,98 +198,14 @@ func (m *Manager) handleMethodFuncInfo(moduleName, methodID string, fn functions

198 "has_history": false,

199 "help": help,

200 "accepted_params": buildAcceptedParams(methodParams),

194 - "required_params": m.buildRequiredParams(moduleName, methodParams),

195 - }

196 -

197 - m.respondJSON(fn, resp)

198 -}

199 -

200 -// respondWithParams wraps the module's data response with current required_params

201 -func (m *Manager) respondWithParams(fn functions.Function, moduleName string, dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

202 - m.respondMethodDataWithParams(

203 - fn,

204 - dataResp,

205 - methodParams,

206 - updateEvery,

207 - buildAcceptedParams,

208 - func(params []funcapi.ParamConfig) []map[string]any {

209 - return m.buildRequiredParams(moduleName, params)

210 - },

211 - )

201 + "required_params": c.buildRequiredParams(moduleName, methodParams),

202 + })

203 }

204

214 -func (m *Manager) respondMethodDataWithParams(

215 - fn functions.Function,

216 - dataResp *funcapi.FunctionResponse,

217 - methodParams []funcapi.ParamConfig,

218 - updateEvery int,

219 - buildAccepted func([]funcapi.ParamConfig) []string,

220 - buildRequired func([]funcapi.ParamConfig) []map[string]any,

221 -) {

222 - if dataResp == nil {

223 - m.respondError(fn, 500, "internal error: module returned nil response")

224 - return

225 - }

226 -

227 - if dataResp.Status >= 400 {

228 - m.respondError(fn, dataResp.Status, "%s", dataResp.Message)

229 - return

230 - }

231 -

232 - paramsForResponse := methodParams

233 - if len(dataResp.RequiredParams) > 0 {

234 - paramsForResponse = funcapi.MergeParamConfigs(paramsForResponse, dataResp.RequiredParams)

235 - }

236 -

237 - resp := map[string]any{

238 - "v": 3,

239 - "update_every": updateEvery,

240 - "status": dataResp.Status,

241 - "type": "table",

242 - "has_history": false,

243 - "help": dataResp.Help,

244 - "accepted_params": buildAccepted(paramsForResponse),

245 - "required_params": buildRequired(paramsForResponse),

246 - }

205 +func (c *Controller) buildRequiredParams(moduleName string, methodParams []funcapi.ParamConfig) []map[string]any {

206 + jobs := c.registry.getJobNames(moduleName)

207

248 - if dataResp.Columns != nil {

249 - resp["columns"] = dataResp.Columns

250 - }

251 - if dataResp.Data != nil {

252 - resp["data"] = dataResp.Data

253 - }

254 - if dataResp.DefaultSortColumn != "" {

255 - resp["default_sort_column"] = dataResp.DefaultSortColumn

256 - }

257 - if len(dataResp.Charts) > 0 {

258 - resp["charts"] = dataResp.Charts

259 - }

260 - if len(dataResp.DefaultCharts) > 0 {

261 - resp["default_charts"] = dataResp.DefaultCharts.Build()

262 - }

263 - if len(dataResp.GroupBy) > 0 {

264 - resp["group_by"] = dataResp.GroupBy

265 - }

266 -

267 - m.respondJSON(fn, resp)

268 -}

269 -

270 -// respondError sends a minimal error response (status + errorMessage).

271 -func (m *Manager) respondError(fn functions.Function, status int, format string, args ...any) {

272 - resp := map[string]any{

273 - "status": status,

274 - "errorMessage": fmt.Sprintf(format, args...),

275 - }

276 - m.respondJSON(fn, resp)

277 -}

278 -

279 -// buildRequiredParams creates the required_params array with current job list

280 -func (m *Manager) buildRequiredParams(moduleName string, methodParams []funcapi.ParamConfig) []map[string]any {

281 - jobs := m.moduleFuncs.getJobNames(moduleName)

282 -

283 - paramConfigs := []funcapi.ParamConfig{

284 - buildJobParamConfig(jobs),

285 - }

208 + paramConfigs := []funcapi.ParamConfig{buildJobParamConfig(jobs)}

209 paramConfigs = append(paramConfigs, methodParams...)

210

211 required := make([]map[string]any, 0, len(paramConfigs))

@@ -292,7 +215,7 @@ func (m *Manager) buildRequiredParams(moduleName string, methodParams []funcapi.

215 return required

216 }

217

295 -func (m *Manager) resolveMethodParamsForJob(ctx context.Context, moduleName, methodID string, methodCfg *funcapi.MethodConfig, job collectorapi.RuntimeJob, handler funcapi.MethodHandler) ([]funcapi.ParamConfig, bool, error) {

218 +func (c *Controller) resolveMethodParamsForJob(ctx context.Context, methodCfg *funcapi.MethodConfig, job collectorapi.RuntimeJob, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error) {

219 methodParams := methodCfg.RequiredParams

220

221 jobParams, err := handler.MethodParams(ctx, methodID)

@@ -316,9 +239,9 @@ func validateParamValues(methodParams []funcapi.ParamConfig, argValues map[strin

239 return fmt.Errorf("parameter '%s' expects a single value for job '%s'", cfg.ID, jobName)

240 }

241 allowed := allowedOptions(cfg.Options)

319 - for _, val := range values {

320 - if !allowed[val] {

321 - return fmt.Errorf("parameter '%s' option '%s' is not supported by job '%s'", cfg.ID, val, jobName)

242 + for _, value := range values {

243 + if !allowed[value] {

244 + return fmt.Errorf("parameter '%s' option '%s' is not supported by job '%s'", cfg.ID, value, jobName)

245 }

246 }

247 }

@@ -336,41 +259,11 @@ func allowedOptions(options []funcapi.ParamOption) map[string]bool {

259 return allowed

260 }

261

339 -// respondJSON sends a JSON response to the function request

340 -// The HTTP status code is extracted from the "status" field in the response

341 -func (m *Manager) respondJSON(fn functions.Function, resp map[string]any) {

342 - data, err := json.Marshal(resp)

343 - if err != nil {

344 - m.Errorf("failed to marshal function response: %v", err)

345 - return

346 - }

347 -

348 - // Extract status code from response for pluginsd protocol

349 - // Default to 200 if not present or not an int

350 - code := 200

351 - if status, ok := resp["status"]; ok {

352 - switch v := status.(type) {

353 - case int:

354 - code = v

355 - case int64:

356 - code = int(v)

357 - case float64:

358 - code = int(v)

359 - }

360 - }

361 -

362 - if m.functionJSONWriter != nil {

363 - m.functionJSONWriter(data, code)

364 - return

365 - }

366 -

367 - m.dyncfgApi.SendJSONWithCode(dyncfg.NewFunction(fn), string(data), code)

368 -}

369 -

262 func parsePayload(raw []byte) map[string]any {

263 if len(raw) == 0 {

264 return nil

265 }

266 +

267 var payload map[string]any

268 if err := json.Unmarshal(raw, &payload); err != nil {

269 return nil

@@ -382,6 +275,7 @@ func parseArgsParams(args []string) map[string][]string {

275 if len(args) == 0 {

276 return nil

277 }

278 +

279 params := make(map[string][]string)

280 for _, arg := range args {

281 if arg == "info" {

@@ -391,8 +285,7 @@ func parseArgsParams(args []string) map[string][]string {

285 if len(parts) != 2 {

286 continue

287 }

394 - key := parts[0]

395 - value := parts[1]

288 + key, value := parts[0], parts[1]

289 if key == "" || value == "" {

290 continue

291 }

@@ -403,36 +296,35 @@ func parseArgsParams(args []string) map[string][]string {

296

297 func paramValues(args map[string][]string, payload map[string]any, key string) []string {

298 if args != nil {

406 - if vals := args[key]; len(vals) > 0 {

407 - return vals

299 + if values := args[key]; len(values) > 0 {

300 + return values

301 }

302 }

303 return extractParamValues(payload, key)

304 }

305

413 -// extractParamValues extracts parameter values from payload, checking selections first.

306 func extractParamValues(payload map[string]any, key string) []string {

307 if payload == nil {

308 return nil

309 }

310 if selections, ok := payload["selections"].(map[string]any); ok {

419 - if vals := extractValues(selections[key]); len(vals) > 0 {

420 - return vals

311 + if values := extractValues(selections[key]); len(values) > 0 {

312 + return values

313 }

314 }

315 return extractValues(payload[key])

316 }

317

426 -func extractValues(val any) []string {

427 - switch v := val.(type) {

318 +func extractValues(value any) []string {

319 + switch current := value.(type) {

320 case string:

429 - if v == "" {

321 + if current == "" {

322 return nil

323 }

432 - return []string{v}

324 + return []string{current}

325 case []any:

326 var out []string

435 - for _, item := range v {

327 + for _, item := range current {

328 if s, ok := item.(string); ok && s != "" {

329 out = append(out, s)

330 }

@@ -440,9 +332,9 @@ func extractValues(val any) []string {

332 return out

333 case []string:

334 var out []string

443 - for _, s := range v {

444 - if s != "" {

445 - out = append(out, s)

335 + for _, item := range current {

336 + if item != "" {

337 + out = append(out, item)

338 }

339 }

340 return out

@@ -455,13 +347,14 @@ func splitCSV(value string) []string {

347 if !strings.Contains(value, ",") {

348 return []string{value}

349 }

350 +

351 parts := strings.Split(value, ",")

352 out := make([]string, 0, len(parts))

460 - for _, p := range parts {

461 - if p == "" {

353 + for _, part := range parts {

354 + if part == "" {

355 continue

356 }

464 - out = append(out, p)

357 + out = append(out, part)

358 }

359 return out

360 }

@@ -475,17 +368,18 @@ func buildJobParamConfig(jobs []string) funcapi.ParamConfig {

368 Disabled: true,

369 })

370 } else {

478 - for i, j := range jobs {

479 - opt := funcapi.ParamOption{

480 - ID: j,

481 - Name: j,

371 + for i, job := range jobs {

372 + option := funcapi.ParamOption{

373 + ID: job,

374 + Name: job,

375 }

376 if i == 0 {

484 - opt.Default = true

377 + option.Default = true

378 }

486 - options = append(options, opt)

379 + options = append(options, option)

380 }

381 }

382 +

383 return funcapi.ParamConfig{

384 ID: paramJob,

385 Name: "Instance",

@@ -498,42 +392,37 @@ func buildJobParamConfig(jobs []string) funcapi.ParamConfig {

392

393 func buildAcceptedParams(methodParams []funcapi.ParamConfig) []string {

394 accepted := []string{paramJob}

501 - for _, p := range methodParams {

502 - if !slices.Contains(accepted, p.ID) {

503 - accepted = append(accepted, p.ID)

395 + for _, param := range methodParams {

396 + if !slices.Contains(accepted, param.ID) {

397 + accepted = append(accepted, param.ID)

398 }

399 }

400 return accepted

401 }

402

509 -// makeJobMethodFuncHandler creates a function handler for a job-specific method.

510 -// Unlike makeMethodFuncHandler, this handler routes directly to a specific job

511 -// without needing the __job parameter (the job is known from the function name).

512 -func (m *Manager) makeJobMethodFuncHandler(moduleName, jobName, methodID string) func(functions.Function) {

403 +func (c *Controller) makeJobMethodFuncHandler(moduleName, jobName, methodID string) func(functions.Function) {

404 return func(fn functions.Function) {

514 - // Check for "info" request

405 if slices.Contains(fn.Args, "info") {

516 - m.handleJobMethodFuncInfo(moduleName, jobName, methodID, fn)

406 + c.handleJobMethodFuncInfo(moduleName, jobName, methodID, fn)

407 return

408 }

409

520 - methodCfg, ok := m.moduleFuncs.getJobMethod(moduleName, jobName, methodID)

410 + methodCfg, ok := c.registry.getJobMethod(moduleName, jobName, methodID)

411 if !ok {

522 - m.respondError(fn, 404, "unknown method '%s' for job '%s:%s'", methodID, moduleName, jobName)

412 + c.respondError(fn, 404, "unknown method '%s' for job '%s:%s'", methodID, moduleName, jobName)

413 return

414 }

415

416 payload := parsePayload(fn.Payload)

417 argValues := parseArgsParams(fn.Args)

418

529 - // Get job WITH generation for race condition detection

530 - job, jobGen := m.moduleFuncs.getJobWithGeneration(moduleName, jobName)

419 + job, jobGen := c.registry.getJobWithGeneration(moduleName, jobName)

420 if job == nil {

532 - m.respondError(fn, 503, "job '%s:%s' is not running", moduleName, jobName)

421 + c.respondError(fn, 503, "job '%s:%s' is not running", moduleName, jobName)

422 return

423 }

424

536 - m.executeMethodRequest(methodExecutionInput{

425 + c.executeMethodRequest(methodExecutionInput{

426 fn: fn,

427 moduleName: moduleName,

428 jobName: jobName,

@@ -545,20 +434,19 @@ func (m *Manager) makeJobMethodFuncHandler(moduleName, jobName, methodID string)

434 payload: payload,

435 argValues: argValues,

436 resolveParams: func(ctx context.Context, methodCfg *funcapi.MethodConfig, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error) {

548 - return m.resolveJobMethodParams(ctx, methodCfg, handler, methodID)

437 + return c.resolveJobMethodParams(ctx, methodCfg, handler, methodID)

438 },

439 respond: func(dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

551 - m.respondJobMethodWithParams(fn, dataResp, methodParams, updateEvery)

440 + c.respondJobMethodWithParams(fn, dataResp, methodParams, updateEvery)

441 },

442 })

443 }

444 }

445

557 -// handleJobMethodFuncInfo handles "info" requests for a job-specific method

558 -func (m *Manager) handleJobMethodFuncInfo(moduleName, jobName, methodID string, fn functions.Function) {

559 - methodCfg, ok := m.moduleFuncs.getJobMethod(moduleName, jobName, methodID)

446 +func (c *Controller) handleJobMethodFuncInfo(moduleName, jobName, methodID string, fn functions.Function) {

447 + methodCfg, ok := c.registry.getJobMethod(moduleName, jobName, methodID)

448 if !ok {

561 - m.respondError(fn, 404, "unknown method '%s' for job '%s:%s'", methodID, moduleName, jobName)

449 + c.respondError(fn, 404, "unknown method '%s' for job '%s:%s'", methodID, moduleName, jobName)

450 return

451 }

452

@@ -573,7 +461,7 @@ func (m *Manager) handleJobMethodFuncInfo(moduleName, jobName, methodID string,

461 updateEvery = methodCfg.UpdateEvery

462 }

463

576 - resp := map[string]any{

464 + c.respondJSON(fn, map[string]any{

465 "v": 3,

466 "update_every": updateEvery,

467 "status": 200,

@@ -582,13 +470,10 @@ func (m *Manager) handleJobMethodFuncInfo(moduleName, jobName, methodID string,

470 "help": help,

471 "accepted_params": buildJobMethodAcceptedParams(methodParams),

472 "required_params": buildJobMethodRequiredParams(methodParams),

585 - }

586 -

587 - m.respondJSON(fn, resp)

473 + })

474 }

475

590 -// resolveJobMethodParams resolves method parameters for a job-specific method

591 -func (m *Manager) resolveJobMethodParams(ctx context.Context, methodCfg *funcapi.MethodConfig, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error) {

476 +func (c *Controller) resolveJobMethodParams(ctx context.Context, methodCfg *funcapi.MethodConfig, handler funcapi.MethodHandler, methodID string) ([]funcapi.ParamConfig, bool, error) {

477 methodParams := methodCfg.RequiredParams

478

479 jobParams, err := handler.MethodParams(ctx, methodID)

@@ -602,30 +487,16 @@ func (m *Manager) resolveJobMethodParams(ctx context.Context, methodCfg *funcapi

487 return funcapi.MergeParamConfigs(methodParams, jobParams), true, nil

488 }

489

605 -// respondJobMethodWithParams wraps the module's data response for job-specific methods

606 -func (m *Manager) respondJobMethodWithParams(fn functions.Function, dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

607 - m.respondMethodDataWithParams(

608 - fn,

609 - dataResp,

610 - methodParams,

611 - updateEvery,

612 - buildJobMethodAcceptedParams,

613 - buildJobMethodRequiredParams,

614 - )

615 -}

616 -

617 -// buildJobMethodAcceptedParams creates accepted_params for job-specific methods (no __job)

490 func buildJobMethodAcceptedParams(methodParams []funcapi.ParamConfig) []string {

491 accepted := make([]string, 0, len(methodParams))

620 - for _, p := range methodParams {

621 - if !slices.Contains(accepted, p.ID) {

622 - accepted = append(accepted, p.ID)

492 + for _, param := range methodParams {

493 + if !slices.Contains(accepted, param.ID) {

494 + accepted = append(accepted, param.ID)

495 }

496 }

497 return accepted

498 }

499

628 -// buildJobMethodRequiredParams creates required_params for job-specific methods (no __job)

500 func buildJobMethodRequiredParams(methodParams []funcapi.ParamConfig) []map[string]any {

501 required := make([]map[string]any, 0, len(methodParams))

502 for _, cfg := range methodParams {

src/go/plugin/agent/jobmgr/funcctl/doc.go new

@@ -0,0 +1,6 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +// Package funcctl owns function and method dispatch state for jobmgr.

4 +// Runtime job lifecycle ownership remains in jobmgr, which feeds job start/stop

5 +// events into the controller.

6 +package funcctl

src/go/plugin/agent/jobmgr/funcctl/registry.go new

+298

@@ -0,0 +1,298 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package funcctl

4 +

5 +import (

6 + "sort"

7 + "sync"

8 +

9 + "github.com/netdata/netdata/go/plugins/pkg/funcapi"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

11 +)

12 +

13 +type moduleFuncRegistry struct {

14 + mu sync.RWMutex

15 + modules map[string]*moduleFunc

16 +}

17 +

18 +type moduleFunc struct {

19 + creator collectorapi.Creator

20 + methods []funcapi.MethodConfig

21 + methodsByID map[string]funcapi.MethodConfig

22 + jobs map[string]*jobEntry

23 + nextGeneration uint64

24 + jobMethods map[string][]funcapi.MethodConfig

25 +}

26 +

27 +type jobEntry struct {

28 + job collectorapi.RuntimeJob

29 + generation uint64

30 +}

31 +

32 +func newModuleFuncRegistry() *moduleFuncRegistry {

33 + return &moduleFuncRegistry{

34 + modules: make(map[string]*moduleFunc),

35 + }

36 +}

37 +

38 +func (r *moduleFuncRegistry) registerModule(name string, creator collectorapi.Creator) {

39 + r.mu.Lock()

40 + defer r.mu.Unlock()

41 +

42 + var methods []funcapi.MethodConfig

43 + if creator.Methods != nil {

44 + methods = creator.Methods()

45 + }

46 +

47 + r.modules[name] = &moduleFunc{

48 + creator: creator,

49 + methods: methods,

50 + methodsByID: indexMethods(methods),

51 + jobs: make(map[string]*jobEntry),

52 + jobMethods: make(map[string][]funcapi.MethodConfig),

53 + }

54 +}

55 +

56 +func indexMethods(methods []funcapi.MethodConfig) map[string]funcapi.MethodConfig {

57 + if len(methods) == 0 {

58 + return nil

59 + }

60 +

61 + idx := make(map[string]funcapi.MethodConfig, len(methods))

62 + for _, method := range methods {

63 + if method.ID == "" {

64 + continue

65 + }

66 + idx[method.ID] = method

67 + }

68 + return idx

69 +}

70 +

71 +func (r *moduleFuncRegistry) addJob(moduleName, jobName string, job collectorapi.RuntimeJob) {

72 + r.mu.Lock()

73 + defer r.mu.Unlock()

74 +

75 + module, ok := r.modules[moduleName]

76 + if !ok {

77 + return

78 + }

79 +

80 + module.nextGeneration++

81 + newGen := module.nextGeneration

82 + module.jobs[jobName] = &jobEntry{

83 + job: job,

84 + generation: newGen,

85 + }

86 +}

87 +

88 +func (r *moduleFuncRegistry) removeJob(moduleName, jobName string) {

89 + r.mu.Lock()

90 + defer r.mu.Unlock()

91 +

92 + if module, ok := r.modules[moduleName]; ok {

93 + delete(module.jobs, jobName)

94 + }

95 +}

96 +

97 +func (r *moduleFuncRegistry) getJobWithGeneration(moduleName, jobName string) (collectorapi.RuntimeJob, uint64) {

98 + r.mu.RLock()

99 + defer r.mu.RUnlock()

100 +

101 + module, ok := r.modules[moduleName]

102 + if !ok {

103 + return nil, 0

104 + }

105 + entry, ok := module.jobs[jobName]

106 + if !ok {

107 + return nil, 0

108 + }

109 + return entry.job, entry.generation

110 +}

111 +

112 +func (r *moduleFuncRegistry) verifyJobGeneration(moduleName, jobName string, expectedGen uint64) bool {

113 + r.mu.RLock()

114 + defer r.mu.RUnlock()

115 +

116 + module, ok := r.modules[moduleName]

117 + if !ok {

118 + return false

119 + }

120 + entry, ok := module.jobs[jobName]

121 + if !ok {

122 + return false

123 + }

124 + if !entry.job.IsRunning() {

125 + return false

126 + }

127 + return entry.generation == expectedGen

128 +}

129 +

130 +func (r *moduleFuncRegistry) getMethod(moduleName, methodID string) (*funcapi.MethodConfig, bool) {

131 + r.mu.RLock()

132 + defer r.mu.RUnlock()

133 +

134 + module, ok := r.modules[moduleName]

135 + if !ok || module.methodsByID == nil {

136 + return nil, false

137 + }

138 + cfg, ok := module.methodsByID[methodID]

139 + if !ok {

140 + return nil, false

141 + }

142 + return &cfg, true

143 +}

144 +

145 +func (r *moduleFuncRegistry) getMethods(moduleName string) []funcapi.MethodConfig {

146 + r.mu.RLock()

147 + defer r.mu.RUnlock()

148 +

149 + module, ok := r.modules[moduleName]

150 + if !ok {

151 + return nil

152 + }

153 + return module.methods

154 +}

155 +

156 +func (r *moduleFuncRegistry) getJobNames(moduleName string) []string {

157 + r.mu.RLock()

158 + defer r.mu.RUnlock()

159 +

160 + module, ok := r.modules[moduleName]

161 + if !ok {

162 + return nil

163 + }

164 +

165 + names := make([]string, 0, len(module.jobs))

166 + for name := range module.jobs {

167 + names = append(names, name)

168 + }

169 + sort.Strings(names)

170 + return names

171 +}

172 +

173 +func (r *moduleFuncRegistry) getJob(moduleName, jobName string) (collectorapi.RuntimeJob, bool) {

174 + r.mu.RLock()

175 + defer r.mu.RUnlock()

176 +

177 + module, ok := r.modules[moduleName]

178 + if !ok {

179 + return nil, false

180 + }

181 + entry, ok := module.jobs[jobName]

182 + if !ok {

183 + return nil, false

184 + }

185 + return entry.job, true

186 +}

187 +

188 +func (r *moduleFuncRegistry) getCreator(moduleName string) (collectorapi.Creator, bool) {

189 + r.mu.RLock()

190 + defer r.mu.RUnlock()

191 +

192 + module, ok := r.modules[moduleName]

193 + if !ok {

194 + return collectorapi.Creator{}, false

195 + }

196 + return module.creator, true

197 +}

198 +

199 +func (r *moduleFuncRegistry) isModuleRegistered(moduleName string) bool {

200 + r.mu.RLock()

201 + defer r.mu.RUnlock()

202 +

203 + _, ok := r.modules[moduleName]

204 + return ok

205 +}

206 +

207 +func (r *moduleFuncRegistry) registerJobMethods(moduleName, jobName string, methods []funcapi.MethodConfig) {

208 + r.mu.Lock()

209 + defer r.mu.Unlock()

210 +

211 + module, ok := r.modules[moduleName]

212 + if !ok {

213 + return

214 + }

215 + module.jobMethods[jobName] = methods

216 +}

217 +

218 +func (r *moduleFuncRegistry) unregisterJobMethods(moduleName, jobName string) {

219 + r.mu.Lock()

220 + defer r.mu.Unlock()

221 +

222 + module, ok := r.modules[moduleName]

223 + if !ok {

224 + return

225 + }

226 + delete(module.jobMethods, jobName)

227 +}

228 +

229 +func (r *moduleFuncRegistry) getJobMethods(moduleName, jobName string) []funcapi.MethodConfig {

230 + r.mu.RLock()

231 + defer r.mu.RUnlock()

232 +

233 + module, ok := r.modules[moduleName]

234 + if !ok {

235 + return nil

236 + }

237 + return module.jobMethods[jobName]

238 +}

239 +

240 +func (r *moduleFuncRegistry) getJobMethod(moduleName, jobName, methodID string) (*funcapi.MethodConfig, bool) {

241 + r.mu.RLock()

242 + defer r.mu.RUnlock()

243 +

244 + module, ok := r.modules[moduleName]

245 + if !ok {

246 + return nil, false

247 + }

248 + methods, ok := module.jobMethods[jobName]

249 + if !ok {

250 + return nil, false

251 + }

252 + for i := range methods {

253 + if methods[i].ID == methodID {

254 + return &methods[i], true

255 + }

256 + }

257 + return nil, false

258 +}

259 +

260 +func (r *moduleFuncRegistry) findMethodCollision(moduleName, jobName, methodID string) (string, bool) {

261 + r.mu.RLock()

262 + defer r.mu.RUnlock()

263 +

264 + module, ok := r.modules[moduleName]

265 + if !ok {

266 + return "", false

267 + }

268 +

269 + if module.methodsByID != nil {

270 + if _, exists := module.methodsByID[methodID]; exists {

271 + return "static method", true

272 + }

273 + }

274 +

275 + for ownerJob, methods := range module.jobMethods {

276 + if ownerJob == jobName {

277 + continue

278 + }

279 + for _, method := range methods {

280 + if method.ID == methodID {

281 + return "job method on " + ownerJob, true

282 + }

283 + }

284 + }

285 +

286 + return "", false

287 +}

288 +

289 +func (r *moduleFuncRegistry) snapshotCreators() map[string]collectorapi.Creator {

290 + r.mu.RLock()

291 + defer r.mu.RUnlock()

292 +

293 + out := make(map[string]collectorapi.Creator, len(r.modules))

294 + for name, module := range r.modules {

295 + out[name] = module.creator

296 + }

297 + return out

298 +}

src/go/plugin/agent/jobmgr/funcctl/response.go new

+135

@@ -0,0 +1,135 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package funcctl

4 +

5 +import (

6 + "encoding/json"

7 + "fmt"

8 +

9 + "github.com/netdata/netdata/go/plugins/pkg/funcapi"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

12 +)

13 +

14 +type methodResponseWriter func(dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int)

15 +

16 +func (c *Controller) respondWithParams(fn functions.Function, moduleName string, dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

17 + c.respondMethodDataWithParams(

18 + fn,

19 + dataResp,

20 + methodParams,

21 + updateEvery,

22 + buildAcceptedParams,

23 + func(params []funcapi.ParamConfig) []map[string]any {

24 + return c.buildRequiredParams(moduleName, params)

25 + },

26 + )

27 +}

28 +

29 +func (c *Controller) respondJobMethodWithParams(fn functions.Function, dataResp *funcapi.FunctionResponse, methodParams []funcapi.ParamConfig, updateEvery int) {

30 + c.respondMethodDataWithParams(

31 + fn,

32 + dataResp,

33 + methodParams,

34 + updateEvery,

35 + buildJobMethodAcceptedParams,

36 + buildJobMethodRequiredParams,

37 + )

38 +}

39 +

40 +func (c *Controller) respondMethodDataWithParams(

41 + fn functions.Function,

42 + dataResp *funcapi.FunctionResponse,

43 + methodParams []funcapi.ParamConfig,

44 + updateEvery int,

45 + buildAccepted func([]funcapi.ParamConfig) []string,

46 + buildRequired func([]funcapi.ParamConfig) []map[string]any,

47 +) {

48 + if dataResp == nil {

49 + c.respondError(fn, 500, "internal error: module returned nil response")

50 + return

51 + }

52 + if dataResp.Status >= 400 {

53 + c.respondError(fn, dataResp.Status, "%s", dataResp.Message)

54 + return

55 + }

56 +

57 + paramsForResponse := methodParams

58 + if len(dataResp.RequiredParams) > 0 {

59 + paramsForResponse = funcapi.MergeParamConfigs(paramsForResponse, dataResp.RequiredParams)

60 + }

61 +

62 + resp := map[string]any{

63 + "v": 3,

64 + "update_every": updateEvery,

65 + "status": dataResp.Status,

66 + "type": "table",

67 + "has_history": false,

68 + "help": dataResp.Help,

69 + "accepted_params": buildAccepted(paramsForResponse),

70 + "required_params": buildRequired(paramsForResponse),

71 + }

72 +

73 + if dataResp.Columns != nil {

74 + resp["columns"] = dataResp.Columns

75 + }

76 + if dataResp.Data != nil {

77 + resp["data"] = dataResp.Data

78 + }

79 + if dataResp.DefaultSortColumn != "" {

80 + resp["default_sort_column"] = dataResp.DefaultSortColumn

81 + }

82 + if len(dataResp.Charts) > 0 {

83 + resp["charts"] = dataResp.Charts

84 + }

85 + if len(dataResp.DefaultCharts) > 0 {

86 + resp["default_charts"] = dataResp.DefaultCharts.Build()

87 + }

88 + if len(dataResp.GroupBy) > 0 {

89 + resp["group_by"] = dataResp.GroupBy

90 + }

91 +

92 + c.respondJSON(fn, resp)

93 +}

94 +

95 +func (c *Controller) respondError(fn functions.Function, status int, format string, args ...any) {

96 + c.respondJSON(fn, map[string]any{

97 + "status": status,

98 + "errorMessage": fmt.Sprintf(format, args...),

99 + })

100 +}

101 +

102 +func (c *Controller) respondJSON(fn functions.Function, resp map[string]any) {

103 + data, err := json.Marshal(resp)

104 + if err != nil {

105 + c.Errorf("failed to marshal function response: %v", err)

106 + c.sendJSON(fn, string(functions.BuildJSONPayload(500, "internal error: failed to encode response")), 500)

107 + return

108 + }

109 +

110 + code := 200

111 + if status, ok := resp["status"]; ok {

112 + switch value := status.(type) {

113 + case int:

114 + code = value

115 + case int64:

116 + code = int(value)

117 + case float64:

118 + code = int(value)

119 + }

120 + }

121 +

122 + c.sendJSON(fn, string(data), code)

123 +}

124 +

125 +func (c *Controller) sendJSON(fn functions.Function, payload string, code int) {

126 + if c.jsonWriter != nil {

127 + c.jsonWriter([]byte(payload), code)

128 + return

129 + }

130 + if c.api == nil {

131 + return

132 + }

133 +

134 + c.api.SendJSONWithCode(dyncfg.NewFunction(fn), payload, code)

135 +}

src/go/plugin/agent/jobmgr/funcdispatch_test.go new

+644

@@ -0,0 +1,644 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "bytes"

7 + "context"

8 + "encoding/json"

9 + "sync"

10 + "testing"

11 + "time"

12 +

13 + "github.com/stretchr/testify/assert"

14 + "github.com/stretchr/testify/require"

15 +

16 + "github.com/netdata/netdata/go/plugins/pkg/funcapi"

17 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

18 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

19 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

20 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

21 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

22 +)

23 +

24 +type mockMethodHandler struct {

25 + paramsFunc func(ctx context.Context, method string) ([]funcapi.ParamConfig, error)

26 + handleFunc func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse

27 +}

28 +

29 +func (m *mockMethodHandler) MethodParams(ctx context.Context, method string) ([]funcapi.ParamConfig, error) {

30 + if m.paramsFunc != nil {

31 + return m.paramsFunc(ctx, method)

32 + }

33 + return nil, nil

34 +}

35 +

36 +func (m *mockMethodHandler) Handle(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

37 + if m.handleFunc != nil {

38 + return m.handleFunc(ctx, method, params)

39 + }

40 + return nil

41 +}

42 +

43 +func (m *mockMethodHandler) Cleanup(context.Context) {}

44 +

45 +func TestExecuteFunction_ModuleMethodPaths(t *testing.T) {

46 + tests := map[string]struct {

47 + methods []funcapi.MethodConfig

48 + fnArgs []string

49 + fnPayload map[string]any

50 + generationRace bool

51 + wantStatus int

52 + wantHelp string

53 + wantAccepted []string

54 + wantRequiredIDs []string

55 + wantDataValue any

56 + wantResolvedJob string

57 + wantErrorContains string

58 + }{

59 + "success with __job resolution": {

60 + methods: []funcapi.MethodConfig{{ID: "details", Help: "details help"}},

61 + fnArgs: []string{"__job:job1"},

62 + wantStatus: 200,

63 + wantHelp: "details help",

64 + wantAccepted: []string{"__job"},

65 + wantRequiredIDs: []string{"__job"},

66 + wantDataValue: "row",

67 + wantResolvedJob: "job1",

68 + },

69 + "info response includes module and method params": {

70 + methods: []funcapi.MethodConfig{{

71 + ID: "details",

72 + Help: "details help",

73 + RequiredParams: []funcapi.ParamConfig{{

74 + ID: "scope",

75 + Name: "Scope",

76 + Selection: funcapi.ParamSelect,

77 + Options: []funcapi.ParamOption{{ID: "default", Name: "Default"}},

78 + }},

79 + }},

80 + fnArgs: []string{"info"},

81 + wantStatus: 200,

82 + wantHelp: "details help",

83 + wantAccepted: []string{"__job", "scope"},

84 + wantRequiredIDs: []string{"__job", "scope"},

85 + },

86 + "generation race returns 503": {

87 + methods: []funcapi.MethodConfig{{ID: "details"}},

88 + fnArgs: []string{"__job:job1"},

89 + generationRace: true,

90 + wantStatus: 503,

91 + wantErrorContains: "replaced during request",

92 + },

93 + "explicit unknown __job in args returns 404": {

94 + methods: []funcapi.MethodConfig{{ID: "details"}},

95 + fnArgs: []string{"__job:missing"},

96 + wantStatus: 404,

97 + wantErrorContains: "unknown job 'missing'",

98 + },

99 + "explicit unknown __job in payload returns 404": {

100 + methods: []funcapi.MethodConfig{{ID: "details"}},

101 + fnPayload: map[string]any{"__job": "missing"},

102 + wantStatus: 404,

103 + wantErrorContains: "unknown job 'missing'",

104 + },

105 + "multiple __job values return 400": {

106 + methods: []funcapi.MethodConfig{{ID: "details"}},

107 + fnPayload: map[string]any{"__job": []string{"job1", "job2"}},

108 + wantStatus: 400,

109 + wantErrorContains: "parameter '__job' expects a single value",

110 + },

111 + }

112 +

113 + for name, tc := range tests {

114 + t.Run(name, func(t *testing.T) {

115 + writer := &jsonWriteCapture{}

116 + var gotJob string

117 + var mgr *Manager

118 +

119 + methodHandler := &mockMethodHandler{}

120 + if len(tc.fnArgs) == 0 || tc.fnArgs[0] != "info" {

121 + methodHandler.handleFunc = func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

122 + if tc.generationRace {

123 + mgr.funcCtl.OnJobStart(&lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"})

124 + return &funcapi.FunctionResponse{Status: 200, Help: "should be replaced"}

125 + }

126 +

127 + gotJob = params.GetOne("__job")

128 + return &funcapi.FunctionResponse{

129 + Status: 200,

130 + Help: tc.wantHelp,

131 + Columns: map[string]any{

132 + "value": map[string]any{"name": "Value"},

133 + },

134 + Data: [][]any{{tc.wantDataValue}},

135 + }

136 + }

137 + }

138 +

139 + mgr = newModuleDispatchTestManager(t, nil, writer.write, methodHandler, tc.methods)

140 + var payload []byte

141 + if tc.fnPayload != nil {

142 + var err error

143 + payload, err = json.Marshal(tc.fnPayload)

144 + require.NoError(t, err)

145 + }

146 + mgr.ExecuteFunction("mod:details", functions.Function{

147 + UID: "module-test",

148 + Timeout: time.Second,

149 + Args: tc.fnArgs,

150 + Payload: payload,

151 + })

152 +

153 + resp := writer.requireResponse(t)

154 + assert.Equal(t, tc.wantStatus, writer.code)

155 + assert.Equal(t, float64(tc.wantStatus), resp["status"])

156 + if tc.wantErrorContains != "" {

157 + assert.Contains(t, resp["errorMessage"], tc.wantErrorContains)

158 + return

159 + }

160 +

161 + if tc.wantResolvedJob != "" {

162 + assert.Equal(t, tc.wantResolvedJob, gotJob)

163 + }

164 + if tc.wantHelp != "" {

165 + assert.Equal(t, tc.wantHelp, resp["help"])

166 + }

167 + assert.Equal(t, tc.wantAccepted, jsonArrayStrings(t, resp["accepted_params"]))

168 +

169 + required := jsonObjectArray(t, resp["required_params"])

170 + require.Len(t, required, len(tc.wantRequiredIDs))

171 + for i, id := range tc.wantRequiredIDs {

172 + assert.Equal(t, id, required[i]["id"])

173 + }

174 + if tc.wantDataValue != nil {

175 + assert.Equal(t, tc.wantDataValue, jsonNestedArrayValue(t, resp["data"], 0, 0))

176 + }

177 + })

178 + }

179 +}

180 +

181 +func TestExecuteFunction_ContextBehavior(t *testing.T) {

182 + tests := map[string]struct {

183 + managerCtx context.Context

184 + wantMarker string

185 + wantMarkerSet bool

186 + wantHasDeadline bool

187 + }{

188 + "uses background fallback before manager context is set": {

189 + wantHasDeadline: true,

190 + },

191 + "uses manager context when available": {

192 + managerCtx: context.WithValue(context.Background(), dispatchContextKey("marker"), "manager"),

193 + wantMarker: "manager",

194 + wantMarkerSet: true,

195 + wantHasDeadline: true,

196 + },

197 + }

198 +

199 + for name, tc := range tests {

200 + t.Run(name, func(t *testing.T) {

201 + writer := &jsonWriteCapture{}

202 + mgr := newModuleDispatchTestManager(t, nil, writer.write, &mockMethodHandler{

203 + handleFunc: func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

204 + require.NotNil(t, ctx)

205 + _, hasDeadline := ctx.Deadline()

206 + assert.Equal(t, tc.wantHasDeadline, hasDeadline)

207 + gotMarker := ctx.Value(dispatchContextKey("marker"))

208 + if tc.wantMarkerSet {

209 + assert.Equal(t, tc.wantMarker, gotMarker)

210 + } else {

211 + assert.Nil(t, gotMarker)

212 + }

213 + return &funcapi.FunctionResponse{Status: 200}

214 + },

215 + }, []funcapi.MethodConfig{{ID: "details"}})

216 + if tc.managerCtx != nil {

217 + mgr.funcCtl.Init(tc.managerCtx)

218 + }

219 +

220 + mgr.ExecuteFunction("mod:details", functions.Function{

221 + UID: "module-context",

222 + Timeout: time.Second,

223 + Args: []string{"__job:job1"},

224 + })

225 +

226 + resp := writer.requireResponse(t)

227 + assert.Equal(t, float64(200), resp["status"])

228 + })

229 + }

230 +}

231 +

232 +func TestJobMethodRegisteredHandlerPaths(t *testing.T) {

233 + tests := map[string]struct {

234 + fnArgs []string

235 + wantRequiredLen int

236 + wantDataValue any

237 + }{

238 + "success path omits __job": {

239 + wantRequiredLen: 0,

240 + wantDataValue: "job1",

241 + },

242 + "info path omits __job": {

243 + fnArgs: []string{"info"},

244 + wantRequiredLen: 0,

245 + },

246 + }

247 +

248 + for name, tc := range tests {

249 + t.Run(name, func(t *testing.T) {

250 + writer := &jsonWriteCapture{}

251 + fnReg := newCapturingFunctionRegistry()

252 + mgr := newJobMethodDispatchTestManager(t, fnReg, writer.write, &mockMethodHandler{

253 + handleFunc: func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

254 + return &funcapi.FunctionResponse{

255 + Status: 200,

256 + Help: "job details help",

257 + Columns: map[string]any{

258 + "value": map[string]any{"name": "Value"},

259 + },

260 + Data: [][]any{{"job1"}},

261 + }

262 + },

263 + }, []funcapi.MethodConfig{{ID: "job-details", Help: "job details help"}})

264 +

265 + handler := fnReg.requireHandler(t, "mod:job-details")

266 + handler(functions.Function{

267 + UID: "job-handler",

268 + Timeout: time.Second,

269 + Args: tc.fnArgs,

270 + })

271 +

272 + resp := writer.requireResponse(t)

273 + assert.Equal(t, float64(200), resp["status"])

274 + assert.NotContains(t, jsonArrayStrings(t, resp["accepted_params"]), "__job")

275 + assert.Len(t, jsonObjectArray(t, resp["required_params"]), tc.wantRequiredLen)

276 + if tc.wantDataValue != nil {

277 + assert.Equal(t, tc.wantDataValue, jsonNestedArrayValue(t, resp["data"], 0, 0))

278 + }

279 + mgr.stopRunningJob("mod_job1")

280 + })

281 + }

282 +}

283 +

284 +func TestFunctionDispatch_ResponsePaths(t *testing.T) {

285 + tests := map[string]struct {

286 + useJSONWriter bool

287 + rebindResponder bool

288 + nilRebindResponder bool

289 + marshalFail bool

290 + wantWriterStatus int

291 + wantResponderUID string

292 + wantResponderJSON string

293 + wantFirstUID string

294 + wantSecondUID string

295 + }{

296 + "JSONWriter takes precedence when configured": {

297 + useJSONWriter: true,

298 + wantWriterStatus: 200,

299 + wantResponderUID: "writer-first",

300 + },

301 + "responder fallback is used when JSONWriter is nil": {

302 + wantResponderUID: "responder-fallback",

303 + wantResponderJSON: "\"status\":200",

304 + },

305 + "responder rebinding updates only the responder-backed path": {

306 + rebindResponder: true,

307 + wantFirstUID: "before-rebind",

308 + wantSecondUID: "after-rebind",

309 + },

310 + "nil responder rebinding preserves the current responder-backed path": {

311 + nilRebindResponder: true,

312 + wantFirstUID: "before-nil-rebind",

313 + wantSecondUID: "after-nil-rebind",

314 + },

315 + "marshal failure falls back to JSONWriter with 500": {

316 + useJSONWriter: true,

317 + marshalFail: true,

318 + wantWriterStatus: 500,

319 + wantResponderUID: "writer-marshal-fail",

320 + },

321 + "marshal failure falls back to responder with 500": {

322 + marshalFail: true,

323 + wantResponderUID: "responder-marshal-fail",

324 + wantResponderJSON: "\"status\":500",

325 + },

326 + }

327 +

328 + for name, tc := range tests {

329 + t.Run(name, func(t *testing.T) {

330 + handler := &mockMethodHandler{

331 + handleFunc: func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

332 + if tc.marshalFail {

333 + return &funcapi.FunctionResponse{

334 + Status: 200,

335 + Data: [][]any{{make(chan int)}},

336 + }

337 + }

338 + return &funcapi.FunctionResponse{Status: 200}

339 + },

340 + }

341 +

342 + if tc.useJSONWriter {

343 + writer := &jsonWriteCapture{}

344 + var responderOut bytes.Buffer

345 +

346 + mgr := newModuleDispatchTestManager(t, dyncfg.NewResponder(netdataapi.New(safewriter.New(&responderOut))), writer.write, handler, []funcapi.MethodConfig{{ID: "details"}})

347 + mgr.ExecuteFunction("mod:details", functions.Function{

348 + UID: tc.wantResponderUID,

349 + Timeout: time.Second,

350 + Args: []string{"__job:job1"},

351 + })

352 +

353 + resp := writer.requireResponse(t)

354 + assert.Equal(t, float64(tc.wantWriterStatus), resp["status"])

355 + if tc.marshalFail {

356 + assert.Contains(t, resp["errorMessage"], "failed to encode response")

357 + }

358 + assert.NotContains(t, responderOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantResponderUID)

359 + return

360 + }

361 +

362 + if tc.rebindResponder {

363 + var firstOut bytes.Buffer

364 + var secondOut bytes.Buffer

365 +

366 + mgr := newModuleDispatchTestManager(t, dyncfg.NewResponder(netdataapi.New(safewriter.New(&firstOut))), nil, handler, []funcapi.MethodConfig{{ID: "details"}})

367 + mgr.ExecuteFunction("mod:details", functions.Function{

368 + UID: tc.wantFirstUID,

369 + Timeout: time.Second,

370 + Args: []string{"__job:job1"},

371 + })

372 +

373 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&secondOut))))

374 + mgr.ExecuteFunction("mod:details", functions.Function{

375 + UID: tc.wantSecondUID,

376 + Timeout: time.Second,

377 + Args: []string{"__job:job1"},

378 + })

379 +

380 + assert.Contains(t, firstOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantFirstUID)

381 + assert.NotContains(t, firstOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantSecondUID)

382 + assert.Contains(t, secondOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantSecondUID)

383 + return

384 + }

385 +

386 + if tc.nilRebindResponder {

387 + var responderOut bytes.Buffer

388 +

389 + mgr := newModuleDispatchTestManager(t, dyncfg.NewResponder(netdataapi.New(safewriter.New(&responderOut))), nil, handler, []funcapi.MethodConfig{{ID: "details"}})

390 + mgr.ExecuteFunction("mod:details", functions.Function{

391 + UID: tc.wantFirstUID,

392 + Timeout: time.Second,

393 + Args: []string{"__job:job1"},

394 + })

395 +

396 + mgr.SetDyncfgResponder(nil)

397 + mgr.ExecuteFunction("mod:details", functions.Function{

398 + UID: tc.wantSecondUID,

399 + Timeout: time.Second,

400 + Args: []string{"__job:job1"},

401 + })

402 +

403 + assert.Contains(t, responderOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantFirstUID)

404 + assert.Contains(t, responderOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantSecondUID)

405 + return

406 + }

407 +

408 + var responderOut bytes.Buffer

409 + mgr := newModuleDispatchTestManager(t, dyncfg.NewResponder(netdataapi.New(safewriter.New(&responderOut))), nil, handler, []funcapi.MethodConfig{{ID: "details"}})

410 + mgr.ExecuteFunction("mod:details", functions.Function{

411 + UID: tc.wantResponderUID,

412 + Timeout: time.Second,

413 + Args: []string{"__job:job1"},

414 + })

415 +

416 + assert.Contains(t, responderOut.String(), "FUNCTION_RESULT_BEGIN "+tc.wantResponderUID)

417 + assert.Contains(t, responderOut.String(), tc.wantResponderJSON)

418 + if tc.marshalFail {

419 + assert.Contains(t, responderOut.String(), "failed to encode response")

420 + }

421 + })

422 + }

423 +}

424 +

425 +func TestCleanup_UnregistersStaticFunctionsBeforeStoppingJobs(t *testing.T) {

426 + fnReg := newCapturingFunctionRegistry()

427 + mgr := New(Config{PluginName: testPluginName, FnReg: fnReg})

428 +

429 + staticCreator := collectorapi.Creator{

430 + Methods: func() []funcapi.MethodConfig {

431 + return []funcapi.MethodConfig{{ID: "static-method"}}

432 + },

433 + }

434 + jobCreator := collectorapi.Creator{

435 + JobMethods: func(_ collectorapi.RuntimeJob) []funcapi.MethodConfig {

436 + return []funcapi.MethodConfig{{ID: "job-method"}}

437 + },

438 + }

439 +

440 + mgr.modules = collectorapi.Registry{

441 + "staticmod": staticCreator,

442 + "jobmod": jobCreator,

443 + }

444 + mgr.funcCtl.RegisterModules(mgr.modules)

445 +

446 + mgr.startRunningJob(&lockProbeJob{fullName: "staticmod_job1", moduleName: "staticmod", name: "job1"})

447 + mgr.startRunningJob(&lockProbeJob{fullName: "jobmod_job1", moduleName: "jobmod", name: "job1"})

448 +

449 + mgr.cleanup()

450 +

451 + unregistered := fnReg.unregisteredNames()

452 + assert.Contains(t, unregistered, "staticmod:static-method")

453 + assert.Contains(t, unregistered, "jobmod:job-method")

454 + assert.Less(

455 + t,

456 + fnReg.unregisteredIndex("staticmod:static-method"),

457 + fnReg.unregisteredIndex("jobmod:job-method"),

458 + "static module cleanup must run before per-job stop cleanup",

459 + )

460 +}

461 +

462 +type dispatchContextKey string

463 +

464 +type jsonWriteCapture struct {

465 + calls int

466 + code int

467 + raw []byte

468 +}

469 +

470 +func (c *jsonWriteCapture) write(payload []byte, code int) {

471 + c.calls++

472 + c.code = code

473 + c.raw = append([]byte(nil), payload...)

474 +}

475 +

476 +func (c *jsonWriteCapture) requireResponse(t *testing.T) map[string]any {

477 + t.Helper()

478 + require.Equal(t, 1, c.calls, "expected exactly one JSON writer call")

479 +

480 + var resp map[string]any

481 + require.NoError(t, json.Unmarshal(c.raw, &resp))

482 + return resp

483 +}

484 +

485 +type capturingFunctionRegistry struct {

486 + mu sync.Mutex

487 + handlers map[string]func(functions.Function)

488 + unregistered []string

489 +}

490 +

491 +func newCapturingFunctionRegistry() *capturingFunctionRegistry {

492 + return &capturingFunctionRegistry{

493 + handlers: make(map[string]func(functions.Function)),

494 + }

495 +}

496 +

497 +func (r *capturingFunctionRegistry) Register(name string, fn func(functions.Function)) {

498 + r.mu.Lock()

499 + r.handlers[name] = fn

500 + r.mu.Unlock()

501 +}

502 +

503 +func (r *capturingFunctionRegistry) Unregister(name string) {

504 + r.mu.Lock()

505 + r.unregistered = append(r.unregistered, name)

506 + delete(r.handlers, name)

507 + r.mu.Unlock()

508 +}

509 +

510 +func (r *capturingFunctionRegistry) RegisterPrefix(string, string, func(functions.Function)) {}

511 +func (r *capturingFunctionRegistry) UnregisterPrefix(string, string) {}

512 +

513 +func (r *capturingFunctionRegistry) requireHandler(t *testing.T, name string) func(functions.Function) {

514 + t.Helper()

515 + r.mu.Lock()

516 + defer r.mu.Unlock()

517 +

518 + handler, ok := r.handlers[name]

519 + require.True(t, ok, "handler %q was not registered", name)

520 + return handler

521 +}

522 +

523 +func (r *capturingFunctionRegistry) unregisteredNames() []string {

524 + r.mu.Lock()

525 + defer r.mu.Unlock()

526 +

527 + out := make([]string, len(r.unregistered))

528 + copy(out, r.unregistered)

529 + return out

530 +}

531 +

532 +func (r *capturingFunctionRegistry) unregisteredIndex(name string) int {

533 + r.mu.Lock()

534 + defer r.mu.Unlock()

535 +

536 + for i, got := range r.unregistered {

537 + if got == name {

538 + return i

539 + }

540 + }

541 + return -1

542 +}

543 +

544 +func newModuleDispatchTestManager(

545 + t *testing.T,

546 + api *dyncfg.Responder,

547 + jsonWriter func([]byte, int),

548 + methodHandler funcapi.MethodHandler,

549 + methods []funcapi.MethodConfig,

550 +) *Manager {

551 + t.Helper()

552 +

553 + mgr := New(Config{

554 + PluginName: testPluginName,

555 + FunctionJSONWriter: jsonWriter,

556 + })

557 + if api != nil {

558 + mgr.SetDyncfgResponder(api)

559 + }

560 +

561 + creator := collectorapi.Creator{

562 + Methods: func() []funcapi.MethodConfig { return methods },

563 + MethodHandler: func(job collectorapi.RuntimeJob) funcapi.MethodHandler {

564 + return methodHandler

565 + },

566 + }

567 + mgr.modules = collectorapi.Registry{"mod": creator}

568 + mgr.funcCtl.RegisterModules(mgr.modules)

569 + mgr.funcCtl.OnJobStart(&lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"})

570 +

571 + return mgr

572 +}

573 +

574 +func newJobMethodDispatchTestManager(

575 + t *testing.T,

576 + fnReg FunctionRegistry,

577 + jsonWriter func([]byte, int),

578 + methodHandler funcapi.MethodHandler,

579 + methods []funcapi.MethodConfig,

580 +) *Manager {

581 + t.Helper()

582 +

583 + mgr := New(Config{

584 + PluginName: testPluginName,

585 + FnReg: fnReg,

586 + FunctionJSONWriter: jsonWriter,

587 + })

588 +

589 + creator := collectorapi.Creator{

590 + JobMethods: func(_ collectorapi.RuntimeJob) []funcapi.MethodConfig { return methods },

591 + MethodHandler: func(job collectorapi.RuntimeJob) funcapi.MethodHandler {

592 + return methodHandler

593 + },

594 + }

595 + mgr.modules = collectorapi.Registry{"mod": creator}

596 + mgr.funcCtl.RegisterModules(mgr.modules)

597 + mgr.startRunningJob(&lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"})

598 +

599 + return mgr

600 +}

601 +

602 +func jsonArrayStrings(t *testing.T, raw any) []string {

603 + t.Helper()

604 +

605 + items, ok := raw.([]any)

606 + require.True(t, ok, "expected []any, got %T", raw)

607 +

608 + out := make([]string, 0, len(items))

609 + for _, item := range items {

610 + s, ok := item.(string)

611 + require.True(t, ok, "expected string item, got %T", item)

612 + out = append(out, s)

613 + }

614 + return out

615 +}

616 +

617 +func jsonObjectArray(t *testing.T, raw any) []map[string]any {

618 + t.Helper()

619 +

620 + items, ok := raw.([]any)

621 + require.True(t, ok, "expected []any, got %T", raw)

622 +

623 + out := make([]map[string]any, 0, len(items))

624 + for _, item := range items {

625 + obj, ok := item.(map[string]any)

626 + require.True(t, ok, "expected map[string]any item, got %T", item)

627 + out = append(out, obj)

628 + }

629 + return out

630 +}

631 +

632 +func jsonNestedArrayValue(t *testing.T, raw any, row, col int) any {

633 + t.Helper()

634 +

635 + rows, ok := raw.([]any)

636 + require.True(t, ok, "expected []any rows, got %T", raw)

637 + require.Len(t, rows, row+1)

638 +

639 + cols, ok := rows[row].([]any)

640 + require.True(t, ok, "expected []any columns, got %T", rows[row])

641 + require.Len(t, cols, col+1)

642 +

643 + return cols[col]

644 +}

src/go/plugin/agent/jobmgr/funcshandler_test.go deleted

-180

@@ -1,180 +0,0 @@

1 -// SPDX-License-Identifier: GPL-3.0-or-later

2 -

3 -package jobmgr

4 -

5 -import (

6 - "context"

7 - "testing"

8 -

9 - "github.com/netdata/netdata/go/plugins/pkg/funcapi"

10 - "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

11 - "github.com/netdata/netdata/go/plugins/plugin/framework/jobruntime"

12 - "github.com/stretchr/testify/assert"

13 -)

14 -

15 -// mockMethodHandler implements funcapi.MethodHandler for testing.

16 -type mockMethodHandler struct {

17 - job *jobruntime.Job

18 - paramsFunc func(ctx context.Context, method string) ([]funcapi.ParamConfig, error)

19 - handleFunc func(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse

20 -}

21 -

22 -func (m *mockMethodHandler) MethodParams(ctx context.Context, method string) ([]funcapi.ParamConfig, error) {

23 - if m.paramsFunc != nil {

24 - return m.paramsFunc(ctx, method)

25 - }

26 - return nil, nil

27 -}

28 -

29 -func (m *mockMethodHandler) Handle(ctx context.Context, method string, params funcapi.ResolvedParams) *funcapi.FunctionResponse {

30 - if m.handleFunc != nil {

31 - return m.handleFunc(ctx, method, params)

32 - }

33 - return nil

34 -}

35 -

36 -func (m *mockMethodHandler) Cleanup(ctx context.Context) {}

37 -

38 -func TestExtractParamValues(t *testing.T) {

39 - tests := map[string]struct {

40 - payload map[string]any

41 - key string

42 - expected []string

43 - }{

44 - "string value": {

45 - payload: map[string]any{"__job": "local"},

46 - key: "__job",

47 - expected: []string{"local"},

48 - },

49 - "array value (single element)": {

50 - payload: map[string]any{"__job": []any{"local"}},

51 - key: "__job",

52 - expected: []string{"local"},

53 - },

54 - "array value (multiple elements)": {

55 - payload: map[string]any{"__sort": []any{"calls", "total_time"}},

56 - key: "__sort",

57 - expected: []string{"calls", "total_time"},

58 - },

59 - "string array value": {

60 - payload: map[string]any{"__job": []string{"local"}},

61 - key: "__job",

62 - expected: []string{"local"},

63 - },

64 - "missing key": {

65 - payload: map[string]any{"__job": "local"},

66 - key: "__sort",

67 - expected: nil,

68 - },

69 - "empty payload": {

70 - payload: map[string]any{},

71 - key: "__job",

72 - expected: nil,

73 - },

74 - "nil in array": {

75 - payload: map[string]any{"__job": []any{nil, "test"}},

76 - key: "__job",

77 - expected: []string{"test"},

78 - },

79 - "empty array": {

80 - payload: map[string]any{"__job": []any{}},

81 - key: "__job",

82 - expected: nil,

83 - },

84 - "non-string value": {

85 - payload: map[string]any{"__job": 123},

86 - key: "__job",

87 - expected: nil,

88 - },

89 - "prefers selections": {

90 - payload: map[string]any{

91 - "__job": "root",

92 - "selections": map[string]any{

93 - "__job": []any{"selected"},

94 - },

95 - },

96 - key: "__job",

97 - expected: []string{"selected"},

98 - },

99 - }

100 -

101 - for name, tc := range tests {

102 - t.Run(name, func(t *testing.T) {

103 - result := extractParamValues(tc.payload, tc.key)

104 - assert.Equal(t, tc.expected, result)

105 - })

106 - }

107 -}

108 -

109 -func TestBuildParams(t *testing.T) {

110 - tests := map[string]struct {

111 - run func(t *testing.T)

112 - }{

113 - "build accepted params": {

114 - run: func(t *testing.T) {

115 - sortDir := funcapi.FieldSortDescending

116 - methodParams := []funcapi.ParamConfig{

117 - {ID: "__sort", Selection: funcapi.ParamSelect, Options: []funcapi.ParamOption{{ID: "calls", Name: "Calls", Sort: &sortDir}}},

118 - {ID: "db"},

119 - {ID: "extra"},

120 - }

121 -

122 - result := buildAcceptedParams(methodParams)

123 - assert.Equal(t, []string{"__job", "__sort", "db", "extra"}, result)

124 - },

125 - },

126 - "build required params uses select type": {

127 - run: func(t *testing.T) {

128 - // Setup a minimal manager with test data.

129 - r := newModuleFuncRegistry()

130 - r.registerModule("postgres", collectorapi.Creator{

131 - Methods: func() []funcapi.MethodConfig {

132 - return []funcapi.MethodConfig{{

133 - ID: "top-queries",

134 - Name: "Top Queries",

135 - }}

136 - },

137 - })

138 - r.addJob("postgres", "master-db", newTestModuleFuncsJob("master-db"))

139 -

140 - mgr := &Manager{moduleFuncs: r}

141 - methodParams := []funcapi.ParamConfig{

142 - {

143 - ID: "__sort",

144 - Name: "Filter By",

145 - Selection: funcapi.ParamSelect,

146 - UniqueView: true,

147 - Options: []funcapi.ParamOption{

148 - {ID: "total_time", Name: "By Total Time", Default: true},

149 - },

150 - },

151 - }

152 - params := mgr.buildRequiredParams("postgres", methodParams)

153 -

154 - assert.Len(t, params, 2, "should have 2 required params: __job, __sort")

155 - for _, param := range params {

156 - paramType, ok := param["type"]

157 - assert.True(t, ok, "param should have type field")

158 - assert.Equal(t, "select", paramType, "param type must be 'select' for single-select, not 'multiselect'")

159 -

160 - assert.Contains(t, param, "id", "param should have id")

161 - assert.Contains(t, param, "name", "param should have name")

162 - assert.Contains(t, param, "options", "param should have options")

163 - assert.Contains(t, param, "unique_view", "param should have unique_view")

164 -

165 - uniqueView, _ := param["unique_view"].(bool)

166 - assert.True(t, uniqueView, "unique_view should be true")

167 - }

168 -

169 - assert.Equal(t, "__job", params[0]["id"])

170 - assert.Equal(t, "__sort", params[1]["id"])

171 - },

172 - },

173 - }

174 -

175 - for name, tc := range tests {

176 - t.Run(name, func(t *testing.T) {

177 - tc.run(t)

178 - })

179 - }

180 -}

src/go/plugin/agent/jobmgr/job_factory.go

+44 -11

@@ -3,6 +3,7 @@

3 package jobmgr

5 import (

6 + "context"

7 "fmt"

8 "io"

9 "os"

@@ -10,6 +11,8 @@ import (

12 "github.com/netdata/netdata/go/plugins/logger"

13 "github.com/netdata/netdata/go/plugins/plugin/agent/internal/naming"

14 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/resolver"

15 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

16 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

17 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

18 "github.com/netdata/netdata/go/plugins/plugin/framework/jobruntime"

@@ -22,35 +25,51 @@ import (

25 type jobFactory struct {

26 logger *logger.Logger

25 - pluginName string

26 - modules collectorapi.Registry

27 - vnodes *vnodeStore

28 - out io.Writer

28 + pluginName string

29 + modules collectorapi.Registry

30 + vnodeLookup func(string) (*vnodes.VirtualNode, bool)

31 + out io.Writer

32 +

33 + validationOnly bool

35 auditMode bool

36 auditAnalyzer metricsaudit.Analyzer

37 auditDataDir string

39 runtimeService runtimecomp.Service

40 +

41 + secretResolver *secretresolver.Resolver

42 + secretStoreSvc secretstore.Service

43 + ctx context.Context

44 }

46 func newJobFactory(m *Manager) *jobFactory {

47 return &jobFactory{

48 logger: m.Logger,

41 - pluginName: m.pluginName,

42 - modules: m.modules,

43 - vnodes: m.vnodes,

44 - out: m.out,

50 + pluginName: m.pluginName,

51 + modules: m.modules,

52 + vnodeLookup: m.vnodesCtl.Lookup,

53 + out: m.out,

55 auditMode: m.auditMode,

56 auditAnalyzer: m.auditAnalyzer,

57 auditDataDir: m.auditDataDir,

59 runtimeService: m.runtimeService,

60 + secretResolver: m.secretResolver,

61 + secretStoreSvc: m.secretsCtl.Service(),

62 + ctx: m.baseContext(),

63 }

64 }

66 +func (f *jobFactory) validate(cfg confgroup.Config) error {

67 + clone := *f

68 + clone.validationOnly = true

69 + _, err := clone.create(cfg)

70 + return err

71 +}

72 +

73 func (f *jobFactory) create(cfg confgroup.Config) (runtimeJob, error) {

74 creator, ok := f.modules[cfg.Module()]

75 if !ok {

@@ -64,7 +83,10 @@ func (f *jobFactory) create(cfg confgroup.Config) (runtimeJob, error) {

84 var vnode *vnodes.VirtualNode

85 if cfg.Vnode() != "" {

67 - n, ok := f.vnodes.Lookup(cfg.Vnode())

86 + if f.vnodeLookup == nil {

87 + return nil, fmt.Errorf("vnode '%s' is not found", cfg.Vnode())

88 + }

89 + n, ok := f.vnodeLookup(cfg.Vnode())

90 if !ok || n == nil {

91 return nil, fmt.Errorf("vnode '%s' is not found", cfg.Vnode())

92 }

@@ -84,7 +106,11 @@ func (f *jobFactory) createV2(cfg confgroup.Config, creator collectorapi.Creator

106 if mod == nil {

107 return nil, fmt.Errorf("module %s CreateV2 returned nil", cfg.Module())

108 }

87 - if err := applyConfig(cfg, mod); err != nil {

109 + storeSnapshot := (*secretstore.Snapshot)(nil)

110 + if f.secretStoreSvc != nil {

111 + storeSnapshot = f.secretStoreSvc.Capture()

112 + }

113 + if err := applyConfig(f.ctx, cfg, mod, f.secretResolver, f.secretStoreSvc, storeSnapshot); err != nil {

114 return nil, err

115 }

116

@@ -119,7 +145,11 @@ func (f *jobFactory) createV1(cfg confgroup.Config, creator collectorapi.Creator

145 }

146

147 mod := creator.Create()

122 - if err := applyConfig(cfg, mod); err != nil {

148 + storeSnapshot := (*secretstore.Snapshot)(nil)

149 + if f.secretStoreSvc != nil {

150 + storeSnapshot = f.secretStoreSvc.Capture()

151 + }

152 + if err := applyConfig(f.ctx, cfg, mod, f.secretResolver, f.secretStoreSvc, storeSnapshot); err != nil {

153 return nil, err

154 }

155

@@ -156,6 +186,9 @@ func (f *jobFactory) createV1(cfg confgroup.Config, creator collectorapi.Creator

186 }

187

188 func (f *jobFactory) createV1CaptureDir(cfg confgroup.Config) (string, error) {

189 + if f.validationOnly {

190 + return "", nil

191 + }

192 if f.auditDataDir == "" {

193 return "", nil

194 }

src/go/plugin/agent/jobmgr/manager.go

+136 -294

@@ -12,10 +12,15 @@ import (

12 "time"

14 "github.com/netdata/netdata/go/plugins/logger"

15 - "github.com/netdata/netdata/go/plugins/pkg/funcapi"

15 "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

16 "github.com/netdata/netdata/go/plugins/pkg/ticker"

17 + "github.com/netdata/netdata/go/plugins/plugin/agent/jobmgr/funcctl"

18 + "github.com/netdata/netdata/go/plugins/plugin/agent/jobmgr/secretsctl"

19 + "github.com/netdata/netdata/go/plugins/plugin/agent/jobmgr/vnodectl"

20 "github.com/netdata/netdata/go/plugins/plugin/agent/policy"

21 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/resolver"

22 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

23 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

24 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

25 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

26 "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

@@ -23,7 +28,6 @@ import (

28 "github.com/netdata/netdata/go/plugins/plugin/framework/metricsaudit"

29 "github.com/netdata/netdata/go/plugins/plugin/framework/runtimecomp"

30 "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

26 - "gopkg.in/yaml.v2"

31 )

33 type Config struct {

@@ -36,6 +40,8 @@ type Config struct {

40 VarLibDir string

41 FnReg FunctionRegistry

42 Vnodes map[string]*vnodes.VirtualNode

43 + SecretStores []secretstore.Config

44 + SecretStoreService secretstore.Service

45 AuditMode bool

46 AuditAnalyzer metricsaudit.Analyzer

47 AuditDataDir string

@@ -72,32 +78,34 @@ func New(cfg Config) *Manager {

78 if vnodesReg == nil {

79 vnodesReg = make(map[string]*vnodes.VirtualNode)

80 }

81 + secretStoreSvc := cfg.SecretStoreService

82 + if secretStoreSvc == nil {

83 + storeCreators := backends.Creators()

84 + secretStoreSvc = secretstore.NewService(storeCreators...)

85 + }

87 mgr := &Manager{

88 Logger: logger.New().With(

89 slog.String("component", "job manager"),

90 ),

80 - pluginName: cfg.PluginName,

81 - out: out,

82 - runModePolicy: cfg.RunModePolicy,

83 - modules: cfg.Modules,

84 - runJob: cfg.RunJob,

85 - configDefaults: cfg.ConfigDefaults,

86 - varLibDir: cfg.VarLibDir,

87 - fnReg: fnReg,

88 - vnodes: newVnodeStore(vnodesReg),

89 -

90 - auditMode: cfg.AuditMode,

91 - auditAnalyzer: cfg.AuditAnalyzer,

92 - auditDataDir: cfg.AuditDataDir,

93 - functionJSONWriter: cfg.FunctionJSONWriter,

94 - runtimeService: cfg.RuntimeService,

95 -

96 - moduleFuncs: newModuleFuncRegistry(),

97 - staticMethodsSeen: make(map[string]struct{}),

91 + pluginName: cfg.PluginName,

92 + out: out,

93 + runModePolicy: cfg.RunModePolicy,

94 + modules: cfg.Modules,

95 + runJobNames: cfg.RunJob,

96 + configDefaults: cfg.ConfigDefaults,

97 + varLibDir: cfg.VarLibDir,

98 + fnReg: fnReg,

99 + initialSecretStores: append([]secretstore.Config(nil), cfg.SecretStores...),

100 +

101 + auditMode: cfg.AuditMode,

102 + auditAnalyzer: cfg.AuditAnalyzer,

103 + auditDataDir: cfg.AuditDataDir,

104 +

105 discoveredConfigs: newDiscoveredConfigsCache(),

99 - seen: seen,

100 - exposed: exposed,

106 + collectorSeen: seen,

107 + collectorExposed: exposed,

108 + secretStoreDeps: newSecretStoreDeps(),

109 runningJobs: newRunningJobsCache(),

110 retryingTasks: newRetryingTasksCache(),

111

@@ -106,16 +114,25 @@ func New(cfg Config) *Manager {

114 rmCh: make(chan confgroup.Config),

115 dyncfgCh: make(chan dyncfg.Function),

116 cmdTestSem: make(chan struct{}, cmdTestWorkerCap),

109 - dyncfgApi: api,

117 +

118 + dyncfgResponder: api,

119 + runtimeService: cfg.RuntimeService,

120 + secretResolver: secretresolver.New(),

121 }

122 + mgr.funcCtl = funcctl.New(funcctl.Options{

123 + Logger: mgr.Logger,

124 + FnReg: fnReg,

125 + API: api,

126 + JSONWriter: cfg.FunctionJSONWriter,

127 + })

128

112 - mgr.collectorCb = &collectorCallbacks{mgr: mgr}

113 - mgr.handler = dyncfg.NewHandler(dyncfg.HandlerOpts[confgroup.Config]{

129 + mgr.collectorCallbacks = &collectorCallbacks{mgr: mgr}

130 + mgr.collectorHandler = dyncfg.NewHandler(dyncfg.HandlerOpts[confgroup.Config]{

131 Logger: mgr.Logger,

132 API: api,

133 Seen: seen,

134 Exposed: exposed,

118 - Callbacks: mgr.collectorCb,

135 + Callbacks: mgr.collectorCallbacks,

136 WaitKey: func(cfg confgroup.Config) string {

137 return cfg.FullName()

138 },

@@ -135,51 +152,75 @@ func New(cfg Config) *Manager {

152 dyncfg.CommandUserconfig,

153 },

154 })

155 + mgr.vnodesCtl = vnodectl.New(vnodectl.Options{

156 + Logger: mgr.Logger,

157 + API: api,

158 + Plugin: cfg.PluginName,

159 + Initial: vnodesReg,

160 + AffectedJobs: mgr.affectedVnodeJobs,

161 + ApplyVnodeUpdate: mgr.applyVnodeUpdate,

162 + })

163 + mgr.secretsCtl = secretsctl.New(secretsctl.Options{

164 + Logger: mgr.Logger,

165 + API: mgr.dyncfgResponder,

166 + Service: secretStoreSvc,

167 + Plugin: mgr.pluginName,

168 + Initial: mgr.initialSecretStores,

169 + AffectedJobs: mgr.affectedJobs,

170 + RestartableAffectedJobs: mgr.restartableAffectedJobs,

171 + RestartDependentJobs: mgr.restartDependentJobs,

172 + })

173

174 return mgr

175 }

176

177 // SetDyncfgResponder allows overriding the default responder (e.g., to silence output in CLI mode).

178 func (m *Manager) SetDyncfgResponder(responder *dyncfg.Responder) {

144 - if responder != nil && m.dyncfgApi != nil {

145 - responder.SetTerminalFinalizer(m.dyncfgApi.TerminalFinalizer())

179 + if responder != nil && m.dyncfgResponder != nil {

180 + responder.SetTerminalFinalizer(m.dyncfgResponder.TerminalFinalizer())

181 }

147 - dyncfg.BindResponder(&m.dyncfgApi, m.handler, responder)

182 + dyncfg.BindResponder(&m.dyncfgResponder, m.collectorHandler, responder)

183 + m.secretsCtl.SetAPI(responder)

184 + m.vnodesCtl.SetAPI(responder)

185 + m.funcCtl.SetAPI(responder)

186 }

187

188 type Manager struct {

189 *logger.Logger

190

153 - pluginName string

154 - out io.Writer

155 - runModePolicy policy.RunModePolicy

156 - modules collectorapi.Registry

157 - runJob []string

158 - configDefaults confgroup.Registry

159 - varLibDir string

160 - fnReg FunctionRegistry

161 - vnodes *vnodeStore

191 + // Static configuration and injected dependencies.

192 + pluginName string

193 + out io.Writer

194 + runModePolicy policy.RunModePolicy

195 + modules collectorapi.Registry

196 + runJobNames []string

197 + configDefaults confgroup.Registry

198 + varLibDir string

199 + fnReg FunctionRegistry

200 + initialSecretStores []secretstore.Config

201

202 // Metrics-audit mode.

203 auditMode bool

204 auditAnalyzer metricsaudit.Analyzer

205 auditDataDir string

206

168 - fileStatus *fileStatus

169 - moduleFuncs *moduleFuncRegistry

170 - // staticMethodsSeen tracks modules whose static methods were already exposed.

171 - // Registration is delayed until the first started job for a module.

172 - staticMethodsSeen map[string]struct{}

173 -

207 + // Persistent caches and runtime state.

208 + fileStatus *fileStatus

209 discoveredConfigs *discoveredConfigs

175 - seen *dyncfg.SeenCache[confgroup.Config]

176 - exposed *dyncfg.ExposedCache[confgroup.Config]

210 + collectorSeen *dyncfg.SeenCache[confgroup.Config]

211 + collectorExposed *dyncfg.ExposedCache[confgroup.Config]

212 + secretStoreDeps *secretStoreDeps

213 retryingTasks *retryingTasks

214 runningJobs *runningJobs

215

180 - handler *dyncfg.Handler[confgroup.Config]

181 - collectorCb *collectorCallbacks

216 + // Controllers and handlers.

217 + funcCtl *funcctl.Controller

218 + collectorHandler *dyncfg.Handler[confgroup.Config]

219 + collectorCallbacks *collectorCallbacks

220 + secretsCtl *secretsctl.Controller

221 + vnodesCtl *vnodectl.Controller

222

223 + // Runtime loop state.

224 ctx context.Context

225 started chan struct{}

226 addCh chan confgroup.Config

@@ -188,41 +229,37 @@ type Manager struct {

229 cmdTestSem chan struct{}

230 cmdTestWG sync.WaitGroup

231

191 - dyncfgApi *dyncfg.Responder

192 -

193 - // FunctionJSONWriter, when set, bypasses Netdata protocol output and writes raw JSON.

194 - functionJSONWriter func(payload []byte, code int)

232 + // Shared service seams.

233 + dyncfgResponder *dyncfg.Responder

234

235 // RuntimeService is an optional runtime/internal metrics registration seam.

236 // When set, V2 jobs may register per-job runtime components.

237 runtimeService runtimecomp.Service

238 +

239 + secretResolver *secretresolver.Resolver

240 }

241

242 func (m *Manager) Run(ctx context.Context, in chan []*confgroup.Group) {

243 m.Info("instance is started")

244 defer func() { m.cleanup(); m.Info("instance is stopped") }()

245 m.ctx = ctx

246 + m.funcCtl.Init(ctx)

247

206 - m.fnReg.RegisterPrefix("config", m.dyncfgCollectorPrefixValue(), dyncfg.WrapHandler(m.dyncfgConfig))

207 - m.fnReg.RegisterPrefix("config", m.dyncfgVnodePrefixValue(), dyncfg.WrapHandler(m.dyncfgConfig))

248 + vnodePrefix := m.vnodesCtl.Prefix()

249 + m.fnReg.RegisterPrefix("config", vnodePrefix, dyncfg.WrapHandler(m.dyncfgConfig))

250 + m.fnReg.RegisterPrefix("config", m.dyncfgSecretStorePrefixValue(), dyncfg.WrapHandler(m.dyncfgConfig))

251

209 - m.dyncfgVnodeModuleCreate()

252 + m.vnodesCtl.CreateTemplates()

253 + m.vnodesCtl.PublishExisting(dyncfg.StatusRunning)

254

211 - m.vnodes.ForEach(func(cfg *vnodes.VirtualNode) bool {

212 - m.dyncfgVnodeJobCreate(cfg, dyncfg.StatusRunning)

213 - return true

214 - })

255 + m.secretsCtl.CreateTemplates()

256 + m.secretsCtl.PublishExisting()

257

216 - for name, creator := range m.modules {

258 + m.fnReg.RegisterPrefix("config", m.dyncfgCollectorPrefixValue(), dyncfg.WrapHandler(m.dyncfgConfig))

259 + for name := range m.modules {

260 m.dyncfgCollectorModuleCreate(name)

218 -

219 - // Register module if it provides static methods OR per-job methods

220 - if creator.Methods != nil || creator.JobMethods != nil {

221 - m.moduleFuncs.registerModule(name, creator)

222 - }

223 -

224 - // Note: Static module methods and per-job methods are registered in startRunningJob.

261 }

262 + m.funcCtl.RegisterModules(m.modules)

263

264 m.loadFileStatus()

265

@@ -258,18 +295,12 @@ func (m *Manager) WaitStarted(ctx context.Context) bool {

295

296 // GetJobNames returns the currently running job names for a module.

297 func (m *Manager) GetJobNames(moduleName string) []string {

261 - return m.moduleFuncs.getJobNames(moduleName)

298 + return m.funcCtl.GetJobNames(moduleName)

299 }

300

301 // ExecuteFunction executes a function handler directly (function name must be module:method).

302 func (m *Manager) ExecuteFunction(functionName string, fn functions.Function) {

266 - moduleName, methodID, err := functions.SplitFunctionName(functionName)

267 - if err != nil {

268 - m.respondError(fn, 400, "%v", err)

269 - return

270 - }

271 - handler := m.makeMethodFuncHandler(moduleName, methodID)

272 - handler(fn)

303 + m.funcCtl.ExecuteFunction(functionName, fn)

304 }

305

306 func (m *Manager) runProcessConfGroups(in chan []*confgroup.Group) {

@@ -284,9 +315,9 @@ func (m *Manager) runProcessConfGroups(in chan []*confgroup.Group) {

315 for _, gr := range groups {

316 a, r := m.discoveredConfigs.add(gr)

317 m.Debugf("received configs: %d/+%d/-%d ('%s')", len(gr.Configs), len(a), len(r), gr.Source)

287 - if len(m.runJob) > 0 {

318 + if len(m.runJobNames) > 0 {

319 a = slices.DeleteFunc(a, func(config confgroup.Config) bool {

289 - return !slices.ContainsFunc(m.runJob, func(name string) bool { return config.Name() == name })

320 + return !slices.ContainsFunc(m.runJobNames, func(name string) bool { return config.Name() == name })

321 })

322 }

323 sendConfigs(m.ctx, m.rmCh, r...)

@@ -298,8 +329,8 @@ func (m *Manager) runProcessConfGroups(in chan []*confgroup.Group) {

329

330 func (m *Manager) run() {

331 for {

301 - if m.handler.WaitingForDecision() {

302 - step, ok := m.handler.NextWaitDecisionStep(m.ctx, m.dyncfgCh)

332 + if m.collectorHandler.WaitingForDecision() {

333 + step, ok := m.collectorHandler.NextWaitDecisionStep(m.ctx, m.dyncfgCh)

334 if !ok {

335 return

336 }

@@ -337,11 +368,11 @@ func (m *Manager) addConfig(cfg confgroup.Config) {

368

369 m.retryingTasks.remove(cfg)

370

340 - m.handler.RememberDiscoveredConfig(cfg)

371 + m.collectorHandler.RememberDiscoveredConfig(cfg)

372

342 - entry, ok := m.exposed.LookupByKey(cfg.ExposedKey())

373 + entry, ok := m.collectorExposed.LookupByKey(cfg.ExposedKey())

374 if !ok {

344 - entry = m.handler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted)

375 + entry = m.collectorHandler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted)

376 } else {

377 sp, ep := cfg.SourceTypePriority(), entry.Cfg.SourceTypePriority()

378 if ep > sp || (ep == sp && entry.Status == dyncfg.StatusRunning) {

@@ -351,30 +382,33 @@ func (m *Manager) addConfig(cfg confgroup.Config) {

382 m.stopRunningJob(entry.Cfg.FullName())

383 m.fileStatus.remove(entry.Cfg)

384 }

354 - entry = m.handler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted) // replace existing exposed

385 + entry = m.collectorHandler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted) // replace existing exposed

386 }

387

357 - m.handler.NotifyJobCreate(entry.Cfg, entry.Status)

388 + m.syncSecretStoreDepsForConfig(entry.Cfg)

389 + m.collectorHandler.NotifyJobCreate(entry.Cfg, entry.Status)

390

391 if m.runModePolicy.AutoEnableDiscovered {

360 - m.handler.CmdEnable(dyncfg.NewFunction(functions.Function{Args: []string{m.dyncfgJobID(entry.Cfg), "enable"}}))

392 + m.collectorHandler.CmdEnable(dyncfg.NewFunction(functions.Function{Args: []string{m.dyncfgJobID(entry.Cfg), "enable"}}))

393 } else {

362 - m.handler.WaitForDecision(entry.Cfg)

394 + m.collectorHandler.WaitForDecision(entry.Cfg)

395 }

396 }

397

398 func (m *Manager) removeConfig(cfg confgroup.Config) {

399 m.retryingTasks.remove(cfg)

400

369 - entry, ok := m.handler.RemoveDiscoveredConfig(cfg)

401 + entry, ok := m.collectorHandler.RemoveDiscoveredConfig(cfg)

402 if !ok {

403 return

404 }

405 + // Stop first so running-state cleanup is applied against existing dependency state.

406 m.stopRunningJob(cfg.FullName())

407 + m.secretStoreDeps.RemoveActiveJob(entry.Cfg.FullName())

408 m.fileStatus.remove(cfg)

409

410 if !isStock(cfg) || entry.Status == dyncfg.StatusRunning {

377 - m.handler.NotifyJobRemove(cfg)

411 + m.collectorHandler.NotifyJobRemove(cfg)

412 }

413 }

414

@@ -402,20 +436,13 @@ func (m *Manager) startRunningJob(job runtimeJob) {

436 m.runningJobs.lock()

437 m.runningJobs.add(job.FullName(), job)

438 m.runningJobs.unlock()

439 + m.secretStoreDeps.setRunning(job.FullName(), true)

440

406 - // Track job for module function routing.

407 - m.moduleFuncs.addJob(job.ModuleName(), job.Name(), job)

408 - m.registerModuleMethodsOnFirstJobStart(job.ModuleName())

409 -

410 - // Register job-specific methods if module provides JobMethods callback

411 - creator, ok := m.modules.Lookup(job.ModuleName())

412 - if !ok || creator.JobMethods == nil {

413 - return

414 - }

415 - methods := creator.JobMethods(job)

416 - if len(methods) > 0 {

417 - m.registerJobMethods(job, methods)

418 - }

441 + // Known behavior: Start runs asynchronously, so function handlers may be

442 + // published before the job flips its running flag. Immediate function calls

443 + // can transiently return 503; this is accepted for now to avoid adding a

444 + // broader runtime readiness contract.

445 + m.funcCtl.OnJobStart(job)

446 }

447

448 func (m *Manager) stopRunningJob(name string) {

@@ -425,35 +452,18 @@ func (m *Manager) stopRunningJob(name string) {

452 m.runningJobs.remove(name)

453 }

454 m.runningJobs.unlock()

428 -

455 if ok {

430 - // Unregister job-specific methods.

431 - m.unregisterJobMethods(job)

432 - // Remove job from module function registry.

433 - m.moduleFuncs.removeJob(job.ModuleName(), job.Name())

434 - // Static module methods remain registered for now. Once Netdata supports

435 - // function removal, this should unregister static methods when the last

436 - // running job for the module is removed.

456 + m.secretStoreDeps.setRunning(name, false)

457 + m.funcCtl.OnJobStop(job)

458 job.Stop()

459 }

460 }

461

462 func (m *Manager) cleanup() {

463 m.fnReg.UnregisterPrefix("config", m.dyncfgCollectorPrefixValue())

464 + m.fnReg.UnregisterPrefix("config", m.dyncfgSecretStorePrefixValue())

465 m.fnReg.UnregisterPrefix("config", m.dyncfgVnodePrefixValue())

444 -

445 - // Unregister module functions

446 - for name, creator := range m.modules {

447 - if creator.Methods != nil {

448 - for _, method := range creator.Methods() {

449 - if method.ID == "" {

450 - continue

451 - }

452 - funcName := fmt.Sprintf("%s:%s", name, method.ID)

453 - m.fnReg.Unregister(funcName)

454 - }

455 - }

456 - }

466 + m.funcCtl.Cleanup()

467

468 for _, job := range m.runningJobs.snapshot() {

469 m.stopRunningJob(job.FullName())

@@ -476,138 +486,12 @@ func (m *Manager) waitCmdTestWorkers() {

486 }

487 }

488

479 -func (m *Manager) registerModuleMethodsOnFirstJobStart(moduleName string) {

480 - if _, ok := m.staticMethodsSeen[moduleName]; ok {

481 - return

482 - }

483 -

484 - creator, ok := m.modules.Lookup(moduleName)

485 - if !ok || creator.Methods == nil {

486 - return

487 - }

488 -

489 - methods := creator.Methods()

490 - for _, method := range methods {

491 - if method.ID == "" {

492 - m.Warningf("skipping function registration for module '%s': empty method ID", moduleName)

493 - continue

494 - }

495 - funcName := fmt.Sprintf("%s:%s", moduleName, method.ID)

496 - m.fnReg.Register(funcName, m.makeMethodFuncHandler(moduleName, method.ID))

497 -

498 - help := method.Help

499 - if help == "" {

500 - help = fmt.Sprintf("%s %s data function", moduleName, method.ID)

501 - }

502 -

503 - // https://github.com/netdata/netdata/blob/1bc1775a17590b3c0fe3a4fe547dc6146d07be89/src/libnetdata/user-auth/http-access.h#L21

504 - const cloudAccess = "0x0013" // SIGNED_ID | SAME_SPACE | SENSITIVE_DATA

505 - access := "0x0000"

506 - if method.RequireCloud {

507 - access = cloudAccess

508 - }

509 - m.dyncfgApi.FunctionGlobal(netdataapi.FunctionGlobalOpts{

510 - Name: funcName,

511 - Timeout: 60,

512 - Help: help,

513 - Tags: "top",

514 - Access: access,

515 - Priority: 100,

516 - Version: 3,

517 - })

518 - }

519 -

520 - m.staticMethodsSeen[moduleName] = struct{}{}

521 -}

522 -

523 -// registerJobMethods registers methods for a specific job with Netdata

524 -func (m *Manager) registerJobMethods(job collectorapi.RuntimeJob, methods []funcapi.MethodConfig) {

525 - planned := make(map[string]struct{}, len(methods))

526 -

527 - for _, method := range methods {

528 - if method.ID == "" {

529 - m.Warningf("skipping job method registration for %s[%s]: empty method ID", job.ModuleName(), job.Name())

530 - continue

531 - }

532 -

533 - funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

534 -

535 - if _, exists := planned[method.ID]; exists {

536 - m.Errorf("job method registration aborted for %s[%s]: duplicate method ID in batch ('%s')", job.ModuleName(), job.Name(), funcName)

537 - return

538 - }

539 - planned[method.ID] = struct{}{}

540 -

541 - if collision, exists := m.moduleFuncs.findMethodCollision(job.ModuleName(), job.Name(), method.ID); exists {

542 - m.Errorf("job method registration aborted for %s[%s]: collision on '%s' (%s)", job.ModuleName(), job.Name(), funcName, collision)

543 - return

544 - }

545 - }

546 -

547 - for _, method := range methods {

548 - if method.ID == "" {

549 - continue

550 - }

551 -

552 - funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

553 -

554 - // Register Go handler for this function

555 - m.fnReg.Register(funcName, m.makeJobMethodFuncHandler(job.ModuleName(), job.Name(), method.ID))

556 -

557 - // Notify Netdata about this function

558 - help := method.Help

559 - if help == "" {

560 - help = fmt.Sprintf("%s %s data function", job.ModuleName(), method.ID)

561 - }

562 -

563 - const cloudAccess = "0x0013" // SIGNED_ID | SAME_SPACE | SENSITIVE_DATA

564 - access := "0x0000"

565 - if method.RequireCloud {

566 - access = cloudAccess

567 - }

568 -

569 - m.dyncfgApi.FunctionGlobal(netdataapi.FunctionGlobalOpts{

570 - Name: funcName,

571 - Timeout: 60,

572 - Help: help,

573 - Tags: "top",

574 - Access: access,

575 - Priority: 100,

576 - Version: 3,

577 - })

578 -

579 - m.Debugf("registered job method: %s for job %s[%s]", funcName, job.ModuleName(), job.Name())

580 - }

581 -

582 - // Store methods in registry for later unregistration

583 - m.moduleFuncs.registerJobMethods(job.ModuleName(), job.Name(), methods)

489 +func (m *Manager) createCollectorJob(cfg confgroup.Config) (runtimeJob, error) {

490 + return newJobFactory(m).create(cfg)

491 }

492

586 -// unregisterJobMethods unregisters methods for a specific job

587 -func (m *Manager) unregisterJobMethods(job collectorapi.RuntimeJob) {

588 - methods := m.moduleFuncs.getJobMethods(job.ModuleName(), job.Name())

589 - if len(methods) == 0 {

590 - return

591 - }

592 -

593 - for _, method := range methods {

594 - if method.ID == "" {

595 - continue

596 - }

597 -

598 - funcName := fmt.Sprintf("%s:%s", job.ModuleName(), method.ID)

599 -

600 - // Unregister Go handler

601 - m.fnReg.Unregister(funcName)

602 -

603 - // Notify Netdata to remove function (no-op until Netdata supports it)

604 - m.dyncfgApi.FunctionRemove(funcName)

605 -

606 - m.Debugf("unregistered job method: %s for job %s[%s]", funcName, job.ModuleName(), job.Name())

607 - }

608 -

609 - // Remove from registry

610 - m.moduleFuncs.unregisterJobMethods(job.ModuleName(), job.Name())

493 +func (m *Manager) validateCollectorJob(cfg confgroup.Config) error {

494 + return newJobFactory(m).validate(cfg)

495 }

496

497 func (m *Manager) baseContext() context.Context {

@@ -617,10 +501,6 @@ func (m *Manager) baseContext() context.Context {

501 return context.Background()

502 }

503

620 -func (m *Manager) createCollectorJob(cfg confgroup.Config) (runtimeJob, error) {

621 - return newJobFactory(m).create(cfg)

622 -}

623 -

504 func runRetryTask(ctx context.Context, out chan<- confgroup.Config, cfg confgroup.Config) {

505 t := time.NewTimer(time.Second * time.Duration(cfg.AutoDetectionRetry()))

506 defer t.Stop()

@@ -649,41 +529,3 @@ func isStock(cfg confgroup.Config) bool {

529 func isDyncfg(cfg confgroup.Config) bool {

530 return cfg.SourceType() == confgroup.TypeDyncfg

531 }

652 -

653 -func newConfigModule(creator collectorapi.Creator) (configModule, error) {

654 - if creator.CreateV2 != nil {

655 - mod := creator.CreateV2()

656 - if mod == nil {

657 - return nil, fmt.Errorf("CreateV2 returned nil")

658 - }

659 - return mod, nil

660 - }

661 - if creator.Create == nil {

662 - return nil, fmt.Errorf("no module creator is defined")

663 - }

664 - mod := creator.Create()

665 - if mod == nil {

666 - return nil, fmt.Errorf("Create returned nil")

667 - }

668 - return mod, nil

669 -}

670 -

671 -func applyConfig(cfg confgroup.Config, module any) error {

672 - bs, err := yaml.Marshal(cfg)

673 - if err != nil {

674 - return err

675 - }

676 - return yaml.Unmarshal(bs, module)

677 -}

678 -

679 -func makeLabels(cfg confgroup.Config) map[string]string {

680 - labels := make(map[string]string)

681 - for name, value := range cfg.Labels() {

682 - n, ok1 := name.(string)

683 - v, ok2 := value.(string)

684 - if ok1 && ok2 {

685 - labels[n] = v

686 - }

687 - }

688 - return labels

689 -}

src/go/plugin/agent/jobmgr/manager_process_test.go

+94 -49

@@ -3,7 +3,9 @@

3 package jobmgr

5 import (

6 + "bytes"

7 "context"

8 + "strings"

9 "sync"

10 "testing"

11 "time"

@@ -12,6 +14,7 @@ import (

14 "github.com/stretchr/testify/require"

16 "github.com/netdata/netdata/go/plugins/pkg/funcapi"

17 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

18 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

19 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

20 "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

@@ -92,7 +95,7 @@ func TestRun_WaitTimeoutClearsGateAndKeepsAccepted(t *testing.T) {

95 cfg2 := prepareStockCfg("success", "wait2")

97 mgr.addCh <- cfg1

95 - require.Eventually(t, mgr.handler.WaitingForDecision, time.Second, 10*time.Millisecond)

98 + require.Eventually(t, mgr.collectorHandler.WaitingForDecision, time.Second, 10*time.Millisecond)

100 secondSent := make(chan struct{})

101 go func() {

@@ -112,7 +115,7 @@ func TestRun_WaitTimeoutClearsGateAndKeepsAccepted(t *testing.T) {

115 t.Fatal("second add did not progress after wait timeout")

116 }

117

115 - entry1, ok := mgr.exposed.LookupByKey(cfg1.ExposedKey())

118 + entry1, ok := mgr.collectorExposed.LookupByKey(cfg1.ExposedKey())

119 require.True(t, ok, "first config must stay exposed after timeout")

120 assert.Equal(t, dyncfg.StatusAccepted, entry1.Status)

121 }

@@ -217,7 +220,7 @@ func TestStartRunningJob_RegistersModuleMethodsOnFirstStartedJob(t *testing.T) {

220 },

221 }

222 mgr.modules = collectorapi.Registry{"mod": creator}

220 - mgr.moduleFuncs.registerModule("mod", creator)

223 + mgr.funcCtl.RegisterModules(mgr.modules)

224

225 job := &lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"}

226 mgr.startRunningJob(job)

@@ -234,7 +237,7 @@ func TestStartRunningJob_DoesNotReregisterModuleMethods(t *testing.T) {

237 },

238 }

239 mgr.modules = collectorapi.Registry{"mod": creator}

237 - mgr.moduleFuncs.registerModule("mod", creator)

240 + mgr.funcCtl.RegisterModules(mgr.modules)

241

242 job1 := &lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"}

243 mgr.startRunningJob(job1)

@@ -247,63 +250,87 @@ func TestStartRunningJob_DoesNotReregisterModuleMethods(t *testing.T) {

250 assert.ElementsMatch(t, []string{"mod:a", "mod:b"}, registered)

251 }

252

250 -func TestRegisterJobMethods_FailFastOnCollisionWithStaticMethod(t *testing.T) {

253 +func TestRun_RegistersDyncfgConfigPrefixes(t *testing.T) {

254 fnReg := &recordingFunctionRegistry{}

255 mgr := New(Config{PluginName: testPluginName, FnReg: fnReg})

253 - mgr.moduleFuncs.registerModule("mod", collectorapi.Creator{

254 - Methods: func() []funcapi.MethodConfig {

255 - return []funcapi.MethodConfig{{ID: "dup"}}

256 - },

257 - })

256

259 - job := &lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"}

260 - mgr.registerJobMethods(job, []funcapi.MethodConfig{{ID: "dup"}})

257 + ctx, cancel := context.WithCancel(context.Background())

258 + defer cancel()

259

262 - assert.Empty(t, fnReg.registeredNames())

263 - assert.Empty(t, mgr.moduleFuncs.getJobMethods("mod", "job1"))

264 -}

260 + in := make(chan []*confgroup.Group)

261 + done := make(chan struct{})

262 + go func() {

263 + mgr.Run(ctx, in)

264 + close(done)

265 + }()

266

266 -func TestRegisterJobMethods_FailFastOnCollisionWithOtherJob(t *testing.T) {

267 - fnReg := &recordingFunctionRegistry{}

268 - mgr := New(Config{PluginName: testPluginName, FnReg: fnReg})

269 - mgr.moduleFuncs.registerModule("mod", collectorapi.Creator{})

270 - mgr.moduleFuncs.registerJobMethods("mod", "jobA", []funcapi.MethodConfig{{ID: "dup"}})

267 + waitCtx, waitCancel := context.WithTimeout(context.Background(), time.Second)

268 + defer waitCancel()

269 + require.True(t, mgr.WaitStarted(waitCtx), "manager did not report started")

270

272 - job := &lockProbeJob{fullName: "mod_jobB", moduleName: "mod", name: "jobB"}

273 - mgr.registerJobMethods(job, []funcapi.MethodConfig{{ID: "dup"}})

271 + cancel()

272 + close(in)

273 +

274 + select {

275 + case <-done:

276 + case <-time.After(2 * time.Second):

277 + t.Fatal("manager did not stop after cancel")

278 + }

279

275 - assert.Empty(t, fnReg.registeredNames())

276 - assert.Empty(t, mgr.moduleFuncs.getJobMethods("mod", "jobB"))

280 + assert.Equal(t, []registeredPrefix{

281 + {name: "config", prefix: mgr.dyncfgVnodePrefixValue()},

282 + {name: "config", prefix: mgr.dyncfgSecretStorePrefixValue()},

283 + {name: "config", prefix: mgr.dyncfgCollectorPrefixValue()},

284 + }, fnReg.registeredPrefixes())

285 }

286

279 -func TestRegisterJobMethods_FailFastOnDuplicateWithinBatch(t *testing.T) {

287 +func TestRun_PublishesVnodesAndSecretstoresBeforeCollectorTemplates(t *testing.T) {

288 fnReg := &recordingFunctionRegistry{}

281 - mgr := New(Config{PluginName: testPluginName, FnReg: fnReg})

282 - mgr.moduleFuncs.registerModule("mod", collectorapi.Creator{})

283 -

284 - job := &lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"}

285 - mgr.registerJobMethods(job, []funcapi.MethodConfig{

286 - {ID: "dup"},

287 - {ID: "dup"},

289 + var buf bytes.Buffer

290 +

291 + mgr := New(Config{

292 + PluginName: testPluginName,

293 + FnReg: fnReg,

294 + Out: &buf,

295 + Modules: collectorapi.Registry{

296 + "mod": collectorapi.Creator{},

297 + },

298 })

299

290 - assert.Empty(t, fnReg.registeredNames())

291 - assert.Empty(t, mgr.moduleFuncs.getJobMethods("mod", "job1"))

292 -}

300 + ctx, cancel := context.WithCancel(context.Background())

301 + defer cancel()

302

294 -func TestRegisterJobMethods_SuccessCommitsAllMethods(t *testing.T) {

295 - fnReg := &recordingFunctionRegistry{}

296 - mgr := New(Config{PluginName: testPluginName, FnReg: fnReg})

297 - mgr.moduleFuncs.registerModule("mod", collectorapi.Creator{})

303 + in := make(chan []*confgroup.Group)

304 + done := make(chan struct{})

305 + go func() {

306 + mgr.Run(ctx, in)

307 + close(done)

308 + }()

309

299 - job := &lockProbeJob{fullName: "mod_job1", moduleName: "mod", name: "job1"}

300 - mgr.registerJobMethods(job, []funcapi.MethodConfig{

301 - {ID: "a"},

302 - {ID: "b"},

303 - })

310 + waitCtx, waitCancel := context.WithTimeout(context.Background(), time.Second)

311 + defer waitCancel()

312 + require.True(t, mgr.WaitStarted(waitCtx), "manager did not report started")

313

305 - assert.ElementsMatch(t, []string{"mod:a", "mod:b"}, fnReg.registeredNames())

306 - assert.Len(t, mgr.moduleFuncs.getJobMethods("mod", "job1"), 2)

314 + cancel()

315 + close(in)

316 +

317 + select {

318 + case <-done:

319 + case <-time.After(2 * time.Second):

320 + t.Fatal("manager did not stop after cancel")

321 + }

322 +

323 + output := buf.String()

324 +

325 + vnodeIdx := strings.Index(output, "CONFIG "+mgr.dyncfgVnodePrefixValue()+" create accepted template /collectors/"+testPluginName+"/Vnodes")

326 + secretIdx := strings.Index(output, "CONFIG "+mgr.dyncfgSecretStoreID(string(secretstore.KindVault))+" create accepted template /collectors/"+testPluginName+"/SecretStores")

327 + collectorIdx := strings.Index(output, "CONFIG "+mgr.dyncfgModID("mod")+" create accepted template /collectors/"+testPluginName+"/Jobs")

328 +

329 + require.NotEqual(t, -1, vnodeIdx, "vnode template publication not found")

330 + require.NotEqual(t, -1, secretIdx, "secretstore template publication not found")

331 + require.NotEqual(t, -1, collectorIdx, "collector template publication not found")

332 + assert.Less(t, vnodeIdx, collectorIdx, "vnode publication must happen before collector template publication")

333 + assert.Less(t, secretIdx, collectorIdx, "secretstore publication must happen before collector template publication")

334 }

335

336 type lockProbeJob struct {

@@ -341,6 +368,7 @@ func (j *lockProbeJob) UpdateVnode(_ *vnodes.VirtualNode) {}

368 type recordingFunctionRegistry struct {

369 mu sync.Mutex

370 registered []string

371 + prefixes []registeredPrefix

372 }

373

374 func (r *recordingFunctionRegistry) Register(name string, _ func(functions.Function)) {

@@ -349,9 +377,13 @@ func (r *recordingFunctionRegistry) Register(name string, _ func(functions.Funct

377 r.mu.Unlock()

378 }

379

352 -func (r *recordingFunctionRegistry) Unregister(string) {}

353 -func (r *recordingFunctionRegistry) RegisterPrefix(string, string, func(functions.Function)) {}

354 -func (r *recordingFunctionRegistry) UnregisterPrefix(string, string) {}

380 +func (r *recordingFunctionRegistry) Unregister(string) {}

381 +func (r *recordingFunctionRegistry) RegisterPrefix(name, prefix string, _ func(functions.Function)) {

382 + r.mu.Lock()

383 + r.prefixes = append(r.prefixes, registeredPrefix{name: name, prefix: prefix})

384 + r.mu.Unlock()

385 +}

386 +func (r *recordingFunctionRegistry) UnregisterPrefix(string, string) {}

387

388 func (r *recordingFunctionRegistry) registeredNames() []string {

389 r.mu.Lock()

@@ -360,3 +392,16 @@ func (r *recordingFunctionRegistry) registeredNames() []string {

392 copy(out, r.registered)

393 return out

394 }

395 +

396 +func (r *recordingFunctionRegistry) registeredPrefixes() []registeredPrefix {

397 + r.mu.Lock()

398 + defer r.mu.Unlock()

399 + out := make([]registeredPrefix, len(r.prefixes))

400 + copy(out, r.prefixes)

401 + return out

402 +}

403 +

404 +type registeredPrefix struct {

405 + name string

406 + prefix string

407 +}

src/go/plugin/agent/jobmgr/manager_test.go

+44

@@ -530,6 +530,8 @@ CONFIG test:collector:fail:name delete

530 }

531

532 func TestManager_Run_Dyncfg_Get(t *testing.T) {

533 + t.Setenv("DYNCFG_GET_SECRET", "resolved-secret-value")

534 +

535 tests := map[string]struct {

536 createSim func() *runSim

537 }{

@@ -596,6 +598,48 @@ CONFIG test:collector:success:test create accepted job /collectors/test/Jobs dyn

598 FUNCTION_RESULT_BEGIN 2-get 200 application/json

599 {"option_str":"1","option_int":1}

600 FUNCTION_RESULT_END

601 +`,

602 + }

603 + },

604 + },

605 + "[get] existing with secret ref remains unresolved": {

606 + createSim: func() *runSim {

607 + cfg := prepareDyncfgCfg("success", "test").

608 + Set("option_str", "${env:DYNCFG_GET_SECRET}")

609 + bs, _ := json.Marshal(cfg)

610 +

611 + return &runSim{

612 + do: func(mgr *Manager, _ chan []*confgroup.Group) {

613 + mgr.dyncfgConfig(dyncfg.NewFunction(functions.Function{

614 + UID: "1-add",

615 + Source: "type=dyncfg",

616 + Args: []string{mgr.dyncfgModID(cfg.Module()), "add", cfg.Name()},

617 + Payload: bs,

618 + }))

619 + mgr.dyncfgConfig(dyncfg.NewFunction(functions.Function{

620 + UID: "2-get",

621 + Args: []string{mgr.dyncfgJobID(cfg), "get"},

622 + }))

623 + },

624 + wantDiscovered: nil,

625 + wantSeen: []confgroup.Config{

626 + cfg,

627 + },

628 + wantExposed: []wantExposedEntry{

629 + {cfg: cfg, status: dyncfg.StatusAccepted},

630 + },

631 + wantRunning: nil,

632 + wantDyncfg: `

633 +

634 +FUNCTION_RESULT_BEGIN 1-add 202 application/json

635 +{"status":202,"message":""}

636 +FUNCTION_RESULT_END

637 +

638 +CONFIG test:collector:success:test create accepted job /collectors/test/Jobs dyncfg 'type=dyncfg' 'schema get enable disable update restart test userconfig remove' 0x0000 0x0000

639 +

640 +FUNCTION_RESULT_BEGIN 2-get 200 application/json

641 +{"option_str":"${env:DYNCFG_GET_SECRET}","option_int":0}

642 +FUNCTION_RESULT_END

643 `,

644 }

645 },

src/go/plugin/agent/jobmgr/modulefuncs.go deleted

-323

@@ -1,323 +0,0 @@

1 -// SPDX-License-Identifier: GPL-3.0-or-later

2 -

3 -package jobmgr

4 -

5 -import (

6 - "sort"

7 - "sync"

8 -

9 - "github.com/netdata/netdata/go/plugins/pkg/funcapi"

10 - "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

11 -)

12 -

13 -// moduleFuncRegistry tracks module-level functions and their running jobs

14 -type moduleFuncRegistry struct {

15 - mu sync.RWMutex

16 -

17 - // moduleName → moduleFunc

18 - // e.g., "postgres" → {methods: [...], jobs: {"master-db": job1, "replica-db": job2}}

19 - modules map[string]*moduleFunc

20 -}

21 -

22 -type moduleFunc struct {

23 - creator collectorapi.Creator // The module creator (has Methods())

24 - methods []funcapi.MethodConfig // Static methods from creator (ordered)

25 - methodsByID map[string]funcapi.MethodConfig

26 - jobs map[string]*jobEntry // jobName → job entry with generation

27 - lastGeneration map[string]uint64 // jobName → last known generation (persists across removals)

28 - jobMethods map[string][]funcapi.MethodConfig // jobName → methods registered for that job

29 -}

30 -

31 -// jobEntry wraps a job with a generation number for race detection

32 -type jobEntry struct {

33 - job collectorapi.RuntimeJob

34 - generation uint64 // Incremented each time this job name is replaced

35 -}

36 -

37 -func newModuleFuncRegistry() *moduleFuncRegistry {

38 - return &moduleFuncRegistry{

39 - modules: make(map[string]*moduleFunc),

40 - }

41 -}

42 -

43 -// registerModule is called at startup for each module that implements FunctionProvider

44 -func (r *moduleFuncRegistry) registerModule(name string, creator collectorapi.Creator) {

45 - r.mu.Lock()

46 - defer r.mu.Unlock()

47 -

48 - var methods []funcapi.MethodConfig

49 - if creator.Methods != nil {

50 - methods = creator.Methods()

51 - }

52 -

53 - r.modules[name] = &moduleFunc{

54 - creator: creator,

55 - methods: methods,

56 - methodsByID: indexMethods(methods),

57 - jobs: make(map[string]*jobEntry),

58 - lastGeneration: make(map[string]uint64),

59 - jobMethods: make(map[string][]funcapi.MethodConfig),

60 - }

61 -}

62 -

63 -func indexMethods(methods []funcapi.MethodConfig) map[string]funcapi.MethodConfig {

64 - if len(methods) == 0 {

65 - return nil

66 - }

67 - idx := make(map[string]funcapi.MethodConfig, len(methods))

68 - for _, m := range methods {

69 - if m.ID == "" {

70 - continue

71 - }

72 - idx[m.ID] = m

73 - }

74 - return idx

75 -}

76 -

77 -// addJob is called when jobs start.

78 -// NOTE: Job names MUST be unique per module. If a job with the same name

79 -// already exists, it is replaced (this handles config reload scenarios)

80 -func (r *moduleFuncRegistry) addJob(moduleName, jobName string, job collectorapi.RuntimeJob) {

81 - r.mu.Lock()

82 - defer r.mu.Unlock()

83 -

84 - mf, ok := r.modules[moduleName]

85 - if !ok {

86 - return // CollectorV1 not registered (doesn't implement FunctionProvider)

87 - }

88 -

89 - // Replace any existing job with the same name

90 - // Increment generation to invalidate any in-flight requests to old job

91 - // Use lastGeneration to ensure generation monotonically increases even across removals

92 - lastGen := mf.lastGeneration[jobName]

93 - newGen := lastGen + 1

94 - mf.jobs[jobName] = &jobEntry{

95 - job: job,

96 - generation: newGen,

97 - }

98 - mf.lastGeneration[jobName] = newGen

99 -}

100 -

101 -// removeJob is called when jobs stop

102 -func (r *moduleFuncRegistry) removeJob(moduleName, jobName string) {

103 - r.mu.Lock()

104 - defer r.mu.Unlock()

105 -

106 - if mf, ok := r.modules[moduleName]; ok {

107 - delete(mf.jobs, jobName)

108 - }

109 -}

110 -

111 -// getJobWithGeneration returns the job and its generation for race detection

112 -func (r *moduleFuncRegistry) getJobWithGeneration(moduleName, jobName string) (collectorapi.RuntimeJob, uint64) {

113 - r.mu.RLock()

114 - defer r.mu.RUnlock()

115 -

116 - mf, ok := r.modules[moduleName]

117 - if !ok {

118 - return nil, 0

119 - }

120 - entry, ok := mf.jobs[jobName]

121 - if !ok {

122 - return nil, 0

123 - }

124 - return entry.job, entry.generation

125 -}

126 -

127 -// verifyJobGeneration checks if the job still has the expected generation

128 -// Returns false if the job was replaced OR stopped during request processing

129 -func (r *moduleFuncRegistry) verifyJobGeneration(moduleName, jobName string, expectedGen uint64) bool {

130 - r.mu.RLock()

131 - defer r.mu.RUnlock()

132 -

133 - mf, ok := r.modules[moduleName]

134 - if !ok {

135 - return false

136 - }

137 - entry, ok := mf.jobs[jobName]

138 - if !ok {

139 - return false // Job was removed

140 - }

141 -

142 - // CRITICAL: Also check if job is still running

143 - // The job could be stopped (but not yet removed) during our request

144 - // This catches the race where generation matches but job.Stop() was called

145 - if !entry.job.IsRunning() {

146 - return false // Job was stopped

147 - }

148 -

149 - return entry.generation == expectedGen

150 -}

151 -

152 -// getMethods returns the method configurations for a module

153 -func (r *moduleFuncRegistry) getMethods(moduleName string) []funcapi.MethodConfig {

154 - r.mu.RLock()

155 - defer r.mu.RUnlock()

156 -

157 - mf, ok := r.modules[moduleName]

158 - if !ok {

159 - return nil

160 - }

161 - return mf.methods

162 -}

163 -

164 -// getMethod returns a method config by ID for a module.

165 -func (r *moduleFuncRegistry) getMethod(moduleName, methodID string) (*funcapi.MethodConfig, bool) {

166 - r.mu.RLock()

167 - defer r.mu.RUnlock()

168 -

169 - mf, ok := r.modules[moduleName]

170 - if !ok || mf.methodsByID == nil {

171 - return nil, false

172 - }

173 - cfg, ok := mf.methodsByID[methodID]

174 - if !ok {

175 - return nil, false

176 - }

177 - return &cfg, true

178 -}

179 -

180 -// getJobNames returns job names in STABLE alphabetical order

181 -// This ensures consistent UI presentation across refreshes

182 -func (r *moduleFuncRegistry) getJobNames(moduleName string) []string {

183 - r.mu.RLock()

184 - defer r.mu.RUnlock()

185 -

186 - mf, ok := r.modules[moduleName]

187 - if !ok {

188 - return nil

189 - }

190 -

191 - // Extract job names and sort for stable ordering

192 - names := make([]string, 0, len(mf.jobs))

193 - for name := range mf.jobs {

194 - names = append(names, name)

195 - }

196 - sort.Strings(names) // Alphabetical order for consistent UI

197 - return names

198 -}

199 -

200 -// getJob returns the job by name for routing requests

201 -func (r *moduleFuncRegistry) getJob(moduleName, jobName string) (collectorapi.RuntimeJob, bool) {

202 - r.mu.RLock()

203 - defer r.mu.RUnlock()

204 -

205 - mf, ok := r.modules[moduleName]

206 - if !ok {

207 - return nil, false

208 - }

209 - entry, ok := mf.jobs[jobName]

210 - if !ok {

211 - return nil, false

212 - }

213 - return entry.job, true

214 -}

215 -

216 -// getCreator returns the module creator for a registered module

217 -func (r *moduleFuncRegistry) getCreator(moduleName string) (collectorapi.Creator, bool) {

218 - r.mu.RLock()

219 - defer r.mu.RUnlock()

220 -

221 - mf, ok := r.modules[moduleName]

222 - if !ok {

223 - return collectorapi.Creator{}, false

224 - }

225 - return mf.creator, true

226 -}

227 -

228 -// isModuleRegistered checks if a module is registered (implements FunctionProvider)

229 -func (r *moduleFuncRegistry) isModuleRegistered(moduleName string) bool {

230 - r.mu.RLock()

231 - defer r.mu.RUnlock()

232 -

233 - _, ok := r.modules[moduleName]

234 - return ok

235 -}

236 -

237 -// registerJobMethods stores methods registered for a specific job

238 -func (r *moduleFuncRegistry) registerJobMethods(moduleName, jobName string, methods []funcapi.MethodConfig) {

239 - r.mu.Lock()

240 - defer r.mu.Unlock()

241 -

242 - mf, ok := r.modules[moduleName]

243 - if !ok {

244 - return

245 - }

246 - mf.jobMethods[jobName] = methods

247 -}

248 -

249 -// unregisterJobMethods removes methods registered for a specific job

250 -func (r *moduleFuncRegistry) unregisterJobMethods(moduleName, jobName string) {

251 - r.mu.Lock()

252 - defer r.mu.Unlock()

253 -

254 - mf, ok := r.modules[moduleName]

255 - if !ok {

256 - return

257 - }

258 - delete(mf.jobMethods, jobName)

259 -}

260 -

261 -// getJobMethods returns methods registered for a specific job

262 -func (r *moduleFuncRegistry) getJobMethods(moduleName, jobName string) []funcapi.MethodConfig {

263 - r.mu.RLock()

264 - defer r.mu.RUnlock()

265 -

266 - mf, ok := r.modules[moduleName]

267 - if !ok {

268 - return nil

269 - }

270 - return mf.jobMethods[jobName]

271 -}

272 -

273 -// getJobMethod returns a specific method registered for a job by method ID

274 -func (r *moduleFuncRegistry) getJobMethod(moduleName, jobName, methodID string) (*funcapi.MethodConfig, bool) {

275 - r.mu.RLock()

276 - defer r.mu.RUnlock()

277 -

278 - mf, ok := r.modules[moduleName]

279 - if !ok {

280 - return nil, false

281 - }

282 - methods, ok := mf.jobMethods[jobName]

283 - if !ok {

284 - return nil, false

285 - }

286 - for i := range methods {

287 - if methods[i].ID == methodID {

288 - return &methods[i], true

289 - }

290 - }

291 - return nil, false

292 -}

293 -

294 -// findMethodCollision checks whether module:method key would collide with already-registered methods.

295 -// It checks static module methods and job methods from other jobs within the same module.

296 -func (r *moduleFuncRegistry) findMethodCollision(moduleName, jobName, methodID string) (collision string, ok bool) {

297 - r.mu.RLock()

298 - defer r.mu.RUnlock()

299 -

300 - mf, ok := r.modules[moduleName]

301 - if !ok {

302 - return "", false

303 - }

304 -

305 - if mf.methodsByID != nil {

306 - if _, exists := mf.methodsByID[methodID]; exists {

307 - return "static method", true

308 - }

309 - }

310 -

311 - for ownerJob, methods := range mf.jobMethods {

312 - if ownerJob == jobName {

313 - continue

314 - }

315 - for _, method := range methods {

316 - if method.ID == methodID {

317 - return "job method on " + ownerJob, true

318 - }

319 - }

320 - }

321 -

322 - return "", false

323 -}

src/go/plugin/agent/jobmgr/modulefuncs_test.go deleted

-258

@@ -1,258 +0,0 @@

1 -// SPDX-License-Identifier: GPL-3.0-or-later

2 -

3 -package jobmgr

4 -

5 -import (

6 - "context"

7 - "io"

8 - "testing"

9 -

10 - "github.com/netdata/netdata/go/plugins/pkg/funcapi"

11 - "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

12 - "github.com/netdata/netdata/go/plugins/plugin/framework/jobruntime"

13 - "github.com/stretchr/testify/assert"

14 - "github.com/stretchr/testify/require"

15 -)

16 -

17 -func TestModuleFuncRegistry_RegisterModule(t *testing.T) {

18 - tests := map[string]struct {

19 - modules []string

20 - expected []string

21 - }{

22 - "single module": {

23 - modules: []string{"postgres"},

24 - expected: []string{"postgres"},

25 - },

26 - "multiple modules": {

27 - modules: []string{"postgres", "mysql", "mssql"},

28 - expected: []string{"mysql", "mssql", "postgres"}, // sorted

29 - },

30 - "duplicate registration overwrites": {

31 - modules: []string{"postgres", "postgres"},

32 - expected: []string{"postgres"},

33 - },

34 - }

35 -

36 - for name, tc := range tests {

37 - t.Run(name, func(t *testing.T) {

38 - r := newModuleFuncRegistry()

39 -

40 - for _, m := range tc.modules {

41 - r.registerModule(m, collectorapi.Creator{

42 - Methods: func() []funcapi.MethodConfig {

43 - return []funcapi.MethodConfig{{ID: "test"}}

44 - },

45 - })

46 - }

47 -

48 - assert.Equal(t, len(tc.expected), len(r.modules))

49 - for _, m := range tc.expected {

50 - assert.True(t, r.isModuleRegistered(m))

51 - }

52 - })

53 - }

54 -}

55 -

56 -func TestModuleFuncRegistry_Operations(t *testing.T) {

57 - tests := map[string]struct {

58 - run func(t *testing.T, r *moduleFuncRegistry)

59 - }{

60 - "add/remove job": {

61 - run: func(t *testing.T, r *moduleFuncRegistry) {

62 - r.registerModule("postgres", collectorapi.Creator{})

63 -

64 - job1 := newTestModuleFuncsJob("job1")

65 - job2 := newTestModuleFuncsJob("job2")

66 -

67 - r.addJob("postgres", "job1", job1)

68 - r.addJob("postgres", "job2", job2)

69 -

70 - names := r.getJobNames("postgres")

71 - assert.ElementsMatch(t, []string{"job1", "job2"}, names)

72 -

73 - got1, ok := r.getJob("postgres", "job1")

74 - assert.True(t, ok)

75 - assert.Equal(t, job1, got1)

76 -

77 - r.removeJob("postgres", "job1")

78 -

79 - names = r.getJobNames("postgres")

80 - assert.ElementsMatch(t, []string{"job2"}, names)

81 -

82 - _, ok = r.getJob("postgres", "job1")

83 - assert.False(t, ok)

84 - },

85 - },

86 - "job replacement increments generation": {

87 - run: func(t *testing.T, r *moduleFuncRegistry) {

88 - r.registerModule("postgres", collectorapi.Creator{})

89 -

90 - job1 := newTestModuleFuncsJob("master")

91 - job2 := newTestModuleFuncsJob("master")

92 -

93 - r.addJob("postgres", "master", job1)

94 - _, gen1 := r.getJobWithGeneration("postgres", "master")

95 - assert.Equal(t, uint64(1), gen1)

96 -

97 - r.addJob("postgres", "master", job2)

98 - got, gen2 := r.getJobWithGeneration("postgres", "master")

99 - assert.Equal(t, uint64(2), gen2)

100 - assert.Equal(t, job2, got)

101 - },

102 - },

103 - "generation verification fails on wrong generation and missing job": {

104 - run: func(t *testing.T, r *moduleFuncRegistry) {

105 - r.registerModule("postgres", collectorapi.Creator{})

106 -

107 - job := newTestModuleFuncsJob("master")

108 - r.addJob("postgres", "master", job)

109 - _, gen := r.getJobWithGeneration("postgres", "master")

110 -

111 - assert.False(t, r.verifyJobGeneration("postgres", "master", gen+1))

112 - r.removeJob("postgres", "master")

113 - assert.False(t, r.verifyJobGeneration("postgres", "master", gen))

114 - },

115 - },

116 - "get methods": {

117 - run: func(t *testing.T, r *moduleFuncRegistry) {

118 - expectedMethods := []funcapi.MethodConfig{

119 - {ID: "top-queries", Name: "Top Queries"},

120 - }

121 -

122 - r.registerModule("postgres", collectorapi.Creator{

123 - Methods: func() []funcapi.MethodConfig {

124 - return expectedMethods

125 - },

126 - })

127 -

128 - assert.Equal(t, expectedMethods, r.getMethods("postgres"))

129 - assert.Nil(t, r.getMethods("nonexistent"))

130 - },

131 - },

132 - "get job names sorted": {

133 - run: func(t *testing.T, r *moduleFuncRegistry) {

134 - r.registerModule("postgres", collectorapi.Creator{})

135 -

136 - r.addJob("postgres", "zebra-db", newTestModuleFuncsJob("zebra"))

137 - r.addJob("postgres", "alpha-db", newTestModuleFuncsJob("alpha"))

138 - r.addJob("postgres", "middle-db", newTestModuleFuncsJob("middle"))

139 -

140 - assert.Equal(t, []string{"alpha-db", "middle-db", "zebra-db"}, r.getJobNames("postgres"))

141 - },

142 - },

143 - "operations on unregistered module are no-op": {

144 - run: func(t *testing.T, r *moduleFuncRegistry) {

145 - r.addJob("nonexistent", "job1", newTestModuleFuncsJob("job1"))

146 - r.removeJob("nonexistent", "job1")

147 -

148 - assert.False(t, r.isModuleRegistered("nonexistent"))

149 - assert.Nil(t, r.getJobNames("nonexistent"))

150 - assert.Nil(t, r.getMethods("nonexistent"))

151 -

152 - _, ok := r.getJob("nonexistent", "job1")

153 - assert.False(t, ok)

154 - },

155 - },

156 - "get creator": {

157 - run: func(t *testing.T, r *moduleFuncRegistry) {

158 - creator := collectorapi.Creator{

159 - JobConfigSchema: "test-schema",

160 - }

161 - r.registerModule("postgres", creator)

162 -

163 - got, ok := r.getCreator("postgres")

164 - require.True(t, ok)

165 - assert.Equal(t, "test-schema", got.JobConfigSchema)

166 -

167 - _, ok = r.getCreator("nonexistent")

168 - assert.False(t, ok)

169 - },

170 - },

171 - }

172 -

173 - for name, tc := range tests {

174 - t.Run(name, func(t *testing.T) {

175 - tc.run(t, newModuleFuncRegistry())

176 - })

177 - }

178 -}

179 -

180 -// newTestModuleFuncsJob creates a minimal job for testing modulefuncs

181 -func newTestModuleFuncsJob(name string) *jobruntime.Job {

182 - return jobruntime.NewJob(jobruntime.JobConfig{

183 - PluginName: "test",

184 - Name: name,

185 - ModuleName: "test",

186 - FullName: "test_" + name,

187 - Module: &collectorapi.MockCollectorV1{},

188 - Out: io.Discard,

189 - UpdateEvery: 1,

190 - AutoDetectEvery: 0,

191 - Priority: 1000,

192 - })

193 -}

194 -

195 -// TestModuleFuncRegistry_Concurrency tests thread safety

196 -func TestModuleFuncRegistry_Concurrency(t *testing.T) {

197 - r := newModuleFuncRegistry()

198 - r.registerModule("postgres", collectorapi.Creator{

199 - Methods: func() []funcapi.MethodConfig {

200 - return []funcapi.MethodConfig{{ID: "test"}}

201 - },

202 - })

203 -

204 - done := make(chan bool)

205 -

206 - // Writer goroutine

207 - go func() {

208 - for i := 0; i < 100; i++ {

209 - job := newTestModuleFuncsJob("job")

210 - r.addJob("postgres", "job", job)

211 - r.removeJob("postgres", "job")

212 - }

213 - done <- true

214 - }()

215 -

216 - // Reader goroutine

217 - go func() {

218 - for i := 0; i < 100; i++ {

219 - _ = r.getJobNames("postgres")

220 - _ = r.getMethods("postgres")

221 - _, _ = r.getJob("postgres", "job")

222 - }

223 - done <- true

224 - }()

225 -

226 - <-done

227 - <-done

228 -}

229 -

230 -// TestModuleFuncRegistry_VerifyJobGeneration_JobStopped tests race detection when job stops

231 -func TestModuleFuncRegistry_VerifyJobGeneration_JobStopped(t *testing.T) {

232 - r := newModuleFuncRegistry()

233 - r.registerModule("postgres", collectorapi.Creator{})

234 -

235 - job := jobruntime.NewJob(jobruntime.JobConfig{

236 - PluginName: "test",

237 - Name: "master",

238 - ModuleName: "postgres",

239 - FullName: "postgres_master",

240 - Module: &collectorapi.MockCollectorV1{

241 - InitFunc: func(context.Context) error { return nil },

242 - CheckFunc: func(context.Context) error { return nil },

243 - ChartsFunc: func() *collectorapi.Charts { return &collectorapi.Charts{} },

244 - CollectFunc: func(context.Context) map[string]int64 { return nil },

245 - },

246 - Out: io.Discard,

247 - UpdateEvery: 1,

248 - AutoDetectEvery: 0,

249 - Priority: 1000,

250 - })

251 -

252 - r.addJob("postgres", "master", job)

253 - _, gen := r.getJobWithGeneration("postgres", "master")

254 -

255 - // Job exists but is not running - verification should fail

256 - // (job.IsRunning() returns false because job hasn't started)

257 - assert.False(t, r.verifyJobGeneration("postgres", "master", gen))

258 -}

src/go/plugin/agent/jobmgr/output_test.go

+1 -1

@@ -21,7 +21,7 @@ func TestNew_UsesConfiguredOutForDyncfgResponder(t *testing.T) {

21 Out: &buf,

22 })

24 - mgr.dyncfgApi.ConfigCreate(netdataapi.ConfigOpts{

24 + mgr.dyncfgResponder.ConfigCreate(netdataapi.ConfigOpts{

25 ID: "test:collector:module",

26 Status: dyncfg.StatusAccepted.String(),

27 ConfigType: dyncfg.ConfigTypeTemplate.String(),

src/go/plugin/agent/jobmgr/secretsctl/cache.go new

+131

@@ -0,0 +1,131 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "fmt"

7 + "strings"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

11 +)

12 +

13 +func (c *Controller) Lookup(key string) (Entry, bool) {

14 + entry, ok := c.lookup(key)

15 + if !ok {

16 + return Entry{}, false

17 + }

18 + return entry, true

19 +}

20 +

21 +func (c *Controller) lookup(key string) (Entry, bool) {

22 + entry, ok := c.lookupInternal(key)

23 + if !ok {

24 + return Entry{}, false

25 + }

26 + return entryFromDyncfg(entry), true

27 +}

28 +

29 +func (c *Controller) lookupInternal(key string) (*dyncfg.Entry[secretstore.Config], bool) {

30 + if strings.TrimSpace(key) == "" || c.exposed == nil {

31 + return nil, false

32 + }

33 + return c.exposed.LookupByKey(key)

34 +}

35 +

36 +func (c *Controller) RememberDiscoveredConfig(cfg secretstore.Config) (Entry, bool, error) {

37 + entry, changed, err := c.rememberDiscoveredConfig(cfg)

38 + if err != nil || !changed || entry == nil {

39 + return Entry{}, changed, err

40 + }

41 + return entryFromDyncfg(entry), true, nil

42 +}

43 +

44 +func (c *Controller) rememberDiscoveredConfig(cfg secretstore.Config) (*dyncfg.Entry[secretstore.Config], bool, error) {

45 + if err := c.validateConfig(cfg); err != nil {

46 + return nil, false, err

47 + }

48 +

49 + c.handler.RememberDiscoveredConfig(cfg)

50 +

51 + entry, ok := c.lookupInternal(cfg.ExposedKey())

52 + if !ok {

53 + entry = c.handler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted)

54 + c.handler.NotifyJobCreate(cfg, dyncfg.StatusAccepted)

55 + return entry, true, nil

56 + }

57 +

58 + sp, ep := cfg.SourceTypePriority(), entry.Cfg.SourceTypePriority()

59 + if ep > sp || (ep == sp && entry.Status == dyncfg.StatusRunning) {

60 + return entry, false, nil

61 + }

62 +

63 + if entry.Status == dyncfg.StatusRunning || entry.Status == dyncfg.StatusFailed {

64 + c.cb.Stop(entry.Cfg)

65 + c.cb.TakeCommandMessage()

66 + }

67 +

68 + entry = c.handler.AddDiscoveredConfig(cfg, dyncfg.StatusAccepted)

69 + c.handler.NotifyJobCreate(cfg, dyncfg.StatusAccepted)

70 + return entry, true, nil

71 +}

72 +

73 +func (c *Controller) RemoveDiscoveredConfig(cfg secretstore.Config) (Entry, bool) {

74 + entry, ok := c.removeDiscoveredConfig(cfg)

75 + if !ok || entry == nil {

76 + return Entry{}, false

77 + }

78 + return entryFromDyncfg(entry), true

79 +}

80 +

81 +func (c *Controller) removeDiscoveredConfig(cfg secretstore.Config) (*dyncfg.Entry[secretstore.Config], bool) {

82 + entry, ok := c.handler.RemoveDiscoveredConfig(cfg)

83 + if !ok {

84 + return nil, false

85 + }

86 +

87 + c.cb.Stop(entry.Cfg)

88 + c.cb.TakeCommandMessage()

89 + c.handler.NotifyJobRemove(entry.Cfg)

90 + return entry, true

91 +}

92 +

93 +func (c *Controller) validateConfig(cfg secretstore.Config) error {

94 + if err := cfg.Validate(); err != nil {

95 + return err

96 + }

97 + if c.service == nil {

98 + return fmt.Errorf("secretstore service is not available")

99 + }

100 + return c.service.Validate(cfg)

101 +}

102 +

103 +func (c *Controller) validateStored(key string) error {

104 + entry, ok := c.lookupInternal(key)

105 + if !ok {

106 + return secretstore.ErrStoreNotFound

107 + }

108 + if c.service == nil {

109 + return fmt.Errorf("secretstore service is not available")

110 + }

111 +

112 + if _, ok := c.service.GetStatus(key); ok {

113 + return c.service.ValidateStored(key)

114 + }

115 +

116 + return c.validateConfig(entry.Cfg)

117 +}

118 +

119 +func (c *Controller) affectedJobsFor(storeKey string) []secretstore.JobRef {

120 + if c.affectedJobs == nil {

121 + return nil

122 + }

123 + return c.affectedJobs(storeKey)

124 +}

125 +

126 +func (c *Controller) restartableAffectedJobsFor(storeKey string) []secretstore.JobRef {

127 + if c.restartableAffectedJobs == nil {

128 + return nil

129 + }

130 + return c.restartableAffectedJobs(storeKey)

131 +}

src/go/plugin/agent/jobmgr/secretsctl/cache_ownership_test.go new

+181

@@ -0,0 +1,181 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "bytes"

7 + "testing"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

14 + "github.com/stretchr/testify/assert"

15 + "github.com/stretchr/testify/require"

16 +)

17 +

18 +func TestControllerRememberDiscoveredConfig_InvalidDoesNotEnterCaches(t *testing.T) {

19 + ctl, _, _ := newVaultControllerTestSubject()

20 +

21 + raw := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{}, "/etc/netdata/secretstores.yaml", confgroup.TypeUser)

22 +

23 + entry, changed, err := ctl.RememberDiscoveredConfig(raw)

24 + require.Error(t, err)

25 + assert.False(t, changed)

26 + assert.Equal(t, Entry{}, entry)

27 + assert.Zero(t, ctl.seen.Count())

28 + assert.Zero(t, ctl.exposed.Count())

29 +}

30 +

31 +func TestControllerRememberDiscoveredConfig_PreservesUnknownFields(t *testing.T) {

32 + ctl, _, _ := newVaultControllerTestSubject()

33 +

34 + cfg := vaultModeTokenConfig()

35 + cfg["ui_note"] = "kept"

36 + cfg["mode_token"].(map[string]any)["extra"] = "kept"

37 +

38 + raw := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", cfg, "/etc/netdata/secretstores.yaml", confgroup.TypeUser)

39 + entry, changed, err := ctl.RememberDiscoveredConfig(raw)

40 + require.NoError(t, err)

41 + require.True(t, changed)

42 +

43 + assert.Equal(t, "kept", entry.Cfg["ui_note"])

44 +

45 + modeToken := entry.Cfg["mode_token"].(map[string]any)

46 + assert.Equal(t, "kept", modeToken["extra"])

47 +}

48 +

49 +func TestControllerSeqExec_FileDefinedConfigBecomesDyncfgOverride(t *testing.T) {

50 + ctl, out, _ := newVaultControllerTestSubject()

51 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

52 +

53 + fileCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", vaultModeTokenConfig(), "file=/etc/netdata/go.d/ss/vault.conf", confgroup.TypeUser)

54 + require.NoError(t, ctl.Service().Add(fileCfg))

55 + ctl.seen.Add(fileCfg)

56 + ctl.exposed.Add(&dyncfg.Entry[secretstore.Config]{

57 + Cfg: fileCfg,

58 + Status: dyncfg.StatusRunning,

59 + })

60 +

61 + updateFn := dyncfg.NewFunction(functions.Function{

62 + UID: "ss-file-update",

63 + ContentType: "application/json",

64 + Payload: mustJSON(t, vaultModeTokenFileConfig()),

65 + Args: []string{

66 + ctl.configID(key),

67 + string(dyncfg.CommandUpdate),

68 + },

69 + })

70 + ctl.SeqExec(updateFn)

71 +

72 + var updateResp map[string]any

73 + mustDecodeFunctionPayload(t, out.String(), "ss-file-update", &updateResp)

74 + assert.Equal(t, float64(200), updateResp["status"])

75 +

76 + entry, ok := ctl.Lookup(key)

77 + require.True(t, ok)

78 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

79 + assert.Equal(t, confgroup.TypeDyncfg, entry.Cfg.SourceType())

80 + assert.Equal(t, confgroup.TypeDyncfg, entry.Cfg.Source())

81 + assert.Equal(t, "token_file", entry.Cfg["mode"])

82 +

83 + removeFn := dyncfg.NewFunction(functions.Function{

84 + UID: "ss-file-remove",

85 + Args: []string{ctl.configID(key), string(dyncfg.CommandRemove)},

86 + })

87 + ctl.SeqExec(removeFn)

88 +

89 + var removeResp map[string]any

90 + mustDecodeFunctionPayload(t, out.String(), "ss-file-remove", &removeResp)

91 + assert.Equal(t, float64(200), removeResp["status"])

92 +

93 + _, ok = ctl.Lookup(key)

94 + assert.False(t, ok)

95 +

96 + seenUser, ok := ctl.seen.LookupByUID(fileCfg.UID())

97 + require.True(t, ok)

98 + assert.Equal(t, fileCfg.UID(), seenUser.UID())

99 + assert.Equal(t, 1, ctl.seen.Count())

100 + assert.Zero(t, ctl.exposed.Count())

101 +}

102 +

103 +func TestControllerRemoveDiscoveredConfig_DoesNotRevealLowerPrioritySeenConfig(t *testing.T) {

104 + ctl, _, _ := newVaultControllerTestSubject()

105 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

106 +

107 + userCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", vaultModeTokenConfig(), "/etc/netdata/secretstores.yaml", confgroup.TypeUser)

108 + entry, changed, err := ctl.RememberDiscoveredConfig(userCfg)

109 + require.NoError(t, err)

110 + require.True(t, changed)

111 + assert.Equal(t, userCfg.UID(), entry.Cfg.UID())

112 +

113 + dyncfgCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", vaultModeTokenFileConfig(), confgroup.TypeDyncfg, confgroup.TypeDyncfg)

114 + entry, changed, err = ctl.RememberDiscoveredConfig(dyncfgCfg)

115 + require.NoError(t, err)

116 + require.True(t, changed)

117 + assert.Equal(t, dyncfgCfg.UID(), entry.Cfg.UID())

118 +

119 + removed, ok := ctl.RemoveDiscoveredConfig(dyncfgCfg)

120 + require.True(t, ok)

121 + assert.Equal(t, dyncfgCfg.UID(), removed.Cfg.UID())

122 +

123 + _, ok = ctl.Lookup(key)

124 + assert.False(t, ok)

125 +

126 + seenUser, ok := ctl.seen.LookupByUID(userCfg.UID())

127 + require.True(t, ok)

128 + assert.Equal(t, userCfg.UID(), seenUser.UID())

129 + assert.Equal(t, 1, ctl.seen.Count())

130 + assert.Zero(t, ctl.exposed.Count())

131 +}

132 +

133 +func TestControllerSeqExec_RemoveFailedUnpublishedStoreDoesNotRestartDependents(t *testing.T) {

134 + ctl, out, seams := newVaultControllerTestSubject()

135 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

136 +

137 + raw := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"mode": "token"}, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

138 + ctl.seen.Add(raw)

139 + ctl.exposed.Add(&dyncfg.Entry[secretstore.Config]{

140 + Cfg: raw,

141 + Status: dyncfg.StatusFailed,

142 + })

143 +

144 + removeFn := dyncfg.NewFunction(functions.Function{

145 + UID: "ss-remove-no-restart",

146 + Args: []string{ctl.configID(key), string(dyncfg.CommandRemove)},

147 + })

148 + ctl.SeqExec(removeFn)

149 +

150 + var resp map[string]any

151 + mustDecodeFunctionPayload(t, out.String(), "ss-remove-no-restart", &resp)

152 + assert.Equal(t, float64(200), resp["status"])

153 + assert.Equal(t, "", resp["message"])

154 + assert.Empty(t, seams.restartCalls)

155 +}

156 +

157 +func newVaultControllerTestSubject() (*Controller, *bytes.Buffer, *controllerSeams) {

158 + return newControllerTestSubjectWithOptions(Options{

159 + Service: secretstore.NewService(backends.Creators()...),

160 + })

161 +}

162 +

163 +func vaultModeTokenConfig() map[string]any {

164 + return map[string]any{

165 + "mode": "token",

166 + "mode_token": map[string]any{

167 + "token": "vault-token",

168 + },

169 + "addr": "https://vault.example",

170 + }

171 +}

172 +

173 +func vaultModeTokenFileConfig() map[string]any {

174 + return map[string]any{

175 + "mode": "token_file",

176 + "mode_token_file": map[string]any{

177 + "path": "/var/lib/netdata/vault.token",

178 + },

179 + "addr": "https://vault.example",

180 + }

181 +}

src/go/plugin/agent/jobmgr/secretsctl/callbacks.go new

+223

@@ -0,0 +1,223 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "errors"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

13 +)

14 +

15 +type secretStoreCallbacks struct {

16 + deps secretStoreCallbackDeps

17 + // commandMessage is written by Start/Update/Stop and consumed by

18 + // TakeCommandMessage. Safety relies on callbacks remaining serialized by

19 + // the jobmgr dyncfg command flow.

20 + commandMessage string

21 +}

22 +

23 +type secretStoreCallbackDeps struct {

24 + pluginName string

25 + service secretstore.Service

26 + restartDependentJobs func(string) string

27 +}

28 +

29 +type codedError struct {

30 + err error

31 + code int

32 +}

33 +

34 +func (e *codedError) Error() string { return e.err.Error() }

35 +func (e *codedError) Unwrap() error { return e.err }

36 +func (e *codedError) Code() int { return e.code }

37 +

38 +func newSecretStoreCallbacks(deps secretStoreCallbackDeps) *secretStoreCallbacks {

39 + return &secretStoreCallbacks{deps: deps}

40 +}

41 +

42 +func (d secretStoreCallbackDeps) restartDependentJobsMessage(storeKey string) string {

43 + if d.restartDependentJobs == nil {

44 + return ""

45 + }

46 + return d.restartDependentJobs(storeKey)

47 +}

48 +

49 +func (d secretStoreCallbackDeps) extractSecretStoreKindFromTemplateID(id string) (secretstore.StoreKind, bool) {

50 + rest, ok := strings.CutPrefix(id, fmt.Sprintf(dyncfgSecretStorePrefixf, d.pluginName))

51 + if !ok || rest == "" || strings.Contains(rest, ":") {

52 + return "", false

53 + }

54 + kind := secretstore.StoreKind(rest)

55 + if d.service == nil {

56 + return "", false

57 + }

58 + _, ok = d.service.DisplayName(kind)

59 + return kind, ok

60 +}

61 +

62 +func (d secretStoreCallbackDeps) extractSecretStoreKey(id string) (string, bool) {

63 + rest, ok := strings.CutPrefix(id, fmt.Sprintf(dyncfgSecretStorePrefixf, d.pluginName))

64 + if !ok || rest == "" {

65 + return "", false

66 + }

67 + kind, name, err := secretstore.ParseStoreKey(rest)

68 + if err != nil {

69 + return "", false

70 + }

71 + return secretstore.StoreKey(kind, name), true

72 +}

73 +

74 +func (d secretStoreCallbackDeps) secretStoreConfigFromPayload(fn dyncfg.Function, name string, kind secretstore.StoreKind) (secretstore.Config, error) {

75 + if err := fn.ValidateHasPayload(); err != nil {

76 + return nil, err

77 + }

78 +

79 + var payload secretstore.Config

80 + if err := fn.UnmarshalPayload(&payload); err != nil {

81 + return nil, fmt.Errorf("invalid configuration format: %w", err)

82 + }

83 + if payload == nil {

84 + payload = secretstore.Config{}

85 + }

86 + payload.SetName(name)

87 + payload.SetKind(kind)

88 + payload.SetSource(confgroup.TypeDyncfg)

89 + payload.SetSourceType(confgroup.TypeDyncfg)

90 + return payload, nil

91 +}

92 +

93 +func (d secretStoreCallbackDeps) validateSecretStoreConfig(cfg secretstore.Config) error {

94 + if err := cfg.Validate(); err != nil {

95 + return err

96 + }

97 + if d.service == nil {

98 + return fmt.Errorf("secretstore service is not available")

99 + }

100 + return d.service.Validate(cfg)

101 +}

102 +

103 +func (d secretStoreCallbackDeps) dyncfgSecretStoreID(id string) string {

104 + return fmt.Sprintf("%s%s", fmt.Sprintf(dyncfgSecretStorePrefixf, d.pluginName), id)

105 +}

106 +

107 +func (cb *secretStoreCallbacks) ExtractKey(fn dyncfg.Function) (key, name string, ok bool) {

108 + if fn.Command() == dyncfg.CommandAdd {

109 + kind, kindOK := cb.deps.extractSecretStoreKindFromTemplateID(fn.ID())

110 + name = fn.JobName()

111 + if !kindOK || name == "" {

112 + return "", "", false

113 + }

114 + return secretstore.StoreKey(kind, name), name, true

115 + }

116 +

117 + key, ok = cb.deps.extractSecretStoreKey(fn.ID())

118 + if !ok {

119 + return "", "", false

120 + }

121 + _, name, err := secretstore.ParseStoreKey(key)

122 + if err != nil {

123 + return "", "", false

124 + }

125 + return key, name, true

126 +}

127 +

128 +func (cb *secretStoreCallbacks) ParseAndValidate(fn dyncfg.Function, name string) (secretstore.Config, error) {

129 + var kind secretstore.StoreKind

130 + if fn.Command() == dyncfg.CommandAdd {

131 + var ok bool

132 + kind, ok = cb.deps.extractSecretStoreKindFromTemplateID(fn.ID())

133 + if !ok {

134 + return nil, fmt.Errorf("invalid template ID for secretstore add: %s", fn.ID())

135 + }

136 + } else {

137 + key, ok := cb.deps.extractSecretStoreKey(fn.ID())

138 + if !ok {

139 + return nil, fmt.Errorf("invalid secretstore ID: %s", fn.ID())

140 + }

141 + var err error

142 + kind, name, err = secretstore.ParseStoreKey(key)

143 + if err != nil {

144 + return nil, err

145 + }

146 + }

147 +

148 + cfg, err := cb.deps.secretStoreConfigFromPayload(fn, name, kind)

149 + if err != nil {

150 + return nil, err

151 + }

152 + if err := cb.deps.validateSecretStoreConfig(cfg); err != nil {

153 + return nil, err

154 + }

155 + return cfg, nil

156 +}

157 +

158 +func (cb *secretStoreCallbacks) Start(cfg secretstore.Config) error {

159 + cb.commandMessage = ""

160 + key := cfg.ExposedKey()

161 + if cb.deps.service == nil {

162 + return &codedError{err: fmt.Errorf("secretstore service is not available"), code: 400}

163 + }

164 +

165 + if _, ok := cb.deps.service.GetStatus(key); ok {

166 + if err := cb.deps.service.Update(key, cfg); err != nil {

167 + return &codedError{err: err, code: secretStoreErrorCode(err)}

168 + }

169 + } else if err := cb.deps.service.Add(cfg); err != nil {

170 + return &codedError{err: err, code: secretStoreErrorCode(err)}

171 + }

172 +

173 + cb.commandMessage = cb.deps.restartDependentJobsMessage(key)

174 + return nil

175 +}

176 +

177 +func (cb *secretStoreCallbacks) Update(oldCfg, newCfg secretstore.Config) error {

178 + cb.commandMessage = ""

179 + key := oldCfg.ExposedKey()

180 + if cb.deps.service == nil {

181 + return &codedError{err: fmt.Errorf("secretstore service is not available"), code: 400}

182 + }

183 +

184 + if _, ok := cb.deps.service.GetStatus(key); ok {

185 + if err := cb.deps.service.Update(key, newCfg); err != nil {

186 + return &codedError{err: err, code: secretStoreErrorCode(err)}

187 + }

188 + } else if err := cb.deps.service.Add(newCfg); err != nil {

189 + return &codedError{err: err, code: secretStoreErrorCode(err)}

190 + }

191 +

192 + cb.commandMessage = cb.deps.restartDependentJobsMessage(key)

193 + return nil

194 +}

195 +

196 +func (cb *secretStoreCallbacks) Stop(cfg secretstore.Config) {

197 + cb.commandMessage = ""

198 + key := cfg.ExposedKey()

199 + if cb.deps.service == nil {

200 + return

201 + }

202 +

203 + if err := cb.deps.service.Remove(key); err != nil {

204 + if errors.Is(err, secretstore.ErrStoreNotFound) {

205 + return

206 + }

207 + return

208 + }

209 + cb.commandMessage = cb.deps.restartDependentJobsMessage(key)

210 +}

211 +

212 +func (*secretStoreCallbacks) OnStatusChange(*dyncfg.Entry[secretstore.Config], dyncfg.Status, dyncfg.Function) {

213 +}

214 +

215 +func (cb *secretStoreCallbacks) TakeCommandMessage() string {

216 + msg := strings.TrimSpace(cb.commandMessage)

217 + cb.commandMessage = ""

218 + return msg

219 +}

220 +

221 +func (cb *secretStoreCallbacks) ConfigID(cfg secretstore.Config) string {

222 + return cb.deps.dyncfgSecretStoreID(cfg.ExposedKey())

223 +}

src/go/plugin/agent/jobmgr/secretsctl/callbacks_test.go new

+219

@@ -0,0 +1,219 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "context"

7 + "encoding/json"

8 + "fmt"

9 + "testing"

10 +

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

14 + "github.com/stretchr/testify/assert"

15 + "github.com/stretchr/testify/require"

16 +)

17 +

18 +const testPluginName = "go.d"

19 +

20 +func TestSecretStoreCallbacks(t *testing.T) {

21 + tests := map[string]struct {

22 + run func(t *testing.T, cb *secretStoreCallbacks, svc secretstore.Service, restart *secretStoreCallbackRestartRecorder)

23 + }{

24 + "isolation without manager": {

25 + run: func(t *testing.T, cb *secretStoreCallbacks, svc secretstore.Service, restart *secretStoreCallbackRestartRecorder) {

26 + addFn := newSecretStoreCallbackFunction(t, "ss-add", testSecretStoreTemplateID(secretstore.KindVault), dyncfg.CommandAdd, "vault_prod", map[string]any{"value": "one"})

27 + key, name, ok := cb.ExtractKey(addFn)

28 + require.True(t, ok)

29 + assert.Equal(t, secretstore.StoreKey(secretstore.KindVault, "vault_prod"), key)

30 + assert.Equal(t, "vault_prod", name)

31 +

32 + cfg, err := cb.ParseAndValidate(addFn, name)

33 + require.NoError(t, err)

34 + assert.Empty(t, restart.calls)

35 + assert.Equal(t, "", cb.TakeCommandMessage())

36 +

37 + require.NoError(t, cb.Start(cfg))

38 + assert.Equal(t, []string{key}, restart.calls)

39 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

40 + assert.Equal(t, testSecretStoreConfigID(key), cb.ConfigID(cfg))

41 +

42 + status, ok := svc.GetStatus(key)

43 + require.True(t, ok)

44 + assert.Equal(t, secretstore.KindVault, status.Kind)

45 + assert.Equal(t, "vault_prod", status.Name)

46 + assert.Equal(t, "one", resolveTestStoreValue(t, svc, key))

47 +

48 + updateFn := newSecretStoreCallbackFunction(t, "ss-update", testSecretStoreConfigID(key), dyncfg.CommandUpdate, "", map[string]any{"value": "two"})

49 + updateKey, updateName, ok := cb.ExtractKey(updateFn)

50 + require.True(t, ok)

51 + assert.Equal(t, key, updateKey)

52 + assert.Equal(t, name, updateName)

53 +

54 + updatedCfg, err := cb.ParseAndValidate(updateFn, "")

55 + require.NoError(t, err)

56 + assert.Len(t, restart.calls, 1)

57 +

58 + require.NoError(t, cb.Update(cfg, updatedCfg))

59 + assert.Equal(t, []string{key, key}, restart.calls)

60 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

61 + assert.Equal(t, "two", resolveTestStoreValue(t, svc, key))

62 +

63 + cb.Stop(updatedCfg)

64 + assert.Equal(t, []string{key, key, key}, restart.calls)

65 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

66 + _, ok = svc.GetStatus(key)

67 + assert.False(t, ok)

68 + },

69 + },

70 + "restart seam called only on mutation": {

71 + run: func(t *testing.T, cb *secretStoreCallbacks, _ secretstore.Service, restart *secretStoreCallbackRestartRecorder) {

72 + addFn := newSecretStoreCallbackFunction(t, "ss-mutate-add", testSecretStoreTemplateID(secretstore.KindVault), dyncfg.CommandAdd, "vault_prod", map[string]any{"value": "one"})

73 + key, name, ok := cb.ExtractKey(addFn)

74 + require.True(t, ok)

75 +

76 + cfg, err := cb.ParseAndValidate(addFn, name)

77 + require.NoError(t, err)

78 + assert.Empty(t, restart.calls)

79 +

80 + assert.Equal(t, testSecretStoreConfigID(key), cb.ConfigID(cfg))

81 + assert.Equal(t, "", cb.TakeCommandMessage())

82 + cb.OnStatusChange(nil, dyncfg.StatusAccepted, addFn)

83 + assert.Empty(t, restart.calls)

84 +

85 + require.NoError(t, cb.Start(cfg))

86 + assert.Equal(t, []string{key}, restart.calls)

87 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

88 +

89 + updateFn := newSecretStoreCallbackFunction(t, "ss-mutate-update", testSecretStoreConfigID(key), dyncfg.CommandUpdate, "", map[string]any{"value": "two"})

90 + updatedCfg, err := cb.ParseAndValidate(updateFn, "")

91 + require.NoError(t, err)

92 + assert.Len(t, restart.calls, 1)

93 +

94 + require.NoError(t, cb.Update(cfg, updatedCfg))

95 + assert.Equal(t, []string{key, key}, restart.calls)

96 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

97 +

98 + cb.Stop(updatedCfg)

99 + assert.Equal(t, []string{key, key, key}, restart.calls)

100 + assert.Equal(t, "restart:"+key, cb.TakeCommandMessage())

101 + },

102 + },

103 + }

104 +

105 + for name, tc := range tests {

106 + t.Run(name, func(t *testing.T) {

107 + cb, svc, restart := newSecretStoreCallbacksTestSubject()

108 + tc.run(t, cb, svc, restart)

109 + })

110 + }

111 +}

112 +

113 +type secretStoreCallbackRestartRecorder struct {

114 + calls []string

115 +}

116 +

117 +func newSecretStoreCallbacksTestSubject() (*secretStoreCallbacks, secretstore.Service, *secretStoreCallbackRestartRecorder) {

118 + svc := newTestSecretStoreService()

119 + restart := &secretStoreCallbackRestartRecorder{}

120 + cb := newSecretStoreCallbacks(secretStoreCallbackDeps{

121 + pluginName: testPluginName,

122 + service: svc,

123 + restartDependentJobs: func(storeKey string) string {

124 + restart.calls = append(restart.calls, storeKey)

125 + return "restart:" + storeKey

126 + },

127 + })

128 + return cb, svc, restart

129 +}

130 +

131 +func newSecretStoreCallbackFunction(t *testing.T, uid, id string, cmd dyncfg.Command, jobName string, payload any) dyncfg.Function {

132 + t.Helper()

133 +

134 + args := []string{id, string(cmd)}

135 + if jobName != "" {

136 + args = append(args, jobName)

137 + }

138 +

139 + fn := functions.Function{

140 + UID: uid,

141 + Args: args,

142 + }

143 + if payload != nil {

144 + fn.ContentType = "application/json"

145 + fn.Payload = mustJSON(t, payload)

146 + }

147 + return dyncfg.NewFunction(fn)

148 +}

149 +

150 +func testSecretStoreTemplateID(kind secretstore.StoreKind) string {

151 + return fmt.Sprintf("%s%s", fmt.Sprintf(dyncfgSecretStorePrefixf, testPluginName), kind)

152 +}

153 +

154 +func testSecretStoreConfigID(key string) string {

155 + return fmt.Sprintf("%s%s", fmt.Sprintf(dyncfgSecretStorePrefixf, testPluginName), key)

156 +}

157 +

158 +func resolveTestStoreValue(t *testing.T, svc secretstore.Service, key string) string {

159 + t.Helper()

160 +

161 + kind, name, err := secretstore.ParseStoreKey(key)

162 + require.NoError(t, err)

163 +

164 + ref := fmt.Sprintf("%s:%s:value", kind, name)

165 + original := fmt.Sprintf("${store:%s}", ref)

166 + value, err := svc.Resolve(context.Background(), svc.Capture(), ref, original)

167 + require.NoError(t, err)

168 + return value

169 +}

170 +

171 +type testStoreConfig struct {

172 + Value string `yaml:"value" json:"value"`

173 +}

174 +

175 +type testStore struct {

176 + testStoreConfig `yaml:",inline" json:""`

177 +}

178 +

179 +func (s *testStore) Configuration() any { return &s.testStoreConfig }

180 +

181 +func (s *testStore) Init(context.Context) error {

182 + if s.testStoreConfig.Value == "" {

183 + return fmt.Errorf("value is required")

184 + }

185 + return nil

186 +}

187 +

188 +func (s *testStore) Publish() secretstore.PublishedStore {

189 + return &testPublishedStore{value: s.testStoreConfig.Value}

190 +}

191 +

192 +type testPublishedStore struct {

193 + value string

194 +}

195 +

196 +func (s *testPublishedStore) Resolve(_ context.Context, req secretstore.ResolveRequest) (string, error) {

197 + if req.Operand != "value" {

198 + return "", fmt.Errorf("unexpected operand %q", req.Operand)

199 + }

200 + return s.value, nil

201 +}

202 +

203 +func newTestSecretStoreService() secretstore.Service {

204 + return secretstore.NewService(secretstore.Creator{

205 + Kind: secretstore.KindVault,

206 + DisplayName: "Vault",

207 + Schema: `{"jsonSchema":{"type":"object","properties":{"value":{"type":"string"}}},"uiSchema":[]}`,

208 + Create: func() secretstore.Store {

209 + return &testStore{}

210 + },

211 + })

212 +}

213 +

214 +func mustJSON(t *testing.T, v any) []byte {

215 + t.Helper()

216 + bs, err := json.Marshal(v)

217 + require.NoError(t, err)

218 + return bs

219 +}

src/go/plugin/agent/jobmgr/secretsctl/controller.go new

+164

@@ -0,0 +1,164 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "fmt"

7 +

8 + "github.com/netdata/netdata/go/plugins/logger"

9 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

12 +)

13 +

14 +const (

15 + dyncfgSecretStorePrefixf = "%s:secretstore:"

16 + dyncfgSecretStorePath = "/collectors/%s/SecretStores"

17 +)

18 +

19 +type Options struct {

20 + Logger *logger.Logger

21 + API *dyncfg.Responder

22 + Seen *dyncfg.SeenCache[secretstore.Config]

23 + Exposed *dyncfg.ExposedCache[secretstore.Config]

24 + Service secretstore.Service

25 + Plugin string

26 + Initial []secretstore.Config

27 +

28 + AffectedJobs func(string) []secretstore.JobRef

29 + RestartableAffectedJobs func(string) []secretstore.JobRef

30 + RestartDependentJobs func(string) string

31 +}

32 +

33 +type Entry struct {

34 + Cfg secretstore.Config

35 + Status dyncfg.Status

36 +}

37 +

38 +type Controller struct {

39 + *logger.Logger

40 +

41 + api *dyncfg.Responder

42 + seen *dyncfg.SeenCache[secretstore.Config]

43 + exposed *dyncfg.ExposedCache[secretstore.Config]

44 + service secretstore.Service

45 + pluginName string

46 + initial []secretstore.Config

47 +

48 + affectedJobs func(string) []secretstore.JobRef

49 + restartableAffectedJobs func(string) []secretstore.JobRef

50 + restartDependentJobs func(string) string

51 +

52 + handler *dyncfg.Handler[secretstore.Config]

53 + cb *secretStoreCallbacks

54 +}

55 +

56 +func New(opts Options) *Controller {

57 + seen := opts.Seen

58 + if seen == nil {

59 + seen = dyncfg.NewSeenCache[secretstore.Config]()

60 + }

61 + exposed := opts.Exposed

62 + if exposed == nil {

63 + exposed = dyncfg.NewExposedCache[secretstore.Config]()

64 + }

65 +

66 + c := &Controller{

67 + Logger: opts.Logger,

68 + api: opts.API,

69 + seen: seen,

70 + exposed: exposed,

71 + service: opts.Service,

72 + pluginName: opts.Plugin,

73 + initial: append([]secretstore.Config(nil), opts.Initial...),

74 + affectedJobs: opts.AffectedJobs,

75 + restartableAffectedJobs: opts.RestartableAffectedJobs,

76 + restartDependentJobs: opts.RestartDependentJobs,

77 + }

78 + c.cb = newSecretStoreCallbacks(secretStoreCallbackDeps{

79 + pluginName: c.pluginName,

80 + service: c.service,

81 + restartDependentJobs: c.restartDependentJobs,

82 + })

83 + c.handler = dyncfg.NewHandler(dyncfg.HandlerOpts[secretstore.Config]{

84 + Logger: c.Logger,

85 + API: c.api,

86 + Seen: c.seen,

87 + Exposed: c.exposed,

88 + Callbacks: c.cb,

89 + Path: fmt.Sprintf(dyncfgSecretStorePath, c.pluginName),

90 + JobCommands: []dyncfg.Command{

91 + dyncfg.CommandSchema,

92 + dyncfg.CommandGet,

93 + dyncfg.CommandUpdate,

94 + dyncfg.CommandTest,

95 + },

96 + })

97 + return c

98 +}

99 +

100 +func (c *Controller) Prefix() string {

101 + return fmt.Sprintf(dyncfgSecretStorePrefixf, c.pluginName)

102 +}

103 +

104 +func (c *Controller) CreateTemplates() {

105 + if c.service == nil || c.api == nil {

106 + return

107 + }

108 + for _, kind := range c.service.Kinds() {

109 + c.api.ConfigCreate(netdataapi.ConfigOpts{

110 + ID: c.templateID(kind),

111 + Status: dyncfg.StatusAccepted.String(),

112 + ConfigType: dyncfg.ConfigTypeTemplate.String(),

113 + Path: fmt.Sprintf(dyncfgSecretStorePath, c.pluginName),

114 + SourceType: "internal",

115 + Source: "internal",

116 + SupportedCommands: dyncfgSecretStoreTemplateCmds(),

117 + })

118 + }

119 +}

120 +

121 +func (c *Controller) PublishExisting() {

122 + if len(c.initial) != 0 {

123 + for _, cfg := range c.initial {

124 + c.publishInitialConfig(cfg)

125 + }

126 + c.initial = nil

127 + return

128 + }

129 +

130 + c.exposed.ForEach(func(_ string, entry *dyncfg.Entry[secretstore.Config]) bool {

131 + c.handler.NotifyJobCreate(entry.Cfg, entry.Status)

132 + return true

133 + })

134 +}

135 +

136 +func (c *Controller) SetAPI(api *dyncfg.Responder) {

137 + if api == nil {

138 + // Nil means "keep the current responder" rather than clearing output wiring.

139 + return

140 + }

141 + c.api = api

142 + if c.handler != nil {

143 + c.handler.SetAPI(api)

144 + }

145 +}

146 +

147 +func (c *Controller) Service() secretstore.Service {

148 + return c.service

149 +}

150 +

151 +func (c *Controller) configID(id string) string {

152 + return fmt.Sprintf("%s%s", c.Prefix(), id)

153 +}

154 +

155 +func (c *Controller) templateID(kind secretstore.StoreKind) string {

156 + return fmt.Sprintf("%s%s", c.Prefix(), kind)

157 +}

158 +

159 +func entryFromDyncfg(entry *dyncfg.Entry[secretstore.Config]) Entry {

160 + if entry == nil {

161 + return Entry{}

162 + }

163 + return Entry{Cfg: entry.Cfg, Status: entry.Status}

164 +}

src/go/plugin/agent/jobmgr/secretsctl/controller_test.go new

+366

@@ -0,0 +1,366 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "bytes"

7 + "encoding/json"

8 + "regexp"

9 + "testing"

10 +

11 + "github.com/netdata/netdata/go/plugins/logger"

12 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

13 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

14 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

15 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

18 + "github.com/stretchr/testify/assert"

19 + "github.com/stretchr/testify/require"

20 +)

21 +

22 +func TestControllerSeqExec(t *testing.T) {

23 + tests := map[string]struct {

24 + run func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams)

25 + }{

26 + "schema dispatch": {

27 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

28 + fn := dyncfg.NewFunction(functions.Function{

29 + UID: "ss-schema",

30 + Args: []string{ctl.templateID(secretstore.KindVault), string(dyncfg.CommandSchema)},

31 + })

32 + ctl.SeqExec(fn)

33 +

34 + var payload any

35 + mustDecodeFunctionPayload(t, out.String(), "ss-schema", &payload)

36 + assert.NotNil(t, payload)

37 + assert.Empty(t, seams.affectedJobsCalls)

38 + assert.Empty(t, seams.restartCalls)

39 + },

40 + },

41 + "test preview uses affected jobs seam": {

42 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

43 + addFn := dyncfg.NewFunction(functions.Function{

44 + UID: "ss-add",

45 + ContentType: "application/json",

46 + Payload: mustJSON(t, map[string]any{"value": "one"}),

47 + Args: []string{ctl.templateID(secretstore.KindVault), string(dyncfg.CommandAdd), "vault_prod"},

48 + })

49 + ctl.SeqExec(addFn)

50 +

51 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

52 + seams.affectedJobs[key] = []secretstore.JobRef{{ID: "mysql:prod", Display: "mysql:prod"}}

53 + seams.restartableJobs[key] = []secretstore.JobRef{{ID: "mysql:prod", Display: "mysql:prod"}}

54 +

55 + testFn := dyncfg.NewFunction(functions.Function{

56 + UID: "ss-test",

57 + Args: []string{ctl.configID(key), string(dyncfg.CommandTest)},

58 + })

59 + ctl.SeqExec(testFn)

60 +

61 + var payload map[string]any

62 + mustDecodeFunctionPayload(t, out.String(), "ss-test", &payload)

63 + assert.Equal(t, float64(202), payload["status"])

64 + assert.Contains(t, payload["message"], "This secretstore is used by jobs: mysql:prod.")

65 + assert.Contains(t, payload["message"], "Running or failed jobs that would be restarted automatically by a change: mysql:prod.")

66 + assert.Equal(t, []string{key}, seams.affectedJobsCalls)

67 + assert.Equal(t, []string{key}, seams.restartableJobsCalls)

68 + },

69 + },

70 + "remove blocks when dependent jobs exist": {

71 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

72 + addFn := dyncfg.NewFunction(functions.Function{

73 + UID: "ss-add-remove-blocked",

74 + ContentType: "application/json",

75 + Payload: mustJSON(t, map[string]any{"value": "one"}),

76 + Args: []string{ctl.templateID(secretstore.KindVault), string(dyncfg.CommandAdd), "vault_prod"},

77 + })

78 + ctl.SeqExec(addFn)

79 +

80 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

81 + seams.affectedJobs[key] = []secretstore.JobRef{{ID: "mysql:prod", Display: "mysql:prod"}, {ID: "nginx:prod", Display: "nginx:prod"}}

82 +

83 + removeFn := dyncfg.NewFunction(functions.Function{

84 + UID: "ss-remove-blocked",

85 + Args: []string{ctl.configID(key), string(dyncfg.CommandRemove)},

86 + })

87 + ctl.SeqExec(removeFn)

88 +

89 + var payload map[string]any

90 + mustDecodeFunctionPayload(t, out.String(), "ss-remove-blocked", &payload)

91 + assert.Equal(t, float64(409), payload["status"])

92 + assert.Equal(t, "The specified secretstore 'vault:vault_prod' is used by jobs (mysql:prod, nginx:prod).", payload["errorMessage"])

93 + _, ok := ctl.Lookup(key)

94 + assert.True(t, ok)

95 + },

96 + },

97 + }

98 +

99 + for name, tc := range tests {

100 + t.Run(name, func(t *testing.T) {

101 + ctl, out, seams := newControllerTestSubject()

102 + tc.run(t, ctl, out, seams)

103 + })

104 + }

105 +}

106 +

107 +func TestControllerSetAPI_NilPreservesResponder(t *testing.T) {

108 + tests := map[string]struct {

109 + uid string

110 + }{

111 + "nil SetAPI keeps existing responder for schema responses": {

112 + uid: "ss-schema-nil-rebind",

113 + },

114 + }

115 +

116 + for name, tc := range tests {

117 + t.Run(name, func(t *testing.T) {

118 + ctl, out, _ := newControllerTestSubject()

119 + ctl.SetAPI(nil)

120 +

121 + fn := dyncfg.NewFunction(functions.Function{

122 + UID: tc.uid,

123 + Args: []string{ctl.templateID(secretstore.KindVault), string(dyncfg.CommandSchema)},

124 + })

125 + ctl.SeqExec(fn)

126 +

127 + var payload any

128 + mustDecodeFunctionPayload(t, out.String(), tc.uid, &payload)

129 + assert.NotNil(t, payload)

130 + })

131 + }

132 +}

133 +

134 +func TestControllerSeqExec_TestStoredWithoutService_Returns400(t *testing.T) {

135 + tests := map[string]struct {

136 + storeKey string

137 + }{

138 + "stored validation without service returns controlled 400": {

139 + storeKey: secretstore.StoreKey(secretstore.KindVault, "vault_prod"),

140 + },

141 + }

142 +

143 + for name, tc := range tests {

144 + t.Run(name, func(t *testing.T) {

145 + var out bytes.Buffer

146 + ctl := New(Options{

147 + Logger: logger.New(),

148 + API: dyncfg.NewResponder(netdataapi.New(safewriter.New(&out))),

149 + Plugin: testPluginName,

150 + })

151 +

152 + cfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "one"}, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

153 + ctl.exposed.Add(&dyncfg.Entry[secretstore.Config]{

154 + Cfg: cfg,

155 + Status: dyncfg.StatusRunning,

156 + })

157 +

158 + fn := dyncfg.NewFunction(functions.Function{

159 + UID: "ss-test-nil-service",

160 + Args: []string{ctl.configID(tc.storeKey), string(dyncfg.CommandTest)},

161 + })

162 +

163 + ctl.SeqExec(fn)

164 +

165 + var payload map[string]any

166 + mustDecodeFunctionPayload(t, out.String(), "ss-test-nil-service", &payload)

167 + assert.Equal(t, float64(400), payload["status"])

168 + assert.Contains(t, payload["errorMessage"], "secretstore service is not available")

169 + })

170 + }

171 +}

172 +

173 +func TestRememberDiscoveredConfig_DrainsRestartFailureMessageFromNonHandlerStop(t *testing.T) {

174 + ctl, _, seams := newControllerTestSubject()

175 + existing := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "one"}, "/etc/netdata/secretstores.yaml", confgroup.TypeUser)

176 + require.NoError(t, ctl.Service().Add(existing))

177 + ctl.seen.Add(existing)

178 + ctl.exposed.Add(&dyncfg.Entry[secretstore.Config]{

179 + Cfg: existing,

180 + Status: dyncfg.StatusRunning,

181 + })

182 +

183 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

184 + seams.restartMessages[key] = "restart failed"

185 +

186 + replacement := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "two"}, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

187 + entry, changed, err := ctl.RememberDiscoveredConfig(replacement)

188 + require.NoError(t, err)

189 + require.True(t, changed)

190 + assert.Equal(t, dyncfg.StatusAccepted, entry.Status)

191 + assert.Equal(t, []string{key}, seams.restartCalls)

192 + _, ok := ctl.Service().GetStatus(key)

193 + assert.False(t, ok)

194 + assert.Equal(t, "", ctl.cb.TakeCommandMessage())

195 +}

196 +

197 +func TestControllerPublishExisting(t *testing.T) {

198 + t.Run("initial valid config publishes running without enable disable commands", func(t *testing.T) {

199 + cfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "one"}, "file=/etc/netdata/go.d/ss/vault.conf", confgroup.TypeUser)

200 +

201 + var out bytes.Buffer

202 + ctl := New(Options{

203 + Logger: logger.New(),

204 + API: dyncfg.NewResponder(netdataapi.New(safewriter.New(&out))),

205 + Plugin: testPluginName,

206 + Service: newTestSecretStoreService(),

207 + Initial: []secretstore.Config{cfg},

208 + })

209 +

210 + ctl.PublishExisting()

211 +

212 + entry, ok := ctl.Lookup(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

213 + require.True(t, ok)

214 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

215 + _, ok = ctl.Service().GetStatus(entry.Cfg.ExposedKey())

216 + assert.True(t, ok)

217 + assert.Contains(t, out.String(), "schema get update test")

218 + assert.NotContains(t, out.String(), "enable")

219 + assert.NotContains(t, out.String(), "disable")

220 + })

221 +

222 + t.Run("keyable startup failure publishes failed", func(t *testing.T) {

223 + cfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{}, "file=/etc/netdata/go.d/ss/vault.conf", confgroup.TypeUser)

224 +

225 + ctl, _, _ := newControllerTestSubjectWithOptions(Options{

226 + Initial: []secretstore.Config{cfg},

227 + })

228 + ctl.PublishExisting()

229 +

230 + entry, ok := ctl.Lookup(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

231 + require.True(t, ok)

232 + assert.Equal(t, dyncfg.StatusFailed, entry.Status)

233 + _, ok = ctl.Service().GetStatus(entry.Cfg.ExposedKey())

234 + assert.False(t, ok)

235 + })

236 +

237 + t.Run("failed higher priority startup config shadows lower priority running config", func(t *testing.T) {

238 + stockCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "stock"}, "file=/usr/lib/netdata/conf.d/go.d/ss/vault.conf", confgroup.TypeStock)

239 + userCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{}, "file=/etc/netdata/go.d/ss/vault.conf", confgroup.TypeUser)

240 +

241 + ctl, _, _ := newControllerTestSubjectWithOptions(Options{

242 + Initial: []secretstore.Config{stockCfg, userCfg},

243 + })

244 + ctl.PublishExisting()

245 +

246 + entry, ok := ctl.Lookup(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

247 + require.True(t, ok)

248 + assert.Equal(t, dyncfg.StatusFailed, entry.Status)

249 + assert.Equal(t, confgroup.TypeUser, entry.Cfg.SourceType())

250 + assert.Equal(t, "file=/etc/netdata/go.d/ss/vault.conf", entry.Cfg.Source())

251 + _, ok = ctl.Service().GetStatus(entry.Cfg.ExposedKey())

252 + assert.False(t, ok)

253 + })

254 +

255 + t.Run("later equal priority valid config replaces earlier failed config", func(t *testing.T) {

256 + failedCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{}, "file=/etc/netdata/go.d/ss/01-vault.conf", confgroup.TypeUser)

257 + validCfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "good"}, "file=/etc/netdata/go.d/ss/02-vault.conf", confgroup.TypeUser)

258 +

259 + ctl, _, _ := newControllerTestSubjectWithOptions(Options{

260 + Initial: []secretstore.Config{failedCfg, validCfg},

261 + })

262 + ctl.PublishExisting()

263 +

264 + entry, ok := ctl.Lookup(secretstore.StoreKey(secretstore.KindVault, "vault_prod"))

265 + require.True(t, ok)

266 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

267 + assert.Equal(t, "file=/etc/netdata/go.d/ss/02-vault.conf", entry.Cfg.Source())

268 + _, ok = ctl.Service().GetStatus(entry.Cfg.ExposedKey())

269 + assert.True(t, ok)

270 + })

271 +}

272 +

273 +type controllerSeams struct {

274 + affectedJobs map[string][]secretstore.JobRef

275 + affectedJobsCalls []string

276 + restartableJobs map[string][]secretstore.JobRef

277 + restartableJobsCalls []string

278 + restartMessages map[string]string

279 + restartCalls []string

280 +}

281 +

282 +func newControllerTestSubject() (*Controller, *bytes.Buffer, *controllerSeams) {

283 + return newControllerTestSubjectWithOptions(Options{})

284 +}

285 +

286 +func newControllerTestSubjectWithOptions(opts Options) (*Controller, *bytes.Buffer, *controllerSeams) {

287 + var out bytes.Buffer

288 + seams := &controllerSeams{

289 + affectedJobs: make(map[string][]secretstore.JobRef),

290 + restartableJobs: make(map[string][]secretstore.JobRef),

291 + restartMessages: make(map[string]string),

292 + }

293 + svc := opts.Service

294 + if svc == nil {

295 + svc = newTestSecretStoreService()

296 + }

297 + log := opts.Logger

298 + if log == nil {

299 + log = logger.New()

300 + }

301 + api := opts.API

302 + if api == nil {

303 + api = dyncfg.NewResponder(netdataapi.New(safewriter.New(&out)))

304 + }

305 + plugin := opts.Plugin

306 + if plugin == "" {

307 + plugin = testPluginName

308 + }

309 + affectedJobs := opts.AffectedJobs

310 + if affectedJobs == nil {

311 + affectedJobs = func(storeKey string) []secretstore.JobRef {

312 + seams.affectedJobsCalls = append(seams.affectedJobsCalls, storeKey)

313 + return seams.affectedJobs[storeKey]

314 + }

315 + }

316 + restartableJobs := opts.RestartableAffectedJobs

317 + if restartableJobs == nil {

318 + restartableJobs = func(storeKey string) []secretstore.JobRef {

319 + seams.restartableJobsCalls = append(seams.restartableJobsCalls, storeKey)

320 + return seams.restartableJobs[storeKey]

321 + }

322 + }

323 + restartDependentJobs := opts.RestartDependentJobs

324 + if restartDependentJobs == nil {

325 + restartDependentJobs = func(storeKey string) string {

326 + seams.restartCalls = append(seams.restartCalls, storeKey)

327 + return seams.restartMessages[storeKey]

328 + }

329 + }

330 + ctl := New(Options{

331 + Logger: log,

332 + API: api,

333 + Plugin: plugin,

334 + Service: svc,

335 + AffectedJobs: affectedJobs,

336 + RestartableAffectedJobs: restartableJobs,

337 + RestartDependentJobs: restartDependentJobs,

338 + Initial: opts.Initial,

339 + Seen: opts.Seen,

340 + Exposed: opts.Exposed,

341 + })

342 + return ctl, &out, seams

343 +}

344 +

345 +func newSecretStoreConfigWithSource(t *testing.T, kind secretstore.StoreKind, name string, cfg map[string]any, source, sourceType string) secretstore.Config {

346 + t.Helper()

347 + bs, err := json.Marshal(cfg)

348 + require.NoError(t, err)

349 + var payload map[string]any

350 + require.NoError(t, json.Unmarshal(bs, &payload))

351 + out := secretstore.Config(payload)

352 + out.SetName(name)

353 + out.SetKind(kind)

354 + out.SetSource(source)

355 + out.SetSourceType(sourceType)

356 + return out

357 +}

358 +

359 +func mustDecodeFunctionPayload(t *testing.T, output, uid string, dst any) {

360 + t.Helper()

361 +

362 + re := regexp.MustCompile(`(?s)FUNCTION_RESULT_BEGIN ` + regexp.QuoteMeta(uid) + ` [^\n]+\n(.*?)\nFUNCTION_RESULT_END`)

363 + match := re.FindStringSubmatch(output)

364 + require.Len(t, match, 2, "function result for uid '%s' not found in output:\n%s", uid, output)

365 + require.NoError(t, json.Unmarshal([]byte(match[1]), dst))

366 +}

src/go/plugin/agent/jobmgr/secretsctl/doc.go new

+11

@@ -0,0 +1,11 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +// Package secretsctl owns the secretstore dyncfg control plane for jobmgr.

4 +//

5 +// It does not own:

6 +// - dyncfg prefix routing

7 +// - dyncfg handoff/channel ownership

8 +// - Manager.run() serialized execution

9 +// - collector-supervisor coordination such as dependent collector restarts

10 +// - dependency-index ownership or writes

11 +package secretsctl

src/go/plugin/agent/jobmgr/secretsctl/dyncfg.go new

+368

@@ -0,0 +1,368 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "encoding/json"

7 + "errors"

8 + "fmt"

9 + "strings"

10 +

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

13 + "gopkg.in/yaml.v2"

14 +)

15 +

16 +func dyncfgSecretStoreTemplateCmds() string {

17 + return dyncfg.JoinCommands(dyncfg.CommandAdd, dyncfg.CommandSchema, dyncfg.CommandUserconfig)

18 +}

19 +

20 +func (c *Controller) SeqExec(fn dyncfg.Function) {

21 + switch fn.Command() {

22 + case dyncfg.CommandSchema:

23 + c.dyncfgCmdSchema(fn)

24 + case dyncfg.CommandGet:

25 + c.dyncfgCmdGet(fn)

26 + case dyncfg.CommandAdd:

27 + c.dyncfgCmdAdd(fn)

28 + case dyncfg.CommandUpdate:

29 + // TODO: file/user -> dyncfg conversion currently reuses generic update ordering,

30 + // which can tear down the old live store before the override is active.

31 + c.handler.CmdUpdate(fn)

32 + case dyncfg.CommandTest:

33 + c.dyncfgCmdTest(fn)

34 + case dyncfg.CommandUserconfig:

35 + c.dyncfgCmdUserconfig(fn)

36 + case dyncfg.CommandRemove:

37 + c.dyncfgCmdRemove(fn)

38 + default:

39 + c.Warningf("dyncfg: function '%s' command '%s' not implemented", fn.Fn().Name, fn.Command())

40 + c.api.SendCodef(fn, 501, "Function '%s' command '%s' is not implemented.", fn.Fn().Name, fn.Command())

41 + }

42 +}

43 +

44 +func (c *Controller) dyncfgCmdAdd(fn dyncfg.Function) {

45 + if err := fn.ValidateArgs(3); err != nil {

46 + c.api.SendCodef(fn, 400, "%v", err)

47 + return

48 + }

49 +

50 + key, name, ok := c.cb.ExtractKey(fn)

51 + if !ok {

52 + c.api.SendCodef(fn, 400, "invalid job ID format.")

53 + return

54 + }

55 + if _, exists := c.lookup(key); exists {

56 + c.api.SendCodef(fn, 409, "The specified secretstore '%s' already exists.", key)

57 + return

58 + }

59 + if err := fn.ValidateHasPayload(); err != nil {

60 + c.api.SendCodef(fn, 400, "%v", err)

61 + return

62 + }

63 + if err := dyncfg.ValidateJobName(name); err != nil {

64 + c.api.SendCodef(fn, 400, "invalid job name '%s': %v.", name, err)

65 + return

66 + }

67 +

68 + kind, ok := c.dyncfgExtractSecretStoreKindFromTemplateID(fn.ID())

69 + if !ok {

70 + c.api.SendCodef(fn, 400, "Invalid template ID for secretstore add: %s.", fn.ID())

71 + return

72 + }

73 +

74 + rawCfg, err := c.dyncfgSecretStoreConfigFromPayload(fn, name, kind)

75 + if err != nil {

76 + c.api.SendCodef(fn, 400, "%v", err)

77 + return

78 + }

79 + if err := rawCfg.Validate(); err != nil {

80 + c.api.SendCodef(fn, 400, "%v", err)

81 + return

82 + }

83 +

84 + cfg, prepErr := c.prepareConfigCandidate(rawCfg)

85 + c.seen.Add(cfg)

86 + entry := &dyncfg.Entry[secretstore.Config]{Cfg: cfg, Status: dyncfg.StatusFailed}

87 + c.exposed.Add(entry)

88 +

89 + code := 200

90 + msg := ""

91 + if prepErr == nil {

92 + if err := c.cb.Start(cfg); err == nil {

93 + entry.Status = dyncfg.StatusRunning

94 + msg = c.cb.TakeCommandMessage()

95 + } else {

96 + prepErr = err

97 + }

98 + }

99 + if prepErr != nil {

100 + code = secretStoreCommandCode(prepErr)

101 + msg = prepErr.Error()

102 + }

103 +

104 + c.api.SendCodef(fn, code, "%s", msg)

105 + c.handler.NotifyJobCreate(cfg, entry.Status)

106 +}

107 +

108 +func (c *Controller) dyncfgCmdSchema(fn dyncfg.Function) {

109 + kind, ok := c.dyncfgExtractSecretStoreKindFromTemplateID(fn.ID())

110 + if !ok {

111 + storeKey, ok := c.dyncfgExtractSecretStoreKey(fn.ID())

112 + if !ok {

113 + c.api.SendCodef(fn, 400, "Invalid ID format for secretstore schema: %s.", fn.ID())

114 + return

115 + }

116 + entry, ok := c.lookup(storeKey)

117 + if !ok {

118 + c.api.SendCodef(fn, 404, "The specified secretstore '%s' is not configured.", storeKey)

119 + return

120 + }

121 + kind = entry.Cfg.Kind()

122 + }

123 +

124 + schema, ok := c.service.Schema(kind)

125 + if !ok {

126 + c.api.SendCodef(fn, 404, "The specified secretstore kind '%s' is not supported.", kind)

127 + return

128 + }

129 +

130 + c.api.SendJSON(fn, schema)

131 +}

132 +

133 +func (c *Controller) dyncfgCmdGet(fn dyncfg.Function) {

134 + storeKey, ok := c.dyncfgExtractSecretStoreKey(fn.ID())

135 + if !ok {

136 + c.api.SendCodef(fn, 400, "Invalid ID format for secretstore get: %s.", fn.ID())

137 + return

138 + }

139 +

140 + entry, ok := c.lookup(storeKey)

141 + if !ok {

142 + c.api.SendCodef(fn, 404, "The specified secretstore '%s' is not configured.", storeKey)

143 + return

144 + }

145 +

146 + cfg, err := c.dyncfgTypedConfigFromRaw(entry.Cfg)

147 + if err != nil {

148 + c.api.SendCodef(fn, 500, "Failed to materialize secretstore configuration: %v.", err)

149 + return

150 + }

151 +

152 + bs, err := json.Marshal(cfg)

153 + if err != nil {

154 + c.api.SendCodef(fn, 500, "Failed to convert configuration into JSON: %v.", err)

155 + return

156 + }

157 +

158 + c.api.SendJSON(fn, string(bs))

159 +}

160 +

161 +func (c *Controller) dyncfgCmdTest(fn dyncfg.Function) {

162 + storeKey, ok := c.dyncfgExtractSecretStoreKey(fn.ID())

163 + if !ok {

164 + c.api.SendCodef(fn, 400, "Invalid ID format for secretstore test: %s.", fn.ID())

165 + return

166 + }

167 +

168 + if !fn.HasPayload() {

169 + if err := c.validateStored(storeKey); err != nil {

170 + c.api.SendCodef(fn, secretStoreErrorCode(err), "%v", err)

171 + return

172 + }

173 + c.dyncfgSendSecretStoreTestImpactMessage(fn, c.affectedJobsFor(storeKey), c.restartableAffectedJobsFor(storeKey), true)

174 + return

175 + }

176 +

177 + entry, ok := c.lookup(storeKey)

178 + if !ok {

179 + c.api.SendCodef(fn, 404, "The specified secretstore '%s' is not configured.", storeKey)

180 + return

181 + }

182 +

183 + cfg, err := c.dyncfgSecretStoreConfigFromPayload(fn, entry.Cfg.Name(), entry.Cfg.Kind())

184 + if err != nil {

185 + c.api.SendCodef(fn, 400, "%v", err)

186 + return

187 + }

188 +

189 + cfg, err = c.prepareConfigCandidate(cfg)

190 + if err != nil {

191 + c.api.SendCodef(fn, 400, "%v", err)

192 + return

193 + }

194 +

195 + if cfg.Hash() == entry.Cfg.Hash() {

196 + c.api.SendCodef(fn, 202, "Submitted configuration does not change the active secretstore.")

197 + return

198 + }

199 +

200 + c.dyncfgSendSecretStoreTestImpactMessage(fn, c.affectedJobsFor(storeKey), c.restartableAffectedJobsFor(storeKey), false)

201 +}

202 +

203 +func (c *Controller) dyncfgCmdUserconfig(fn dyncfg.Function) {

204 + kind, ok := c.dyncfgExtractSecretStoreKindFromTemplateID(fn.ID())

205 + if !ok {

206 + c.api.SendCodef(fn, 400, "Invalid template ID for secretstore userconfig: %s.", fn.ID())

207 + return

208 + }

209 + if err := fn.ValidateHasPayload(); err != nil {

210 + c.api.SendCodef(fn, 400, "%v", err)

211 + return

212 + }

213 +

214 + cfg, err := c.dyncfgTypedConfigFromPayload(fn, kind)

215 + if err != nil {

216 + c.api.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

217 + return

218 + }

219 +

220 + bs, err := yaml.Marshal(cfg)

221 + if err != nil {

222 + c.api.SendCodef(fn, 500, "Failed to convert configuration into YAML: %v.", err)

223 + return

224 + }

225 +

226 + c.api.SendYAML(fn, string(bs))

227 +}

228 +

229 +func (c *Controller) dyncfgCmdRemove(fn dyncfg.Function) {

230 + storeKey, ok := c.dyncfgExtractSecretStoreKey(fn.ID())

231 + if !ok {

232 + c.api.SendCodef(fn, 400, "Invalid ID format for secretstore remove: %s.", fn.ID())

233 + return

234 + }

235 +

236 + if _, ok := c.lookup(storeKey); !ok {

237 + c.api.SendCodef(fn, 404, "The specified secretstore '%s' is not configured.", storeKey)

238 + return

239 + }

240 +

241 + if affected := formatAffectedJobs(c.affectedJobsFor(storeKey)); affected != "" {

242 + c.api.SendCodef(fn, 409, "The specified secretstore '%s' is used by jobs (%s).", storeKey, affected)

243 + return

244 + }

245 +

246 + c.handler.CmdRemove(fn)

247 +}

248 +

249 +func (c *Controller) dyncfgTypedConfigFromPayload(fn dyncfg.Function, kind secretstore.StoreKind) (any, error) {

250 + cfg, err := c.dyncfgNewTypedConfig(kind)

251 + if err != nil {

252 + return nil, err

253 + }

254 + if err := fn.UnmarshalPayload(cfg); err != nil {

255 + return nil, err

256 + }

257 + return cfg, nil

258 +}

259 +

260 +func (c *Controller) dyncfgTypedConfigFromRaw(rawCfg secretstore.Config) (any, error) {

261 + cfg, err := c.dyncfgNewTypedConfig(rawCfg.Kind())

262 + if err != nil {

263 + return nil, err

264 + }

265 +

266 + bs, err := yaml.Marshal(rawCfg)

267 + if err != nil {

268 + return nil, err

269 + }

270 + if err := yaml.Unmarshal(bs, cfg); err != nil {

271 + return nil, err

272 + }

273 + return cfg, nil

274 +}

275 +

276 +func (c *Controller) dyncfgNewTypedConfig(kind secretstore.StoreKind) (any, error) {

277 + if c.service == nil {

278 + return nil, fmt.Errorf("secretstore service is not available")

279 + }

280 +

281 + store, ok := c.service.New(kind)

282 + if !ok {

283 + return nil, fmt.Errorf("the specified secretstore kind '%s' is not supported", kind)

284 + }

285 +

286 + cfg := store.Configuration()

287 + if cfg == nil {

288 + return nil, fmt.Errorf("secretstore kind '%s' does not provide configuration", kind)

289 + }

290 + return cfg, nil

291 +}

292 +

293 +func (c *Controller) dyncfgExtractSecretStoreKindFromTemplateID(id string) (secretstore.StoreKind, bool) {

294 + return c.cb.deps.extractSecretStoreKindFromTemplateID(id)

295 +}

296 +

297 +func (c *Controller) dyncfgExtractSecretStoreKey(id string) (string, bool) {

298 + return c.cb.deps.extractSecretStoreKey(id)

299 +}

300 +

301 +func (c *Controller) dyncfgSecretStoreConfigFromPayload(fn dyncfg.Function, name string, kind secretstore.StoreKind) (secretstore.Config, error) {

302 + return c.cb.deps.secretStoreConfigFromPayload(fn, name, kind)

303 +}

304 +

305 +func formatAffectedJobs(refs []secretstore.JobRef) string {

306 + if len(refs) == 0 {

307 + return ""

308 + }

309 +

310 + var b strings.Builder

311 + for i, ref := range refs {

312 + if i > 0 {

313 + b.WriteString(", ")

314 + }

315 + if ref.Display != "" {

316 + b.WriteString(ref.Display)

317 + } else {

318 + b.WriteString(ref.ID)

319 + }

320 + }

321 + return b.String()

322 +}

323 +

324 +func (c *Controller) dyncfgSendSecretStoreTestImpactMessage(fn dyncfg.Function, refs, restartable []secretstore.JobRef, validationOnly bool) {

325 + affected := formatAffectedJobs(refs)

326 + restartableAffected := formatAffectedJobs(restartable)

327 + if validationOnly {

328 + if affected != "" {

329 + if restartableAffected != "" {

330 + c.api.SendCodef(fn, 202, "Stored configuration is valid. This secretstore is used by jobs: %s. Running or failed jobs that would be restarted automatically by a change: %s.", affected, restartableAffected)

331 + return

332 + }

333 + c.api.SendCodef(fn, 202, "Stored configuration is valid. This secretstore is used by jobs: %s. No running or failed jobs would be restarted automatically by a change.", affected)

334 + return

335 + }

336 + c.api.SendCodef(fn, 202, "Stored configuration is valid. No jobs are currently using this secretstore.")

337 + return

338 + }

339 +

340 + if affected != "" {

341 + if restartableAffected != "" {

342 + c.api.SendCodef(fn, 202, "Updated configuration is used by jobs: %s. Running or failed jobs that would be restarted automatically: %s.", affected, restartableAffected)

343 + return

344 + }

345 + c.api.SendCodef(fn, 202, "Updated configuration is used by jobs: %s. No running or failed jobs would be restarted automatically.", affected)

346 + return

347 + }

348 + c.api.SendCodef(fn, 202, "No jobs currently use this secretstore.")

349 +}

350 +

351 +func secretStoreErrorCode(err error) int {

352 + switch {

353 + case errors.Is(err, secretstore.ErrStoreExists):

354 + return 409

355 + case errors.Is(err, secretstore.ErrStoreNotFound):

356 + return 404

357 + default:

358 + return 400

359 + }

360 +}

361 +

362 +func secretStoreCommandCode(err error) int {

363 + var ce interface{ Code() int }

364 + if errors.As(err, &ce) {

365 + return ce.Code()

366 + }

367 + return secretStoreErrorCode(err)

368 +}

src/go/plugin/agent/jobmgr/secretsctl/initial.go new

+49

@@ -0,0 +1,49 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretsctl

4 +

5 +import (

6 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

7 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

8 +)

9 +

10 +func (c *Controller) publishInitialConfig(rawCfg secretstore.Config) {

11 + cfg, prepErr := c.prepareConfigCandidate(rawCfg)

12 + c.handler.RememberDiscoveredConfig(cfg)

13 +

14 + if existing, ok := c.lookupInternal(cfg.ExposedKey()); ok {

15 + if shouldKeepExisting(cfg, existing) {

16 + return

17 + }

18 + if existing.Status == dyncfg.StatusRunning || existing.Status == dyncfg.StatusFailed {

19 + c.cb.Stop(existing.Cfg)

20 + c.cb.TakeCommandMessage()

21 + }

22 + }

23 +

24 + entry := &dyncfg.Entry[secretstore.Config]{Cfg: cfg, Status: dyncfg.StatusFailed}

25 + if prepErr == nil {

26 + if err := c.cb.Start(cfg); err == nil {

27 + entry.Status = dyncfg.StatusRunning

28 + }

29 + }

30 +

31 + c.exposed.Add(entry)

32 + c.handler.NotifyJobCreate(cfg, entry.Status)

33 +}

34 +

35 +func (c *Controller) prepareConfigCandidate(cfg secretstore.Config) (secretstore.Config, error) {

36 + return cfg, c.validateConfig(cfg)

37 +}

38 +

39 +func shouldKeepExisting(cfg secretstore.Config, existing *dyncfg.Entry[secretstore.Config]) bool {

40 + sp, ep := cfg.SourceTypePriority(), existing.Cfg.SourceTypePriority()

41 + if ep > sp {

42 + return true

43 + }

44 + if ep < sp {

45 + return false

46 + }

47 +

48 + return existing.Status == dyncfg.StatusRunning

49 +}

src/go/plugin/agent/jobmgr/secretstore_deps.go new

+337

@@ -0,0 +1,337 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "fmt"

7 + "regexp"

8 + "sort"

9 + "strings"

10 + "sync"

11 +

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

14 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

15 +)

16 +

17 +var (

18 + reSecretRef = regexp.MustCompile(`\$\{([^}]+)\}`)

19 + reUpperShorthand = regexp.MustCompile(`^[A-Z_][A-Z0-9_]*$`)

20 +)

21 +

22 +type secretStoreDeps struct {

23 + mu sync.RWMutex

24 +

25 + jobs map[string]*secretStoreJobState

26 +

27 + // storeKey -> internal job key -> ref

28 + exposed map[string]map[string]secretstore.JobRef

29 + running map[string]map[string]secretstore.JobRef

30 +}

31 +

32 +type secretStoreJobState struct {

33 + display string

34 + stores map[string]struct{}

35 + running bool

36 +}

37 +

38 +func newSecretStoreDeps() *secretStoreDeps {

39 + return &secretStoreDeps{

40 + jobs: make(map[string]*secretStoreJobState),

41 + exposed: make(map[string]map[string]secretstore.JobRef),

42 + running: make(map[string]map[string]secretstore.JobRef),

43 + }

44 +}

45 +

46 +func (d *secretStoreDeps) SetActiveJobStores(internalKey, display string, storeKeys []string) {

47 + internalKey = strings.TrimSpace(internalKey)

48 + if internalKey == "" {

49 + return

50 + }

51 + if display == "" {

52 + display = internalKey

53 + }

54 +

55 + normalized := normalizeStoreKeys(storeKeys)

56 +

57 + d.mu.Lock()

58 + defer d.mu.Unlock()

59 +

60 + state, ok := d.jobs[internalKey]

61 + if !ok {

62 + state = &secretStoreJobState{}

63 + d.jobs[internalKey] = state

64 + }

65 +

66 + for storeKey := range state.stores {

67 + d.removeRefLocked(d.exposed, storeKey, internalKey)

68 + if state.running {

69 + d.removeRefLocked(d.running, storeKey, internalKey)

70 + }

71 + }

72 +

73 + state.display = display

74 + state.stores = make(map[string]struct{}, len(normalized))

75 + for _, storeKey := range normalized {

76 + state.stores[storeKey] = struct{}{}

77 + ref := secretstore.JobRef{ID: internalKey, Display: display}

78 + d.addRefLocked(d.exposed, storeKey, ref)

79 + if state.running {

80 + d.addRefLocked(d.running, storeKey, ref)

81 + }

82 + }

83 +}

84 +

85 +func (d *secretStoreDeps) RemoveActiveJob(internalKey string) {

86 + internalKey = strings.TrimSpace(internalKey)

87 + if internalKey == "" {

88 + return

89 + }

90 +

91 + d.mu.Lock()

92 + defer d.mu.Unlock()

93 +

94 + state, ok := d.jobs[internalKey]

95 + if !ok {

96 + return

97 + }

98 +

99 + for storeKey := range state.stores {

100 + d.removeRefLocked(d.exposed, storeKey, internalKey)

101 + d.removeRefLocked(d.running, storeKey, internalKey)

102 + }

103 + delete(d.jobs, internalKey)

104 +}

105 +

106 +func (d *secretStoreDeps) setRunning(internalKey string, running bool) {

107 + internalKey = strings.TrimSpace(internalKey)

108 + if internalKey == "" {

109 + return

110 + }

111 +

112 + d.mu.Lock()

113 + defer d.mu.Unlock()

114 +

115 + state, ok := d.jobs[internalKey]

116 + if !ok {

117 + if !running {

118 + return

119 + }

120 + state = &secretStoreJobState{display: internalKey, stores: map[string]struct{}{}}

121 + d.jobs[internalKey] = state

122 + }

123 +

124 + if state.running == running {

125 + return

126 + }

127 +

128 + state.running = running

129 + for storeKey := range state.stores {

130 + ref := secretstore.JobRef{ID: internalKey, Display: state.display}

131 + if running {

132 + d.addRefLocked(d.running, storeKey, ref)

133 + } else {

134 + d.removeRefLocked(d.running, storeKey, internalKey)

135 + }

136 + }

137 +}

138 +

139 +func (d *secretStoreDeps) Impacted(storeKey string) (exposed []secretstore.JobRef, running []secretstore.JobRef) {

140 + storeKey = strings.TrimSpace(storeKey)

141 + if storeKey == "" {

142 + return nil, nil

143 + }

144 +

145 + d.mu.RLock()

146 + defer d.mu.RUnlock()

147 +

148 + exposed = collectSortedRefs(d.exposed[storeKey])

149 + running = collectSortedRefs(d.running[storeKey])

150 + return exposed, running

151 +}

152 +

153 +func (d *secretStoreDeps) addRefLocked(index map[string]map[string]secretstore.JobRef, storeKey string, ref secretstore.JobRef) {

154 + jobs, ok := index[storeKey]

155 + if !ok {

156 + jobs = make(map[string]secretstore.JobRef)

157 + index[storeKey] = jobs

158 + }

159 + jobs[ref.ID] = ref

160 +}

161 +

162 +func (d *secretStoreDeps) removeRefLocked(index map[string]map[string]secretstore.JobRef, storeKey, internalKey string) {

163 + jobs, ok := index[storeKey]

164 + if !ok {

165 + return

166 + }

167 + delete(jobs, internalKey)

168 + if len(jobs) == 0 {

169 + delete(index, storeKey)

170 + }

171 +}

172 +

173 +func collectSortedRefs(m map[string]secretstore.JobRef) []secretstore.JobRef {

174 + if len(m) == 0 {

175 + return nil

176 + }

177 + refs := make([]secretstore.JobRef, 0, len(m))

178 + for _, ref := range m {

179 + refs = append(refs, ref)

180 + }

181 + sort.Slice(refs, func(i, j int) bool {

182 + if refs[i].ID == refs[j].ID {

183 + return refs[i].Display < refs[j].Display

184 + }

185 + return refs[i].ID < refs[j].ID

186 + })

187 + return refs

188 +}

189 +

190 +func normalizeStoreKeys(storeKeys []string) []string {

191 + if len(storeKeys) == 0 {

192 + return nil

193 + }

194 + seen := make(map[string]struct{}, len(storeKeys))

195 + for _, key := range storeKeys {

196 + kind, name, err := secretstore.ParseStoreKey(key)

197 + if err != nil {

198 + continue

199 + }

200 + seen[secretstore.StoreKey(kind, name)] = struct{}{}

201 + }

202 + if len(seen) == 0 {

203 + return nil

204 + }

205 + out := make([]string, 0, len(seen))

206 + for key := range seen {

207 + out = append(out, key)

208 + }

209 + sort.Strings(out)

210 + return out

211 +}

212 +

213 +func extractSecretStoreKeys(cfg confgroup.Config) []string {

214 + seen := make(map[string]struct{})

215 + extractSecretStoreKeysFromValue(cfg, seen)

216 + if len(seen) == 0 {

217 + return nil

218 + }

219 + keys := make([]string, 0, len(seen))

220 + for key := range seen {

221 + keys = append(keys, key)

222 + }

223 + sort.Strings(keys)

224 + return keys

225 +}

226 +

227 +func extractSecretStoreKeysFromValue(v any, seen map[string]struct{}) {

228 + switch value := v.(type) {

229 + case string:

230 + extractSecretStoreKeysFromString(value, seen)

231 + case confgroup.Config:

232 + for k, entry := range value {

233 + if isSecretStoreInternalKey(k) {

234 + continue

235 + }

236 + extractSecretStoreKeysFromValue(entry, seen)

237 + }

238 + case map[string]any:

239 + for k, entry := range value {

240 + if isSecretStoreInternalKey(k) {

241 + continue

242 + }

243 + extractSecretStoreKeysFromValue(entry, seen)

244 + }

245 + case map[any]any:

246 + for rawKey, entry := range value {

247 + if key, ok := rawKey.(string); ok && isSecretStoreInternalKey(key) {

248 + continue

249 + }

250 + extractSecretStoreKeysFromValue(entry, seen)

251 + }

252 + case []any:

253 + for _, entry := range value {

254 + extractSecretStoreKeysFromValue(entry, seen)

255 + }

256 + }

257 +}

258 +

259 +func extractSecretStoreKeysFromString(value string, seen map[string]struct{}) {

260 + if !strings.Contains(value, "${") {

261 + return

262 + }

263 + matches := reSecretRef.FindAllStringSubmatch(value, -1)

264 + for _, match := range matches {

265 + if len(match) < 2 {

266 + continue

267 + }

268 + inner := match[1]

269 + scheme, rest, hasScheme := strings.Cut(inner, ":")

270 + if !hasScheme {

271 + if reUpperShorthand.MatchString(inner) {

272 + continue

273 + }

274 + continue

275 + }

276 + if scheme != "store" {

277 + continue

278 + }

279 + kindPart, tail, ok := strings.Cut(rest, ":")

280 + if !ok {

281 + continue

282 + }

283 + namePart, _, ok := strings.Cut(tail, ":")

284 + if !ok {

285 + continue

286 + }

287 + kind := secretstore.StoreKind(strings.TrimSpace(kindPart))

288 + name := strings.TrimSpace(namePart)

289 + if !kind.IsValid() {

290 + continue

291 + }

292 + if err := dyncfg.ValidateJobName(name); err != nil {

293 + continue

294 + }

295 + seen[secretstore.StoreKey(kind, name)] = struct{}{}

296 + }

297 +}

298 +

299 +func isSecretStoreInternalKey(k string) bool {

300 + return strings.HasPrefix(k, "__") && strings.HasSuffix(k, "__")

301 +}

302 +

303 +func secretStoreDisplay(cfg confgroup.Config) string {

304 + module := strings.TrimSpace(cfg.Module())

305 + job := strings.TrimSpace(cfg.Name())

306 + switch {

307 + case module != "" && job != "":

308 + return fmt.Sprintf("%s:%s", module, job)

309 + case cfg.FullName() != "":

310 + return cfg.FullName()

311 + default:

312 + return job

313 + }

314 +}

315 +

316 +func (m *Manager) syncSecretStoreDepsForConfig(cfg confgroup.Config) {

317 + if m.secretStoreDeps == nil {

318 + return

319 + }

320 + m.secretStoreDeps.SetActiveJobStores(cfg.FullName(), secretStoreDisplay(cfg), extractSecretStoreKeys(cfg))

321 +}

322 +

323 +func (m *Manager) syncSecretStoreDepsByFunction(fn dyncfg.Function) {

324 + if m.secretStoreDeps == nil || m.collectorCallbacks == nil {

325 + return

326 + }

327 + key, _, ok := m.collectorCallbacks.ExtractKey(fn)

328 + if !ok {

329 + return

330 + }

331 + entry, ok := m.collectorExposed.LookupByKey(key)

332 + if !ok {

333 + m.secretStoreDeps.RemoveActiveJob(key)

334 + return

335 + }

336 + m.syncSecretStoreDepsForConfig(entry.Cfg)

337 +}

src/go/plugin/agent/jobmgr/secretstore_deps_test.go new

+198

@@ -0,0 +1,198 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "bytes"

7 + "context"

8 + "testing"

9 +

10 + "github.com/stretchr/testify/assert"

11 + "github.com/stretchr/testify/require"

12 +

13 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

14 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

15 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

18 +)

19 +

20 +func TestExtractSecretStoreKeys(t *testing.T) {

21 + tests := map[string]struct {

22 + cfg map[string]any

23 + want []string

24 + }{

25 + "single store reference": {

26 + cfg: map[string]any{

27 + "password": "${store:aws-sm:aws_prod:db/password}",

28 + },

29 + want: []string{"aws-sm:aws_prod"},

30 + },

31 + "multiple refs with non-store refs ignored": {

32 + cfg: map[string]any{

33 + "dsn": "postgres://${env:USER}:${store:vault:vault_prod:secret/data/db#password}@host/${store:aws-sm:aws_prod:db#pw}",

34 + },

35 + want: []string{"aws-sm:aws_prod", "vault:vault_prod"},

36 + },

37 + "nested structures and internal keys": {

38 + cfg: map[string]any{

39 + "__source__": "${store:vault:ignored:foo}",

40 + "outer": map[string]any{

41 + "items": []any{

42 + "${store:gcp-sm:gcp_prod:project/secret/latest}",

43 + map[any]any{"x": "${store:azure-kv:az_prod:vault/secret}"},

44 + },

45 + },

46 + },

47 + want: []string{"azure-kv:az_prod", "gcp-sm:gcp_prod"},

48 + },

49 + "invalid store refs are ignored": {

50 + cfg: map[string]any{

51 + "a": "${store::bad:secret}",

52 + "b": "${store:bad-id:thing:path}",

53 + "c": "${store:vault:good_1:path}",

54 + },

55 + want: []string{"vault:good_1"},

56 + },

57 + }

58 +

59 + for name, tc := range tests {

60 + t.Run(name, func(t *testing.T) {

61 + cfg := prepareUserCfg("mod", "job")

62 + for k, v := range tc.cfg {

63 + cfg[k] = v

64 + }

65 + got := extractSecretStoreKeys(cfg)

66 + assert.Equal(t, tc.want, got)

67 + })

68 + }

69 +}

70 +

71 +func TestSecretStoreDepsImpacted(t *testing.T) {

72 + tests := map[string]struct {

73 + run func(d *secretStoreDeps)

74 + storeKey string

75 + wantExposed []secretstore.JobRef

76 + wantRunning []secretstore.JobRef

77 + }{

78 + "set active and running": {

79 + run: func(d *secretStoreDeps) {

80 + d.SetActiveJobStores("mysql_prod", "mysql:prod", []string{"vault:vault_prod"})

81 + d.setRunning("mysql_prod", true)

82 + },

83 + storeKey: "vault:vault_prod",

84 + wantExposed: []secretstore.JobRef{

85 + {ID: "mysql_prod", Display: "mysql:prod"},

86 + },

87 + wantRunning: []secretstore.JobRef{

88 + {ID: "mysql_prod", Display: "mysql:prod"},

89 + },

90 + },

91 + "replace active stores keeps running alignment": {

92 + run: func(d *secretStoreDeps) {

93 + d.SetActiveJobStores("mysql_prod", "mysql:prod", []string{"vault:vault_prod"})

94 + d.setRunning("mysql_prod", true)

95 + d.SetActiveJobStores("mysql_prod", "mysql:prod", []string{"aws-sm:aws_prod"})

96 + },

97 + storeKey: "aws-sm:aws_prod",

98 + wantExposed: []secretstore.JobRef{

99 + {ID: "mysql_prod", Display: "mysql:prod"},

100 + },

101 + wantRunning: []secretstore.JobRef{

102 + {ID: "mysql_prod", Display: "mysql:prod"},

103 + },

104 + },

105 + "remove active job clears impacted": {

106 + run: func(d *secretStoreDeps) {

107 + d.SetActiveJobStores("mysql_prod", "mysql:prod", []string{"vault:vault_prod"})

108 + d.setRunning("mysql_prod", true)

109 + d.RemoveActiveJob("mysql_prod")

110 + },

111 + storeKey: "vault:vault_prod",

112 + wantExposed: nil,

113 + wantRunning: nil,

114 + },

115 + "non-running job appears only in exposed": {

116 + run: func(d *secretStoreDeps) {

117 + d.SetActiveJobStores("redis_prod", "redis:prod", []string{"gcp-sm:gcp_prod"})

118 + d.setRunning("redis_prod", false)

119 + },

120 + storeKey: "gcp-sm:gcp_prod",

121 + wantExposed: []secretstore.JobRef{

122 + {ID: "redis_prod", Display: "redis:prod"},

123 + },

124 + wantRunning: nil,

125 + },

126 + }

127 +

128 + for name, tc := range tests {

129 + t.Run(name, func(t *testing.T) {

130 + deps := newSecretStoreDeps()

131 + tc.run(deps)

132 +

133 + exposed, running := deps.Impacted(tc.storeKey)

134 + assert.Equal(t, tc.wantExposed, exposed)

135 + assert.Equal(t, tc.wantRunning, running)

136 + })

137 + }

138 +}

139 +

140 +func TestDyncfgTestDoesNotMutateSecretStoreDeps(t *testing.T) {

141 + tests := map[string]struct{}{

142 + "command test leaves dependency index unchanged": {},

143 + }

144 +

145 + for name := range tests {

146 + t.Run(name, func(t *testing.T) {

147 + var buf bytes.Buffer

148 + mgr := newCollectorTestManager()

149 + mgr.ctx = context.Background()

150 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&buf))))

151 +

152 + cfg := prepareDyncfgCfg("success", "job")

153 + cfg["password"] = "${store:vault:vault_prod:secret/data/mysql#password}"

154 + mgr.syncSecretStoreDepsForConfig(cfg)

155 + mgr.secretStoreDeps.setRunning(cfg.FullName(), true)

156 +

157 + beforeExposed, beforeRunning := mgr.secretStoreDeps.Impacted("vault:vault_prod")

158 + require.Len(t, beforeExposed, 1)

159 + require.Len(t, beforeRunning, 1)

160 +

161 + fn := dyncfg.NewFunction(functions.Function{

162 + UID: "secretstore-deps-test",

163 + ContentType: "application/json",

164 + Payload: mustMarshalCollectorConfigPayload(t, prepareDyncfgCfg("success", "job")),

165 + Args: []string{mgr.dyncfgModID("success"), string(dyncfg.CommandTest), "job"},

166 + })

167 +

168 + mgr.dyncfgCollectorSeqExec(fn)

169 + mgr.cmdTestWG.Wait()

170 +

171 + afterExposed, afterRunning := mgr.secretStoreDeps.Impacted("vault:vault_prod")

172 + assert.Equal(t, beforeExposed, afterExposed)

173 + assert.Equal(t, beforeRunning, afterRunning)

174 + })

175 + }

176 +}

177 +

178 +func TestSecretStoreDepsNoStateLeakOnRemoveThenStop(t *testing.T) {

179 + tests := map[string]struct{}{

180 + "setRunning false on missing state is no-op": {},

181 + }

182 +

183 + for name := range tests {

184 + t.Run(name, func(t *testing.T) {

185 + deps := newSecretStoreDeps()

186 +

187 + deps.SetActiveJobStores("mysql_prod", "mysql:prod", []string{"vault:vault_prod"})

188 + deps.setRunning("mysql_prod", true)

189 + deps.RemoveActiveJob("mysql_prod")

190 + deps.setRunning("mysql_prod", false)

191 +

192 + assert.Empty(t, deps.jobs)

193 + exposed, running := deps.Impacted("vault:vault_prod")

194 + assert.Empty(t, exposed)

195 + assert.Empty(t, running)

196 + })

197 + }

198 +}

src/go/plugin/agent/jobmgr/secretstore_flow_test.go new

+331

@@ -0,0 +1,331 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package jobmgr

4 +

5 +import (

6 + "bytes"

7 + "context"

8 + "fmt"

9 + "testing"

10 + "time"

11 +

12 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

13 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

14 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/resolver"

15 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

18 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

19 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

20 + "github.com/stretchr/testify/assert"

21 + "github.com/stretchr/testify/require"

22 +)

23 +

24 +type TestStoreConfig struct {

25 + Value string `yaml:"value" json:"value"`

26 +}

27 +

28 +type testStore struct {

29 + TestStoreConfig `yaml:",inline" json:""`

30 +}

31 +

32 +func (s *testStore) Configuration() any { return &s.TestStoreConfig }

33 +

34 +func (s *testStore) Init(context.Context) error {

35 + if s.TestStoreConfig.Value == "" {

36 + return fmt.Errorf("value is required")

37 + }

38 + return nil

39 +}

40 +

41 +func (s *testStore) Publish() secretstore.PublishedStore {

42 + return &testPublishedStore{value: s.TestStoreConfig.Value}

43 +}

44 +

45 +type testPublishedStore struct {

46 + value string

47 +}

48 +

49 +func (s *testPublishedStore) Resolve(_ context.Context, req secretstore.ResolveRequest) (string, error) {

50 + if req.Operand != "value" {

51 + return "", fmt.Errorf("unexpected operand %q", req.Operand)

52 + }

53 + return s.value, nil

54 +}

55 +

56 +type secretAwareCollector struct {

57 + collectorapi.Base

58 + Config collectorapi.MockConfiguration `yaml:",inline" json:""`

59 +}

60 +

61 +func (c *secretAwareCollector) Configuration() any { return c.Config }

62 +func (c *secretAwareCollector) Check(context.Context) error { return nil }

63 +func (c *secretAwareCollector) Cleanup(context.Context) {}

64 +func (c *secretAwareCollector) Charts() *collectorapi.Charts { return &collectorapi.Charts{} }

65 +func (c *secretAwareCollector) Collect(context.Context) map[string]int64 {

66 + return map[string]int64{"value": 1}

67 +}

68 +

69 +func (c *secretAwareCollector) Init(context.Context) error {

70 + if c.Config.OptionStr != "good" {

71 + return fmt.Errorf("secret is not usable: %s", c.Config.OptionStr)

72 + }

73 + return nil

74 +}

75 +

76 +func newTestSecretStoreService() secretstore.Service {

77 + return secretstore.NewService(secretstore.Creator{

78 + Kind: secretstore.KindVault,

79 + DisplayName: "Vault",

80 + Schema: `{"jsonSchema":{"type":"object","properties":{"value":{"type":"string"}}},"uiSchema":[]}`,

81 + Create: func() secretstore.Store {

82 + return &testStore{}

83 + },

84 + })

85 +}

86 +

87 +func TestApplyConfig_ResolvesStoreReferenceWithKindAndName(t *testing.T) {

88 + svc := newTestSecretStoreService()

89 + raw := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "resolved-secret"}, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

90 + require.NoError(t, svc.Add(raw))

91 +

92 + cfg := prepareDyncfgCfg("success", "secret-job").

93 + Set("option_str", "${store:vault:vault_prod:value}").

94 + Set("option_int", 7)

95 +

96 + module := &collectorapi.MockCollectorV1{}

97 + err := applyConfig(t.Context(), cfg, module, secretresolver.New(), svc, svc.Capture())

98 + require.NoError(t, err)

99 +

100 + assert.Equal(t, "resolved-secret", module.Config.OptionStr)

101 + assert.Equal(t, 7, module.Config.OptionInt)

102 +}

103 +

104 +func TestRun_StartupLoadedSecretStoreIsAvailableToFirstCollectorStart(t *testing.T) {

105 + initial := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{"value": "good"}, "file=/etc/netdata/go.d/ss/vault.conf", confgroup.TypeUser)

106 + mgr := New(Config{

107 + PluginName: testPluginName,

108 + SecretStores: []secretstore.Config{initial},

109 + SecretStoreService: newTestSecretStoreService(),

110 + })

111 + mgr.modules = collectorapi.Registry{

112 + "gated": {

113 + Create: func() collectorapi.CollectorV1 { return &secretAwareCollector{} },

114 + },

115 + }

116 +

117 + var out bytes.Buffer

118 + mgr.SetDyncfgResponder(dyncfg.NewResponder(netdataapi.New(safewriter.New(&out))))

119 + mgr.runModePolicy.AutoEnableDiscovered = true

120 +

121 + ctx, cancel := context.WithCancel(context.Background())

122 + in := make(chan []*confgroup.Group)

123 + done := make(chan struct{})

124 + go func() {

125 + defer close(done)

126 + mgr.Run(ctx, in)

127 + }()

128 +

129 + waitCtx, waitCancel := context.WithTimeout(context.Background(), time.Second)

130 + defer waitCancel()

131 + require.True(t, mgr.WaitStarted(waitCtx), "manager did not report started")

132 +

133 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

134 + entry, ok := mgr.lookupSecretStoreEntry(key)

135 + require.True(t, ok)

136 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

137 + _, ok = mustSecretStoreService(t, mgr).GetStatus(key)

138 + assert.True(t, ok)

139 +

140 + cfg := prepareDyncfgCfg("gated", "startup").

141 + Set("option_str", "${store:vault:vault_prod:value}").

142 + Set("option_int", 1)

143 + mgr.addConfig(cfg)

144 +

145 + jobEntry, ok := mgr.lookupExposedByFullName(cfg.FullName())

146 + require.True(t, ok)

147 + assert.Equal(t, dyncfg.StatusRunning, jobEntry.Status)

148 + require.Len(t, mgr.runningJobs.snapshot(), 1)

149 + assert.Equal(t, cfg.FullName(), mgr.runningJobs.snapshot()[0].FullName())

150 +

151 + cancel()

152 + close(in)

153 +

154 + select {

155 + case <-done:

156 + case <-time.After(2 * time.Second):

157 + t.Fatal("manager did not stop after cancel")

158 + }

159 +}

160 +

161 +func TestDyncfgSecretStoreUpdate_DependentRestartBehavior(t *testing.T) {

162 + tests := map[string]struct {

163 + run func(t *testing.T, mgr *Manager, out *bytes.Buffer)

164 + }{

165 + "restarts failed dependent after store is fixed": {

166 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

167 + mgr.modules["gated"] = collectorapi.Creator{

168 + Create: func() collectorapi.CollectorV1 { return &secretAwareCollector{} },

169 + }

170 +

171 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

172 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", map[string]any{"value": "good"}, dyncfg.StatusRunning)

173 +

174 + cfg := prepareDyncfgCfg("gated", "mysql").

175 + Set("option_str", "${store:vault:vault_prod:value}").

176 + Set("option_int", 1)

177 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

178 + Cfg: cfg,

179 + Status: dyncfg.StatusRunning,

180 + })

181 + mgr.syncSecretStoreDepsForConfig(cfg)

182 + require.NoError(t, mgr.collectorCallbacks.Start(cfg))

183 +

184 + _, running := mgr.secretStoreDeps.Impacted(key)

185 + require.Len(t, running, 1)

186 + assert.Equal(t, cfg.FullName(), running[0].ID)

187 +

188 + badFn := dyncfg.NewFunction(functions.Function{

189 + UID: "ss-update-bad",

190 + ContentType: "application/json",

191 + Payload: mustJSON(t, map[string]any{"value": "bad"}),

192 + Args: []string{

193 + mgr.dyncfgSecretStoreID(key),

194 + string(dyncfg.CommandUpdate),

195 + },

196 + })

197 + mgr.dyncfgSecretStoreSeqExec(badFn)

198 +

199 + var badResp map[string]any

200 + mustDecodeFunctionPayload(t, out.String(), "ss-update-bad", &badResp)

201 + assert.Equal(t, float64(200), badResp["status"])

202 + assert.Contains(t, badResp["message"], "Secretstore change applied, but dependent collector restarts failed")

203 + assert.Contains(t, badResp["message"], "gated:mysql")

204 +

205 + entry, ok := mgr.lookupExposedByFullName(cfg.FullName())

206 + require.True(t, ok)

207 + assert.Equal(t, dyncfg.StatusFailed, entry.Status)

208 +

209 + _, running = mgr.secretStoreDeps.Impacted(key)

210 + assert.Empty(t, running)

211 +

212 + goodFn := dyncfg.NewFunction(functions.Function{

213 + UID: "ss-update-good",

214 + ContentType: "application/json",

215 + Payload: mustJSON(t, map[string]any{"value": "good"}),

216 + Args: []string{

217 + mgr.dyncfgSecretStoreID(key),

218 + string(dyncfg.CommandUpdate),

219 + },

220 + })

221 + mgr.dyncfgSecretStoreSeqExec(goodFn)

222 +

223 + var goodResp map[string]any

224 + mustDecodeFunctionPayload(t, out.String(), "ss-update-good", &goodResp)

225 + assert.Equal(t, float64(200), goodResp["status"])

226 + assert.Equal(t, "", goodResp["message"])

227 +

228 + entry, ok = mgr.lookupExposedByFullName(cfg.FullName())

229 + require.True(t, ok)

230 + assert.Equal(t, dyncfg.StatusRunning, entry.Status)

231 +

232 + _, running = mgr.secretStoreDeps.Impacted(key)

233 + require.Len(t, running, 1)

234 + assert.Equal(t, cfg.FullName(), running[0].ID)

235 + },

236 + },

237 + "ignores accepted and disabled dependents": {

238 + run: func(t *testing.T, mgr *Manager, out *bytes.Buffer) {

239 + key := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

240 + seedSecretStore(t, mgr, secretstore.KindVault, "vault_prod", map[string]any{"value": "good"}, dyncfg.StatusRunning)

241 +

242 + acceptedCfg := prepareDyncfgCfg("success", "accepted")

243 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

244 + Cfg: acceptedCfg,

245 + Status: dyncfg.StatusAccepted,

246 + })

247 + mgr.secretStoreDeps.SetActiveJobStores(acceptedCfg.FullName(), "success:accepted", []string{key})

248 +

249 + disabledCfg := prepareDyncfgCfg("success", "disabled")

250 + mgr.collectorExposed.Add(&dyncfg.Entry[confgroup.Config]{

251 + Cfg: disabledCfg,

252 + Status: dyncfg.StatusDisabled,

253 + })

254 + mgr.secretStoreDeps.SetActiveJobStores(disabledCfg.FullName(), "success:disabled", []string{key})

255 +

256 + updateFn := dyncfg.NewFunction(functions.Function{

257 + UID: "ss-update-ignored",

258 + ContentType: "application/json",

259 + Payload: mustJSON(t, map[string]any{"value": "better"}),

260 + Args: []string{

261 + mgr.dyncfgSecretStoreID(key),

262 + string(dyncfg.CommandUpdate),

263 + },

264 + })

265 + mgr.dyncfgSecretStoreSeqExec(updateFn)

266 +

267 + var resp map[string]any

268 + mustDecodeFunctionPayload(t, out.String(), "ss-update-ignored", &resp)

269 + assert.Equal(t, float64(200), resp["status"])

270 + assert.Equal(t, "", resp["message"])

271 +

272 + acceptedEntry, ok := mgr.lookupExposedByFullName(acceptedCfg.FullName())

273 + require.True(t, ok)

274 + assert.Equal(t, dyncfg.StatusAccepted, acceptedEntry.Status)

275 +

276 + disabledEntry, ok := mgr.lookupExposedByFullName(disabledCfg.FullName())

277 + require.True(t, ok)

278 + assert.Equal(t, dyncfg.StatusDisabled, disabledEntry.Status)

279 +

280 + _, running := mgr.secretStoreDeps.Impacted(key)

281 + assert.Empty(t, running)

282 + },

283 + },

284 + }

285 +

286 + for name, tc := range tests {

287 + t.Run(name, func(t *testing.T) {

288 + mgr, out := newDyncfgSecretStoreTestManagerWithService(newTestSecretStoreService())

289 + tc.run(t, mgr, out)

290 + })

291 + }

292 +}

293 +

294 +func TestDyncfgSecretStoreGet_CanonicalJSONDoesNotExposeUnknownFields(t *testing.T) {

295 + mgr, out := newDyncfgSecretStoreTestManagerWithService(newTestSecretStoreService())

296 +

297 + cfg := newSecretStoreConfigWithSource(t, secretstore.KindVault, "vault_prod", map[string]any{

298 + "value": "resolved-secret",

299 + "ignored": "drop-me",

300 + }, confgroup.TypeDyncfg, confgroup.TypeDyncfg)

301 + _, changed, err := mgr.rememberSecretStoreConfig(cfg)

302 + require.NoError(t, err)

303 + require.True(t, changed)

304 +

305 + getFn := dyncfg.NewFunction(functions.Function{

306 + UID: "ss-get-canonical",

307 + Args: []string{mgr.dyncfgSecretStoreID(secretstore.StoreKey(secretstore.KindVault, "vault_prod")), string(dyncfg.CommandGet)},

308 + })

309 + mgr.dyncfgSecretStoreSeqExec(getFn)

310 +

311 + var got map[string]any

312 + mustDecodeFunctionPayload(t, out.String(), "ss-get-canonical", &got)

313 + assert.Equal(t, "resolved-secret", got["value"])

314 + _, ok := got["ignored"]

315 + assert.False(t, ok)

316 +}

317 +

318 +func TestSecretStoreConfigFromPayload_PreservesKindAndNameForStoreSyntax(t *testing.T) {

319 + mgr, _ := newDyncfgSecretStoreTestManagerWithService(newTestSecretStoreService())

320 +

321 + fn := dyncfg.NewFunction(functions.Function{

322 + ContentType: "application/json",

323 + Payload: mustJSON(t, map[string]any{"value": "resolved-secret"}),

324 + })

325 + cfg, err := mgr.secretStoreConfigFromPayload(fn, "vault_prod", secretstore.KindVault)

326 + require.NoError(t, err)

327 +

328 + assert.Equal(t, "vault_prod", cfg.Name())

329 + assert.Equal(t, secretstore.KindVault, cfg.Kind())

330 + assert.Equal(t, "resolved-secret", cfg["value"])

331 +}

src/go/plugin/agent/jobmgr/sim_test.go

+6 -6

@@ -110,8 +110,8 @@ func (s *runSim) run(t *testing.T) {

110 expectedResults := strings.Count(s.wantDyncfg, "FUNCTION_RESULT_END")

111 require.Eventually(t, func() bool {

112 return countDiscovered(mgr) == len(s.wantDiscovered) &&

113 - mgr.seen.Count() == len(s.wantSeen) &&

114 - mgr.exposed.Count() == len(s.wantExposed) &&

113 + mgr.collectorSeen.Count() == len(s.wantSeen) &&

114 + mgr.collectorExposed.Count() == len(s.wantExposed) &&

115 runningSetMatches(mgr.runningJobs.snapshot(), s.wantRunning) &&

116 out.FuncResultCount() >= expectedResults

117 }, timeout, 10*time.Millisecond, "manager state did not settle before shutdown")

@@ -172,19 +172,19 @@ func (s *runSim) run(t *testing.T) {

172 require.Truef(t, ok, "discoveredConfigs: source %s config %d is not found", cfg.Source(), cfg.Hash())

173 }

174

175 - wantLen, gotLen = len(s.wantSeen), mgr.seen.Count()

175 + wantLen, gotLen = len(s.wantSeen), mgr.collectorSeen.Count()

176 require.Equalf(t, wantLen, gotLen, "seen: different len (want %d got %d)", wantLen, gotLen)

177

178 for _, cfg := range s.wantSeen {

179 - _, ok := mgr.seen.Lookup(cfg)

179 + _, ok := mgr.collectorSeen.Lookup(cfg)

180 require.Truef(t, ok, "seen: config '%s' is not found", cfg.UID())

181 }

182

183 - wantLen, gotLen = len(s.wantExposed), mgr.exposed.Count()

183 + wantLen, gotLen = len(s.wantExposed), mgr.collectorExposed.Count()

184 require.Equalf(t, wantLen, gotLen, "exposed: different len (want %d got %d)", wantLen, gotLen)

185

186 for _, we := range s.wantExposed {

187 - entry, ok := mgr.exposed.LookupByKey(we.cfg.ExposedKey())

187 + entry, ok := mgr.collectorExposed.LookupByKey(we.cfg.ExposedKey())

188 require.Truef(t, ok && we.cfg.UID() == entry.Cfg.UID(), "exposed: config '%s' is not found", we.cfg.UID())

189 require.Truef(t, we.status == entry.Status, "exposed: wrong status for '%s', want %s got %s", we.cfg.UID(), we.status, entry.Status)

190 }

src/go/plugin/agent/jobmgr/vnodectl/controller.go new

+118

@@ -0,0 +1,118 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vnodectl

4 +

5 +import (

6 + "fmt"

7 +

8 + "github.com/netdata/netdata/go/plugins/logger"

9 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

10 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

13 +)

14 +

15 +const (

16 + dyncfgVnodeIDf = "%s:vnode"

17 + dyncfgVnodePath = "/collectors/%s/Vnodes"

18 +)

19 +

20 +type Options struct {

21 + Logger *logger.Logger

22 + API *dyncfg.Responder

23 + Plugin string

24 + Initial map[string]*vnodes.VirtualNode

25 +

26 + AffectedJobs func(string) []string

27 + ApplyVnodeUpdate func(string, *vnodes.VirtualNode)

28 +}

29 +

30 +type Controller struct {

31 + *logger.Logger

32 +

33 + api *dyncfg.Responder

34 + pluginName string

35 + store *vnodeStore

36 +

37 + affectedJobs func(string) []string

38 + applyVnodeUpdate func(string, *vnodes.VirtualNode)

39 +}

40 +

41 +func New(opts Options) *Controller {

42 + log := opts.Logger

43 + if log == nil {

44 + log = logger.New()

45 + }

46 +

47 + return &Controller{

48 + Logger: log,

49 + api: opts.API,

50 + pluginName: opts.Plugin,

51 + store: newVnodeStore(opts.Initial),

52 + affectedJobs: opts.AffectedJobs,

53 + applyVnodeUpdate: opts.ApplyVnodeUpdate,

54 + }

55 +}

56 +

57 +func (c *Controller) Prefix() string {

58 + return fmt.Sprintf(dyncfgVnodeIDf, c.pluginName)

59 +}

60 +

61 +func (c *Controller) SetAPI(api *dyncfg.Responder) {

62 + if api == nil {

63 + // Nil means "keep the current responder" rather than clearing output wiring.

64 + return

65 + }

66 + c.api = api

67 +}

68 +

69 +func (c *Controller) Lookup(name string) (*vnodes.VirtualNode, bool) {

70 + if c.store == nil {

71 + return nil, false

72 + }

73 + return c.store.Lookup(name)

74 +}

75 +

76 +func (c *Controller) CreateTemplates() {

77 + if c.api == nil {

78 + return

79 + }

80 + c.api.ConfigCreate(netdataapi.ConfigOpts{

81 + ID: c.Prefix(),

82 + Status: dyncfg.StatusAccepted.String(),

83 + ConfigType: dyncfg.ConfigTypeTemplate.String(),

84 + Path: fmt.Sprintf(dyncfgVnodePath, c.pluginName),

85 + SourceType: "internal",

86 + Source: "internal",

87 + SupportedCommands: dyncfgVnodeModCmds(),

88 + })

89 +}

90 +

91 +func (c *Controller) PublishExisting(status dyncfg.Status) {

92 + if c.store == nil {

93 + return

94 + }

95 + c.store.ForEach(func(cfg *vnodes.VirtualNode) bool {

96 + c.createJob(cfg, status)

97 + return true

98 + })

99 +}

100 +

101 +func (c *Controller) configID(name string) string {

102 + return fmt.Sprintf("%s:%s", c.Prefix(), name)

103 +}

104 +

105 +func (c *Controller) createJob(cfg *vnodes.VirtualNode, status dyncfg.Status) {

106 + if c.api == nil || cfg == nil {

107 + return

108 + }

109 + c.api.ConfigCreate(netdataapi.ConfigOpts{

110 + ID: c.configID(cfg.Name),

111 + Status: status.String(),

112 + ConfigType: dyncfg.ConfigTypeJob.String(),

113 + Path: fmt.Sprintf(dyncfgVnodePath, c.pluginName),

114 + SourceType: cfg.SourceType,

115 + Source: cfg.Source,

116 + SupportedCommands: dyncfgVnodeJobCmds(cfg.SourceType == confgroup.TypeDyncfg),

117 + })

118 +}

src/go/plugin/agent/jobmgr/vnodectl/controller_test.go new

+476

@@ -0,0 +1,476 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vnodectl

4 +

5 +import (

6 + "bytes"

7 + "encoding/json"

8 + "fmt"

9 + "regexp"

10 + "testing"

11 +

12 + "github.com/netdata/netdata/go/plugins/logger"

13 + "github.com/netdata/netdata/go/plugins/pkg/netdataapi"

14 + "github.com/netdata/netdata/go/plugins/pkg/safewriter"

15 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

16 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

17 + "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

18 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

19 + "github.com/stretchr/testify/assert"

20 + "github.com/stretchr/testify/require"

21 +)

22 +

23 +const testPluginName = "test"

24 +

25 +func TestControllerSeqExec(t *testing.T) {

26 + tests := map[string]struct {

27 + initial map[string]*vnodes.VirtualNode

28 + run func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams)

29 + }{

30 + "schema dispatch": {

31 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

32 + fn := dyncfg.NewFunction(functions.Function{UID: "vn-schema", Args: []string{ctl.Prefix(), string(dyncfg.CommandSchema)}})

33 + ctl.SeqExec(fn)

34 +

35 + assert.Contains(t, out.String(), "FUNCTION_RESULT_BEGIN vn-schema 200 application/json")

36 + assert.Empty(t, seams.affectedJobsCalls)

37 + assert.Empty(t, seams.applyCalls)

38 + },

39 + },

40 + "userconfig generation": {

41 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

42 + fn := dyncfg.NewFunction(functions.Function{

43 + UID: "vn-userconfig",

44 + ContentType: "application/json",

45 + Payload: mustJSON(t, map[string]any{

46 + "guid": "11111111-1111-1111-1111-111111111111",

47 + "labels": map[string]string{"env": "prod"},

48 + }),

49 + Args: []string{ctl.Prefix(), string(dyncfg.CommandUserconfig), "db"},

50 + })

51 + ctl.SeqExec(fn)

52 +

53 + body := mustFunctionBody(t, out.String(), "vn-userconfig")

54 + assert.Contains(t, out.String(), "FUNCTION_RESULT_BEGIN vn-userconfig 200 application/yaml")

55 + assert.Contains(t, body, "name: db")

56 + assert.Contains(t, body, "hostname: db")

57 + assert.Empty(t, seams.affectedJobsCalls)

58 + assert.Empty(t, seams.applyCalls)

59 + },

60 + },

61 + "get returns stored config": {

62 + initial: map[string]*vnodes.VirtualNode{

63 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

64 + },

65 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

66 + fn := dyncfg.NewFunction(functions.Function{UID: "vn-get", Args: []string{ctl.configID("db"), string(dyncfg.CommandGet)}})

67 + ctl.SeqExec(fn)

68 +

69 + var payload map[string]any

70 + mustDecodeFunctionPayload(t, out.String(), "vn-get", &payload)

71 + assert.Equal(t, "db", payload["name"])

72 + assert.Equal(t, "11111111-1111-1111-1111-111111111111", payload["guid"])

73 + assert.Empty(t, seams.affectedJobsCalls)

74 + assert.Empty(t, seams.applyCalls)

75 + },

76 + },

77 + "add applies vnode update seam": {

78 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

79 + fn := dyncfg.NewFunction(functions.Function{

80 + UID: "vn-add",

81 + ContentType: "application/json",

82 + Payload: mustJSON(t, map[string]any{

83 + "guid": "11111111-1111-1111-1111-111111111111",

84 + "labels": map[string]string{"env": "prod"},

85 + }),

86 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), "db"},

87 + })

88 + ctl.SeqExec(fn)

89 +

90 + var payload map[string]any

91 + mustDecodeFunctionPayload(t, out.String(), "vn-add", &payload)

92 + assert.Equal(t, float64(202), payload["status"])

93 + assert.Equal(t, []string{"db"}, seams.applyCalls)

94 + cfg, ok := ctl.Lookup("db")

95 + require.True(t, ok)

96 + assert.Equal(t, confgroup.TypeDyncfg, cfg.SourceType)

97 + },

98 + },

99 + "add no-op keeps apply seam unused": {

100 + initial: map[string]*vnodes.VirtualNode{

101 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

102 + },

103 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

104 + fn := dyncfg.NewFunction(functions.Function{

105 + UID: "vn-add-noop",

106 + ContentType: "application/json",

107 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

108 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), "db"},

109 + })

110 + ctl.SeqExec(fn)

111 +

112 + var payload map[string]any

113 + mustDecodeFunctionPayload(t, out.String(), "vn-add-noop", &payload)

114 + assert.Equal(t, float64(202), payload["status"])

115 + assert.Empty(t, seams.applyCalls)

116 + },

117 + },

118 + "add equal user vnode rewrites stored source metadata to dyncfg": {

119 + initial: map[string]*vnodes.VirtualNode{

120 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeUser),

121 + },

122 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

123 + fn := dyncfg.NewFunction(functions.Function{

124 + UID: "vn-add-promote-user",

125 + Source: "user=alice",

126 + ContentType: "application/json",

127 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

128 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), "db"},

129 + })

130 + ctl.SeqExec(fn)

131 +

132 + var payload map[string]any

133 + mustDecodeFunctionPayload(t, out.String(), "vn-add-promote-user", &payload)

134 + assert.Equal(t, float64(202), payload["status"])

135 + assert.Equal(t, []string{"db"}, seams.applyCalls)

136 +

137 + cfg, ok := ctl.Lookup("db")

138 + require.True(t, ok)

139 + assert.Equal(t, confgroup.TypeDyncfg, cfg.SourceType)

140 + assert.Equal(t, "user=alice", cfg.Source)

141 + },

142 + },

143 + "duplicate hostname is rejected": {

144 + initial: map[string]*vnodes.VirtualNode{

145 + "other": testVnode("other", "shared", "22222222-2222-2222-2222-222222222222", confgroup.TypeUser),

146 + },

147 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

148 + fn := dyncfg.NewFunction(functions.Function{

149 + UID: "vn-add-dup-host",

150 + ContentType: "application/json",

151 + Payload: mustJSON(t, map[string]any{

152 + "guid": "33333333-3333-3333-3333-333333333333",

153 + "hostname": "shared",

154 + }),

155 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), "db"},

156 + })

157 + ctl.SeqExec(fn)

158 +

159 + var payload map[string]any

160 + mustDecodeFunctionPayload(t, out.String(), "vn-add-dup-host", &payload)

161 + assert.Equal(t, float64(400), payload["status"])

162 + assert.Contains(t, fmt.Sprint(payload["errorMessage"]), "duplicate virtual node hostname")

163 + assert.Empty(t, seams.applyCalls)

164 + },

165 + },

166 + "update applies vnode update seam": {

167 + initial: map[string]*vnodes.VirtualNode{

168 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

169 + },

170 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

171 + fn := dyncfg.NewFunction(functions.Function{

172 + UID: "vn-update",

173 + ContentType: "application/json",

174 + Payload: mustJSON(t, map[string]any{

175 + "guid": "11111111-1111-1111-1111-111111111111",

176 + "labels": map[string]string{"team": "db"},

177 + }),

178 + Args: []string{ctl.configID("db"), string(dyncfg.CommandUpdate)},

179 + })

180 + ctl.SeqExec(fn)

181 +

182 + var payload map[string]any

183 + mustDecodeFunctionPayload(t, out.String(), "vn-update", &payload)

184 + assert.Equal(t, float64(202), payload["status"])

185 + assert.Equal(t, []string{"db"}, seams.applyCalls)

186 + },

187 + },

188 + "update no-op keeps apply seam unused": {

189 + initial: map[string]*vnodes.VirtualNode{

190 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

191 + },

192 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

193 + fn := dyncfg.NewFunction(functions.Function{

194 + UID: "vn-update-noop",

195 + ContentType: "application/json",

196 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

197 + Args: []string{ctl.configID("db"), string(dyncfg.CommandUpdate)},

198 + })

199 + ctl.SeqExec(fn)

200 +

201 + var payload map[string]any

202 + mustDecodeFunctionPayload(t, out.String(), "vn-update-noop", &payload)

203 + assert.Equal(t, float64(202), payload["status"])

204 + assert.Empty(t, seams.applyCalls)

205 + },

206 + },

207 + "update equal user vnode rewrites stored source metadata to dyncfg": {

208 + initial: map[string]*vnodes.VirtualNode{

209 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeUser),

210 + },

211 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

212 + fn := dyncfg.NewFunction(functions.Function{

213 + UID: "vn-update-promote-user",

214 + Source: "user=alice",

215 + ContentType: "application/json",

216 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

217 + Args: []string{ctl.configID("db"), string(dyncfg.CommandUpdate)},

218 + })

219 + ctl.SeqExec(fn)

220 +

221 + var payload map[string]any

222 + mustDecodeFunctionPayload(t, out.String(), "vn-update-promote-user", &payload)

223 + assert.Equal(t, float64(202), payload["status"])

224 + assert.Equal(t, []string{"db"}, seams.applyCalls)

225 +

226 + cfg, ok := ctl.Lookup("db")

227 + require.True(t, ok)

228 + assert.Equal(t, confgroup.TypeDyncfg, cfg.SourceType)

229 + assert.Equal(t, "user=alice", cfg.Source)

230 + },

231 + },

232 + "update rejects duplicate hostname": {

233 + initial: map[string]*vnodes.VirtualNode{

234 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

235 + "other": testVnode("other", "shared", "22222222-2222-2222-2222-222222222222", confgroup.TypeUser),

236 + },

237 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

238 + fn := dyncfg.NewFunction(functions.Function{

239 + UID: "vn-update-dup-host",

240 + ContentType: "application/json",

241 + Payload: mustJSON(t, map[string]any{

242 + "guid": "11111111-1111-1111-1111-111111111111",

243 + "hostname": "shared",

244 + }),

245 + Args: []string{ctl.configID("db"), string(dyncfg.CommandUpdate)},

246 + })

247 + ctl.SeqExec(fn)

248 +

249 + var payload map[string]any

250 + mustDecodeFunctionPayload(t, out.String(), "vn-update-dup-host", &payload)

251 + assert.Equal(t, float64(400), payload["status"])

252 + assert.Contains(t, fmt.Sprint(payload["errorMessage"]), "duplicate virtual node hostname")

253 + assert.Empty(t, seams.applyCalls)

254 + },

255 + },

256 + "update rejects duplicate guid": {

257 + initial: map[string]*vnodes.VirtualNode{

258 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

259 + "other": testVnode("other", "other", "22222222-2222-2222-2222-222222222222", confgroup.TypeUser),

260 + },

261 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

262 + fn := dyncfg.NewFunction(functions.Function{

263 + UID: "vn-update-dup-guid",

264 + ContentType: "application/json",

265 + Payload: mustJSON(t, map[string]any{

266 + "guid": "22222222-2222-2222-2222-222222222222",

267 + }),

268 + Args: []string{ctl.configID("db"), string(dyncfg.CommandUpdate)},

269 + })

270 + ctl.SeqExec(fn)

271 +

272 + var payload map[string]any

273 + mustDecodeFunctionPayload(t, out.String(), "vn-update-dup-guid", &payload)

274 + assert.Equal(t, float64(400), payload["status"])

275 + assert.Contains(t, fmt.Sprint(payload["errorMessage"]), "duplicate virtual node guid")

276 + assert.Empty(t, seams.applyCalls)

277 + },

278 + },

279 + "invalid guid is rejected": {

280 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

281 + fn := dyncfg.NewFunction(functions.Function{

282 + UID: "vn-invalid-guid",

283 + ContentType: "application/json",

284 + Payload: mustJSON(t, map[string]any{"guid": "bad-guid"}),

285 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), "db"},

286 + })

287 + ctl.SeqExec(fn)

288 +

289 + var payload map[string]any

290 + mustDecodeFunctionPayload(t, out.String(), "vn-invalid-guid", &payload)

291 + assert.Equal(t, float64(400), payload["status"])

292 + assert.Empty(t, seams.applyCalls)

293 + },

294 + },

295 + "empty vnode name is rejected": {

296 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

297 + fn := dyncfg.NewFunction(functions.Function{

298 + UID: "vn-empty-name",

299 + ContentType: "application/json",

300 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

301 + Args: []string{ctl.Prefix(), string(dyncfg.CommandAdd), ""},

302 + })

303 + ctl.SeqExec(fn)

304 +

305 + var payload map[string]any

306 + mustDecodeFunctionPayload(t, out.String(), "vn-empty-name", &payload)

307 + assert.Equal(t, float64(400), payload["status"])

308 + assert.Contains(t, fmt.Sprint(payload["errorMessage"]), "Missing vnode name")

309 + assert.Empty(t, seams.applyCalls)

310 + },

311 + },

312 + "remove rejects non dyncfg source": {

313 + initial: map[string]*vnodes.VirtualNode{

314 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeUser),

315 + },

316 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

317 + fn := dyncfg.NewFunction(functions.Function{UID: "vn-remove-user", Args: []string{ctl.configID("db"), string(dyncfg.CommandRemove)}})

318 + ctl.SeqExec(fn)

319 +

320 + var payload map[string]any

321 + mustDecodeFunctionPayload(t, out.String(), "vn-remove-user", &payload)

322 + assert.Equal(t, float64(405), payload["status"])

323 + assert.Empty(t, seams.affectedJobsCalls)

324 + },

325 + },

326 + "remove uses affected jobs seam": {

327 + initial: map[string]*vnodes.VirtualNode{

328 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

329 + },

330 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

331 + seams.affectedJobs["db"] = []string{"mysql:prod"}

332 +

333 + fn := dyncfg.NewFunction(functions.Function{UID: "vn-remove-running", Args: []string{ctl.configID("db"), string(dyncfg.CommandRemove)}})

334 + ctl.SeqExec(fn)

335 +

336 + var payload map[string]any

337 + mustDecodeFunctionPayload(t, out.String(), "vn-remove-running", &payload)

338 + assert.Equal(t, float64(409), payload["status"])

339 + assert.Contains(t, fmt.Sprint(payload["errorMessage"]), "referenced by configs")

340 + assert.Equal(t, []string{"db"}, seams.affectedJobsCalls)

341 + _, ok := ctl.Lookup("db")

342 + assert.True(t, ok)

343 + },

344 + },

345 + "test preview uses affected jobs seam": {

346 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer, seams *controllerSeams) {

347 + seams.affectedJobs["db"] = []string{"mysql:prod"}

348 +

349 + fn := dyncfg.NewFunction(functions.Function{

350 + UID: "vn-test",

351 + ContentType: "application/json",

352 + Payload: mustJSON(t, map[string]any{"guid": "11111111-1111-1111-1111-111111111111"}),

353 + Args: []string{ctl.Prefix(), string(dyncfg.CommandTest), "db"},

354 + })

355 + ctl.SeqExec(fn)

356 +

357 + var payload map[string]any

358 + mustDecodeFunctionPayload(t, out.String(), "vn-test", &payload)

359 + assert.Equal(t, float64(202), payload["status"])

360 + assert.Equal(t, "Updated configuration will affect configs: mysql:prod.", payload["message"])

361 + assert.Equal(t, []string{"db"}, seams.affectedJobsCalls)

362 + assert.Empty(t, seams.applyCalls)

363 + },

364 + },

365 + }

366 +

367 + for name, tc := range tests {

368 + t.Run(name, func(t *testing.T) {

369 + ctl, out, seams := newControllerTestSubject(tc.initial)

370 + tc.run(t, ctl, out, seams)

371 + })

372 + }

373 +}

374 +

375 +func TestControllerPublicationAndLookup(t *testing.T) {

376 + tests := map[string]struct {

377 + run func(t *testing.T, ctl *Controller, out *bytes.Buffer)

378 + }{

379 + "create module and publish existing": {

380 + run: func(t *testing.T, ctl *Controller, out *bytes.Buffer) {

381 + ctl.CreateTemplates()

382 + ctl.PublishExisting(dyncfg.StatusRunning)

383 +

384 + assert.Contains(t, out.String(), "CONFIG test:vnode create accepted template /collectors/test/Vnodes")

385 + assert.Contains(t, out.String(), "CONFIG test:vnode:db create running job /collectors/test/Vnodes")

386 +

387 + cfg, ok := ctl.Lookup("db")

388 + require.True(t, ok)

389 + assert.Equal(t, "db", cfg.Name)

390 + },

391 + },

392 + }

393 +

394 + for name, tc := range tests {

395 + t.Run(name, func(t *testing.T) {

396 + ctl, out, _ := newControllerTestSubject(map[string]*vnodes.VirtualNode{

397 + "db": testVnode("db", "db", "11111111-1111-1111-1111-111111111111", confgroup.TypeDyncfg),

398 + })

399 + tc.run(t, ctl, out)

400 + })

401 + }

402 +}

403 +

404 +func TestControllerSetAPI_NilPreservesResponder(t *testing.T) {

405 + tests := map[string]struct {

406 + uid string

407 + }{

408 + "nil SetAPI keeps existing responder for schema responses": {

409 + uid: "vn-schema-nil-rebind",

410 + },

411 + }

412 +

413 + for name, tc := range tests {

414 + t.Run(name, func(t *testing.T) {

415 + ctl, out, _ := newControllerTestSubject(nil)

416 + ctl.SetAPI(nil)

417 +

418 + fn := dyncfg.NewFunction(functions.Function{

419 + UID: tc.uid,

420 + Args: []string{ctl.Prefix(), string(dyncfg.CommandSchema)},

421 + })

422 + ctl.SeqExec(fn)

423 +

424 + assert.Contains(t, out.String(), "FUNCTION_RESULT_BEGIN "+tc.uid+" 200 application/json")

425 + })

426 + }

427 +}

428 +

429 +type controllerSeams struct {

430 + affectedJobs map[string][]string

431 + affectedJobsCalls []string

432 + applyCalls []string

433 +}

434 +

435 +func newControllerTestSubject(initial map[string]*vnodes.VirtualNode) (*Controller, *bytes.Buffer, *controllerSeams) {

436 + var out bytes.Buffer

437 + seams := &controllerSeams{affectedJobs: make(map[string][]string)}

438 + ctl := New(Options{

439 + Logger: logger.New(),

440 + API: dyncfg.NewResponder(netdataapi.New(safewriter.New(&out))),

441 + Plugin: testPluginName,

442 + Initial: initial,

443 + AffectedJobs: func(vnode string) []string {

444 + seams.affectedJobsCalls = append(seams.affectedJobsCalls, vnode)

445 + return seams.affectedJobs[vnode]

446 + },

447 + ApplyVnodeUpdate: func(name string, _ *vnodes.VirtualNode) {

448 + seams.applyCalls = append(seams.applyCalls, name)

449 + },

450 + })

451 + return ctl, &out, seams

452 +}

453 +

454 +func testVnode(name, hostname, guid, sourceType string) *vnodes.VirtualNode {

455 + return &vnodes.VirtualNode{Name: name, Hostname: hostname, GUID: guid, Source: sourceType, SourceType: sourceType}

456 +}

457 +

458 +func mustJSON(t *testing.T, v any) []byte {

459 + t.Helper()

460 + bs, err := json.Marshal(v)

461 + require.NoError(t, err)

462 + return bs

463 +}

464 +

465 +func mustFunctionBody(t *testing.T, output, uid string) string {

466 + t.Helper()

467 + re := regexp.MustCompile("(?s)FUNCTION_RESULT_BEGIN " + regexp.QuoteMeta(uid) + " [^\\n]+\\n(.*?)\\nFUNCTION_RESULT_END")

468 + match := re.FindStringSubmatch(output)

469 + require.Len(t, match, 2, "function result for uid '%s' not found in output:\n%s", uid, output)

470 + return match[1]

471 +}

472 +

473 +func mustDecodeFunctionPayload(t *testing.T, output, uid string, dst any) {

474 + t.Helper()

475 + require.NoError(t, json.Unmarshal([]byte(mustFunctionBody(t, output, uid)), dst))

476 +}

src/go/plugin/agent/jobmgr/vnodectl/doc.go new

@@ -0,0 +1,5 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +// Package vnodectl owns vnode dyncfg control-plane logic and vnode store ownership.

4 +// Routing, handoff, and runtime coordination remain in jobmgr.

5 +package vnodectl

src/go/plugin/agent/jobmgr/vnodectl/dyncfg.go new

+323

@@ -0,0 +1,323 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vnodectl

4 +

5 +import (

6 + "encoding/json"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/google/uuid"

11 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

14 + "gopkg.in/yaml.v2"

15 +)

16 +

17 +func dyncfgVnodeModCmds() string {

18 + return dyncfg.JoinCommands(

19 + dyncfg.CommandAdd,

20 + dyncfg.CommandSchema,

21 + dyncfg.CommandUserconfig,

22 + dyncfg.CommandTest,

23 + )

24 +}

25 +

26 +func dyncfgVnodeJobCmds(isDyncfgJob bool) string {

27 + cmds := []dyncfg.Command{

28 + dyncfg.CommandUserconfig,

29 + dyncfg.CommandSchema,

30 + dyncfg.CommandGet,

31 + dyncfg.CommandUpdate,

32 + dyncfg.CommandTest,

33 + }

34 + if isDyncfgJob {

35 + cmds = append(cmds, dyncfg.CommandRemove)

36 + }

37 + return dyncfg.JoinCommands(cmds...)

38 +}

39 +

40 +func (c *Controller) SeqExec(fn dyncfg.Function) {

41 + switch fn.Command() {

42 + case dyncfg.CommandSchema:

43 + c.dyncfgCmdSchema(fn)

44 + case dyncfg.CommandUserconfig:

45 + c.dyncfgCmdUserconfig(fn)

46 + case dyncfg.CommandGet:

47 + c.dyncfgCmdGet(fn)

48 + case dyncfg.CommandAdd:

49 + c.dyncfgCmdAdd(fn)

50 + case dyncfg.CommandUpdate:

51 + c.dyncfgCmdUpdate(fn)

52 + case dyncfg.CommandRemove:

53 + c.dyncfgCmdRemove(fn)

54 + case dyncfg.CommandTest:

55 + c.dyncfgCmdTest(fn)

56 + default:

57 + c.Warningf("dyncfg: function '%s' command '%s' not implemented", fn.Fn().Name, fn.Command())

58 + c.api.SendCodef(fn, 501, "Function '%s' command '%s' is not implemented.", fn.Fn().Name, fn.Command())

59 + }

60 +}

61 +

62 +func (c *Controller) dyncfgCmdSchema(fn dyncfg.Function) {

63 + c.api.SendJSON(fn, vnodes.ConfigSchema)

64 +}

65 +

66 +func (c *Controller) dyncfgCmdUserconfig(fn dyncfg.Function) {

67 + bs, err := dyncfgVnodeUserconfigFromPayload(fn)

68 + if err != nil {

69 + c.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", dyncfg.CommandUserconfig, err)

70 + c.api.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

71 + return

72 + }

73 +

74 + c.api.SendYAML(fn, string(bs))

75 +}

76 +

77 +func (c *Controller) dyncfgCmdGet(fn dyncfg.Function) {

78 + name := strings.TrimPrefix(fn.ID(), c.Prefix()+":")

79 + cfg, ok := c.Lookup(name)

80 + if !ok {

81 + c.Warningf("dyncfg: %s: vnode %s not found", dyncfg.CommandGet, name)

82 + c.api.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

83 + return

84 + }

85 +

86 + bs, err := json.Marshal(cfg)

87 + if err != nil {

88 + c.Warningf("dyncfg: %s: vnode job %s failed to json marshal config: %v", dyncfg.CommandGet, name, err)

89 + c.api.SendCodef(fn, 500, "Failed to convert configuration into JSON: %v.", err)

90 + return

91 + }

92 +

93 + c.api.SendJSON(fn, string(bs))

94 +}

95 +

96 +func (c *Controller) dyncfgCmdAdd(fn dyncfg.Function) {

97 + if err := fn.ValidateArgs(3); err != nil {

98 + c.Warningf("dyncfg: %s: %v", dyncfg.CommandAdd, err)

99 + c.api.SendCodef(fn, 400, "%v", err)

100 + return

101 + }

102 + if !fn.HasPayload() {

103 + c.Warningf("dyncfg: %s: vnode job %s missing configuration payload.", dyncfg.CommandAdd, fn.JobName())

104 + c.api.SendCodef(fn, 400, "Missing configuration payload.")

105 + return

106 + }

107 +

108 + name := fn.JobName()

109 + if name == "" {

110 + c.Warningf("dyncfg: %s: missing vnode name", dyncfg.CommandAdd)

111 + c.api.SendCodef(fn, 400, "Missing vnode name.")

112 + return

113 + }

114 + if err := dyncfg.ValidateJobName(name); err != nil {

115 + c.Warningf("dyncfg: %s: unacceptable vnode name '%s': %v", dyncfg.CommandAdd, name, err)

116 + c.api.SendCodef(fn, 400, "Unacceptable vnode name '%s': %v.", name, err)

117 + return

118 + }

119 + cfg, err := dyncfgVnodeConfigFromPayload(fn)

120 + if err != nil {

121 + c.Warningf("dyncfg: %s: vnode job %s: failed to create config from payload: %v", dyncfg.CommandAdd, name, err)

122 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid configuration format: %v.", err)

123 + return

124 + }

125 + if err := uuid.Validate(cfg.GUID); err != nil {

126 + c.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", dyncfg.CommandAdd, name, err)

127 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

128 + return

129 + }

130 +

131 + dyncfgUpdateVnodeConfig(cfg, name, fn)

132 + if err := c.verifyVnodeUnique(cfg); err != nil {

133 + c.Warningf("dyncfg: %s: vnode job %s: %v", dyncfg.CommandAdd, name, err)

134 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload: %v.", err)

135 + return

136 + }

137 +

138 + if orig, ok := c.Lookup(name); ok && sameStoredVnode(orig, cfg) {

139 + c.api.SendCodef(fn, 202, "")

140 + c.createJob(cfg, dyncfg.StatusRunning)

141 + return

142 + }

143 +

144 + if _, err := c.store.Upsert(cfg); err != nil {

145 + c.Warningf("dyncfg: %s: vnode job %s: %v", dyncfg.CommandAdd, name, err)

146 + c.api.SendCodef(fn, 400, "Failed to update vnode configuration: %v.", err)

147 + return

148 + }

149 +

150 + c.applyUpdate(name, cfg)

151 + c.api.SendCodef(fn, 202, "")

152 + c.createJob(cfg, dyncfg.StatusRunning)

153 +}

154 +

155 +func (c *Controller) dyncfgCmdUpdate(fn dyncfg.Function) {

156 + name := strings.TrimPrefix(fn.ID(), c.Prefix()+":")

157 + orig, ok := c.Lookup(name)

158 + if !ok {

159 + c.Warningf("dyncfg: %s: vnode %s not found", dyncfg.CommandUpdate, name)

160 + c.api.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

161 + return

162 + }

163 +

164 + cfg, err := dyncfgVnodeConfigFromPayload(fn)

165 + if err != nil {

166 + c.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", dyncfg.CommandUpdate, err)

167 + c.api.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

168 + return

169 + }

170 + if err := uuid.Validate(cfg.GUID); err != nil {

171 + c.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", dyncfg.CommandUpdate, name, err)

172 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

173 + return

174 + }

175 +

176 + dyncfgUpdateVnodeConfig(cfg, name, fn)

177 + if err := c.verifyVnodeUnique(cfg); err != nil {

178 + c.Warningf("dyncfg: %s: vnode job %s: %v", dyncfg.CommandUpdate, name, err)

179 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload: %v.", err)

180 + return

181 + }

182 + if sameStoredVnode(orig, cfg) {

183 + c.api.SendCodef(fn, 202, "")

184 + return

185 + }

186 +

187 + if _, err := c.store.Upsert(cfg); err != nil {

188 + c.Warningf("dyncfg: %s: vnode job %s: %v", dyncfg.CommandUpdate, name, err)

189 + c.api.SendCodef(fn, 400, "Failed to update vnode configuration: %v.", err)

190 + return

191 + }

192 +

193 + c.applyUpdate(name, cfg)

194 + c.api.SendCodef(fn, 202, "")

195 + c.createJob(cfg, dyncfg.StatusRunning)

196 +}

197 +

198 +func (c *Controller) dyncfgCmdRemove(fn dyncfg.Function) {

199 + name := strings.TrimPrefix(fn.ID(), c.Prefix()+":")

200 + vnode, ok := c.Lookup(name)

201 + if !ok {

202 + c.Warningf("dyncfg: %s: vnode %s not found", dyncfg.CommandRemove, name)

203 + c.api.SendCodef(fn, 404, "The specified vnode '%s' is not registered.", name)

204 + return

205 + }

206 + if vnode.SourceType != confgroup.TypeDyncfg {

207 + c.Warningf("dyncfg: %s: module vnode %s: can not remove vnode of type %s", dyncfg.CommandRemove, vnode.Name, vnode.SourceType)

208 + c.api.SendCodef(fn, 405, "Removing vnode of type '%s' is not supported. Only 'dyncfg' vnodes can be removed.", vnode.SourceType)

209 + return

210 + }

211 +

212 + if affected := c.affectedJobsFor(vnode.Name); affected != "" {

213 + c.Warningf("dyncfg: %s: vnode %s is referenced by configs (%s)", dyncfg.CommandRemove, name, affected)

214 + c.api.SendCodef(fn, 409, "The specified vnode '%s' is referenced by configs (%s).", name, affected)

215 + return

216 + }

217 +

218 + c.store.Remove(name)

219 + c.api.ConfigDelete(fn.ID())

220 + c.api.SendCodef(fn, 200, "")

221 +}

222 +

223 +func (c *Controller) dyncfgCmdTest(fn dyncfg.Function) {

224 + if err := fn.ValidateArgs(3); err != nil {

225 + c.Warningf("dyncfg: %s: %v", dyncfg.CommandTest, err)

226 + c.api.SendCodef(fn, 400, "%v", err)

227 + return

228 + }

229 +

230 + name := fn.JobName()

231 + cfg, err := dyncfgVnodeConfigFromPayload(fn)

232 + if err != nil {

233 + c.Warningf("dyncfg: %s: vnode: failed to create config from payload: %v", dyncfg.CommandTest, err)

234 + c.api.SendCodef(fn, 400, "Invalid configuration format. Failed to create configuration from payload: %v.", err)

235 + return

236 + }

237 + if err := uuid.Validate(cfg.GUID); err != nil {

238 + c.Warningf("dyncfg: %s: vnode job %s: invalid guid: %v", dyncfg.CommandTest, name, err)

239 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload. Invalid guid format: %v.", err)

240 + return

241 + }

242 +

243 + dyncfgUpdateVnodeConfig(cfg, name, fn)

244 + if err := c.verifyVnodeUnique(cfg); err != nil {

245 + c.Warningf("dyncfg: %s: vnode job %s: %v", dyncfg.CommandTest, name, err)

246 + c.api.SendCodef(fn, 400, "Failed to create configuration from payload: %v.", err)

247 + return

248 + }

249 +

250 + if affected := c.affectedJobsFor(cfg.Name); affected != "" {

251 + c.api.SendCodef(fn, 202, "Updated configuration will affect configs: %s.", affected)

252 + return

253 + }

254 + c.api.SendCodef(fn, 202, "No configs will be affected by this change.")

255 +}

256 +

257 +func (c *Controller) affectedJobsFor(vnode string) string {

258 + if c.affectedJobs == nil {

259 + return ""

260 + }

261 + return strings.Join(c.affectedJobs(vnode), ", ")

262 +}

263 +

264 +func (c *Controller) applyUpdate(name string, cfg *vnodes.VirtualNode) {

265 + if c.applyVnodeUpdate != nil {

266 + c.applyVnodeUpdate(name, cfg)

267 + }

268 +}

269 +

270 +func (c *Controller) verifyVnodeUnique(newCfg *vnodes.VirtualNode) error {

271 + var err error

272 + c.store.ForEach(func(cfg *vnodes.VirtualNode) bool {

273 + if cfg.Name == newCfg.Name {

274 + return true

275 + }

276 + if cfg.Hostname == newCfg.Hostname {

277 + err = fmt.Errorf("duplicate virtual node hostname detected (job '%s')", cfg.Name)

278 + return false

279 + }

280 + if cfg.GUID == newCfg.GUID {

281 + err = fmt.Errorf("duplicate virtual node guid detected (job '%s')", cfg.Name)

282 + return false

283 + }

284 + return true

285 + })

286 + return err

287 +}

288 +

289 +func dyncfgUpdateVnodeConfig(cfg *vnodes.VirtualNode, name string, fn dyncfg.Function) {

290 + cfg.SourceType = confgroup.TypeDyncfg

291 + cfg.Source = fn.Source()

292 + cfg.Name = name

293 + if cfg.Hostname == "" {

294 + cfg.Hostname = name

295 + }

296 +}

297 +

298 +func dyncfgVnodeConfigFromPayload(fn dyncfg.Function) (*vnodes.VirtualNode, error) {

299 + var cfg vnodes.VirtualNode

300 + if err := fn.UnmarshalPayload(&cfg); err != nil {

301 + return nil, err

302 + }

303 + return &cfg, nil

304 +}

305 +

306 +func dyncfgVnodeUserconfigFromPayload(fn dyncfg.Function) ([]byte, error) {

307 + cfg, err := dyncfgVnodeConfigFromPayload(fn)

308 + if err != nil {

309 + return nil, err

310 + }

311 +

312 + name := fn.JobName()

313 + if name == "" {

314 + name = "test"

315 + }

316 + dyncfgUpdateVnodeConfig(cfg, name, fn)

317 +

318 + bs, err := yaml.Marshal([]any{cfg})

319 + if err != nil {

320 + return nil, err

321 + }

322 + return bs, nil

323 +}

src/go/plugin/agent/jobmgr/vnodectl/store.go renamed

+19 -11

@@ -1,6 +1,6 @@

1 // SPDX-License-Identifier: GPL-3.0-or-later

3 -package jobmgr

3 +package vnodectl

5 import (

6 "fmt"

@@ -8,8 +8,8 @@ import (

8 "github.com/netdata/netdata/go/plugins/plugin/framework/vnodes"

9 )

11 -// vnodeStore owns manager vnode state. It is intentionally lock-free because

12 -// mutations are serialized by Manager.run and startup initialization.

11 +// vnodeStore is intentionally lock-free because mutations are serialized by jobmgr.

12 +// Any caller outside that serialized path must add its own synchronization first.

13 type vnodeStore struct {

14 items map[string]*vnodes.VirtualNode

15 }

@@ -26,23 +26,23 @@ func (s *vnodeStore) Lookup(name string) (*vnodes.VirtualNode, bool) {

26 return cfg, ok

27 }

29 -func (s *vnodeStore) Upsert(cfg *vnodes.VirtualNode) (changed bool, affectedJobNames []string, err error) {

29 +func (s *vnodeStore) Upsert(cfg *vnodes.VirtualNode) (bool, error) {

30 if cfg == nil {

31 - return false, nil, fmt.Errorf("nil vnode config")

31 + return false, fmt.Errorf("nil vnode config")

32 }

33 - if orig, ok := s.items[cfg.Name]; ok && orig.Equal(cfg) {

34 - return false, nil, nil

33 + if orig, ok := s.items[cfg.Name]; ok && sameStoredVnode(orig, cfg) {

34 + return false, nil

35 }

36 s.items[cfg.Name] = cfg

37 - return true, nil, nil

37 + return true, nil

38 }

40 -func (s *vnodeStore) Remove(name string) (removed bool, err error) {

40 +func (s *vnodeStore) Remove(name string) bool {

41 if _, ok := s.items[name]; !ok {

42 - return false, nil

42 + return false

43 }

44 delete(s.items, name)

45 - return true, nil

45 + return true

46 }

48 func (s *vnodeStore) ForEach(fn func(cfg *vnodes.VirtualNode) bool) {

@@ -52,3 +52,11 @@ func (s *vnodeStore) ForEach(fn func(cfg *vnodes.VirtualNode) bool) {

52 }

53 }

54 }

55 +

56 +func sameStoredVnode(orig, next *vnodes.VirtualNode) bool {

57 + if orig == nil || next == nil {

58 + return orig == next

59 + }

60 + return orig.Equal(next) &&

61 + orig.SourceType == next.SourceType

62 +}

src/go/plugin/agent/secrets/resolver/cmd.go new

+46

@@ -0,0 +1,46 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "errors"

8 + "fmt"

9 + "io"

10 + "os/exec"

11 + "path/filepath"

12 + "strings"

13 + "time"

14 +)

15 +

16 +func (r *Resolver) resolveCmd(ctx context.Context, cmdLine, original string) (string, error) {

17 + parts := strings.Fields(cmdLine)

18 + if len(parts) == 0 {

19 + return "", fmt.Errorf("resolving secret '%s': empty command", original)

20 + }

21 + if !filepath.IsAbs(parts[0]) {

22 + return "", fmt.Errorf("resolving secret '%s': command path must be absolute, got '%s'", original, parts[0])

23 + }

24 +

25 + if ctx == nil {

26 + ctx = context.Background()

27 + }

28 + timeout := r.cmdTimeout

29 + if timeout <= 0 {

30 + timeout = 10 * time.Second

31 + }

32 + ctx, cancel := context.WithTimeout(ctx, timeout)

33 + defer cancel()

34 +

35 + cmd := exec.CommandContext(ctx, parts[0], parts[1:]...)

36 + cmd.Stderr = io.Discard

37 + out, err := cmd.Output()

38 + if err != nil {

39 + if errors.Is(ctx.Err(), context.DeadlineExceeded) {

40 + return "", fmt.Errorf("resolving secret '%s': command timed out after %s", original, timeout)

41 + }

42 + return "", fmt.Errorf("resolving secret '%s': command failed: %w", original, err)

43 + }

44 +

45 + return strings.TrimSpace(string(out)), nil

46 +}

src/go/plugin/agent/secrets/resolver/cmd_test.go new

+98

@@ -0,0 +1,98 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "os"

7 + "path/filepath"

8 + "runtime"

9 + "testing"

10 +

11 + "github.com/stretchr/testify/assert"

12 + "github.com/stretchr/testify/require"

13 +)

14 +

15 +func TestResolveCmd(t *testing.T) {

16 + tests := map[string]struct {

17 + onWindowsSkip bool

18 + buildCfg func(t *testing.T) map[string]any

19 + wantErrContains string

20 + wantValue string

21 + field string

22 + }{

23 + "success": {

24 + onWindowsSkip: true,

25 + buildCfg: func(t *testing.T) map[string]any {

26 + return map[string]any{"password": "${cmd:/bin/echo hello}"}

27 + },

28 + field: "password",

29 + wantValue: "hello",

30 + },

31 + "trims output": {

32 + onWindowsSkip: true,

33 + buildCfg: func(t *testing.T) map[string]any {

34 + return map[string]any{"val": "${cmd:/bin/echo secretval}"}

35 + },

36 + field: "val",

37 + wantValue: "secretval",

38 + },

39 + "relative path rejected": {

40 + buildCfg: func(t *testing.T) map[string]any {

41 + return map[string]any{"val": "${cmd:echo hello}"}

42 + },

43 + wantErrContains: "command path must be absolute",

44 + },

45 + "empty command": {

46 + buildCfg: func(t *testing.T) map[string]any {

47 + return map[string]any{"val": "${cmd:}"}

48 + },

49 + wantErrContains: "empty command",

50 + },

51 + "nonexistent command": {

52 + buildCfg: func(t *testing.T) map[string]any {

53 + return map[string]any{"val": "${cmd:/nonexistent/command}"}

54 + },

55 + wantErrContains: "command failed",

56 + },

57 + "command with args": {

58 + onWindowsSkip: true,

59 + buildCfg: func(t *testing.T) map[string]any {

60 + return map[string]any{"val": "${cmd:/usr/bin/printf %s secret}"}

61 + },

62 + field: "val",

63 + wantValue: "secret",

64 + },

65 + "script file": {

66 + onWindowsSkip: true,

67 + buildCfg: func(t *testing.T) map[string]any {

68 + dir := t.TempDir()

69 + script := filepath.Join(dir, "secret.sh")

70 + require.NoError(t, os.WriteFile(script, []byte("#!/bin/sh\necho mysecret\n"), 0700))

71 + return map[string]any{"val": "${cmd:" + script + "}"}

72 + },

73 + field: "val",

74 + wantValue: "mysecret",

75 + },

76 + }

77 +

78 + for name, tc := range tests {

79 + t.Run(name, func(t *testing.T) {

80 + if tc.onWindowsSkip && runtime.GOOS == "windows" {

81 + t.Skip("skipping on windows")

82 + }

83 +

84 + resolver := New()

85 + cfg := tc.buildCfg(t)

86 + err := resolver.Resolve(cfg)

87 +

88 + if tc.wantErrContains != "" {

89 + require.Error(t, err)

90 + assert.Contains(t, err.Error(), tc.wantErrContains)

91 + return

92 + }

93 +

94 + require.NoError(t, err)

95 + assert.Equal(t, tc.wantValue, cfg[tc.field])

96 + })

97 + }

98 +}

src/go/plugin/agent/secrets/resolver/env.go new

+18

@@ -0,0 +1,18 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "os"

9 + "strings"

10 +)

11 +

12 +func (r *Resolver) resolveEnv(_ context.Context, name, original string) (string, error) {

13 + val, ok := os.LookupEnv(name)

14 + if !ok {

15 + return "", fmt.Errorf("resolving secret '%s': environment variable '%s' is not set", original, name)

16 + }

17 + return strings.TrimSpace(val), nil

18 +}

src/go/plugin/agent/secrets/resolver/file.go new

+22

@@ -0,0 +1,22 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "os"

9 + "path/filepath"

10 + "strings"

11 +)

12 +

13 +func (r *Resolver) resolveFile(_ context.Context, path, original string) (string, error) {

14 + if !filepath.IsAbs(path) {

15 + return "", fmt.Errorf("resolving secret '%s': file path must be absolute, got '%s'", original, path)

16 + }

17 + data, err := os.ReadFile(path)

18 + if err != nil {

19 + return "", fmt.Errorf("resolving secret '%s': %w", original, err)

20 + }

21 + return strings.TrimSpace(string(data)), nil

22 +}

src/go/plugin/agent/secrets/resolver/resolver.go new

+143

@@ -0,0 +1,143 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "regexp"

9 + "strings"

10 +)

11 +

12 +type StoreRefResolver func(ctx context.Context, ref, original string) (string, error)

13 +

14 +// reSecretRef matches ${...} references in strings.

15 +var reSecretRef = regexp.MustCompile(`\$\{([^}]+)\}`)

16 +

17 +// Resolve walks the config map and resolves all secret references in string values.

18 +func (r *Resolver) Resolve(cfg map[string]any) error {

19 + return r.ResolveWithStoreResolver(context.Background(), cfg, nil)

20 +}

21 +

22 +func (r *Resolver) ResolveWithStoreResolver(ctx context.Context, cfg map[string]any, storeResolver StoreRefResolver) error {

23 + if r == nil {

24 + return fmt.Errorf("secret resolver is nil")

25 + }

26 + if ctx == nil {

27 + ctx = context.Background()

28 + }

29 + r.ensureDefaults()

30 + return r.resolveMap(ctx, cfg, storeResolver)

31 +}

32 +

33 +func (r *Resolver) resolveMap(ctx context.Context, m map[string]any, storeResolver StoreRefResolver) error {

34 + for k, v := range m {

35 + // skip internal keys (__key__)

36 + if isInternalKey(k) {

37 + continue

38 + }

39 + resolved, err := r.resolveValue(ctx, v, storeResolver)

40 + if err != nil {

41 + return err

42 + }

43 + m[k] = resolved

44 + }

45 + return nil

46 +}

47 +

48 +func (r *Resolver) resolveMapAny(ctx context.Context, m map[any]any, storeResolver StoreRefResolver) error {

49 + for k, v := range m {

50 + s, ok := k.(string)

51 + if ok && isInternalKey(s) {

52 + continue

53 + }

54 + resolved, err := r.resolveValue(ctx, v, storeResolver)

55 + if err != nil {

56 + return err

57 + }

58 + m[k] = resolved

59 + }

60 + return nil

61 +}

62 +

63 +func (r *Resolver) resolveSlice(ctx context.Context, s []any, storeResolver StoreRefResolver) error {

64 + for i, v := range s {

65 + resolved, err := r.resolveValue(ctx, v, storeResolver)

66 + if err != nil {

67 + return err

68 + }

69 + s[i] = resolved

70 + }

71 + return nil

72 +}

73 +

74 +func (r *Resolver) resolveValue(ctx context.Context, v any, storeResolver StoreRefResolver) (any, error) {

75 + switch val := v.(type) {

76 + case string:

77 + return r.resolveString(ctx, val, storeResolver)

78 + case map[string]any:

79 + return val, r.resolveMap(ctx, val, storeResolver)

80 + case map[any]any:

81 + return val, r.resolveMapAny(ctx, val, storeResolver)

82 + case []any:

83 + return val, r.resolveSlice(ctx, val, storeResolver)

84 + default:

85 + return v, nil

86 + }

87 +}

88 +

89 +func (r *Resolver) resolveString(ctx context.Context, s string, storeResolver StoreRefResolver) (string, error) {

90 + if !strings.Contains(s, "${") {

91 + return s, nil

92 + }

93 +

94 + var resolveErr error

95 +

96 + result := reSecretRef.ReplaceAllStringFunc(s, func(match string) string {

97 + if resolveErr != nil {

98 + return match

99 + }

100 +

101 + // extract inner content between ${ and }

102 + inner := match[2 : len(match)-1]

103 +

104 + val, err := r.resolveRef(ctx, inner, match, storeResolver)

105 + if err != nil {

106 + resolveErr = err

107 + return match

108 + }

109 + return val

110 + })

111 +

112 + if resolveErr != nil {

113 + return "", resolveErr

114 + }

115 + return result, nil

116 +}

117 +

118 +func (r *Resolver) resolveRef(ctx context.Context, ref, original string, storeResolver StoreRefResolver) (string, error) {

119 + scheme, name, hasScheme := strings.Cut(ref, ":")

120 +

121 + if !hasScheme {

122 + // no scheme — leave unchanged

123 + return original, nil

124 + }

125 +

126 + if scheme == "store" {

127 + if storeResolver == nil {

128 + return "", fmt.Errorf("resolving secret '%s': secretstore resolver is not configured", original)

129 + }

130 + return storeResolver(ctx, name, original)

131 + }

132 +

133 + provider, ok := r.providers[scheme]

134 + if !ok {

135 + return "", fmt.Errorf("resolving secret '%s': unknown secret provider '%s'", original, scheme)

136 + }

137 +

138 + return provider(ctx, name, original)

139 +}

140 +

141 +func isInternalKey(k string) bool {

142 + return strings.HasPrefix(k, "__") && strings.HasSuffix(k, "__")

143 +}

src/go/plugin/agent/secrets/resolver/resolver_object.go new

+36

@@ -0,0 +1,36 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "time"

8 +)

9 +

10 +// Resolver resolves secret references in config maps.

11 +// In v2 it only owns local builtin resolvers and the optional secretstore callback.

12 +type Resolver struct {

13 + cmdTimeout time.Duration

14 +

15 + providers map[string]func(context.Context, string, string) (string, error)

16 +}

17 +

18 +// New creates a resolver with local builtin provider defaults.

19 +func New() *Resolver {

20 + r := &Resolver{}

21 + r.ensureDefaults()

22 + return r

23 +}

24 +

25 +func (r *Resolver) ensureDefaults() {

26 + if r.cmdTimeout <= 0 {

27 + r.cmdTimeout = 10 * time.Second

28 + }

29 + if r.providers == nil {

30 + r.providers = map[string]func(context.Context, string, string) (string, error){

31 + "env": r.resolveEnv,

32 + "file": r.resolveFile,

33 + "cmd": r.resolveCmd,

34 + }

35 + }

36 +}

src/go/plugin/agent/secrets/resolver/resolver_test.go new

+433

@@ -0,0 +1,433 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretresolver

4 +

5 +import (

6 + "context"

7 + "errors"

8 + "os"

9 + "path/filepath"

10 + "testing"

11 +

12 + "github.com/stretchr/testify/assert"

13 + "github.com/stretchr/testify/require"

14 +)

15 +

16 +func TestResolve(t *testing.T) {

17 + tests := map[string]struct {

18 + buildCfg func(t *testing.T) map[string]any

19 + wantErrContains string

20 + assertCfg func(t *testing.T, cfg map[string]any)

21 + }{

22 + "env ref": {

23 + buildCfg: func(t *testing.T) map[string]any {

24 + t.Setenv("TEST_SECRET_USER", "admin")

25 + return map[string]any{"username": "${env:TEST_SECRET_USER}"}

26 + },

27 + assertCfg: func(t *testing.T, cfg map[string]any) {

28 + assert.Equal(t, "admin", cfg["username"])

29 + },

30 + },

31 + "env ref trims surrounding whitespace": {

32 + buildCfg: func(t *testing.T) map[string]any {

33 + t.Setenv("TEST_SECRET_TRIMMED", " admin \n")

34 + return map[string]any{"username": "${env:TEST_SECRET_TRIMMED}"}

35 + },

36 + assertCfg: func(t *testing.T, cfg map[string]any) {

37 + assert.Equal(t, "admin", cfg["username"])

38 + },

39 + },

40 + "file ref": {

41 + buildCfg: func(t *testing.T) map[string]any {

42 + path := filepath.Join(t.TempDir(), "secret.txt")

43 + require.NoError(t, os.WriteFile(path, []byte(" s3cret\n"), 0600))

44 + return map[string]any{"password": "${file:" + path + "}"}

45 + },

46 + assertCfg: func(t *testing.T, cfg map[string]any) {

47 + assert.Equal(t, "s3cret", cfg["password"])

48 + },

49 + },

50 + "nested maps": {

51 + buildCfg: func(t *testing.T) map[string]any {

52 + t.Setenv("TEST_SECRET_HOST", "db.local")

53 + return map[string]any{

54 + "database": map[string]any{

55 + "host": "${env:TEST_SECRET_HOST}",

56 + "port": 5432,

57 + },

58 + }

59 + },

60 + assertCfg: func(t *testing.T, cfg map[string]any) {

61 + inner := cfg["database"].(map[string]any)

62 + assert.Equal(t, "db.local", inner["host"])

63 + assert.Equal(t, 5432, inner["port"])

64 + },

65 + },

66 + "map any any": {

67 + buildCfg: func(t *testing.T) map[string]any {

68 + t.Setenv("TEST_SECRET_LABEL", "prod")

69 + return map[string]any{

70 + "labels": map[any]any{"env": "${env:TEST_SECRET_LABEL}"},

71 + }

72 + },

73 + assertCfg: func(t *testing.T, cfg map[string]any) {

74 + labels := cfg["labels"].(map[any]any)

75 + assert.Equal(t, "prod", labels["env"])

76 + },

77 + },

78 + "array with strings": {

79 + buildCfg: func(t *testing.T) map[string]any {

80 + t.Setenv("TEST_SECRET_ITEM", "resolved")

81 + return map[string]any{

82 + "items": []any{"plain", "${env:TEST_SECRET_ITEM}", 42},

83 + }

84 + },

85 + assertCfg: func(t *testing.T, cfg map[string]any) {

86 + items := cfg["items"].([]any)

87 + assert.Equal(t, "plain", items[0])

88 + assert.Equal(t, "resolved", items[1])

89 + assert.Equal(t, 42, items[2])

90 + },

91 + },

92 + "multiple refs in one string": {

93 + buildCfg: func(t *testing.T) map[string]any {

94 + t.Setenv("TEST_SECRET_DB_USER", "root")

95 + t.Setenv("TEST_SECRET_DB_PASS", "p@ss")

96 + return map[string]any{

97 + "dsn": "${env:TEST_SECRET_DB_USER}:${env:TEST_SECRET_DB_PASS}@tcp(localhost)/db",

98 + }

99 + },

100 + assertCfg: func(t *testing.T, cfg map[string]any) {

101 + assert.Equal(t, "root:p@ss@tcp(localhost)/db", cfg["dsn"])

102 + },

103 + },

104 + "uppercase no-scheme left alone": {

105 + buildCfg: func(t *testing.T) map[string]any {

106 + return map[string]any{"token": "${MY_TOKEN}"}

107 + },

108 + assertCfg: func(t *testing.T, cfg map[string]any) {

109 + assert.Equal(t, "${MY_TOKEN}", cfg["token"])

110 + },

111 + },

112 + "lowercase no-scheme left alone": {

113 + buildCfg: func(t *testing.T) map[string]any {

114 + return map[string]any{"template": "${lower_case}"}

115 + },

116 + assertCfg: func(t *testing.T, cfg map[string]any) {

117 + assert.Equal(t, "${lower_case}", cfg["template"])

118 + },

119 + },

120 + "unknown scheme error": {

121 + buildCfg: func(t *testing.T) map[string]any {

122 + return map[string]any{"val": "${nosuchprovider:secret/data/pass}"}

123 + },

124 + wantErrContains: "unknown secret provider 'nosuchprovider'",

125 + },

126 + "legacy remote vault syntax is rejected": {

127 + buildCfg: func(t *testing.T) map[string]any {

128 + return map[string]any{"val": "${vault:secret/data/pass#key}"}

129 + },

130 + wantErrContains: "unknown secret provider 'vault'",

131 + },

132 + "legacy remote aws syntax is rejected": {

133 + buildCfg: func(t *testing.T) map[string]any {

134 + return map[string]any{"val": "${aws-sm:mysecret}"}

135 + },

136 + wantErrContains: "unknown secret provider 'aws-sm'",

137 + },

138 + "missing env var error": {

139 + buildCfg: func(t *testing.T) map[string]any {

140 + return map[string]any{"password": "${env:DEFINITELY_NOT_SET_12345}"}

141 + },

142 + wantErrContains: "environment variable 'DEFINITELY_NOT_SET_12345' is not set",

143 + },

144 + "missing file error": {

145 + buildCfg: func(t *testing.T) map[string]any {

146 + path := filepath.Join(t.TempDir(), "nonexistent_secret_file")

147 + return map[string]any{"secret": "${file:" + path + "}"}

148 + },

149 + wantErrContains: "resolving secret '${file:",

150 + },

151 + "relative file path error": {

152 + buildCfg: func(t *testing.T) map[string]any {

153 + return map[string]any{"secret": "${file:relative/secret.txt}"}

154 + },

155 + wantErrContains: "file path must be absolute",

156 + },

157 + "internal keys skipped": {

158 + buildCfg: func(t *testing.T) map[string]any {

159 + return map[string]any{

160 + "__source__": "${env:SHOULD_NOT_RESOLVE}",

161 + "__source_type__": "${env:SHOULD_NOT_RESOLVE}",

162 + "__provider__": "${env:SHOULD_NOT_RESOLVE}",

163 + }

164 + },

165 + assertCfg: func(t *testing.T, cfg map[string]any) {

166 + assert.Equal(t, "${env:SHOULD_NOT_RESOLVE}", cfg["__source__"])

167 + assert.Equal(t, "${env:SHOULD_NOT_RESOLVE}", cfg["__source_type__"])

168 + assert.Equal(t, "${env:SHOULD_NOT_RESOLVE}", cfg["__provider__"])

169 + },

170 + },

171 + "non-string values untouched": {

172 + buildCfg: func(t *testing.T) map[string]any {

173 + return map[string]any{

174 + "port": 8080,

175 + "enabled": true,

176 + "ratio": 3.14,

177 + "nothing": nil,

178 + }

179 + },

180 + assertCfg: func(t *testing.T, cfg map[string]any) {

181 + assert.Equal(t, 8080, cfg["port"])

182 + assert.Equal(t, true, cfg["enabled"])

183 + assert.Equal(t, 3.14, cfg["ratio"])

184 + assert.Nil(t, cfg["nothing"])

185 + },

186 + },

187 + "no refs no changes": {

188 + buildCfg: func(t *testing.T) map[string]any {

189 + return map[string]any{"host": "localhost", "port": 3306}

190 + },

191 + assertCfg: func(t *testing.T, cfg map[string]any) {

192 + assert.Equal(t, "localhost", cfg["host"])

193 + assert.Equal(t, 3306, cfg["port"])

194 + },

195 + },

196 + "empty map": {

197 + buildCfg: func(t *testing.T) map[string]any { return map[string]any{} },

198 + assertCfg: func(t *testing.T, cfg map[string]any) {

199 + assert.Empty(t, cfg)

200 + },

201 + },

202 + "missing uppercase no-scheme left alone": {

203 + buildCfg: func(t *testing.T) map[string]any {

204 + return map[string]any{"token": "${MISSING_SHORTHAND_VAR_12345}"}

205 + },

206 + assertCfg: func(t *testing.T, cfg map[string]any) {

207 + assert.Equal(t, "${MISSING_SHORTHAND_VAR_12345}", cfg["token"])

208 + },

209 + },

210 + "mixed refs and plain text": {

211 + buildCfg: func(t *testing.T) map[string]any {

212 + t.Setenv("TEST_SECRET_PROTO", "https")

213 + return map[string]any{"url": "${env:TEST_SECRET_PROTO}://example.com/api"}

214 + },

215 + assertCfg: func(t *testing.T, cfg map[string]any) {

216 + assert.Equal(t, "https://example.com/api", cfg["url"])

217 + },

218 + },

219 + "deeply nested": {

220 + buildCfg: func(t *testing.T) map[string]any {

221 + t.Setenv("TEST_SECRET_DEEP", "found")

222 + return map[string]any{

223 + "level1": map[string]any{

224 + "level2": map[string]any{"level3": "${env:TEST_SECRET_DEEP}"},

225 + },

226 + }

227 + },

228 + assertCfg: func(t *testing.T, cfg map[string]any) {

229 + l1 := cfg["level1"].(map[string]any)

230 + l2 := l1["level2"].(map[string]any)

231 + assert.Equal(t, "found", l2["level3"])

232 + },

233 + },

234 + "file ref and env ref together": {

235 + buildCfg: func(t *testing.T) map[string]any {

236 + t.Setenv("TEST_SECRET_FUSER", "admin")

237 + path := filepath.Join(t.TempDir(), "pass.txt")

238 + require.NoError(t, os.WriteFile(path, []byte("hunter2\n"), 0600))

239 + return map[string]any{

240 + "dsn": "${env:TEST_SECRET_FUSER}:${file:" + path + "}@host",

241 + }

242 + },

243 + assertCfg: func(t *testing.T, cfg map[string]any) {

244 + assert.Equal(t, "admin:hunter2@host", cfg["dsn"])

245 + },

246 + },

247 + "internal key in nested map": {

248 + buildCfg: func(t *testing.T) map[string]any {

249 + return map[string]any{

250 + "sub": map[string]any{

251 + "__meta__": "${env:SHOULD_NOT_RESOLVE}",

252 + "normal": "plain",

253 + },

254 + }

255 + },

256 + assertCfg: func(t *testing.T, cfg map[string]any) {

257 + sub := cfg["sub"].(map[string]any)

258 + assert.Equal(t, "${env:SHOULD_NOT_RESOLVE}", sub["__meta__"])

259 + assert.Equal(t, "plain", sub["normal"])

260 + },

261 + },

262 + "empty env name error": {

263 + buildCfg: func(t *testing.T) map[string]any {

264 + return map[string]any{"val": "${env:}"}

265 + },

266 + wantErrContains: "environment variable '' is not set",

267 + },

268 + "empty file name error": {

269 + buildCfg: func(t *testing.T) map[string]any {

270 + return map[string]any{"val": "${file:}"}

271 + },

272 + wantErrContains: "resolving secret",

273 + },

274 + "multiple refs one failure": {

275 + buildCfg: func(t *testing.T) map[string]any {

276 + t.Setenv("TEST_SECRET_OK", "good")

277 + return map[string]any{

278 + "dsn": "${env:TEST_SECRET_OK}:${env:MISSING_VAR_12345}@host",

279 + }

280 + },

281 + wantErrContains: "MISSING_VAR_12345",

282 + },

283 + "array in nested map": {

284 + buildCfg: func(t *testing.T) map[string]any {

285 + t.Setenv("TEST_SECRET_ARR", "val")

286 + return map[string]any{

287 + "outer": map[string]any{

288 + "list": []any{"${env:TEST_SECRET_ARR}", "static"},

289 + },

290 + }

291 + },

292 + assertCfg: func(t *testing.T, cfg map[string]any) {

293 + outer := cfg["outer"].(map[string]any)

294 + list := outer["list"].([]any)

295 + assert.Equal(t, "val", list[0])

296 + assert.Equal(t, "static", list[1])

297 + },

298 + },

299 + }

300 +

301 + for name, tc := range tests {

302 + t.Run(name, func(t *testing.T) {

303 + resolver := New()

304 + cfg := tc.buildCfg(t)

305 + err := resolver.Resolve(cfg)

306 +

307 + if tc.wantErrContains != "" {

308 + require.Error(t, err)

309 + assert.Contains(t, err.Error(), tc.wantErrContains)

310 + return

311 + }

312 +

313 + require.NoError(t, err)

314 + if tc.assertCfg != nil {

315 + tc.assertCfg(t, cfg)

316 + }

317 + })

318 + }

319 +}

320 +

321 +func TestResolveWithStoreResolver(t *testing.T) {

322 + tests := map[string]struct {

323 + cfg map[string]any

324 + storeResolver StoreRefResolver

325 + wantErrContains string

326 + assertCfg func(t *testing.T, cfg map[string]any)

327 + }{

328 + "store ref with resolver": {

329 + cfg: map[string]any{

330 + "password": "${store:vault:vault_prod:secret/data/mysql#password}",

331 + },

332 + storeResolver: func(ctx context.Context, ref, original string) (string, error) {

333 + require.NotNil(t, ctx)

334 + if ref == "vault:vault_prod:secret/data/mysql#password" && original == "${store:vault:vault_prod:secret/data/mysql#password}" {

335 + return "resolved-secret", nil

336 + }

337 + return "", errors.New("unexpected ref")

338 + },

339 + assertCfg: func(t *testing.T, cfg map[string]any) {

340 + assert.Equal(t, "resolved-secret", cfg["password"])

341 + },

342 + },

343 + "store ref without resolver": {

344 + cfg: map[string]any{

345 + "password": "${store:vault:vault_prod:secret/data/mysql#password}",

346 + },

347 + wantErrContains: "secretstore resolver is not configured",

348 + },

349 + "store resolver error bubbles": {

350 + cfg: map[string]any{

351 + "password": "${store:vault:vault_prod:secret/data/mysql#password}",

352 + },

353 + storeResolver: func(ctx context.Context, ref, original string) (string, error) {

354 + require.NotNil(t, ctx)

355 + return "", errors.New("store not configured")

356 + },

357 + wantErrContains: "store not configured",

358 + },

359 + "mixed env and store refs": {

360 + cfg: map[string]any{

361 + "dsn": "${env:TEST_SR_USER}:${store:aws-sm:aws_prod:app/db#password}@host",

362 + },

363 + storeResolver: func(ctx context.Context, ref, original string) (string, error) {

364 + require.NotNil(t, ctx)

365 + assert.Equal(t, "aws-sm:aws_prod:app/db#password", ref)

366 + return "p@ss", nil

367 + },

368 + assertCfg: func(t *testing.T, cfg map[string]any) {

369 + assert.Equal(t, "admin:p@ss@host", cfg["dsn"])

370 + },

371 + },

372 + "store resolver receives canceled context": {

373 + cfg: map[string]any{

374 + "password": "${store:vault:vault_prod:secret/data/mysql#password}",

375 + },

376 + storeResolver: func(ctx context.Context, ref, original string) (string, error) {

377 + <-ctx.Done()

378 + return "", ctx.Err()

379 + },

380 + wantErrContains: context.Canceled.Error(),

381 + },

382 + }

383 +

384 + for name, tc := range tests {

385 + t.Run(name, func(t *testing.T) {

386 + if name == "mixed env and store refs" {

387 + t.Setenv("TEST_SR_USER", "admin")

388 + }

389 +

390 + resolver := New()

391 + ctx := context.Background()

392 + if name == "store resolver receives canceled context" {

393 + var cancel context.CancelFunc

394 + ctx, cancel = context.WithCancel(context.Background())

395 + cancel()

396 + }

397 +

398 + err := resolver.ResolveWithStoreResolver(ctx, tc.cfg, tc.storeResolver)

399 +

400 + if tc.wantErrContains != "" {

401 + require.Error(t, err)

402 + assert.Contains(t, err.Error(), tc.wantErrContains)

403 + return

404 + }

405 +

406 + require.NoError(t, err)

407 + if tc.assertCfg != nil {

408 + tc.assertCfg(t, tc.cfg)

409 + }

410 + })

411 + }

412 +}

413 +

414 +func TestResolveRefUsesProviderRegistry(t *testing.T) {

415 + resolver := New()

416 +

417 + called := false

418 + resolver.providers["stub"] = func(ctx context.Context, ref, original string) (string, error) {

419 + called = true

420 + require.NotNil(t, ctx)

421 + assert.Equal(t, "name", ref)

422 + assert.Equal(t, "${stub:name}", original)

423 + return "resolved-by-stub", nil

424 + }

425 +

426 + cfg := map[string]any{

427 + "value": "${stub:name}",

428 + }

429 +

430 + require.NoError(t, resolver.Resolve(cfg))

431 + assert.True(t, called)

432 + assert.Equal(t, "resolved-by-stub", cfg["value"])

433 +}

src/go/plugin/agent/secrets/secretstore/backends/aws/config_schema.json new

+40

@@ -0,0 +1,40 @@

1 +{

2 + "jsonSchema": {

3 + "$schema": "http://json-schema.org/draft-07/schema#",

4 + "type": "object",

5 + "title": "AWS Secrets Manager secretstore configuration.",

6 + "properties": {

7 + "auth_mode": {

8 + "title": "Auth mode",

9 + "description": "How AWS credentials are obtained.",

10 + "type": "string",

11 + "enum": [

12 + "env",

13 + "ecs",

14 + "imds"

15 + ],

16 + "default": "env"

17 + },

18 + "region": {

19 + "title": "Region",

20 + "description": "AWS region used for Secrets Manager requests.",

21 + "type": "string"

22 + }

23 + },

24 + "required": [

25 + "auth_mode",

26 + "region"

27 + ]

28 + },

29 + "uiSchema": {

30 + "uiOptions": {

31 + "fullPage": true

32 + },

33 + "auth_mode": {

34 + "ui:widget": "radio",

35 + "ui:options": {

36 + "inline": true

37 + }

38 + }

39 + }

40 +}

src/go/plugin/agent/secrets/secretstore/backends/aws/init.go new

+35

@@ -0,0 +1,35 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package aws

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 +)

12 +

13 +func (s *store) init(_ context.Context) error {

14 + switch strings.TrimSpace(s.Config.AuthMode) {

15 + case "env", "ecs", "imds":

16 + s.Config.AuthMode = strings.TrimSpace(s.Config.AuthMode)

17 + default:

18 + return fmt.Errorf("auth_mode '%s' is invalid for kind '%s'", s.Config.AuthMode, secretstore.KindAWSSM)

19 + }

20 +

21 + region := strings.TrimSpace(s.Config.Region)

22 + if region == "" {

23 + return fmt.Errorf("region is required")

24 + }

25 + s.Config.Region = region

26 +

27 + published := &publishedStore{

28 + provider: s.provider,

29 + mode: s.Config.AuthMode,

30 + regionValue: region,

31 + }

32 +

33 + s.published = published

34 + return nil

35 +}

src/go/plugin/agent/secrets/secretstore/backends/aws/provider.go new

+71

@@ -0,0 +1,71 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package aws

4 +

5 +import (

6 + "context"

7 + _ "embed"

8 + "net/http"

9 + "time"

10 +

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

13 +)

14 +

15 +//go:embed config_schema.json

16 +var configSchema string

17 +

18 +type Config struct {

19 + AuthMode string `json:"auth_mode" yaml:"auth_mode"`

20 + Region string `json:"region" yaml:"region"`

21 +}

22 +

23 +type credentials struct {

24 + accessKeyID string

25 + secretAccessKey string

26 + sessionToken string

27 +}

28 +

29 +type provider struct {

30 + apiClient *http.Client

31 + imdsClient *http.Client

32 + endpoint string

33 + now func() time.Time

34 +}

35 +

36 +type store struct {

37 + Config `yaml:",inline" json:""`

38 + provider *provider

39 + published *publishedStore

40 +}

41 +

42 +type publishedStore struct {

43 + provider *provider

44 + mode string

45 + regionValue string

46 +}

47 +

48 +func New() secretstore.Creator {

49 + p := &provider{

50 + apiClient: httpx.APIClient(10 * time.Second),

51 + imdsClient: httpx.NoProxyClient(2 * time.Second),

52 + now: time.Now,

53 + }

54 +

55 + return secretstore.Creator{

56 + Kind: secretstore.KindAWSSM,

57 + DisplayName: "AWS Secrets Manager",

58 + Schema: configSchema,

59 + Create: p.create,

60 + }

61 +}

62 +

63 +func (p *provider) create() secretstore.Store {

64 + return &store{provider: p}

65 +}

66 +

67 +func (s *store) Configuration() any { return &s.Config }

68 +

69 +func (s *store) Init(ctx context.Context) error { return s.init(ctx) }

70 +

71 +func (s *store) Publish() secretstore.PublishedStore { return s.published }

src/go/plugin/agent/secrets/secretstore/backends/aws/resolve.go new

+325

@@ -0,0 +1,325 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package aws

4 +

5 +import (

6 + "context"

7 + "crypto/hmac"

8 + "crypto/sha256"

9 + "encoding/hex"

10 + "encoding/json"

11 + "fmt"

12 + "io"

13 + "net/http"

14 + "sort"

15 + "strings"

16 +

17 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

18 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/envx"

19 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

20 +)

21 +

22 +func (s *publishedStore) Resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

23 + return s.resolve(ctx, req)

24 +}

25 +

26 +func (s *publishedStore) resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

27 + secretName, jsonKey, _ := strings.Cut(req.Operand, "#")

28 + if secretName == "" {

29 + return "", fmt.Errorf("resolving secret '%s': store '%s': secret name is empty", req.Original, req.StoreKey)

30 + }

31 +

32 + creds, err := s.credentials(ctx)

33 + if err != nil {

34 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

35 + }

36 +

37 + region, err := s.region()

38 + if err != nil {

39 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

40 + }

41 +

42 + secretString, err := s.secretValue(ctx, creds, region, secretName, req.Original)

43 + if err != nil {

44 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

45 + }

46 +

47 + if jsonKey == "" {

48 + return secretString, nil

49 + }

50 +

51 + var parsed map[string]any

52 + if err := json.Unmarshal([]byte(secretString), &parsed); err != nil {

53 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing SecretString as JSON: %w", req.Original, req.StoreKey, err)

54 + }

55 + val, ok := parsed[jsonKey]

56 + if !ok {

57 + return "", fmt.Errorf("resolving secret '%s': store '%s': key '%s' not found in SecretString JSON", req.Original, req.StoreKey, jsonKey)

58 + }

59 + if value, ok := val.(string); ok {

60 + return value, nil

61 + }

62 + b, err := json.Marshal(val)

63 + if err != nil {

64 + return "", fmt.Errorf("resolving secret '%s': store '%s': encoding value for key '%s': %w", req.Original, req.StoreKey, jsonKey, err)

65 + }

66 + return string(b), nil

67 +}

68 +

69 +func (s *publishedStore) region() (string, error) {

70 + if s.regionValue == "" {

71 + return "", fmt.Errorf("region is required")

72 + }

73 + return s.regionValue, nil

74 +}

75 +

76 +func (s *publishedStore) credentials(ctx context.Context) (*credentials, error) {

77 + switch s.mode {

78 + case "env":

79 + return envCredentials()

80 + case "ecs":

81 + uri, ok := envx.Lookup("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")

82 + if !ok || uri == "" {

83 + return nil, fmt.Errorf("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is not set")

84 + }

85 + return s.ecsCredentials(ctx, uri)

86 + case "imds":

87 + return s.imdsCredentials(ctx)

88 + default:

89 + return nil, fmt.Errorf("auth_mode '%s' is invalid for aws-sm", s.mode)

90 + }

91 +}

92 +

93 +func envCredentials() (*credentials, error) {

94 + ak, ok := envx.Lookup("AWS_ACCESS_KEY_ID")

95 + if !ok || ak == "" {

96 + return nil, fmt.Errorf("AWS_ACCESS_KEY_ID is not set")

97 + }

98 + sk, ok := envx.Lookup("AWS_SECRET_ACCESS_KEY")

99 + if !ok || sk == "" {

100 + return nil, fmt.Errorf("AWS_SECRET_ACCESS_KEY is not set")

101 + }

102 + token, _ := envx.Lookup("AWS_SESSION_TOKEN")

103 + return &credentials{accessKeyID: ak, secretAccessKey: sk, sessionToken: token}, nil

104 +}

105 +

106 +func (s *publishedStore) ecsCredentials(ctx context.Context, relativeURI string) (*credentials, error) {

107 + req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://169.254.170.2"+relativeURI, nil)

108 + if err != nil {

109 + return nil, fmt.Errorf("creating ECS credentials request: %w", err)

110 + }

111 + resp, err := s.provider.imdsClient.Do(req)

112 + if err != nil {

113 + return nil, fmt.Errorf("ECS credentials request failed: %w", err)

114 + }

115 + defer resp.Body.Close()

116 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

117 + if err != nil {

118 + return nil, fmt.Errorf("reading ECS credentials response: %w", err)

119 + }

120 + if resp.StatusCode != http.StatusOK {

121 + return nil, fmt.Errorf("ECS credentials returned HTTP %d: %s", resp.StatusCode, httpx.TruncateBody(body))

122 + }

123 + var result struct {

124 + AccessKeyID string `json:"AccessKeyId"`

125 + SecretAccessKey string `json:"SecretAccessKey"`

126 + Token string `json:"Token"`

127 + }

128 + if err := json.Unmarshal(body, &result); err != nil {

129 + return nil, fmt.Errorf("parsing ECS credentials response: %w", err)

130 + }

131 + if result.AccessKeyID == "" || result.SecretAccessKey == "" {

132 + return nil, fmt.Errorf("ECS credentials response missing required fields")

133 + }

134 + return &credentials{

135 + accessKeyID: result.AccessKeyID,

136 + secretAccessKey: result.SecretAccessKey,

137 + sessionToken: result.Token,

138 + }, nil

139 +}

140 +

141 +func (s *publishedStore) imdsCredentials(ctx context.Context) (*credentials, error) {

142 + tokenReq, err := http.NewRequestWithContext(ctx, http.MethodPut, "http://169.254.169.254/latest/api/token", nil)

143 + if err != nil {

144 + return nil, fmt.Errorf("creating IMDS token request: %w", err)

145 + }

146 + tokenReq.Header.Set("X-aws-ec2-metadata-token-ttl-seconds", "21600")

147 + tokenResp, err := s.provider.imdsClient.Do(tokenReq)

148 + if err != nil {

149 + return nil, fmt.Errorf("IMDS token request failed: %w", err)

150 + }

151 + defer tokenResp.Body.Close()

152 + tokenBody, err := io.ReadAll(io.LimitReader(tokenResp.Body, 1<<20))

153 + if err != nil {

154 + return nil, fmt.Errorf("reading IMDS token response: %w", err)

155 + }

156 + if tokenResp.StatusCode != http.StatusOK {

157 + return nil, fmt.Errorf("IMDS token request returned HTTP %d", tokenResp.StatusCode)

158 + }

159 + imdsToken := strings.TrimSpace(string(tokenBody))

160 +

161 + roleReq, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://169.254.169.254/latest/meta-data/iam/security-credentials/", nil)

162 + if err != nil {

163 + return nil, fmt.Errorf("creating IMDS role request: %w", err)

164 + }

165 + roleReq.Header.Set("X-aws-ec2-metadata-token", imdsToken)

166 + roleResp, err := s.provider.imdsClient.Do(roleReq)

167 + if err != nil {

168 + return nil, fmt.Errorf("IMDS role request failed: %w", err)

169 + }

170 + defer roleResp.Body.Close()

171 + roleBody, err := io.ReadAll(io.LimitReader(roleResp.Body, 1<<20))

172 + if err != nil {

173 + return nil, fmt.Errorf("reading IMDS role response: %w", err)

174 + }

175 + if roleResp.StatusCode != http.StatusOK {

176 + return nil, fmt.Errorf("IMDS role request returned HTTP %d", roleResp.StatusCode)

177 + }

178 + role := strings.TrimSpace(string(roleBody))

179 + if role == "" {

180 + return nil, fmt.Errorf("IMDS returned empty role name")

181 + }

182 +

183 + credReq, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://169.254.169.254/latest/meta-data/iam/security-credentials/"+role, nil)

184 + if err != nil {

185 + return nil, fmt.Errorf("creating IMDS credentials request: %w", err)

186 + }

187 + credReq.Header.Set("X-aws-ec2-metadata-token", imdsToken)

188 + credResp, err := s.provider.imdsClient.Do(credReq)

189 + if err != nil {

190 + return nil, fmt.Errorf("IMDS credentials request failed: %w", err)

191 + }

192 + defer credResp.Body.Close()

193 + credBody, err := io.ReadAll(io.LimitReader(credResp.Body, 1<<20))

194 + if err != nil {

195 + return nil, fmt.Errorf("reading IMDS credentials response: %w", err)

196 + }

197 + if credResp.StatusCode != http.StatusOK {

198 + return nil, fmt.Errorf("IMDS credentials request returned HTTP %d", credResp.StatusCode)

199 + }

200 + var result struct {

201 + AccessKeyID string `json:"AccessKeyId"`

202 + SecretAccessKey string `json:"SecretAccessKey"`

203 + Token string `json:"Token"`

204 + }

205 + if err := json.Unmarshal(credBody, &result); err != nil {

206 + return nil, fmt.Errorf("parsing IMDS credentials response: %w", err)

207 + }

208 + if result.AccessKeyID == "" || result.SecretAccessKey == "" {

209 + return nil, fmt.Errorf("IMDS credentials response missing required fields")

210 + }

211 + return &credentials{accessKeyID: result.AccessKeyID, secretAccessKey: result.SecretAccessKey, sessionToken: result.Token}, nil

212 +}

213 +

214 +func (s *publishedStore) secretValue(ctx context.Context, creds *credentials, region, secretName, original string) (string, error) {

215 + endpoint := s.provider.endpoint

216 + host := secretsManagerHost(region)

217 + if endpoint == "" {

218 + endpoint = "https://" + host + "/"

219 + }

220 + secretIDJSON, err := json.Marshal(secretName)

221 + if err != nil {

222 + return "", fmt.Errorf("resolving secret '%s': encoding secret name: %w", original, err)

223 + }

224 + payload := `{"SecretId":` + string(secretIDJSON) + `}`

225 + now := s.provider.now().UTC()

226 + timestamp := now.Format("20060102T150405Z")

227 + datestamp := now.Format("20060102")

228 + headers := map[string]string{

229 + "host": host,

230 + "x-amz-date": timestamp,

231 + "x-amz-target": "secretsmanager.GetSecretValue",

232 + "content-type": "application/x-amz-json-1.1",

233 + }

234 + if creds.sessionToken != "" {

235 + headers["x-amz-security-token"] = creds.sessionToken

236 + }

237 + authHeader := sigV4Sign("POST", "/", "", headers, payload, creds, region, datestamp, timestamp)

238 + httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, strings.NewReader(payload))

239 + if err != nil {

240 + return "", fmt.Errorf("resolving secret '%s': creating request: %w", original, err)

241 + }

242 + for k, v := range headers {

243 + httpReq.Header.Set(k, v)

244 + }

245 + httpReq.Host = host

246 + httpReq.Header.Set("Authorization", authHeader)

247 + resp, err := s.provider.apiClient.Do(httpReq)

248 + if err != nil {

249 + return "", fmt.Errorf("resolving secret '%s': request failed: %w", original, err)

250 + }

251 + defer resp.Body.Close()

252 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

253 + if err != nil {

254 + return "", fmt.Errorf("resolving secret '%s': reading response: %w", original, err)

255 + }

256 + if resp.StatusCode != http.StatusOK {

257 + return "", fmt.Errorf("resolving secret '%s': AWS Secrets Manager returned HTTP %d: %s", original, resp.StatusCode, httpx.TruncateBody(body))

258 + }

259 + var result struct {

260 + SecretString *string `json:"SecretString"`

261 + }

262 + if err := json.Unmarshal(body, &result); err != nil {

263 + return "", fmt.Errorf("resolving secret '%s': parsing response: %w", original, err)

264 + }

265 + if result.SecretString == nil {

266 + return "", fmt.Errorf("resolving secret '%s': SecretString is empty (binary secrets are not supported)", original)

267 + }

268 + return *result.SecretString, nil

269 +}

270 +

271 +func secretsManagerHost(region string) string {

272 + suffix := "amazonaws.com"

273 + if strings.HasPrefix(region, "cn-") {

274 + suffix = "amazonaws.com.cn"

275 + }

276 + return fmt.Sprintf("secretsmanager.%s.%s", region, suffix)

277 +}

278 +

279 +func sigV4Sign(method, uri, query string, headers map[string]string, payload string, creds *credentials, region, datestamp, timestamp string) string {

280 + canonicalHeaders, signedHeaders := canonicalHeaders(headers)

281 + payloadHash := sha256Hex([]byte(payload))

282 + canonicalRequest := strings.Join([]string{method, uri, query, canonicalHeaders, signedHeaders, payloadHash}, "\n")

283 + scope := datestamp + "/" + region + "/secretsmanager/aws4_request"

284 + stringToSign := strings.Join([]string{"AWS4-HMAC-SHA256", timestamp, scope, sha256Hex([]byte(canonicalRequest))}, "\n")

285 + signingKey := deriveSigningKey(creds.secretAccessKey, datestamp, region)

286 + signature := hex.EncodeToString(hmacSHA256(signingKey, []byte(stringToSign)))

287 + return fmt.Sprintf("AWS4-HMAC-SHA256 Credential=%s/%s, SignedHeaders=%s, Signature=%s", creds.accessKeyID, scope, signedHeaders, signature)

288 +}

289 +

290 +func canonicalHeaders(headers map[string]string) (string, string) {

291 + norm := make(map[string]string, len(headers))

292 + keys := make([]string, 0, len(headers))

293 + for k, v := range headers {

294 + lk := strings.ToLower(k)

295 + norm[lk] = v

296 + keys = append(keys, lk)

297 + }

298 + sort.Strings(keys)

299 + var canonical strings.Builder

300 + for _, k := range keys {

301 + canonical.WriteString(k)

302 + canonical.WriteByte(':')

303 + canonical.WriteString(strings.TrimSpace(norm[k]))

304 + canonical.WriteByte('\n')

305 + }

306 + return canonical.String(), strings.Join(keys, ";")

307 +}

308 +

309 +func deriveSigningKey(secretKey, datestamp, region string) []byte {

310 + kDate := hmacSHA256([]byte("AWS4"+secretKey), []byte(datestamp))

311 + kRegion := hmacSHA256(kDate, []byte(region))

312 + kService := hmacSHA256(kRegion, []byte("secretsmanager"))

313 + return hmacSHA256(kService, []byte("aws4_request"))

314 +}

315 +

316 +func hmacSHA256(key, data []byte) []byte {

317 + h := hmac.New(sha256.New, key)

318 + _, _ = h.Write(data)

319 + return h.Sum(nil)

320 +}

321 +

322 +func sha256Hex(data []byte) string {

323 + sum := sha256.Sum256(data)

324 + return hex.EncodeToString(sum[:])

325 +}

src/go/plugin/agent/secrets/secretstore/backends/aws/resolve_test.go new

+80

@@ -0,0 +1,80 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package aws

4 +

5 +import (

6 + "context"

7 + "net/http"

8 + "net/http/httptest"

9 + "testing"

10 + "time"

11 +

12 + "github.com/stretchr/testify/assert"

13 + "github.com/stretchr/testify/require"

14 +)

15 +

16 +func TestSecretsManagerHost(t *testing.T) {

17 + tests := map[string]struct {

18 + region string

19 + wantHost string

20 + }{

21 + "standard partition": {

22 + region: "us-east-1",

23 + wantHost: "secretsmanager.us-east-1.amazonaws.com",

24 + },

25 + "china partition": {

26 + region: "cn-north-1",

27 + wantHost: "secretsmanager.cn-north-1.amazonaws.com.cn",

28 + },

29 + }

30 +

31 + for name, tc := range tests {

32 + t.Run(name, func(t *testing.T) {

33 + assert.Equal(t, tc.wantHost, secretsManagerHost(tc.region))

34 + })

35 + }

36 +}

37 +

38 +func TestSecretValue_CustomEndpointUsesSignedHostHeader(t *testing.T) {

39 + tests := map[string]struct {

40 + region string

41 + wantHost string

42 + }{

43 + "standard partition": {

44 + region: "us-east-1",

45 + wantHost: "secretsmanager.us-east-1.amazonaws.com",

46 + },

47 + "china partition": {

48 + region: "cn-north-1",

49 + wantHost: "secretsmanager.cn-north-1.amazonaws.com.cn",

50 + },

51 + }

52 +

53 + for name, tc := range tests {

54 + t.Run(name, func(t *testing.T) {

55 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

56 + assert.Equal(t, tc.wantHost, r.Host)

57 + w.Header().Set("Content-Type", "application/x-amz-json-1.1")

58 + _, _ = w.Write([]byte(`{"SecretString":"value"}`))

59 + }))

60 + defer srv.Close()

61 +

62 + store := &publishedStore{

63 + provider: &provider{

64 + apiClient: srv.Client(),

65 + endpoint: srv.URL + "/",

66 + now: func() time.Time {

67 + return time.Date(2026, time.March, 18, 12, 0, 0, 0, time.UTC)

68 + },

69 + },

70 + }

71 +

72 + value, err := store.secretValue(context.Background(), &credentials{

73 + accessKeyID: "AKID",

74 + secretAccessKey: "SECRET",

75 + }, tc.region, "db/password", "${store:aws-sm:aws_prod:db/password}")

76 + require.NoError(t, err)

77 + assert.Equal(t, "value", value)

78 + })

79 + }

80 +}

src/go/plugin/agent/secrets/secretstore/backends/azure/config_schema.json new

+101

@@ -0,0 +1,101 @@

1 +{

2 + "jsonSchema": {

3 + "$schema": "http://json-schema.org/draft-07/schema#",

4 + "type": "object",

5 + "title": "Azure Key Vault secretstore configuration.",

6 + "properties": {

7 + "mode": {

8 + "title": "Mode",

9 + "description": "Azure authentication mode.",

10 + "type": "string",

11 + "enum": [

12 + "client",

13 + "managed_identity"

14 + ],

15 + "default": "client"

16 + }

17 + },

18 + "required": [

19 + "mode"

20 + ],

21 + "dependencies": {

22 + "mode": {

23 + "oneOf": [

24 + {

25 + "properties": {

26 + "mode": {

27 + "const": "client"

28 + },

29 + "mode_client": {

30 + "title": "Client",

31 + "description": "Client credentials used when mode is `client`.",

32 + "type": "object",

33 + "properties": {

34 + "tenant_id": {

35 + "title": "Tenant ID",

36 + "description": "Azure tenant ID.",

37 + "type": "string"

38 + },

39 + "client_id": {

40 + "title": "Client ID",

41 + "description": "Azure application / service principal client ID.",

42 + "type": "string"

43 + },

44 + "client_secret": {

45 + "title": "Client Secret",

46 + "description": "Azure application / service principal client secret.",

47 + "type": "string",

48 + "sensitive": true

49 + }

50 + },

51 + "required": [

52 + "tenant_id",

53 + "client_id",

54 + "client_secret"

55 + ]

56 + }

57 + },

58 + "required": [

59 + "mode_client"

60 + ]

61 + },

62 + {

63 + "properties": {

64 + "mode": {

65 + "const": "managed_identity"

66 + },

67 + "mode_managed_identity": {

68 + "title": "Managed Identity",

69 + "description": "Managed identity settings used when mode is `managed_identity`.",

70 + "type": "object",

71 + "properties": {

72 + "client_id": {

73 + "title": "Client ID",

74 + "description": "Optional client ID of a user-assigned managed identity.",

75 + "type": "string"

76 + }

77 + }

78 + }

79 + }

80 + }

81 + ]

82 + }

83 + }

84 + },

85 + "uiSchema": {

86 + "uiOptions": {

87 + "fullPage": true

88 + },

89 + "mode": {

90 + "ui:widget": "radio",

91 + "ui:options": {

92 + "inline": true

93 + }

94 + },

95 + "mode_client": {

96 + "client_secret": {

97 + "ui:widget": "password"

98 + }

99 + }

100 + }

101 +}

src/go/plugin/agent/secrets/secretstore/backends/azure/init.go new

+61

@@ -0,0 +1,61 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package azure

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 +)

12 +

13 +func (s *store) init(_ context.Context) error {

14 + published := &publishedStore{provider: s.provider}

15 +

16 + switch strings.TrimSpace(s.Config.Mode) {

17 + case "client":

18 + if s.Config.ModeClient == nil {

19 + return fmt.Errorf("mode_client is required when mode is 'client'")

20 + }

21 + tenantID := strings.TrimSpace(s.Config.ModeClient.TenantID)

22 + if tenantID == "" {

23 + return fmt.Errorf("mode_client.tenant_id is required")

24 + }

25 + clientID := strings.TrimSpace(s.Config.ModeClient.ClientID)

26 + if clientID == "" {

27 + return fmt.Errorf("mode_client.client_id is required")

28 + }

29 + clientSecret := strings.TrimSpace(s.Config.ModeClient.ClientSecret)

30 + if clientSecret == "" {

31 + return fmt.Errorf("mode_client.client_secret is required")

32 + }

33 + s.Config.Mode = "client"

34 + s.Config.ModeClient.TenantID = tenantID

35 + s.Config.ModeClient.ClientID = clientID

36 + s.Config.ModeClient.ClientSecret = clientSecret

37 + s.Config.ModeManagedIdentity = nil

38 + published.mode = s.Config.Mode

39 + published.clientTenantID = tenantID

40 + published.clientID = clientID

41 + published.clientSecret = clientSecret

42 + case "managed_identity":

43 + s.Config.Mode = "managed_identity"

44 + s.Config.ModeClient = nil

45 + published.mode = s.Config.Mode

46 + if s.Config.ModeManagedIdentity != nil {

47 + clientID := strings.TrimSpace(s.Config.ModeManagedIdentity.ClientID)

48 + if clientID != "" {

49 + s.Config.ModeManagedIdentity.ClientID = clientID

50 + published.managedIdentityClientID = clientID

51 + } else {

52 + s.Config.ModeManagedIdentity = nil

53 + }

54 + }

55 + default:

56 + return fmt.Errorf("mode '%s' is invalid for kind '%s'", s.Config.Mode, secretstore.KindAzureKV)

57 + }

58 +

59 + s.published = published

60 + return nil

61 +}

src/go/plugin/agent/secrets/secretstore/backends/azure/provider.go new

+81

@@ -0,0 +1,81 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package azure

4 +

5 +import (

6 + "context"

7 + _ "embed"

8 + "net/http"

9 + "regexp"

10 + "time"

11 +

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

13 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

14 +)

15 +

16 +var (

17 + //go:embed config_schema.json

18 + configSchema string

19 + reAzureSafeName = regexp.MustCompile(`^[a-zA-Z0-9-]+$`)

20 +)

21 +

22 +type Config struct {

23 + Mode string `json:"mode" yaml:"mode"`

24 + ModeClient *ModeClientConfig `json:"mode_client,omitempty" yaml:"mode_client,omitempty"`

25 + ModeManagedIdentity *ModeManagedIdentityConfig `json:"mode_managed_identity,omitempty" yaml:"mode_managed_identity,omitempty"`

26 +}

27 +

28 +type ModeClientConfig struct {

29 + TenantID string `json:"tenant_id" yaml:"tenant_id"`

30 + ClientID string `json:"client_id" yaml:"client_id"`

31 + ClientSecret string `json:"client_secret" yaml:"client_secret"`

32 +}

33 +

34 +type ModeManagedIdentityConfig struct {

35 + ClientID string `json:"client_id,omitempty" yaml:"client_id,omitempty"`

36 +}

37 +

38 +type provider struct {

39 + apiClient *http.Client

40 + imdsClient *http.Client

41 + loginEndpointURL string

42 +}

43 +

44 +type store struct {

45 + Config `yaml:",inline" json:""`

46 + provider *provider

47 + published *publishedStore

48 +}

49 +

50 +type publishedStore struct {

51 + provider *provider

52 + mode string

53 + clientTenantID string

54 + clientID string

55 + clientSecret string

56 + managedIdentityClientID string

57 +}

58 +

59 +func New() secretstore.Creator {

60 + p := &provider{

61 + apiClient: httpx.APIClient(10 * time.Second),

62 + imdsClient: httpx.NoProxyClient(2 * time.Second),

63 + }

64 +

65 + return secretstore.Creator{

66 + Kind: secretstore.KindAzureKV,

67 + DisplayName: "Azure Key Vault",

68 + Schema: configSchema,

69 + Create: p.create,

70 + }

71 +}

72 +

73 +func (p *provider) create() secretstore.Store {

74 + return &store{provider: p}

75 +}

76 +

77 +func (s *store) Configuration() any { return &s.Config }

78 +

79 +func (s *store) Init(ctx context.Context) error { return s.init(ctx) }

80 +

81 +func (s *store) Publish() secretstore.PublishedStore { return s.published }

src/go/plugin/agent/secrets/secretstore/backends/azure/resolve.go new

+172

@@ -0,0 +1,172 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package azure

4 +

5 +import (

6 + "context"

7 + "encoding/json"

8 + "fmt"

9 + "io"

10 + "net/http"

11 + "net/url"

12 + "strings"

13 +

14 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

15 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

16 +)

17 +

18 +func (s *publishedStore) Resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

19 + return s.resolve(ctx, req)

20 +}

21 +

22 +func (s *publishedStore) resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

23 + vaultName, secretName, ok := splitOperand(req.Operand)

24 + if !ok {

25 + return "", fmt.Errorf("resolving secret '%s': store '%s': operand must be in format 'vault-name/secret-name'", req.Original, req.StoreKey)

26 + }

27 + if !reAzureSafeName.MatchString(vaultName) {

28 + return "", fmt.Errorf("resolving secret '%s': store '%s': invalid vault name '%s'", req.Original, req.StoreKey, vaultName)

29 + }

30 + if !reAzureSafeName.MatchString(secretName) {

31 + return "", fmt.Errorf("resolving secret '%s': store '%s': invalid secret name '%s'", req.Original, req.StoreKey, secretName)

32 + }

33 +

34 + token, err := s.accessToken(ctx)

35 + if err != nil {

36 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

37 + }

38 +

39 + httpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("https://%s.vault.azure.net/secrets/%s?api-version=7.4", vaultName, secretName), nil)

40 + if err != nil {

41 + return "", fmt.Errorf("resolving secret '%s': store '%s': creating request: %w", req.Original, req.StoreKey, err)

42 + }

43 + httpReq.Header.Set("Authorization", "Bearer "+token)

44 +

45 + resp, err := s.provider.apiClient.Do(httpReq)

46 + if err != nil {

47 + return "", fmt.Errorf("resolving secret '%s': store '%s': request failed: %w", req.Original, req.StoreKey, err)

48 + }

49 + defer resp.Body.Close()

50 +

51 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

52 + if err != nil {

53 + return "", fmt.Errorf("resolving secret '%s': store '%s': reading response: %w", req.Original, req.StoreKey, err)

54 + }

55 + if resp.StatusCode != http.StatusOK {

56 + return "", fmt.Errorf("resolving secret '%s': store '%s': Azure Key Vault returned HTTP %d: %s", req.Original, req.StoreKey, resp.StatusCode, httpx.TruncateBody(body))

57 + }

58 +

59 + var result struct {

60 + Value string `json:"value"`

61 + }

62 + if err := json.Unmarshal(body, &result); err != nil {

63 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing response: %w", req.Original, req.StoreKey, err)

64 + }

65 + if result.Value == "" {

66 + return "", fmt.Errorf("resolving secret '%s': store '%s': Azure Key Vault returned empty secret value", req.Original, req.StoreKey)

67 + }

68 + return result.Value, nil

69 +}

70 +

71 +func splitOperand(operand string) (string, string, bool) {

72 + vaultName, secretName, ok := strings.Cut(operand, "/")

73 + return vaultName, secretName, ok && vaultName != "" && secretName != ""

74 +}

75 +

76 +func (s *publishedStore) accessToken(ctx context.Context) (string, error) {

77 + switch s.mode {

78 + case "client":

79 + if s.clientTenantID == "" {

80 + return "", fmt.Errorf("mode_client.tenant_id is required")

81 + }

82 + if s.clientID == "" {

83 + return "", fmt.Errorf("mode_client.client_id is required")

84 + }

85 + if s.clientSecret == "" {

86 + return "", fmt.Errorf("mode_client.client_secret is required")

87 + }

88 + return s.clientCredentialsToken(ctx, s.clientTenantID, s.clientID, s.clientSecret)

89 + case "managed_identity":

90 + var clientID string

91 + if s.managedIdentityClientID != "" {

92 + clientID = s.managedIdentityClientID

93 + }

94 + return s.managedIdentityToken(ctx, clientID)

95 + default:

96 + return "", fmt.Errorf("mode '%s' is invalid for azure-kv", s.mode)

97 + }

98 +}

99 +

100 +func (s *publishedStore) clientCredentialsToken(ctx context.Context, tenantID, clientID, clientSecret string) (string, error) {

101 + tokenURL := s.provider.loginEndpointURL

102 + if tokenURL == "" {

103 + tokenURL = fmt.Sprintf("https://login.microsoftonline.com/%s/oauth2/v2.0/token", tenantID)

104 + }

105 + form := url.Values{

106 + "client_id": {clientID},

107 + "client_secret": {clientSecret},

108 + "scope": {"https://vault.azure.net/.default"},

109 + "grant_type": {"client_credentials"},

110 + }

111 + req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenURL, strings.NewReader(form.Encode()))

112 + if err != nil {

113 + return "", fmt.Errorf("creating client credentials token request: %w", err)

114 + }

115 + req.Header.Set("Content-Type", "application/x-www-form-urlencoded")

116 + resp, err := s.provider.apiClient.Do(req)

117 + if err != nil {

118 + return "", fmt.Errorf("client credentials token request failed: %w", err)

119 + }

120 + defer resp.Body.Close()

121 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

122 + if err != nil {

123 + return "", fmt.Errorf("reading client credentials token response: %w", err)

124 + }

125 + if resp.StatusCode != http.StatusOK {

126 + return "", fmt.Errorf("client credentials token request returned HTTP %d: %s", resp.StatusCode, httpx.TruncateBody(body))

127 + }

128 + var result struct {

129 + AccessToken string `json:"access_token"`

130 + }

131 + if err := json.Unmarshal(body, &result); err != nil {

132 + return "", fmt.Errorf("parsing client credentials token response: %w", err)

133 + }

134 + if result.AccessToken == "" {

135 + return "", fmt.Errorf("client credentials token response missing access_token")

136 + }

137 + return result.AccessToken, nil

138 +}

139 +

140 +func (s *publishedStore) managedIdentityToken(ctx context.Context, clientID string) (string, error) {

141 + reqURL := "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://vault.azure.net"

142 + if clientID != "" {

143 + reqURL += "&client_id=" + url.QueryEscape(clientID)

144 + }

145 + req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)

146 + if err != nil {

147 + return "", fmt.Errorf("creating managed identity token request: %w", err)

148 + }

149 + req.Header.Set("Metadata", "true")

150 + resp, err := s.provider.imdsClient.Do(req)

151 + if err != nil {

152 + return "", fmt.Errorf("managed identity token request failed: %w", err)

153 + }

154 + defer resp.Body.Close()

155 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

156 + if err != nil {

157 + return "", fmt.Errorf("reading managed identity token response: %w", err)

158 + }

159 + if resp.StatusCode != http.StatusOK {

160 + return "", fmt.Errorf("managed identity token request returned HTTP %d: %s", resp.StatusCode, httpx.TruncateBody(body))

161 + }

162 + var result struct {

163 + AccessToken string `json:"access_token"`

164 + }

165 + if err := json.Unmarshal(body, &result); err != nil {

166 + return "", fmt.Errorf("parsing managed identity token response: %w", err)

167 + }

168 + if result.AccessToken == "" {

169 + return "", fmt.Errorf("managed identity token response missing access_token")

170 + }

171 + return result.AccessToken, nil

172 +}

src/go/plugin/agent/secrets/secretstore/backends/gcp/config_schema.json new

+72

@@ -0,0 +1,72 @@

1 +{

2 + "jsonSchema": {

3 + "$schema": "http://json-schema.org/draft-07/schema#",

4 + "type": "object",

5 + "title": "Google Secret Manager secretstore configuration.",

6 + "properties": {

7 + "mode": {

8 + "title": "Mode",

9 + "description": "GCP authentication mode.",

10 + "type": "string",

11 + "enum": [

12 + "metadata",

13 + "service_account_file"

14 + ],

15 + "default": "metadata"

16 + }

17 + },

18 + "required": [

19 + "mode"

20 + ],

21 + "dependencies": {

22 + "mode": {

23 + "oneOf": [

24 + {

25 + "properties": {

26 + "mode": {

27 + "const": "metadata"

28 + }

29 + }

30 + },

31 + {

32 + "properties": {

33 + "mode": {

34 + "const": "service_account_file"

35 + },

36 + "mode_service_account_file": {

37 + "title": "Service Account File",

38 + "description": "Service account file settings used when mode is `service_account_file`.",

39 + "type": "object",

40 + "properties": {

41 + "path": {

42 + "title": "Path",

43 + "description": "Path to the service account JSON file.",

44 + "type": "string",

45 + "minLength": 1

46 + }

47 + },

48 + "required": [

49 + "path"

50 + ]

51 + }

52 + },

53 + "required": [

54 + "mode_service_account_file"

55 + ]

56 + }

57 + ]

58 + }

59 + }

60 + },

61 + "uiSchema": {

62 + "uiOptions": {

63 + "fullPage": true

64 + },

65 + "mode": {

66 + "ui:widget": "radio",

67 + "ui:options": {

68 + "inline": true

69 + }

70 + }

71 + }

72 +}

src/go/plugin/agent/secrets/secretstore/backends/gcp/init.go new

+39

@@ -0,0 +1,39 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package gcp

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 +)

12 +

13 +func (s *store) init(_ context.Context) error {

14 + published := &publishedStore{provider: s.provider}

15 +

16 + switch strings.TrimSpace(s.Config.Mode) {

17 + case "metadata":

18 + s.Config.Mode = "metadata"

19 + s.Config.ModeServiceAccountFile = nil

20 + published.mode = s.Config.Mode

21 + case "service_account_file":

22 + if s.Config.ModeServiceAccountFile == nil {

23 + return fmt.Errorf("mode_service_account_file is required when mode is 'service_account_file'")

24 + }

25 + path := strings.TrimSpace(s.Config.ModeServiceAccountFile.Path)

26 + if path == "" {

27 + return fmt.Errorf("mode_service_account_file.path is required")

28 + }

29 + s.Config.Mode = "service_account_file"

30 + s.Config.ModeServiceAccountFile.Path = path

31 + published.mode = s.Config.Mode

32 + published.serviceAccountFilePath = path

33 + default:

34 + return fmt.Errorf("mode '%s' is invalid for kind '%s'", s.Config.Mode, secretstore.KindGCPSM)

35 + }

36 +

37 + s.published = published

38 + return nil

39 +}

src/go/plugin/agent/secrets/secretstore/backends/gcp/provider.go new

+74

@@ -0,0 +1,74 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package gcp

4 +

5 +import (

6 + "context"

7 + _ "embed"

8 + "net/http"

9 + "regexp"

10 + "time"

11 +

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

13 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

14 +)

15 +

16 +var (

17 + //go:embed config_schema.json

18 + configSchema string

19 + reGCPSafeProjectID = regexp.MustCompile(`^[a-zA-Z0-9._:-]+$`)

20 + reGCPSafeName = regexp.MustCompile(`^[a-zA-Z0-9_-]+$`)

21 +)

22 +

23 +type Config struct {

24 + Mode string `json:"mode" yaml:"mode"`

25 + ModeServiceAccountFile *ModeServiceAccountFileConfig `json:"mode_service_account_file,omitempty" yaml:"mode_service_account_file,omitempty"`

26 +}

27 +

28 +type ModeServiceAccountFileConfig struct {

29 + Path string `json:"path" yaml:"path"`

30 +}

31 +

32 +type provider struct {

33 + apiClient *http.Client

34 + metadataClient *http.Client

35 + secretEndpoint string

36 + now func() time.Time

37 +}

38 +

39 +type store struct {

40 + Config `yaml:",inline" json:""`

41 + provider *provider

42 + published *publishedStore

43 +}

44 +

45 +type publishedStore struct {

46 + provider *provider

47 + mode string

48 + serviceAccountFilePath string

49 +}

50 +

51 +func New() secretstore.Creator {

52 + p := &provider{

53 + apiClient: httpx.APIClient(10 * time.Second),

54 + metadataClient: httpx.NoProxyClient(2 * time.Second),

55 + now: time.Now,

56 + }

57 +

58 + return secretstore.Creator{

59 + Kind: secretstore.KindGCPSM,

60 + DisplayName: "Google Secret Manager",

61 + Schema: configSchema,

62 + Create: p.create,

63 + }

64 +}

65 +

66 +func (p *provider) create() secretstore.Store {

67 + return &store{provider: p}

68 +}

69 +

70 +func (s *store) Configuration() any { return &s.Config }

71 +

72 +func (s *store) Init(ctx context.Context) error { return s.init(ctx) }

73 +

74 +func (s *store) Publish() secretstore.PublishedStore { return s.published }

src/go/plugin/agent/secrets/secretstore/backends/gcp/resolve.go new

+232

@@ -0,0 +1,232 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package gcp

4 +

5 +import (

6 + "context"

7 + "crypto"

8 + cryptorand "crypto/rand"

9 + "crypto/rsa"

10 + "crypto/sha256"

11 + "crypto/x509"

12 + "encoding/base64"

13 + "encoding/json"

14 + "encoding/pem"

15 + "fmt"

16 + "io"

17 + "net/http"

18 + "net/url"

19 + "os"

20 + "strings"

21 +

22 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

23 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

24 +)

25 +

26 +func (s *publishedStore) Resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

27 + return s.resolve(ctx, req)

28 +}

29 +

30 +func (s *publishedStore) resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

31 + project, secretName, version, ok := parseOperand(req.Operand)

32 + if !ok {

33 + return "", fmt.Errorf("resolving secret '%s': store '%s': operand must be in format 'project/secret' or 'project/secret/version'", req.Original, req.StoreKey)

34 + }

35 + if !reGCPSafeProjectID.MatchString(project) {

36 + return "", fmt.Errorf("resolving secret '%s': store '%s': invalid project ID '%s'", req.Original, req.StoreKey, project)

37 + }

38 + if !reGCPSafeName.MatchString(secretName) {

39 + return "", fmt.Errorf("resolving secret '%s': store '%s': invalid secret name '%s'", req.Original, req.StoreKey, secretName)

40 + }

41 + if !reGCPSafeName.MatchString(version) {

42 + return "", fmt.Errorf("resolving secret '%s': store '%s': invalid version '%s'", req.Original, req.StoreKey, version)

43 + }

44 +

45 + token, err := s.accessToken(ctx)

46 + if err != nil {

47 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

48 + }

49 +

50 + baseURL := s.provider.secretEndpoint

51 + if baseURL == "" {

52 + baseURL = "https://secretmanager.googleapis.com"

53 + }

54 + httpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s/v1/projects/%s/secrets/%s/versions/%s:access", baseURL, project, secretName, version), nil)

55 + if err != nil {

56 + return "", fmt.Errorf("resolving secret '%s': store '%s': creating request: %w", req.Original, req.StoreKey, err)

57 + }

58 + httpReq.Header.Set("Authorization", "Bearer "+token)

59 + resp, err := s.provider.apiClient.Do(httpReq)

60 + if err != nil {

61 + return "", fmt.Errorf("resolving secret '%s': store '%s': request failed: %w", req.Original, req.StoreKey, err)

62 + }

63 + defer resp.Body.Close()

64 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

65 + if err != nil {

66 + return "", fmt.Errorf("resolving secret '%s': store '%s': reading response: %w", req.Original, req.StoreKey, err)

67 + }

68 + if resp.StatusCode != http.StatusOK {

69 + return "", fmt.Errorf("resolving secret '%s': store '%s': GCP Secret Manager returned HTTP %d: %s", req.Original, req.StoreKey, resp.StatusCode, httpx.TruncateBody(body))

70 + }

71 + var result struct {

72 + Payload struct {

73 + Data string `json:"data"`

74 + } `json:"payload"`

75 + }

76 + if err := json.Unmarshal(body, &result); err != nil {

77 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing response: %w", req.Original, req.StoreKey, err)

78 + }

79 + decoded, err := base64.StdEncoding.DecodeString(result.Payload.Data)

80 + if err != nil {

81 + return "", fmt.Errorf("resolving secret '%s': store '%s': decoding secret data: %w", req.Original, req.StoreKey, err)

82 + }

83 + return string(decoded), nil

84 +}

85 +

86 +func parseOperand(operand string) (string, string, string, bool) {

87 + project, rest, ok := strings.Cut(operand, "/")

88 + if !ok || project == "" || rest == "" {

89 + return "", "", "", false

90 + }

91 + secretName, version, hasVersion := strings.Cut(rest, "/")

92 + if !hasVersion || version == "" {

93 + version = "latest"

94 + }

95 + return project, secretName, version, secretName != ""

96 +}

97 +

98 +func (s *publishedStore) accessToken(ctx context.Context) (string, error) {

99 + switch s.mode {

100 + case "metadata":

101 + return s.metadataToken(ctx)

102 + case "service_account_file":

103 + path := s.serviceAccountFilePath

104 + if path == "" {

105 + return "", fmt.Errorf("mode_service_account_file.path is required")

106 + }

107 + return s.serviceAccountToken(ctx, path)

108 + default:

109 + return "", fmt.Errorf("mode '%s' is invalid for gcp-sm", s.mode)

110 + }

111 +}

112 +

113 +func (s *publishedStore) metadataToken(ctx context.Context) (string, error) {

114 + req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token", nil)

115 + if err != nil {

116 + return "", fmt.Errorf("creating metadata token request: %w", err)

117 + }

118 + req.Header.Set("Metadata-Flavor", "Google")

119 + resp, err := s.provider.metadataClient.Do(req)

120 + if err != nil {

121 + return "", fmt.Errorf("metadata token request failed: %w", err)

122 + }

123 + defer resp.Body.Close()

124 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

125 + if err != nil {

126 + return "", fmt.Errorf("reading metadata token response: %w", err)

127 + }

128 + if resp.StatusCode != http.StatusOK {

129 + return "", fmt.Errorf("metadata token request returned HTTP %d: %s", resp.StatusCode, httpx.TruncateBody(body))

130 + }

131 + var result struct {

132 + AccessToken string `json:"access_token"`

133 + }

134 + if err := json.Unmarshal(body, &result); err != nil {

135 + return "", fmt.Errorf("parsing metadata token response: %w", err)

136 + }

137 + if result.AccessToken == "" {

138 + return "", fmt.Errorf("metadata token response missing access_token")

139 + }

140 + return result.AccessToken, nil

141 +}

142 +

143 +func (s *publishedStore) serviceAccountToken(ctx context.Context, credFile string) (string, error) {

144 + data, err := os.ReadFile(credFile)

145 + if err != nil {

146 + return "", fmt.Errorf("reading service account file '%s': %w", credFile, err)

147 + }

148 + var sa struct {

149 + ClientEmail string `json:"client_email"`

150 + PrivateKey string `json:"private_key"`

151 + TokenURI string `json:"token_uri"`

152 + }

153 + if err := json.Unmarshal(data, &sa); err != nil {

154 + return "", fmt.Errorf("parsing service account JSON: %w", err)

155 + }

156 + if sa.ClientEmail == "" || sa.PrivateKey == "" || sa.TokenURI == "" {

157 + return "", fmt.Errorf("service account JSON missing required fields (client_email, private_key, token_uri)")

158 + }

159 + now := s.provider.now().Unix()

160 + signedJWT, err := createSignedJWT(sa.ClientEmail, sa.TokenURI, sa.PrivateKey, now)

161 + if err != nil {

162 + return "", err

163 + }

164 + form := url.Values{

165 + "grant_type": {"urn:ietf:params:oauth:grant-type:jwt-bearer"},

166 + "assertion": {signedJWT},

167 + }

168 + httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, sa.TokenURI, strings.NewReader(form.Encode()))

169 + if err != nil {

170 + return "", fmt.Errorf("creating token exchange request: %w", err)

171 + }

172 + httpReq.Header.Set("Content-Type", "application/x-www-form-urlencoded")

173 + resp, err := s.provider.apiClient.Do(httpReq)

174 + if err != nil {

175 + return "", fmt.Errorf("token exchange request failed: %w", err)

176 + }

177 + defer resp.Body.Close()

178 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

179 + if err != nil {

180 + return "", fmt.Errorf("reading token exchange response: %w", err)

181 + }

182 + if resp.StatusCode != http.StatusOK {

183 + return "", fmt.Errorf("token exchange returned HTTP %d: %s", resp.StatusCode, httpx.TruncateBody(body))

184 + }

185 + var result struct {

186 + AccessToken string `json:"access_token"`

187 + }

188 + if err := json.Unmarshal(body, &result); err != nil {

189 + return "", fmt.Errorf("parsing token exchange response: %w", err)

190 + }

191 + if result.AccessToken == "" {

192 + return "", fmt.Errorf("token exchange response missing access_token")

193 + }

194 + return result.AccessToken, nil

195 +}

196 +

197 +func createSignedJWT(clientEmail, tokenURI, privateKeyPEM string, nowUnix int64) (string, error) {

198 + header := `{"alg":"RS256","typ":"JWT"}`

199 + claimsMap := map[string]any{

200 + "iss": clientEmail,

201 + "scope": "https://www.googleapis.com/auth/cloud-platform",

202 + "aud": tokenURI,

203 + "iat": nowUnix,

204 + "exp": nowUnix + 3600,

205 + }

206 + claimsJSON, err := json.Marshal(claimsMap)

207 + if err != nil {

208 + return "", fmt.Errorf("marshaling JWT claims: %w", err)

209 + }

210 + headerB64 := base64.RawURLEncoding.EncodeToString([]byte(header))

211 + claimsB64 := base64.RawURLEncoding.EncodeToString(claimsJSON)

212 + unsigned := headerB64 + "." + claimsB64

213 +

214 + block, _ := pem.Decode([]byte(privateKeyPEM))

215 + if block == nil {

216 + return "", fmt.Errorf("failed to decode PEM private key")

217 + }

218 + key, err := x509.ParsePKCS8PrivateKey(block.Bytes)

219 + if err != nil {

220 + return "", fmt.Errorf("parsing private key: %w", err)

221 + }

222 + rsaKey, ok := key.(*rsa.PrivateKey)

223 + if !ok {

224 + return "", fmt.Errorf("private key is not RSA")

225 + }

226 + hashed := sha256.Sum256([]byte(unsigned))

227 + sig, err := rsa.SignPKCS1v15(cryptorand.Reader, rsaKey, crypto.SHA256, hashed[:])

228 + if err != nil {

229 + return "", fmt.Errorf("signing JWT: %w", err)

230 + }

231 + return unsigned + "." + base64.RawURLEncoding.EncodeToString(sig), nil

232 +}

src/go/plugin/agent/secrets/secretstore/backends/registry.go new

+20

@@ -0,0 +1,20 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package backends

4 +

5 +import (

6 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

7 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends/aws"

8 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends/azure"

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends/gcp"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends/vault"

11 +)

12 +

13 +func Creators() []secretstore.Creator {

14 + return []secretstore.Creator{

15 + aws.New(),

16 + azure.New(),

17 + gcp.New(),

18 + vault.New(),

19 + }

20 +}

src/go/plugin/agent/secrets/secretstore/backends/vault/config_schema.json new

+112

@@ -0,0 +1,112 @@

1 +{

2 + "jsonSchema": {

3 + "$schema": "http://json-schema.org/draft-07/schema#",

4 + "type": "object",

5 + "title": "Vault secretstore configuration.",

6 + "properties": {

7 + "mode": {

8 + "title": "Mode",

9 + "description": "How Vault authentication is provided.",

10 + "type": "string",

11 + "enum": [

12 + "token",

13 + "token_file"

14 + ],

15 + "default": "token"

16 + },

17 + "addr": {

18 + "title": "Address",

19 + "description": "Vault server address / base URL.",

20 + "type": "string"

21 + },

22 + "namespace": {

23 + "title": "Namespace",

24 + "description": "Optional Vault namespace.",

25 + "type": "string"

26 + },

27 + "tls_skip_verify": {

28 + "title": "Skip TLS verification",

29 + "description": "Disable TLS certificate verification for Vault requests.",

30 + "type": "boolean",

31 + "default": false

32 + }

33 + },

34 + "required": [

35 + "mode",

36 + "addr"

37 + ],

38 + "dependencies": {

39 + "mode": {

40 + "oneOf": [

41 + {

42 + "properties": {

43 + "mode": {

44 + "const": "token"

45 + },

46 + "mode_token": {

47 + "title": "Token",

48 + "description": "Token settings used when mode is `token`.",

49 + "type": "object",

50 + "properties": {

51 + "token": {

52 + "title": "Token",

53 + "description": "Vault token value.",

54 + "type": "string",

55 + "sensitive": true

56 + }

57 + },

58 + "required": [

59 + "token"

60 + ]

61 + }

62 + },

63 + "required": [

64 + "mode_token"

65 + ]

66 + },

67 + {

68 + "properties": {

69 + "mode": {

70 + "const": "token_file"

71 + },

72 + "mode_token_file": {

73 + "title": "Token File",

74 + "description": "Token file settings used when mode is `token_file`.",

75 + "type": "object",

76 + "properties": {

77 + "path": {

78 + "title": "Path",

79 + "description": "Path to a file containing the Vault token.",

80 + "type": "string"

81 + }

82 + },

83 + "required": [

84 + "path"

85 + ]

86 + }

87 + },

88 + "required": [

89 + "mode_token_file"

90 + ]

91 + }

92 + ]

93 + }

94 + }

95 + },

96 + "uiSchema": {

97 + "uiOptions": {

98 + "fullPage": true

99 + },

100 + "mode": {

101 + "ui:widget": "radio",

102 + "ui:options": {

103 + "inline": true

104 + }

105 + },

106 + "mode_token": {

107 + "token": {

108 + "ui:widget": "password"

109 + }

110 + }

111 + }

112 +}

src/go/plugin/agent/secrets/secretstore/backends/vault/init.go new

+62

@@ -0,0 +1,62 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vault

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 +)

12 +

13 +func (s *store) init(_ context.Context) error {

14 + published := &publishedStore{

15 + provider: s.provider,

16 + }

17 +

18 + switch strings.TrimSpace(s.Config.Mode) {

19 + case "token":

20 + if s.Config.ModeToken == nil {

21 + return fmt.Errorf("mode_token is required when mode is 'token'")

22 + }

23 + token := strings.TrimSpace(s.Config.ModeToken.Token)

24 + if token == "" {

25 + return fmt.Errorf("mode_token.token is required")

26 + }

27 + s.Config.Mode = "token"

28 + s.Config.ModeToken.Token = token

29 + s.Config.ModeTokenFile = nil

30 + published.mode = s.Config.Mode

31 + published.tokenValue = token

32 + case "token_file":

33 + if s.Config.ModeTokenFile == nil {

34 + return fmt.Errorf("mode_token_file is required when mode is 'token_file'")

35 + }

36 + path := strings.TrimSpace(s.Config.ModeTokenFile.Path)

37 + if path == "" {

38 + return fmt.Errorf("mode_token_file.path is required")

39 + }

40 + s.Config.Mode = "token_file"

41 + s.Config.ModeTokenFile.Path = path

42 + s.Config.ModeToken = nil

43 + published.mode = s.Config.Mode

44 + published.tokenFilePath = path

45 + default:

46 + return fmt.Errorf("mode '%s' is invalid for kind '%s'", s.Config.Mode, secretstore.KindVault)

47 + }

48 +

49 + addr := strings.TrimSpace(s.Config.Addr)

50 + if addr == "" {

51 + return fmt.Errorf("addr is required")

52 + }

53 + s.Config.Addr = addr

54 + published.addr = addr

55 +

56 + s.Config.Namespace = strings.TrimSpace(s.Config.Namespace)

57 + published.namespaceValue = s.Config.Namespace

58 + published.tlsSkipVerify = s.Config.TLSSkipVerify

59 +

60 + s.published = published

61 + return nil

62 +}

src/go/plugin/agent/secrets/secretstore/backends/vault/provider.go new

+78

@@ -0,0 +1,78 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vault

4 +

5 +import (

6 + "context"

7 + _ "embed"

8 + "net/http"

9 + "time"

10 +

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

13 +)

14 +

15 +//go:embed config_schema.json

16 +var configSchema string

17 +

18 +type Config struct {

19 + Mode string `json:"mode" yaml:"mode"`

20 + ModeToken *ModeTokenConfig `json:"mode_token,omitempty" yaml:"mode_token,omitempty"`

21 + ModeTokenFile *ModeTokenFileConfig `json:"mode_token_file,omitempty" yaml:"mode_token_file,omitempty"`

22 + Addr string `json:"addr" yaml:"addr"`

23 + Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`

24 + TLSSkipVerify bool `json:"tls_skip_verify,omitempty" yaml:"tls_skip_verify,omitempty"`

25 +}

26 +

27 +type ModeTokenConfig struct {

28 + Token string `json:"token" yaml:"token"`

29 +}

30 +

31 +type ModeTokenFileConfig struct {

32 + Path string `json:"path" yaml:"path"`

33 +}

34 +

35 +type provider struct {

36 + httpClient *http.Client

37 + httpClientInsecure *http.Client

38 +}

39 +

40 +type store struct {

41 + Config `yaml:",inline" json:""`

42 + provider *provider

43 + published *publishedStore

44 +}

45 +

46 +type publishedStore struct {

47 + provider *provider

48 + mode string

49 + tokenValue string

50 + tokenFilePath string

51 + addr string

52 + namespaceValue string

53 + tlsSkipVerify bool

54 +}

55 +

56 +func New() secretstore.Creator {

57 + p := &provider{

58 + httpClient: httpx.VaultClient(10 * time.Second),

59 + httpClientInsecure: httpx.VaultInsecureClient(10 * time.Second),

60 + }

61 +

62 + return secretstore.Creator{

63 + Kind: secretstore.KindVault,

64 + DisplayName: "Vault",

65 + Schema: configSchema,

66 + Create: p.create,

67 + }

68 +}

69 +

70 +func (p *provider) create() secretstore.Store {

71 + return &store{provider: p}

72 +}

73 +

74 +func (s *store) Configuration() any { return &s.Config }

75 +

76 +func (s *store) Init(ctx context.Context) error { return s.init(ctx) }

77 +

78 +func (s *store) Publish() secretstore.PublishedStore { return s.published }

src/go/plugin/agent/secrets/secretstore/backends/vault/resolve.go new

+195

@@ -0,0 +1,195 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vault

4 +

5 +import (

6 + "context"

7 + "encoding/json"

8 + "fmt"

9 + "io"

10 + "net/http"

11 + "os"

12 + "strings"

13 +

14 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

15 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/httpx"

16 +)

17 +

18 +func (s *publishedStore) Resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

19 + return s.resolve(ctx, req)

20 +}

21 +

22 +func (s *publishedStore) resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

23 + path, key, ok := strings.Cut(req.Operand, "#")

24 + if !ok || key == "" {

25 + return "", fmt.Errorf("resolving secret '%s': store '%s': operand must be in format 'path#key'", req.Original, req.StoreKey)

26 + }

27 + if path == "" {

28 + return "", fmt.Errorf("resolving secret '%s': store '%s': vault path is empty", req.Original, req.StoreKey)

29 + }

30 + if strings.Contains(path, "..") || strings.ContainsAny(path, "?#") {

31 + return "", fmt.Errorf("resolving secret '%s': store '%s': vault path contains invalid characters", req.Original, req.StoreKey)

32 + }

33 +

34 + addr, err := s.address()

35 + if err != nil {

36 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

37 + }

38 +

39 + token, err := s.token()

40 + if err != nil {

41 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

42 + }

43 +

44 + httpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimRight(addr, "/")+"/v1/"+path, nil)

45 + if err != nil {

46 + return "", fmt.Errorf("resolving secret '%s': store '%s': %w", req.Original, req.StoreKey, err)

47 + }

48 + httpReq.Header.Set("X-Vault-Token", token)

49 + if ns, ok := s.namespace(); ok {

50 + httpReq.Header.Set("X-Vault-Namespace", ns)

51 + }

52 +

53 + client := s.provider.httpClient

54 + if s.skipVerify() {

55 + client = s.provider.httpClientInsecure

56 + }

57 +

58 + resp, err := client.Do(httpReq)

59 + if err != nil {

60 + return "", fmt.Errorf("resolving secret '%s': store '%s': vault request failed: %w", req.Original, req.StoreKey, err)

61 + }

62 + defer resp.Body.Close()

63 +

64 + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))

65 + if err != nil {

66 + return "", fmt.Errorf("resolving secret '%s': store '%s': reading vault response: %w", req.Original, req.StoreKey, err)

67 + }

68 + if resp.StatusCode != http.StatusOK {

69 + return "", fmt.Errorf("resolving secret '%s': store '%s': vault returned HTTP %d: %s", req.Original, req.StoreKey, resp.StatusCode, httpx.TruncateBody(body))

70 + }

71 + return parseResponse(body, key, req)

72 +}

73 +

74 +func (s *publishedStore) address() (string, error) {

75 + if s.addr == "" {

76 + return "", fmt.Errorf("addr is required")

77 + }

78 + return s.addr, nil

79 +}

80 +

81 +func (s *publishedStore) namespace() (string, bool) {

82 + if s.namespaceValue == "" {

83 + return "", false

84 + }

85 + return s.namespaceValue, true

86 +}

87 +

88 +func (s *publishedStore) skipVerify() bool {

89 + return s.tlsSkipVerify

90 +}

91 +

92 +func (s *publishedStore) token() (string, error) {

93 + switch s.mode {

94 + case "token":

95 + if s.tokenValue == "" {

96 + return "", fmt.Errorf("mode_token.token is required")

97 + }

98 + return s.tokenValue, nil

99 + case "token_file":

100 + path := s.tokenFilePath

101 + if path == "" {

102 + return "", fmt.Errorf("mode_token_file.path is required")

103 + }

104 + data, err := os.ReadFile(path)

105 + if err != nil {

106 + return "", fmt.Errorf("cannot read token file '%s': %w", path, err)

107 + }

108 + token := strings.TrimSpace(string(data))

109 + if token == "" {

110 + return "", fmt.Errorf("token file '%s' is empty", path)

111 + }

112 + return token, nil

113 + default:

114 + return "", fmt.Errorf("mode '%s' is invalid for vault", s.mode)

115 + }

116 +}

117 +

118 +func parseResponse(body []byte, key string, req secretstore.ResolveRequest) (string, error) {

119 + var resp vaultReadResponse

120 + if err := json.Unmarshal(body, &resp); err != nil {

121 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing vault response: %w", req.Original, req.StoreKey, err)

122 + }

123 +

124 + var kvV2 vaultKV2ReadPayload

125 + if err := json.Unmarshal(resp.Data, &kvV2); err != nil {

126 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing vault response data: %w", req.Original, req.StoreKey, err)

127 + }

128 + if kvV2.isDetected() {

129 + data, err := unmarshalVaultDataPayload(kvV2.Data)

130 + if err != nil {

131 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing vault response data: %w", req.Original, req.StoreKey, err)

132 + }

133 + if val, ok := data[key]; ok {

134 + return valueToString(val)

135 + }

136 + return "", fmt.Errorf("resolving secret '%s': store '%s': key '%s' not found in vault response", req.Original, req.StoreKey, key)

137 + }

138 +

139 + payload, err := unmarshalVaultDataPayload(resp.Data)

140 + if err != nil {

141 + return "", fmt.Errorf("resolving secret '%s': store '%s': parsing vault response data: %w", req.Original, req.StoreKey, err)

142 + }

143 + if val, ok := payload[key]; ok {

144 + return valueToString(val)

145 + }

146 +

147 + return "", fmt.Errorf("resolving secret '%s': store '%s': key '%s' not found in vault response", req.Original, req.StoreKey, key)

148 +}

149 +

150 +type vaultReadResponse struct {

151 + Data json.RawMessage `json:"data"`

152 +}

153 +

154 +type vaultKV2ReadPayload struct {

155 + Data json.RawMessage `json:"data"`

156 + Metadata *vaultKV2Metadata `json:"metadata"`

157 +}

158 +

159 +type vaultKV2Metadata struct {

160 + CreatedTime *string `json:"created_time"`

161 + DeletionTime *string `json:"deletion_time"`

162 + Destroyed *bool `json:"destroyed"`

163 + Version *int `json:"version"`

164 +}

165 +

166 +func (p vaultKV2ReadPayload) isDetected() bool {

167 + // KV v2 read responses expose secret values under data.data and include

168 + // version metadata alongside it. Requiring the standard metadata fields

169 + // avoids misclassifying KV v1 secrets that happen to have top-level

170 + // keys named "data" and "metadata".

171 + return len(p.Data) != 0 && p.Metadata != nil &&

172 + p.Metadata.CreatedTime != nil &&

173 + p.Metadata.DeletionTime != nil &&

174 + p.Metadata.Destroyed != nil &&

175 + p.Metadata.Version != nil

176 +}

177 +

178 +func unmarshalVaultDataPayload(data json.RawMessage) (map[string]any, error) {

179 + var payload map[string]any

180 + if err := json.Unmarshal(data, &payload); err != nil {

181 + return nil, err

182 + }

183 + return payload, nil

184 +}

185 +

186 +func valueToString(val any) (string, error) {

187 + if s, ok := val.(string); ok {

188 + return s, nil

189 + }

190 + b, err := json.Marshal(val)

191 + if err != nil {

192 + return "", fmt.Errorf("encoding vault value: %w", err)

193 + }

194 + return string(b), nil

195 +}

src/go/plugin/agent/secrets/secretstore/backends/vault/resolve_test.go new

+69

@@ -0,0 +1,69 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package vault

4 +

5 +import (

6 + "testing"

7 +

8 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

9 + "github.com/stretchr/testify/assert"

10 + "github.com/stretchr/testify/require"

11 +)

12 +

13 +func TestParseResponse(t *testing.T) {

14 + req := secretstore.ResolveRequest{

15 + StoreKey: "vault:vault_prod",

16 + StoreKind: secretstore.KindVault,

17 + StoreName: "vault_prod",

18 + Original: "${store:vault:vault_prod:secret/data/mysql#password}",

19 + }

20 +

21 + const kv2Body = `{"data":{"data":{"password":"s3cr3t"},"metadata":{"created_time":"2024-01-01T00:00:00Z","deletion_time":"","destroyed":false,"version":3}}}`

22 +

23 + tests := map[string]struct {

24 + body string

25 + key string

26 + want string

27 + wantErrContains string

28 + }{

29 + "kv1 returns requested key": {

30 + body: `{"data":{"password":"s3cr3t","username":"netdata"}}`,

31 + key: "password",

32 + want: "s3cr3t",

33 + },

34 + "kv1 with top-level data key still uses kv1 lookup": {

35 + body: `{"data":{"data":{"nested":"value"},"password":"s3cr3t"}}`,

36 + key: "password",

37 + want: "s3cr3t",

38 + },

39 + "kv2 returns requested nested key": {

40 + body: kv2Body,

41 + key: "password",

42 + want: "s3cr3t",

43 + },

44 + "kv2 does not leak metadata envelope field": {

45 + body: kv2Body,

46 + key: "metadata",

47 + wantErrContains: "key 'metadata' not found",

48 + },

49 + "kv2 does not leak data envelope field": {

50 + body: kv2Body,

51 + key: "data",

52 + wantErrContains: "key 'data' not found",

53 + },

54 + }

55 +

56 + for name, tc := range tests {

57 + t.Run(name, func(t *testing.T) {

58 + got, err := parseResponse([]byte(tc.body), tc.key, req)

59 + if tc.wantErrContains != "" {

60 + require.Error(t, err)

61 + assert.Contains(t, err.Error(), tc.wantErrContains)

62 + return

63 + }

64 +

65 + require.NoError(t, err)

66 + assert.Equal(t, tc.want, got)

67 + })

68 + }

69 +}

src/go/plugin/agent/secrets/secretstore/compile.go new

+37

@@ -0,0 +1,37 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import "gopkg.in/yaml.v2"

6 +

7 +func cloneConfig(in Config) Config {

8 + if len(in) == 0 {

9 + return nil

10 + }

11 +

12 + type plain Config

13 + bs, err := yaml.Marshal((plain)(in))

14 + if err != nil {

15 + return nil

16 + }

17 +

18 + var out Config

19 + if err := yaml.Unmarshal(bs, &out); err != nil {

20 + return nil

21 + }

22 + return out

23 +}

24 +

25 +func cloneStoreStatus(status StoreStatus) StoreStatus {

26 + out := status

27 + out.LastValidation = cloneValidationStatus(status.LastValidation)

28 + return out

29 +}

30 +

31 +func cloneValidationStatus(status *ValidationStatus) *ValidationStatus {

32 + if status == nil {

33 + return nil

34 + }

35 + copyStatus := *status

36 + return &copyStatus

37 +}

src/go/plugin/agent/secrets/secretstore/doc.go new

+12

@@ -0,0 +1,12 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +// Package secretstore defines control-plane contracts for go.d v2 secretstores.

4 +//

5 +// Current scope:

6 +// - core data model types

7 +// - raw-config and runtime contracts

8 +// - manager and snapshot provider interfaces

9 +//

10 +// Runtime behavior (validation, normalization, publication, and resolution)

11 +// is implemented by the service/runtime packages.

12 +package secretstore

src/go/plugin/agent/secrets/secretstore/fileconfig.go new

+123

@@ -0,0 +1,123 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "fmt"

7 + "os"

8 + "path/filepath"

9 + "strings"

10 +

11 + "github.com/netdata/netdata/go/plugins/pkg/pluginconfig"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

13 + "gopkg.in/yaml.v2"

14 +)

15 +

16 +type fileRootConfig struct {

17 + Jobs []map[string]any `yaml:"jobs"`

18 +}

19 +

20 +func LoadFileConfigs(roots []string) ([]Config, []error) {

21 + var cfgs []Config

22 + var errs []error

23 +

24 + for _, root := range roots {

25 + root = strings.TrimSpace(root)

26 + if root == "" {

27 + continue

28 + }

29 +

30 + dir := filepath.Join(root, "ss")

31 + entries, err := os.ReadDir(dir)

32 + if err != nil {

33 + continue

34 + }

35 +

36 + for _, entry := range entries {

37 + if !entry.Type().IsRegular() {

38 + continue

39 + }

40 +

41 + ext := strings.ToLower(filepath.Ext(entry.Name()))

42 + if ext != ".conf" && ext != ".yaml" && ext != ".yml" {

43 + continue

44 + }

45 +

46 + path := filepath.Join(dir, entry.Name())

47 + stem := strings.TrimSpace(strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))

48 +

49 + jobs, jobErrs := loadFileConfigJobs(path)

50 + errs = append(errs, jobErrs...)

51 + if len(jobs) == 0 {

52 + continue

53 + }

54 +

55 + sourceType := fileConfigSourceType(path)

56 + source := "file=" + path

57 + for i, job := range jobs {

58 + cfg := Config(job)

59 + if strings.TrimSpace(cfg.Name()) == "" {

60 + errs = append(errs, fmt.Errorf("secretstore file config '%s' job %d: store name is required", path, i+1))

61 + continue

62 + }

63 + if cfg.Kind() == "" {

64 + cfg.SetKind(StoreKind(stem))

65 + }

66 + cfg.SetSource(source)

67 + cfg.SetSourceType(sourceType)

68 + if err := validateFileConfig(cfg); err != nil {

69 + errs = append(errs, fmt.Errorf("secretstore file config '%s' job %d: %w", path, i+1, err))

70 + continue

71 + }

72 + cfgs = append(cfgs, cfg)

73 + }

74 + }

75 + }

76 +

77 + return cfgs, errs

78 +}

79 +

80 +func loadFileConfigJobs(path string) ([]map[string]any, []error) {

81 + data, err := os.ReadFile(path)

82 + if err != nil {

83 + return nil, []error{fmt.Errorf("secretstore file config '%s': %w", path, err)}

84 + }

85 +

86 + var root fileRootConfig

87 + if err := yaml.Unmarshal(data, &root); err != nil {

88 + return nil, []error{fmt.Errorf("secretstore file config '%s': %w", path, err)}

89 + }

90 +

91 + return root.Jobs, nil

92 +}

93 +

94 +func fileConfigSourceType(path string) string {

95 + if pluginconfig.IsStock(path) {

96 + return confgroup.TypeStock

97 + }

98 + return confgroup.TypeUser

99 +}

100 +

101 +func validateFileConfig(cfg Config) error {

102 + if cfg == nil {

103 + return fmt.Errorf("store config is nil")

104 + }

105 + if strings.TrimSpace(cfg.Name()) == "" {

106 + return fmt.Errorf("store name is required")

107 + }

108 + if err := validateStoreName(cfg.Name()); err != nil {

109 + return fmt.Errorf("invalid store name '%s': %w", cfg.Name(), err)

110 + }

111 + if strings.TrimSpace(string(cfg.Kind())) == "" {

112 + return fmt.Errorf("store kind is required")

113 + }

114 + if strings.TrimSpace(cfg.Source()) == "" {

115 + return fmt.Errorf("store source is required")

116 + }

117 + switch cfg.SourceType() {

118 + case confgroup.TypeUser, confgroup.TypeStock:

119 + return nil

120 + default:

121 + return fmt.Errorf("invalid store source type '%s'", cfg.SourceType())

122 + }

123 +}

src/go/plugin/agent/secrets/secretstore/fileconfig_test.go new

+132

@@ -0,0 +1,132 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "os"

7 + "path/filepath"

8 + "strings"

9 + "testing"

10 +

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

13 + "github.com/stretchr/testify/assert"

14 + "github.com/stretchr/testify/require"

15 +)

16 +

17 +func TestLoadFileConfigs(t *testing.T) {

18 + t.Run("loads user and stock roots with filename defaults", func(t *testing.T) {

19 + base := t.TempDir()

20 + userRoot := filepath.Join(base, "etc", "netdata", "go.d")

21 + stockRoot := filepath.Join(base, "usr", "lib", "netdata", "conf.d", "go.d")

22 +

23 + mustWriteSecretStoreConfigFile(t, filepath.Join(userRoot, "ss", "vault.conf"), `

24 +jobs:

25 + - name: vault_prod

26 + mode: token

27 + mode_token:

28 + token: vault-token

29 + addr: https://vault.example

30 +`)

31 + mustWriteSecretStoreConfigFile(t, filepath.Join(userRoot, "ss", "azure-kv.yaml"), `

32 +jobs:

33 + - name: azure_prod

34 + mode: managed_identity

35 +`)

36 + mustWriteSecretStoreConfigFile(t, filepath.Join(stockRoot, "ss", "aws-sm.conf"), `

37 +jobs:

38 + - name: aws_prod

39 + auth_mode: env

40 + region: us-east-1

41 +`)

42 + mustWriteSecretStoreConfigFile(t, filepath.Join(stockRoot, "ss", "gcp-sm.yml"), `

43 +jobs:

44 + - name: gcp_prod

45 + mode: metadata

46 +`)

47 +

48 + cfgs, errs := secretstore.LoadFileConfigs([]string{userRoot, stockRoot})

49 + require.Empty(t, errs)

50 + require.Len(t, cfgs, 4)

51 +

52 + assert.Equal(t, secretstore.KindAzureKV, cfgs[0].Kind())

53 + assert.Equal(t, confgroup.TypeUser, cfgs[0].SourceType())

54 + assert.Contains(t, cfgs[0].Source(), "file="+filepath.Join(userRoot, "ss", "azure-kv.yaml"))

55 +

56 + assert.Equal(t, secretstore.KindVault, cfgs[1].Kind())

57 + assert.Equal(t, confgroup.TypeUser, cfgs[1].SourceType())

58 +

59 + assert.Equal(t, secretstore.KindAWSSM, cfgs[2].Kind())

60 + assert.Equal(t, confgroup.TypeStock, cfgs[2].SourceType())

61 +

62 + assert.Equal(t, secretstore.KindGCPSM, cfgs[3].Kind())

63 + assert.Equal(t, confgroup.TypeStock, cfgs[3].SourceType())

64 + })

65 +

66 + t.Run("keeps explicit kind and allows unknown file stem", func(t *testing.T) {

67 + base := t.TempDir()

68 + userRoot := filepath.Join(base, "etc", "netdata", "go.d")

69 +

70 + mustWriteSecretStoreConfigFile(t, filepath.Join(userRoot, "ss", "custom.conf"), `

71 +jobs:

72 + - name: explicit_kind

73 + kind: vault

74 + mode: token

75 + mode_token:

76 + token: vault-token

77 + addr: https://vault.example

78 + - name: inferred_kind

79 + mode: token

80 + mode_token:

81 + token: custom-token

82 + addr: https://vault.example

83 +`)

84 +

85 + cfgs, errs := secretstore.LoadFileConfigs([]string{userRoot})

86 + require.Empty(t, errs)

87 + require.Len(t, cfgs, 2)

88 + assert.Equal(t, secretstore.KindVault, cfgs[0].Kind())

89 + assert.Equal(t, secretstore.StoreKind("custom"), cfgs[1].Kind())

90 + })

91 +

92 + t.Run("skips malformed files and non-keyable jobs", func(t *testing.T) {

93 + base := t.TempDir()

94 + userRoot := filepath.Join(base, "etc", "netdata", "go.d")

95 +

96 + mustWriteSecretStoreConfigFile(t, filepath.Join(userRoot, "ss", "vault.conf"), `

97 +jobs:

98 + - mode: token

99 + mode_token:

100 + token: missing-name

101 + addr: https://vault.example

102 + - name: vault_prod

103 + mode: token

104 + mode_token:

105 + token: vault-token

106 + addr: https://vault.example

107 +`)

108 + mustWriteSecretStoreConfigFile(t, filepath.Join(userRoot, "ss", "gcp-sm.conf"), `

109 +jobs:

110 + - name: gcp_prod

111 + mode: [

112 +`)

113 +

114 + cfgs, errs := secretstore.LoadFileConfigs([]string{userRoot})

115 + require.Len(t, cfgs, 1)

116 + assert.Equal(t, "vault_prod", cfgs[0].Name())

117 + assert.Len(t, errs, 2)

118 + var messages []string

119 + for _, err := range errs {

120 + messages = append(messages, err.Error())

121 + }

122 + joined := strings.Join(messages, "\n")

123 + assert.Contains(t, joined, "store name is required")

124 + assert.Contains(t, joined, "secretstore file config")

125 + })

126 +}

127 +

128 +func mustWriteSecretStoreConfigFile(t *testing.T, path, content string) {

129 + t.Helper()

130 + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))

131 + require.NoError(t, os.WriteFile(path, []byte(content), 0o644))

132 +}

src/go/plugin/agent/secrets/secretstore/helpers.go new

@@ -0,0 +1,9 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import "github.com/netdata/netdata/go/plugins/plugin/framework/dyncfg"

6 +

7 +func validateStoreName(name string) error {

8 + return dyncfg.ValidateJobName(name)

9 +}

src/go/plugin/agent/secrets/secretstore/internal/envx/envx.go new

+68

@@ -0,0 +1,68 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package envx

4 +

5 +import (

6 + "fmt"

7 + "os"

8 + "regexp"

9 + "strings"

10 +)

11 +

12 +var reEnvVarID = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)

13 +

14 +func Lookup(name string) (string, bool) {

15 + v, ok := os.LookupEnv(name)

16 + if !ok {

17 + return "", false

18 + }

19 + return strings.TrimSpace(v), true

20 +}

21 +

22 +func ResolveRequired(selector, fieldPath string) (string, error) {

23 + selector = strings.TrimSpace(selector)

24 + if selector == "" {

25 + return "", fmt.Errorf("%s selector is empty", fieldPath)

26 + }

27 + val, ok := Lookup(selector)

28 + if !ok || strings.TrimSpace(val) == "" {

29 + return "", fmt.Errorf("%s references env '%s' which is not set", fieldPath, selector)

30 + }

31 + return strings.TrimSpace(val), nil

32 +}

33 +

34 +func ResolveOptional(selector string) (string, bool) {

35 + selector = strings.TrimSpace(selector)

36 + if selector == "" {

37 + return "", false

38 + }

39 + val, ok := Lookup(selector)

40 + if !ok || strings.TrimSpace(val) == "" {

41 + return "", false

42 + }

43 + return strings.TrimSpace(val), true

44 +}

45 +

46 +func ValidateSelector(name, fieldPath string) error {

47 + name = strings.TrimSpace(name)

48 + if name == "" {

49 + return fmt.Errorf("%s is required", fieldPath)

50 + }

51 + if !reEnvVarID.MatchString(name) {

52 + return fmt.Errorf("%s must reference an environment variable name", fieldPath)

53 + }

54 + return nil

55 +}

56 +

57 +func ParseBool(name string) bool {

58 + v, ok := Lookup(name)

59 + if !ok {

60 + return false

61 + }

62 + switch strings.ToLower(strings.TrimSpace(v)) {

63 + case "1", "true", "yes":

64 + return true

65 + default:

66 + return false

67 + }

68 +}

src/go/plugin/agent/secrets/secretstore/internal/httpx/httpx.go new

+68

@@ -0,0 +1,68 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package httpx

4 +

5 +import (

6 + "crypto/tls"

7 + "net/http"

8 + "strings"

9 + "time"

10 +)

11 +

12 +func noRedirect(*http.Request, []*http.Request) error {

13 + return http.ErrUseLastResponse

14 +}

15 +

16 +func APIClient(timeout time.Duration) *http.Client {

17 + return &http.Client{Timeout: timeout}

18 +}

19 +

20 +func NoProxyClient(timeout time.Duration) *http.Client {

21 + transport := cloneDefaultTransport()

22 + transport.Proxy = nil

23 +

24 + return &http.Client{

25 + Timeout: timeout,

26 + Transport: transport,

27 + }

28 +}

29 +

30 +func VaultClient(timeout time.Duration) *http.Client {

31 + return &http.Client{

32 + Timeout: timeout,

33 + CheckRedirect: noRedirect,

34 + }

35 +}

36 +

37 +func VaultInsecureClient(timeout time.Duration) *http.Client {

38 + transport := cloneDefaultTransport()

39 + if transport.TLSClientConfig != nil {

40 + transport.TLSClientConfig = transport.TLSClientConfig.Clone()

41 + } else {

42 + transport.TLSClientConfig = &tls.Config{}

43 + }

44 + transport.TLSClientConfig.InsecureSkipVerify = true

45 +

46 + return &http.Client{

47 + Timeout: timeout,

48 + Transport: transport,

49 + CheckRedirect: noRedirect,

50 + }

51 +}

52 +

53 +func cloneDefaultTransport() *http.Transport {

54 + transport, ok := http.DefaultTransport.(*http.Transport)

55 + if !ok || transport == nil {

56 + return &http.Transport{}

57 + }

58 + return transport.Clone()

59 +}

60 +

61 +func TruncateBody(body []byte) string {

62 + const maxLen = 200

63 + s := strings.TrimSpace(string(body))

64 + if len(s) > maxLen {

65 + return s[:maxLen] + "..."

66 + }

67 + return s

68 +}

src/go/plugin/agent/secrets/secretstore/internal/httpx/httpx_test.go new

+83

@@ -0,0 +1,83 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package httpx

4 +

5 +import (

6 + "net/http"

7 + "net/http/httptest"

8 + "testing"

9 + "time"

10 +

11 + "github.com/stretchr/testify/assert"

12 + "github.com/stretchr/testify/require"

13 +)

14 +

15 +func TestVaultInsecureClient_PreservesDefaultTransportBehavior(t *testing.T) {

16 + client := VaultInsecureClient(5 * time.Second)

17 +

18 + transport, ok := client.Transport.(*http.Transport)

19 + require.True(t, ok)

20 +

21 + defaultTransport, ok := http.DefaultTransport.(*http.Transport)

22 + require.True(t, ok)

23 + require.NotNil(t, defaultTransport.Proxy)

24 +

25 + assert.Equal(t, 5*time.Second, client.Timeout)

26 + require.NotNil(t, client.CheckRedirect)

27 +

28 + assert.NotSame(t, defaultTransport, transport)

29 + assert.NotNil(t, transport.Proxy)

30 + assert.Equal(t, defaultTransport.MaxIdleConns, transport.MaxIdleConns)

31 + assert.Equal(t, defaultTransport.IdleConnTimeout, transport.IdleConnTimeout)

32 + assert.Equal(t, defaultTransport.TLSHandshakeTimeout, transport.TLSHandshakeTimeout)

33 +

34 + require.NotNil(t, transport.TLSClientConfig)

35 + assert.True(t, transport.TLSClientConfig.InsecureSkipVerify)

36 +}

37 +

38 +func TestVaultClient_DoesNotFollowTemporaryRedirects(t *testing.T) {

39 + tokenSeen := make(chan string, 1)

40 + target := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

41 + tokenSeen <- r.Header.Get("X-Vault-Token")

42 + w.WriteHeader(http.StatusOK)

43 + }))

44 + defer target.Close()

45 +

46 + redirect := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

47 + http.Redirect(w, r, target.URL, http.StatusTemporaryRedirect)

48 + }))

49 + defer redirect.Close()

50 +

51 + client := VaultClient(5 * time.Second)

52 + req, err := http.NewRequest(http.MethodGet, redirect.URL, nil)

53 + require.NoError(t, err)

54 + req.Header.Set("X-Vault-Token", "vault-token")

55 +

56 + resp, err := client.Do(req)

57 + require.NoError(t, err)

58 + defer resp.Body.Close()

59 +

60 + assert.Equal(t, http.StatusTemporaryRedirect, resp.StatusCode)

61 + select {

62 + case seen := <-tokenSeen:

63 + t.Fatalf("unexpected redirected request carrying token %q", seen)

64 + default:

65 + }

66 +}

67 +

68 +func TestNoProxyClient_PreservesDefaultTransportBehavior(t *testing.T) {

69 + client := NoProxyClient(2 * time.Second)

70 +

71 + transport, ok := client.Transport.(*http.Transport)

72 + require.True(t, ok)

73 +

74 + defaultTransport, ok := http.DefaultTransport.(*http.Transport)

75 + require.True(t, ok)

76 +

77 + assert.Equal(t, 2*time.Second, client.Timeout)

78 + assert.NotSame(t, defaultTransport, transport)

79 + assert.Nil(t, transport.Proxy)

80 + assert.Equal(t, defaultTransport.MaxIdleConns, transport.MaxIdleConns)

81 + assert.Equal(t, defaultTransport.IdleConnTimeout, transport.IdleConnTimeout)

82 + assert.Equal(t, defaultTransport.TLSHandshakeTimeout, transport.TLSHandshakeTimeout)

83 +}

src/go/plugin/agent/secrets/secretstore/internal/specutil/specutil.go new

+57

@@ -0,0 +1,57 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package specutil

4 +

5 +import (

6 + "fmt"

7 + "regexp"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/internal/envx"

12 +)

13 +

14 +var reStoreName = regexp.MustCompile(`^[A-Za-z0-9_]+$`)

15 +

16 +func NormalizeStoreName(name string) (string, error) {

17 + name = strings.TrimSpace(name)

18 + if name == "" {

19 + return "", fmt.Errorf("store name is required")

20 + }

21 + if !reStoreName.MatchString(name) {

22 + return "", fmt.Errorf("store name '%s' must match %s", name, reStoreName.String())

23 + }

24 + return name, nil

25 +}

26 +

27 +func ValidateStoreKind(key string, kind, expected secretstore.StoreKind) error {

28 + if !kind.IsValid() {

29 + return fmt.Errorf("store '%s': invalid kind '%s'", key, kind)

30 + }

31 + if kind != expected {

32 + return fmt.Errorf("store '%s': invalid kind '%s'", key, kind)

33 + }

34 + return nil

35 +}

36 +

37 +func RequireEnvSelector(v, path string) (string, error) {

38 + v = strings.TrimSpace(v)

39 + if v == "" {

40 + return "", fmt.Errorf("%s is required", path)

41 + }

42 + if err := envx.ValidateSelector(v, path); err != nil {

43 + return "", err

44 + }

45 + return v, nil

46 +}

47 +

48 +func OptionalEnvSelector(v, path string) (string, error) {

49 + v = strings.TrimSpace(v)

50 + if v == "" {

51 + return "", nil

52 + }

53 + if err := envx.ValidateSelector(v, path); err != nil {

54 + return "", err

55 + }

56 + return v, nil

57 +}

src/go/plugin/agent/secrets/secretstore/provider_parity_test.go new

+144

@@ -0,0 +1,144 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "testing"

7 +

8 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

10 + "github.com/stretchr/testify/assert"

11 + "github.com/stretchr/testify/require"

12 +)

13 +

14 +func TestProviderSchemaAndValidationParity(t *testing.T) {

15 + svc := secretstore.NewService(backends.Creators()...)

16 +

17 + tests := map[string]struct {

18 + kind secretstore.StoreKind

19 + valid map[string]any

20 + invalid map[string]any

21 + wantErrContains string

22 + assertSchemaShape func(t *testing.T, schema map[string]any)

23 + }{

24 + "aws": {

25 + kind: secretstore.KindAWSSM,

26 + valid: map[string]any{

27 + "name": "aws_prod",

28 + "auth_mode": "env",

29 + "region": "us-east-1",

30 + },

31 + invalid: map[string]any{

32 + "name": "aws_prod",

33 + "auth_mode": "env",

34 + },

35 + wantErrContains: "region is required",

36 + assertSchemaShape: func(t *testing.T, schema map[string]any) {

37 + jsonSchema := schema["jsonSchema"].(map[string]any)

38 + assert.Contains(t, jsonSchema["required"], "auth_mode")

39 + assert.Contains(t, jsonSchema["required"], "region")

40 + _, ok := jsonSchema["allOf"]

41 + assert.False(t, ok)

42 + },

43 + },

44 + "azure": {

45 + kind: secretstore.KindAzureKV,

46 + valid: map[string]any{

47 + "name": "azure_prod",

48 + "mode": "managed_identity",

49 + },

50 + invalid: map[string]any{

51 + "name": "azure_prod",

52 + "mode": "client",

53 + "mode_client": map[string]any{

54 + "client_id": "client-id",

55 + },

56 + },

57 + wantErrContains: "mode_client.tenant_id is required",

58 + assertSchemaShape: func(t *testing.T, schema map[string]any) {

59 + jsonSchema := schema["jsonSchema"].(map[string]any)

60 + uiSchema := schema["uiSchema"].(map[string]any)

61 + assert.Contains(t, jsonSchema["required"], "mode")

62 + deps := jsonSchema["dependencies"].(map[string]any)

63 + assert.Contains(t, deps, "mode")

64 + modeClient := uiSchema["mode_client"].(map[string]any)

65 + clientSecret := modeClient["client_secret"].(map[string]any)

66 + assert.Equal(t, "password", clientSecret["ui:widget"])

67 + },

68 + },

69 + "gcp": {

70 + kind: secretstore.KindGCPSM,

71 + valid: map[string]any{

72 + "name": "gcp_prod",

73 + "mode": "metadata",

74 + },

75 + invalid: map[string]any{

76 + "name": "gcp_prod",

77 + "mode": "service_account_file",

78 + "mode_service_account_file": map[string]any{},

79 + },

80 + wantErrContains: "mode_service_account_file.path is required",

81 + assertSchemaShape: func(t *testing.T, schema map[string]any) {

82 + jsonSchema := schema["jsonSchema"].(map[string]any)

83 + assert.Contains(t, jsonSchema["required"], "mode")

84 + deps := jsonSchema["dependencies"].(map[string]any)

85 + assert.Contains(t, deps, "mode")

86 + },

87 + },

88 + "vault": {

89 + kind: secretstore.KindVault,

90 + valid: map[string]any{

91 + "name": "vault_prod",

92 + "mode": "token",

93 + "mode_token": map[string]any{

94 + "token": "vault-token",

95 + },

96 + "addr": "https://vault.example",

97 + },

98 + invalid: map[string]any{

99 + "name": "vault_prod",

100 + "mode": "token_file",

101 + "addr": "https://vault.example",

102 + },

103 + wantErrContains: "mode_token_file is required",

104 + assertSchemaShape: func(t *testing.T, schema map[string]any) {

105 + jsonSchema := schema["jsonSchema"].(map[string]any)

106 + uiSchema := schema["uiSchema"].(map[string]any)

107 + assert.Contains(t, jsonSchema["required"], "addr")

108 + assert.Contains(t, jsonSchema["required"], "mode")

109 + assert.NotContains(t, jsonSchema["required"], "kind")

110 + deps := jsonSchema["dependencies"].(map[string]any)

111 + assert.Contains(t, deps, "mode")

112 + modeToken := uiSchema["mode_token"].(map[string]any)

113 + token := modeToken["token"].(map[string]any)

114 + assert.Equal(t, "password", token["ui:widget"])

115 + },

116 + },

117 + }

118 +

119 + for name, tc := range tests {

120 + t.Run(name, func(t *testing.T) {

121 + displayName, ok := svc.DisplayName(tc.kind)

122 + require.True(t, ok)

123 + assert.NotEmpty(t, displayName)

124 +

125 + schema, ok := svc.Schema(tc.kind)

126 + require.True(t, ok)

127 + schemaObj := decodeSchema(t, schema)

128 + _, ok = schemaObj["jsonSchema"].(map[string]any)

129 + require.True(t, ok)

130 + _, ok = schemaObj["uiSchema"].(map[string]any)

131 + require.True(t, ok)

132 + if tc.assertSchemaShape != nil {

133 + tc.assertSchemaShape(t, schemaObj)

134 + }

135 +

136 + err := svc.Validate(newStoreFromConfig(t, svc, tc.kind, tc.valid))

137 + require.NoError(t, err)

138 +

139 + err = svc.Validate(newStoreFromConfig(t, svc, tc.kind, tc.invalid))

140 + require.Error(t, err)

141 + assert.ErrorContains(t, err, tc.wantErrContains)

142 + })

143 + }

144 +}

src/go/plugin/agent/secrets/secretstore/raw_config.go new

+154

@@ -0,0 +1,154 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "fmt"

7 + "strings"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

10 +

11 + "github.com/gohugoio/hashstructure"

12 +)

13 +

14 +const (

15 + keyName = "name"

16 + keyKind = "kind"

17 +

18 + ikeySource = "__source__"

19 + ikeySourceType = "__source_type__"

20 +)

21 +

22 +// Config is the external raw secretstore object shape used by dyncfg/cache layers.

23 +// It carries structural metadata alongside provider payload and excludes __ metadata

24 +// from content hashing.

25 +type Config map[string]any

26 +

27 +func (c Config) Set(key string, value any) Config { c[key] = value; return c }

28 +

29 +func StoreKey(kind StoreKind, name string) string {

30 + kind = StoreKind(strings.TrimSpace(string(kind)))

31 + name = strings.TrimSpace(name)

32 + if kind == "" || name == "" {

33 + return ""

34 + }

35 + return string(kind) + ":" + name

36 +}

37 +

38 +func ParseStoreKey(key string) (StoreKind, string, error) {

39 + key = strings.TrimSpace(key)

40 + if key == "" {

41 + return "", "", fmt.Errorf("store key is required")

42 + }

43 +

44 + kindPart, name, ok := strings.Cut(key, ":")

45 + if !ok {

46 + return "", "", fmt.Errorf("store key '%s' must be in format 'kind:name'", key)

47 + }

48 +

49 + kind := StoreKind(strings.TrimSpace(kindPart))

50 + name = strings.TrimSpace(name)

51 + if !kind.IsValid() {

52 + return "", "", fmt.Errorf("invalid store kind '%s'", kind)

53 + }

54 + if err := validateStoreName(name); err != nil {

55 + return "", "", fmt.Errorf("invalid store name '%s': %w", name, err)

56 + }

57 +

58 + return kind, name, nil

59 +}

60 +

61 +func (c Config) Name() string {

62 + v, _ := c[keyName].(string)

63 + return v

64 +}

65 +

66 +func (c Config) Kind() StoreKind {

67 + switch v := c[keyKind].(type) {

68 + case StoreKind:

69 + return v

70 + case string:

71 + return StoreKind(v)

72 + default:

73 + return ""

74 + }

75 +}

76 +

77 +func (c Config) Source() string {

78 + v, _ := c[ikeySource].(string)

79 + return v

80 +}

81 +

82 +func (c Config) SourceType() string {

83 + v, _ := c[ikeySourceType].(string)

84 + return v

85 +}

86 +

87 +func (c Config) SetName(v string) Config { return c.Set(keyName, v) }

88 +func (c Config) SetKind(v StoreKind) Config { return c.Set(keyKind, string(v)) }

89 +func (c Config) SetSource(v string) Config { return c.Set(ikeySource, v) }

90 +func (c Config) SetSourceType(v string) Config { return c.Set(ikeySourceType, v) }

91 +

92 +func (c Config) ExposedKey() string {

93 + return StoreKey(c.Kind(), c.Name())

94 +}

95 +

96 +func (c Config) UID() string {

97 + if c.Source() == "" || c.ExposedKey() == "" {

98 + return ""

99 + }

100 + return c.Source() + ":" + c.ExposedKey()

101 +}

102 +

103 +func (c Config) SourceTypePriority() int {

104 + switch c.SourceType() {

105 + case confgroup.TypeDyncfg:

106 + return 16

107 + case confgroup.TypeUser:

108 + return 8

109 + case confgroup.TypeStock:

110 + return 2

111 + default:

112 + return 0

113 + }

114 +}

115 +

116 +func (c Config) HashIncludeMap(_ string, k, _ any) (bool, error) {

117 + s := k.(string)

118 + return !strings.HasPrefix(s, "__") && !strings.HasSuffix(s, "__"), nil

119 +}

120 +

121 +func (c Config) Hash() uint64 {

122 + hash, _ := hashstructure.Hash(c, nil)

123 + return hash

124 +}

125 +

126 +func (c Config) Validate() error {

127 + if c == nil {

128 + return fmt.Errorf("store config is nil")

129 + }

130 +

131 + name := c.Name()

132 + if name == "" {

133 + return fmt.Errorf("store name is required")

134 + }

135 + if err := validateStoreName(name); err != nil {

136 + return fmt.Errorf("invalid store name '%s': %w", name, err)

137 + }

138 +

139 + kind := c.Kind()

140 + if !kind.IsValid() {

141 + return fmt.Errorf("invalid store kind '%s'", kind)

142 + }

143 +

144 + if c.Source() == "" {

145 + return fmt.Errorf("store source is required")

146 + }

147 +

148 + switch c.SourceType() {

149 + case confgroup.TypeDyncfg, confgroup.TypeUser, confgroup.TypeStock:

150 + return nil

151 + default:

152 + return fmt.Errorf("invalid store source type '%s'", c.SourceType())

153 + }

154 +}

src/go/plugin/agent/secrets/secretstore/raw_config_test.go new

+175

@@ -0,0 +1,175 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "testing"

7 +

8 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

9 + "github.com/stretchr/testify/assert"

10 + "github.com/stretchr/testify/require"

11 + "gopkg.in/yaml.v2"

12 +)

13 +

14 +func TestConfigValidateAndIdentity(t *testing.T) {

15 + cfg := Config{

16 + "name": "prod",

17 + "kind": string(KindVault),

18 + "__source__": "/etc/netdata/secretstores.yaml",

19 + "__source_type__": confgroup.TypeUser,

20 + "auth": map[string]any{

21 + "mode": "token",

22 + },

23 + }

24 +

25 + require.NoError(t, cfg.Validate())

26 + assert.Equal(t, "prod", cfg.Name())

27 + assert.Equal(t, KindVault, cfg.Kind())

28 + assert.Equal(t, "vault:prod", cfg.ExposedKey())

29 + assert.Equal(t, "/etc/netdata/secretstores.yaml:vault:prod", cfg.UID())

30 +}

31 +

32 +func TestConfigValidateRejectsInvalidStructuralFields(t *testing.T) {

33 + tests := map[string]struct {

34 + cfg Config

35 + want string

36 + }{

37 + "nil": {

38 + cfg: nil,

39 + want: "store config is nil",

40 + },

41 + "missing name": {

42 + cfg: Config{

43 + "kind": string(KindVault),

44 + "__source__": "src",

45 + "__source_type__": confgroup.TypeDyncfg,

46 + },

47 + want: "store name is required",

48 + },

49 + "invalid name": {

50 + cfg: Config{

51 + "name": "prod:west",

52 + "kind": string(KindVault),

53 + "__source__": "src",

54 + "__source_type__": confgroup.TypeDyncfg,

55 + },

56 + want: "invalid store name",

57 + },

58 + "invalid kind": {

59 + cfg: Config{

60 + "name": "prod",

61 + "kind": "wat",

62 + "__source__": "src",

63 + "__source_type__": confgroup.TypeDyncfg,

64 + },

65 + want: "invalid store kind",

66 + },

67 + "missing source": {

68 + cfg: Config{

69 + "name": "prod",

70 + "kind": string(KindVault),

71 + "__source_type__": confgroup.TypeDyncfg,

72 + },

73 + want: "store source is required",

74 + },

75 + "invalid source type": {

76 + cfg: Config{

77 + "name": "prod",

78 + "kind": string(KindVault),

79 + "__source__": "src",

80 + "__source_type__": "invalid",

81 + },

82 + want: "invalid store source type",

83 + },

84 + }

85 +

86 + for name, tc := range tests {

87 + t.Run(name, func(t *testing.T) {

88 + err := tc.cfg.Validate()

89 + require.Error(t, err)

90 + assert.ErrorContains(t, err, tc.want)

91 + })

92 + }

93 +}

94 +

95 +func TestConfigHashExcludesMetadataKeys(t *testing.T) {

96 + base := Config{

97 + "name": "prod",

98 + "kind": string(KindVault),

99 + "auth": map[string]any{

100 + "mode": "token",

101 + "mode_token": map[string]any{

102 + "token": "vault-token",

103 + },

104 + },

105 + "__source__": "user.yaml",

106 + "__source_type__": confgroup.TypeUser,

107 + }

108 +

109 + metaChanged := Config{

110 + "name": "prod",

111 + "kind": string(KindVault),

112 + "auth": map[string]any{

113 + "mode": "token",

114 + "mode_token": map[string]any{

115 + "token": "vault-token",

116 + },

117 + },

118 + "__source__": "dyncfg",

119 + "__source_type__": confgroup.TypeDyncfg,

120 + }

121 +

122 + payloadChanged := Config{

123 + "name": "prod",

124 + "kind": string(KindVault),

125 + "auth": map[string]any{

126 + "mode": "token",

127 + "mode_token": map[string]any{

128 + "token": "other-token",

129 + },

130 + },

131 + "__source__": "user.yaml",

132 + "__source_type__": confgroup.TypeUser,

133 + }

134 +

135 + assert.Equal(t, base.Hash(), metaChanged.Hash())

136 + assert.NotEqual(t, base.Hash(), payloadChanged.Hash())

137 +}

138 +

139 +func TestConfigHashMatchesAcrossEquivalentMapShapes(t *testing.T) {

140 + rawMap := Config{

141 + "name": "prod",

142 + "kind": string(KindVault),

143 + "__source__": "user.yaml",

144 + "__source_type__": confgroup.TypeUser,

145 + "auth": map[string]any{

146 + "mode": "token",

147 + "selectors": []any{

148 + "VAULT_TOKEN",

149 + map[string]any{"alt": "VAULT_TOKEN_ALT"},

150 + },

151 + },

152 + }

153 +

154 + var rawYAML Config

155 + require.NoError(t, yaml.Unmarshal([]byte(`

156 +name: prod

157 +kind: vault

158 +__source__: user.yaml

159 +__source_type__: user

160 +auth:

161 + mode: token

162 + selectors:

163 + - VAULT_TOKEN

164 + - alt: VAULT_TOKEN_ALT

165 +`), &rawYAML))

166 +

167 + assert.Equal(t, rawMap.Hash(), rawYAML.Hash())

168 +}

169 +

170 +func TestConfigSourceTypePriority(t *testing.T) {

171 + assert.Equal(t, 16, Config{"__source_type__": confgroup.TypeDyncfg}.SourceTypePriority())

172 + assert.Equal(t, 8, Config{"__source_type__": confgroup.TypeUser}.SourceTypePriority())

173 + assert.Equal(t, 2, Config{"__source_type__": confgroup.TypeStock}.SourceTypePriority())

174 + assert.Equal(t, 0, Config{"__source_type__": "other"}.SourceTypePriority())

175 +}

src/go/plugin/agent/secrets/secretstore/runtime_resolver.go new

+61

@@ -0,0 +1,61 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "strings"

9 +)

10 +

11 +// runtimeResolver resolves ${store:<kind>:<name>:<operand>} references against a captured snapshot.

12 +type runtimeResolver struct{}

13 +

14 +func newRuntimeResolver() *runtimeResolver { return &runtimeResolver{} }

15 +

16 +func (r *runtimeResolver) resolveContext(ctx context.Context, snapshot *Snapshot, ref, original string) (string, error) {

17 + if ctx == nil {

18 + ctx = context.Background()

19 + }

20 + select {

21 + case <-ctx.Done():

22 + return "", ctx.Err()

23 + default:

24 + }

25 +

26 + kindPart, rest, ok := strings.Cut(ref, ":")

27 + if !ok {

28 + return "", fmt.Errorf("resolving secret '%s': store reference must be in format 'kind:name:operand'", original)

29 + }

30 + namePart, operand, ok := strings.Cut(rest, ":")

31 + if !ok {

32 + return "", fmt.Errorf("resolving secret '%s': store reference must be in format 'kind:name:operand'", original)

33 + }

34 +

35 + kind := StoreKind(strings.TrimSpace(kindPart))

36 + name := strings.TrimSpace(namePart)

37 + operand = strings.TrimSpace(operand)

38 + if !kind.IsValid() || name == "" || operand == "" {

39 + return "", fmt.Errorf("resolving secret '%s': store reference must be in format 'kind:name:operand'", original)

40 + }

41 + storeKey := StoreKey(kind, name)

42 +

43 + if snapshot == nil {

44 + return "", fmt.Errorf("resolving secret '%s': secretstore '%s' is not configured", original, storeKey)

45 + }

46 + store, ok := snapshot.lookupStore(storeKey)

47 + if !ok {

48 + return "", fmt.Errorf("resolving secret '%s': secretstore '%s' is not configured", original, storeKey)

49 + }

50 + if store.published == nil {

51 + return "", fmt.Errorf("resolving secret '%s': secretstore '%s' has no published resolver state", original, storeKey)

52 + }

53 +

54 + return store.published.Resolve(ctx, ResolveRequest{

55 + StoreKey: storeKey,

56 + StoreKind: kind,

57 + StoreName: name,

58 + Operand: operand,

59 + Original: original,

60 + })

61 +}

src/go/plugin/agent/secrets/secretstore/runtime_resolver_provider_test.go new

+58

@@ -0,0 +1,58 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "context"

7 + "testing"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

11 + "github.com/stretchr/testify/assert"

12 + "github.com/stretchr/testify/require"

13 +)

14 +

15 +func TestProviderBackedRuntimeResolverOperandValidation(t *testing.T) {

16 + svc := secretstore.NewService(backends.Creators()...)

17 + for _, entry := range providerBackedConfigs() {

18 + err := svc.Add(newStoreFromConfig(t, svc, entry.kind, entry.config))

19 + require.NoError(t, err)

20 + }

21 +

22 + snapshot := svc.Capture()

23 +

24 + tests := map[string]struct {

25 + ref string

26 + original string

27 + wantErrContains string

28 + }{

29 + "aws": {

30 + ref: "aws-sm:aws_prod:#jsonKey",

31 + original: "${store:aws-sm:aws_prod:#jsonKey}",

32 + wantErrContains: "secret name is empty",

33 + },

34 + "azure": {

35 + ref: "azure-kv:azure_prod:not-a-secret-ref",

36 + original: "${store:azure-kv:azure_prod:not-a-secret-ref}",

37 + wantErrContains: "operand must be in format 'vault-name/secret-name'",

38 + },

39 + "gcp": {

40 + ref: "gcp-sm:gcp_prod:not-a-secret-ref",

41 + original: "${store:gcp-sm:gcp_prod:not-a-secret-ref}",

42 + wantErrContains: "operand must be in format 'project/secret' or 'project/secret/version'",

43 + },

44 + "vault": {

45 + ref: "vault:vault_prod:not-a-vault-ref",

46 + original: "${store:vault:vault_prod:not-a-vault-ref}",

47 + wantErrContains: "operand must be in format 'path#key'",

48 + },

49 + }

50 +

51 + for name, tc := range tests {

52 + t.Run(name, func(t *testing.T) {

53 + _, err := svc.Resolve(context.Background(), snapshot, tc.ref, tc.original)

54 + require.Error(t, err)

55 + assert.Contains(t, err.Error(), tc.wantErrContains)

56 + })

57 + }

58 +}

src/go/plugin/agent/secrets/secretstore/runtime_resolver_test.go new

+52

@@ -0,0 +1,52 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "testing"

7 +

8 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

10 + "github.com/stretchr/testify/assert"

11 + "github.com/stretchr/testify/require"

12 +)

13 +

14 +func TestRuntimeResolverResolveErrors(t *testing.T) {

15 + enabledSvc := secretstore.NewService(backends.Creators()...)

16 + err := enabledSvc.Add(newStoreFromConfig(t, enabledSvc, secretstore.KindVault, testSingleVaultConfig()))

17 + require.NoError(t, err)

18 +

19 + tests := map[string]struct {

20 + snapshot *secretstore.Snapshot

21 + ref string

22 + original string

23 + wantErrContains string

24 + }{

25 + "invalid store ref format": {

26 + snapshot: enabledSvc.Capture(),

27 + ref: "invalid",

28 + original: "${store:invalid}",

29 + wantErrContains: "store reference must be in format",

30 + },

31 + "store not configured": {

32 + snapshot: secretstore.NewService(backends.Creators()...).Capture(),

33 + ref: "vault:missing:secret/data/app#password",

34 + original: "${store:vault:missing:secret/data/app#password}",

35 + wantErrContains: "secretstore 'vault:missing' is not configured",

36 + },

37 + "bad vault operand": {

38 + snapshot: enabledSvc.Capture(),

39 + ref: "vault:vault_prod:secret/data/app",

40 + original: "${store:vault:vault_prod:secret/data/app}",

41 + wantErrContains: "operand must be in format 'path#key'",

42 + },

43 + }

44 +

45 + for name, tc := range tests {

46 + t.Run(name, func(t *testing.T) {

47 + _, err := enabledSvc.Resolve(t.Context(), tc.snapshot, tc.ref, tc.original)

48 + require.Error(t, err)

49 + assert.Contains(t, err.Error(), tc.wantErrContains)

50 + })

51 + }

52 +}

src/go/plugin/agent/secrets/secretstore/service.go new

+40

@@ -0,0 +1,40 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import "context"

6 +

7 +type Service interface {

8 + Capture() *Snapshot

9 + Resolve(ctx context.Context, snapshot *Snapshot, ref, original string) (string, error)

10 +

11 + Kinds() []StoreKind

12 + DisplayName(kind StoreKind) (string, bool)

13 + Schema(kind StoreKind) (string, bool)

14 + New(kind StoreKind) (Store, bool)

15 +

16 + GetStatus(key string) (StoreStatus, bool)

17 +

18 + Validate(cfg Config) error

19 + ValidateStored(key string) error

20 + Add(cfg Config) error

21 + Update(key string, cfg Config) error

22 + Remove(key string) error

23 +}

24 +

25 +type Creator struct {

26 + Kind StoreKind

27 + DisplayName string

28 + Schema string

29 + Create func() Store

30 +}

31 +

32 +type Store interface {

33 + Configuration() any

34 + Init(context.Context) error

35 + Publish() PublishedStore

36 +}

37 +

38 +type PublishedStore interface {

39 + Resolve(ctx context.Context, req ResolveRequest) (string, error)

40 +}

src/go/plugin/agent/secrets/secretstore/service_hardening_test.go new

+354

@@ -0,0 +1,354 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "context"

7 + "encoding/json"

8 + "errors"

9 + "sync"

10 + "sync/atomic"

11 + "testing"

12 +

13 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

14 + "github.com/stretchr/testify/assert"

15 + "github.com/stretchr/testify/require"

16 +)

17 +

18 +type fakePublished struct {

19 + blockOnCtx *atomic.Bool

20 + requireNonNilCtx *atomic.Bool

21 +}

22 +

23 +func (p *fakePublished) Resolve(ctx context.Context, req secretstore.ResolveRequest) (string, error) {

24 + if p.requireNonNilCtx != nil {

25 + p.requireNonNilCtx.Store(ctx != nil)

26 + if ctx == nil {

27 + return "", errors.New("nil context")

28 + }

29 + }

30 + if p.blockOnCtx != nil && p.blockOnCtx.Load() {

31 + <-ctx.Done()

32 + return "", ctx.Err()

33 + }

34 + return req.Operand, nil

35 +}

36 +

37 +type fakeConfig struct {

38 + Auth map[string]any `json:"auth,omitempty" yaml:"auth,omitempty"`

39 +}

40 +

41 +type fakeStore struct {

42 + cfg fakeConfig

43 + failInit *atomic.Bool

44 + blockOnCtx *atomic.Bool

45 + requireNonNilCtx *atomic.Bool

46 + published secretstore.PublishedStore

47 +}

48 +

49 +func (s *fakeStore) Configuration() any { return &s.cfg }

50 +func (s *fakeStore) Publish() secretstore.PublishedStore {

51 + return s.published

52 +}

53 +

54 +func (s *fakeStore) Init(context.Context) error {

55 + if s.failInit != nil && s.failInit.Load() {

56 + return errors.New("simulated validation error")

57 + }

58 + if len(s.cfg.Auth) == 0 {

59 + return errors.New("auth is required")

60 + }

61 + s.published = &fakePublished{

62 + blockOnCtx: s.blockOnCtx,

63 + requireNonNilCtx: s.requireNonNilCtx,

64 + }

65 + return nil

66 +}

67 +

68 +type validateRaceStore struct {

69 + cfg fakeConfig

70 + initCount *atomic.Int32

71 + validateStarted chan struct{}

72 + validateRelease <-chan struct{}

73 + published secretstore.PublishedStore

74 +}

75 +

76 +func (s *validateRaceStore) Configuration() any { return &s.cfg }

77 +func (s *validateRaceStore) Publish() secretstore.PublishedStore {

78 + return s.published

79 +}

80 +

81 +func (s *validateRaceStore) Init(context.Context) error {

82 + if s.initCount != nil && s.initCount.Add(1) == 2 {

83 + close(s.validateStarted)

84 + <-s.validateRelease

85 + }

86 + if len(s.cfg.Auth) == 0 {

87 + return errors.New("auth is required")

88 + }

89 + s.published = &fakePublished{}

90 + return nil

91 +}

92 +

93 +func newFakeCreator(kind secretstore.StoreKind, failInit, blockOnCtx *atomic.Bool) secretstore.Creator {

94 + return newFakeCreatorWithCtxProbe(kind, failInit, blockOnCtx, nil)

95 +}

96 +

97 +func newFakeCreatorWithCtxProbe(kind secretstore.StoreKind, failInit, blockOnCtx, requireNonNilCtx *atomic.Bool) secretstore.Creator {

98 + schema := map[string]any{

99 + "jsonSchema": map[string]any{

100 + "type": "object",

101 + "properties": map[string]any{

102 + "auth": map[string]any{"type": "object"},

103 + },

104 + "required": []any{"auth"},

105 + },

106 + "uiSchema": map[string]any{},

107 + }

108 + bs, err := json.Marshal(schema)

109 + if err != nil {

110 + panic(err)

111 + }

112 +

113 + return secretstore.Creator{

114 + Kind: kind,

115 + DisplayName: "Fake Provider",

116 + Schema: string(bs),

117 + Create: func() secretstore.Store {

118 + return &fakeStore{

119 + failInit: failInit,

120 + blockOnCtx: blockOnCtx,

121 + requireNonNilCtx: requireNonNilCtx,

122 + }

123 + },

124 + }

125 +}

126 +

127 +func newValidateRaceCreator(kind secretstore.StoreKind, validateStarted chan struct{}, validateRelease <-chan struct{}) secretstore.Creator {

128 + schema := map[string]any{

129 + "jsonSchema": map[string]any{

130 + "type": "object",

131 + "properties": map[string]any{

132 + "auth": map[string]any{"type": "object"},

133 + },

134 + "required": []any{"auth"},

135 + },

136 + "uiSchema": map[string]any{},

137 + }

138 + bs, err := json.Marshal(schema)

139 + if err != nil {

140 + panic(err)

141 + }

142 +

143 + var initCount atomic.Int32

144 + return secretstore.Creator{

145 + Kind: kind,

146 + DisplayName: "Fake Provider",

147 + Schema: string(bs),

148 + Create: func() secretstore.Store {

149 + return &validateRaceStore{

150 + initCount: &initCount,

151 + validateStarted: validateStarted,

152 + validateRelease: validateRelease,

153 + }

154 + },

155 + }

156 +}

157 +

158 +func newFakeStore(_ *testing.T, _ secretstore.Service, kind secretstore.StoreKind, cfg fakeConfig, name string) secretstore.Config {

159 + bs, err := json.Marshal(cfg)

160 + if err != nil {

161 + panic(err)

162 + }

163 + var payload map[string]any

164 + if err := json.Unmarshal(bs, &payload); err != nil {

165 + panic(err)

166 + }

167 + out := secretstore.Config(payload)

168 + out.SetName(name)

169 + out.SetKind(kind)

170 + out.SetSource("dyncfg")

171 + out.SetSourceType("dyncfg")

172 + return out

173 +}

174 +

175 +func TestServiceStatusLifecycle(t *testing.T) {

176 + var failInit atomic.Bool

177 + svc := secretstore.NewService(newFakeCreator(secretstore.KindVault, &failInit, nil))

178 +

179 + store := newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

180 + Auth: map[string]any{"mode": "token_env"},

181 + }, "vault_prod")

182 +

183 + err := svc.Add(store)

184 + require.NoError(t, err)

185 +

186 + failInit.Store(true)

187 + storeKey := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

188 + err = svc.ValidateStored(storeKey)

189 + require.Error(t, err)

190 +

191 + status, ok := svc.GetStatus(storeKey)

192 + require.True(t, ok)

193 + require.NotNil(t, status.LastValidation)

194 + assert.False(t, status.LastValidation.OK)

195 + assert.Equal(t, "simulated validation error", status.LastErrorSummary)

196 +

197 + failInit.Store(false)

198 + err = svc.ValidateStored(storeKey)

199 + require.NoError(t, err)

200 +

201 + status, ok = svc.GetStatus(storeKey)

202 + require.True(t, ok)

203 + require.NotNil(t, status.LastValidation)

204 + assert.True(t, status.LastValidation.OK)

205 + assert.Empty(t, status.LastErrorSummary)

206 +}

207 +

208 +func TestServiceResolveHonorsCanceledContext(t *testing.T) {

209 + var blockOnCtx atomic.Bool

210 + blockOnCtx.Store(true)

211 +

212 + svc := secretstore.NewService(newFakeCreator(secretstore.KindVault, nil, &blockOnCtx))

213 + store := newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

214 + Auth: map[string]any{"mode": "token_env"},

215 + }, "vault_prod")

216 + err := svc.Add(store)

217 + require.NoError(t, err)

218 +

219 + ctx, cancel := context.WithCancel(context.Background())

220 + cancel()

221 +

222 + _, err = svc.Resolve(ctx, svc.Capture(), "vault:vault_prod:secret", "${store:vault:vault_prod:secret}")

223 + require.ErrorIs(t, err, context.Canceled)

224 +}

225 +

226 +func TestServiceResolve_NormalizesNilContext(t *testing.T) {

227 + var requireNonNilCtx atomic.Bool

228 +

229 + svc := secretstore.NewService(newFakeCreatorWithCtxProbe(secretstore.KindVault, nil, nil, &requireNonNilCtx))

230 + store := newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

231 + Auth: map[string]any{"mode": "token_env"},

232 + }, "vault_prod")

233 + err := svc.Add(store)

234 + require.NoError(t, err)

235 +

236 + val, err := svc.Resolve(nil, svc.Capture(), "vault:vault_prod:secret/data/app#key", "${store:vault:vault_prod:secret/data/app#key}")

237 + require.NoError(t, err)

238 + assert.Equal(t, "secret/data/app#key", val)

239 + assert.True(t, requireNonNilCtx.Load())

240 +}

241 +

242 +func TestServiceConcurrentResolveAndMutation(t *testing.T) {

243 + svc := secretstore.NewService(newFakeCreator(secretstore.KindVault, nil, nil))

244 + baseCfg := fakeConfig{

245 + Auth: map[string]any{"mode": "token_env"},

246 + }

247 +

248 + err := svc.Add(newFakeStore(t, svc, secretstore.KindVault, baseCfg, "vault_prod"))

249 + require.NoError(t, err)

250 +

251 + var wg sync.WaitGroup

252 + errCh := make(chan error, 32)

253 +

254 + wg.Add(1)

255 + go func() {

256 + defer wg.Done()

257 + for i := 0; i < 100; i++ {

258 + snapshot := svc.Capture()

259 + val, err := svc.Resolve(context.Background(), snapshot, "vault:vault_prod:secret/data/app#key", "${store:vault:vault_prod:secret/data/app#key}")

260 + if err != nil {

261 + errCh <- err

262 + return

263 + }

264 + if val != "secret/data/app#key" {

265 + errCh <- errors.New("unexpected resolved value")

266 + return

267 + }

268 + }

269 + }()

270 +

271 + wg.Add(1)

272 + go func() {

273 + defer wg.Done()

274 + for i := 0; i < 100; i++ {

275 + updateCfg := baseCfg

276 + if i%2 == 0 {

277 + updateCfg.Auth = map[string]any{

278 + "mode": "token_env",

279 + "tag": "alt",

280 + }

281 + }

282 + if err := svc.Update(secretstore.StoreKey(secretstore.KindVault, "vault_prod"), newFakeStore(t, svc, secretstore.KindVault, updateCfg, "vault_prod")); err != nil {

283 + errCh <- err

284 + return

285 + }

286 + }

287 + }()

288 +

289 + wg.Wait()

290 + close(errCh)

291 +

292 + for err := range errCh {

293 + require.NoError(t, err)

294 + }

295 +}

296 +

297 +func TestServiceValidateStored_RemovedDuringValidationReturnsNotFound(t *testing.T) {

298 + validateStarted := make(chan struct{})

299 + validateRelease := make(chan struct{})

300 +

301 + svc := secretstore.NewService(newValidateRaceCreator(secretstore.KindVault, validateStarted, validateRelease))

302 + store := newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

303 + Auth: map[string]any{"mode": "token_env"},

304 + }, "vault_prod")

305 + require.NoError(t, svc.Add(store))

306 +

307 + storeKey := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

308 + errCh := make(chan error, 1)

309 +

310 + go func() {

311 + errCh <- svc.ValidateStored(storeKey)

312 + }()

313 +

314 + <-validateStarted

315 + require.NoError(t, svc.Remove(storeKey))

316 + close(validateRelease)

317 +

318 + err := <-errCh

319 + require.Error(t, err)

320 + assert.ErrorIs(t, err, secretstore.ErrStoreNotFound)

321 +}

322 +

323 +func TestServiceValidateStored_UpdatedDuringValidationReturnsRetryWithoutOverwritingStatus(t *testing.T) {

324 + validateStarted := make(chan struct{})

325 + validateRelease := make(chan struct{})

326 +

327 + svc := secretstore.NewService(newValidateRaceCreator(secretstore.KindVault, validateStarted, validateRelease))

328 + store := newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

329 + Auth: map[string]any{"mode": "token_env"},

330 + }, "vault_prod")

331 + require.NoError(t, svc.Add(store))

332 +

333 + storeKey := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

334 + errCh := make(chan error, 1)

335 +

336 + go func() {

337 + errCh <- svc.ValidateStored(storeKey)

338 + }()

339 +

340 + <-validateStarted

341 + require.NoError(t, svc.Update(storeKey, newFakeStore(t, svc, secretstore.KindVault, fakeConfig{

342 + Auth: map[string]any{"mode": "token_env", "tag": "new"},

343 + }, "vault_prod")))

344 + close(validateRelease)

345 +

346 + err := <-errCh

347 + require.Error(t, err)

348 + assert.Contains(t, err.Error(), "changed during validation")

349 +

350 + status, ok := svc.GetStatus(storeKey)

351 + require.True(t, ok)

352 + assert.Nil(t, status.LastValidation)

353 + assert.Empty(t, status.LastErrorSummary)

354 +}

src/go/plugin/agent/secrets/secretstore/service_impl.go new

+436

@@ -0,0 +1,436 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "context"

7 + "fmt"

8 + "sort"

9 + "sync"

10 + "sync/atomic"

11 + "time"

12 +

13 + "gopkg.in/yaml.v2"

14 +)

15 +

16 +type storeRecord struct {

17 + rawConfig Config

18 + configHash uint64

19 + status StoreStatus

20 + published PublishedStore

21 +}

22 +

23 +type preparedStore struct {

24 + key string

25 + rawConfig Config

26 + configHash uint64

27 + status StoreStatus

28 + published PublishedStore

29 +}

30 +

31 +type serviceState struct {

32 + snapshot *Snapshot

33 + records map[string]storeRecord

34 +}

35 +

36 +type creatorRegistry struct {

37 + kinds []StoreKind

38 + byKind map[StoreKind]Creator

39 +}

40 +

41 +type inMemoryService struct {

42 + mu sync.Mutex

43 + state atomic.Pointer[serviceState]

44 + now func() time.Time

45 + resolver *runtimeResolver

46 + registry creatorRegistry

47 +}

48 +

49 +func NewService(creators ...Creator) Service {

50 + return newInMemoryService(creators...)

51 +}

52 +

53 +func newInMemoryService(creators ...Creator) Service {

54 + s := &inMemoryService{

55 + now: time.Now,

56 + resolver: newRuntimeResolver(),

57 + registry: newCreatorRegistry(creators...),

58 + }

59 + s.state.Store(&serviceState{

60 + snapshot: &Snapshot{

61 + generation: 0,

62 + publishedAt: s.now().UTC(),

63 + stores: map[string]publishedRecord{},

64 + },

65 + records: map[string]storeRecord{},

66 + })

67 + return s

68 +}

69 +

70 +func (s *inMemoryService) Capture() *Snapshot {

71 + state := s.state.Load()

72 + if state == nil {

73 + return &Snapshot{stores: map[string]publishedRecord{}}

74 + }

75 + return cloneSnapshot(state.snapshot)

76 +}

77 +

78 +func (s *inMemoryService) Resolve(ctx context.Context, snapshot *Snapshot, ref, original string) (string, error) {

79 + if ctx != nil {

80 + select {

81 + case <-ctx.Done():

82 + return "", ctx.Err()

83 + default:

84 + }

85 + }

86 + return s.resolver.resolveContext(ctx, snapshot, ref, original)

87 +}

88 +

89 +func (s *inMemoryService) Kinds() []StoreKind {

90 + return append([]StoreKind(nil), s.registry.kinds...)

91 +}

92 +

93 +func (s *inMemoryService) DisplayName(kind StoreKind) (string, bool) {

94 + creator, ok := s.registry.byKind[kind]

95 + if !ok {

96 + return "", false

97 + }

98 + return creator.DisplayName, true

99 +}

100 +

101 +func (s *inMemoryService) Schema(kind StoreKind) (string, bool) {

102 + creator, ok := s.registry.byKind[kind]

103 + if !ok {

104 + return "", false

105 + }

106 + return creator.Schema, true

107 +}

108 +

109 +func (s *inMemoryService) New(kind StoreKind) (Store, bool) {

110 + creator, ok := s.registry.byKind[kind]

111 + if !ok || creator.Create == nil {

112 + return nil, false

113 + }

114 + store := creator.Create()

115 + if store == nil {

116 + return nil, false

117 + }

118 + return store, true

119 +}

120 +

121 +func (s *inMemoryService) GetStatus(key string) (StoreStatus, bool) {

122 + key, err := normalizeStoreKey(key)

123 + if err != nil {

124 + return StoreStatus{}, false

125 + }

126 +

127 + state := s.state.Load()

128 + if state == nil {

129 + return StoreStatus{}, false

130 + }

131 + record, ok := state.records[key]

132 + if !ok {

133 + return StoreStatus{}, false

134 + }

135 + return cloneStoreStatus(record.status), true

136 +}

137 +

138 +func (s *inMemoryService) Validate(cfg Config) error {

139 + _, err := s.prepareConfig(context.Background(), cfg)

140 + return err

141 +}

142 +

143 +func (s *inMemoryService) ValidateStored(key string) error {

144 + key, err := normalizeStoreKey(key)

145 + if err != nil {

146 + return err

147 + }

148 +

149 + state := s.state.Load()

150 + if state == nil {

151 + return storeNotConfiguredError(key)

152 + }

153 + record, ok := state.records[key]

154 + if !ok {

155 + return storeNotConfiguredError(key)

156 + }

157 + validatedHash := record.configHash

158 +

159 + _, err = s.prepareConfig(context.Background(), record.rawConfig)

160 +

161 + validation := &ValidationStatus{

162 + CheckedAt: s.now().UTC(),

163 + OK: err == nil,

164 + }

165 +

166 + s.mu.Lock()

167 + defer s.mu.Unlock()

168 +

169 + current := s.state.Load()

170 + if current == nil {

171 + return storeNotConfiguredError(key)

172 + }

173 + updated, ok := current.records[key]

174 + if !ok {

175 + return storeNotConfiguredError(key)

176 + }

177 + if updated.configHash != validatedHash {

178 + return fmt.Errorf("store '%s' changed during validation; retry", key)

179 + }

180 + updated.status.LastValidation = validation

181 + if err != nil {

182 + updated.status.LastErrorSummary = err.Error()

183 + } else {

184 + updated.status.LastErrorSummary = ""

185 + }

186 +

187 + records := cloneRecords(current.records)

188 + records[key] = updated

189 + s.state.Store(&serviceState{

190 + snapshot: current.snapshot,

191 + records: records,

192 + })

193 +

194 + return err

195 +}

196 +

197 +func (s *inMemoryService) Add(cfg Config) error {

198 + prepared, err := s.prepareConfig(context.Background(), cfg)

199 + if err != nil {

200 + return err

201 + }

202 +

203 + s.mu.Lock()

204 + defer s.mu.Unlock()

205 +

206 + state := s.state.Load()

207 + if state == nil {

208 + state = &serviceState{

209 + snapshot: &Snapshot{stores: map[string]publishedRecord{}},

210 + records: map[string]storeRecord{},

211 + }

212 + }

213 + if _, ok := state.records[prepared.key]; ok {

214 + return storeAlreadyExistsError(prepared.key)

215 + }

216 +

217 + records := cloneRecords(state.records)

218 + records[prepared.key] = prepared.record()

219 + snapshot := newSnapshot(state.snapshot.Generation()+1, s.now().UTC(), records)

220 + s.state.Store(&serviceState{snapshot: snapshot, records: records})

221 + return nil

222 +}

223 +

224 +func (s *inMemoryService) Update(key string, cfg Config) error {

225 + key, err := normalizeStoreKey(key)

226 + if err != nil {

227 + return err

228 + }

229 +

230 + state := s.state.Load()

231 + if state == nil {

232 + return storeNotConfiguredError(key)

233 + }

234 + before, ok := state.records[key]

235 + if !ok {

236 + return storeNotConfiguredError(key)

237 + }

238 +

239 + prepared, err := s.prepareConfig(context.Background(), cfg)

240 + if err != nil {

241 + return err

242 + }

243 + if prepared.key != key {

244 + return fmt.Errorf("store key mismatch: path key '%s' differs from config key '%s'", key, prepared.key)

245 + }

246 +

247 + s.mu.Lock()

248 + defer s.mu.Unlock()

249 +

250 + current := s.state.Load()

251 + if current == nil {

252 + return storeNotConfiguredError(key)

253 + }

254 + before, ok = current.records[key]

255 + if !ok {

256 + return storeNotConfiguredError(key)

257 + }

258 +

259 + if before.configHash == prepared.configHash {

260 + return nil

261 + }

262 +

263 + records := cloneRecords(current.records)

264 + records[key] = prepared.record()

265 + snapshot := newSnapshot(current.snapshot.Generation()+1, s.now().UTC(), records)

266 + s.state.Store(&serviceState{snapshot: snapshot, records: records})

267 + return nil

268 +}

269 +

270 +func (s *inMemoryService) Remove(key string) error {

271 + key, err := normalizeStoreKey(key)

272 + if err != nil {

273 + return err

274 + }

275 +

276 + s.mu.Lock()

277 + defer s.mu.Unlock()

278 +

279 + state := s.state.Load()

280 + if state == nil {

281 + return storeNotConfiguredError(key)

282 + }

283 + if _, ok := state.records[key]; !ok {

284 + return storeNotConfiguredError(key)

285 + }

286 +

287 + records := cloneRecords(state.records)

288 + delete(records, key)

289 + snapshot := newSnapshot(state.snapshot.Generation()+1, s.now().UTC(), records)

290 + s.state.Store(&serviceState{snapshot: snapshot, records: records})

291 + return nil

292 +}

293 +

294 +func newCreatorRegistry(creators ...Creator) creatorRegistry {

295 + reg := creatorRegistry{

296 + byKind: make(map[StoreKind]Creator, len(creators)),

297 + }

298 + for _, creator := range creators {

299 + if creator.Kind == "" || creator.Create == nil {

300 + continue

301 + }

302 + reg.byKind[creator.Kind] = creator

303 + }

304 + reg.kinds = make([]StoreKind, 0, len(reg.byKind))

305 + for kind := range reg.byKind {

306 + reg.kinds = append(reg.kinds, kind)

307 + }

308 + sort.Slice(reg.kinds, func(i, j int) bool { return reg.kinds[i] < reg.kinds[j] })

309 + return reg

310 +}

311 +

312 +func (s *inMemoryService) prepareConfig(ctx context.Context, cfg Config) (preparedStore, error) {

313 + if cfg == nil {

314 + return preparedStore{}, fmt.Errorf("store config is nil")

315 + }

316 + if ctx == nil {

317 + ctx = context.Background()

318 + }

319 +

320 + raw := cloneConfig(cfg)

321 + if raw == nil {

322 + return preparedStore{}, fmt.Errorf("store config is nil")

323 + }

324 + if err := raw.Validate(); err != nil {

325 + return preparedStore{}, err

326 + }

327 +

328 + kind := raw.Kind()

329 + name := raw.Name()

330 + key := raw.ExposedKey()

331 +

332 + store, ok := s.New(kind)

333 + if !ok {

334 + return preparedStore{}, fmt.Errorf("store kind '%s' is not supported", kind)

335 + }

336 + if store.Configuration() == nil {

337 + return preparedStore{}, fmt.Errorf("store '%s': configuration is nil", key)

338 + }

339 +

340 + bs, err := yaml.Marshal(raw)

341 + if err != nil {

342 + return preparedStore{}, fmt.Errorf("store '%s': marshaling raw config: %w", key, err)

343 + }

344 + if err := yaml.Unmarshal(bs, store.Configuration()); err != nil {

345 + return preparedStore{}, fmt.Errorf("store '%s': invalid provider payload: %w", key, err)

346 + }

347 +

348 + if err := store.Init(ctx); err != nil {

349 + return preparedStore{}, err

350 + }

351 +

352 + published := store.Publish()

353 + if published == nil {

354 + return preparedStore{}, fmt.Errorf("store '%s': published resolver state is nil", key)

355 + }

356 +

357 + return preparedStore{

358 + key: key,

359 + rawConfig: raw,

360 + configHash: raw.Hash(),

361 + status: StoreStatus{

362 + Name: name,

363 + Kind: kind,

364 + },

365 + published: published,

366 + }, nil

367 +}

368 +

369 +func newSnapshot(generation uint64, publishedAt time.Time, records map[string]storeRecord) *Snapshot {

370 + stores := make(map[string]publishedRecord, len(records))

371 + for key, record := range records {

372 + stores[key] = publishedRecord{

373 + published: record.published,

374 + }

375 + }

376 + return &Snapshot{

377 + generation: generation,

378 + publishedAt: publishedAt,

379 + stores: stores,

380 + }

381 +}

382 +

383 +func cloneRecords(in map[string]storeRecord) map[string]storeRecord {

384 + if len(in) == 0 {

385 + return map[string]storeRecord{}

386 + }

387 + out := make(map[string]storeRecord, len(in))

388 + for key, record := range in {

389 + out[key] = storeRecord{

390 + rawConfig: cloneConfig(record.rawConfig),

391 + configHash: record.configHash,

392 + status: cloneStoreStatus(record.status),

393 + published: record.published,

394 + }

395 + }

396 + return out

397 +}

398 +

399 +func normalizeStoreKey(key string) (string, error) {

400 + kind, name, err := ParseStoreKey(key)

401 + if err != nil {

402 + return "", err

403 + }

404 + return StoreKey(kind, name), nil

405 +}

406 +

407 +func (p preparedStore) record() storeRecord {

408 + return storeRecord{

409 + rawConfig: cloneConfig(p.rawConfig),

410 + configHash: p.configHash,

411 + status: cloneStoreStatus(p.status),

412 + published: p.published,

413 + }

414 +}

415 +

416 +type wrappedStoreError struct {

417 + msg string

418 + err error

419 +}

420 +

421 +func (e wrappedStoreError) Error() string { return e.msg }

422 +func (e wrappedStoreError) Unwrap() error { return e.err }

423 +

424 +func storeAlreadyExistsError(key string) error {

425 + return wrappedStoreError{

426 + msg: fmt.Sprintf("store '%s' already exists", key),

427 + err: ErrStoreExists,

428 + }

429 +}

430 +

431 +func storeNotConfiguredError(key string) error {

432 + return wrappedStoreError{

433 + msg: fmt.Sprintf("store '%s' is not configured", key),

434 + err: ErrStoreNotFound,

435 + }

436 +}

src/go/plugin/agent/secrets/secretstore/service_impl_test.go new

+182

@@ -0,0 +1,182 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "errors"

7 + "testing"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore/backends"

11 + "github.com/stretchr/testify/assert"

12 + "github.com/stretchr/testify/require"

13 + "gopkg.in/yaml.v2"

14 +)

15 +

16 +func TestServiceRegistryMetadata(t *testing.T) {

17 + svc := secretstore.NewService(backends.Creators()...)

18 +

19 + assert.Equal(t, []secretstore.StoreKind{secretstore.KindAWSSM, secretstore.KindAzureKV, secretstore.KindGCPSM, secretstore.KindVault}, svc.Kinds())

20 +

21 + name, ok := svc.DisplayName(secretstore.KindVault)

22 + require.True(t, ok)

23 + assert.Equal(t, "Vault", name)

24 +

25 + schema, ok := svc.Schema(secretstore.KindVault)

26 + require.True(t, ok)

27 + schemaObject := decodeSchema(t, schema)

28 + jsonSchema, ok := schemaObject["jsonSchema"].(map[string]any)

29 + require.True(t, ok)

30 + _, ok = jsonSchema["properties"].(map[string]any)["kind"]

31 + assert.False(t, ok)

32 + _, ok = schemaObject["uiSchema"].(map[string]any)

33 + assert.True(t, ok)

34 +}

35 +

36 +func TestServiceStatusAndGenerationLifecycle(t *testing.T) {

37 + svc := secretstore.NewService(backends.Creators()...)

38 +

39 + config := testSingleVaultConfig()

40 + err := svc.Add(newStoreFromConfig(t, svc, secretstore.KindVault, config))

41 + require.NoError(t, err)

42 + assert.Equal(t, uint64(1), svc.Capture().Generation())

43 +

44 + storeKey := secretstore.StoreKey(secretstore.KindVault, "vault_prod")

45 + status, ok := svc.GetStatus(storeKey)

46 + require.True(t, ok)

47 + assert.Equal(t, "vault_prod", status.Name)

48 + assert.Equal(t, secretstore.KindVault, status.Kind)

49 + assert.Nil(t, status.LastValidation)

50 +

51 + runtimeUpdate := testSingleVaultConfig()

52 + runtimeUpdate["mode"] = "token_file"

53 + runtimeUpdate["mode_token"] = nil

54 + runtimeUpdate["mode_token_file"] = map[string]any{

55 + "path": "/var/lib/netdata/vault.token",

56 + }

57 + err = svc.Update(storeKey, newStoreFromConfig(t, svc, secretstore.KindVault, runtimeUpdate))

58 + require.NoError(t, err)

59 + assert.Equal(t, uint64(2), svc.Capture().Generation())

60 +

61 + err = svc.Update(storeKey, newStoreFromConfig(t, svc, secretstore.KindVault, runtimeUpdate))

62 + require.NoError(t, err)

63 + assert.Equal(t, uint64(2), svc.Capture().Generation())

64 +

65 + err = svc.ValidateStored(storeKey)

66 + require.NoError(t, err)

67 +

68 + status, ok = svc.GetStatus(storeKey)

69 + require.True(t, ok)

70 + require.NotNil(t, status.LastValidation)

71 + assert.True(t, status.LastValidation.OK)

72 +}

73 +

74 +func TestServiceUpdate_UnknownFieldOnlyChangeCountsAsChange(t *testing.T) {

75 + svc := secretstore.NewService(backends.Creators()...)

76 +

77 + base := testSingleVaultConfig()

78 + err := svc.Add(newStoreFromConfig(t, svc, secretstore.KindVault, base))

79 + require.NoError(t, err)

80 + assert.Equal(t, uint64(1), svc.Capture().Generation())

81 +

82 + changed := testSingleVaultConfig()

83 + changed["ui_note"] = "kept"

84 + err = svc.Update(secretstore.StoreKey(secretstore.KindVault, "vault_prod"), newStoreFromConfig(t, svc, secretstore.KindVault, changed))

85 + require.NoError(t, err)

86 + assert.Equal(t, uint64(2), svc.Capture().Generation())

87 +}

88 +

89 +func TestServiceValidateAcceptsYAMLDecodedNestedMaps(t *testing.T) {

90 + svc := secretstore.NewService(backends.Creators()...)

91 +

92 + var cfg secretstore.Config

93 + require.NoError(t, yaml.Unmarshal([]byte(`

94 +mode: token

95 +mode_token:

96 + token: vault-token

97 +addr: https://vault.example

98 +`), &cfg))

99 + cfg.SetName("vault_prod")

100 + cfg.SetKind(secretstore.KindVault)

101 + cfg.SetSource("dyncfg")

102 + cfg.SetSourceType("dyncfg")

103 +

104 + require.NoError(t, svc.Validate(cfg))

105 +}

106 +

107 +func TestServiceUsesSentinelErrors(t *testing.T) {

108 + svc := secretstore.NewService(backends.Creators()...)

109 +

110 + err := svc.Add(newStoreFromConfig(t, svc, secretstore.KindVault, testSingleVaultConfig()))

111 + require.NoError(t, err)

112 +

113 + err = svc.Add(newStoreFromConfig(t, svc, secretstore.KindVault, testSingleVaultConfig()))

114 + require.Error(t, err)

115 + assert.ErrorIs(t, err, secretstore.ErrStoreExists)

116 +

117 + missing := testSingleVaultConfig()

118 + missing["name"] = "missing"

119 + err = svc.Update(secretstore.StoreKey(secretstore.KindVault, "missing"), newStoreFromConfig(t, svc, secretstore.KindVault, missing))

120 + require.Error(t, err)

121 + assert.ErrorIs(t, err, secretstore.ErrStoreNotFound)

122 +

123 + err = svc.Remove(secretstore.StoreKey(secretstore.KindVault, "missing"))

124 + require.Error(t, err)

125 + assert.True(t, errors.Is(err, secretstore.ErrStoreNotFound))

126 +}

127 +

128 +func TestProviderBackedValidationContracts(t *testing.T) {

129 + svc := secretstore.NewService(backends.Creators()...)

130 +

131 + err := svc.Validate(newStoreFromConfig(t, svc, secretstore.KindAWSSM, map[string]any{

132 + "name": "aws_prod",

133 + "auth_mode": "env",

134 + }))

135 + require.Error(t, err)

136 + assert.ErrorContains(t, err, "region is required")

137 +

138 + err = svc.Validate(newStoreFromConfig(t, svc, secretstore.KindVault, map[string]any{

139 + "name": "vault_prod",

140 + "mode": "token",

141 + "mode_token": map[string]any{

142 + "token": "vault-token",

143 + },

144 + }))

145 + require.Error(t, err)

146 + assert.ErrorContains(t, err, "addr is required")

147 +

148 + awsSchema, ok := svc.Schema(secretstore.KindAWSSM)

149 + require.True(t, ok)

150 + awsSchemaObject := decodeSchema(t, awsSchema)

151 + awsJSONSchema, ok := awsSchemaObject["jsonSchema"].(map[string]any)

152 + require.True(t, ok)

153 + assert.Contains(t, awsJSONSchema["required"], "auth_mode")

154 + assert.NotContains(t, awsJSONSchema["required"], "kind")

155 + assert.Contains(t, awsJSONSchema["required"], "region")

156 +

157 + vaultSchema, ok := svc.Schema(secretstore.KindVault)

158 + require.True(t, ok)

159 + vaultSchemaObject := decodeSchema(t, vaultSchema)

160 + vaultJSONSchema, ok := vaultSchemaObject["jsonSchema"].(map[string]any)

161 + require.True(t, ok)

162 + assert.Contains(t, vaultJSONSchema["required"], "addr")

163 +}

164 +

165 +func TestProviderBackedAddAcrossKinds(t *testing.T) {

166 + svc := secretstore.NewService(backends.Creators()...)

167 +

168 + for _, entry := range providerBackedConfigs() {

169 + entry := entry

170 + t.Run(string(entry.kind), func(t *testing.T) {

171 + err := svc.Add(newStoreFromConfig(t, svc, entry.kind, entry.config))

172 + require.NoError(t, err)

173 +

174 + status, ok := svc.GetStatus(secretstore.StoreKey(entry.kind, entry.name))

175 + require.True(t, ok)

176 + assert.Equal(t, entry.kind, status.Kind)

177 +

178 + err = svc.ValidateStored(secretstore.StoreKey(entry.kind, entry.name))

179 + require.NoError(t, err)

180 + })

181 + }

182 +}

src/go/plugin/agent/secrets/secretstore/snapshot.go new

+62

@@ -0,0 +1,62 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import "time"

6 +

7 +type publishedRecord struct {

8 + published PublishedStore

9 +}

10 +

11 +type Snapshot struct {

12 + generation uint64

13 + publishedAt time.Time

14 + stores map[string]publishedRecord

15 +}

16 +

17 +func (s *Snapshot) Generation() uint64 {

18 + if s == nil {

19 + return 0

20 + }

21 + return s.generation

22 +}

23 +

24 +func (s *Snapshot) PublishedAt() time.Time {

25 + if s == nil {

26 + return time.Time{}

27 + }

28 + return s.publishedAt

29 +}

30 +

31 +func (s *Snapshot) lookupStore(key string) (publishedRecord, bool) {

32 + if s == nil {

33 + return publishedRecord{}, false

34 + }

35 + store, ok := s.stores[key]

36 + if !ok {

37 + return publishedRecord{}, false

38 + }

39 + return store, true

40 +}

41 +

42 +func cloneSnapshot(s *Snapshot) *Snapshot {

43 + if s == nil {

44 + return &Snapshot{stores: map[string]publishedRecord{}}

45 + }

46 + return &Snapshot{

47 + generation: s.generation,

48 + publishedAt: s.publishedAt,

49 + stores: clonePublishedRecords(s.stores),

50 + }

51 +}

52 +

53 +func clonePublishedRecords(in map[string]publishedRecord) map[string]publishedRecord {

54 + if len(in) == 0 {

55 + return map[string]publishedRecord{}

56 + }

57 + out := make(map[string]publishedRecord, len(in))

58 + for id, store := range in {

59 + out[id] = store

60 + }

61 + return out

62 +}

src/go/plugin/agent/secrets/secretstore/snapshot_test.go new

+65

@@ -0,0 +1,65 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "context"

7 + "testing"

8 +

9 + "github.com/stretchr/testify/assert"

10 +)

11 +

12 +type testPublishedStore struct{}

13 +

14 +func (testPublishedStore) Resolve(_ context.Context, _ ResolveRequest) (string, error) {

15 + return "", nil

16 +}

17 +

18 +func TestSnapshotLookupStore(t *testing.T) {

19 + tests := map[string]struct {

20 + snapshot *Snapshot

21 + id string

22 + wantOK bool

23 + }{

24 + "nil snapshot": {

25 + snapshot: nil,

26 + id: "s1",

27 + wantOK: false,

28 + },

29 + "empty stores map": {

30 + snapshot: &Snapshot{stores: map[string]publishedRecord{}},

31 + id: "s1",

32 + wantOK: false,

33 + },

34 + "not found": {

35 + snapshot: &Snapshot{

36 + stores: map[string]publishedRecord{

37 + "s1": {},

38 + },

39 + },

40 + id: "s2",

41 + wantOK: false,

42 + },

43 + "found": {

44 + snapshot: &Snapshot{

45 + stores: map[string]publishedRecord{

46 + "s1": {published: testPublishedStore{}},

47 + },

48 + },

49 + id: "s1",

50 + wantOK: true,

51 + },

52 + }

53 +

54 + for name, tc := range tests {

55 + t.Run(name, func(t *testing.T) {

56 + store, ok := tc.snapshot.lookupStore(tc.id)

57 + if !assert.Equal(t, tc.wantOK, ok, "lookupStore(%q) ok mismatch", tc.id) {

58 + return

59 + }

60 + if ok {

61 + assert.NotNil(t, store.published)

62 + }

63 + })

64 + }

65 +}

src/go/plugin/agent/secrets/secretstore/test_helpers_test.go new

+136

@@ -0,0 +1,136 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore_test

4 +

5 +import (

6 + "encoding/json"

7 + "testing"

8 +

9 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

10 + "github.com/stretchr/testify/require"

11 +)

12 +

13 +type providerBackedConfig struct {

14 + kind secretstore.StoreKind

15 + name string

16 + config map[string]any

17 +}

18 +

19 +func providerBackedConfigs() []providerBackedConfig {

20 + return []providerBackedConfig{

21 + {

22 + kind: secretstore.KindAWSSM,

23 + name: "aws_prod",

24 + config: map[string]any{

25 + "name": "aws_prod",

26 + "auth_mode": "env",

27 + "region": "us-east-1",

28 + },

29 + },

30 + {

31 + kind: secretstore.KindAzureKV,

32 + name: "azure_prod",

33 + config: map[string]any{

34 + "name": "azure_prod",

35 + "mode": "managed_identity",

36 + },

37 + },

38 + {

39 + kind: secretstore.KindGCPSM,

40 + name: "gcp_prod",

41 + config: map[string]any{

42 + "name": "gcp_prod",

43 + "mode": "metadata",

44 + },

45 + },

46 + {

47 + kind: secretstore.KindVault,

48 + name: "vault_prod",

49 + config: map[string]any{

50 + "name": "vault_prod",

51 + "mode": "token",

52 + "mode_token": map[string]any{

53 + "token": "vault-token",

54 + },

55 + "addr": "https://vault.example",

56 + },

57 + },

58 + }

59 +}

60 +

61 +func testSingleVaultConfig() map[string]any {

62 + return map[string]any{

63 + "name": "vault_prod",

64 + "mode": "token",

65 + "mode_token": map[string]any{

66 + "token": "vault-token",

67 + },

68 + "addr": "https://vault.example",

69 + }

70 +}

71 +

72 +func newStoreFromConfig(t *testing.T, _ secretstore.Service, kind secretstore.StoreKind, cfg map[string]any) secretstore.Config {

73 + t.Helper()

74 +

75 + name, _ := cfg["name"].(string)

76 + if name == "" {

77 + name, _ = cfg["id"].(string)

78 + }

79 + require.NotEmpty(t, name)

80 +

81 + payload := secretstore.Config(cloneTestMap(cfg))

82 + delete(payload, "name")

83 + delete(payload, "id")

84 + delete(payload, "kind")

85 + delete(payload, "enabled")

86 + delete(payload, "description")

87 + payload.SetName(name)

88 + payload.SetKind(kind)

89 + payload.SetSource("dyncfg")

90 + payload.SetSourceType("dyncfg")

91 + return payload

92 +}

93 +

94 +func decodeSchema(t *testing.T, raw string) map[string]any {

95 + t.Helper()

96 +

97 + var out map[string]any

98 + require.NoError(t, json.Unmarshal([]byte(raw), &out))

99 + return out

100 +}

101 +

102 +func cloneTestMap(in map[string]any) map[string]any {

103 + if len(in) == 0 {

104 + return nil

105 + }

106 + out := make(map[string]any, len(in))

107 + for k, v := range in {

108 + switch tv := v.(type) {

109 + case map[string]any:

110 + out[k] = cloneTestMap(tv)

111 + case []any:

112 + out[k] = cloneTestSlice(tv)

113 + default:

114 + out[k] = tv

115 + }

116 + }

117 + return out

118 +}

119 +

120 +func cloneTestSlice(in []any) []any {

121 + if len(in) == 0 {

122 + return nil

123 + }

124 + out := make([]any, 0, len(in))

125 + for _, v := range in {

126 + switch tv := v.(type) {

127 + case map[string]any:

128 + out = append(out, cloneTestMap(tv))

129 + case []any:

130 + out = append(out, cloneTestSlice(tv))

131 + default:

132 + out = append(out, tv)

133 + }

134 + }

135 + return out

136 +}

src/go/plugin/agent/secrets/secretstore/types.go new

+57

@@ -0,0 +1,57 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import "time"

6 +

7 +type StoreKind string

8 +

9 +const (

10 + KindVault StoreKind = "vault"

11 + KindAWSSM StoreKind = "aws-sm"

12 + KindAzureKV StoreKind = "azure-kv"

13 + KindGCPSM StoreKind = "gcp-sm"

14 +)

15 +

16 +func (k StoreKind) IsValid() bool {

17 + switch k {

18 + case KindVault, KindAWSSM, KindAzureKV, KindGCPSM:

19 + return true

20 + default:

21 + return false

22 + }

23 +}

24 +

25 +type ValidationStatus struct {

26 + CheckedAt time.Time `json:"checked_at" yaml:"checked_at"`

27 + OK bool `json:"ok" yaml:"ok"`

28 +}

29 +

30 +type StoreStatus struct {

31 + Name string `json:"name" yaml:"name"`

32 + Kind StoreKind `json:"kind" yaml:"kind"`

33 + LastValidation *ValidationStatus `json:"last_validation,omitempty" yaml:"last_validation,omitempty"`

34 + LastErrorSummary string `json:"last_error_summary,omitempty" yaml:"last_error_summary,omitempty"`

35 +}

36 +

37 +type ResolveRequest struct {

38 + StoreKey string `json:"store_key" yaml:"store_key"`

39 + StoreKind StoreKind `json:"store_kind" yaml:"store_kind"`

40 + StoreName string `json:"store_name" yaml:"store_name"`

41 + Operand string `json:"operand" yaml:"operand"`

42 + Original string `json:"original" yaml:"original"`

43 +}

44 +

45 +var (

46 + ErrStoreExists = errStore("store already exists")

47 + ErrStoreNotFound = errStore("store is not configured")

48 +)

49 +

50 +type errStore string

51 +

52 +func (e errStore) Error() string { return string(e) }

53 +

54 +type JobRef struct {

55 + ID string `json:"id" yaml:"id"`

56 + Display string `json:"display" yaml:"display"`

57 +}

src/go/plugin/agent/secrets/secretstore/types_test.go new

+29

@@ -0,0 +1,29 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +package secretstore

4 +

5 +import (

6 + "testing"

7 +

8 + "github.com/stretchr/testify/assert"

9 +)

10 +

11 +func TestStoreKindIsValid(t *testing.T) {

12 + tests := map[string]struct {

13 + kind StoreKind

14 + want bool

15 + }{

16 + "vault": {kind: KindVault, want: true},

17 + "aws-sm": {kind: KindAWSSM, want: true},

18 + "azure-kv": {kind: KindAzureKV, want: true},

19 + "gcp-sm": {kind: KindGCPSM, want: true},

20 + "empty": {kind: "", want: false},

21 + "unknown": {kind: "foobar", want: false},

22 + }

23 +

24 + for name, tc := range tests {

25 + t.Run(name, func(t *testing.T) {

26 + assert.Equal(t, tc.want, tc.kind.IsValid())

27 + })

28 + }

29 +}

src/go/plugin/agent/setup.go

+17

@@ -8,6 +8,7 @@ import (

9 "github.com/netdata/netdata/go/plugins/pkg/pluginconfig"

10 "github.com/netdata/netdata/go/plugins/plugin/agent/discovery"

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

13 "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

14 "github.com/netdata/netdata/go/plugins/plugin/framework/functions"

@@ -182,6 +183,22 @@ func (a *Agent) setupVnodeRegistry() map[string]*vnodes.VirtualNode {

183 return reg

184 }

185

186 +func (a *Agent) setupSecretStoreConfigs() []secretstore.Config {

187 + a.Debugf("looking for 'ss/' in %v", a.CollectorsConfDir)

188 + if len(a.CollectorsConfDir) == 0 {

189 + return nil

190 + }

191 +

192 + cfgs, errs := secretstore.LoadFileConfigs(a.CollectorsConfDir)

193 + for _, err := range errs {

194 + a.Warningf("%v", err)

195 + }

196 + if len(cfgs) > 0 {

197 + a.Infof("loaded %d secretstore configs from 'ss/'", len(cfgs))

198 + }

199 + return cfgs

200 +}

201 +

202 func loadYAML(conf any, path string) error {

203 f, err := os.Open(path)

204 if err != nil {

src/go/plugin/agent/setup_test.go

+61

@@ -3,10 +3,14 @@

3 package agent

5 import (

6 + "os"

7 + "path/filepath"

8 "testing"

10 "github.com/netdata/netdata/go/plugins/plugin/agent/discovery"

11 + "github.com/netdata/netdata/go/plugins/plugin/agent/secrets/secretstore"

12 "github.com/netdata/netdata/go/plugins/plugin/framework/collectorapi"

13 + "github.com/netdata/netdata/go/plugins/plugin/framework/confgroup"

15 "github.com/stretchr/testify/assert"

16 "github.com/stretchr/testify/require"

@@ -209,6 +213,63 @@ func TestIsStockConfig(t *testing.T) {

213 assert.False(t, isStockConfig("/etc/netdata/go.d/module.conf"))

214 }

215

216 +func TestAgent_setupSecretStoreConfigs(t *testing.T) {

217 + t.Run("loads merged configs from collectors dirs only", func(t *testing.T) {

218 + base := t.TempDir()

219 + configRoot := filepath.Join(base, "etc", "netdata")

220 + userCollectors := filepath.Join(base, "etc", "netdata", "go.d")

221 + stockCollectors := filepath.Join(base, "usr", "lib", "netdata", "conf.d", "go.d")

222 +

223 + mustWriteAgentSecretStoreConfigFile(t, filepath.Join(configRoot, "ss", "vault.conf"), `

224 +jobs:

225 + - name: ignored

226 + mode: token

227 + mode_token:

228 + token: config-dir-should-not-load

229 + addr: https://vault.example

230 +`)

231 + mustWriteAgentSecretStoreConfigFile(t, filepath.Join(userCollectors, "ss", "vault.conf"), `

232 +jobs:

233 + - name: vault_prod

234 + mode: token

235 + mode_token:

236 + token: user-token

237 + addr: https://vault.example

238 +`)

239 + mustWriteAgentSecretStoreConfigFile(t, filepath.Join(stockCollectors, "ss", "aws-sm.conf"), `

240 +jobs:

241 + - name: aws_prod

242 + auth_mode: env

243 + region: us-east-1

244 +`)

245 +

246 + agent := &Agent{

247 + ConfigDir: []string{configRoot},

248 + CollectorsConfDir: []string{userCollectors, stockCollectors},

249 + }

250 +

251 + cfgs := agent.setupSecretStoreConfigs()

252 + require.Len(t, cfgs, 2)

253 + assert.Equal(t, "vault_prod", cfgs[0].Name())

254 + assert.Equal(t, secretstore.KindVault, cfgs[0].Kind())

255 + assert.Equal(t, confgroup.TypeUser, cfgs[0].SourceType())

256 + assert.Equal(t, "aws_prod", cfgs[1].Name())

257 + assert.Equal(t, secretstore.KindAWSSM, cfgs[1].Kind())

258 + assert.Equal(t, confgroup.TypeStock, cfgs[1].SourceType())

259 + })

260 +

261 + t.Run("no collectors config dirs returns nil", func(t *testing.T) {

262 + agent := &Agent{}

263 + assert.Nil(t, agent.setupSecretStoreConfigs())

264 + })

265 +}

266 +

267 +func mustWriteAgentSecretStoreConfigFile(t *testing.T, path, content string) {

268 + t.Helper()

269 + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))

270 + require.NoError(t, os.WriteFile(path, []byte(content), 0o644))

271 +}

272 +

273 func TestAgent_buildDiscoveryConf(t *testing.T) {

274 providers := []discovery.ProviderFactory{

275 discovery.NewProviderFactory("noop", nil),

src/go/plugin/framework/dyncfg/handler.go

+19 -5

@@ -50,6 +50,12 @@ type CodedError interface {

50 Code() int

51 }

53 +// CommandMessageSource optionally provides a success/warning message

54 +// for the command that just completed.

55 +type CommandMessageSource interface {

56 + TakeCommandMessage() string

57 +}

58 +

59 // HandlerOpts configures the handler with component-specific settings.

60 type HandlerOpts[C Config] struct {

61 Logger *logger.Logger

@@ -82,6 +88,14 @@ type Handler[C Config] struct {

88 waitGate *waitGate[C]

89 }

91 +func takeCommandMessage[C Config](cb Callbacks[C]) string {

92 + msgSrc, ok := any(cb).(CommandMessageSource)

93 + if !ok {

94 + return ""

95 + }

96 + return msgSrc.TakeCommandMessage()

97 +}

98 +

99 // WaitTimeoutEvent describes a wait gate timeout transition.

100 type WaitTimeoutEvent struct {

101 Key string

@@ -506,7 +520,7 @@ func (h *Handler[C]) CmdEnable(fn Function) {

520 }

521

522 entry.Status = StatusRunning

509 - h.api.SendCodef(fn, 200, "")

523 + h.api.SendCodef(fn, 200, "%s", takeCommandMessage(h.cb))

524 h.NotifyJobStatus(entry.Cfg, StatusRunning)

525 h.cb.OnStatusChange(entry, oldStatus, fn)

526 }

@@ -537,7 +551,7 @@ func (h *Handler[C]) CmdDisable(fn Function) {

551 h.cb.Stop(entry.Cfg)

552

553 entry.Status = StatusDisabled

540 - h.api.SendCodef(fn, 200, "")

554 + h.api.SendCodef(fn, 200, "%s", takeCommandMessage(h.cb))

555 h.NotifyJobStatus(entry.Cfg, StatusDisabled)

556 h.cb.OnStatusChange(entry, oldStatus, fn)

557 }

@@ -565,7 +579,7 @@ func (h *Handler[C]) CmdRemove(fn Function) {

579 h.exposed.Remove(entry.Cfg)

580 h.cb.Stop(entry.Cfg)

581

568 - h.api.SendCodef(fn, 200, "")

582 + h.api.SendCodef(fn, 200, "%s", takeCommandMessage(h.cb))

583 h.NotifyJobRemove(entry.Cfg)

584 }

585

@@ -632,7 +646,7 @@ func (h *Handler[C]) CmdUpdate(fn Function) {

646 if isConversion {

647 h.NotifyJobCreate(newCfg, StatusDisabled)

648 }

635 - h.api.SendCodef(fn, 200, "")

649 + h.api.SendCodef(fn, 200, "%s", takeCommandMessage(h.cb))

650 h.NotifyJobStatus(newCfg, StatusDisabled)

651 h.cb.OnStatusChange(newEntry, oldStatus, fn)

652 return

@@ -672,7 +686,7 @@ func (h *Handler[C]) CmdUpdate(fn Function) {

686 if isConversion {

687 h.NotifyJobCreate(newCfg, StatusRunning)

688 }

675 - h.api.SendCodef(fn, 200, "")

689 + h.api.SendCodef(fn, 200, "%s", takeCommandMessage(h.cb))

690 h.NotifyJobStatus(newCfg, StatusRunning)

691 h.cb.OnStatusChange(newEntry, oldStatus, fn)

692 }

src/go/plugin/framework/dyncfg/helpers.go

@@ -14,6 +14,7 @@ func WrapHandler(handler func(Function)) func(functions.Function) {

14 // BindResponder swaps a component responder and keeps handler API in sync.

15 func BindResponder[C Config](dst **Responder, handler *Handler[C], responder *Responder) {

16 if responder == nil {

17 + // Nil means "keep the current responder" rather than clearing output wiring.

18 return

19 }

20 *dst = responder

src/go/plugin/go.d/config/go.d/ss/aws-sm.conf new

+15

@@ -0,0 +1,15 @@

1 +## Uncomment and adjust the secretstores you want Netdata to load at startup.

2 +## File-defined secretstores are loaded from go.d/ss during agent startup.

3 +

4 +#jobs:

5 +# - name: aws_prod

6 +# auth_mode: env

7 +# region: us-east-1

8 +#

9 +# - name: aws_ecs

10 +# auth_mode: ecs

11 +# region: us-east-1

12 +#

13 +# - name: aws_imds

14 +# auth_mode: imds

15 +# region: us-east-1

src/go/plugin/go.d/config/go.d/ss/azure-kv.conf new

+15

@@ -0,0 +1,15 @@

1 +## Uncomment and adjust the secretstores you want Netdata to load at startup.

2 +## File-defined secretstores are loaded from go.d/ss during agent startup.

3 +

4 +#jobs:

5 +# - name: azure_prod

6 +# mode: client

7 +# mode_client:

8 +# tenant_id: 00000000-0000-0000-0000-000000000000

9 +# client_id: 00000000-0000-0000-0000-000000000000

10 +# client_secret: your-client-secret

11 +#

12 +# - name: azure_vm

13 +# mode: managed_identity

14 +# mode_managed_identity:

15 +# client_id: 00000000-0000-0000-0000-000000000000

src/go/plugin/go.d/config/go.d/ss/gcp-sm.conf new

+11

@@ -0,0 +1,11 @@

1 +## Uncomment and adjust the secretstores you want Netdata to load at startup.

2 +## File-defined secretstores are loaded from go.d/ss during agent startup.

3 +

4 +#jobs:

5 +# - name: gcp_metadata

6 +# mode: metadata

7 +#

8 +# - name: gcp_service_account

9 +# mode: service_account_file

10 +# mode_service_account_file:

11 +# path: /etc/netdata/gcp-service-account.json

src/go/plugin/go.d/config/go.d/ss/vault.conf new

+17

@@ -0,0 +1,17 @@

1 +## Uncomment and adjust the secretstores you want Netdata to load at startup.

2 +## File-defined secretstores are loaded from go.d/ss during agent startup.

3 +

4 +#jobs:

5 +# - name: vault_prod

6 +# mode: token

7 +# mode_token:

8 +# token: your-vault-token

9 +# addr: https://vault.example

10 +# namespace: admin

11 +# tls_skip_verify: false

12 +#

13 +# - name: vault_prod_file_token

14 +# mode: token_file

15 +# mode_token_file:

16 +# path: /var/lib/netdata/vault.token

17 +# addr: https://vault.example

src/go/plugin/ibm.d/config/ibm.d/websphere_pmi.conf

+2 -2

@@ -108,7 +108,7 @@

108 # # - name: prod_was_node1

109 # # url: https://was-prod-01.example.com:9443/wasPerfTool/servlet/perfservlet

110 # # username: monitor

111 -# # password: ${WEBSPHERE_PASSWORD} # Use environment variable

111 +# # password: ${env:WEBSPHERE_PASSWORD} # Use environment variable

112 # # tls_skip_verify: true

113 # # cluster_name: ProdCluster

114 # # node_name: ProdNode01

@@ -122,7 +122,7 @@

122 # # - name: dmgr

123 # # url: https://dmgr.example.com:9043/wasPerfTool/servlet/perfservlet

124 # # username: wasadmin

125 -# # password: ${DMGR_PASSWORD}

125 +# # password: ${env:DMGR_PASSWORD}

126 # # server_type: dmgr

127 # # collect_cluster_metrics: true

128 # # collect_webapp_metrics: false