@cryptotaxi247 / netdata-1 / commits / 61197faf9

added dyncfg docs (#20187)

Co-authored-by: ilyam8 <ilya@netdata.cloud>

Costa Tsaousis committed Apr 28, 2025 at 10:38 UTC 61197faf9ee59efd815260f5d7f2a1f36117e8e3
3 files changed +1035
docs/developer-and-contributor-corner/dyncfg.md new
+116
@@ -0,0 +1,116 @@
1 +# Dynamic Configuration (DynCfg)
2 +
3 +Dynamic Configuration (DynCfg) is a system in Netdata that enables both internal and external plugins/modules to expose their configurations dynamically to users through a unified interface. This document provides an overview of the DynCfg system and directs developers to detailed implementation documentation.
4 +
5 +## Overview
6 +
7 +DynCfg provides a centralized mechanism for:
8 +
9 +1. Registering configuration objects from any plugin or module
10 +2. Providing a unified interface for users to view and modify these configurations
11 +3. Persisting configurations between Netdata agent restarts
12 +4. Validating configuration changes through the originating plugin/module
13 +5. Standardizing configuration UI using JSON Schema
14 +
15 +Key features:
16 +
17 +- Plugins can expose multiple configuration objects
18 +- Each configuration object has a unique ID
19 +- The owning plugin validates configuration changes before being committed
20 +- The DynCfg manager maintains the state of all dynamic configurations
21 +- JSON Schema is used to define the structure of configuration objects
22 +- The UI is based on adaptations of the react-jsonschema-form project
23 +
24 +## Architecture
25 +
26 +DynCfg consists of these key components:
27 +
28 +1. **DynCfg Manager**: Core system that tracks configurations and routes commands
29 +2. **Internal Plugin API**: Used by modules inside the Netdata agent
30 +3. **External Plugin API**: Used by independent plugins communicating via plugins.d protocol
31 +4. **Web API**: Exposes configuration management to users and applications
32 +
33 +## Configuration Types
34 +
35 +DynCfg supports three types of configurations:
36 +
37 +- **SINGLE**: A standalone configuration object (e.g., systemd-journal directories)
38 +- **TEMPLATE**: A blueprint for creating multiple related configurations (e.g., Nginx collector template)
39 +- **JOB**: A specific configuration instance derived from a template (e.g., a specific Nginx server to monitor)
40 +
41 +## Implementation Documentation
42 +
43 +For detailed implementation guidance, refer to these documents:
44 +
45 +### For Internal Modules/Plugins
46 +
47 +If you're developing an internal Netdata module or plugin, see:
48 +
49 +👉 [**Internal DynCfg Implementation Guide**](/src/daemon/dyncfg/README.md)
50 +
51 +This document covers:
52 +
53 +- Low-level and high-level APIs
54 +- Configuration ID structure
55 +- Response codes and status handling
56 +- Action behavior for different configuration types
57 +- JSON Schema implementation
58 +- API access and endpoints
59 +- Best practices
60 +
61 +### For External Plugins
62 +
63 +If you're developing an external plugin that communicates with Netdata using the plugins.d protocol, see:
64 +
65 +👉 [**External Plugin DynCfg Implementation Guide**](/src/plugins.d/DYNCFG.md)
66 +
67 +This document covers:
68 +
69 +- Plugin protocol commands and responses
70 +- Registering configurations
71 +- Handling configuration commands
72 +- Responding to status changes
73 +- Schema handling
74 +- Working examples
75 +
76 +## Example Implementations
77 +
78 +For reference, you can study these existing implementations:
79 +
80 +### Health Alerts System (Internal)
81 +
82 +The health module uses DynCfg to manage alert definitions. Key files:
83 +
84 +- `src/health/health_dyncfg.c`: Implements DynCfg integration for health alerts
85 +- Uses the high-level API for internal plugins
86 +
87 +### systemd-journal.plugin (External)
88 +
89 +The systemd-journal.plugin is a C-based external plugin that uses DynCfg. Key files:
90 +
91 +- `src/collectors/systemd-journal.plugin/systemd-journal-dyncfg.c`: Implements a SINGLE configuration for journal directories
92 +
93 +### go.d.plugin (External)
94 +
95 +go.d.plugin is a Go-based external plugin that uses DynCfg to manage job configurations:
96 +
97 +- Implements templates and jobs for various data collectors
98 +- Dynamically generates JSON Schema based on Go struct tags
99 +
100 +## Best Practices
101 +
102 +When implementing DynCfg for your module or plugin:
103 +
104 +1. **Use Clear ID Structure**: Follow the component:category:name pattern
105 +2. **Choose Logical Paths**: The path parameter affects UI organization
106 +3. **Validate Thoroughly**: Always validate configuration changes before accepting
107 +4. **Provide Detailed Errors**: Help users understand why a configuration was rejected
108 +5. **Document Your Schema**: Include good descriptions in your JSON Schema
109 +6. **Respect Type-Action Relationships**: Different actions behave differently for each configuration type
110 +7. **Return Appropriate Status Codes**: Use the correct response codes for each situation
111 +
112 +## More Information
113 +
114 +For more details about using the Netdata Agent UI to manage dynamic configurations, see the [Netdata Cloud documentation](https://learn.netdata.cloud/docs/agent/web/gui/).
115 +
116 +To learn about developing for Netdata, see the [Developer Corner](https://learn.netdata.cloud/docs/agent/contribute/).
src/daemon/dyncfg/README.md new
+451
@@ -0,0 +1,451 @@
1 +# Dynamic Configuration (DynCfg)
2 +
3 +Dynamic Configuration (DynCfg) is a system in Netdata that enables both internal and external plugins/modules to expose their configurations dynamically to users through a unified interface. This document explains how DynCfg works and how to integrate your module with it.
4 +
5 +## Overview
6 +
7 +DynCfg provides a centralized mechanism for:
8 +
9 +1. Registering configuration objects from any plugin or module
10 +2. Providing a unified interface for users to view and modify these configurations
11 +3. Persisting configurations between Netdata agent restarts
12 +4. Validating configuration changes through the originating plugin/module
13 +5. Standardizing configuration UI using JSON Schema
14 +
15 +Key features:
16 +
17 +- Plugins can expose multiple configuration objects
18 +- Each configuration object has a unique ID
19 +- The owning plugin validates configuration changes before being committed
20 +- The DynCfg manager maintains the state of all dynamic configurations
21 +- JSON Schema is used to define the structure of configuration objects
22 +- The UI is based on adaptations of the react-jsonschema-form project
23 +
24 +## Architecture
25 +
26 +DynCfg consists of several API layers:
27 +
28 +1. **Low-level API**: Core functionality used by both internal and external plugins
29 +2. **High-level API**: Simplified API for internal plugins and modules
30 +3. **External Plugin API**: Used by external plugins like go.d.plugin
31 +
32 +## Integration Approaches
33 +
34 +There are two main ways to integrate with DynCfg:
35 +
36 +1. **High-level API for Internal Plugins**: Used by the health alerts system (documented here)
37 +2. **External Plugin API**: Used by go.d.plugin (see [src/plugins.d/DYNCFG.md](/src/plugins.d/DYNCFG.md) for detailed documentation)
38 +
39 +### Core Concepts
40 +
41 +### Configuration ID Structure
42 +
43 +The configuration ID is a crucial part of the DynCfg system. It serves as a unique identifier and determines how the configuration appears in the UI.
44 +
45 +#### ID Format and Hierarchy
46 +
47 +Configuration IDs typically follow a colon-separated hierarchical structure:
48 +
49 +```
50 +component:category:name
51 +```
52 +
53 +Where:
54 +
55 +- **component**: The main module or plugin (e.g., "health", "go.d", "systemd-journal")
56 +- **category**: Optional subcategory (e.g., "alert", "job")
57 +- **name**: Specific configuration name
58 +
59 +For example:
60 +
61 +- `health:alert:prototype`: Health alert prototype template
62 +- `health:alert:prototype:ram_usage`: Specific health alert prototype
63 +- `go.d:nginx`: Nginx collector template
64 +- `go.d:nginx:local_server`: Specific Nginx collector job
65 +- `systemd-journal:monitored-directories`: Systemd journal configuration
66 +
67 +#### Templates and Jobs
68 +
69 +For templates and jobs, the ID follows a specific pattern:
70 +
71 +1. **Template ID**: `component:template_name`
72 +2. **Job ID**: `component:template_name:job_name`
73 +
74 +The part before the last colon in a job ID must match an existing template ID.
75 +
76 +#### UI Organization
77 +
78 +The first component of the ID is used to organize configurations in the UI, creating separate tabs or sections. This allows for logical grouping of related configurations.
79 +
80 +When choosing an ID, consider:
81 +
82 +- UI organization (first component)
83 +- Logical grouping (middle components)
84 +- Uniqueness (complete ID)
85 +- Readability for users
86 +
87 +### Configuration Types
88 +
89 +- **DYNCFG_TYPE_SINGLE**: A single configuration object
90 +- **DYNCFG_TYPE_TEMPLATE**: A template for creating multiple job configurations
91 +- **DYNCFG_TYPE_JOB**: A specific job configuration (derived from a template)
92 +
93 +### Response Codes
94 +
95 +DynCfg uses HTTP-like response codes to indicate the status of operations. These are crucial for both internal and external plugins to properly communicate with the DynCfg system:
96 +
97 +#### Success Codes (2xx)
98 +
99 +- **DYNCFG_RESP_RUNNING (200)**: Configuration was accepted and is currently running
100 +- **DYNCFG_RESP_ACCEPTED (202)**: Configuration was accepted but not yet running
101 +- **DYNCFG_RESP_ACCEPTED_DISABLED (298)**: Configuration was accepted but is currently disabled
102 +- **DYNCFG_RESP_ACCEPTED_RESTART_REQUIRED (299)**: Configuration was accepted but requires a restart to apply
103 +
104 +#### Error Codes (4xx, 5xx)
105 +
106 +Standard HTTP error codes are used, including:
107 +
108 +- **HTTP_RESP_BAD_REQUEST (400)**: Invalid request or configuration
109 +- **HTTP_RESP_NOT_FOUND (404)**: Requested configuration was not found
110 +- **HTTP_RESP_INTERNAL_SERVER_ERROR (500)**: An internal error occurred
111 +- **HTTP_RESP_NOT_IMPLEMENTED (501)**: The requested operation is not implemented
112 +
113 +When implementing a callback function, always return the appropriate response code to indicate the status of the operation. The DynCfg system uses these codes to determine how to handle the configuration and what to display to the user.
114 +
115 +### Source Types
116 +
117 +- **DYNCFG_SOURCE_TYPE_INTERNAL**: Configuration defined internally within Netdata
118 +- **DYNCFG_SOURCE_TYPE_DYNCFG**: Configuration created/modified through DynCfg
119 +- **DYNCFG_SOURCE_TYPE_USER**: Configuration from user-provided files
120 +
121 +#### Supported Commands
122 +
123 +- **DYNCFG_CMD_SCHEMA**: Get JSON schema for the configuration
124 +- **DYNCFG_CMD_GET**: Get the current configuration
125 +- **DYNCFG_CMD_UPDATE**: Update the configuration
126 +- **DYNCFG_CMD_DISABLE**: Disable the configuration
127 +- **DYNCFG_CMD_ENABLE**: Enable the configuration
128 +- **DYNCFG_CMD_ADD**: Add a new job (for templates only)
129 +- **DYNCFG_CMD_REMOVE**: Remove a job (for DYNCFG_SOURCE_TYPE_DYNCFG jobs only)
130 +- **DYNCFG_CMD_TEST**: Test a configuration without applying it
131 +- **DYNCFG_CMD_RESTART**: Restart the configuration
132 +- **DYNCFG_CMD_USERCONFIG**: Get the configuration in a user-friendly format (used for conf files)
133 +
134 +## Implementing DynCfg for Internal Plugins
135 +
136 +Here's how to implement DynCfg for an internal plugin/module:
137 +
138 +### 1. Register Your Configuration
139 +
140 +For internal plugins, the Netdata daemon already handles initialization and shutdown of the DynCfg system. You don't need to call `dyncfg_init()` or `dyncfg_shutdown()` in your plugin code.
141 +
142 +Register your configurations when your plugin initializes:
143 +
144 +```c
145 +bool dyncfg_add(
146 + RRDHOST *host, // The host this configuration belongs to (localhost for global configs)
147 + const char *id, // Unique ID for this configuration
148 + const char *path, // Path for UI organization
149 + DYNCFG_STATUS status, // Initial status (ACCEPTED, DISABLED, etc.)
150 + DYNCFG_TYPE type, // SINGLE, TEMPLATE, or JOB
151 + DYNCFG_SOURCE_TYPE source_type, // INTERNAL, DYNCFG, USER
152 + const char *source, // Source identifier (e.g., "internal")
153 + DYNCFG_CMDS cmds, // Supported commands (bitwise OR of DYNCFG_CMD_* values)
154 + HTTP_ACCESS view_access, // Access permissions for viewing
155 + HTTP_ACCESS edit_access, // Access permissions for editing
156 + dyncfg_cb_t cb, // Callback function for handling commands
157 + void *data // User data passed to the callback
158 +);
159 +```
160 +
161 +Example (from health_dyncfg.c):
162 +
163 +```c
164 +dyncfg_add(
165 + localhost,
166 + DYNCFG_HEALTH_ALERT_PROTOTYPE_PREFIX,
167 + "/health/alerts/prototypes",
168 + DYNCFG_STATUS_ACCEPTED,
169 + DYNCFG_TYPE_TEMPLATE,
170 + DYNCFG_SOURCE_TYPE_INTERNAL,
171 + "internal",
172 + DYNCFG_CMD_SCHEMA | DYNCFG_CMD_ADD | DYNCFG_CMD_ENABLE | DYNCFG_CMD_DISABLE | DYNCFG_CMD_USERCONFIG,
173 + HTTP_ACCESS_NONE,
174 + HTTP_ACCESS_NONE,
175 + dyncfg_health_cb,
176 + NULL
177 +);
178 +```
179 +
180 +### 3. Implement a Callback Function
181 +
182 +```c
183 +int your_dyncfg_callback(
184 + const char *transaction, // Transaction ID
185 + const char *id, // Configuration ID
186 + DYNCFG_CMDS cmd, // Command being executed
187 + const char *add_name, // For ADD: the name to add
188 + BUFFER *payload, // Command payload
189 + usec_t *stop_monotonic_ut, // Timeout info
190 + bool *cancelled, // Whether the operation was cancelled
191 + BUFFER *result, // Buffer to write results to
192 + HTTP_ACCESS access, // Access level of the user
193 + const char *source, // Configuration source
194 + void *data // User data passed during registration
195 +) {
196 + // Handle the command
197 + switch(cmd) {
198 + case DYNCFG_CMD_SCHEMA:
199 + // Return JSON schema
200 + buffer_json_initialize(result, "\"", "\"", 0, true, BUFFER_JSON_OPTIONS_MINIFY);
201 + // Add your schema here
202 + buffer_json_finalize(result);
203 + return HTTP_RESP_OK;
204 +
205 + case DYNCFG_CMD_GET:
206 + // Return current configuration
207 + buffer_json_initialize(result, "\"", "\"", 0, true, BUFFER_JSON_OPTIONS_MINIFY);
208 + // Add your configuration here
209 + buffer_json_finalize(result);
210 + return HTTP_RESP_OK;
211 +
212 + case DYNCFG_CMD_UPDATE:
213 + // Process configuration update
214 + // If update successful
215 + return DYNCFG_RESP_RUNNING; // or DYNCFG_RESP_ACCEPTED if not running yet
216 +
217 + // If restart is required
218 + // return DYNCFG_RESP_ACCEPTED_RESTART_REQUIRED;
219 +
220 + // If validation fails
221 + // return dyncfg_default_response(result, HTTP_RESP_BAD_REQUEST, "your error message");
222 +
223 + case DYNCFG_CMD_DISABLE:
224 + // Handle disabling the configuration
225 + return DYNCFG_RESP_ACCEPTED_DISABLED;
226 +
227 + case DYNCFG_CMD_ENABLE:
228 + // Handle enabling the configuration
229 + return DYNCFG_RESP_RUNNING; // or DYNCFG_RESP_ACCEPTED if not running yet
230 +
231 + // Handle other commands
232 +
233 + default:
234 + // Unsupported command
235 + return dyncfg_default_response(result, HTTP_RESP_BAD_REQUEST, "unsupported command");
236 + }
237 +}
238 +```
239 +
240 +### 4. Configuration Lifecycle Management
241 +
242 +Unregister configurations only when they’re no longer conceptually relevant (such as when a feature becomes unavailable), not during plugin or module shutdown:
243 +
244 +```c
245 +dyncfg_del(host, id); // Remove a specific configuration that no longer applies
246 +```
247 +
248 +When a plugin or module exits, the Netdata Functions manager automatically stops accepting requests for its functions. The configurations will remain in the DynCfg system and will become active again when the plugin restarts.
249 +
250 +The Netdata daemon also handles shutting down the entire DynCfg system, so you don't need to call `dyncfg_shutdown()` in your plugin code.
251 +
252 +## Implementing DynCfg for External Plugins
253 +
254 +External plugins like go.d.plugin use a different approach based on the plugins.d protocol. For detailed information on implementing DynCfg for external plugins, please refer to the [External Plugins DynCfg documentation](/src/plugins.d/DYNCFG.md).
255 +
256 +This documentation covers:
257 +
258 +- How to register configurations using the CONFIG command
259 +- How to respond to configuration commands
260 +- How to update configuration status
261 +- How to delete configurations
262 +- Detailed examples and best practices
263 +
264 +## JSON Schema for Configuration UI
265 +
266 +The UI for modifying configurations is generated from JSON Schema. Netdata supports two ways to provide schema definitions:
267 +
268 +### 1. Static Schema Files
269 +
270 +Netdata first attempts to load schema files from the disk before calling the plugin or module. Schema files should be placed in:
271 +
272 +- `CONFIG_DIR/schema.d/` (user-provided schemas, typically `/etc/netdata/schema.d/`)
273 +- `LIBCONFIG_DIR/schema.d/` (stock schemas, typically `/usr/lib/netdata/conf.d/schema.d/`)
274 +
275 +Schema files should be named after the configuration ID with `.json` extension, for example:
276 +
277 +- `health:alert:prototype.json`
278 +- `go.d:nginx.json`
279 +
280 +### 2. Dynamic Schema Generation
281 +
282 +If no static schema file is found, Netdata will call the plugin or module with the `DYNCFG_CMD_SCHEMA` command:
283 +
284 +1. For internal plugins, the callback function should return a JSON Schema document
285 +2. For external plugins, the plugin should respond to the schema command with a JSON Schema document
286 +
287 +This gives you flexibility to either:
288 +
289 +- Use static schema files for simple, fixed configurations
290 +- Generate schemas dynamically for more complex configurations that may change based on runtime conditions
291 +
292 +Example JSON Schema:
293 +
294 +```json
295 +{
296 + "type": "object",
297 + "properties": {
298 + "url": {
299 + "type": "string",
300 + "format": "uri",
301 + "title": "Server URL",
302 + "description": "The URL of the server to connect to"
303 + },
304 + "timeout": {
305 + "type": "integer",
306 + "minimum": 1,
307 + "maximum": 60,
308 + "title": "Timeout",
309 + "description": "Connection timeout in seconds"
310 + },
311 + "auth": {
312 + "type": "object",
313 + "title": "Authentication",
314 + "properties": {
315 + "username": {
316 + "type": "string",
317 + "title": "Username"
318 + },
319 + "password": {
320 + "type": "string",
321 + "format": "password",
322 + "title": "Password"
323 + }
324 + }
325 + }
326 + },
327 + "required": [
328 + "url"
329 + ]
330 +}
331 +```
332 +
333 +## Action Behavior by Configuration Type
334 +
335 +Different actions behave differently depending on the configuration type. The following table explains what happens when each action is applied to different configuration types:
336 +
337 +| Action | SINGLE | TEMPLATE | JOB |
338 +|----------------|--------------------------------------|--------------------------------------------------|-------------------------------------------------------|
339 +| **SCHEMA** | Returns schema from file or plugin | Returns schema from file or plugin | Uses template's schema |
340 +| **GET** | Returns current configuration | Not supported (templates don't maintain data) | Returns current configuration |
341 +| **UPDATE** | Updates single configuration | Not supported (templates don't maintain data) | Updates job configuration |
342 +| **ADD** | Not supported | Creates a new job based on template | Not supported |
343 +| **REMOVE** | Not supported | Not supported | Removes job (only for DYNCFG_SOURCE_TYPE_DYNCFG jobs) |
344 +| **ENABLE** | Enables single configuration | Enables template AND all its jobs | Enables job (fails if its template is disabled) |
345 +| **DISABLE** | Disables single configuration | Disables template AND all its jobs | Disables job |
346 +| **RESTART** | Sends restart command to plugin | Restarts all template's jobs | Restarts specific job |
347 +| **TEST** | Tests configuration without applying | Tests a new job configuration | Tests updated job configuration |
348 +| **USERCONFIG** | Returns user-friendly configuration | Returns template for user-friendly configuration | Returns user-friendly configuration |
349 +
350 +**Important Notes:**
351 +
352 +- When a template is disabled, all its jobs are automatically disabled regardless of their individual settings
353 +- Jobs can’t be enabled if their parent template is disabled
354 +- Template schemas are used for all jobs created from that template
355 +- The REMOVE action is only available for jobs created via DynCfg (with source type DYNCFG_SOURCE_TYPE_DYNCFG)
356 +- RESTART on a template recursively restarts all jobs associated with that template
357 +
358 +## API Access
359 +
360 +The DynCfg system exposes configurations via the Netdata API for management:
361 +
362 +### Tree API
363 +
364 +Netdata provides a tree API that returns the entire DynCfg tree or a specific subpath:
365 +
366 +```
367 +/api/v3/config?action=tree&path=/collectors
368 +```
369 +
370 +Parameters:
371 +
372 +- `action`: Set to "tree" to get the configuration tree
373 +- `path`: The configuration path to start from (e.g., "/collectors", "/health/alerts")
374 +- `id` (optional): Return only a specific configuration ID
375 +
376 +This returns a JSON structure of all configurations, organized by path, with their status, commands, access controls, and other metadata.
377 +
378 +### Configuration Actions API
379 +
380 +Individual configurations can be managed via:
381 +
382 +```
383 +/api/v3/config?action=<command>&id=<configuration_id>&name=<name>
384 +```
385 +
386 +Where:
387 +
388 +- `action`: One of the supported commands (get, update, add, remove, etc.)
389 +- `id`: The configuration ID to act upon
390 +- `name`: Required for add/test actions (new job name)
391 +
392 +The API automatically routes commands to the appropriate plugin or module that registered the configuration.
393 +
394 +## Best Practices
395 +
396 +1. **Use Clear IDs**: Configuration IDs should be clear and hierarchical (e.g., "mymodule:config1")
397 +2. **Choose Appropriate Paths**: The path parameter controls UI organization - select logical paths for easy navigation
398 +3. **Validate Thoroughly**: Always validate configuration changes before accepting them
399 +4. **Provide Helpful Errors**: Return detailed error messages when rejections occur
400 +5. **Document Your Schema**: Include descriptions for all properties in your JSON Schema
401 +6. **Clean Up**: Remove configurations when modules are unloaded or no longer needed
402 +7. **Security**: Set appropriate view/edit access controls for sensitive configurations
403 +
404 +## Example Implementations
405 +
406 +### Health Alerts System (Internal Plugin)
407 +
408 +Health alerts use DynCfg to manage alert definitions. Key files:
409 +
410 +- `src/health/health_dyncfg.c`: Implements DynCfg integration for health alerts
411 +- Handles alert prototype templates and individual alert configurations
412 +
413 +The health module uses the high-level API with IDs like:
414 +
415 +- `health:alert:prototype` for the alert template
416 +- `health:alert:prototype:ram_usage` for specific alert prototypes
417 +
418 +It supports multiple configuration objects, validation of alert definitions, and conversion between different configuration formats (JSON and traditional Netdata health configuration syntax).
419 +
420 +### systemd-journal.plugin (External Plugin)
421 +
422 +The systemd-journal.plugin is an external plugin written in C that uses DynCfg to manage journal directory configurations:
423 +
424 +- `src/collectors/systemd-journal.plugin/systemd-journal-dyncfg.c`: Implements DynCfg integration
425 +- Registers as `systemd-journal:monitored-directories` ID
426 +- Uses a SINGLE configuration type for its directory list
427 +- Provides validation of directory paths for security
428 +- Implements GET and UPDATE commands
429 +
430 +This is a good example of an external plugin using the DynCfg system for a single configuration object.
431 +
432 +### go.d.plugin (External Plugin)
433 +
434 +go.d.plugin uses DynCfg to manage job configurations. It:
435 +
436 +1. Registers configurations through the plugins.d protocol
437 +2. Generates dynamic JSON Schema based on Go struct tags
438 +3. Handles configuration updates for collecting jobs
439 +
440 +It uses IDs like:
441 +
442 +- `go.d:nginx` for the Nginx collector template
443 +- `go.d:nginx:local_server` for a specific Nginx collector job
444 +
445 +go.d.plugin demonstrates how external plugins can leverage DynCfg to provide dynamic configuration capabilities through the plugins.d protocol.
446 +
447 +## Debugging Tips
448 +
449 +1. Set the environment variable `NETDATA_DEBUG_DYNCFG=1` to enable debug logging for DynCfg
450 +2. Check `/var/lib/netdata/config/` for persisted configuration files
451 +3. Inspect configurations via the API: `/api/v3/config?id=<your-config-id>`
src/plugins.d/DYNCFG.md new
+468
@@ -0,0 +1,468 @@
1 +# Dynamic Configuration for External Plugins
2 +
3 +External plugins in Netdata can expose dynamic configuration capabilities through the DynCfg system. This document explains how to implement DynCfg in external plugins using the plugins.d protocol.
4 +
5 +## Overview
6 +
7 +The DynCfg system allows external plugins to:
8 +
9 +1. Register configurable entities (both single configurations and templates for creating jobs)
10 +2. Receive configuration commands from users
11 +3. Validate and apply configurations
12 +4. Persist configurations between Netdata agent restarts
13 +
14 +## Protocol Commands
15 +
16 +DynCfg for external plugins uses the following plugins.d protocol commands:
17 +
18 +1. `CONFIG`: Sent from the plugin to Netdata to register, update status, or delete configurations
19 +2. `FUNCTION`/`FUNCTION_PAYLOAD_BEGIN`: Received by the plugin to handle configuration commands
20 +3. `FUNCTION_RESULT_BEGIN`: Sent from the plugin to respond to commands
21 +
22 +## Implementing DynCfg in External Plugins
23 +
24 +### 1. Register a Configuration
25 +
26 +To register a configuration, the plugin sends the CONFIG command:
27 +
28 +```
29 +CONFIG <id> CREATE <status> <type> <path> <source_type> <source> <cmds> <view_access> <edit_access>
30 +```
31 +
32 +Where:
33 +
34 +- `id` is a unique identifier for the configurable entity (e.g., "go.d:nginx")
35 +- `status` can be:
36 + - `accepted`: Configuration is accepted but not running
37 + - `running`: Configuration is accepted and running
38 + - `failed`: Plugin fails to run the configuration
39 + - `incomplete`: Plugin needs additional settings
40 + - `disabled`: Configuration is disabled by a user
41 +- `type` can be:
42 + - `single`: A single configuration object (not addable or removable by users)
43 + - `template`: A template for creating multiple job configurations
44 + - `job`: A specific job configuration (derived from a template)
45 +- `path` is the UI organization path (usually "/collectors") that determines where in the configuration tree the item will appear in the UI. This is separate from the ID and controls the hierarchical navigation structure.
46 +- `source_type` can be:
47 + - `internal`: Based on internal code settings
48 + - `stock`: Default configurations
49 + - `user`: User configurations via a file
50 + - `dyncfg`: Configuration received via this mechanism
51 + - `discovered`: Dynamically discovered by the plugin
52 +- `source` provides more details about the exact source
53 +- `cmds` is a space or pipe (|) separated list of supported commands:
54 + - `schema`: Get JSON schema for the configuration
55 + - `get`: Get current configuration values
56 + - `update`: Receive configuration updates
57 + - `add`: Receive job creation commands (templates only)
58 + - `remove`: Remove a configuration (jobs only)
59 + - `enable`/`disable`: Enable or disable the configuration
60 + - `test`: Test a configuration without applying it
61 + - `restart`: Restart the configuration
62 + - `userconfig`: Get user-friendly configuration format
63 +- `view_access` and `edit_access` are permission bitmaps (use 0 for default permissions)
64 +
65 +Example:
66 +
67 +```
68 +CONFIG go.d:nginx CREATE accepted template /collectors internal internal schema|add|enable|disable 0 0
69 +CONFIG go.d:nginx:local_server CREATE running job /collectors dyncfg user schema|get|update|remove|enable|disable|restart 0 0
70 +```
71 +
72 +### 2. Respond to Configuration Commands
73 +
74 +The plugin receives configuration commands from Netdata as plugin functions. These come in two forms:
75 +
76 +#### Without Payload:
77 +
78 +```
79 +FUNCTION <transaction_id> <timeout_ms> "config <id> <command>" "<http_access>" "<source>"
80 +```
81 +
82 +Used for commands like: `schema`, `get`, `remove`, `enable`, `disable`, `restart`
83 +
84 +Example:
85 +
86 +```
87 +FUNCTION abcd1234 60 "config go.d:nginx:local_server get" "member" "netdata-cli"
88 +```
89 +
90 +#### With Payload:
91 +
92 +```
93 +FUNCTION_PAYLOAD_BEGIN <transaction_id> <timeout_ms> "config <id> <command>" "<http_access>" "<source>" "<content_type>"
94 +<payload_data>
95 +FUNCTION_PAYLOAD_END
96 +```
97 +
98 +Used for commands like: `update`, `add`, `test` that require additional data.
99 +
100 +Example:
101 +
102 +```
103 +FUNCTION_PAYLOAD_BEGIN abcd1234 60 "config go.d:nginx:local_server update" "member" "netdata-cli" "application/json"
104 +{
105 + "url": "http://localhost:80/stub_status",
106 + "timeout": 5,
107 + "update_every": 10
108 +}
109 +FUNCTION_PAYLOAD_END
110 +```
111 +
112 +### 3. Process Commands and Respond
113 +
114 +After receiving a command, the plugin should process it and respond with a function result:
115 +
116 +```
117 +FUNCTION_RESULT_BEGIN <transaction_id> <http_status_code> <content_type> <expiration>
118 +<result_data>
119 +FUNCTION_RESULT_END
120 +```
121 +
122 +Where:
123 +
124 +- `transaction_id` is the same ID received in the original command
125 +- `http_status_code` is the standard HTTP response code:
126 + - `200`: Success (DYNCFG_RESP_RUNNING) - Configuration accepted and running
127 + - `202`: Accepted (DYNCFG_RESP_ACCEPTED) - Configuration accepted but not running yet
128 + - `298`: Accepted but disabled (DYNCFG_RESP_ACCEPTED_DISABLED)
129 + - `299`: Accepted but restart required (DYNCFG_RESP_ACCEPTED_RESTART_REQUIRED)
130 + - `400`: Bad request - Invalid configuration
131 + - `404`: Not found - Configuration not found
132 + - `500`: Internal server error
133 +- `content_type` is typically "application/json"
134 +- `expiration` is the absolute timestamp (unix epoch) for result expiration
135 +
136 +The result data depends on the command:
137 +
138 +- `schema`: Return JSON Schema document
139 +- `get`: Return current configuration values
140 +- Other commands: Return a success or error message
141 +
142 +Success response example:
143 +
144 +```
145 +FUNCTION_RESULT_BEGIN abcd1234 200 application/json 0
146 +{
147 + "status": 200,
148 + "message": "Configuration updated successfully"
149 +}
150 +FUNCTION_RESULT_END
151 +```
152 +
153 +Error response example:
154 +
155 +```
156 +FUNCTION_RESULT_BEGIN abcd1234 400 application/json 0
157 +{
158 + "status": 400,
159 + "error_message": "Invalid URL format"
160 +}
161 +FUNCTION_RESULT_END
162 +```
163 +
164 +### 4. Update Configuration Status
165 +
166 +To update the status of a configuration after it's been created:
167 +
168 +```
169 +CONFIG <id> STATUS <new_status>
170 +```
171 +
172 +Example:
173 +
174 +```
175 +CONFIG go.d:nginx:local_server STATUS running
176 +```
177 +
178 +This is useful when a configuration transitions from "accepted" to "running" or "failed" after being tested.
179 +
180 +### 5. Delete a Configuration
181 +
182 +When a configuration is no longer available (e.g., the monitored service is removed):
183 +
184 +```
185 +CONFIG <id> DELETE
186 +```
187 +
188 +Example:
189 +
190 +```
191 +CONFIG go.d:nginx:local_server DELETE
192 +```
193 +
194 +## JSON Schema for Configuration UI
195 +
196 +DynCfg uses JSON Schema to define the structure of configuration objects, which is used to generate the UI.
197 +
198 +### Static Schema Files (Optional)
199 +
200 +Before calling the plugin, Netdata will first attempt to find a static schema file. You can provide static schema files in:
201 +
202 +- `CONFIG_DIR/schema.d/` (user-provided schemas, typically `/etc/netdata/schema.d/`)
203 +- `LIBCONFIG_DIR/schema.d/` (stock schemas, typically `/usr/lib/netdata/conf.d/schema.d/`)
204 +
205 +Schema files should be named after the configuration ID with `.json` extension:
206 +
207 +```
208 +/etc/netdata/schema.d/go.d:nginx.json
209 +```
210 +
211 +This approach is useful for stable schemas that don't change frequently.
212 +
213 +### Dynamic Schema Generation
214 +
215 +If no static schema file is found, Netdata will send a `schema` command to the plugin. When handling a `schema` request, the plugin should return a JSON Schema document:
216 +
217 +```json
218 +{
219 + "type": "object",
220 + "properties": {
221 + "url": {
222 + "type": "string",
223 + "format": "uri",
224 + "title": "Server URL",
225 + "description": "The URL of the Nginx stub_status endpoint"
226 + },
227 + "timeout": {
228 + "type": "integer",
229 + "minimum": 1,
230 + "maximum": 60,
231 + "title": "Timeout",
232 + "description": "Connection timeout in seconds"
233 + },
234 + "update_every": {
235 + "type": "integer",
236 + "minimum": 1,
237 + "title": "Update Every",
238 + "description": "Data collection frequency in seconds"
239 + }
240 + },
241 + "required": [
242 + "url"
243 + ]
244 +}
245 +```
246 +
247 +For templates, the schema will be used when users add new jobs based on the template.
248 +
249 +## Action Behavior Reference
250 +
251 +When implementing DynCfg in your external plugin, be aware of how actions should behave based on the configuration type:
252 +
253 +| Action | TEMPLATE | JOB |
254 +|----------------|-----------------------------------------|-----------------------------------------|
255 +| **SCHEMA** | Return schema for creating new jobs | Use template's schema |
256 +| **GET** | Not applicable | Return current configuration |
257 +| **UPDATE** | Not applicable | Update configuration and apply if valid |
258 +| **ADD** | Create new job from template | Not applicable |
259 +| **REMOVE** | Not supported | Remove job (only for user-created jobs) |
260 +| **ENABLE** | Enable template and all its jobs | Enable specific job |
261 +| **DISABLE** | Disable template and all its jobs | Disable specific job |
262 +| **RESTART** | Restart all jobs based on template | Restart specific job |
263 +| **TEST** | Test a potential job configuration | Test configuration changes |
264 +| **USERCONFIG** | Return template in user-friendly format | Return job in user-friendly format |
265 +
266 +**Important Implementation Notes:**
267 +
268 +- When a template is disabled, send DISABLE commands to all jobs of that template
269 +- Reject ENABLE commands for jobs if their template is disabled
270 +- For job SCHEMA requests, return the same schema as the template
271 +- REMOVE should only work on dynamically added jobs, not ones from static configurations
272 +- Return appropriate response codes to indicate the status (running, accepted, disabled)
273 +
274 +## External Plugin Examples
275 +
276 +### C-based External Plugin (systemd-journal.plugin)
277 +
278 +The systemd-journal.plugin is a C-based external plugin that uses DynCfg to manage journal directory configurations. It implements a SINGLE configuration type to manage the list of journald directories to monitor:
279 +
280 +```c
281 +// Register the configuration
282 +functions_evloop_dyncfg_add(
283 + wg,
284 + "systemd-journal:monitored-directories", // ID
285 + "/logs/systemd-journal", // UI Path
286 + DYNCFG_STATUS_RUNNING, // Status
287 + DYNCFG_TYPE_SINGLE, // Type - single configuration
288 + DYNCFG_SOURCE_TYPE_INTERNAL, // Source type
289 + "internal", // Source
290 + DYNCFG_CMD_SCHEMA | DYNCFG_CMD_GET | DYNCFG_CMD_UPDATE, // Supported commands
291 + HTTP_ACCESS_NONE, // View permissions
292 + HTTP_ACCESS_NONE, // Edit permissions
293 + systemd_journal_directories_dyncfg_cb, // Callback function
294 + NULL // User data
295 +);
296 +```
297 +
298 +Key points about its implementation:
299 +
300 +- Uses a single, non-removable configuration object
301 +- Supports schema, get, and update commands
302 +- Validates directory paths for security
303 +- Updates the systemd-journal watcher when configuration changes
304 +
305 +### Go-based External Plugin (go.d.plugin)
306 +
307 +Here's a complete example showing how a Go-based external plugin might implement DynCfg for an Nginx module:
308 +
309 +### 1. Register the Template and Jobs on Startup
310 +
311 +```
312 +# Register the template for Nginx configurations
313 +CONFIG go.d:nginx CREATE accepted template /collectors internal internal schema|add|enable|disable 0 0
314 +
315 +# Register existing jobs
316 +CONFIG go.d:nginx:local_server CREATE running job /collectors user /etc/netdata/go.d/nginx.conf schema|get|update|remove|enable|disable|restart 0 0
317 +CONFIG go.d:nginx:production CREATE running job /collectors user /etc/netdata/go.d/nginx.conf schema|get|update|remove|enable|disable|restart 0 0
318 +```
319 +
320 +### 2. Handle Schema Command
321 +
322 +When receiving:
323 +
324 +```
325 +FUNCTION abcd1234 60 "config go.d:nginx schema" "member" "netdata-cli"
326 +```
327 +
328 +Respond with:
329 +
330 +```
331 +FUNCTION_RESULT_BEGIN abcd1234 200 application/json 0
332 +{
333 + "type": "object",
334 + "properties": {
335 + "url": {
336 + "type": "string",
337 + "format": "uri",
338 + "title": "Server URL",
339 + "description": "The URL of the Nginx stub_status endpoint"
340 + },
341 + "timeout": {
342 + "type": "integer",
343 + "minimum": 1,
344 + "maximum": 60,
345 + "title": "Timeout",
346 + "description": "Connection timeout in seconds"
347 + },
348 + "update_every": {
349 + "type": "integer",
350 + "minimum": 1,
351 + "title": "Update Every",
352 + "description": "Data collection frequency in seconds"
353 + }
354 + },
355 + "required": ["url"]
356 +}
357 +FUNCTION_RESULT_END
358 +```
359 +
360 +### 3. Handle Get Command
361 +
362 +When receiving:
363 +
364 +```
365 +FUNCTION abcd1234 60 "config go.d:nginx:local_server get" "member" "netdata-cli"
366 +```
367 +
368 +Respond with:
369 +
370 +```
371 +FUNCTION_RESULT_BEGIN abcd1234 200 application/json 0
372 +{
373 + "url": "http://localhost:80/stub_status",
374 + "timeout": 5,
375 + "update_every": 10
376 +}
377 +FUNCTION_RESULT_END
378 +```
379 +
380 +### 4. Handle Update Command
381 +
382 +When receiving:
383 +
384 +```
385 +FUNCTION_PAYLOAD_BEGIN abcd1234 60 "config go.d:nginx:local_server update" "member" "netdata-cli" "application/json"
386 +{
387 + "url": "http://localhost:8080/stub_status",
388 + "timeout": 3,
389 + "update_every": 5
390 +}
391 +FUNCTION_PAYLOAD_END
392 +```
393 +
394 +Process the update and respond:
395 +
396 +```
397 +FUNCTION_RESULT_BEGIN abcd1234 200 application/json 0
398 +{
399 + "status": 200,
400 + "message": "Configuration updated successfully"
401 +}
402 +FUNCTION_RESULT_END
403 +```
404 +
405 +If a restart is required:
406 +
407 +```
408 +FUNCTION_RESULT_BEGIN abcd1234 299 application/json 0
409 +{
410 + "status": 299,
411 + "message": "Configuration updated, restart required to apply changes"
412 +}
413 +FUNCTION_RESULT_END
414 +```
415 +
416 +### 5. Handle Add Command (for templates)
417 +
418 +When receiving:
419 +
420 +```
421 +FUNCTION_PAYLOAD_BEGIN abcd1234 60 "config go.d:nginx add" "member" "netdata-cli" "application/json"
422 +{
423 + "name": "staging",
424 + "url": "http://staging:80/stub_status",
425 + "timeout": 5,
426 + "update_every": 10
427 +}
428 +FUNCTION_PAYLOAD_END
429 +```
430 +
431 +Process the new job and respond:
432 +
433 +```
434 +FUNCTION_RESULT_BEGIN abcd1234 200 application/json 0
435 +{
436 + "status": 200,
437 + "message": "Job 'staging' created successfully"
438 +}
439 +FUNCTION_RESULT_END
440 +```
441 +
442 +Then register the new job:
443 +
444 +```
445 +CONFIG go.d:nginx:staging CREATE running job /collectors dyncfg netdata-cli schema|get|update|remove|enable|disable|restart 0 0
446 +```
447 +
448 +## Best Practices
449 +
450 +1. **Use Consistent IDs**: Follow the pattern `component:template_name` for templates and `component:template_name:job_name` for jobs
451 +2. **Validate Thoroughly**: Always validate configuration changes before accepting them
452 +3. **Include Descriptive Messages**: Provide helpful error messages when rejections occur
453 +4. **Document Your Schema**: Include clear titles and descriptions for all properties in your JSON Schema
454 +5. **Handle Errors Gracefully**: Return appropriate HTTP status codes and error messages
455 +6. **Update Status Promptly**: When a configuration changes state (e.g., from "accepted" to "running"), update its status
456 +7. **Clean Up Configurations**: When a monitored resource is gone, delete its configuration with `CONFIG id DELETE`
457 +
458 +## Debugging Tips
459 +
460 +1. Set `NETDATA_DEBUG_DYNCFG=1` environment variable when running Netdata to see detailed logs
461 +2. If configurations aren't being registered, check for errors in the plugin output
462 +3. Verify configuration files are saved in `/var/lib/netdata/config/`
463 +4. Test configurations via the API: `/api/v3/config?id=<your-config-id>`
464 +
465 +## Related Documentation
466 +
467 +- [Main DynCfg Documentation](/src/daemon/dyncfg/README.md) - Core DynCfg system concepts and APIs
468 +- [Plugins.d Protocol](/src/plugins.d/README.md) - Complete documentation of the plugins.d protocol