25
multi_instance: true
26
data_collection:
27
metrics_description: |
28
- This collector runs Nagios-compatible checks, tracks the execution state of each configured job, measures how long each check takes to run, and automatically charts any performance data the check prints. For non-counter perfdata, Netdata also derives a plugin-scoped threshold-state chart for visualization and a static `nagios.job.perfdata.threshold_state` duplicate for alerting. When the check does not provide warning or critical ranges, the threshold state is `no_threshold`.
28
+ This collector runs [Nagios-compatible plugins](https://www.nagios-plugins.org/) and custom scripts, providing:
29
+
30
+ - **Check state monitoring** — tracks whether each check returns OK, WARNING, CRITICAL, or UNKNOWN
31
+ - **Execution metrics** — measures run duration, CPU time, and memory usage of each check
32
+ - **Automatic performance data charts** — any [Nagios performance data](https://nagios-plugins.org/doc/guidelines.html) in the check output is parsed and charted automatically
33
+ - **Threshold-based alerting** — when performance data includes warning/critical thresholds, Netdata derives threshold state and creates built-in alerts
34
method_description: |
30
- Netdata runs the configured Nagios-compatible command for each job, reads the process exit code to determine the check state, and parses the command output into a summary line, optional long output, and optional performance data. Any performance data found after the `|` separator is converted into charts automatically. The main perfdata value becomes a chart, and non-counter metrics also get derived threshold-state output in two forms: a plugin-scoped chart for visualization and a static `nagios.job.perfdata.threshold_state` chart labeled by `perfdata_value` for stock alerting. If the check does not provide warning or critical ranges, that threshold state is `no_threshold`. You can use packaged Nagios plugins or your own scripts, and you can control how often checks run, how retries behave, and when checks are allowed to run by using the job configuration.
35
+ Netdata executes each configured command on a schedule, reads the process exit code to determine the check state, and parses the standard output for a status message and optional [performance data](https://nagios-plugins.org/doc/guidelines.html). Any performance data is automatically converted into charts.
36
+
37
+ :::tip
38
+
39
+ You can use packaged [Nagios plugins](https://www.nagios-plugins.org/) or write your own scripts — any executable that follows the Nagios plugin output format will work.
40
+
41
+ :::
42
+
43
+ <details open><summary><strong>Nagios Plugin Output Format</strong></summary>
44
+
45
+ A Nagios-compatible plugin communicates through two channels: the **process exit code** and **standard output**. For the full specification, see the [Nagios Plugin Development Guidelines](https://nagios-plugins.org/doc/guidelines.html).
46
+
47
+ #### Exit Codes
48
+
49
+ The exit code is the **only** thing that determines the check state — the output text is for display purposes only.
50
+
51
+ | Exit Code | State | Meaning |
52
+ |:---------:|:------|:--------|
53
+ | 0 | **OK** | Check passed |
54
+ | 1 | **WARNING** | Above warning threshold or degraded |
55
+ | 2 | **CRITICAL** | Above critical threshold or service down |
56
+ | 3 | **UNKNOWN** | Invalid arguments or internal error |
57
+
58
+ #### Standard Output
59
+
60
+ The output follows this structure:
61
+
62
+ ```text
63
+ STATUS TEXT | perfdata1=val;warn;crit;min;max perfdata2=val
64
+ LONG OUTPUT LINE 1
65
+ LONG OUTPUT LINE 2 | more_perfdata=val
66
+ ```
67
+
68
+ | Part | Description |
69
+ |:-----|:------------|
70
+ | **Status text** | Text before the pipe on the first line. Shown as the job's status message. |
71
+ | **Performance data** | Text after the pipe on any line. Parsed into charts automatically. |
72
+ | **Long output** | Lines 2+ before the pipe. Additional detail text. |
73
+
74
+ > **Note:** The pipe separator is optional. Without it, the entire first line is the status text and no performance data charts are created.
75
+
76
+ #### Performance Data Format
77
+
78
+ Each performance data metric uses this format:
79
+
80
+ ```text
81
+ 'label'=value[UOM];[warn];[crit];[min];[max]
82
+ ```
83
+
84
+ | Field | Required | Description |
85
+ |:------|:--------:|:------------|
86
+ | `label` | Yes | Metric name. Quote with single quotes if it contains spaces. |
87
+ | `value` | Yes | Numeric value. |
88
+ | `UOM` | No | Unit of measurement (see table below). |
89
+ | `warn` | No | Warning threshold range. |
90
+ | `crit` | No | Critical threshold range. |
91
+ | `min` | No | Minimum possible value. |
92
+ | `max` | No | Maximum possible value. |
93
+
94
+ Separate multiple metrics with spaces.
95
+
96
+ **Supported Units of Measurement (UOM):**
97
+
98
+ | UOM | Meaning | How Netdata charts it |
99
+ |:----|:--------|:----------------------|
100
+ | *(none)* | Unitless number | Charted as-is |
101
+ | `s` | Seconds (also `ms`, `us`, `ns`) | Normalized to seconds |
102
+ | `%` | Percentage | Charted as percentage |
103
+ | `B` | Bytes (also `KB`, `MB`, `GB`, `TB`) | Charted in bytes |
104
+ | `b` | Bits (also `Kb`, `Mb`, `Gb`, `Tb`) | Charted in bits |
105
+ | `c` | Continuous counter | Charted as incremental rate |
106
+
107
+ #### Threshold Ranges
108
+
109
+ Thresholds use the format `[@]start:end`, where a bare number like `10` is shorthand for `0:10` and `~` represents negative infinity (no lower bound). An alert triggers when the value falls **outside** the range (or **inside** with the `@` prefix):
110
+
111
+ | Range | Alert when... |
112
+ |:------|:--------------|
113
+ | `10` | value < 0 or value > 10 |
114
+ | `10:` | value < 10 |
115
+ | `~:10` | value > 10 |
116
+ | `10:20` | value < 10 or value > 20 |
117
+ | `@10:20` | 10 ≤ value ≤ 20 |
118
+
119
+ When `warn` and `crit` ranges are provided on non-counter metrics, Netdata automatically derives a threshold state (ok / warning / critical) and creates charts with built-in alerts.
120
+
121
+ **Common threshold patterns:**
122
+
123
+ | I want to alert when... | `warn` | `crit` |
124
+ |:------------------------|:-------|:-------|
125
+ | Value exceeds a limit (e.g., response time > 2s) | `~:2` | `~:5` |
126
+ | Value drops below a floor (e.g., free space < 10%) | `10:` | `5:` |
127
+ | Value is outside a band (e.g., temperature 20–80) | `20:80` | `10:90` |
128
+
129
+ #### Example
130
+
131
+ A minimal Nagios-compatible script:
132
+
133
+ ```bash
134
+ #!/bin/sh
135
+ echo "OK - 85% free memory | free_pct=85%;20:;10:;0;100 used_kb=2380912KB;;;0;16380000"
136
+ exit 0
137
+ ```
138
+
139
+ This produces:
140
+ - **Check state**: OK (exit code 0)
141
+ - **Status text**: `OK - 85% free memory`
142
+ - **Charts**: `free_pct` (percentage with warning/critical thresholds) and `used_kb` (bytes)
143
+
144
+ </details>
145
+
146
+ :::info
147
+
148
+ **Retry behavior:** When a check returns a non-OK state, Netdata does not alert immediately. The check enters a **soft state** and retries at the `retry_interval` rate. Only after `max_check_attempts` consecutive failures does it become a **hard state** and trigger alerts. If the check recovers during retries, it returns to OK without alerting. The `retry` dimension on state charts indicates a soft state is in progress.
149
+
150
+ :::
151
default_behavior:
152
auto_detection:
153
description: |
182
```
183
184
Make sure the configured command path exists and is executable by the `netdata` user.
65
- - title: Write Nagios-compatible checks
185
+ - title: Prepare custom check scripts
186
description: |
67
- A compatible check uses two things:
68
-
69
- - the **exit code** to tell Netdata whether the result is OK, WARNING, CRITICAL, or UNKNOWN
70
- - the **command output** to show a human-readable message and optional performance data
71
-
72
- Use these exit codes:
73
-
74
- - `0` = OK
75
- - `1` = WARNING
76
- - `2` = CRITICAL
77
- - `3` = UNKNOWN
78
-
79
- The first output line should follow this pattern:
80
-
81
- ```text
82
- <summary text> | <perfdata>
83
- ```
84
-
85
- The `|` separator is optional:
187
+ If you are writing your own check scripts instead of using packaged Nagios plugins:
188
87
- - everything before `|` is the human-readable summary
88
- - everything after `|` is performance data used for automatic charts
89
-
90
- The summary should be short and useful because it is the main status text shown for the job. If the script prints multiple lines, Netdata uses the first line as the summary and keeps the remaining lines as long output.
91
-
92
- Each performance-data item follows this format:
93
-
94
- ```text
95
- 'label'=value[UOM];warn;crit;min;max
96
- ```
97
-
98
- Only `label` and `value` are required. The threshold and range fields are optional. Separate multiple metrics with spaces.
99
-
100
- Common units include:
101
-
102
- - `%` for percentages
103
- - `s`, `ms`, `us` for durations
104
- - `B`, `KB`, `MB`, `GB` for sizes
105
- - `c` for counters
106
-
107
- Example output:
108
-
109
- ```text
110
- OK - 85.5% free memory | free_pct=85.5%;20;10;0;100 free_kb=13999088KB;;;0;16380000
111
- ```
112
-
113
- In that example:
114
-
115
- - the exit code decides the state
116
- - `OK - 85.5% free memory` is the summary line
117
- - `free_pct=85.5%;20;10;0;100` creates a percentage metric
118
- - `free_kb=13999088KB;;;0;16380000` creates a size metric
119
- - the warning and critical ranges on non-counter metrics are also used to derive threshold-state output for both visualization and alerting
120
-
121
- Good rules to follow:
122
-
123
- - return the correct exit code
124
- - keep the first line short and readable
125
- - put performance data after `|`
126
- - separate multiple metrics with spaces
127
- - quote labels if they contain spaces
128
-
129
- Minimal example:
130
-
131
- ```bash
132
- #!/bin/sh
133
- echo "CPU OK - 20% used | cpu=20%;80;90"
134
- exit 0
135
- ```
189
+ - Place scripts anywhere accessible to the `netdata` user (e.g., `/usr/local/lib/netdata/checks/`)
190
+ - Make scripts executable: `chmod +x /path/to/script.sh`
191
+ - Test as the `netdata` user to verify permissions and environment: `sudo -u netdata /path/to/script.sh`
192
+ - Verify the exit code: `echo $?` (must be 0, 1, 2, or 3)
193
+ - Verify the output matches the Nagios plugin output format described in the Overview above
194
configuration:
195
file:
196
name: scripts.d/nagios.conf
202
enabled: true
203
list:
204
- name: update_every
147
- description: How often Netdata evaluates the job schedule, in seconds.
205
+ description: How often the collector's internal scheduler ticks, in seconds. Controls chart granularity. In most cases you only need to set `check_interval`.
206
default_value: 10
207
required: false
208
group: Collection
151
- - name: autodetection_retry
152
- description: How often Netdata retries failed auto-detection jobs, in seconds. Set `0` to keep auto-detection disabled.
153
- default_value: 0
154
- required: false
155
- group: Collection
209
210
- name: plugin
158
- description: Absolute path to the Nagios-compatible executable to run. This can be a packaged Nagios plugin or your own executable. If you need a script interpreter, point `plugin` to that interpreter and pass the script path in `args`. The command should return exit code `0`, `1`, `2`, or `3` and may print performance data after `|`.
211
+ description: Absolute path to the Nagios-compatible executable to run. This can be a packaged Nagios plugin or your own executable. If you need a script interpreter, point `plugin` to that interpreter and pass the script path in `args`. The command should return exit code `0`, `1`, `2`, or `3` and may print performance data after <code>|</code>.
212
default_value: ""
213
required: true
214
group: Target
218
required: false
219
group: Target
220
- name: arg_values
168
- description: Values exposed to `$ARG1$` through `$ARG32$` for macro expansion.
221
+ description: Values exposed to `$ARG1$` through `$ARG32$` for macro expansion. The first value maps to `$ARG1$`, the second to `$ARG2$`, and so on.
222
default_value: ""
223
required: false
224
group: Target
249
required: false
250
group: Scheduling
251
- name: check_period
199
- description: Name of the time period that controls when the job is allowed to run. Outside this period, the check does not execute and the public job state becomes `paused`.
252
+ description: Name of the [time period](/src/go/plugin/scripts.d/pkg/timeperiod/README.md) that controls when the job is allowed to run. The built-in `24x7` period (always allowed) is the default. Outside the active period, the check does not execute and the job state becomes `paused`.
253
default_value: 24x7
254
required: false
255
group: Scheduling
256
- name: time_periods
204
- description: Custom named time periods defined inside the same job.
257
+ description: Custom named [time periods](/src/go/plugin/scripts.d/pkg/timeperiod/README.md) defined inside the same job. Supports `weekly`, `nth_weekday`, and `date` rule types.
258
default_value: ""
259
required: false
260
group: Scheduling
264
default_value: ""
265
required: false
266
group: Environment
267
+ detailed_description: |
268
+ A key-value map of environment variables injected into the check's process. Use this when your script depends on variables that are not part of the collector's default environment.
269
+
270
+ ```yaml
271
+ jobs:
272
+ - name: oracle_check
273
+ plugin: /usr/local/bin/check_oracle.sh
274
+ environment:
275
+ ORACLE_HOME: /opt/oracle/product/19c
276
+ LD_LIBRARY_PATH: /opt/oracle/product/19c/lib
277
+ ```
278
- name: custom_vars
279
description: Custom service variables exposed to the check as Nagios-style macros.
280
default_value: ""
281
required: false
282
group: Environment
283
+ detailed_description: |
284
+ A key-value map of custom service variables. Each entry is exposed as a `NAGIOS__SERVICE<UPPERCASE_KEY>` environment variable and can be referenced in `args` using the Nagios macro syntax `$_SERVICE<KEY>$`.
285
+
286
+ ```yaml
287
+ jobs:
288
+ - name: check_db
289
+ plugin: /usr/lib/nagios/plugins/check_pgsql
290
+ args: ["-H", "$_SERVICEDBHOST$", "-d", "$_SERVICEDBNAME$"]
291
+ custom_vars:
292
+ DBHOST: db.example.com
293
+ DBNAME: production
294
+ ```
295
296
- name: vnode
297
description: Associate the job with a virtual node so the check can use host-specific labels and macros.
320
check_interval: 1m
321
retry_interval: 30s
322
max_check_attempts: 3
247
- - name: Custom script
248
- description: Run your own Nagios-compatible shell script.
323
+ - name: End-to-end custom script
324
+ description: |
325
+ Write a custom check script, then configure Netdata to run it.
326
+
327
+ **1. Create the script** (e.g., `/usr/local/lib/netdata/checks/check_api.sh`):
328
+
329
+ ```bash
330
+ #!/bin/sh
331
+ # Check HTTP endpoint health
332
+ URL="http://localhost:8080/health"
333
+
334
+ response=$(curl -s -o /dev/null -w "%{http_code} %{time_total}" --max-time 5 "$URL" 2>/dev/null)
335
+ curl_exit=$?
336
+
337
+ if [ "$curl_exit" -ne 0 ]; then
338
+ echo "UNKNOWN - Could not connect to $URL (curl exit code $curl_exit)"
339
+ exit 3
340
+ fi
341
+
342
+ http_code=$(echo "$response" | cut -d' ' -f1)
343
+ response_time=$(echo "$response" | cut -d' ' -f2)
344
+
345
+ if [ "$http_code" -ge 500 ]; then
346
+ echo "CRITICAL - $URL returned HTTP $http_code | response_time=${response_time}s;2;5;0;"
347
+ exit 2
348
+ elif [ "$http_code" -ne 200 ]; then
349
+ echo "WARNING - $URL returned HTTP $http_code | response_time=${response_time}s;2;5;0;"
350
+ exit 1
351
+ fi
352
+
353
+ echo "OK - $URL returned HTTP $http_code | response_time=${response_time}s;2;5;0;"
354
+ exit 0
355
+ ```
356
+
357
+ **2. Make it executable and test it:**
358
+
359
+ ```bash
360
+ chmod +x /usr/local/lib/netdata/checks/check_api.sh
361
+ sudo -u netdata /usr/local/lib/netdata/checks/check_api.sh
362
+ echo "Exit code: $?"
363
+ ```
364
+
365
+ **3. Add the configuration below, then restart Netdata** (`sudo systemctl restart netdata`). After restarting, look for `nagios.job.execution_state` and related charts in the Netdata dashboard.
366
+ config: |
367
+ jobs:
368
+ - name: api_health
369
+ plugin: /usr/local/lib/netdata/checks/check_api.sh
370
+ timeout: 10s
371
+ check_interval: 1m
372
+ retry_interval: 30s
373
+ max_check_attempts: 3
374
+ - name: Custom script (minimal)
375
+ description: Run your own Nagios-compatible shell script with minimal configuration.
376
config: |
377
jobs:
378
- name: custom_memory_check
430
Nagios checks run with a limited execution environment rather than inheriting the full Netdata process environment. If the script depends on extra variables, set them explicitly in `environment` instead of relying on ambient shell state.
431
- name: Built-in alerts cover warning and critical states only
432
description: |
306
- This collector installs stock Netdata health alerts for the `warning` and `critical` states on `nagios.job.execution_state` and `nagios.job.perfdata_threshold_state`. Both stock alert families suppress soft retry states by checking that `retry` is not active. If you also want alerts for `unknown`, `timeout`, `paused`, or more specific perfdata behavior, build your own rules on top of these contexts. The `job.perfdata.threshold_state` chart uses the `perfdata_value` label to identify which perfdata metric each threshold state belongs to.
433
+ This collector installs stock Netdata health alerts for the `warning` and `critical` states on `nagios.job.execution_state` and `nagios.job.perfdata_threshold_state`. Both stock alert families suppress soft retry states by checking that `retry` is not active. If you also want alerts for `unknown`, `timeout`, `paused`, or more specific perfdata behavior, build your own rules on top of these contexts. The `nagios.job.perfdata_threshold_state` chart uses the `perfdata_value` label to identify which perfdata metric each threshold state belongs to.
434
+ - name: Configuration changes are not picked up
435
+ description: |
436
+ After editing `scripts.d/nagios.conf`, restart the Netdata Agent for changes to take effect: `sudo systemctl restart netdata`.
437
+ - name: Script stderr output is not visible
438
+ description: |
439
+ Netdata captures the check's standard output for status and performance data parsing. Standard error (stderr) is logged by the collector but not used for state or charts. If your script writes errors to stderr, check the Netdata error log for details.
440
- name: Windows checks need an executable entry point
441
description: |
442
The collector runs the command named in `plugin` directly. On Windows, point `plugin` to an executable or to an interpreter such as `powershell.exe` and pass the script path in `args`.
462
title: Metrics
463
enabled: false
464
description: |
332
- Each configured job exposes execution-state and execution-resource charts. Netdata also emits a plugin-scoped copy of the job state named `nagios.perfdata.<plugin>.job.execution_state` so the state appears under each configured check section in the UI. If a check prints Nagios performance data, Netdata also creates additional value charts automatically from the values emitted by that check. For non-counter perfdata, Netdata creates both a plugin-scoped threshold-state chart for visualization and a static `nagios.job.perfdata.threshold_state` duplicate labeled by `perfdata_value` for alerting. Counter perfdata currently exposes only the value chart.
465
+ Each configured job produces execution state and resource usage charts. When a check emits Nagios performance data, additional charts are created automatically for each metric. Non-counter perfdata with warning/critical thresholds also get threshold state charts for alerting.
466
availability: []
467
scopes:
468
- name: job
471
- name: nagios_job
472
description: Job name as defined in the configuration.
473
- name: perfdata_value
341
- description: Normalized perfdata identity in the form `<class>_<metric_key>`, used by the static threshold-state duplicate.
474
+ description: Identifies which performance data metric a threshold state belongs to. Format is `<unit_class>_<label>`, where `<unit_class>` is derived from the UOM (`time`, `bytes`, `bits`, `percent`, or `generic`) and `<label>` is the sanitized metric label from the check output. For example, `repl_lag=5s` produces `time_repl_lag`.
475
metrics:
476
- name: nagios.job.execution_state
344
- description: Current job execution state for the check. Normal plugin results use `ok`, `warning`, `critical`, or `unknown`; collector-detected check timeouts use `timeout`; jobs blocked by `check_period` use `paused`. While a non-OK result is still retrying, the `retry` flag is also active on this bitset chart. Netdata also emits a plugin-scoped duplicate named `nagios.perfdata.<plugin>.job.execution_state` for UI grouping.
477
+ description: Current state of the check job. Values are ok, warning, critical, unknown, timeout (check exceeded configured timeout), or paused (outside check_period). The retry dimension indicates the check is retrying before reaching hard state.
478
unit: state
479
chart_type: line
480
dimensions:
485
- name: timeout
486
- name: paused
487
- name: retry
355
- - name: nagios.job.perfdata.threshold_state
356
- description: Static alert-oriented duplicate of the non-counter perfdata threshold-state signal. Use the `perfdata_value` label to select the normalized perfdata identity, such as `bytes_used` or `time_latency`. Values are `no_threshold`, `ok`, `warning`, or `critical`. While the source check result is still retrying, this bitset chart also exposes `retry`.
488
+ - name: nagios.job.perfdata_threshold_state
489
+ description: Threshold state derived from performance data warning/critical ranges. Use the `perfdata_value` label to identify the specific metric. Values are no_threshold (check provides no ranges), ok, warning, or critical. The retry dimension indicates the check is retrying.
490
unit: state
491
chart_type: line
492
dimensions:
496
- name: critical
497
- name: retry
498
- name: nagios.job.execution_duration
366
- description: Wall-clock duration recorded when the check runs. Non-due cycles report zero.
499
+ description: How long the check command took to run.
500
unit: seconds
501
chart_type: line
502
dimensions:
503
- name: duration
504
- name: nagios.job.execution_cpu_total
372
- description: CPU time used when the check runs. Non-due cycles report zero. This chart is available on non-Windows platforms.
505
+ description: CPU time (user + system) consumed by the check command. Available on non-Windows platforms.
506
unit: seconds
507
chart_type: line
508
dimensions:
509
- name: total
510
- name: nagios.job.execution_max_rss
378
- description: Peak RSS memory used when the check runs. Non-due cycles report zero. This chart is available on non-Windows platforms.
511
+ description: Peak memory (RSS) used by the check command. Available on non-Windows platforms.
512
unit: bytes
513
chart_type: line
514
dimensions: