Fix alerts-configuration-manager.md (#16679)
Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>
Fotis Voutsas committed
Dec 28, 2023 at 15:45 UTC
6f55edf7f35be8afbc20bdb320545d2c355ea2a0
1 file changed
+53
-53
docs/cloud/alerts-configuration-manager.md
+53
-53
@@ -4,68 +4,68 @@ The Netdata Alerts Configuration Manager enables users with [Business subscripti
4
5
## Using Alerts Configuration Manager
6
7
-- Go to the `Metrics` tab and navigate to the `chart` you want to alert on.
7
+1. Go to the **Metrics** tab and navigate to the chart you want to configure an alert for.
8
9
-- Click the `Alert icon` on the top right corner of the chart.
10
-
9
+2. Click the **Alert icon** on the top right corner of the chart.
10
+ <!---->
11
12
-- Alert Configuration Manager will open up with the `default` thresholds. Modify the configuration as required and the alert definition on the right will be updated dynamically.
13
-
12
+3. The Alert Configuration Manager will open up with the default thresholds. You can modify the configuration as required and the alert definition on the right will be updated dynamically.
13
+ <!---->
14
15
-- If you want more fine-grained control or access to more advanced settings, enable `Show advanced`
16
-
15
+4. If you want more fine-grained control or access to more advanced settings, enable **Show advanced**
16
+ <!---->
17
18
-- Copy the alert definition that is generated in the code box and add it to an existing [health configuration file](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#edit-health-configuration-files) or a new custom file under `<path to netdata install>/etc/netdata/health.d/` on a `Parent Agent` or a `Standalone Child Agent`.
19
-
18
+5. Copy the alert definition that is generated in the code box and add it to an existing [health configuration file](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#edit-health-configuration-files) or a new custom file under `<path to netdata install>/etc/netdata/health.d/` on a Parent Agent or a Standalone Child Agent.
19
+ <!---->
20
21
-- Reload Netdata Alert Health checks `<path to netdata install>/usr/sbin/netdatacli reload-health` and the new alert is now configured.
21
+6. Reload Netdata Alert Health checks `<path to netdata install>/usr/sbin/netdatacli reload-health` and the new alert is now configured.
22
23
24
## Alerts Configuration Manager Sections
25
26
-- **Alert Name, Description and Summary Section**
27
-
28
-
29
- - **Alert Template Name**: This field uniquely identifies an alert and corresponds to the [template](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-alarm-or-template) field of the Alert configuration. The Alerts Configuration Manager provides a default name for an Alert template but we recommend you to modify this to have a meaningful name for your configured alert.
30
- - **Alert Template Description**: This field describes the alert and corresponds to the [info](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-info) line of the Alert configuration.
31
- - **Alert Summary**: This field enables the users to customize a title for the Alert Notifications (via [Notification integrations](https://learn.netdata.cloud/docs/alerting/notifications/centralized-cloud-notifications)) and corresponds to the [summary](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-summary) line of the Alert configuration.
26
+- **Alert Detection Method**
27
+ <!---->
28
+ An alert is triggered whenever a metric crosses a threshold:
29
+ - Based on a standard `threshold`
30
+ - Based on metric `variance`
31
+ - Based on the `anomaly rate` of the metric
32
+
33
33
-- **Alert Type**
34
-
35
- The users can select the type of Alert that they want to create:
36
- - Standard `threshold` based alerts.
37
- - `Variance` based alerts
38
- - `Anomalies` based alerts
34
+- **Metrics Lookup, Filtering and Formula Section**
35
+ <!---->
36
+
37
+ - **Metrics Lookup**: This is the **Evaluate** line of fields in the modal and it defines the parameters for the database lookup that is needed to get the value that will be compared against the alert definition. It corresponds to the [`lookup`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-lookup) line of the Alert configuration file. The Alerts Configuration Manager provides a default selection for the lookup and can be modified to suit your requirements. The parameters that can be modified are:
38
+ - METHOD (`avg`, `sum`, `min`, `max`, `cv`, `stddev`)
39
+ - COMPUTATION (sum of all dimensions or individually for each dimension)
40
+ - DIMENSIONS (All dimensions, or a selection of dimensions)
41
+ - DURATION (the period in time to run the lookup)
42
+ - OPTIONS (`absolute`, `unaligned`, `percentage`, `min2max`)
43
+
44
+ - **Alert Filtering**: This functionality can be accessed through the **Show advanced** checkbox and it allows for filtering the alert health checks to be run only for specific components of the infrastructure. It helps in achieving a fine-grained configuration for any given alert.
45
+ - `HOSTS` / `NODES` - By default all hosts are selected. You can pick nodes from the dropdown list, or enter a wildcard matching a list of hosts that you want the alert health check to run on. This field corresponds to the [`hosts`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-hosts) line of the Alert configuration file.
46
+ - `INSTANCES` - All instances are selected by default. You can pick instances from the dropdown list, or enter a wildcard matching a list of instances that you want the alert health check to run on. This field corresponds to the [`charts`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-charts) line of the Alert configuration file.
47
+ - `CHART LABELS` - All chart labels are selected by default. You can pick a chart label from the dropdown list or enter a wildcard matching a list of chart labels that you want the alert health check to run on. This field corresponds to the [`chart labels`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-chart-labels) line of the Alert configuration file.
48
+ - `OS` - All Operating Systems are selected by default. You can choose which OS(s) an alert health check should run on. This field corresponds to the [`os`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-os) line of the Alert configuration file.
49
+
50
+ - **Formula / Calculation**: This field is available through the **Show advanced** checkbox and it is used to define a formula to be run on top of the `lookup` value. The result of the lookup is available in the `$this` variable, and after the formula is run, the result is also stored in `$this` and can be accessed while setting the alert thresholds. This field corresponds to the [`calc`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-calc) line of the Alert configuration file.
51
+
52
+- **Alerting conditions**
53
+ <!---->
54
+ - **Warning and Critical Thresholds**: These fields are used to set the thresholds for the `Warning` and `Critical` alert states, while also having the option to set the condition for the alert to be raised if it is `above` or `below` the given threshold. If the advanced settings are selected, a **formula** option can also be used, to define a custom formula instead of a threshold. These fields correspond to the [`warn` and `crit`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-lines-warn-and-crit) lines of the Alert configuration file.
55
+ - **Recovery Thresholds**: This field is available through the **Show advanced** checkbox, and it is used to set the threshold that the metric value needs to meet to de-escalate from a given severity status, like `Critical to Warning` and from `Warning to Clear`. The logic is appended to the `warn` and `crit` lines of the Alert configuration file and resembles a one-line `IF-THEN-ELSE` clause.
56
+ - **Check Interval**: This field is used to define the frequency of the health check for the alert and corresponds to the [`every`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-every) line of the Alert configuration file.
57
+ - **Delay Notifications**: This field is available through the **Show advanced** checkbox and it is used to set delay parameters on notifications for an alert severity `escalation` or `de-escalation`. It corresponds to the [`delay`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-delay) line of the Alert configuration file.
58
+
59
+ - **Agent Specific Options**: These options are only available on the `Netdata Agent` and not honored on `Netdata Cloud`. They can be accessed through the **Show advanced** checkbox.
60
+ <!---->
61
+ - **Repeat Notifications**: This field defines the repeat frequency for the alert notification when the alert is in either `warning` or `critical` status and corresponds to the [`repeat`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-repeat) line of the Alert configuration file.
62
+ - **Send to**: This field is used to define a user role to which the alert notifications will be sent. If set to `silent`, then the alert won't be sent to any role. It corresponds to the [`to`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-to) line of the Alert configuration file.
63
+ - **Custom Exec Script**: This field is used to define a custom script that will be executed when the alert is triggered (but needs to be carefully designed as it needs to call the `health_alarm_notify.sh` module) and corresponds to the [`exec`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-exec) line of the Alert configuration file.
64
65
+- **Alert Name, Description and Summary Section**
66
+ <!---->
67
41
-- **Metrics Lookup, Filtering and Formula Section**
42
-
43
-
44
- - **Metrics Lookup**: This field defines the parameters of a database lookup that is needed to compute the value that will be compared against the alert definition and corresponds to the `[lookup](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-lookup)` line of the Alert configuration. The Alerts Configuration Manager provides a default selection for the lookup and can be modified to suit your requirements. The parameters that you are allowed to modify are:
45
- - METHOD `(avg, sum, min, max, cv, stddev)`
46
- - COMPUTATION `(sum of all dimensions or individually for each dimension)`
47
- - DIMENSIONS `(All dimensions, or a selection of dimensions)`
48
- - DURATION `(the period in time to run the lookup)` and
49
- - OPTIONS `(absolute, unaligned, percentage, min2max)`. This field is available in the `Basic Options`.
50
-
51
- - **Alert Filtering**: This field enables the users to filter the alert health checks to be run only for specific components of the infrastructure and helps in fine-grained configuration of the alerts. This is only available in the `Advance Options`.
52
- - `HOSTS / NODES` - By default all hosts are selected. You can choose / enter a wildcard matching a list of hosts you want this alert health check to run on. This field corresponds to the [hosts](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-hosts) line of the Alert configuration.
53
- - `INSTANCES` - By default all instances are selected. You can choose / enter a wildcard matching a list of instances you want this alert health check to run on. This field corresponds to the [charts](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-charts) line of the Alert configuration.
54
- - `CHART LABELS` - By default all chart labels are selected. You can choose a chart label and select or enter a wildcard matching a list of chart label values you want this alert health check to run on. This field corresponds to the [chart labels](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-chart-labels) line of the Alert configuration.
55
- - `OS` - By default all Operating Systems are selected. You can choose which OSes are relevant for this alert health check to run on. This field corresponds to the [os](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-os) line of the Alert configuration.
56
-
57
- - **Formula / Calculation**: This field enables the user to define a formula that will be run on top of the `Lookup` above. The result of the lookup is available in `$this` and after defining the formula, the result of the `formula` is also stored in `$this` and can be accessed while setting the alert thresholds. This field corresponds to the [calc](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-calc) line of the Alert configuration. This is only available in the `Advance Options`.
58
-
59
-- **Alert Thresholds**
60
-
61
- - **Warning and Critical Threshold**: This field enables the users to set a threshold to raise alerts as `Warning` or `Critical` and corresponds to the [warn](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-lines-warn-and-crit) line of the Alert configuration.
62
- - **Recovery Thresholds**: This field enables the users to set a different threshold to de-escalate the severity of an alert from `Critical to Warning` or `Warning to Clear`.
63
- - **Delay Notifications**: This field enables the users to set a delay on notifications for an alert severity `escalation` or `de-escalation` and corresponds to the [delay](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-delay) line of the Alert configuration.
64
- - **Check Interval**: This field enables the users to define the frequency of the health check for the alert being defined and corresponds to the [every](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-every) line of the Alert configuration.
65
-
66
-- **Agent Specific Options**
67
- These options are only available on the `Netdata Agent` and not honored on `Netdata Cloud`.
68
-
69
- - **Repeat Notifications**: This field allows the users to define if they want to repeat sending the `alert notifications` when the alert is active and corresponds to the [repeat](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-repeat) line of the Alert configuration.
70
- - **Send to**: This field enables the users to define a `user role` to which the alert notifications need to be sent or `silence` the notifications and corresponds to the [to](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-to) line of the Alert configuration.
71
- - **Custom Exec Script**: This field enables the users to define a custom script to be executed when the alert is triggered (but needs to be carefully designed as it needs to call the `health_alarm_notify.sh` module) and corresponds to the [exec](https://learn.netdata.cloud/docs/alerting/health-configuration-reference#alert-line-exec) line of the Alert Configuration.
68
+ - **Alert Template Name**: This field uniquely identifies an alert and corresponds to the [`template`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-alarm-or-template) field of the Alert configuration file.
69
+ The Alerts Configuration Manager provides a default name for an Alert template but we recommend you modify this to have a meaningful name for your configured alert.
70
+ - **Alert Template Description**: This field provides a brief explanation of the alert and corresponds to the [`info`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-info) line of the Alert configuration file.
71
+ - **Alert Summary**: This field enables the users to create a custom title for the alert notification (via [Notification integrations](https://learn.netdata.cloud/docs/alerting/notifications/centralized-cloud-notifications)) and corresponds to the [`summary`](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alert-line-summary) line of the Alert configuration file.