|
1
|
# Notifications |
|
2
|
|
|
3
|
Netdata supports two ways to send alert notifications: **from Netdata Cloud** or **from the Netdata Agent**. |
|
4
|
|
|
5
|
You can use either or both depending on how your infrastructure is set up. |
|
6
|
|
|
7
|
:::tip |
|
8
|
|
|
9
|
Need alerts *fast*? |
|
10
|
Use Cloud for a centralized setup, or Agent for full control on each node. |
|
11
|
|
|
12
|
::: |
|
13
|
|
|
14
|
## Notification Methods |
|
15
|
|
|
16
|
### Netdata Cloud (Centralized) |
|
17
|
|
|
18
|
Netdata Cloud collects alert data from all connected nodes and sends notifications through your configured integrations. |
|
19
|
|
|
20
|
[See supported Cloud integrations](https://learn.netdata.cloud/docs/alerts-&-notifications/notifications/centralized-cloud-notifications) |
|
21
|
|
|
22
|
**Popular integrations:** |
|
23
|
|
|
24
|
- Amazon SNS |
|
25
|
- Slack |
|
26
|
- Discord |
|
27
|
- Splunk |
|
28
|
- Microsoft Teams |
|
29
|
- Mattermost |
|
30
|
|
|
31
|
### Netdata Agent (Local) |
|
32
|
|
|
33
|
The Agent sends alerts directly from the node, even if it's offline or not connected to the Cloud. |
|
34
|
|
|
35
|
[See supported Agent integrations](https://learn.netdata.cloud/docs/alerts-&-notifications/notifications/agent-dispatched-notifications) |
|
36
|
|
|
37
|
**Popular integrations:** |
|
38
|
|
|
39
|
- Email |
|
40
|
- Slack |
|
41
|
- PagerDuty |
|
42
|
- Twilio |
|
43
|
- Telegram |
|
44
|
- Opsgenie |
|
45
|
|
|
46
|
## Alert Intelligence and Severity Levels |
|
47
|
|
|
48
|
Netdata's alerts are designed to minimize false positives and prevent alarm fatigue. |
|
49
|
|
|
50
|
### Alert Severity Levels |
|
51
|
|
|
52
|
| Level | Description | Typical Action | |
|
53
|
|--------------|-------------------------------------------------------------------|-----------------------------------| |
|
54
|
| **CLEAR** | The metric has returned to normal range | No action needed | |
|
55
|
| **WARNING** | The metric shows concerning behavior that requires attention | Investigate during business hours | |
|
56
|
| **CRITICAL** | The metric indicates a serious problem requiring immediate action | Immediate response required | |
|
57
|
|
|
58
|
These severity levels help you prioritize your response and can be routed to different notification channels based on urgency. |
|
59
|
|
|
60
|
### Preventing Alert Fatigue |
|
61
|
|
|
62
|
| Feature | Benefit | |
|
63
|
|---------------------------|-----------------------------------------------------------------------| |
|
64
|
| **Intelligent Defaults** | Thresholds carefully selected based on real-world experience | |
|
65
|
| **Hysteresis Protection** | Prevents notification floods when metrics fluctuate around thresholds | |
|
66
|
| **Notification Delays** | Configurable delays ensure transient issues don't trigger alerts | |
|
67
|
| **Role-Based Routing** | Ensures alerts reach only the appropriate stakeholders | |
|
68
|
|
|
69
|
:::tip |
|
70
|
|
|
71
|
You can configure different notification channels for different severity levels. For example, you can send WARNING alerts to Slack and CRITICAL alerts to PagerDuty. |
|
72
|
|
|
73
|
::: |
|
74
|
|
|
75
|
## Troubleshooting Assistance |
|
76
|
|
|
77
|
When you receive an alert, Netdata provides tools to help you understand and resolve the issue: |
|
78
|
|
|
79
|
### Netdata Assistant |
|
80
|
|
|
81
|
The [Netdata Assistant](https://learn.netdata.cloud/docs/machine-learning-and-anomaly-detection/ai-powered-troubleshooting-assistant) is an AI-powered feature that guides you through troubleshooting alerts by providing: |
|
82
|
|
|
83
|
- Clear explanations of what the alert means |
|
84
|
- Assessment of potential causes |
|
85
|
- Recommended troubleshooting steps |
|
86
|
- Links to relevant documentation |
|
87
|
|
|
88
|
The Assistant window follows you as you navigate through dashboards, making troubleshooting faster and more efficient. |
|
89
|
|
|
90
|
### Community Resources |
|
91
|
|
|
92
|
For more complex issues, you can access the [Alerts Troubleshooting space](https://community.netdata.cloud/c/alerts/28) in our community forum, where you'll find: |
|
93
|
|
|
94
|
- Detailed information about all built-in alerts |
|
95
|
- Recommended troubleshooting actions from experts |
|
96
|
- A searchable history of previously solved issues |
|
97
|
- Ability to ask questions and share your own solutions |
|
98
|
|
|
99
|
## Which One Should I Use? |
|
100
|
|
|
101
|
Choose the option that fits your needs: |
|
102
|
|
|
103
|
| Use Case | Best Option | |
|
104
|
|---------------------------------------|---------------| |
|
105
|
| Manage multiple nodes centrally | Netdata Cloud | |
|
106
|
| Fewer configs, alerts from one place | Netdata Cloud | |
|
107
|
| Full control at node level | Netdata Agent | |
|
108
|
| No internet or external dependencies | Netdata Agent | |
|
109
|
| Fine-tuned control per system/service | Netdata Agent | |
|
110
|
| Want both simplicity and flexibility | Use **both** | |
|
111
|
|
|
112
|
## Creating and configuring alerts |
|
113
|
|
|
114
|
Netdata ships with hundreds of pre-configured alerts. You can also create your own or tune existing ones: |
|
115
|
|
|
116
|
- **[Alerts Automation](/docs/netdata-ai/alerts-automation/alerts-automation.md)** — Describe what you want in plain English; AI suggests alerts, generates the configuration, and tests it against historical data before deployment |
|
117
|
- **[Alerts Configuration Manager](/docs/alerts-and-notifications/creating-alerts-with-netdata-alerts-configuration-manager.md)** — Visual UI wizard for creating and editing alerts |
|
118
|
- **[Manual configuration](/src/health/REFERENCE.md)** — Edit `health.d/*.conf` files directly for full control over alert syntax |
|
119
|
|
|
120
|
## Next Steps |
|
121
|
|
|
122
|
- [Set up Cloud Notifications](/docs/alerts-and-notifications/notifications/centralized-cloud-notifications/centralized-cloud-notifications-reference.md) |
|
123
|
- [Set up Agent Notifications](/src/health/notifications/README.md) |
|
124
|
- [Create alerts with AI](/docs/netdata-ai/alerts-automation/alerts-automation.md) |