Netdata ai (#20309)
Co-authored-by: ilyam8 <ilya@netdata.cloud>
kanelatechnical committed
May 20, 2025 at 19:43 UTC
8da25ac7a9897f4d2f55005baaeb2c2d2b799796
5 files changed
+669
-146
docs/category-overview-pages/machine-learning-and-assisted-troubleshooting.md
+45
-10
@@ -1,19 +1,54 @@
1
-# Machine Learning and Anomaly Detection
1
+# Netdata AI
2
3
-Netdata includes advanced Machine Learning capabilities to help you detect and resolve anomalies in your infrastructure before they escalate into critical issues. These features provide real-time insights and proactive monitoring to improve system reliability.
3
+Boost your monitoring and troubleshooting capabilities with Netdata's AI-powered features.
4
5
-## Key Features
5
+Netdata AI helps you **detect anomalies, understand metric relationships, and resolve issues quickly** with intelligent assistance all designed to make your infrastructure management smarter, faster, and bulletproof.
6
7
-### Anomaly Detection with K-Means Clustering
7
+## What Can Netdata AI Do For You?
8
9
-Netdata trains K-means clustering models to detect anomalies in your infrastructure. These models power the [Anomaly Advisor](/docs/dashboards-and-charts/anomaly-advisor-tab.md), which visually highlights anomalies on the dashboard, allowing you to quickly identify and investigate unexpected behavior.
9
+Netdata AI combines powerful machine learning capabilities with intuitive interfaces to help you:
10
11
-### Metric Correlations
11
+1. **Detect anomalies automatically** before they escalate into critical issues
12
+2. **Understand relationships** between metrics during troubleshooting
13
+3. **Get expert guidance** when resolving alerts and performance problems
14
13
-Netdata enables metric correlation analysis through the dashboard. This feature uses the [Two-sample Kolmogorov-Smirnov test](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test#Two-sample_Kolmogorov%E2%80%93Smirnov_test) and volume heuristic measures to help you understand relationships between different metrics and identify potential causes of anomalies.
15
+## Machine Learning and Anomaly Detection
16
15
-### Netdata Assistant for Troubleshooting
17
+Our ML-powered anomaly detection works silently in the background, monitoring your metrics and identifying unusual patterns.
18
17
-The [Netdata Assistant](/docs/netdata-assistant.md) provides AI-driven assistance for troubleshooting alerts and anomalies. You can interact with it directly to get explanations, recommendations, and next steps based on detected anomalies and system behavior.
19
+| Feature | What It Does For You |
20
+|----------------------------------|------------------------------------------------------------------------------|
21
+| **Unsupervised Learning** | Works automatically without requiring manual training or labeling of data |
22
+| **Multiple Model Consensus** | Reduces false positives by 99% by requiring agreement across multiple models |
23
+| **Real-time Anomaly Bits** | Flags unusual metrics instantly, with zero storage overhead |
24
+| **Anomaly Rate Visualization** | Highlights anomalous time periods in your dashboard for quick investigation |
25
+| **Node-Level Anomaly Detection** | Identifies when your entire system is behaving unusually |
26
+| **Metric Correlations** | Helps you find relationships between metrics to pinpoint root causes |
27
19
-These Machine Learning features enhance observability and streamline incident response, helping you maintain system health with greater efficiency.
\ No newline at end of file
28
+Learn more in the [Machine Learning and Anomaly Detection](/src/ml/README.md) documentation.
29
+
30
+## Netdata Assistant
31
+
32
+When alerts trigger or anomalies emerge, Netdata Assistant serves as your AI-powered troubleshooting companion.
33
+
34
+| Feature | What It Does For You |
35
+|----------------------------|-----------------------------------------------------------------------|
36
+| **Alert Context** | Explains what each alert means and why you should care about it |
37
+| **Guided Troubleshooting** | Offers step-by-step instructions tailored to your specific situation |
38
+| **Persistent Window** | Follows you throughout your dashboards as you investigate issues |
39
+| **Curated Resources** | Provides links to relevant documentation to deepen your understanding |
40
+| **Time-Saving** | Eliminates the need for searching documentation or online forums |
41
+
42
+Learn more about [Netdata Assistant](/docs/netdata-assistant.md) and how it helps streamline your troubleshooting workflow.
43
+
44
+## Getting Started
45
+
46
+Netdata AI features are enabled by default with the standard installation. The machine learning capabilities require the `dbengine` database mode, which is the default setting.
47
+
48
+To start exploring:
49
+
50
+1. **Anomaly Detection**: Check the [Anomaly Advisor tab](/docs/dashboards-and-charts/anomaly-advisor-tab.md) to see detected anomalies
51
+2. **Metric Correlations**: Use the Metric Correlations button in the dashboard to analyze relationships between metrics
52
+3. **Netdata Assistant**: Click the Assistant button in the Alerts tab when troubleshooting alerts
53
+
54
+These AI features work seamlessly with Netdata's other capabilities, enhancing your overall monitoring and troubleshooting experience without requiring any AI expertise.
docs/metric-correlations.md
+262
-31
@@ -1,66 +1,297 @@
1
# Metric Correlations
2
3
-The Metric Correlations feature helps you quickly identify metrics and charts relevant to a specific time window of interest, allowing for faster root cause analysis.
3
+The **Metric Correlations** feature helps you quickly identify metrics and charts relevant to a specific time window of interest, allowing for faster root cause analysis.
4
5
-By filtering the standard Netdata dashboard to display only the most relevant charts, Metric Correlations makes it easier to pinpoint anomalies and investigate issues.
5
+:::tip
6
7
-Since it leverages every available metric in your infrastructure with up to 1-second granularity, Metric Correlations provides highly accurate insights.
7
+By filtering your standard Netdata dashboard to **display only the most relevant charts**, Metric Correlations make it easier for you to pinpoint anomalies and investigate issues.
8
+
9
+:::
10
+
11
+Since it leverages every available metric in your infrastructure with up to 1-second granularity, **Metric Correlations provides you with highly accurate insights**.
12
13
## Using Metric Correlations
14
15
When viewing the [Metrics tab or a single-node dashboard](/docs/dashboards-and-charts/metrics-tab-and-single-node-tabs.md), you'll find the **Metric Correlations** button in the top-right corner.
16
13
-To start:
17
+<details>
18
+<summary><strong>To start:</strong></summary><br/>
19
20
1. Click **Metric Correlations**.
16
-2. [Highlight a selection of metrics](/docs/dashboards-and-charts/netdata-charts.md#highlight) on a single chart. The selected timeframe must be at least 15 seconds.
17
-3. The menu displays details about the selected area and reference baseline. Metric Correlations compares the highlighted window to a reference baseline, which is four times its length and precedes it immediately.
18
-4. Click **Find Correlations**. This button is only active if a valid timeframe is selected.
19
-5. The process evaluates all available metrics and returns a filtered Netdata dashboard showing only the most changed metrics between the baseline and the highlighted window.
21
+2. Highlight a selection of metrics on a single chart. **The selected timeframe must be at least 15 seconds**.
22
+3. The menu displays details about your selected area and reference baseline. Metric Correlations compares your highlighted window to a reference baseline, which is four times its length and precedes it immediately.
23
+4. Click **Find Correlations**.
24
+
25
+:::note
26
+
27
+This button is only active if you've selected a valid timeframe.
28
+
29
+:::
30
+
31
+5. **The process evaluates all your available metrics and returns a filtered Netdata dashboard** showing only the most changed metrics between the baseline and your highlighted window.
32
6. If needed, select another window and press **Find Correlations** again to refine your analysis.
33
34
+</details>
35
+
36
+## Integration with Anomaly Detection
37
+
38
+You can combine Metric Correlations with Anomaly Detection for powerful troubleshooting:
39
+
40
+:::tip
41
+
42
+When you notice an anomaly in your system, use Metric Correlations with the **Anomaly Rate** data type to quickly identify which metrics are contributing to the anomalous behavior.
43
+
44
+:::
45
+
46
+### How to Use Together
47
+
48
+```mermaid
49
+flowchart TD
50
+ %% Node styling
51
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
52
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
53
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
54
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
55
+
56
+ A[Spot a spike in the<br/>node anomaly rate chart] --> B[Highlight that<br/>time period]
57
+ B --> C[Select Anomaly Rate<br/>as data type<br/>and Volume as method]
58
+ C --> D[Click Find Correlations]
59
+ D --> E[Review metrics with<br/>highest anomaly rates]
60
+ E --> F[Examine these metrics<br/>in detail to determine<br/>root cause]
61
+
62
+ %% Apply styles
63
+ class A,B neutral
64
+ class C,D warning
65
+ class E success
66
+ class F danger
67
+```
68
+
69
+:::tip
70
+
71
+**This workflow helps you move from detecting** that *"something is wrong"* **to understanding** exactly which components are behaving abnormally, significantly reducing your troubleshooting time.
72
+
73
+:::
74
+
75
+## API Access
76
+
77
+You can access anomaly detection data and use it with metric correlations through Netdata's API:
78
+
79
+<details>
80
+<summary><strong>Querying Anomaly Bits</strong></summary><br/>
81
+
82
+To get the anomaly bits for any metric, add the `options=anomaly-bit` parameter to your API query:
83
+
84
+```
85
+https://your-netdata-node/api/v1/data?chart=system.cpu&dimensions=user&after=-60&options=anomaly-bit
86
+```
87
+
88
+Sample response:
89
+
90
+```json
91
+{
92
+ "labels": [
93
+ "time",
94
+ "user"
95
+ ],
96
+ "data": [
97
+ [
98
+ 1684852570,
99
+ 0
100
+ ],
101
+ [
102
+ 1684852569,
103
+ 0
104
+ ],
105
+ [
106
+ 1684852568,
107
+ 0
108
+ ],
109
+ [
110
+ 1684852567,
111
+ 0
112
+ ],
113
+ [
114
+ 1684852566,
115
+ 0
116
+ ],
117
+ [
118
+ 1684852565,
119
+ 0
120
+ ],
121
+ [
122
+ 1684852564,
123
+ 0
124
+ ],
125
+ [
126
+ 1684852563,
127
+ 0
128
+ ],
129
+ [
130
+ 1684852562,
131
+ 0
132
+ ],
133
+ [
134
+ 1684852561,
135
+ 0
136
+ ]
137
+ ]
138
+}
139
+```
140
+
141
+</details>
142
+
143
+<details>
144
+<summary><strong>Querying Anomaly Rates</strong></summary><br/>
145
+
146
+For anomaly rates over a time window, use the same parameter but with aggregated data:
147
+
148
+```
149
+https://your-netdata-node/api/v1/data?chart=system.cpu&dimensions=user&after=-600&before=0&points=10&options=anomaly-bit
150
+```
151
+
152
+Sample response showing the percentage of time each metric was anomalous:
153
+
154
+```json
155
+{
156
+ "labels": [
157
+ "time",
158
+ "user"
159
+ ],
160
+ "data": [
161
+ [
162
+ 1684852770,
163
+ 0
164
+ ],
165
+ [
166
+ 1684852710,
167
+ 20
168
+ ],
169
+ [
170
+ 1684852650,
171
+ 0
172
+ ],
173
+ [
174
+ 1684852590,
175
+ 10
176
+ ],
177
+ [
178
+ 1684852530,
179
+ 0
180
+ ],
181
+ [
182
+ 1684852470,
183
+ 0
184
+ ],
185
+ [
186
+ 1684852410,
187
+ 30
188
+ ],
189
+ [
190
+ 1684852350,
191
+ 0
192
+ ],
193
+ [
194
+ 1684852290,
195
+ 0
196
+ ],
197
+ [
198
+ 1684852230,
199
+ 0
200
+ ]
201
+ ]
202
+}
203
+```
204
+
205
+</details>
206
+
207
+:::tip
208
+
209
+You can programmatically access this data to build custom dashboards or alerts based on anomaly patterns in your infrastructure.
210
+
211
+:::
212
+
213
## Metric Correlations Options
214
24
-Metric Correlations offers adjustable parameters for deeper data exploration. Since different data types and incidents require different approaches, these settings allow for flexible analysis.
215
+Metric Correlations offer adjustable parameters for deeper data exploration. Since different data types and incidents require different approaches, **these settings allow for flexible analysis**.
216
26
-### Method
217
+<details>
218
+<summary><strong>Method</strong></summary><br/>
219
220
Two algorithms are available for scoring metrics based on changes between the baseline and highlight windows:
221
30
-- **`KS2` (Kolmogorov-Smirnov Test)**: A statistical method comparing distributions between the highlighted and baseline windows to detect significant changes. [Implementation details](https://github.com/netdata/netdata/blob/d917f9831c0a1638ef4a56580f321eb6c9a88037/database/metric_correlations.c#L212).
31
-- **`Volume`**: A heuristic approach based on percentage change in averages, designed to handle edge cases. [Implementation details](https://github.com/netdata/netdata/blob/d917f9831c0a1638ef4a56580f321eb6c9a88037/database/metric_correlations.c#L516).
222
+* **`KS2` (Kolmogorov-Smirnov Test)**: A statistical method comparing distributions between the highlighted and baseline windows to detect significant changes. [Implementation details](https://github.com/netdata/netdata/blob/d917f9831c0a1638ef4a56580f321eb6c9a88037/database/metric_correlations.c#L212).
223
+* **`Volume`**: A heuristic approach based on percentage change in averages, designed to handle edge cases. [Implementation details](https://github.com/netdata/netdata/blob/d917f9831c0a1638ef4a56580f321eb6c9a88037/database/metric_correlations.c#L516).
224
+
225
+</details>
226
+
227
+<details>
228
+<summary><strong>Aggregation</strong></summary><br/>
229
33
-### Aggregation
230
+To accommodate different window lengths, Netdata aggregates your raw data as needed. The default aggregation method is `Average`, but you can also choose `Median`, `Min`, `Max`, or `Stddev`.
231
+</details>
232
35
-To accommodate different window lengths, Netdata aggregates raw data as needed. The default aggregation method is `Average`, but you can also choose `Median`, `Min`, `Max`, or `Stddev`.
233
+<details>
234
+<summary><strong>Data Type</strong></summary><br/>
235
37
-### Data Type
236
+Netdata assigns an [Anomaly Bit](https://github.com/netdata/netdata/tree/master/src/ml#anomaly-bit) to each of your metrics in real-time, flagging whether it deviates significantly from normal behavior. You can analyze either raw data or anomaly rates:
237
39
-Netdata assigns an [Anomaly Bit](https://github.com/netdata/netdata/tree/master/src/ml#anomaly-bit) to each metric in real-time, flagging whether it deviates significantly from normal behavior. You can analyze either raw data or anomaly rates:
238
+* **`Metrics`**: Runs Metric Correlations on your raw metric values.
239
+* **`Anomaly Rate`**: Runs Metric Correlations on anomaly rates for each of your metrics.
240
41
-- **`Metrics`**: Runs Metric Correlations on raw metric values.
42
-- **`Anomaly Rate`**: Runs Metric Correlations on anomaly rates for each metric.
241
+</details>
242
243
## Metric Correlations on the Agent
244
245
Metric Correlations (MC) requests to Netdata Cloud are handled in two ways:
246
48
-1. If MC is enabled on any node, the request is routed to the highest-level node (a Parent node or the node itself).
49
-2. If MC is not enabled on any node, Netdata Cloud processes the request by collecting data from nodes and computing correlations on its backend.
247
+1. **If MC is enabled** on any of your nodes, the request is routed to the highest-level node (a Parent node or the node itself).
248
+2. **If MC is not enabled** on any of your nodes, Netdata Cloud processes the request by collecting data from your nodes and computing correlations on its backend.
249
+
250
+## Interpreting Combined Results
251
+
252
+When you use Metric Correlations together with Anomaly Detection, you'll want to understand how to interpret the results:
253
+
254
+:::tip
255
+
256
+**High anomaly rates combined with significant metric changes** often indicate genuine issues rather than false positives.
257
+
258
+:::
259
+
260
+Here's how to interpret different scenarios:
261
+
262
+| Anomaly Rate | Metric Correlation | Interpretation |
263
+|--------------|--------------------|------------------------------------------------------|
264
+| High | Strong | Likely a significant issue affecting system behavior |
265
+| High | Weak | Possible edge case or intermittent issue |
266
+| Low | Strong | Normal but significant change in system behavior |
267
+| Low | Weak | Likely normal system operation |
268
+
269
+:::tip
270
+
271
+By examining both the anomaly rate and the correlation strength, you can prioritize your troubleshooting efforts more effectively.
272
+
273
+:::
274
275
## Usage Tips
276
53
-- When running Metric Correlations from the [Metrics tab](/docs/dashboards-and-charts/metrics-tab-and-single-node-tabs.md) across multiple nodes, refine your results by grouping by node:
54
- 1. Run MC on all nodes if you're unsure which ones are relevant.
55
- 2. Group the most interesting charts by node to determine whether changes affect all nodes or just a subset.
56
- 3. If a subset of nodes stands out, filter for those nodes and rerun MC to get more precise results.
277
+:::tip
278
+
279
+When running Metric Correlations from the [Metrics tab](/docs/dashboards-and-charts/metrics-tab-and-single-node-tabs.md) across multiple nodes, refine your results by grouping by node:
280
+
281
+1. Run MC on all your nodes if you're unsure which ones are relevant.
282
+2. Group the most interesting charts by node to determine whether changes affect all your nodes or just a subset.
283
+3. If a subset of your nodes stands out, filter for those nodes and rerun MC to get more precise results.
284
+
285
+Choose the **`Volume`** algorithm for sparse metrics (e.g., request latency with few requests). Otherwise, use **`KS2`**.
286
+
287
+- **`KS2`** is ideal for detecting complex distribution changes in your metrics, such as shifts in variance.
288
+- **`Volume`** is better for detecting your metrics that were inactive and then spiked (or vice versa).
289
+
290
+**Example:**
291
58
-- Choose the **`Volume`** algorithm for sparse metrics (e.g., request latency with few requests). Otherwise, use **`KS2`**.
59
- - **`KS2`** is ideal for detecting complex distribution changes, such as shifts in variance.
60
- - **`Volume`** is better for detecting metrics that were inactive and then spiked (or vice versa).
292
+- `Volume` can highlight network traffic suddenly turning on in your system.
293
+- `KS2` can detect entropy distribution changes in your data missed by `Volume`.
294
62
- **Example:**
63
- - `Volume` can highlight network traffic suddenly turning on.
64
- - `KS2` can detect entropy distribution changes missed by `Volume`.
295
+Combine **`Volume`** and **`Anomaly Rate`** to identify the most anomalous metrics within your selected timeframe. Expand the anomaly rate chart to visualize results more clearly.
296
66
-- Combine **`Volume`** and **`Anomaly Rate`** to identify the most anomalous metrics within a timeframe. Expand the anomaly rate chart to visualize results more clearly.
\ No newline at end of file
297
+:::
docs/netdata-assistant.md
+78
-3
@@ -1,9 +1,22 @@
1
# Alert Troubleshooting with Netdata Assistant
2
3
-The Netdata Assistant leverages large language models and community knowledge to simplify alert troubleshooting and root cause analysis.
3
+**The Netdata Assistant leverages large language models and community knowledge** to simplify alert troubleshooting and root cause analysis.
4
+
5
+This AI-powered tool helps you understand alerts quickly, **especially during critical situations**.
6
+
7
+| Feature | Benefit |
8
+|---------------------------|-----------------------------------------------------------------------------------------------------------------------|
9
+| **Follows Your Workflow** | The Assistant window stays with you as you navigate through Netdata dashboards during your troubleshooting process. |
10
+| **Works at Any Hour** | Especially valuable during after-hours emergencies when you might not have team support available. |
11
+| **Contextual Knowledge** | Combines Netdata's community expertise with the power of large language models to provide relevant advice. |
12
+| **Time-Saving** | Eliminates the need for searches across multiple documentation sources or community forums. |
13
+| **Non-Intrusive** | Provides helpful guidance without taking control away from you - you remain in charge of the troubleshooting process. |
14
15
## Using Netdata Assistant
16
17
+<details>
18
+<summary><strong>Accessing the Assistant</strong></summary><br/>
19
+
20
1. Navigate to the **Alerts** tab.
21
2. If there are active alerts, the **Actions** column will have an **Assistant** button.
22
@@ -11,10 +24,72 @@ The Netdata Assistant leverages large language models and community knowledge to
24
25
3. Click the **Assistant** button to open a floating window with tailored troubleshooting insights.
26
27
+4. If there are no active alerts, you can still access the Assistant from the **Alert Configuration** view.
28
+
29
+</details>
30
+
31
+<details>
32
+<summary><strong>Understanding Assistant Information</strong></summary><br/>
33
+
34
+When you open the Assistant, you'll see:
35
+
36
+1. **Alert Context**: Explanation of what the alert means and why it's occurring
37
+
38
39
16
-4. If you need more details, the Assistant provides useful resource links to help with further investigation.
40
+2. **Troubleshooting Steps**: Recommended actions to address the issue
41
+
42
+3. **Importance Level**: Context on how critical this alert is for your system
43
+
44
+4. **Resource Links**: Curated documentation and external resources for further investigation
45
46
47
20
-5. If there are no active alerts, you can still access the Assistant from the **Alert Configuration** view.
\ No newline at end of file
48
+</details>
49
+
50
+## How Netdata Assistant Helps You
51
+
52
+:::tip
53
+
54
+Netdata Assistant is designed to reduce your troubleshooting time by providing contextual information exactly when you need it.
55
+
56
+:::
57
+
58
+<div style="background-color: #f8f9fa; padding: 15px; border-radius: 5px; margin-bottom: 20px;">
59
+<p><strong>🔍 Immediate Alert Context</strong> - Get explanations of what alerts mean without searching online</p>
60
+<p><strong>⚠️ Impact Assessment</strong> - Understand why the alert matters to your system's health</p>
61
+<p><strong>🛠️ Guided Troubleshooting</strong> - Receive customized steps for your specific situation</p>
62
+<p><strong>📚 Curated Resources</strong> - Access relevant documentation for deeper investigation</p>
63
+<p><strong>🔄 Persistent Assistance</strong> - Keep the Assistant window with you throughout your troubleshooting journey</p>
64
+</div>
65
+
66
+## Practical Example
67
+
68
+Here's how Netdata Assistant can help in a real-world scenario:
69
+
70
+```mermaid
71
+flowchart LR
72
+ %% Node styling
73
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
74
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
75
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
76
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
77
+
78
+ A["🕒 3 AM Alert<br/>load average 15"] --> B["Without Assistant:<br/>🔍 Google searches"]
79
+ A --> C["With Assistant:<br/>🤖 Click Assistant button"]
80
+
81
+ B --> D["🕰️ Time wasted<br/>Stress increased"]
82
+
83
+ subgraph AssistantProcess ["Assistant Process"]
84
+ direction LR
85
+ E["📊 Explanation of<br/>system load cause"] --> F["🛠️ Specific troubleshooting<br/>steps provided"] --> G["🔄 Assistant follows as<br/>you check metrics"] --> H["📚 Quick access to<br/>additional resources"] --> I["⚡ Issue resolved faster<br/>with confidence"]
86
+ end
87
+
88
+ C --> E
89
+
90
+ %% Apply styles
91
+ class A,B,D danger
92
+ class C,E,F,G,H,I success
93
+```
94
+
95
+By using Netdata Assistant, you can resolve issues faster and with more confidence, even during stressful situations.
src/ml/README.md
+123
-49
@@ -1,62 +1,140 @@
1
-# Machine Learning Models and Anomaly Detection in Netdata
1
+# Machine Learning and Anomaly Detection
2
3
## Overview
4
5
-Machine learning helps detect patterns and anomalies in large datasets, enabling early issue identification before they escalate.
5
+You can leverage machine learning to detect patterns and anomalies in your large datasets, enabling you to identify issues early before they escalate.
6
7
-At Netdata, we developed **Anomaly Advisor**, a tool designed to improve troubleshooting, reduce mean time to resolution, and prevent issues from escalating. You can access it through the [Netdata dashboard](/docs/dashboards-and-charts/README.md).
7
+Netdata offers **Anomaly Advisor**, a tool designed to improve your troubleshooting experience, reduce mean time to resolution, and prevent issues from escalating. You can access it through the [Netdata dashboard](/docs/dashboards-and-charts/README.md).
8
9
-> **Note**
10
->
11
-> To configure ML on your nodes, check the [ML configuration documentation](/src/ml/ml-configuration.md).
9
+:::tip
10
13
----
11
+To configure ML on your nodes, check the [ML configuration documentation](/src/ml/ml-configuration.md).
12
+
13
+:::
14
15
## Design Principles
16
17
-Netdata’s machine learning models follow these key principles:
17
+When you use Netdata's machine learning models, you benefit from these key principles:
18
+
19
+| Principle | Description |
20
+|----------------------------|----------------------------------------------------------------------------------------------------------------------------|
21
+| **Unsupervised Learning** | Models operate independently without requiring your input |
22
+| **Real-time Performance** | While ML impacts CPU usage, you won't experience any compromise to Netdata's high-fidelity, real-time monitoring |
23
+| **Seamless Integration** | ML-based insights are fully embedded into your existing Netdata infrastructure monitoring and troubleshooting workflow |
24
+| **Assistance Over Alerts** | ML helps you investigate potential issues rather than triggering unnecessary alerts - no 3 AM wake-ups for minor anomalies |
25
+| **Many Light Models** | Netdata uses many lightweight models instead of a few heavy ones, optimizing for resource usage while maintaining accuracy |
26
+| **Scalable Architecture** | The system is designed to handle thousands of metrics simultaneously, scoring each one every second with minimal latency |
27
+
28
+:::note
29
+
30
+Netdata deliberately avoids using deep learning models, as they would introduce heavy dependencies and resource requirements that wouldn't align with Netdata's goal of running efficiently on any Linux system. Instead, the implementation uses the lightweight [dlib](https://github.com/davisking/dlib) library and spreads training costs over a wide window to minimize performance impact.
31
+
32
+:::
33
+
34
+## Types of Anomalies You Can Detect
35
+
36
+| Anomaly Type | Description | Business Impact |
37
+|--------------------------|-------------------------------------------------------------------|------------------------------------------|
38
+| **Point Anomalies** | Unusually high or low values compared to historical data | Early warning of service degradation |
39
+| **Contextual Anomalies** | Sequences of values that deviate from expected patterns | Identification of unusual usage patterns |
40
+| **Collective Anomalies** | Multivariate anomalies where a combination of metrics appears off | Detection of complex system issues |
41
+| **Concept Drifts** | Gradual shifts leading to a new baseline | Recognition of evolving system behavior |
42
+| **Change Points** | Sudden shifts resulting in a new normal state | Identification of system changes |
43
+
44
+## How Netdata ML Works
45
+
46
+```mermaid
47
+flowchart TD
48
+ %% Node styling
49
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
50
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
51
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
52
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
53
+
54
+ %% Training Process
55
+ subgraph TrainingProcess["Training Process"]
56
+ direction TB
57
+ Raw["Raw Metrics<br/>(Last 4 Hours)"] --> Preprocess["Preprocess<br/>Feature Vectors"]
58
+ Preprocess --> Train["Train k-means<br/>(k=2)"]
59
+ Train --> Model["Trained Model"]
60
+ end
61
+
62
+ %% Multiple Models
63
+ subgraph MultipleModels["Multiple Models Over Time"]
64
+ direction TB
65
+ M1["Model 1<br/>Recent Data"]
66
+ M2["Model 2<br/>Older Data"]
67
+ M3["Model 3<br/>Even Older Data"]
68
+ MN["Model N<br/>Up to 2 Days Old"]
69
+ end
70
+
71
+ %% Detection Process
72
+ subgraph DetectionProcess["Detection Process"]
73
+ direction TB
74
+ NewData["New Metrics"] --> DistCalc["Calculate Euclidean Distance<br/>to Cluster Centers"]
75
+ DistCalc --> Threshold{"Distance > 99th<br/>Percentile?"}
76
+ Threshold -->|Yes| FlagA["Flag as Anomalous<br/>in This Model"]
77
+ Threshold -->|No| FlagN["Flag as Normal<br/>in This Model"]
78
+ end
79
+
80
+ %% Consensus
81
+ subgraph Consensus["Consensus Decision"]
82
+ direction TB
83
+ AllResults["Results from All Models"] --> AllAgree{"All Models<br/>Agree it's<br/>Anomalous?"}
84
+ AllAgree -->|Yes| SetBit["Set Anomaly Bit = 100<br/>(True)"]
85
+ AllAgree -->|No| ClearBit["Set Anomaly Bit = 0<br/>(False)"]
86
+ end
87
+
88
+ %% Flow between subgraphs
89
+ TrainingProcess --> MultipleModels
90
+ MultipleModels --> DetectionProcess
91
+ DetectionProcess --> Consensus
92
+
93
+ %% Apply styles
94
+ class Raw,Preprocess,NewData,AllResults neutral
95
+ class Train,Model,M1,M2,M3,MN,DistCalc success
96
+ class Threshold,FlagA,FlagN,AllAgree warning
97
+ class SetBit,ClearBit danger
98
+```
99
19
-| Principle | Description |
20
-|----------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
21
-| **Unsupervised Learning** | Models operate independently without requiring user input. |
22
-| **Real-time Performance** | While ML impacts CPU usage, it won't compromise Netdata's high-fidelity, real-time monitoring. |
23
-| **Seamless Integration** | ML-based insights are fully embedded into Netdata's existing infrastructure monitoring and troubleshooting. |
24
-| **Assistance Over Alerts** | ML helps users investigate potential issues rather than triggering unnecessary alerts. It won't wake you up at 3 AM for minor anomalies. |
100
+### Training & Detection
101
26
----
102
+When you enable ML, Netdata trains an unsupervised model for each of your metrics. By default, this model is a [k-means clustering](https://en.wikipedia.org/wiki/K-means_clustering) algorithm (with k=2) trained on the last 4 hours of your data. Instead of just analyzing raw values, the model works with preprocessed feature vectors to improve your detection accuracy.
103
28
-## Types of Anomalies Detected
104
+:::important
105
30
-Netdata identifies several anomaly types:
106
+To reduce false positives in your environment, Netdata trains multiple models per time-series, covering over two days of data. **An anomaly is flagged only if all models agree on it, eliminating 99% of false positives**. This approach of requiring consensus across models trained on different time scales makes the system highly resistant to spurious anomalies while still being sensitive to real issues.
107
32
-- **Point Anomalies**: Unusually high or low values compared to historical data.
33
-- **Contextual Anomalies**: Sequences of values that deviate from expected patterns.
34
-- **Collective Anomalies**: Multivariate anomalies where a combination of metrics appears off.
35
-- **Concept Drifts**: Gradual shifts leading to a new baseline.
36
-- **Change Points**: Sudden shifts resulting in a new normal state.
108
+:::
109
38
----
110
+The anomaly detection algorithm uses the [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance) between recent metric patterns and the learned cluster centers. If this distance exceeds a threshold based on the 99th percentile of training data, that model considers the metric anomalous.
111
40
-## How Netdata’s ML Models Work
112
+### Anomaly Bit
113
42
-### Training & Detection
114
+Each trained model assigns an **anomaly score** at every time step based on how far your data deviates from learned clusters. If the score exceeds the 99th percentile of training data, the **anomaly bit** is set to `true` (100); otherwise, it remains `false` (0).
115
44
-Once ML is enabled, Netdata trains an unsupervised model for each metric. By default, this model is a [k-means clustering](https://en.wikipedia.org/wiki/K-means_clustering) algorithm trained on the last 4 hours of data. Instead of just analyzing raw values, the model works with preprocessed feature vectors to improve detection accuracy.
116
+**Key benefits you'll experience:**
117
46
-To reduce false positives, Netdata trains multiple models per time-series, covering over two days of data. An anomaly is flagged only if **all** models agree on it, eliminating 99% of false positives.
118
+- No additional storage overhead since the anomaly bit is embedded in Netdata's floating point number format
119
+- The query engine automatically computes anomaly rates without requiring extra queries
120
48
-### Anomaly Bit
121
+:::note
122
50
-Each trained model assigns an **anomaly score** at every time step based on how far the data deviates from learned clusters. If the score exceeds the 99th percentile of training data, the **anomaly bit** is set to `true` (100); otherwise, it remains `false` (0).
123
+The anomaly bit is quite literally a bit in Netdata's [internal storage representation](https://github.com/netdata/netdata/blob/89f22f056ca2aae5d143da9a4e94fcab1f7ee1b8/libnetdata/storage_number/storage_number.c#L83). This ingenious design means that for every metric collected, Netdata can also track whether it's anomalous without increasing storage requirements.
124
52
-**Key benefits:**
125
+:::
126
54
-- No additional storage overhead since the anomaly bit is embedded in Netdata’s floating point number format.
55
-- The query engine automatically computes anomaly rates without requiring extra queries.
127
+You can access the anomaly bits through Netdata's API by adding the `options=anomaly-bit` parameter to your query. For example:
128
+
129
+```
130
+https://your-node/api/v1/data?chart=system.cpu&dimensions=user&after=-10&options=anomaly-bit
131
+```
132
+
133
+This would return anomaly bits for the last 10 seconds of CPU user data, with values of either 0 (normal) or 100 (anomalous).
134
135
### Anomaly Rate
136
59
-Netdata calculates **Node Anomaly Rate (NAR)** and **Dimension Anomaly Rate (DAR)** based on anomaly bits. Here’s an example matrix:
137
+You can see **Node Anomaly Rate (NAR)** and **Dimension Anomaly Rate (DAR)** calculated based on anomaly bits. Here's an example matrix:
138
139
| Time | d1 | d2 | d3 | d4 | d5 | **NAR** |
140
|---------|---------|---------|---------|---------|---------|-----------------------|
@@ -72,33 +150,29 @@ Netdata calculates **Node Anomaly Rate (NAR)** and **Dimension Anomaly Rate (DAR
150
| t10 | 0 | 0 | 0 | 0 | 0 | **0%** |
151
| **DAR** | **10%** | **30%** | **20%** | **20%** | **30%** | **_NAR_t1-10 = 22%_** |
152
75
-- **DAR (Dimension Anomaly Rate):** Average anomalies for a specific metric over time.
76
-- **NAR (Node Anomaly Rate):** Average anomalies across all metrics at a given time.
77
-- **Overall anomaly rate:** Computed across the entire dataset for deeper insights.
153
+- **DAR (Dimension Anomaly Rate):** Average anomalies for a specific metric over time
154
+- **NAR (Node Anomaly Rate):** Average anomalies across all metrics at a given time
155
+- **Overall anomaly rate:** Computed across your entire dataset for deeper insights
156
157
### Node-Level Anomaly Detection
158
81
-Netdata tracks the percentage of anomaly bits over time. When the **Node Anomaly Rate (NAR)** exceeds a set threshold and remains high for a period, a **node anomaly event** is triggered. These events are recorded in the `new_anomaly_event` dimension on the `anomaly_detection.anomaly_detection` chart.
159
+Netdata tracks the percentage of anomaly bits over time for you. When the **Node Anomaly Rate (NAR)** exceeds a set threshold and remains high for a period, a **node anomaly event** is triggered. These events are recorded in the `new_anomaly_event` dimension on the `anomaly_detection.anomaly_detection` chart.
160
83
----
161
+## Viewing Anomaly Data in Your Netdata Dashboard
162
85
-## Viewing Anomaly Data in Netdata
163
+Once you enable ML, you'll have access to an **Anomaly Detection** menu with key charts:
164
87
-Once ML is enabled, Netdata provides an **Anomaly Detection** menu with key charts:
88
-
89
-- **`anomaly_detection.dimensions`**: Number of dimensions flagged as anomalous.
90
-- **`anomaly_detection.anomaly_rate`**: Percentage of anomalous dimensions.
91
-- **`anomaly_detection.anomaly_detection`**: Flags (0 or 1) indicating when an anomaly event occurs.
165
+- **`anomaly_detection.dimensions`**: Number of dimensions flagged as anomalous
166
+- **`anomaly_detection.anomaly_rate`**: Percentage of anomalous dimensions
167
+- **`anomaly_detection.anomaly_detection`**: Flags (0 or 1) indicating when an anomaly event occurs
168
169
These insights help you quickly assess potential issues and take action before they escalate.
170
95
----
96
-
171
## Summary
172
99
-Netdata’s machine learning models provide reliable, real-time anomaly detection with minimal false positives. By embedding ML within existing observability workflows, Netdata enhances troubleshooting and ensures proactive monitoring without unnecessary alerts.
173
+With Netdata ML, you get reliable, real-time anomaly detection with minimal false positives. By incorporating ML within your existing observability workflows, you can enhance troubleshooting and ensure proactive monitoring without unnecessary alerts.
174
101
-For more details, check out:
175
+For more information:
176
177
- [Anomaly Advisor](/docs/dashboards-and-charts/anomaly-advisor-tab.md)
104
-- [ML Configuration Guide](/src/ml/ml-configuration.md)
\ No newline at end of file
178
+- [ML Configuration Guide](/src/ml/ml-configuration.md)
src/ml/ml-configuration.md
+161
-53
@@ -1,10 +1,10 @@
1
# ML Configuration
2
3
-Netdata's [Machine Learning](/src/ml/README.md) capabilities are enabled by default if the [Database mode](/src/database/README.md) is set to `db = dbengine`.
3
+You can use Netdata's [Machine Learning](/src/ml/README.md) capabilities to detect anomalies in your infrastructure metrics. This feature is enabled by default if your [Database mode](/src/database/README.md) is set to `db = dbengine`.
4
5
## Enabling or Disabling Machine Learning
6
7
-To enable or disable Machine Learning capabilities on a node:
7
+To enable or disable Machine Learning capabilities on your node:
8
9
1. [Edit `netdata.conf`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config).
10
2. In the `[ml]` section:
@@ -13,6 +13,35 @@ To enable or disable Machine Learning capabilities on a node:
13
- Leave it at the default `auto` to enable ML only when [Database mode](/src/database/README.md) is set to `dbengine`.
14
3. [Restart Netdata](/docs/netdata-agent/start-stop-restart.md).
15
16
+## Technical Implementation
17
+
18
+Netdata implements machine learning using the lightweight [dlib](https://github.com/davisking/dlib) C++ library. This implementation choice allows Netdata to:
19
+
20
+```mermaid
21
+flowchart TD
22
+ %% Node styling
23
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
24
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
25
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
26
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
27
+
28
+ Dlib[dlib C++ Library Implementation]
29
+
30
+ Dlib --> Efficiency[Run efficiently on<br/>any system without<br/>heavy dependencies]
31
+ Dlib --> Resources[Minimize resource usage<br/>while maintaining<br/>high accuracy]
32
+ Dlib --> Codebase[Operate within the<br/>constraints of agent's<br/>C/C++ codebase]
33
+
34
+ %% Apply styles
35
+ class Dlib warning
36
+ class Efficiency,Resources,Codebase success
37
+```
38
+
39
+:::note
40
+
41
+The ML implementation prioritizes minimal storage impact. Model objects are designed to be small, with each trained model typically requiring only a few kilobytes of memory.
42
+
43
+:::
44
+
45
## Available Configuration Parameters
46
47
Below is a list of all available configuration parameters and their default values:
@@ -41,9 +70,86 @@ Below is a list of all available configuration parameters and their default valu
70
# delete models older than = 7d
71
```
72
73
+## Multiple Models and False Positive Reduction
74
+
75
+One of the **key strengths of Netdata's ML implementation is its use of multiple models** to virtually eliminate false positives.
76
+
77
+When you configure the `number of models per dimension` parameter (default: 18), you're specifying how many trained models Netdata maintains for each metric.
78
+
79
+:::note
80
+
81
+The default setting of 18 models spanning approximately 54 hours (with models trained every 3 hours) ensures that anomalies are verified across different time frames before being flagged.
82
+
83
+:::
84
+
85
+### How Multiple Models Work Together
86
+
87
+When your system detects a potential anomaly:
88
+
89
+1. **The new data point is evaluated against all trained models** (up to 18 by default)
90
+2. **Each model independently determines if the data point is anomalous** based on its training period
91
+3. Only if **ALL** models agree that the data point is anomalous will Netdata set the anomaly bit to 1 (anomalous)
92
+
93
+```mermaid
94
+flowchart TD
95
+ %% Node styling
96
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
97
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
98
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
99
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
100
+
101
+ %% Input data
102
+ NewData[New Metric Data Point] --> M1 & M2 & M3 & M4
103
+
104
+ %% Models
105
+ subgraph Models[Multiple Time-Scale Models]
106
+ M1[Model 1<br/>Recent 3h]
107
+ M2[Model 2<br/>3-6h ago]
108
+ M3[Model 3<br/>6-9h ago]
109
+ M4[Model N<br/>Older periods]
110
+ end
111
+
112
+ %% Decisions
113
+ M1 --> D1{Anomalous?}
114
+ M2 --> D2{Anomalous?}
115
+ M3 --> D3{Anomalous?}
116
+ M4 --> D4{Anomalous?}
117
+
118
+ %% Results
119
+ D1 -->|Yes| R1[Model 1: Anomaly]
120
+ D1 -->|No| N1[Model 1: Normal]
121
+ D2 -->|Yes| R2[Model 2: Anomaly]
122
+ D2 -->|No| N2[Model 2: Normal]
123
+ D3 -->|Yes| R3[Model 3: Anomaly]
124
+ D3 -->|No| N3[Model 3: Normal]
125
+ D4 -->|Yes| R4[Model N: Anomaly]
126
+ D4 -->|No| N4[Model N: Normal]
127
+
128
+ %% Consensus
129
+ R1 & R2 & R3 & R4 --> AllYes{All<br/>Models<br/>Agree?}
130
+ N1 & N2 & N3 & N4 ---> AllNo[Set Anomaly Bit = 0<br/>Normal]
131
+ AllYes -->|Yes| AllAnom[Set Anomaly Bit = 1<br/>Anomalous]
132
+ AllYes -->|No| AllNo
133
+
134
+ %% Apply styles
135
+ class NewData neutral
136
+ class M1,M2,M3,M4 success
137
+ class D1,D2,D3,D4,AllYes warning
138
+ class R1,R2,R3,R4,AllAnom danger
139
+ class N1,N2,N3,N4,AllNo neutral
140
+```
141
+
142
+:::note
143
+
144
+This consensus approach **reduces false positives by approximately 99%**, ensuring that you only see alerts for genuine anomalies that persist across different time scales.
145
+
146
+:::
147
+
148
+**Models trained on different time frames capture various patterns in your data**, making the system robust against temporary fluctuations.
149
+
150
## Configuration Examples
151
46
-If you want to run ML on a parent instead of at the edge, the examples below illustrate various configurations.
152
+If you want to **run ML on a parent instead of at the edge**, the examples below illustrate various configurations.
153
154
This example assumes three child nodes [streaming](/docs/observability-centralization-points/metrics-centralization-points/README.md) to one parent node. It shows different ways to configure ML:
155
@@ -53,13 +159,24 @@ This example assumes three child nodes [streaming](/docs/observability-centraliz
159
160
```mermaid
161
flowchart BT
56
- C1["Netdata Child 0\nML enabled"]
57
- C2["Netdata Child 1\nML enabled"]
58
- C3["Netdata Child 2\nML disabled"]
59
- P1["Netdata Parent\n(ML enabled for itself and Child 1 & 2)"]
162
+ %% Node styling
163
+ classDef neutral fill:#f9f9f9,stroke:#000000,color:#000000,stroke-width:2px
164
+ classDef success fill:#4caf50,stroke:#000000,color:#000000,stroke-width:2px
165
+ classDef warning fill:#ffeb3b,stroke:#000000,color:#000000,stroke-width:2px
166
+ classDef danger fill:#f44336,stroke:#000000,color:#000000,stroke-width:2px
167
+
168
+ C1["Netdata Child 0<br/>ML enabled"]
169
+ C2["Netdata Child 1<br/>ML enabled"]
170
+ C3["Netdata Child 2<br/>ML disabled"]
171
+ P1["Netdata Parent<br/>(ML enabled for itself<br/>and Child 1 & 2)"]
172
C1 --> P1
173
C2 --> P1
174
C3 --> P1
175
+
176
+ %% Apply styles
177
+ class C1,C2 success
178
+ class C3 neutral
179
+ class P1 warning
180
```
181
182
```text
@@ -85,59 +202,50 @@ flowchart BT
202
enabled = no
203
```
204
88
-## Parameter Descriptions (Min/Max Values)
89
-
90
-### General Settings
91
-
92
-- **`enabled`**: Controls whether ML is enabled.
93
- - `yes` to enable.
94
- - `no` to disable.
95
- - `auto` lets Netdata decide based on database mode.
96
-
97
-- **`maximum num samples to train`** (`3600` - `86400`): Defines the maximum training period. The default of `21600` trains on the last 6 hours of data.
98
-
99
-- **`minimum num samples to train`** (`900` - `21600`): The minimum amount of data needed to train a model. If less than `900` samples (15 minutes of data) are available, training is skipped.
100
-
101
-- **`train every`** (`3h` - `6h`): Determines how often models are retrained. The default of `3h` means retraining occurs every three hours. Training is staggered to distribute system load.
102
-
103
-### Model Behavior
104
-
105
-- **`number of models per dimension`** (`1` - `168`): Specifies how many trained models per dimension are used for anomaly detection. The default of `18` means models trained over the last ~54 hours are considered.
205
+## Resource Considerations
206
107
-- **`dbengine anomaly rate every`** (`30` - `900`): Defines how frequently Netdata aggregates anomaly bits into a single chart.
207
+When configuring machine learning, it's important to understand the resource implications:
208
109
-### Feature Processing
209
+:::note
210
111
-- **`num samples to diff`** (`0` - `1`): Determines whether ML operates on raw data (`0`) or differences (`1`). Using differences helps detect anomalies in cyclical patterns.
211
+Netdata's ML is designed to be lightweight, using approximately 1–2% of a single CPU core under default settings on a typical system.
212
113
-- **`num samples to smooth`** (`0` - `5`): Controls data smoothing. The default of `3` averages the last three values to reduce noise.
213
+:::
214
115
-- **`num samples to lag`** (`0` - `5`): Defines how many past values are included in the feature vector. The default `5` helps the model detect patterns over time.
215
+Several configuration options directly impact resource usage:
216
117
-### Training Efficiency
217
+- **Increasing `number of models per dimension`** increases memory usage as more models are stored
218
+- **Decreasing `train every`** increases CPU usage as models are trained more frequently
219
+- **Increasing `maximum num samples to train`** increases memory usage during training but may improve accuracy
220
+- **Adjusting `random sampling ratio`** allows you to control how much data is used during training (lower values reduce CPU usage)
221
119
-- **`random sampling ratio`** (`0.2` - `1.0`): Controls the fraction of data used for training. The default `0.2` means 20% of available data is used, reducing system load.
222
+For resource-constrained systems, consider these adjustments:
223
121
-- **`maximum number of k-means iterations`**: Limits iterations during k-means clustering (leave at default in most cases).
224
+- Set `random sampling ratio = 0.1` to reduce training data by half from default
225
+- Use `number of models per dimension = 6` to maintain multiple model consensus with reduced memory footprint
226
+- Increase `train every = 6h` to reduce training frequency
227
123
-### Anomaly Detection Sensitivity
124
-
125
-- **`dimension anomaly score threshold`** (`0.01` - `5.00`): Sets the threshold for flagging an anomaly. The default `0.99` flags values that are in the top 1% of anomalies based on training data.
126
-
127
-- **`host anomaly rate threshold`** (`0.1` - `10.0`): Defines the percentage of dimensions that must be anomalous for the host to be considered anomalous. The default `1.0` means more than 1% must be anomalous.
128
-
129
-### Anomaly Detection Grouping
130
-
131
-- **`anomaly detection grouping method`**: Defines the method used to calculate the node-level anomaly rate.
132
-
133
-- **`anomaly detection grouping duration`** (`1m` - `15m`): Determines the time window for calculating anomaly rates. The default `5m` calculates anomalies over a 5-minute rolling window.
134
-
135
-### Skipping Hosts and Charts
136
-
137
-- **`hosts to skip from training`**: Allows excluding specific child hosts from training. The default `!*` means no hosts are skipped.
138
-
139
-- **`charts to skip from training`**: Excludes charts from anomaly detection. By default, Netdata-related charts are excluded to prevent false anomalies caused by normal dashboard activity.
140
-
141
-### Model Retention
228
+## Parameter Descriptions (Min/Max Values)
229
143
-- **`delete models older than`** (`1d` - `7d`): Defines how long old models are stored. The default `7d` removes unused models after seven days.
\ No newline at end of file
230
+# ML Parameter Settings
231
+
232
+| Category | Parameter | Range | Description |
233
+|-----------------------------------|----------------------------------------|------------------|------------------------------------------------------------------------------------------------------------------------------------------|
234
+| **General Settings** | `enabled` | `yes/no/auto` | Controls whether ML is enabled. `yes` to enable, `no` to disable, `auto` lets Netdata decide based on database mode. |
235
+| | `maximum num samples to train` | `3600` - `86400` | Defines the maximum training period. Default `21600` trains on your last 6 hours of data. |
236
+| | `minimum num samples to train` | `900` - `21600` | Minimum data needed to train a model. Training is skipped if less than `900` samples (15 minutes) are available. |
237
+| | `train every` | `3h` - `6h` | How often models are retrained. Default `3h` means retraining every three hours. Training is staggered to distribute system load. |
238
+| **Model Behavior** | `number of models per dimension` | `1` - `168` | Specifies how many trained models per dimension are used. Default `18` means models trained over the last ~54 hours are considered. |
239
+| | `dbengine anomaly rate every` | `30` - `900` | How frequently Netdata aggregates anomaly bits into a single chart. |
240
+| **Feature Processing** | `num samples to diff` | `0` - `1` | Determines whether ML operates on raw data (`0`) or differences (`1`). Using differences helps detect anomalies in cyclical patterns. |
241
+| | `num samples to smooth` | `0` - `5` | Controls data smoothing. Default `3` averages the last three values to reduce noise. |
242
+| | `num samples to lag` | `0` - `5` | How many past values are included in the feature vector. Default `5` helps detect patterns over time. |
243
+| **Training Efficiency** | `random sampling ratio` | `0.2` - `1.0` | Fraction of data used for training. Default `0.2` means 20% of available data is used, reducing system load while maintaining accuracy. |
244
+| | `maximum number of k-means iterations` | - | Limits iterations during k-means clustering (leave at default in most cases). |
245
+| **Anomaly Detection Sensitivity** | `dimension anomaly score threshold` | `0.01` - `5.00` | Threshold for flagging an anomaly. Default `0.99` flags values in the top 1% of anomalies based on training data. |
246
+| | `host anomaly rate threshold` | `0.1` - `10.0` | Percentage of dimensions that must be anomalous for host to be considered anomalous. Default `1.0` means more than 1% must be anomalous. |
247
+| **Anomaly Detection Grouping** | `anomaly detection grouping method` | - | Method used to calculate node-level anomaly rate. |
248
+| | `anomaly detection grouping duration` | `1m` - `15m` | Time window for calculating anomaly rates. Default `5m` calculates over a 5-minute rolling window. |
249
+| **Skipping Hosts and Charts** | `hosts to skip from training` | - | Excludes specific child hosts from training. Default `!*` means no hosts are skipped. |
250
+| | `charts to skip from training` | - | Excludes charts from anomaly detection. By default, Netdata-related charts are excluded. |
251
+| **Model Retention** | `delete models older than` | `1d` - `7d` | How long old models are stored. Default `7d` removes unused models after seven days. |