Update machine-learning-and-assisted-troubleshooting.md
Costa Tsaousis committed
Feb 20, 2024 at 19:06 UTC
89c058a3a371aaeb67c4352bcc710d5cc2179a9e
1 file changed
+22
docs/category-overview-pages/machine-learning-and-assisted-troubleshooting.md
+22
@@ -73,3 +73,25 @@ The query engine of Netdata uses this bit to compute anomaly rates while it exec
73
## What is the Anomaly Rate (AR)?
74
75
The Anomaly Rate of a query, is a percentage, representing the number of samples in the query found anomalous, vs the total number of samples participating in the query.
76
+
77
+## How it works - a more technical presentation
78
+
79
+For each time-series Netdata trains every 3 hours, a `k-means clustering` model, using the last 6 hours of samples collected for it.
80
+
81
+Rather than using raw samples of each time-series, the model works on a preprocessed "feature vector" of recent smoothed and differenced values.
82
+
83
+This enables the model to detect a wider range of potentially anomalous patterns as opposed to just point anomalies like big spikes or drops.
84
+
85
+Some of the types of anomalies Netdata detects are:
86
+
87
+1. **Point Anomalies** or **Strange Points**: Single points that represent very big or very small values, not seen before (in some statistical sense).
88
+2. **Contextual Anomalies** or **Strange Patterns**: Not strange points in their own, but unexpected sequences of points, given the history of the time-series.
89
+3. **Collective Anomalies** or **Strange Multivariate Patterns**: Neither strange points nor strange patterns, but in global sense something looks off.
90
+4. **Concept Drifts** or **Strange Trends**: A slow and steady drift to a new state.
91
+5. **Change Point Detection** or **Strange Step**: A shift occurred and gradually a new normal is established.
92
+
93
+For a visual representation, check this infographic:
94
+
95
+
96
+
97
+A more detailed explanation can be found on [this (informal) presentation](https://docs.google.com/presentation/d/18zkCvU3nKP-Bw_nQZuXTEa4PIVM6wppH3VUnAauq-RU/edit#slide=id.p).