@cryptotaxi247 / netdata-1 / commits / 284825e2d

docs: Add comprehensive scalability architecture documentation (#21018)

* Fix MDX parsing errors in realtime-monitoring.md Replace '<5%' with 'Less than 5%' and '<200' with 'Less than 200' to prevent MDX parser from interpreting them as JSX elements. The MDX parser was failing with 'Unexpected character 5' error when encountering '<5' pattern. * Improve ASCII art diagram with proper symbols - Replace X's with block characters (█) for better visual representation - Use arrow symbols (↓ ↑) instead of pipes for clearer directional indicators - Use box drawing characters (┬ ─ │) for cleaner timeline structure - Fix typo: 'missaligned' to 'misaligned' - Improve clarity of event timeline labels * docs: Add comprehensive scalability architecture documentation - Document edge-native architecture with process-and-store-at-edge philosophy - Explain Parents as intelligent workload distributors, not just centralization - Detail ML training flexibility (edge or Parents with model streaming) - Show resource savings with offloaded Agents (<2% CPU, <150 MiB RAM, zero disk I/O) - Include performance comparisons (Netdata vs Prometheus 2025 test) - Add University of Amsterdam energy efficiency study validation - Document intelligent Parent clustering without work duplication - Explain automation vs monitoring alert separation strategy - Provide Parent sizing guidelines and deployment strategies - Add comprehensive FAQ section addressing common questions This document serves as both a pre-sales resource and high-level deployment guide, explaining how Netdata scales from 1 to 100,000+ nodes without architectural changes. * docs: Add scalability.md to documentation map Register the new scalability architecture document in the documentation index.

Costa Tsaousis committed Sep 21, 2025 at 00:52 UTC 284825e2d20675fd3b92f0e97c922d8166e119b1

3 files changed +347 -55

docs/.map/map.csv

@@ -2,6 +2,7 @@ custom_edit_url,sidebar_label,learn_status,learn_rel_path,description

2 https://github.com/netdata/netdata/edit/master/docs/welcome-to-netdata.md,Welcome to Netdata,Published,Welcome to Netdata,

3 https://github.com/netdata/netdata/edit/master/docs/netdata-enterprise-evaluation-corrected.md,Enterprise Evaluation Guide,Published,Welcome to Netdata,

4 https://github.com/netdata/netdata/edit/master/docs/realtime-monitoring.md,Real-time Monitoring,Published,Welcome to Netdata,

5 +https://github.com/netdata/netdata/edit/master/docs/scalability.md,Scalability,Published,Welcome to Netdata,

6 https://github.com/netdata/netdata/edit/master/docs/fleet-configuration-management.md,Fleet Deployment and Configuration Management,Published,Welcome to Netdata,

7 https://github.com/netdata/netdata/edit/master/docs/getting-started-netdata/guide.md,Getting Started,Published,root,

8 https://github.com/netdata/netdata/edit/master/docs/Demo-Sites.md,Live Demo,Published,root,

docs/realtime-monitoring.md

+56 -55

@@ -1,8 +1,8 @@

1 # Real-Time Monitoring: The Netdata Standard

3 -## Executive Summary

3 +## TL;DR

5 -Netdata defines what real-time monitoring truly means: **1-second data collection and 1–2 second total latency from event to insight**, at scale. While most monitoring systems operate on 10–60-second intervals, Netdata provides _true_ sub-2-second visibility without overhead. This difference is critical: most operational anomalies last under 10 seconds, which means traditional monitoring misses them completely.

5 +Netdata defines what real-time monitoring truly means: **1-second data collection and 1-second latency from collection to visualization**, providing a worst-case latency of less that 2 seconds from event to insight, at any scale. While most monitoring systems operate on 10–60-second intervals, Netdata provides _true_ sub-2-second visibility without overhead. This difference is critical: most operational anomalies last under 10 seconds, which means traditional monitoring misses them completely.

7 With Netdata, organizations gain:

@@ -34,19 +34,19 @@ As Richard Hackathorn wrote in _[The BI Watch: Real-Time to Real-Value](https://

34 Wikipedia defines **[real-time business intelligence](https://en.wikipedia.org/wiki/Real-time_business_intelligence)** as a range from _milliseconds to ≤ 5 seconds_ after an event has occurred, and identifies three types of latency involved:

36 - **Data latency** - time to collect and store data

37 -- **Analysis latency** - time to process data into information

37 +- **Analysis latency** - time to process data

38 - **Action latency** - time to act on the data (e.g., visualize or alert in observability)

40 -For a system to qualify as _real-time_, the **sum of all three latencies** must remain within the real-time window.

40 +For a system to qualify as _real-time_, the **sum of all three latencies** must remain within the real-time window of ≤ 5 seconds.

42 ### The Netdata Standard for Real-Time

43 Based on these definitions, Netdata establishes this practical taxonomy:

45 -Classification | Total Latency | Netdata's Position

46 ----:|:---:|:---

47 -**Real-time** | ≤ 5 seconds | ✅ Netdata: 1-2 seconds total

48 -**Near real-time** | 5-30 seconds | ❌ Most "modern" monitoring tools

49 -**Not real-time** | > 30 seconds | ❌ Traditional monitoring systems

45 +| Classification | Total Latency | Netdata's Position |

46 +| -----------------: | :-----------: | :------------------------------- |

47 +| **Real-time** | ≤ 5 seconds | ✅ Netdata: 1-2 seconds total |

48 +| **Near real-time** | 5-30 seconds | ❌ Most "modern" monitoring tools |

49 +| **Not real-time** | > 30 seconds | ❌ Traditional monitoring systems |

51 **Netdata is designed to keep the sum of all three latencies under 2 seconds**, making it one of the few monitoring systems that qualifies as truly real-time at scale, under rigorous definitions.

@@ -54,13 +54,13 @@ Classification | Total Latency | Netdata's Position

55 ### The Pillars of Operational Excellence

57 -1. **Faster Mean Time to Resolution (MTTR)**

57 +1. **Faster Mean Time to Resolution (MTTR)**

58 Engineers see the immediate effects of their actions. When a database is slow, they alter an index and instantly observe queries running faster and resource relief. Without real-time monitoring, this iterative troubleshooting process takes 10-30x longer.

59 -2. **Accurate Incident Detection**

59 +2. **Accurate Incident Detection**

60 Real-time monitoring reveals the true behavior of systems. An application using 100% CPU for 2 seconds then 0% for 8 seconds is fundamentally different from one using steady 20% CPU. Averaged metrics hide critical patterns, confuse operations teams, and delay root cause identification.

61 -3. **Preventing Cascading Failures**

61 +3. **Preventing Cascading Failures**

62 Problems compound exponentially. A 3-second resource spike can trigger a 30-second cascade if not caught immediately. Real-time monitoring catches the spark before it becomes a fire.

63 -4. **Security Threat Detection**

63 +4. **Security Threat Detection**

64 Modern attacks happen in seconds: port scans (2-3 seconds), crypto-miner activation (instant CPU spikes), memory scanning attempts (burst patterns). These are invisible to systems monitoring at 30-second intervals.

66 ### Use Cases That Demand Real-Time

@@ -86,13 +86,13 @@ DBAs using Netdata can immediately see the effects of index changes, query plan

86 Container lifespans can be seconds. Pod scheduling, startup, and shutdown events often complete in under 5 seconds. Traditional monitoring completely misses these critical events. Netdata captures the full lifecycle.

88 #### Typical System Administration

89 -**Scenario 1: Disk throughput**

89 +**Scenario 1: Disk throughput**

90 An application performs disk reads at **500 MB/s for 5 seconds**, then is idle for 10 seconds. Can the application be made faster?

92 - **Other monitoring solutions** show 15-second averages at 167MB/s. SREs: "The application can be made faster, the disks can provide up to 500MB/s. Contact the developers".

93 - **Netdata** shows saturation for 5 seconds, then idle. SREs: "The application is already maxing the disks. Install faster disks."

95 -**Scenario 2: Network saturation**

95 +**Scenario 2: Network saturation**

96 A sensitive transactional database stalls for 10 seconds every 5 minutes.

98 - **Other monitoring solutions** (1-minute averages) show network usage rising slightly, from **200 Mb/s → 220 Mb/s** once every 5 minutes. This looks harmless, almost noise.

@@ -102,11 +102,11 @@ With coarse averages, every team sees evidence that _someone else_ is at fault.

102

103 ## The Anatomy of Netdata’s Latency

104

105 -Netdata is designed to keep the **sum of all three latencies 1 second **. The worst case scenario for Netdata is 2 seconds:

105 +Netdata is designed to keep the **sum of all three latencies at about 1 second **. with the worst case scenario at 2 seconds:

106

107 -1. **Data latency:** 1 second (collection interval)

107 +1. **Data latency:** up to 1 second from event

108 2. **Analysis latency:** microseconds (negligible, CPU-speed dependent)

109 -3. **Action latency:** 1 second (visualization and alert updates)

109 +3. **Action latency:** up to 1 second from collection

110

111 This means:

112

@@ -118,19 +118,20 @@ Graphically:

118 ```

119 Data Collection Pace

120

121 - the interesting event

122 - | event collected

123 - | |

124 ---|XXXXXXXXXX|----------|----------|----------| <- data collection pace

125 - |--- 1s ---|--- 1s ---|--- 1s ---|--- 1s ---|

121 + the interesting event started

122 + ↓ event collected

123 + ↓ ↓

124 +──┬██████████┬──────────┬──────────┬──────────┬ ← data collection pace

125 + │ ── 1s ── │ ── 1s ── │ ── 1s ── │ ── 1s ── │

126

127 -Visualization Pace (a few ms missaligned to collection - worst case)

127 +Visualization Pace (a few ms misaligned to collection - worst case)

128

129 - UI fetches everything collected

130 - |

131 ---------|----------|XXXXXXXXXX|----------|----- <- visualization pace

132 - |

133 - the event visualized

129 + UI fetches everything collected

130 + ↓

131 +┬──────────┬──────────┬██████████┬──────────┬── ← visualization pace

132 +│ ── 1s ── │ ── 1s ── │ ── 1s ── │ ── 1s ── │

133 + ↑

134 + event visualized

135 ```

136

137 The shaded boxes show the slices where an event may fall. Because both collection and visualization run on 1-second ticks, the event is guaranteed to be visible within 2 seconds.

@@ -164,20 +165,20 @@ Netdata is a distributed-by-design platform. It scales horizontally by adding mo

165 ## How Netdata Compares to Other Monitoring Solutions

166 While most tools can be _configured_ for faster polling, their core architecture is not optimized for sustained, pervasive, per-second collection without excessive overhead or cost. This table reflects their typical, real-world deployment:

167

168 -|---:|:---:|:---:|:---|

169 +| -----------------------: | :---------------------: | :------------: | :---------------------------------------- |

182

183 ### The Prometheus + Grafana Reality

184 - Designed around 10-30 second scrape intervals; 1-second scraping is anti-pattern that overloads targets.

@@ -200,10 +201,10 @@ Our [stress test](https://www.netdata.cloud/blog/netdata-vs-prometheus-2025/) ag

201 - Cloud processing and indexing add another 5-30 seconds of latency.

202 - **Verdict**: A powerful analytics platform, but its architecture imposes fundamental latency constraints. It is not real-time.

203

203 -**Q: Datadog claims "real-time" features. How is Netdata different?**

204 +**Q: Datadog claims "real-time" features. How is Netdata different?**

205 A: Datadog's "real-time" typically refers to live-tail for logs or near-live infrastructure views. Their metric pipeline latency is 20-90 seconds. Netdata provides true 1-2 second latency for all metrics, everywhere.

206

206 -**Q: Couldn't Datadog just collect faster?**

207 +**Q: Couldn't Datadog just collect faster?**

208 A: No. Their business and architectural model prevents it:

209

210 - **Cost Prohibitive:** 1-second collection increases metric volume 15-60x, making their pricing model untenable for customers.

@@ -217,7 +218,7 @@ Netdata's edge-native architecture eliminates these bottlenecks by collecting, s

218 - Relies on sampling for high-volume events, sacrificing fidelity.

219 - **Verdict**: Powerful for code-level application insights, but its minute-level granularity misses crucial system-level patterns. Not real-time.

220

220 -**Q: New Relic has 1-minute metrics. Isn't that enough?**

221 +**Q: New Relic has 1-minute metrics. Isn't that enough?**

222 A: Absolutely not. They hide the most critical performance patterns. A 5-second query running every 20 seconds appears as a benign 25% load average in New Relic but is immediately obvious as a damaging spike pattern in Netdata.

223

224 ## The Netdata Real-Time Monitoring Manifesto

@@ -261,31 +262,31 @@ It is common for large Netdata deployments to process millions of metrics per se

262

263 ## Real-Time Monitoring FAQ

264

264 -**Q: Is Netdata truly real-time?**

265 +**Q: Is Netdata truly real-time?**

266 A: Yes. Netdata provides 1-second granularity monitoring with a total latency of 1-2 seconds from event to visualization, making it the fastest and only true real-time monitoring solution proven at scale. This is 10-60x faster than the typical "near real-time" solutions.

267

267 -**Q: Why is 1-second resolution critical compared to 10 or 30-second?**

268 +**Q: Why is 1-second resolution critical compared to 10 or 30-second?**

269 A: Most operational anomalies have a duration of 2-10 seconds. With 30-second monitoring, you are blind to over 90% of incidents. With 10-second monitoring, you still miss roughly 50%. Netdata's 1-second monitoring captures the full spectrum of system behavior.

270

270 -**Q: Doesn't per-second monitoring create unsustainable overhead?**

271 +**Q: Doesn't per-second monitoring create unsustainable overhead?**

272 A: No. This is a common misconception. Netdata is engineered for extreme efficiency, typically using less than 5% of a single CPU core. According to the [University of Amsterdam study](https://www.ivanomalavolta.com/files/papers/ICSOC_2023.pdf), Netdata is the most energy-efficient tool for monitoring Docker-based systems. The study also shows Netdata excels in CPU usage, RAM usage, and execution time compared to other monitoring solutions.

273

273 -**Q: How does Netdata handle network outages?**

274 +**Q: How does Netdata handle network outages?**

275 A: Netdata Agents buffer metrics locally on disk during network partitions and automatically replay the buffered data once the connection is restored. This ensures zero data loss. Gaps only appear if the local system is too stressed to even collect data, which is itself a critical alert. Also, each Netdata Agent and Parent provide their own dashboards allowing continue troubleshooting at extreme conditions.

276

276 -**Q: Can Netdata handle cloud and container environments?**

277 +**Q: Can Netdata handle cloud and container environments?**

278 A: Yes, natively. Netdata provides automatic discovery and per-second monitoring for Kubernetes, Docker, and all major cloud platforms. It collects cgroups metrics directly from the kernel. The short lifespans of containers are perfectly aligned with Netdata's real-time model.

279

279 -**Q: What is the core architectural difference between Netdata and Prometheus/Datadog?**

280 +**Q: What is the core architectural difference between Netdata and Prometheus/Datadog?**

281 A: Netdata is a distributed, edge-native system designed for real-time data. Prometheus is a pull-based centralized scrapers, and Datadog is a cloud-based SaaS platform. These fundamental models impose inherent latency that Netdata's architecture avoids entirely.

282

282 -**Q: Is real-time necessary for every single metric?**

283 +**Q: Is real-time necessary for every single metric?**

284 A: Netdata uses intelligence, not dogma. It automatically adjusts collection frequency for slow-changing metrics (like static configuration details) or for applications that are sensitive to frequent polling, while maintaining 1-second collection for all dynamic system and application metrics.

285

285 -**Q: Does querying more time-series slow down the dashboard?**

286 +**Q: Does querying more time-series slow down the dashboard?**

287 A: Rendering thousands of lines on a chart is a browser limitation, not a Netdata limitation. For these high-cardinality views, Netdata provides instant aggregated views with the ability to drill down to specific metrics in real-time.

288

288 -**Q: What are the trade-offs for being real-time?**

289 +**Q: What are the trade-offs for being real-time?**

290 A: The trade-off is architectural complexity. Distributing intelligence to the edge, synchronizing data in real-time without central bottlenecks, and managing a fleet implicitly rather than explicitly is a significantly harder engineering problem. We solved this so you don't have to choose between real-time visibility and operational cost.

291

292 ## Summary

docs/scalability.md new

+290

@@ -0,0 +1,290 @@

1 +# Scalability: Monitoring at Any Scale

2 +

3 +## TL;DR

4 +

5 +Netdata scales from a single node to 100,000+ nodes without architectural changes, maintaining 1-second granularity and sub-2-second latency at any scale. The distributed edge-native architecture ensures that adding more nodes doesn't degrade performance - each node operates independently while collaborating seamlessly.

6 +

7 +## The Problem with Centralization

8 +

9 +Traditional observability assumes one thing: push all data to a central database, then query it for dashboards and alerts.

10 +

11 +This works. Until it doesn't.

12 +

13 +When scale breaks the model, teams face two options - and both are wrong:

14 +

15 +### Option 1: Reduce the Workload

16 +Lower granularity. Drop cardinality. Filter. Sample.

17 +

18 +**This is a trap and a paradox.** If you knew which data you'd need during a crisis, you could predict the crisis and prevent it. By definition, an unpredictable event is the one that will be invisible in your downsampled dataset. You're betting your incident response on being able to predict the unpredictable.

19 +

20 +### Option 2: Scale the Database

21 +Build giant, expensive clusters. Add more cores. More RAM. More everything.

22 +

23 +**This is a money pit.** In many organizations today, observability costs more than the services being monitored. We routinely encounter companies where 40-50% of infrastructure budget goes to monitoring pipelines, plus teams of specialists to keep them alive.

24 +

25 +## The Netdata Way: Process and Store at the Edge

26 +

27 +**Instead of centralizing data, distribute the code.** This is the heart of Netdata's philosophy and design.

28 +

29 +Every Netdata Agent is a full observability engine:

30 +- Collects metrics at the edge

31 +- Stores data locally in multi-tier storage

32 +- Runs ML-based anomaly detection in real-time

33 +- Runs health checks and triggers alerts

34 +- Serves dashboards and APIs independently

35 +

36 +When you need high availability, persistent storage for ephemeral nodes, reduced load on production systems, or on-premises dashboards, Parents aggregate streams without becoming bottlenecks - because the heavy lifting already happened at the edge.

37 +

38 +This distributed architecture delivers results that speak for themselves:

39 +

40 +- **No loss of fidelity** - every metric, every second, always visible

41 +- **No blind spots** - no sampling, no cherry-picking

42 +- **No scaling tax** - adding nodes adds observability, not exponential cost curves

43 +

44 +Once you pass ~500 nodes, you’re naturally in the multi-million metrics/s range. What looks “heroic” elsewhere is simply normal operating conditions with Netdata.

45 +

46 +## Proof: The Numbers Don't Lie

47 +

48 +### Independent Validation: University of Amsterdam Study (2023)

49 +**Study:** "An Empirical Evaluation of the Energy and Performance Overhead of Monitoring Tools on Docker-Based Systems"

50 +**Conference:** ICSOC 2023 (International Conference on Service-Oriented Computing)

51 +**DOI:** 10.1007/978-3-031-48421-6_13

52 +

53 +**Finding:** **Netdata is the most energy-efficient monitoring solution**, with the lowest CPU and memory overhead - even while collecting data every second and running anomaly detection at the edge.

54 +

55 +### Head-to-Head: Netdata vs Prometheus (2025)

56 +We tested a single installation Netdata Parent and Prometheus, at **4.6 million metrics per second** - the scale you hit with just 1,000 nodes. This is how the systems compare for ingestion:

57 +

59 +| ------------------------------- | ----------------- | ---------------- | -------------------------------------------- |

61 +| **Memory Usage** | ~47 GiB | ~383 GiB | 88% less RAM |

64 +| **Sample completeness** | ~100% | ~93.7% | Zero data loss |

65 +| **Query latency (2hr window)** | ~0.11s | ~1.8s | 16x faster - 22x faster in long term queries |

66 +

67 +**Critical insight:** This isn't exotic scale. **Every Netdata deployment with >500 nodes runs at millions of metrics per second.** Our users don't even notice - because the architecture absorbs it.

68 +

69 +## Architecture: Built for Planet Scale

70 +

71 +### Core Components

72 +

74 +| ---------- | --------------------------------- | --------------------------------------- | -------------------------------------- | ------------------------ |

78 +

79 +**Offloading:** Agents can offload ML, alerting, dashboards, and retention to Parents - typically cutting agent CPU ≈50%, RAM ≈25%, and eliminating disk I/O entirely.

80 +

81 +### The Edge Advantage

82 +

83 +Each Agent is autonomous:

84 +- **Collects** 3,000-20,000 metrics per second per node

85 +- **Stores** data in tiered storage (raw + aggregated)

86 +- **Detects** anomalies using unsupervised ML

87 +- **Triggers** alerts in real-time

88 +- **Serves** local dashboards and APIs

89 +- **Streams** to Parents for aggregation

90 +

91 +This means:

92 +- No data loss if Parent is down (Agents buffer locally)

93 +- No performance degradation as you scale (work stays distributed)

94 +- No architectural changes from 1 to 100,000 nodes

95 +

96 +## The Parent Advantage: Intelligent Workload Distribution

97 +

98 +### Why Parents Should Be Your Default

99 +

100 +Parents aren't just centralization points - they're intelligent workload distributors that can reduce the resource footprint on production systems.

101 +

102 +With Parents, Agents can offload:

103 +- **ML Training** - Parents train models, Agents just collect (50% CPU reduction)

104 +- **Health Checks** - Parents run all alerts, Agents focus on collection

105 +- **Persistent Storage** - Agents run in RAM-only mode with zero disk I/O

106 +- **Dashboard Serving** - Parents handle all queries and visualizations

107 +

108 +A fully offloaded Agent uses <2% CPU, <150 MiB RAM, and zero disk I/O - a fraction of a standalone Agent.

109 +

110 +### ML Intelligence: Train Where It Makes Sense

111 +

112 +Netdata's ML models flow with the metrics stream, giving you complete flexibility:

113 +

114 +**Option 1: ML at the Edge (default)**

115 +- Agents train their own models locally

116 +- Models stream to Parents along with metrics

117 +- Parents receive pre-computed ML results

118 +- Best for: Systems with available CPU, need for immediate local anomaly detection

119 +

120 +**Option 2: ML at Parents**

121 +- Agents disable ML training (50% CPU savings)

122 +- First Parent trains models for all Agents

123 +- Models shared with other Parents in cluster

124 +- Best for: Resource-constrained production systems, centralized ML management

125 +

126 +The architecture adapts to your needs - train where you have resources, use everywhere.

127 +

128 +### When You Need Parents

129 +

130 +**We recommend Parents by default:**

131 +- Future-proof your architecture (same setup works at 10 or 100,000 nodes)

132 +- Reduce production system load even at small scale

133 +- Provide unified dashboards and centralized alerting

134 +- Enable high availability and disaster recovery

135 +- Cost less than the resources they save on production systems

136 +

137 +**Parents are essential when you have:**

138 +- **Ephemeral systems** - Kubernetes pods, auto-scaling VMs that disappear

139 +- **Resource constraints** - Systems where every CPU cycle matters

140 +- **On-premises requirements** - Multi-node view without Cloud connectivity

141 +- **Network restrictions** - Agents can't reach Cloud due to firewalls/policies

142 +

143 +### Parent Sizing Guidelines

144 +

145 +| Nodes per Parent | Metrics/sec | Resources |

146 +| :--------------: | :---------: | :------------------- |

147 +| ~100 nodes | ~0.5M/sec | 5 cores, 20 GiB RAM |

148 +| ~250 nodes | ~1M/sec | 10 cores, 40 GiB RAM |

149 +| ~500 nodes | ~2M/sec | 20 cores, 80 GiB RAM |

150 +

151 +**Key principle:** Scale horizontally with more Parents, not vertically with bigger Parents. Beyond 500 nodes per Parent, resource usage grows non-linearly.

152 +

153 +### Parent Placement Strategy

154 +

155 +- **Keep Parents close** to their Agents (same datacenter, region, or cloud zone)

156 +- **Minimize network hops** to reduce latency and bandwidth costs

157 +- **Deploy per region** in multi-region architectures

158 +- **Use multiple Parents** rather than one giant Parent

159 +

160 +### High Availability & Intelligent Clustering

161 +

162 +Parents work together intelligently to eliminate duplicate work:

163 +

164 +- **Active-active Parents** with automatic work distribution

165 +- **ML model sharing** - First Parent trains, others receive models

166 +- **Automatic failover** - Agents reconnect to available Parents

167 +- **Local buffering** - Agents retain 1+ hour of data during Parent downtime

168 +- **Streaming replication** between Parents for complete redundancy

169 +- **Federated queries** across all Parents via Netdata Cloud

170 +

171 +**Key insight:** Clustering without double-spend: In an active-active cluster, the first Parent that sees a child trains the model; peers reuse it. You get HA without multiplying heavy work.

172 +

173 +### Alerts: Automation vs Monitoring

174 +

175 +Netdata separates automation from monitoring, letting you optimize both:

176 +

177 +**Agents: Local Automation**

178 +- Keep only alerts that trigger local scripts

179 +- Example: "If CPU > 90%, scale this service"

180 +- Immediate response, no network dependency

181 +- Minimal overhead when selective

182 +

183 +**Parents: Human Monitoring**

184 +- Run comprehensive health checks for all Agents

185 +- Send notifications to teams via Cloud or integrations

186 +- Correlate issues across multiple systems

187 +- Rich context for troubleshooting

188 +

189 +This dual approach means production systems only run automation-critical alerts while Parents handle the hundreds of monitoring alerts that humans need to see.

190 +

191 +## Storage: Efficient Multi-Tier Architecture

192 +

193 +### Three-Tier Storage System

194 +

196 +| ---------- | ---------- | -------------------------- | --------------- | ----------------- |

200 +

201 +All tiers update in parallel - no post-processing or compaction jobs needed.

202 +

203 +### Storage Efficiency

204 +

205 +- **0.6 bytes per sample** - industry's most efficient

206 +- **Gorilla + ZSTD compression** for optimal size/speed

207 +- **WORM design** - append-only, no expensive compaction

208 +

209 +## Why This Architecture Wins

210 +

211 +### For Operations Teams

212 +- **No blind spots** during incidents - all data available

213 +- **No architectural rewrites** as you scale

214 +- **No sampling lottery** - the metric you need is always there

215 +- **No specialized skills** required - it just works

216 +

217 +### For Finance

218 +- **Predictable costs** - linear scaling, no surprises

219 +- **Lower TCO** - fewer resources for same visibility

220 +- **Energy efficient** - independently validated lowest overhead

221 +- **Reduced team size** - less complexity to manage

222 +

223 +### For Developers

224 +- **Per-second granularity** - see what actually happened

225 +- **Real-time anomaly detection** - catch issues immediately

226 +- **Local dashboards** - debug without central bottlenecks

227 +- **Full cardinality** - every dimension tracked

228 +

229 +## The Bottom Line

230 +

231 +Through intelligent workload distribution between Parents and Agents:

232 +

233 +- **ML trains where you have resources** (edge or Parents, your choice)

234 +- **Alerts run where they matter** (automation locally, monitoring centrally)

235 +- **Storage happens where it's cheap** (Parents, not production)

236 +- **Millions of metrics per second is normal** (not heroic)

237 +- **HA doesn't multiply overhead** (intelligent clustering)

238 +

239 +This isn't just optimization. It's a fundamentally different architecture that recognizes observability shouldn't compete with your applications for resources.

240 +

241 +**Welcome to observability that makes your infrastructure better, not heavier.**

242 +

243 +## FAQ

244 +

245 +**Q: How many nodes can a single Netdata Parent handle?**

246 +A: We recommend running Parents with up to 500 Agents (1.5M metrics/s). We have customers running larger Parents, but resources increase and performance decreases non-linearly.

247 +

248 +**Q: What happens if a Parent goes down?**

249 +A: If the Parent was clustered, agents will connect to the other Parent and replicate to it any metrics collected during the transition. If there is no other Parent to connect to, Agents keep collecting and storing data locally, which will be replicated to the Parent when it becomes available. Note that the replication of past metrics uses only tier-0 (high-res data), so Agents must have enough retention in tier-0 to avoid gaps in the charts.

250 +

251 +**Q: Do I always need Parents?**

252 +A: No. Agents alone may be enough. Parents are usually required when you have ephemeral nodes.

253 +

254 +**Q: How much overhead does Netdata introduce on my systems?**

255 +A: Less than 5% CPU and ~200 MiB RAM per agent in standalone mode. Offloaded agents (streaming to a Parent) drop to <2% CPU and ~150 MiB RAM with zero disk I/O. Netdata is designed to be "polite citizen" to production workloads, so it spreads its workload across time and avoids all kinds of sudden and intense spikes.

256 +

257 +**Q: How efficient is Netdata’s storage?**

258 +A: Tier 0 (per-second) is ~0.6 bytes/sample - the industry’s most efficient. Tiers 1 & 2 keep per-minute and per-hour aggregates, letting you retain months or years of history cheaply.

259 +

260 +**Q: How do I deploy Parents in multi-region or multi-cloud setups?**

261 +A: Place Parents close to the agents they serve (same DC/region/AZ). Deploy multiple Parents per region for HA. Use Netdata Cloud to unify dashboards and queries across Parents.

262 +

263 +**Q: What’s the difference between monitoring and automation alerts?**

264 +A: Since Netdata evaluates alerts at the edge, it allows you to specify scripts to be executed when an alert triggers. This enables automation, e.g. "restart service if API endpoint is not responding".

265 +

266 +**Q: Is Netdata really energy-efficient?**

267 +A: Yes. A peer-reviewed 2023 study (ICSOC, University of Amsterdam) found Netdata to be the most energy-efficient tool among the ones tested, with the lowest CPU and RAM overhead even at 1-second collection.

268 +

269 +**Q: Is 100,000+ nodes single installation real?**

270 +A: Yes. Even Netdata Cloud SaaS itself (our commercial service) is such a single installation that serves way more than 100k reachable nodes.

271 +

272 +**Q: Do you promote per-second collection and unlimited metrics because your revenue depends on volume?**

273 +A: No. Our commercial offerings are priced per node, with volume discounts (smaller price as the number of nodes increases). Our revenue is not related to the number of metrics or the volume of observability data collected or viewed. We designed Netdata for maximum performance at scale and volume for your benefit. Not ours.

274 +

275 +**Q: If I have multiple Parents, how Netdata Cloud provides unified dashboards?**

276 +A: Think of Netdata Cloud as the headend of a distributed database. Each Netdata Parent and Agent dynamically becomes part of that database. So, Netdata Cloud queries them all in parallel, to provide the unified view required.

277 +

278 +**Q: Is querying 100 remote systems in parallel slower than querying a bigger one locally?**

279 +A: There is some extra Network latency involved, but this is usually small (a few ms), because the data transferred are tiny (your web browser will receive 500-1000 points max, even if the query is 10 days of per-second data). However, the aggregate horse power and parallelism of 100 totally independent systems is orders of magnitude more, compared to any single local system. The queries are actually quite faster.

280 +

281 +## Next Steps

282 +

283 +- **[Deploy your first Agent](./deployment/agents.md)** - Start monitoring in 60 seconds

284 +- **[Configure Parents](./deployment/parents.md)** - Scale to hundreds of nodes

285 +- **[Design for Enterprise](./deployment/enterprise.md)** - Architect for thousands

286 +- **[Try Netdata Cloud](https://netdata.cloud)** - Unified visibility across everything

287 +

288 +---

289 +

290 +*Based on real production deployments, independent research (University of Amsterdam, ICSOC 2023), and comparative testing (2025). All metrics and resource usage figures represent typical production scenarios.*