Revert "Deployment Guides: add and update documentation for deployment strate" (#20728)
Ilya Mashchenko committed
Jul 27, 2025 at 10:17 UTC
6c6ec0943f3b9031d622948ebbd496d454e3ac6e
8 files changed
+588
-999
docs/deployment-guides/README.md
+26
-56
@@ -1,72 +1,42 @@
1
# Deployment Guides
2
3
-Get Netdata up and running in your infrastructure. Choose a deployment method that fits your needs.
3
+Netdata provides real-time monitoring for various infrastructure types, from small IoT devices to complex hybrid environments that combine on-premise and cloud infrastructure. It supports bare-metal servers, virtual machines, and containers.
4
5
-## Quick Start
5
+## Core Components of a Netdata Deployment
6
7
-:::tip Getting Started
7
+A Netdata deployment consists of three main components:
8
9
-- **Testing Netdata?** → [Docker deployment](/packaging/docker/README.md) (2 minutes, easy cleanup)
10
-- **Monitoring one server?** → [Standalone installation](/docs/deployment-guides/standalone-deployment.md) (1 minute, upgradeable)
11
-- **Production ready?** → [Parent-Child setup](/docs/deployment-guides/deployment-with-centralization-points.md) (recommended)
9
+### 1. Netdata Agents
10
13
-:::
11
+Netdata Agents collect real-time metrics from your infrastructure's physical or virtual nodes, including applications and containers running on them. They are open-source and licensed under GPL v3+.
12
15
-## Deployment Methods
13
+### 2. Netdata Parents
14
17
-### Standalone
15
+Netdata Parents serve as central aggregation points for monitoring data. They help reduce the resource load on individual Netdata Agents, provide high availability for collected metrics, extend data retention, and enable better isolation of monitored nodes.
16
19
-Single Netdata Agent monitoring one system. Perfect for getting started or monitoring individual servers.
17
+- Netdata Parents are built using the same Netdata Agent software.
18
+- Any Netdata Agent can function as both an Agent for a node and a Parent for other Agents.
19
+- Deploying multiple Netdata Parents ensures redundancy and seamless integration with Netdata Cloud.
20
21
-**Best for:** Testing or simple single-server monitoring
21
+### 3. Netdata Cloud
22
23
-**Setup time:** < 1 minute
23
+Netdata Cloud is a SaaS platform that unifies all Netdata Agents and Parents into a distributed, scalable monitoring solution. It provides:
24
25
-[→ Deploy Standalone Agent](/docs/deployment-guides/standalone-deployment.md)
25
+- Centralized infrastructure monitoring
26
+- Advanced data analysis and visualization tools
27
+- Customizable dashboards
28
+- User management features
29
+- Alerting and anomaly detection capabilities
30
27
-### Parent-Child Streaming (Recommended)
31
+## Key Features of Netdata Agents
32
29
-The recommended production setup. Stream metrics from Child Agents to centralized Parent nodes for better data persistence and resource optimization.
33
+Netdata Agents offer a modular monitoring solution with capabilities that include:
34
31
-**Best for:** Production environments of any size, high availability requirements
35
+- Extensive data collection through built-in plugins
36
+- A high-performance time-series database optimized for real-time analytics
37
+- A query engine for flexible data retrieval
38
+- Integrated health monitoring and alerting
39
+- Machine learning-based anomaly detection
40
+- Exporting of metrics to third-party systems
41
33
-**Setup time:** 10-15 minutes
34
-
35
-[→ Deploy Parent-Child Setup](/docs/deployment-guides/deployment-with-centralization-points.md)
36
-
37
-### Kubernetes
38
-
39
-Deploy Netdata across your Kubernetes clusters with our Helm chart. Required for proper Kubernetes monitoring.
40
-
41
-**Best for:** Kubernetes environments (required for full K8s observability)
42
-
43
-**Setup time:** 5-10 minutes
44
-
45
-[→ Deploy on Kubernetes](https://github.com/netdata/helmchart#netdata-helm-chart-for-kubernetes-deployments)
46
-
47
-### Docker
48
-
49
-Run Netdata in containers for quick testing. Note: Some features are limited compared to host installation.
50
-
51
-**Best for:** Quick testing, ephemeral environments
52
-
53
-**Setup time:** 2-5 minutes
54
-
55
-[→ Deploy with Docker](/packaging/docker/README.md)
56
-
57
-## Which Deployment Should I Choose?
58
-
59
-| Environment | Recommended Method | Why |
60
-|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------|
61
-| **Production servers** | [Parent-Child](/docs/deployment-guides/deployment-with-centralization-points.md) | Best data persistence, resource optimization, and high availability |
62
-| **Kubernetes** | [Helm Chart](https://github.com/netdata/helmchart#netdata-helm-chart-for-kubernetes-deployments) | Required for K8s API access and pod metadata collection |
63
-| **Testing/Development** | [Standalone](/docs/deployment-guides/standalone-deployment.md) or [Docker](/packaging/docker/README.md) | Quick setup, easy to remove |
64
-| **Single server** | [Standalone](/docs/deployment-guides/standalone-deployment.md) (upgrade to [Parent-Child](/docs/deployment-guides/deployment-with-centralization-points.md) later) | Start simple, upgrade when ready for production |
65
-
66
-:::warning Important Notes
67
-
68
-- **Kubernetes**: Always use our Helm chart. Direct host installation won't have access to K8s API for pod metadata and service discovery.
69
-- **Docker**: Limited feature set compared to host installation. Best for testing, not recommended for production.
70
-- **Production**: Parent-Child is recommended regardless of cluster size for better reliability and data persistence.
71
-
72
-:::
42
+This structured deployment allows for scalable, efficient monitoring of any infrastructure, ensuring optimal performance and proactive issue resolution.
docs/deployment-guides/deployment-strategies.md
+53
-72
@@ -1,40 +1,40 @@
1
-# Configuration Examples
1
+# Deployment Examples
2
3
-These practical examples will help you optimize Netdata for various real-world deployment scenarios.
3
+## Deployment Options Overview
4
5
-## Single Agent Configuration
5
+This section provides a quick overview of a few common deployment options for Netdata.
6
7
-Single Agents work great out of the box with sensible defaults.
7
+You can read about [Standalone Deployment](/docs/deployment-guides/standalone-deployment.md) and [Deployment with Centralization Points](/docs/deployment-guides/deployment-with-centralization-points.md) in the documentation inside this section.
8
9
-:::tip
9
+The sections below go into configuration examples about these deployment concepts.
10
11
-Check out our [configuration documentation](/docs/netdata-agent/configuration/README.md) for all available options.
11
+## Deployment Configuration Details
12
13
-:::
13
+### Stand-alone
14
15
-## Parent-Child Configuration Examples
15
+The stand-alone setup is configured out of the box with reasonable defaults, but please consult our [configuration documentation](/docs/netdata-agent/configuration/README.md) for more details.
16
17
-### Lightweight Child Configuration
17
+### Parent – Child
18
19
-This example helps you make Children super lightweight by offloading work to Parents, minimizing resource usage on production systems.
19
+For setups involving Parent and Child Agents, they need to be configured for [streaming](/docs/observability-centralization-points/metrics-centralization-points/configuration.md), through the configuration file `stream.conf`.
20
21
-<details>
22
-<summary><strong>Click to see lightweight Child configuration</strong></summary><br/>
21
+This will instruct the Child to stream data to the Parent and the Parent to accept streaming connections for one or more Child Agents. To secure this connection, both need a shared API key (to replace the string `API_KEY` in the examples below). Additionally, the Child can be configured with one or more addresses of Parent Agents (`PARENT_IP_ADDRESS`).
22
24
-**Why minimize Child footprint?**
25
-We don't recommend connecting Children to Cloud directly. This reduces the Netdata Agent footprint on your production systems, as some capabilities can be switched OFF for the Child and kept ON for the Parent.
23
+An API key is a key created with `uuidgen` and is used for authentication and/or customization on the Parent side. For example, a Child can stream using the API key, and a Parent can be configured to accept connections from the Child, but it can also apply different options for Children by using multiple different API keys. The easiest setup uses just one API key for all Child Agents.
24
27
-**What this does:**
25
+#### Child config
26
29
-- Stores metrics in RAM only (zero disk I/O)
30
-- Disables machine learning (Parent handles it)
31
-- Disables alerts (Parent handles them)
32
-- Keeps only 20 minutes of data locally
33
-- Restricts dashboard to localhost only
27
+As mentioned above, we do not recommend connecting the Child to Cloud directly during your setup.
28
35
-Edit `netdata.conf` on the Child using the [edit-config](/docs/netdata-agent/configuration/README.md#locate-your-config-directory) script:
29
+This is done to reduce the footprint of the Netdata Agent on your production system, as some capabilities can be switched OFF for the Child and kept ON for the Parent.
30
37
-```ini
31
+In this example, Machine Learning and Alerting are disabled for the Child, so that the Parent can take the load. We also use RAM instead of disk to store metrics with limited retention, covering temporary network issues.
32
+
33
+##### netdata.conf
34
+
35
+On the child node, edit `netdata.conf` by using the [edit-config](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script and set the following parameters:
36
+
37
+```text
38
[db]
39
# https://github.com/netdata/netdata/blob/master/src/database/README.md
40
# none = no retention, ram = some retention in ram
@@ -60,9 +60,11 @@ Edit `netdata.conf` on the Child using the [edit-config](/docs/netdata-agent/con
60
# enable running new plugins = no
61
```
62
63
-Edit `stream.conf` on the Child using the [edit-config](/docs/netdata-agent/configuration/README.md#locate-your-config-directory) script:
63
+##### stream.conf
64
65
-```ini
65
+To edit `stream.conf`, use again the [edit-config](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script and set the following parameters:
66
+
67
+```text
68
[stream]
69
# Stream metrics to another Netdata
70
enabled = yes
@@ -72,33 +74,24 @@ Edit `stream.conf` on the Child using the [edit-config](/docs/netdata-agent/conf
74
api key = API_KEY
75
```
76
75
-**Perfect for:** IoT devices, containers, or any resource-constrained system.
76
-
77
-</details><br/>
77
+#### Parent config
78
79
-### Parent with Tiered Storage
79
+For the Parent, besides setting up streaming, this example also provides configuration for multiple [tiers of metrics storage](/src/database/README.md#tiers), for 10 Children, with about 2k metrics each. This allows for:
80
81
-This example helps you configure a Parent with intelligent storage tiers to store different time ranges at different resolutions.
81
+- 1s granularity at tier 0 for 1 week
82
+- 1m granularity at tier 1 for 1 month
83
+- 1h granularity at tier 2 for 1 year
84
83
-<details>
84
-<summary><strong>Click to see Parent with tiered storage configuration</strong></summary><br/>
85
+Requiring:
86
86
-This example provides configuration for multiple [tiers of metrics storage](/src/database/README.md#tiers), for 10 Children with about 2k metrics each.
87
+- 25GB of disk
88
+- 3.5GB of RAM (2.5GB under pressure)
89
88
-**What this gives you:**
90
+##### netdata.conf
91
90
-- 1-second resolution for 1 week (recent data at full detail)
91
-- 1-minute resolution for 1 month (medium-term trends)
92
-- 1-hour resolution for 1 year (long-term patterns)
92
+On the Parent, edit `netdata.conf` by using the [edit-config](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script and set the following parameters:
93
94
-**Resource requirements:**
95
-
96
-- Disk space: 25GB total
97
-- RAM usage: 3.5GB typical (2.5GB under memory pressure)
98
-
99
-Edit `netdata.conf` on the Parent using the [edit-config](/docs/netdata-agent/configuration/README.md#locate-your-config-directory) script:
100
-
101
-```ini
94
+```text
95
[db]
96
mode = dbengine
97
dbengine tier backfill = new
@@ -124,36 +117,25 @@ Edit `netdata.conf` on the Parent using the [edit-config](/docs/netdata-agent/co
117
# bind to = *
118
```
119
127
-Edit `stream.conf` on the Parent using the [edit-config](/docs/netdata-agent/configuration/README.md#locate-your-config-directory) script:
120
+##### stream.conf
121
+
122
+On the Parent node, edit `stream.conf` by using the [edit-config](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script and set the following parameters:
123
129
-```ini
124
+```yaml
125
[API_KEY]
131
- # Accept metrics streaming from other Agents with the specified API key
132
- enabled = yes
126
+ # Accept metrics streaming from other Agents with the specified API key
127
+ enabled = yes
128
```
129
135
-**Perfect for:** Central monitoring servers with enough storage for historical data.
136
-
137
-</details><br/>
138
-
139
-### Active-Active Parents
140
-
141
-This example guides you through setting up active-active Parents that sync with each other for high availability.
142
-
143
-<details>
144
-<summary><strong>Click to see high availability configuration</strong></summary><br/>
130
+### Active–Active Parents
131
146
-:::info
132
+To set up active–active streaming between Parent 1 and Parent 2, Parent 1 needs to be instructed to stream data to Parent 2 and Parent 2 to stream data to Parent 1. The Child Agents need to be configured with the addresses of both Parent Agents. An Agent will only connect to one Parent at a time, falling back to the next upon failure. These examples use the same API key between Parent Agents and for connections for Child Agents.
133
148
-- To set up active-active streaming between Parent 1 and Parent 2, Parent 1 needs to be instructed to stream data to Parent 2 and Parent 2 to stream data to Parent 1.
134
+On both Netdata Parent and all Child Agents, edit `stream.conf` by using the [edit-config](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script:
135
150
-- The Child Agents need to be configured with the addresses of both Parent Agents. An Agent will only connect to one Parent at a time, falling back to the next upon failure.
136
+#### stream.conf on Parent 1
137
152
-:::
153
-
154
-**Parent 1 stream.conf:**
155
-
156
-```ini
138
+```text
139
[stream]
140
# Stream metrics to another Netdata
141
enabled = yes
@@ -164,11 +146,12 @@ This example guides you through setting up active-active Parents that sync with
146
[API_KEY]
147
# Accept metrics streams from Parent 2 and Child Agents
148
enabled = yes
149
+
150
```
151
169
-**Parent 2 stream.conf:**
152
+#### stream.conf on Parent 2
153
171
-```ini
154
+```text
155
[stream]
156
# Stream metrics to another Netdata
157
enabled = yes
@@ -180,9 +163,9 @@ This example guides you through setting up active-active Parents that sync with
163
enabled = yes
164
```
165
183
-**Children stream.conf:**
166
+#### stream.conf on Child Agents
167
185
-```ini
168
+```text
169
[stream]
170
# Stream metrics to another Netdata
171
enabled = yes
@@ -192,8 +175,6 @@ This example guides you through setting up active-active Parents that sync with
175
api key = API_KEY
176
```
177
195
-</details><br/>
196
-
178
## Further Reading
179
180
We strongly recommend the following configuration changes for production deployments:
docs/deployment-guides/deployment-with-centralization-points.md
+97
-179
@@ -1,204 +1,122 @@
1
-# Parents: Your Centralization Points
1
+# Deployment with Centralization Points
2
3
-***Parents*** are Netdata Agents that collect and store data from other Agents ("***Children***"). They act as centralization points for your observability data.
3
+An observability centralization point can centralize both metrics and logs. The sending systems are called Children, while the receiving systems are called Parents.
4
5
-## How It Works
5
+When metrics and logs are centralized, the Children are never queried for metrics and logs. The Netdata Parents have all the data needed to satisfy queries.
6
7
-1. **You designate some Agents as Parents** - Configure them to receive streaming data
8
-2. **Children stream their data to Parents** - They push metrics continuously
9
-3. **Parents store and process everything** - All metrics and logs from all Children
10
-4. **You access Parents for dashboards and alerts** - Centralized monitoring interface
11
-5. **Cloud queries Parents when configured** - Reduces load on production systems
7
+- **Metrics** are centralized by Netdata, with a feature we call **Streaming**. The Parents listen for incoming connections and permit access only to Children that connect to it with the right API key. Children are configured to push their metrics to the Parents, and they initiate the connections to do so.
8
13
-:::info
9
+- **Logs** are centralized with methodologies provided by `systemd-journald`. This involves installing `systemd-journal-remote` on both the Parent and the Children, and configuring the keys required for this communication.
10
15
-Parents give you centralized collection with distributed architecture benefits.
11
+| Feature | How it works |
12
+|:---------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|
13
+| Unified infrastructure dashboards for metrics | Yes, at Netdata Cloud |
14
+| Unified infrastructure dashboards for logs | All logs are accessible via the same dashboard at Netdata Cloud, although they are unified per Netdata Parent |
15
+| Centrally configured alerts | Yes, at Netdata Parents |
16
+| Centrally dispatched alert notifications | Yes, at Netdata Cloud |
17
+| Data are exclusively on-prem | Yes, Netdata Cloud queries Netdata Agents to satisfy dashboard queries. |
18
17
-:::
19
+A configuration with 2 observability centralization points looks like this:
20
19
-## What Parents Do
20
-
21
-Parents are specialized Netdata installations that you can configure to **receive, store, and process** observability data (metrics and logs) from multiple other systems in your infrastructure.
22
-
23
-These Parents give you several core functions:
24
-
25
-* **Receiving and storing** metrics and logs from multiple systems
26
-* **Processing and analyzing** your collected data
27
-* **Running health checks and alerts**
28
-* Providing **unified dashboards** across all your systems
29
-* **Replicating data** for your historical analysis
30
-
31
-:::info
32
-
33
-This **distributed yet centralized** approach gives you the benefits of both decentralized collection and centralized analysis.
34
-
35
-:::
36
-
37
-## Why Use Parents
38
-
39
-| Use Case | Description | Benefits |
40
-|-------------------------------------------------|------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
41
-| **Ephemeral Systems** | Ideal for your Kubernetes nodes or temporary VMs that frequently go offline | You retain metrics and logs for analysis and troubleshooting even after node termination |
42
-| **Limited Resources** | Offloads observability tasks from your systems with low disk space, CPU, RAM, or I/O bandwidth | Your production systems run efficiently without performance trade-offs |
43
-| **Multi-Node Dashboards Without Netdata Cloud** | Aggregates data from all your nodes for centralized dashboards | You get Cloud-like functionality in environments that prefer or require on-premises solutions |
44
-| **Restricted Netdata Cloud Access** | Acts as a bridge when your monitored systems can't connect to Netdata Cloud | You can still use Cloud features despite firewall restrictions or security policies |
21
+```mermaid
22
+flowchart LR
23
+ WEB[["One unified
24
+ dashboard
25
+ for all nodes"]]
26
+ NC(["<b>Netdata Cloud</b>
27
+ decides which Agents
28
+ need to be queried"])
29
+ SA1["Netdata at AWS
30
+ A1"]
31
+ SA2["Netdata at AWS
32
+ A2"]
33
+ SAN["Netdata at AWS
34
+ AN"]
35
+ PA["<b>Netdata Parent A</b>
36
+ at AWS
37
+ having all metrics & logs
38
+ for all Ax nodes"]
39
+ SB1["Netdata On-Prem
40
+ B1"]
41
+ SB2["Netdata On-Prem
42
+ B2"]
43
+ SBN["Netdata On-Prem
44
+ BN"]
45
+ PB["<b>Netdata Parent B</b>
46
+ On-Prem
47
+ having all metrics & logs
48
+ for all Bx nodes"]
49
+ WEB -->|query| NC -->|query| PA & PB
50
+ PA ---|stream| SA1 & SA2 & SAN
51
+ PB ---|stream| SB1 & SB2 & SBN
52
+```
53
46
-## How Multiple Parents Work
54
+Netdata Cloud queries the Netdata Parents to provide aggregated dashboard views.
55
48
-<details>
49
-<summary><strong>Click to see Parent architecture options</strong></summary><br/>
56
+For alerts, the dispatch of notifications looks like in the following chart:
57
58
```mermaid
52
-flowchart TB
53
- subgraph architectures["Parent Architecture Options"]
54
- direction TB
55
-
56
- subgraph single["Single Parent"]
57
- SP[SP]
58
- SC1[SC1]
59
- SC2[SC2]
60
- SC3[SC3]
61
- SP("**Parent**<br/>All data in one place")
62
- SC1("Child 1")
63
- SC2("Child 2")
64
- SC3("Child 3")
65
- SC1 --> SP
66
- SC2 --> SP
67
- SC3 --> SP
68
- end
69
-
70
- subgraph multiple["Multiple Parents"]
71
- MP1[MP1]
72
- MP2[MP2]
73
- MC1[MC1]
74
- MC2[MC2]
75
- MC3[MC3]
76
- MC4[MC4]
77
- MP1("**Parent 1**<br/>Region/Team A")
78
- MP2("**Parent 2**<br/>Region/Team B")
79
- MC1("Child 1")
80
- MC2("Child 2")
81
- MC3("Child 3")
82
- MC4("Child 4")
83
- MC1 --> MP1
84
- MC2 --> MP1
85
- MC3 --> MP2
86
- MC4 --> MP2
87
- end
88
-
89
- subgraph ha["High Availability"]
90
- HP1[HP1]
91
- HP2[HP2]
92
- HC1[HC1]
93
- HC2[HC2]
94
- HP1("**Parent 1**<br/>Active")
95
- HP2("**Parent 2**<br/>Active")
96
- HC1("Child 1")
97
- HC2("Child 2")
98
- HC1 --> HP1
99
- HC2 --> HP1
100
- HC1 -.-> HP2
101
- HC2 -.-> HP2
102
- HP1 <--> HP2
103
- end
104
- end
105
-
106
- classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
107
- classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
108
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
109
- classDef innerStyle fill: #f0f8ff, stroke: #87ceeb, stroke-width: 2px, color: #2c3e50, rx: 12, ry: 12
110
- class SP parent
111
- class MP1 parent
112
- class MP2 parent
113
- class HP1 parent
114
- class HP2 parent
115
- class SC1 child
116
- class SC2 child
117
- class SC3 child
118
- class MC1 child
119
- class MC2 child
120
- class MC3 child
121
- class MC4 child
122
- class HC1 child
123
- class HC2 child
124
- class architectures subgraphStyle
125
- class single innerStyle
126
- class multiple innerStyle
127
- class ha innerStyle
59
+flowchart LR
60
+ NC(["<b>Netdata Cloud</b>
61
+ applies silencing
62
+ & user settings"])
63
+ SA1["Netdata at AWS
64
+ A1"]
65
+ SA2["Netdata at AWS
66
+ A2"]
67
+ SAN["Netdata at AWS
68
+ AN"]
69
+ PA["<b>Netdata Parent A</b>
70
+ at AWS
71
+ having all metrics & logs
72
+ for all Ax nodes"]
73
+ SB1["Netdata On-Prem
74
+ B1"]
75
+ SB2["Netdata On-Prem
76
+ B2"]
77
+ SBN["Netdata On-Prem
78
+ BN"]
79
+ PB["<b>Netdata Parent B</b>
80
+ On-Prem
81
+ having all metrics & logs
82
+ for all Bx nodes"]
83
+ EMAIL[["<b>e-mail</b>
84
+ notifications"]]
85
+ MOBILEAPP[["<b>Netdata Mobile App</b>
86
+ notifications"]]
87
+ SLACK[["<b>Slack</b>
88
+ notifications"]]
89
+ OTHER[["Other
90
+ notifications"]]
91
+ PA & PB -->|alert transitions| NC -->|notification| EMAIL & MOBILEAPP & SLACK & OTHER
92
+ SA1 & SA2 & SAN ---|stream| PA
93
+ SB1 & SB2 & SBN ---|stream| PB
94
```
95
130
-</details><br/>
96
+## Active–Active Parent Deployment
97
132
-| Scenario | Operation | Advantages |
133
-|-----------------------------|------------------------------------------------------------|--------------------------------------------------------------------------|
134
-| **With Netdata Cloud** | Queries all your Parents in parallel for a unified view | You get a seamless experience regardless of your underlying architecture |
135
-| **Without Netdata Cloud** | Your Parents consolidate data from connected systems | You have a local view of metrics and logs without external dependencies |
136
-| **High Availability Setup** | Your Parents share data with each other, forming a cluster | You won't lose data if one Parent fails |
98
+For high availability, Parents can be configured to stream data for their Children between them, and keep their data sets in sync. Children are configured with the addresses of both Parents, but will only stream to one of them at a time. When one Parent becomes unavailable, the Child reconnects to the other. When the first Parent becomes available again, that Parent will catch up by receiving the backlog from the second.
99
138
-## Technical Implementation
100
+With both Parent Agents connected to Netdata Cloud, it will route queries to either of them transparently, depending on their availability. Alerts trigger on either Parent will stream to Cloud, and Cloud will deduplicate and debounce state changes to prevent spurious notifications.
101
140
-Parents consist of two major components you can deploy:
102
+## Configuration steps for deploying Netdata with Observability Centralization Points
103
142
-1. **Metrics Centralization** - Uses Netdata's streaming and replication features to centralize your metrics data
143
-2. **Logs Centralization** - Uses systemd-journald methodologies to centralize your log data
104
+For Metrics:
105
145
-You can configure your systems to connect to **multiple Parents** for redundancy. If a connection fails, they automatically switch to an available alternative.
106
+- Install Netdata Agents on all systems and the Netdata Parents.
107
147
-In a **high-availability setup**, your Parents can form a cluster by sharing data with each other, ensuring all points have a complete copy of all your metrics and logs.
108
+- Configure `stream.conf` at the Netdata Parents to enable streaming access with an API key.
109
149
-<details>
150
-<summary><strong>Click to see how high availability works</strong></summary><br/>
110
+- Configure `stream.conf` at the Netdata Children to enable streaming to the configured Netdata Parents.
111
152
-```mermaid
153
-flowchart TB
154
- NC[NC]
155
- NC("**Netdata Cloud**<br/>Queries available Parents")
156
-
157
- subgraph infrastructure["Your Infrastructure"]
158
- direction TB
159
- P1[P1]
160
- P2[P2]
161
- C1[C1]
162
- C2[C2]
163
- C3[C3]
164
- C4[C4]
165
- P1("**Parent 1**<br/>Active")
166
- P2("**Parent 2**<br/>Active")
167
- C1("Child 1")
168
- C2("Child 2")
169
- C3("Child 3")
170
- C4("Child 4")
171
- C1 -->|primary| P1
172
- C2 -->|primary| P1
173
- C3 -->|primary| P2
174
- C4 -->|primary| P2
175
- C1 -.->|failover| P2
176
- C2 -.->|failover| P2
177
- C3 -.->|failover| P1
178
- C4 -.->|failover| P1
179
- P1 <-->|sync| P2
180
- end
181
-
182
- NC <--> P1
183
- NC <--> P2
184
- classDef cloud fill: #e8f4fd, stroke: #4a90e2, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
185
- classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
186
- classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
187
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
188
- class NC cloud
189
- class P1 parent
190
- class P2 parent
191
- class C1 child
192
- class C2 child
193
- class C3 child
194
- class C4 child
195
- class infrastructure subgraphStyle
196
-```
112
+Check the [related section in our documentation](/docs/observability-centralization-points/metrics-centralization-points/README.md) for more info.
113
+
114
+For Logs:
115
198
-</details><br/>
116
+- Install `systemd-journal-remote` on all systems and the Netdata Parents.
117
200
-:::tip
118
+- Configure `systemd-journal-remote` at the Netdata Parents to enable logs reception.
119
202
-Check out our [Parent-Child Deployment Guide](/docs/deployment-guides/deployment-with-centralization-points.md) for step-by-step instructions.
120
+- Configure `systemd-journal-upload` at the Netdata Children to enable transmission of their logs to the Netdata Parents.
121
204
-:::
122
+Check the [related section in our documentation](/docs/observability-centralization-points/logs-centralization-points-with-systemd-journald/README.md) for more info.
\ No newline at end of file
docs/deployment-guides/standalone-deployment.md
+76
-141
@@ -1,167 +1,102 @@
1
-# Single Agent Deployment
1
+# Standalone Deployment
2
3
-The simplest way to use Netdata - install it, and you're monitoring. Each Agent works independently with zero configuration.
3
+Netdata provides real-time monitoring out of the box. By default, each Netdata Agent functions as a standalone monitoring system with no additional configuration required.
4
5
-## Single Agents With Netdata Cloud (Recommended)
5
+## Standalone Agents Without Netdata Cloud
6
7
-Get the best experience - one dashboard for all your systems, mobile alerts, and team collaboration. Your data stays on your servers.
7
+Each Netdata Agent operates independently and provides its own monitoring dashboard and alerting system.
8
9
-### What You Get
9
+### Features
10
11
-| Feature | How it Works |
12
-|---------------------------------|------------------------------------------|
13
-| **Unified metrics dashboard** | ✓ See all Agents in one place |
14
-| **Unified logs view** | ✓ Access all logs from Cloud |
15
-| **Central alert configuration** | Each Agent still manages its own alerts |
16
-| **Central notifications** | ✓ Cloud handles all notifications |
17
-| **Data stays on-premise** | ✓ Cloud queries your Agents in real-time |
11
+| Feature | How it works |
12
+|---------|-------------|
13
+| **Infrastructure dashboards for metrics** | No, each Netdata Agent provides its own dashboard. |
14
+| **Infrastructure dashboards for logs** | No, logs are only accessible per individual Netdata Agent. |
15
+| **Centralized alert configuration** | No, each Netdata Agent has its own alert settings. |
16
+| **Centralized alert notifications** | No, each Netdata Agent sends notifications independently. |
17
+| **On-prem data retention** | Yes, all collected data remains on the monitored system. |
18
19
-<details>
20
-<summary><strong>Click to see visual representation of the architecture</strong></summary><br/>
19
+Each Netdata Agent is accessible via a unique URL: `http://agent-ip:19999`.
20
21
```mermaid
23
-flowchart TB
24
- NC[NC]
25
- Users[Users]
26
- Notifications[Notifications]
27
- NC("**Netdata Cloud**<br/>• Unified dashboards<br/>• Central notifications<br/>• Access from anywhere")
28
- Users("**One Dashboard**<br/>for all your systems")
29
- Notifications("**Alert Notifications**<br/>Email, Slack, Mobile App")
30
- Users <--> NC
31
- NC --> Notifications
32
-
33
- subgraph infrastructure["Your Infrastructure"]
34
- direction TB
35
- Agents[Agents]
36
- Data[Data]
37
- Agents("**Netdata Agents**<br/>Agent 1, Agent 2, Agent 3")
38
- Data("**Your Metrics**<br/>Stay on your servers")
39
- Agents <--> Data
40
- end
41
-
42
- NC <--> Agents
43
- classDef cloud fill: #e8f4fd, stroke: #4a90e2, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
44
- classDef users fill: #fff2e8, stroke: #f39c12, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
45
- classDef notifications fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
46
- classDef agents fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
47
- classDef data fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
48
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
49
- class NC cloud
50
- class Users users
51
- class Notifications notifications
52
- class Agents agents
53
- class Data data
54
- class infrastructure subgraphStyle
22
+flowchart LR
23
+ WEB["Multiple Independent Dashboards"]
24
+ S1["Standalone Netdata 1"]
25
+ S2["Standalone Netdata 2"]
26
+ SN["Standalone Netdata N"]
27
+ WEB -->|URL 1| S1
28
+ WEB -->|URL 2| S2
29
+ WEB -->|URL N| SN
30
```
31
57
-</details>
58
-
59
-### Setup
60
-
61
-Getting started is simple:
62
-
63
-1. **Sign up for Netdata Cloud** (it's free)
64
-2. **Get your connection command** - Once logged in, you have three ways to get it:
65
- - Navigate to **Space Settings** → **Nodes** → Click **"+"**
66
- - Go to **Nodes** tab → Click **Add nodes**
67
- - Visit **Integrations** page → Select your OS
68
-
69
-3. **Run the installation command** that includes your unique claim token and room information
70
-
71
-**What happens next:**
72
-
73
-- The command automatically detects your OS
74
-- Installs the latest Netdata Agent
75
-- Connects to your Cloud Space
76
-- Your node appears live in seconds
77
-- Charts start streaming real-time data immediately
78
-
79
-:::tip
80
-
81
-Get detailed instructions on how to connect Agents to Cloud in our [Connect Agent to Cloud Guide](https://github.com/netdata/netdata/blob/master/src/claim/README.md).
82
-
83
-:::
84
-
85
-### Optional Optimizations
86
-
87
-- Disable local Agent notifications (Cloud handles them better)
88
-- Restrict local dashboard access for security (use Cloud instead)
89
-
90
-## Single Agents Without Cloud
91
-
92
-You can also run Agents independently, though you'll miss out on unified dashboards and mobile alerts.
93
-
94
-<details>
95
-<summary><strong>Click to see visual representation of standalone architecture</strong></summary><br/>
32
+Each agent also manages its own alert notifications:
33
34
```mermaid
98
-flowchart TB
99
- subgraph infrastructure["Your Infrastructure"]
100
- direction TB
101
- A1["Agent 1"]
102
- A2["Agent 2"]
103
- A3["Agent 3"]
104
- D1["Dashboard 1<br/>:19999"]
105
- D2["Dashboard 2<br/>:19999"]
106
- D3["Dashboard 3<br/>:19999"]
107
- N1["Alerts"]
108
- N2["Alerts"]
109
- N3["Alerts"]
110
- A1 --> D1
111
- A2 --> D2
112
- A3 --> D3
113
- A1 --> N1
114
- A2 --> N2
115
- A3 --> N3
116
- end
117
-
118
- classDef agents fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
119
- classDef dashboards fill: #fff2e8, stroke: #f39c12, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
120
- classDef alerts fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
121
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
122
- class A1 agents
123
- class A2 agents
124
- class A3 agents
125
- class D1 dashboards
126
- class D2 dashboards
127
- class D3 dashboards
128
- class N1 alerts
129
- class N2 alerts
130
- class N3 alerts
131
- class infrastructure subgraphStyle
35
+flowchart LR
36
+ S1["Standalone Netdata 1"]
37
+ S2["Standalone Netdata 2"]
38
+ SN["Standalone Netdata N"]
39
+ EMAIL["Email notifications"]
40
+ SLACK["Slack notifications"]
41
+ OTHER["Other notifications"]
42
+ S1 & S2 & SN .-> SLACK
43
+ S1 & S2 & SN ---> EMAIL
44
+ S1 & S2 & SN ==> OTHER
45
```
46
134
-</details>
135
-
136
-### Setup
47
+### Configuration Steps
48
138
-:::tip
49
+- Install Netdata Agents on each system.
50
+- Access each Agent individually via its URL (`http://agent-ip:19999`).
51
140
-Check the [Version & Platform](https://learn.netdata.cloud/docs/netdata-agent/versions-&-platforms) that's suitable for your needs and install the Agent.
52
+## Standalone Agents With Netdata Cloud
53
142
-:::
54
+Connecting Netdata Agents to Netdata Cloud enables centralized monitoring while keeping collected data on-premise.
55
144
-1. Install Netdata on each system
145
-2. Access each dashboard at `http://agent-ip:19999`
146
-3. Configure alerts individually on each Agent
56
+### Features
57
148
-## When to Use Each Approach
58
+| Feature | Description |
59
+|---------|-------------|
60
+| **Infrastructure dashboards for metrics** | Yes, Netdata Cloud provides unified charts aggregating metrics from all systems. |
61
+| **Infrastructure dashboards for logs** | Logs from all agents are accessible in Netdata Cloud (though not merged into a single view). |
62
+| **Centralized alert configuration** | No, each Netdata Agent maintains its own alert settings. |
63
+| **Centralized alert notifications** | Yes, Netdata Cloud manages and dispatches notifications. |
64
+| **On-prem data retention** | Yes, Netdata Cloud queries Netdata Agents in real time. |
65
150
-**Use Cloud-connected Agents when:**
66
+Connecting Netdata Agents to Netdata Cloud enables a unified monitoring view without requiring additional infrastructure setup.
67
152
-- You want one dashboard for everything
153
-- You need mobile alerts
154
-- Multiple people need access
155
-- You're managing more than one system
68
+```mermaid
69
+flowchart LR
70
+ WEB["Unified Dashboard for All Nodes"]
71
+ NC["Netdata Cloud"]
72
+ S1["Standalone Netdata 1"]
73
+ S2["Standalone Netdata 2"]
74
+ SN["Standalone Netdata N"]
75
+ WEB -->|queries| NC
76
+ NC -->|queries| S1 & S2 & SN
77
+```
78
157
-**Use standalone Agents only when:**
79
+Alert notifications are managed centrally in Netdata Cloud:
80
159
-- You have strict air-gapped requirements
160
-- You're testing on a single system
161
-- Cloud connectivity is not possible
81
+```mermaid
82
+flowchart LR
83
+ EMAIL["Email notifications"]
84
+ MOBILEAPP["Netdata Mobile App notifications"]
85
+ SLACK["Slack notifications"]
86
+ OTHER["Other notifications"]
87
+ NC["Netdata Cloud"]
88
+ S1["Standalone Netdata 1"]
89
+ S2["Standalone Netdata 2"]
90
+ SN["Standalone Netdata N"]
91
+ NC -->|notification| EMAIL & MOBILEAPP & SLACK & OTHER
92
+ S1 & S2 & SN -->|alert transition| NC
93
+```
94
163
-:::note
95
+> **Note:** Alerts are still triggered by Netdata Agents. Netdata Cloud manages notifications.
96
165
-Without Netdata Cloud, each Agent operates independently - you'll need to check multiple dashboards, configure alerts on each system separately, and won't receive mobile notifications. Cloud connection is free and keeps your data on-premise while providing a unified view.
97
+### Configuration Steps
98
167
-:::
99
+- Install Netdata Agents using the installation commands provided by Netdata Cloud to automatically link them to your Space.
100
+- Alternatively, install Netdata Agents manually and connect them via the command line or dashboard.
101
+- **Optional:** Disable direct dashboard access for security.
102
+- **Optional:** Disable individual agent notifications to prevent duplicate alerts (Netdata Agents send email alerts by default if an MTA is detected).
\ No newline at end of file
docs/nodes-ephemerality.md
+109
-123
@@ -1,186 +1,172 @@
1
-# Node Types and Lifecycle Strategies
2
-
3
-Netdata categorizes nodes as **ephemeral** or **permanent** to help you tailor alerting, cleanup, and monitoring strategies for dynamic or static infrastructures.
1
+# Nodes Ephemerality in Netdata
2
3
## Node Types
4
7
-| Type | Description | Common Use Cases |
8
-|---------------|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
9
-| **Ephemeral** | Expected to disconnect or reconnect frequently | • Auto-scaling cloud instances<br />• Dynamic containers and VMs<br />• IoT devices with intermittent connectivity<br />• Test environments |
10
-| **Permanent** | Expected to maintain continuous connectivity | • Production servers<br />• Core infrastructure nodes<br />• Critical monitoring systems<br />• Stable database servers |
5
+Netdata categorizes nodes into two types:
6
+
7
+| Type | Description | Common Use Cases |
8
+|---------------|------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
9
+| **Ephemeral** | Expected to disconnect or reconnect frequently | • Auto-scaling cloud instances<br />• Dynamic containers and VMs<br />• IoT devices with intermittent connectivity<br />• Development/test environments with frequent restarts |
10
+| **Permanent** | Expected to maintain continuous connectivity | • Production servers<br />• Core infrastructure nodes<br />• Critical monitoring systems<br />• Stable database servers |
11
12
:::note
13
14
-Disconnections in **permanent nodes** may indicate system failures and require immediate attention.
14
+Disconnections in permanent nodes indicate potential system failures and require immediate attention.
15
16
:::
17
18
-### Key Benefits of Ephemeral Nodes
18
+### Key Benefits
19
20
-1. **Reduced Alert Noise**: Disconnection alerts apply only to permanent nodes.
21
-2. **Support for Dynamic Infrastructure**: Designate temporary resources as ephemeral to avoid false alarms.
22
-3. **Automated Cleanup**: Configure retention policies for ephemeral nodes to keep dashboards uncluttered.
20
+1. **Reduced Alert Noise**: Disconnection alerts now apply only to permanent nodes, helping you focus on actual issues.
21
+2. **Improved Dynamic Infrastructure Support**: You can designate auto-scaling cloud instances, containers, and other temporary resources as ephemeral to prevent unnecessary alerts.
22
+3. **Automated Node Cleanup**: You can configure ephemeral nodes to be automatically removed based on your preferred retention periods, keeping your dashboards relevant and uncluttered.
23
24
## Configuring Ephemeral Nodes
25
26
+```mermaid
27
+flowchart TD
28
+ A[Start:<br> Node Permanent by Default] -->|Step 1| B[Open netdata.conf<br> on Target Node]
29
+ B -->|Step 2| C["Add Configuration"]
30
+ C -->|Step 3| D[Restart the Node]
31
+ D --> E[Node Now Marked<br>Ephemeral]
32
+ E --> F[_is_ephemeral<br>Label Applied]
33
+ F --> G[Label Propagates to Parents<br>and Cloud]
34
+
35
+ style A fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
36
+ style B fill:#4caf50,stroke:#333,stroke-width:1px,color:white
37
+ style C fill:#4caf50,stroke:#333,stroke-width:1px,color:white
38
+ style D fill:#4caf50,stroke:#333,stroke-width:1px,color:white
39
+ style E fill:#ffeb3b,stroke:#333,stroke-width:1px,color:black
40
+ style F fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
41
+ style G fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
42
+```
43
+
44
By default, Netdata treats all nodes as permanent. To mark a node as ephemeral:
45
46
1. Open the `netdata.conf` file on the target node.
47
2. Add the following configuration:
30
-
48
```ini
49
[global]
33
- is ephemeral node = yes
50
+ is ephemeral node = yes
51
```
52
+3. Restart the node.
53
36
-3. Restart the Netdata Agent.
37
-
38
-This applies the `_is_ephemeral` host label, which propagates to your Parents and Netdata Cloud.
39
-
40
-<details>
41
-<summary><strong>Click to see visual representation of configuration flow</strong></summary><br/>
42
-
43
-```mermaid
44
-flowchart TD
45
- A[Node is Permanent by Default] -->|Step 1| B[Open netdata.conf on Target Node]
46
- B -->|Step 2| C[Add Configuration]
47
- C -->|Step 3| D[Restart the Node]
48
- D --> E[Node Now Marked as Ephemeral]
49
- E --> F[_is_ephemeral Label Applied]
50
- F --> G[Label Propagates to Parents and Cloud]
51
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
52
- classDef label fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
53
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
54
- class A step
55
- class B step
56
- class C step
57
- class D step
58
- class E label
59
- class F label
60
- class G subgraphStyle
61
-```
62
-
63
-</details>
54
+This setting applies the `_is_ephemeral` host label, which propagates to your Netdata Parents and Netdata Cloud.
55
56
## Alerts for Parent Nodes
57
67
-Netdata v2.3.0 introduces two alerts specific to permanent nodes:
58
+Netdata v2.3.0 introduces two new alerts specifically for permanent nodes:
59
69
-| Alert | Trigger Condition |
70
-|-----------------------------|---------------------------------------------------------|
71
-| `streaming_never_connected` | A permanent node has never connected to a Parent. |
72
-| `streaming_disconnected` | A previously connected permanent node has disconnected. |
60
+| Alert | Trigger Condition |
61
+|-----------------------------|-----------------------------------------------------------|
62
+| `streaming_never_connected` | A permanent node has never connected to a Netdata Parent. |
63
+| `streaming_disconnected` | A previously connected permanent node has disconnected. |
64
74
-## Monitoring and Managing Node Status
65
+## Monitoring Child Node Status
66
76
-### Mark Permanently Offline Nodes as Ephemeral
67
+```mermaid
68
+flowchart TD
69
+ A[Start:<br> Permanently Offline Node] -->|Run CLI Command| B[Use netdatacli Utility]
70
+ B -->|Specify Target| C[Specify Node<br> to Mark as Ephemeral]
71
+ C --> D[Node Marked as Ephemeral]
72
+ D --> E[Metrics Data Remains<br> Available]
73
+ D --> F[Active Alerts Cleared]
74
+ D --> G{Node Reconnects?}
75
+ G -->|Yes, without config| H[Reverts to<br> Permanent Status]
76
+ G -->|No| I[Remains Ephemeral]
77
+
78
+ style A fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
79
+ style B fill:#4caf50,stroke:#333,stroke-width:1px,color:white
80
+ style C fill:#4caf50,stroke:#333,stroke-width:1px,color:white
81
+ style D fill:#ffeb3b,stroke:#333,stroke-width:1px,color:black
82
+ style E fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
83
+ style F fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
84
+ style G fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
85
+ style H fill:#f44336,stroke:#333,stroke-width:1px,color:white
86
+ style I fill:#ffeb3b,stroke:#333,stroke-width:1px,color:black
87
+```
88
78
-To mark nodes (including virtual ones) as ephemeral:
89
+To mark permanently offline nodes, including virtual nodes, as ephemeral:
90
91
```bash
92
netdatacli mark-stale-nodes-ephemeral <node_id | machine_guid | hostname | ALL_NODES>
93
```
94
84
-This keeps historical data queryable and clears active alerts.
95
+This keeps the previously collected metrics data available for querying and clears any active alerts.
96
86
-<details>
87
-<summary><strong>Click to see visual representation of CLI workflow</strong></summary><br/>
97
+:::note
98
+
99
+Nodes will revert to permanent status if they reconnect unless explicitly configured as ephemeral in `netdata.conf`.
100
+
101
+:::
102
+
103
+### remove-stale-node
104
105
```mermaid
106
flowchart TD
91
- A[Offline Node Detected] -->|Run CLI Command| B[Use netdatacli mark-stale-nodes-ephemeral]
92
- B --> C[Node Marked as Ephemeral]
93
- C --> D[Metrics Remain Available]
94
- C --> E[Active Alerts Cleared]
95
- C --> F{Node Reconnects?}
96
- F -->|Yes - no config| G[Reverts to Permanent]
97
- F -->|No| H[Remains Ephemeral]
98
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
99
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
100
- class A step
101
- class B step
102
- class C step
103
- class D step
104
- class E step
105
- class F alert
106
- class G alert
107
- class H alert
107
+ A[Start:<br>Offline Node Detected] -->|Use CLI Tool| B[Run Node Removal<br> Command]
108
+ B -->|Specify Target| C[Select Node to Remove]
109
+ C --> D[Node Removed from System]
110
+ D --> E[Node No Longer Queryable]
111
+ D --> F[Alerts for Node Cleared]
112
+
113
+ style A fill:#4caf50,stroke:#333,stroke-width:1px,color:white
114
+ style B fill:#4caf50,stroke:#333,stroke-width:1px,color:white
115
+ style C fill:#4caf50,stroke:#333,stroke-width:1px,color:white
116
+ style D fill:#f44336,stroke:#333,stroke-width:1px,color:white
117
+ style E fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
118
+ style F fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
119
```
120
110
-</details>
111
-
112
-### Removing Offline Nodes
113
-
121
To fully remove permanently offline nodes:
122
123
```bash
124
netdatacli remove-stale-node <node_id | machine_guid | hostname | ALL_NODES>
125
```
126
120
-:::note
127
+This is like the `mark-stale-nodes-ephemeral` subcommand, but it also removes the nodes so they are no longer available for querying.
128
122
-For detailed instructions on removing nodes from Netdata Cloud (including **offline** and **stale** nodes, bulk operations, and UI-based removal), see the [Remove Node Guide](https://github.com/netdata/netdata/edit/master/docs/learn/remove-node.md). This covers scenarios where UI removal is disabled due to parent-child configured relationships.
129
+## Cloud Integration
130
124
-:::
131
+In Netdata Cloud, ephemeral nodes remain visible but marked as 'stale' as long as at least one Agent reports having queryable metrics data for that node. Once all Agents report the node as offline, ephemeral nodes are automatically removed from the Cloud.
132
126
-<details>
127
-<summary><strong>Click to see visual representation of node removal flow</strong></summary><br/>
133
+From v2.3.0 onward, Netdata Cloud sends unreachable-node notifications **only for permanent nodes**, reducing unnecessary alerts.
134
+
135
+## Automatically Removing Ephemeral Nodes
136
137
```mermaid
138
flowchart TD
131
- A[Offline Node Detected] -->|Run CLI Tool| B[Execute remove-stale-node Command]
132
- B --> C[Node Removed from System]
133
- C --> D[Node No Longer Queryable]
134
- C --> E[Alerts for Node Cleared]
135
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
136
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
137
- class A step
138
- class B step
139
- class C step
140
- class D step
141
- class E step
139
+ A[Start:<br> Configure Auto-Removal] -->|Edit Configuration| B[Open netdata.conf<br> on Parent Nodes]
140
+ B -->|Add Configuration| C["Add: [db]\ncleanup<br> ephemeral hosts after = 1d"]
141
+ C -->|Restart Node| D[Restart Netdata]
142
+ D --> E[Ephemeral Node<br> Disconnects]
143
+ E -->|Wait Period| F{24 Hours Passed?}
144
+ F -->|Yes| G[Node<br> Automatically Removed]
145
+ F -->|No| H[Node Remains in System]
146
+ G -->|If All Parents Remove Node| I[Node Removed from Cloud]
147
+
148
+ style A fill:#4caf50,stroke:#333,stroke-width:1px,color:white
149
+ style B fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
150
+ style C fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
151
+ style D fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
152
+ style E fill:#ffeb3b,stroke:#333,stroke-width:1px,color:black
153
+ style F fill:#f9f9f9,stroke:#333,stroke-width:1px,color:black
154
+ style G fill:#f44336,stroke:#333,stroke-width:1px,color:white
155
+ style H fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333
156
+ style I fill:#f44336,stroke:#333,stroke-width:1px,color:white
157
```
158
144
-</details>
145
-
146
-## Automatically Removing Ephemeral Nodes
147
-
148
-To enable automatic cleanup of ephemeral nodes:
159
+By default, Netdata does not automatically remove disconnected ephemeral nodes. **To enable automatic cleanup**:
160
161
1. Open the `netdata.conf` file on Netdata Parent nodes.
162
2. Add the following configuration:
152
-
163
```ini
164
[db]
155
- cleanup ephemeral hosts after = 1d
165
+ cleanup ephemeral hosts after = 1d
166
```
167
+3. Restart the node.
168
158
-3. Restart the Netdata Agent.
169
+This setting removes ephemeral nodes from queries after 24 hours of disconnection. Once all parent nodes remove a node, Netdata Cloud automatically deletes it as well.
170
160
-This removes ephemeral nodes after 24 hours of disconnection. Once all Parents purge the node, it is automatically removed from Netdata Cloud.
171
162
-<details>
163
-<summary><strong>Click to see visual representation of auto-removal process</strong></summary><br/>
164
-
165
-```mermaid
166
-flowchart TD
167
- A[Configure Auto-Removal in netdata.conf] --> B[Restart Parent Nodes]
168
- B --> C[Ephemeral Node Disconnects]
169
- C --> D{Wait Period Elapsed?}
170
- D -->|Yes| E[Node Automatically Removed]
171
- D -->|No| F[Node Remains in System]
172
- E --> G{All Parents Removed Node?}
173
- G -->|Yes| H[Node Removed from Cloud]
174
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
175
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
176
- class A step
177
- class B step
178
- class C step
179
- class D step
180
- class E step
181
- class F step
182
- class G step
183
- class H step
184
-```
172
186
-</details>
docs/observability-centralization-points/README.md
+46
-166
@@ -1,26 +1,10 @@
1
-# Parents: Your Centralization Points
1
+# Observability Centralization Points
2
3
-***Parents*** are Netdata Agents that collect and store data from other Agents ("***Children***"). They act as centralization points for your observability data.
3
+## What Are Centralization Points?
4
5
-## How It Works
5
+Observability Centralization Points are specialized Netdata installations that you can configure to **receive, store, and process** observability data (metrics and logs) from multiple other systems in your infrastructure.
6
7
-1. **You designate some Agents as Parents** - Configure them to receive streaming data
8
-2. **Children stream their data to Parents** - They push metrics continuously
9
-3. **Parents store and process everything** - All metrics and logs from all Children
10
-4. **You access Parents for dashboards and alerts** - Centralized monitoring interface
11
-5. **Cloud queries Parents when configured** - Reduces load on production systems
12
-
13
-:::info
14
-
15
-Parents give you centralized collection with distributed architecture benefits.
16
-
17
-:::
18
-
19
-## What Parents Do
20
-
21
-Parents are specialized Netdata installations that you can configure to **receive, store, and process** observability data (metrics and logs) from multiple other systems in your infrastructure.
22
-
23
-These Parents give you several core functions:
7
+These centralization points give you several core functions:
8
9
* **Receiving and storing** metrics and logs from multiple systems
10
* **Processing and analyzing** your collected data
@@ -30,171 +14,67 @@ These Parents give you several core functions:
14
15
This **distributed yet centralized** approach gives you the benefits of both decentralized collection and centralized analysis.
16
33
-## Why Use Parents
17
+## Why Use Centralization Points?
18
19
| Use Case | Description | Benefits |
36
-|-------------------------------------------------|------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
20
+|-------------------------------------------------|------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
21
| **Ephemeral Systems** | Ideal for your Kubernetes nodes or temporary VMs that frequently go offline | You retain metrics and logs for analysis and troubleshooting even after node termination |
22
| **Limited Resources** | Offloads observability tasks from your systems with low disk space, CPU, RAM, or I/O bandwidth | Your production systems run efficiently without performance trade-offs |
23
| **Multi-Node Dashboards Without Netdata Cloud** | Aggregates data from all your nodes for centralized dashboards | You get Cloud-like functionality in environments that prefer or require on-premises solutions |
24
| **Restricted Netdata Cloud Access** | Acts as a bridge when your monitored systems can't connect to Netdata Cloud | You can still use Cloud features despite firewall restrictions or security policies |
25
42
-## How Multiple Parents Work
26
+## How Multiple Centralization Points Work
27
44
-<details>
45
-<summary><strong>Click to see Parent architecture options</strong></summary><br/>
28
+| Scenario | Operation | Advantages |
29
+|-----------------------------|--------------------------------------------------------------------------|--------------------------------------------------------------------------|
30
+| **With Netdata Cloud** | Queries all your centralization points in parallel for a unified view | You get a seamless experience regardless of your underlying architecture |
31
+| **Without Netdata Cloud** | Your centralization points consolidate data from connected systems | You have a local view of metrics and logs without external dependencies |
32
+| **High Availability Setup** | Your centralization points share data with each other, forming a cluster | You won't lose data if one centralization point fails |
33
34
```mermaid
48
-flowchart TB
49
- subgraph architectures["Parent Architecture Options"]
50
- direction TB
51
-
52
- subgraph single["Single Parent"]
53
- SP[SP]
54
- SC1[SC1]
55
- SC2[SC2]
56
- SC3[SC3]
57
- SP("**Parent**<br/>All data in one place")
58
- SC1("Child 1")
59
- SC2("Child 2")
60
- SC3("Child 3")
61
- SC1 --> SP
62
- SC2 --> SP
63
- SC3 --> SP
64
- end
65
-
66
- subgraph multiple["Multiple Parents"]
67
- MP1[MP1]
68
- MP2[MP2]
69
- MC1[MC1]
70
- MC2[MC2]
71
- MC3[MC3]
72
- MC4[MC4]
73
- MP1("**Parent 1**<br/>Region/Team A")
74
- MP2("**Parent 2**<br/>Region/Team B")
75
- MC1("Child 1")
76
- MC2("Child 2")
77
- MC3("Child 3")
78
- MC4("Child 4")
79
- MC1 --> MP1
80
- MC2 --> MP1
81
- MC3 --> MP2
82
- MC4 --> MP2
83
- end
84
-
85
- subgraph ha["High Availability"]
86
- HP1[HP1]
87
- HP2[HP2]
88
- HC1[HC1]
89
- HC2[HC2]
90
- HP1("**Parent 1**<br/>Active")
91
- HP2("**Parent 2**<br/>Active")
92
- HC1("Child 1")
93
- HC2("Child 2")
94
- HC1 --> HP1
95
- HC2 --> HP1
96
- HC1 -.-> HP2
97
- HC2 -.-> HP2
98
- HP1 <--> HP2
99
- end
100
- end
101
-
102
- classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
103
- classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
104
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
105
- classDef innerStyle fill: #f0f8ff, stroke: #87ceeb, stroke-width: 2px, color: #2c3e50, rx: 12, ry: 12
106
- class SP parent
107
- class MP1 parent
108
- class MP2 parent
109
- class HP1 parent
110
- class HP2 parent
111
- class SC1 child
112
- class SC2 child
113
- class SC3 child
114
- class MC1 child
115
- class MC2 child
116
- class MC3 child
117
- class MC4 child
118
- class HC1 child
119
- class HC2 child
120
- class architectures subgraphStyle
121
- class single innerStyle
122
- class multiple innerStyle
123
- class ha innerStyle
35
+graph TD
36
+ A[Centralization Points<br>Architecture] --> B[Single Centralization Point<br> Setup]
37
+ A --> C[Multiple Independent<br>Centralization Points]
38
+ A --> D[High Availability Cluster]
39
+
40
+ B --> B1[All systems stream<br>to one centralization point]
41
+ C --> C1[Systems divided<br>by region/service/team]
42
+ D --> D1[Centralization points<br>share data with each other]
43
+
44
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
45
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
46
+class A default;
47
+class B,C,D,B1,C1,D1 green;
48
```
49
126
-</details><br/>
127
-
128
-| Scenario | Operation | Advantages |
129
-|-----------------------------|------------------------------------------------------------|--------------------------------------------------------------------------|
130
-| **With Netdata Cloud** | Queries all your Parents in parallel for a unified view | You get a seamless experience regardless of your underlying architecture |
131
-| **Without Netdata Cloud** | Your Parents consolidate data from connected systems | You have a local view of metrics and logs without external dependencies |
132
-| **High Availability Setup** | Your Parents share data with each other, forming a cluster | You won't lose data if one Parent fails |
133
-
50
## Technical Implementation
51
136
-Parents consist of two major components you can deploy:
52
+Observability Centralization Points consist of two major components you can deploy:
53
54
1. **Metrics Centralization** - Uses Netdata's streaming and replication features to centralize your metrics data
55
2. **Logs Centralization** - Uses systemd-journald methodologies to centralize your log data
56
141
-You can configure your systems to connect to **multiple Parents** for redundancy. If a connection fails, they automatically switch to an available alternative.
142
-
143
-In a **high-availability setup**, your Parents can form a cluster by sharing data with each other, ensuring all points have a complete copy of all your metrics and logs.
57
+You can configure your systems to connect to **multiple centralization points** for redundancy. If a connection fails, they automatically switch to an available alternative.
58
145
-<details>
146
-<summary><strong>Click to see how high availability works</strong></summary><br/>
59
+In a **high-availability setup**, your centralization points can form a cluster by sharing data with each other, ensuring all points have a complete copy of all your metrics and logs.
60
61
```mermaid
149
-flowchart TB
150
- NC[NC]
151
- NC("**Netdata Cloud**<br/>Queries available Parents")
152
-
153
- subgraph infrastructure["Your Infrastructure"]
154
- direction TB
155
- P1[P1]
156
- P2[P2]
157
- C1[C1]
158
- C2[C2]
159
- C3[C3]
160
- C4[C4]
161
- P1("**Parent 1**<br/>Active")
162
- P2("**Parent 2**<br/>Active")
163
- C1("Child 1")
164
- C2("Child 2")
165
- C3("Child 3")
166
- C4("Child 4")
167
- C1 -->|primary| P1
168
- C2 -->|primary| P1
169
- C3 -->|primary| P2
170
- C4 -->|primary| P2
171
- C1 -.->|failover| P2
172
- C2 -.->|failover| P2
173
- C3 -.->|failover| P1
174
- C4 -.->|failover| P1
175
- P1 <-->|sync| P2
176
- end
177
-
178
- NC <--> P1
179
- NC <--> P2
180
- classDef cloud fill: #e8f4fd, stroke: #4a90e2, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
181
- classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
182
- classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
183
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
184
- class NC cloud
185
- class P1 parent
186
- class P2 parent
187
- class C1 child
188
- class C2 child
189
- class C3 child
190
- class C4 child
191
- class infrastructure subgraphStyle
62
+graph TD
63
+ CP1[Centralization Point 1] --- CP2[Centralization Point 2]
64
+
65
+ S1[System 1] --> CP1
66
+ S2[System 2] --> CP1
67
+ S3[System 3] --> CP2
68
+ S4[System 4] --> CP2
69
+
70
+ S1 -.-> CP2
71
+ S2 -.-> CP2
72
+ S3 -.-> CP1
73
+ S4 -.-> CP1
74
+
75
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
76
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
77
+classDef blue fill:#2196F3,stroke:#333,stroke-width:1px,color:white;
78
+class CP1,CP2 default;
79
+class S1,S2,S3,S4 blue;
80
```
193
-
194
-</details><br/>
195
-
196
-:::tip
197
-
198
-Check out our [Parent-Child Deployment Guide](https://github.com/netdata/netdata/edit/master/docs/deployment-guides/deployment-with-centralization-points.md) for step-by-step instructions.
199
-
200
-:::
docs/observability-centralization-points/best-practices.md
+63
-95
@@ -1,126 +1,94 @@
1
-# Parent Configuration Best Practices
1
+# Best Practices for Observability Centralization Points
2
3
-## Critical Factors to Consider
3
+## Critical factors to consider
4
5
-When setting up Parents, consider the following:
5
+When setting up Observability Centralization Points, consider the following:
6
7
| Factor | Description | Impact |
8
|---------------------------------------------|--------------------------------------|-----------------------------------------------------------------------------------------------|
9
-| **System Volume** | The number of monitored systems | Larger infrastructures may need multiple Parents to maintain performance |
9
+| **System Volume** | The number of monitored systems | Larger infrastructures may need multiple centralization points to maintain performance |
10
| **Data Transfer Costs** | Bandwidth usage between environments | Strategic placement reduces egress bandwidth costs in multi-cloud or hybrid environments |
11
-| **Usability Without Netdata Cloud** | Standalone operation considerations | Fewer Parents simplifies access and management |
11
+| **Usability Without Netdata Cloud** | Standalone operation considerations | Fewer centralization points simplifies access and management |
12
| **Optimized Deployment with Netdata Cloud** | Cloud integration benefits | Provides complete infrastructure view with optimized security, cost, and operational controls |
13
14
-<details>
15
-<summary><strong>Click to see deployment optimization factors</strong></summary><br/>
16
-
14
```mermaid
18
-flowchart TB
19
- A[A]
20
- B[B]
21
- C[C]
22
- D[D]
23
- B1[B1]
24
- C1[C1]
25
- D1[D1]
26
- A("**Optimized Deployment**<br/>with Netdata Cloud")
27
- B("Security")
28
- C("Cost")
29
- D("Operational Needs")
30
- B1("Internet access controls")
31
- C1("Bandwidth and<br/>resource allocation")
32
- D1("Regional, service, or<br/>team-based isolation")
33
- A --> B
34
- A --> C
35
- A --> D
36
- B --> B1
37
- C --> C1
38
- D --> D1
39
- classDef default fill: #f9f9f9, stroke: #333, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
40
- classDef factors fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
41
- class A default
42
- class B factors
43
- class C factors
44
- class D factors
45
- class B1 factors
46
- class C1 factors
47
- class D1 factors
15
+graph TD
16
+ A[Optimized Deployment<br>with Netdata Cloud] --> B[Security]
17
+ A --> C[Cost]
18
+ A --> D[Operational Needs]
19
+
20
+ B --> B1[Internet access controls]
21
+ C --> C1[Bandwidth and<br>resource allocation]
22
+ D --> D1[Regional, service, or<br>team-based isolation]
23
+
24
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
25
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
26
+class A default;
27
+class B,C,D,B1,C1,D1 green;
28
```
29
50
-</details><br/>
51
-
30
## Cost Optimization Strategies
31
54
-Netdata helps you keep observability efficient and cost-effective:
32
+Netdata is designed to keep observability efficient and cost-effective. To manage costs:
33
56
-| Strategy | Description | Benefit |
57
-|--------------------------------------------|----------------------------------------|-------------------------------------------------------------------------------------------------|
58
-| **Scale Out** | Use multiple smaller Parents | Improves efficiency and performance across distributed systems |
59
-| **Use Existing Resources** | Leverage spare capacity | Minimize additional hardware costs by using available resources |
60
-| **Centralized or Separate Logs & Metrics** | Choose storage approach based on needs | Optimize based on access patterns, retention policies, and compliance requirements |
61
-| **Flexible Configuration Management** | Customize each Parent | Control costs with unique retention and alert settings tailored for different teams or services |
62
-
63
-<details>
64
-<summary><strong>Click to see cost optimization strategies</strong></summary><br/>
34
+| Strategy | Description | Benefit |
35
+|--------------------------------------------|--------------------------------------------|-------------------------------------------------------------------------------------------------|
36
+| **Scale Out** | Use multiple smaller centralization points | Improves efficiency and performance across distributed systems |
37
+| **Use Existing Resources** | Leverage spare capacity | Minimize additional hardware costs by using available resources |
38
+| **Centralized or Separate Logs & Metrics** | Choose storage approach based on needs | Optimize based on access patterns, retention policies, and compliance requirements |
39
+| **Flexible Configuration Management** | Customize each centralization point | Control costs with unique retention and alert settings tailored for different teams or services |
40
41
```mermaid
67
-flowchart TB
68
- A[A]
69
- B[B]
70
- C[C]
71
- D[D]
72
- E[E]
73
- B1[B1]
74
- C1[C1]
75
- D1[D1]
76
- E1[E1]
77
- A("**Cost Optimization**<br/>Strategies")
78
- B("Scale Out")
79
- C("Use Existing<br/>Resources")
80
- D("Centralized or<br/>Separate Logs & Metrics")
81
- E("Flexible<br/>Configuration Management")
82
- B1("Multiple smaller<br/>Parents")
83
- C1("Leverage spare capacity")
84
- D1("Based on access needs,<br/>retention policies,<br/>and compliance")
85
- E1("Unique settings for<br/>different teams or services")
86
- A --> B
87
- A --> C
88
- A --> D
89
- A --> E
90
- B --> B1
91
- C --> C1
92
- D --> D1
93
- E --> E1
94
- classDef default fill: #f9f9f9, stroke: #333, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
95
- classDef strategies fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
96
- class A default
97
- class B strategies
98
- class C strategies
99
- class D strategies
100
- class E strategies
101
- class B1 strategies
102
- class C1 strategies
103
- class D1 strategies
104
- class E1 strategies
42
+graph TD
43
+ A[Cost Optimization<br>Strategies] --> B[Scale Out]
44
+ A --> C[Use Existing<br>Resources]
45
+ A --> D[Centralized or<br>Separate Logs & Metrics]
46
+ A --> E[Flexible<br>Configuration Management]
47
+
48
+ B --> B1[Multiple smaller<br>centralization points]
49
+ C --> C1[Leverage spare capacity]
50
+ D --> D1[Based on access needs,<br>retention policies,<br>and compliance]
51
+ E --> E1[Unique settings for<br>different teams or services]
52
+
53
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
54
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
55
+class A default;
56
+class B,C,D,E,B1,C1,D1,E1 green;
57
```
58
107
-</details><br/>
108
-
59
## Advantages of Netdata's Approach
60
61
Netdata provides several benefits over other observability solutions:
62
63
| Advantage | Description | Value |
64
|----------------------------------|--------------------------------------------|-----------------------------------------------------------------------|
115
-| **Scalability & Flexibility** | Multiple independent Parents | Customized observability by region, service, or team |
116
-| **Resilience & Reliability** | Built-in replication | Observability continues even if a Parent fails |
65
+| **Scalability & Flexibility** | Multiple independent centralization points | Customized observability by region, service, or team |
66
+| **Resilience & Reliability** | Built-in replication | Observability continues even if a centralization point fails |
67
| **Optimized Cost & Performance** | Distributed workloads | Prevents bottlenecks and improves resource efficiency |
68
| **Ease of Use** | Minimal setup and maintenance | Reduces complexity and operational overhead |
69
| **On-Prem Control** | Data remains within your infrastructure | Enhanced security and compliance, even when using Netdata Cloud |
70
| **Comprehensive Observability** | Segmented infrastructure with unified view | Deep visibility with tailored retention, alerts, and machine learning |
71
122
-:::tip
72
+```mermaid
73
+graph TD
74
+ A[Advantages of<br>Netdata's Approach] --> B[Scalability & Flexibility]
75
+ A --> C[Resilience & Reliability]
76
+ A --> D[Optimized Cost &<br>Performance]
77
+ A --> E[Ease of Use]
78
+ A --> F[On-Prem Control]
79
+ A --> G[Comprehensive<br>Observability]
80
+
81
+ B --> B1[Customized observability<br>by region, service, or team]
82
+ C --> C1[Observability continues<br> even if a centralization<br> point fails]
83
+ D --> D1[Prevents bottlenecks<br>and improves<br>resource efficiency]
84
+ E --> E1[Minimal setup and maintenance]
85
+ F --> F1[Data remains within<br>your infrastructure]
86
+ G --> G1[Unified view with<br>tailored segments]
87
+
88
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
89
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
90
+class A default;
91
+class B,C,D,E,F,G,B1,C1,D1,E1,F1,G1 green;
92
+```
93
94
Following these best practices helps you maintain a **cost-effective**, **high-performance** observability setup with Netdata.
125
-
126
-:::
src/streaming/README.md
+118
-167
@@ -1,16 +1,16 @@
1
-# Parent-Child Configuration Reference
1
+# Streaming and Replication Reference
2
3
## Introduction
4
5
This guide covers Netdata's advanced streaming and replication capabilities, which allow you to build centralized observability points across your infrastructure.
6
7
-**[Streaming and replication](https://learn.netdata.cloud/docs/developer-and-contributor-corner/glossary#r)** work together to send metrics data from one Netdata Agent (Child) to another Netdata Agent (Parent). Streaming sends metrics in real-time, while replication ensures historical data is copied, as well, maintaining complete data integrity even after connection interruptions.
7
+**[Streaming and replication](https://learn.netdata.cloud/docs/developer-and-contributor-corner/glossary#r)** work together to send metrics data from one Netdata Agent (child) to another Netdata Agent (parent). Streaming sends metrics in real-time, while replication ensures historical data is copied, as well, maintaining complete data integrity even after connection interruptions.
8
9
:::tip
10
11
If you're new to Netdata streaming or prefer a guided approach, [jump to our step-by-step guide](#step-by-step-setup-guide) at the end of this document. The guide will walk you through setting up a basic streaming configuration and then connecting to the comprehensive reference sections as needed.
12
13
-For a quick reference on setting up the Parent-Child relationship, see the [Configuration Examples](#complete-configuration-examples) or refer to our comprehensive [Parents: Your Centralization Points](https://learn.netdata.cloud/docs/deployment-guides/parents-your-centralization-points) documentation for more details.
13
+For a quick reference on setting up the parent-child relationship, see the [example configurations](#complete-configuration-examples) or refer to our comprehensive [Centralized Deployment Guide](https://learn.netdata.cloud/docs/deployment-guides/centralized) for more details.
14
15
:::
16
@@ -18,77 +18,56 @@ For a quick reference on setting up the Parent-Child relationship, see the [Conf
18
19
Before diving into configuration details, it's important to understand the key concepts behind Netdata's streaming architecture:
20
21
-<details>
22
-<summary><strong>Click to see how streaming and replication work</strong></summary><br/>
23
-
21
```mermaid
25
-flowchart TB
26
- subgraph infrastructure["Your Infrastructure"]
27
- direction TB
28
- C1[C1]
29
- C2[C2]
30
- P[P]
31
- C1("**Child 1**<br/>Collects metrics")
32
- C2("**Child 2**<br/>Collects metrics")
33
- P("**Parent**<br/>Stores all metrics")
34
- C1 -->|Streams real - time metrics| P
35
- C1 -.->|Replicates historical data| P
36
- C2 -->|Streams real - time metrics| P
37
- C2 -.->|Replicates historical data| P
38
- end
39
-
40
- U[U]
41
- U("**You**<br/>Access unified dashboard")
42
- P -->|Presents all data| U
43
- classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
44
- classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
45
- classDef user fill: #fff2e8, stroke: #f39c12, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
46
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
47
- class C1 child
48
- class C2 child
49
- class P parent
50
- class U user
51
- class infrastructure subgraphStyle
22
+graph TD
23
+ A[Child Node] -->|Streams real-time metrics| B[Parent Node]
24
+ A -->|Replicates historical data| B
25
+ C[Child Node] -->|Streams real-time metrics| B
26
+ C -->|Replicates historical data| B
27
+ B -->|Presents unified dashboard| D[Monitoring User]
28
+
29
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
30
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
31
+class A,C green;
32
+class B,D default;
33
```
34
54
-</details><br/>
55
-
35
### Parent-Child Relationship
36
37
- **Child nodes** (data sources) collect metrics directly from systems they monitor
59
-- **Parent nodes** (data centralization points) receive, store, and visualize metrics from multiple Child nodes
60
-- **A single Parent can receive data from many Children**, creating a centralized monitoring hub
61
-- **Child nodes maintain independence and continue collecting metrics** even if the connection to a Parent is lost
38
+- **Parent nodes** (data centralization points) receive, store, and visualize metrics from multiple child nodes
39
+- **A single parent can receive data from many children**, creating a centralized monitoring hub
40
+- **Child nodes maintain independence and continue collecting metrics** even if the connection to a parent is lost
41
42
### Data Flow
43
44
1. **Collection**: Child nodes collect metrics from their local systems
66
-2. **Streaming**: Child nodes send real-time metrics to Parent nodes
67
-3. **Replication**: When a connection is established or restored, Child nodes replicate historical data to ensure Parents have complete history
45
+2. **Streaming**: Child nodes send real-time metrics to parent nodes
46
+3. **Replication**: When a connection is established or restored, child nodes replicate historical data to ensure parents have complete history
47
4. **Storage**: Parent nodes store metrics based on their configured retention policies
69
-5. **Visualization**: Users access the Parent node's dashboard to view metrics from all connected Child nodes
48
+5. **Visualization**: Users access the parent node's dashboard to view metrics from all connected child nodes
49
50
### Benefits of This Architecture
51
52
- **Efficiency**: Distribute collection workload across multiple nodes while centralizing visualization
74
-- **Resilience**: Maintain data collection even during network disruptions or Parent node failures
75
-- **Scalability**: Add more Child nodes or additional Parent nodes as your infrastructure grows
53
+- **Resilience**: Maintain data collection even during network disruptions or parent node failures
54
+- **Scalability**: Add more child nodes or additional parent nodes as your infrastructure grows
55
- **Flexibility**: Configure retention, alerts, and dashboards according to your specific needs
56
57
## Quick Reference
58
80
-| Task | Configuration | Example |
81
-|------------------------------------------|-------------------------------------------|----------------------------------------------------------------|
82
-| Enable streaming on a Child | Set `enabled = yes` in `[stream]` section | `[stream]`<br/>`enabled = yes`<br/>`destination = 192.168.1.5` |
83
-| Configure a Parent to accept connections | Create an `[API_KEY]` section | `[API_KEY]`<br/>`enabled = yes`<br/>`allow from = *` |
84
-| Set up high availability | Configure multiple destinations on Child | `[stream]`<br/>`destination = parent1:19999 parent2:19999` |
85
-| Filter which metrics to send | Use `send charts matching` setting | `send charts matching = system.* !system.uptime` |
59
+| Task | Configuration | Example |
60
+|------------------------------------------|-------------------------------------------|------------------------------------------------------------------|
61
+| Enable streaming on a child | Set `enabled = yes` in `[stream]` section | `[stream]`<br />`enabled = yes`<br />`destination = 192.168.1.5` |
62
+| Configure a parent to accept connections | Create an `[API_KEY]` section | `[API_KEY]`<br />`enabled = yes`<br />`allow from = *` |
63
+| Set up high availability | Configure multiple destinations on child | `[stream]`<br />`destination = parent1:19999 parent2:19999` |
64
+| Filter which metrics to send | Use `send charts matching` setting | `send charts matching = system.* !system.uptime` |
65
66
## Configuration Overview
67
68
Netdata's streaming capabilities are configured through two key files:
69
91
-- **`stream.conf`** – Controls streaming behavior, including Parent and Child configurations.
70
+- **`stream.conf`** – Controls streaming behavior, including parent and child configurations.
71
- **`netdata.conf`** – Contains global settings that can impact streaming.
72
73
To edit these files, navigate to your Netdata configuration directory (typically `/etc/netdata`) and run:
@@ -105,9 +84,9 @@ sudo ./edit-config netdata.conf
84
85
The `stream.conf` file has three main sections:
86
108
-1. **`[stream]`** – With these settings, you can configure how Child nodes send metrics.
109
-2. **`[API_KEY]`** – Here you can define settings for authentication and access control between Parents and Children.
110
-3. **`[MACHINE_GUID]`** – This area lets you customize settings for specific Child nodes by their unique ID.
87
+1. **`[stream]`** – With these settings, you can configure how child nodes send metrics.
88
+2. **`[API_KEY]`** – Here you can define settings for authentication and access control between parents and children.
89
+3. **`[MACHINE_GUID]`** – This area lets you customize settings for specific child nodes by their unique ID.
90
91
### Identifying a Node's GUID
92
@@ -123,60 +102,32 @@ This file is generated automatically the first time Netdata starts and remains u
102
103
For a production-ready streaming setup, consider the following best practices:
104
126
-<details>
127
-<summary><strong>Click to see deployment best practices</strong></summary><br/>
128
-
105
```mermaid
130
-flowchart TB
131
- A[A]
132
- B[B]
133
- C[C]
134
- D[D]
135
- E[E]
136
- B1[B1]
137
- C1[C1]
138
- D1[D1]
139
- E1[E1]
140
- A("**Recommended Strategies**")
141
- B("Multiple Parent Nodes")
142
- C("Optimized Data Retention")
143
- D("Secure Communications")
144
- E("Performance Monitoring")
145
- B1("Improved redundancy<br/>and resilience")
146
- C1("Balance storage costs<br/>and data availability")
147
- D1("Enable encryption<br/>and authentication")
148
- E1("Regular log and<br/>metric reviews")
149
- A --> B
150
- A --> C
151
- A --> D
152
- A --> E
153
- B --> B1
154
- C --> C1
155
- D --> D1
156
- E --> E1
157
- classDef default fill: #f9f9f9, stroke: #333, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
158
- classDef strategies fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
159
- class A default
160
- class B strategies
161
- class C strategies
162
- class D strategies
163
- class E strategies
164
- class B1 strategies
165
- class C1 strategies
166
- class D1 strategies
167
- class E1 strategies
106
+graph TD
107
+ A[Recommended Strategies] --> B[Multiple Parent Nodes]
108
+ A --> C[Optimized Data Retention]
109
+ A --> D[Secure Communications]
110
+ A --> E[Performance Monitoring]
111
+
112
+ B --> B1[Improved redundancy<br>and resilience]
113
+ C --> C1[Balance storage costs<br>and data availability]
114
+ D --> D1[Enable encryption<br>and authentication]
115
+ E --> E1[Regular log and<br>metric reviews]
116
+
117
+classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px,color:#333;
118
+classDef green fill:#4caf50,stroke:#333,stroke-width:1px,color:black;
119
+class A default;
120
+class B,C,D,E,B1,C1,D1,E1 green;
121
```
122
170
-</details><br/>
171
-
123
:::tip
124
125
### Multiple Parent Nodes
126
176
-Setting up multiple Parent nodes creates redundancy in your monitoring infrastructure. **If one Parent fails, Child nodes can automatically switch to another available Parent.** This approach:
127
+Setting up multiple parent nodes creates redundancy in your monitoring infrastructure. **If one parent fails, child nodes can automatically switch to another available parent.** This approach:
128
129
- **Prevents single points of failure** in your monitoring system
179
-- **Allows for maintenance** on Parent nodes without monitoring interruptions
130
+- **Allows for maintenance** on parent nodes without monitoring interruptions
131
- Can be **geographically distributed to reduce latency** for global deployments
132
133
### Optimized Data Retention
@@ -199,7 +150,7 @@ Configure data retention settings based on your specific monitoring needs:
150
151
Regularly evaluate the **health of your** streaming **setup**:
152
202
-- **Monitor network traffic** between Child and Parent nodes
153
+- **Monitor network traffic** between child and parent nodes
154
- **Check for buffer overflows** or connection issues
155
- **Adjust settings** like buffer size based on actual performance
156
@@ -211,37 +162,37 @@ Regularly evaluate the **health of your** streaming **setup**:
162
163
### `[stream]` Section (Child Node Settings)
164
214
-With these settings, you can configure how your Child nodes send metrics to Parent nodes.
165
+With these settings, you can configure how your child nodes send metrics to parent nodes.
166
167
| Setting | Default | Description |
168
|-------------------------------------------------|---------------------------|---------------------------------------------------------------------|
169
| `enabled` | `no` | Enables streaming. Set to `yes` to allow this node to send metrics. |
219
-| [`destination`](#destination) | (empty) | Defines one or more Parent nodes to send data to. |
170
+| [`destination`](#destination) | (empty) | Defines one or more parent nodes to send data to. |
171
| `ssl skip certificate verification` | `yes` | Accepts self-signed or expired SSL certificates. |
172
| `CApath` | `/etc/ssl/certs/` | Directory for trusted SSL certificates. |
173
| `CAfile` | `/etc/ssl/certs/cert.pem` | File containing trusted certificates. |
223
-| `api key` | (empty) | API key used by the Child to authenticate with the Parent. |
174
+| `api key` | (empty) | API key used by the child to authenticate with the parent. |
175
| `timeout` | `1m` | Connection timeout duration. |
176
| `default port` | `19999` | Default port for streaming if not specified in `destination`. |
177
| [`send charts matching`](#send-charts-matching) | `*` | Filters which charts are streamed. |
178
| `buffer size bytes` | `10485760` | Buffer size (10MB by default). Increase for higher latencies. |
228
-| `reconnect delay` | `5s` | Time before retrying connection to the Parent. |
179
+| `reconnect delay` | `5s` | Time before retrying connection to the parent. |
180
| `initial clock resync iterations` | `60` | Syncs chart clocks during startup. |
230
-| `parent using h2o` | `no` | Set to `yes` if connecting to a Parent using the H2O web server. |
181
+| `parent using h2o` | `no` | Set to `yes` if connecting to a parent using the H2O web server. |
182
183
### `[API_KEY]` Section (Parent Node Authentication)
184
234
-Here you can define settings for authentication and access control between Parents and Children.
185
+Here you can define settings for authentication and access control between parents and children.
186
187
| Setting | Default | Description |
188
|------------------------------|------------|-------------------------------------------------------------|
189
| `enabled` | `no` | Enables or disables this API key. |
190
| `type` | `api` | Defines the section as an API key configuration. |
240
-| [`allow from`](#allow-from) | `*` | Specifies which Child nodes (IP addresses) can connect. |
241
-| `retention` | `1h` | How long to keep Child node metrics in RAM-based storage. |
191
+| [`allow from`](#allow-from) | `*` | Specifies which child nodes (IP addresses) can connect. |
192
+| `retention` | `1h` | How long to keep child node metrics in RAM-based storage. |
193
| [`db`](#db) | `dbengine` | Specifies the database type for this API key. |
194
| `health enabled` | `auto` | Controls alerts and notifications (`auto`, `yes`, or `no`). |
244
-| `postpone alerts on connect` | `1m` | Delay alerts for a period after the Child connects. |
195
+| `postpone alerts on connect` | `1m` | Delay alerts for a period after the child connects. |
196
| `health log retention` | `5d` | Duration (in seconds) to keep health log events. |
197
| `proxy enabled` | (empty) | Enables routing metrics through a proxy. |
198
| `proxy destination` | (empty) | IP and port of the proxy server. |
@@ -249,20 +200,20 @@ Here you can define settings for authentication and access control between Paren
200
| `send charts matching` | `*` | Defines which charts to stream. |
201
| `enable compression` | `yes` | Enables or disables data compression. |
202
| `enable replication` | `yes` | Enables or disables data replication. |
252
-| `replication period` | `1d` | Maximum time window replicated from each Child. |
203
+| `replication period` | `1d` | Maximum time window replicated from each child. |
204
| `replication step` | `10m` | Time interval for each replication step. |
254
-| `is ephemeral node` | `no` | Marks the Child as ephemeral (removes it after inactivity). |
205
+| `is ephemeral node` | `no` | Marks the child as ephemeral (removes it after inactivity). |
206
207
### `[MACHINE_GUID]` Section (Per-Node Customization)
208
258
-This area lets you customize settings for specific Child nodes by their unique ID.
209
+This area lets you customize settings for specific child nodes by their unique ID.
210
211
| Setting | Default | Description |
212
|------------------------------|------------|----------------------------------------------------------|
213
| `enabled` | `no` | Enables or disables this specific node's configuration. |
214
| `type` | `machine` | Defines the section as a machine-specific configuration. |
215
| [`allow from`](#allow-from) | `*` | Lists IP addresses allowed to stream metrics. |
265
-| `retention` | `3600` | Retention period for Child metrics in RAM-based storage. |
216
+| `retention` | `3600` | Retention period for child metrics in RAM-based storage. |
217
| [`db`](#db) | `dbengine` | Database type for this node. |
218
| `health enabled` | `auto` | Controls alerts (`auto`, `yes`, `no`). |
219
| `postpone alerts on connect` | `1m` | Delay alerts for a period after connection. |
@@ -281,10 +232,10 @@ This area lets you customize settings for specific Child nodes by their unique I
232
233
#### `destination`
234
284
-Defines Parent nodes for streaming using the format:
235
+Defines parent nodes for streaming using the format:
236
`[PROTOCOL:]HOST[%INTERFACE][:PORT][:SSL]`
237
287
-- **PROTOCOL**: `tcp`, `udp`, or `unix` (only `tcp` and `unix` are supported for Parents).
238
+- **PROTOCOL**: `tcp`, `udp`, or `unix` (only `tcp` and `unix` are supported for parents).
239
- **HOST**: IPv4, IPv6 (in brackets `[ ]`), hostname, or Unix domain socket path.
240
- **INTERFACE** (IPv6 only): Network interface to use.
241
- **PORT**: Port number or service name.
@@ -294,7 +245,7 @@ Example (TCP connection with SSL to `203.0.113.0` on port `20000`):
245
246
```ini
247
[stream]
297
- # Send metrics securely to the Parent at 203.0.113.0:20000
248
+ # Send metrics securely to the parent at 203.0.113.0:20000
249
destination = tcp:203.0.113.0:20000:SSL
250
```
251
@@ -321,7 +272,7 @@ Controls which charts are streamed.
272
273
#### `allow from`
274
324
-Defines which Child nodes (by IP) can connect.
275
+Defines which child nodes (by IP) can connect.
276
277
- Allow a single IP:
278
@@ -404,32 +355,32 @@ Manage database settings for data storage and retention.
355
[stream]
356
# Enable streaming on this node
357
enabled = yes
407
- # Connect to Parent at 192.168.1.5 port 19999
358
+ # Connect to parent at 192.168.1.5 port 19999
359
destination = 192.168.1.5
409
- # Use the same API key defined on the Parent
360
+ # Use the same API key defined on the parent
361
api key = 11111111-2222-3333-4444-555555555555
362
```
363
364
### High Availability Setup with Multiple Parents
365
415
-**Parent nodes configuration (stream.conf on both Parents):**
366
+**Parent nodes configuration (stream.conf on both parents):**
367
368
```ini
418
-# Configuration for accepting metrics from Children
369
+# Configuration for accepting metrics from children
370
[11111111-2222-3333-4444-555555555555]
371
enabled = yes
372
allow from = *
373
db = dbengine
374
424
-# Configuration for accepting metrics from other Parents
375
+# Configuration for accepting metrics from other parents
376
[22222222-3333-4444-5555-666666666666]
377
enabled = yes
427
- # Only allow the other Parent's IP
378
+ # Only allow the other parent's IP
379
allow from = 192.168.1.5 192.168.1.6
380
db = dbengine
381
```
382
432
-**First Parent node's configuration for streaming to the second Parent:**
383
+**First parent node's configuration for streaming to the second parent:**
384
385
```ini
386
[stream]
@@ -438,7 +389,7 @@ Manage database settings for data storage and retention.
389
api key = 22222222-3333-4444-5555-666666666666
390
```
391
441
-**Second Parent node's configuration for streaming to the first Parent:**
392
+**Second parent node's configuration for streaming to the first parent:**
393
394
```ini
395
[stream]
@@ -452,7 +403,7 @@ Manage database settings for data storage and retention.
403
```ini
404
[stream]
405
enabled = yes
455
- # List both Parents for failover
406
+ # List both parents for failover
407
destination = 192.168.1.5 192.168.1.6
408
api key = 11111111-2222-3333-4444-555555555555
409
```
@@ -461,21 +412,21 @@ Manage database settings for data storage and retention.
412
413
If the streaming configuration is working correctly, you'll see logs similar to the following.
414
464
-On the Parent side:
415
+On the parent side:
416
417
```
418
2017-03-09 09:38:52: netdata: INFO : STREAM [receive from [10.11.12.86]:38564]: new client connection.
419
2017-03-09 09:38:52: netdata: INFO : STREAM xxx [10.11.12.86]:38564: receive thread created (task id 27721)
420
```
421
471
-On the Child side:
422
+On the child side:
423
424
```
425
2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: connecting...
426
2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: established communication - sending metrics...
427
```
428
478
-Both Parent and Child nodes log information in `/var/log/netdata/error.log`.
429
+Both parent and child nodes log information in `/var/log/netdata/error.log`.
430
431
## Troubleshooting
432
@@ -509,7 +460,7 @@ Slow network connections or high-latency links can cause the streaming buffer to
460
- Increase buffer size in `stream.conf`: `buffer size bytes = 20971520` (20MB)
461
- Check network bandwidth and latency between nodes
462
- Consider reducing the collection frequency on high-volume metrics
512
-- If possible, place Parent nodes closer (network-wise) to Child nodes
463
+- If possible, place parent nodes closer (network-wise) to child nodes
464
465
</details>
466
@@ -519,7 +470,7 @@ Slow network connections or high-latency links can cause the streaming buffer to
470
471
**Symptoms:**
472
522
-- Child can't establish a connection to a Parent
473
+- Child can't establish a connection to a parent
474
- Repeated reconnection attempts
475
476
**Child logs:**
@@ -529,13 +480,13 @@ ERROR : STREAM_SENDER[HOSTNAME] : Failed to connect to 'PARENT IP', port 'PARENT
480
```
481
482
**What's happening:**
532
-This error indicates network connectivity problems between the Child and Parent nodes. It could be due to firewall rules, incorrect IP addresses, or the Parent node not running.
483
+This error indicates network connectivity problems between the child and parent nodes. It could be due to firewall rules, incorrect IP addresses, or the parent node not running.
484
485
**Solutions:**
486
487
- Verify firewalls allow traffic on port 19999 (or your custom port)
537
-- Check that the Parent node is running and listening on the correct interface
538
-- Verify that the IP address/hostname is correct in Child's configuration
488
+- Check that the parent node is running and listening on the correct interface
489
+- Verify that the IP address/hostname is correct in child's configuration
490
- Test basic connectivity with tools like `ping` or `telnet`
491
- Check network routing between the nodes
492
@@ -557,12 +508,12 @@ STREAM [receive from [child HOSTNAME]:child IP]: `API key 'VALUE' is not allowed
508
```
509
510
**What's happening:**
560
-The Parent node is rejecting the connection because the API key doesn't match or the Child's IP address is not allowed by the `allow from` setting.
511
+The parent node is rejecting the connection because the API key doesn't match or the child's IP address is not allowed by the `allow from` setting.
512
513
**Solutions:**
514
564
-- Verify if the API key matches exactly between Parent and Child
565
-- Check that the `allow from` setting permits the Child's IP address
515
+- Verify if the API key matches exactly between parent and child
516
+- Check that the `allow from` setting permits the child's IP address
517
- Ensure GUID formats are valid
518
- Check for whitespace or invisible characters in the API key
519
- Remember that API keys are case-sensitive
@@ -584,7 +535,7 @@ ERROR : STREAM_SENDER[CHILD HOSTNAME] : STREAM child HOSTNAME [send to PARENT HO
535
```
536
537
**What's happening:**
587
-The Child node is connecting to the destination, but the server is not responding with the expected Netdata streaming protocol. This commonly occurs when there's a mismatch in SSL/TLS settings or when the destination is not a Netdata server.
538
+The child node is connecting to the destination, but the server is not responding with the expected Netdata streaming protocol. This commonly occurs when there's a mismatch in SSL/TLS settings or when the destination is not a Netdata server.
539
540
**Solutions:**
541
@@ -601,15 +552,15 @@ The Child node is connecting to the destination, but the server is not respondin
552
553
**Symptoms:**
554
604
-- Data inconsistencies between Parent and Child
555
+- Data inconsistencies between parent and child
556
- Gaps in metrics collection
557
558
**What's happening:**
608
-When the database settings between Parent and Child nodes don't match, it can cause inconsistencies in how data is stored and displayed. The most common cause is different memory modes or retention settings.
559
+When the database settings between parent and child nodes don't match, it can cause inconsistencies in how data is stored and displayed. The most common cause is different memory modes or retention settings.
560
561
**Solutions:**
562
612
-- Check for mismatch in the `[db].db` settings between the Parent and Child
563
+- Check for mismatch in the `[db].db` settings between the parent and child
564
- Ensure database retention settings are compatible
565
- Verify replication is enabled and properly configured
566
- Make sure both nodes are using the same (or compatible) database engine
@@ -620,17 +571,17 @@ When the database settings between Parent and Child nodes don't match, it can ca
571
## FAQ
572
573
<details>
623
-<summary><strong>Can I stream to multiple Parents simultaneously?</strong></summary>
574
+<summary><strong>Can I stream to multiple parents simultaneously?</strong></summary>
575
<br/>
576
626
-No, you can't stream to multiple Parents at the same time. However, you can configure multiple destinations for failover. Your Child node will connect to the first available Parent in the list.
577
+No, you can't stream to multiple parents at the same time. However, you can configure multiple destinations for failover. Your child node will connect to the first available parent in the list.
578
</details>
579
580
<details>
581
<summary><strong>How does replication work with interrupted connections?</strong></summary>
582
<br/>
583
633
-When you re-establish a connection, your Child node will replicate historical data based on the `replication period` setting. This ensures your Parent has a complete history even after interruptions.
584
+When you re-establish a connection, your child node will replicate historical data based on the `replication period` setting. This ensures your parent has a complete history even after interruptions.
585
</details>
586
587
<details>
@@ -641,7 +592,7 @@ Your streaming setup will be very efficient, especially with compression enabled
592
</details>
593
594
<details>
644
-<summary><strong>Can I filter which metrics are sent to the Parent?</strong></summary>
595
+<summary><strong>Can I filter which metrics are sent to the parent?</strong></summary>
596
<br/>
597
598
Yes, you can use the `send charts matching` setting to include or exclude specific metrics from streaming. This works with wildcard patterns, giving you precise control over what metrics are transferred.
@@ -655,10 +606,10 @@ You can enable SSL in the destination setting by adding `:SSL` at the end. Confi
606
</details>
607
608
<details>
658
-<summary><strong>Do I need to configure streaming on every Child node?</strong></summary>
609
+<summary><strong>Do I need to configure streaming on every child node?</strong></summary>
610
<br/>
611
661
-Yes, you need to configure each Child node with its own streaming configuration. However, you can use configuration management tools to deploy a standard configuration across your infrastructure, making this process more efficient.
612
+Yes, you need to configure each child node with its own streaming configuration. However, you can use configuration management tools to deploy a standard configuration across your infrastructure, making this process more efficient.
613
</details>
614
615
## Step-by-Step Setup Guide
@@ -671,7 +622,7 @@ This guide will walk you through setting up Netdata streaming between nodes. **B
622
623
Before configuring streaming, ensure you have:
624
674
-1. At least two Netdata instances are installed (one to act as Parent, one as Child)
625
+1. At least two Netdata instances are installed (one to act as parent, one as child)
626
2. Network connectivity between the instances
627
3. Administrative access to edit configuration files on both systems
628
@@ -681,9 +632,9 @@ Before configuring streaming, ensure you have:
632
<summary><strong>Step 2: Generate an API Key</strong></summary>
633
<br/>
634
684
-The API key is used to authenticate the connection between Parent and Child nodes.
635
+The API key is used to authenticate the connection between parent and child nodes.
636
686
-1. On the Parent node, generate a UUID to use as your API key:
637
+1. On the parent node, generate a UUID to use as your API key:
638
639
```bash
640
uuidgen
@@ -703,7 +654,7 @@ The API key is used to authenticate the connection between Parent and Child node
654
<summary><strong>Step 3: Configure the Parent Node</strong></summary>
655
<br/>
656
706
-The Parent node receives and stores metrics from Child nodes.
657
+The parent node receives and stores metrics from child nodes.
658
659
1. Open the stream configuration file for editing:
660
@@ -731,7 +682,7 @@ The Parent node receives and stores metrics from Child nodes.
682
:::tip
683
684
**Deployment Strategy**
734
-For critical environments, consider setting up at least two Parent nodes for redundancy. Each Parent should have enough disk space for your required retention period.
685
+For critical environments, consider setting up at least two parent nodes for redundancy. Each parent should have enough disk space for your required retention period.
686
687
:::
688
@@ -741,16 +692,16 @@ For critical environments, consider setting up at least two Parent nodes for red
692
<summary><strong>Step 4: Configure the Child Node</strong></summary>
693
<br/>
694
744
-The Child node streams its metrics to the Parent node.
695
+The child node streams its metrics to the parent node.
696
746
-1. Open the stream configuration file on the Child node:
697
+1. Open the stream configuration file on the child node:
698
699
```bash
700
cd /etc/netdata
701
sudo ./edit-config stream.conf
702
```
703
753
-2. Find the `[stream]` section and update it (replace PARENT_IP with your Parent's actual IP address):
704
+2. Find the `[stream]` section and update it (replace PARENT_IP with your parent's actual IP address):
705
706
```ini
707
[stream]
@@ -761,7 +712,7 @@ The Child node streams its metrics to the Parent node.
712
713
3. Save and close the file
714
764
-4. Restart Netdata on the Child node:
715
+4. Restart Netdata on the child node:
716
717
```bash
718
sudo systemctl restart netdata
@@ -783,7 +734,7 @@ For production environments, enable SSL by adding `:SSL` to your destination. Th
734
735
Check that streaming is working properly between your nodes.
736
786
-1. Check the Netdata logs on the Parent node:
737
+1. Check the Netdata logs on the parent node:
738
739
```bash
740
tail -f /var/log/netdata/error.log | grep STREAM
@@ -796,14 +747,14 @@ Check that streaming is working properly between your nodes.
747
STREAM xxx [CHILD_IP]: receive thread created (task id xxxxx)
748
```
749
799
-3. On the Child node, you should see:
750
+3. On the child node, you should see:
751
752
```
753
STREAM xxx [send to PARENT_IP:19999]: connecting...
754
STREAM xxx [send to PARENT_IP:19999]: established communication - sending metrics...
755
```
756
806
-4. Open the Netdata dashboard on the Parent node (http://PARENT_IP:19999) and look for the Child node's hostname in the menu
757
+4. Open the Netdata dashboard on the parent node (http://PARENT_IP:19999) and look for the child node's hostname in the menu
758
759
:::tip
760
@@ -822,7 +773,7 @@ Now that you have a working basic setup, you can customize it based on your depl
773
774
### To Filter Which Metrics Are Streamed (Optimize Performance)
775
825
-Add the following to the Child's `[stream]` section:
776
+Add the following to the child's `[stream]` section:
777
778
```ini
779
[stream]
@@ -832,7 +783,7 @@ Add the following to the Child's `[stream]` section:
783
784
### To Enable SSL Encryption (Security Enhancement)
785
835
-1. On the Child node, update the destination to include SSL:
786
+1. On the child node, update the destination to include SSL:
787
788
```ini
789
[stream]
@@ -848,23 +799,23 @@ Add the following to the Child's `[stream]` section:
799
800
### To Set Up Multiple Parents for High Availability (Redundancy Strategy)
801
851
-1. Configure multiple destinations on the Child:
802
+1. Configure multiple destinations on the child:
803
804
```ini
805
[stream]
806
destination = PARENT1_IP:19999 PARENT2_IP:19999
807
```
808
858
-2. The Child will connect to the first available Parent and automatically switch if that connection fails
809
+2. The child will connect to the first available parent and automatically switch if that connection fails
810
811
### Optimizing Data Retention (Storage Strategy)
812
862
-On the Parent node, you can [configure retention settings](/src/database/CONFIGURATION.md#retention-settings) to control how long metrics are stored.
813
+On the parent node, you can [configure retention settings](/src/database/CONFIGURATION.md#retention-settings) to control how long metrics are stored.
814
815
:::tip
816
817
**Advanced**
867
-For large-scale deployments, consider setting up Parent-to-Parent streaming to create a hierarchical architecture that balances local responsiveness with centralized monitoring.
818
+For large-scale deployments, consider setting up parent-to-parent streaming to create a hierarchical architecture that balances local responsiveness with centralized monitoring.
819
820
:::
821
@@ -881,7 +832,7 @@ Verify that:
832
- Firewalls allow traffic on port 19999
833
- Both Netdata instances are running
834
- The API key matches exactly on both systems
884
-- The Parent IP address is correct
835
+- The parent IP address is correct
836
837
</details>
838
@@ -892,7 +843,7 @@ Verify that:
843
Verify that:
844
845
- The connection is established (check logs)
895
-- The Child node hasn't been excluded with `allow from` settings
846
+- The child node hasn't been excluded with `allow from` settings
847
- The metrics aren't being filtered out with `send charts matching`
848
849
</details>
@@ -903,7 +854,7 @@ Verify that:
854
855
If you're using SSL encryption:
856
906
-- Make sure `:SSL` is added to the destination on the Child node
857
+- Make sure both `:SSL` is added to the destination on the child node
858
- Set `ssl skip certificate verification = yes` if using self-signed certificates
859
- Check that certificate paths are correct if using custom certificates
860