deployment-with-centralization-points.md · cryptotaxi247/netdata

Code Commits Issues Pulls Sessions CI/CD

master

md 211 lines 8.65 KB

Rendered Raw

1	# Parents: Your Centralization Points
2
3	*Parents* are Netdata Agents that collect and store data from other Agents ("*Children*"). They act as centralization points for your observability data.
4
5	## How It Works
6
7	1. You designate some Agents as Parents - Configure them to receive streaming data
8	2. Children stream their data to Parents - They push metrics continuously
9	3. Parents store and process everything - All metrics and logs from all Children
10	4. You access Parents for dashboards and alerts - Centralized monitoring interface
11	5. Cloud queries Parents when configured - Reduces load on production systems
12
13	:::info
14
15	Parents give you centralized collection with distributed architecture benefits.
16
17	:::
18
19	## What Parents Do
20
21	Parents are specialized Netdata installations that you can configure to receive, store, and process observability data (metrics and logs) from multiple other systems in your infrastructure.
22
23	These Parents give you several core functions:
24
25	* Receiving and storing metrics and logs from multiple systems
26	* Processing and analyzing your collected data
27	* Running health checks and alerts
28	* Providing unified dashboards across all your systems
29	* Replicating data for your historical analysis
30
31	:::info
32
33	This distributed yet centralized approach gives you the benefits of both decentralized collection and centralized analysis.
34
35	:::
36
37	## Why Use Parents
38
39	\| Use Case \| Description \| Benefits \|
40	\|-------------------------------------------------\|------------------------------------------------------------------------------------------------\|-----------------------------------------------------------------------------------------------\|
41	\| Ephemeral Systems \| Ideal for your Kubernetes nodes or temporary VMs that frequently go offline \| You retain metrics and logs for analysis and troubleshooting even after node termination \|
42	\| Limited Resources \| Offloads observability tasks from your systems with low disk space, CPU, RAM, or I/O bandwidth \| Your production systems run efficiently without performance trade-offs \|
43	\| Multi-Node Dashboards Without Netdata Cloud \| Aggregates data from all your nodes for centralized dashboards \| You get Cloud-like functionality in environments that prefer or require on-premises solutions \|
44	\| Restricted Netdata Cloud Access \| Acts as a bridge when your monitored systems can't connect to Netdata Cloud \| You can still use Cloud features despite firewall restrictions or security policies \|
45
46	:::tip
47
48	If you want the full Netdata Cloud experience within your own infrastructure, consider [Netdata Cloud Enterprise On-Prem](/docs/netdata-cloud/versions.md) — a self-hosted version that deploys the complete Cloud control plane in your environment. For the complete On-Prem documentation, see [Netdata Cloud On-Prem](https://github.com/netdata/netdata-cloud-onprem/blob/master/docs/learn.netdata.cloud/README.md).
49
50	:::
51
52	## How Multiple Parents Work
53
54	<details>
55	<summary><strong>Click to see Parent architecture options</strong></summary><br/>
56
57	```mermaid
58	flowchart TB
59	subgraph architectures["Parent Architecture Options"]
60	direction TB
61
62	subgraph single["Single Parent"]
63	SP[SP]
64	SC1[SC1]
65	SC2[SC2]
66	SC3[SC3]
67	SP("Parent<br/>All data in one place")
68	SC1("Child 1")
69	SC2("Child 2")
70	SC3("Child 3")
71	SC1 --> SP
72	SC2 --> SP
73	SC3 --> SP
74	end
75
76	subgraph multiple["Multiple Parents"]
77	MP1[MP1]
78	MP2[MP2]
79	MC1[MC1]
80	MC2[MC2]
81	MC3[MC3]
82	MC4[MC4]
83	MP1("Parent 1<br/>Region/Team A")
84	MP2("Parent 2<br/>Region/Team B")
85	MC1("Child 1")
86	MC2("Child 2")
87	MC3("Child 3")
88	MC4("Child 4")
89	MC1 --> MP1
90	MC2 --> MP1
91	MC3 --> MP2
92	MC4 --> MP2
93	end
94
95	subgraph ha["High Availability"]
96	HP1[HP1]
97	HP2[HP2]
98	HC1[HC1]
99	HC2[HC2]
100	HP1("Parent 1<br/>Active")
101	HP2("Parent 2<br/>Active")
102	HC1("Child 1")
103	HC2("Child 2")
104	HC1 --> HP1
105	HC2 --> HP1
106	HC1 -.-> HP2
107	HC2 -.-> HP2
108	HP1 <--> HP2
109	end
110	end
111
112	classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
113	classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
114	classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
115	classDef innerStyle fill: #f0f8ff, stroke: #87ceeb, stroke-width: 2px, color: #2c3e50, rx: 12, ry: 12
116	class SP parent
117	class MP1 parent
118	class MP2 parent
119	class HP1 parent
120	class HP2 parent
121	class SC1 child
122	class SC2 child
123	class SC3 child
124	class MC1 child
125	class MC2 child
126	class MC3 child
127	class MC4 child
128	class HC1 child
129	class HC2 child
130	class architectures subgraphStyle
131	class single innerStyle
132	class multiple innerStyle
133	class ha innerStyle
134	```
135
136	</details><br/>
137
138	\| Scenario \| Operation \| Advantages \|
139	\|-----------------------------\|------------------------------------------------------------\|--------------------------------------------------------------------------\|
140	\| With Netdata Cloud \| Queries all your Parents in parallel for a unified view \| You get a seamless experience regardless of your underlying architecture \|
141	\| Without Netdata Cloud \| Your Parents consolidate data from connected systems \| You have a local view of metrics and logs without external dependencies \|
142	\| High Availability Setup \| Your Parents share data with each other, forming a cluster \| You won't lose data if one Parent fails \|
143
144	## Technical Implementation
145
146	Parents consist of two major components you can deploy:
147
148	1. [Metrics Centralization](/docs/observability-centralization-points/metrics-centralization-points/README.md) - Uses Netdata's streaming and replication features to centralize your metrics data
149
150	2. [Logs Centralization](/docs/observability-centralization-points/logs-centralization-points-with-systemd-journald/README.md) - Uses systemd-journald methodologies to centralize your log data
151
152	You can configure your systems to connect to multiple Parents for redundancy. If a connection fails, they automatically switch to an available alternative.
153
154	In a high-availability setup, your Parents can form a cluster by sharing data with each other, ensuring all points have a complete copy of all your metrics and logs.
155
156	<details>
157	<summary><strong>Click to see how high availability works</strong></summary><br/>
158
159	```mermaid
160	flowchart TB
161	NC[NC]
162	NC("Netdata Cloud<br/>Queries available Parents")
163
164	subgraph infrastructure["Your Infrastructure"]
165	direction TB
166	P1[P1]
167	P2[P2]
168	C1[C1]
169	C2[C2]
170	C3[C3]
171	C4[C4]
172	P1("Parent 1<br/>Active")
173	P2("Parent 2<br/>Active")
174	C1("Child 1")
175	C2("Child 2")
176	C3("Child 3")
177	C4("Child 4")
178	C1 -->\|primary\| P1
179	C2 -->\|primary\| P1
180	C3 -->\|primary\| P2
181	C4 -->\|primary\| P2
182	C1 -.->\|failover\| P2
183	C2 -.->\|failover\| P2
184	C3 -.->\|failover\| P1
185	C4 -.->\|failover\| P1
186	P1 <-->\|sync\| P2
187	end
188
189	NC <--> P1
190	NC <--> P2
191	classDef cloud fill: #e8f4fd, stroke: #4a90e2, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
192	classDef parent fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
193	classDef child fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
194	classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
195	class NC cloud
196	class P1 parent
197	class P2 parent
198	class C1 child
199	class C2 child
200	class C3 child
201	class C4 child
202	class infrastructure subgraphStyle
203	```
204
205	</details><br/>
206
207	:::tip
208
209	Check out our [Parent-Child Deployment Guide](/docs/deployment-guides/deployment-with-centralization-points.md) for step-by-step instructions.
210
211	:::