| 1 | # Parent Configuration Best Practices |
| 2 | |
| 3 | A Parent node is a Netdata Agent configured to receive metrics from multiple Child nodes. It acts as the central long-term storage layer, providing a unified view, longer retention, and high availability when used with replication. |
| 4 | |
| 5 | ## Critical Factors to Consider |
| 6 | |
| 7 | When setting up Parents, consider the following: |
| 8 | |
| 9 | | Factor | Description | Impact | |
| 10 | |---------------------------------------------|--------------------------------------|--------------------------------------------------------------------------------------------------| |
| 11 | | **System Volume** | The number of monitored systems | Larger infrastructures may need multiple Parents to maintain performance | |
| 12 | | **Data Transfer Costs** | Bandwidth usage between environments | Strategic placement reduces egress bandwidth costs in multi-cloud or hybrid environments | |
| 13 | | **Usability Without Netdata Cloud** | Standalone operation considerations | Fewer Parents simplifies access and management | |
| 14 | | **Optimized Deployment with Netdata Cloud** | Cloud integration benefits | Provides complete infrastructure view with optimized security, cost, and operational controls | |
| 15 | | **Data Retention & Metric Volume** | Disk planning for Parent nodes | Estimate disk needs based on total metrics streamed from children and configured retention tiers | |
| 16 | |
| 17 | ## Deployment Optimization Factors |
| 18 | |
| 19 | ```mermaid |
| 20 | flowchart TB |
| 21 | A("Optimized Deployment<br/>with Netdata Cloud") |
| 22 | B("Security") |
| 23 | C("Cost") |
| 24 | D("Operational Needs") |
| 25 | B1("Internet access controls") |
| 26 | C1("Bandwidth and<br/>resource allocation") |
| 27 | D1("Regional, service, or<br/>team-based isolation") |
| 28 | A --> B |
| 29 | A --> C |
| 30 | A --> D |
| 31 | B --> B1 |
| 32 | C --> C1 |
| 33 | D --> D1 |
| 34 | classDef alert fill: #ffeb3b, stroke: #000000, stroke-width: 3px, color: #000000 |
| 35 | classDef neutral fill: #f9f9f9, stroke: #000000, stroke-width: 3px, color: #000000 |
| 36 | classDef complete fill: #4caf50, stroke: #000000, stroke-width: 3px, color: #000000 |
| 37 | class A neutral |
| 38 | class B complete |
| 39 | class C complete |
| 40 | class D complete |
| 41 | class B1 complete |
| 42 | class C1 complete |
| 43 | class D1 complete |
| 44 | ``` |
| 45 | |
| 46 | ## Critical Retention Configuration |
| 47 | |
| 48 | :::warning |
| 49 | |
| 50 | **Default retention settings will not work for production.** Netdata defaults to 1GB space limit per tier with combined time/space retention. You'll hit the 1GB limit within hours or days, causing data loss much sooner than your configured time limits. |
| 51 | |
| 52 | **You must configure retention properly before deployment.** |
| 53 | |
| 54 | ::: |
| 55 | |
| 56 | ### Choosing Your Retention Strategy |
| 57 | |
| 58 | Netdata supports three retention strategies. Choose the one that best fits your environment: |
| 59 | |
| 60 | 1. **Time-based retention** (recommended for predictable retention periods): |
| 61 | |
| 62 | Guarantees data is kept for a fixed time, regardless of disk usage (assuming you have enough disk space) |
| 63 | |
| 64 | ```ini |
| 65 | [db] |
| 66 | dbengine tier 0 retention time = 30d |
| 67 | dbengine tier 0 retention size = 0 |
| 68 | dbengine tier 1 retention time = 6mo |
| 69 | dbengine tier 1 retention size = 0 |
| 70 | dbengine tier 2 retention time = 5y |
| 71 | dbengine tier 2 retention size = 0 |
| 72 | ``` |
| 73 | |
| 74 | 2. **Space-based retention** (recommended for predictable disk usage): |
| 75 | |
| 76 | Targets keeping storage usage within defined limits, at the cost of variable retention duration. |
| 77 | |
| 78 | ```ini |
| 79 | [db] |
| 80 | dbengine tier 0 retention size = 500GB |
| 81 | dbengine tier 0 retention time = 0 |
| 82 | dbengine tier 1 retention size = 200GB |
| 83 | dbengine tier 1 retention time = 0 |
| 84 | dbengine tier 2 retention size = 100GB |
| 85 | dbengine tier 2 retention time = 0 |
| 86 | ``` |
| 87 | |
| 88 | 3. **Combined retention** (use with caution): |
| 89 | |
| 90 | Uses both time and space limits. Data is dropped as soon as either limit is reached. |
| 91 | |
| 92 | ```ini |
| 93 | [db] |
| 94 | dbengine tier 0 retention time = 30d |
| 95 | dbengine tier 0 retention size = 500GB # Must be large enough to hold 30 days of data! |
| 96 | dbengine tier 1 retention time = 6mo |
| 97 | dbengine tier 1 retention size = 200GB # Must be large enough to hold 6 months of data! |
| 98 | dbengine tier 2 retention time = 5y |
| 99 | dbengine tier 2 retention size = 100GB # Must be large enough to hold 5 years of data! |
| 100 | ``` |
| 101 | |
| 102 | :::warning |
| 103 | |
| 104 | Retention size limits are soft targets, not hard caps. Actual disk usage can exceed the configured limit, especially on tier 0 with high metric volumes from streaming Children. Always provision more disk space than your configured limit to avoid unexpected disk-full conditions. For the detailed enforcement behavior, see [Retention Size Enforcement](/src/database/README.md#retention-size-enforcement). |
| 105 | |
| 106 | ::: |
| 107 | |
| 108 | :::tip |
| 109 | |
| 110 | - For Parent nodes with millions of metrics, expect to allocate 100GB-1TB+ per tier. |
| 111 | - Setting `retention size = 0` means unlimited space (not zero space). This works well with time-based retention if you have sufficient disk capacity. |
| 112 | - Always validate retention sizing in staging before production to avoid premature data loss. |
| 113 | |
| 114 | ::: |
| 115 | |
| 116 | ## Estimating Disk Retention by Metric Volume on Parent Nodes |
| 117 | |
| 118 | Parent nodes are the central long-term storage layer in a Netdata infrastructure. They receive all metrics streamed from children and store them according to tiered retention settings. |
| 119 | |
| 120 | | Tier | Sample Resolution | Typical Compressed Size per Sample | |
| 121 | |--------|------------------------------------|------------------------------------| |
| 122 | | Tier 0 | per second (native) | ~0.6 B / sample | |
| 123 | | Tier 1 | per minute (60× aggregate) | ~6 B / sample | |
| 124 | | Tier 2 | per hour (60× aggregate of Tier 1) | ~18 B / sample | |
| 125 | |
| 126 | ### Example Calculation |
| 127 | |
| 128 | Assume a Parent configured with: |
| 129 | |
| 130 | - **Tier 0:** 30 days retention (per-second resolution) |
| 131 | - **Tier 1:** 6 months retention (per-minute resolution) |
| 132 | - **Tier 2:** 5 years retention (per-hour resolution) |
| 133 | |
| 134 | One metric would consume approximately **3.7 MB** across tiers. |
| 135 | For **1,000,000 metrics streamed to the Parent**, this equals **≈ 3.7 TB**. |
| 136 | |
| 137 | Adding 5–15% overhead for replication buffers, indexes, and metadata, plan for **≈ 4 TB per million metrics** under this retention policy. |
| 138 | |
| 139 | ### Configuration Example for Production Deployments |
| 140 | |
| 141 | ```ini |
| 142 | [db] |
| 143 | mode = dbengine |
| 144 | update every = 1 |
| 145 | storage tiers = 3 |
| 146 | |
| 147 | # Tier 0: per-second data for 30 days |
| 148 | dbengine tier 0 retention time = 30d |
| 149 | # No size limit - let time control retention |
| 150 | |
| 151 | # Tier 1: per-minute data for 6 months |
| 152 | dbengine tier 1 update every iterations = 60 |
| 153 | dbengine tier 1 retention time = 6mo |
| 154 | |
| 155 | # Tier 2: per-hour data for 5 years |
| 156 | dbengine tier 2 update every iterations = 60 |
| 157 | dbengine tier 2 retention time = 5y |
| 158 | ``` |
| 159 | |
| 160 | ## Cost Optimization Strategies |
| 161 | |
| 162 | Netdata helps you keep observability efficient and cost-effective: |
| 163 | |
| 164 | | Strategy | Description | Benefit | |
| 165 | |--------------------------------------------|----------------------------------------|-------------------------------------------------------------------------------------------------| |
| 166 | | **Scale Out** | Use multiple smaller Parents | Improves efficiency and performance across distributed systems | |
| 167 | | **Use Existing Resources** | Leverage spare capacity | Minimize additional hardware costs by using available resources | |
| 168 | | **Centralized or Separate Logs & Metrics** | Choose storage approach based on needs | Optimize based on access patterns, retention policies, and compliance requirements | |
| 169 | | **Flexible Configuration Management** | Customize each Parent | Control costs with unique retention and alert settings tailored for different teams or services | |
| 170 | | **Right-size Retention Based on Metrics** | Tune tier retention and sampling | Directly control disk cost by shortening or lengthening retention tiers where appropriate | |
| 171 | |
| 172 | ```mermaid |
| 173 | flowchart TB |
| 174 | A("Cost Optimization<br/>Strategies") |
| 175 | B("Scale Out") |
| 176 | C("Use Existing<br/>Resources") |
| 177 | D("Centralized or<br/>Separate Logs & Metrics") |
| 178 | E("Flexible<br/>Configuration Management") |
| 179 | B1("Multiple smaller<br/>Parents") |
| 180 | C1("Leverage spare capacity") |
| 181 | D1("Based on access needs,<br/>retention policies,<br/>and compliance") |
| 182 | E1("Unique settings for<br/>different teams or services") |
| 183 | A --> B |
| 184 | A --> C |
| 185 | A --> D |
| 186 | A --> E |
| 187 | B --> B1 |
| 188 | C --> C1 |
| 189 | D --> D1 |
| 190 | E --> E1 |
| 191 | classDef alert fill: #ffeb3b, stroke: #000000, stroke-width: 3px, color: #000000 |
| 192 | classDef neutral fill: #f9f9f9, stroke: #000000, stroke-width: 3px, color: #000000 |
| 193 | classDef complete fill: #4caf50, stroke: #000000, stroke-width: 3px, color: #000000 |
| 194 | class A neutral |
| 195 | class B complete |
| 196 | class C complete |
| 197 | class D complete |
| 198 | class E complete |
| 199 | class B1 complete |
| 200 | class C1 complete |
| 201 | class D1 complete |
| 202 | class E1 complete |
| 203 | ``` |
| 204 | |
| 205 | ## Advantages of Netdata's Approach |
| 206 | |
| 207 | Netdata provides several benefits over other observability solutions: |
| 208 | |
| 209 | | Advantage | Description | Value | |
| 210 | |-----------------------------------|--------------------------------------------|-----------------------------------------------------------------------| |
| 211 | | **Scalability & Flexibility** | Multiple independent Parents | Customized observability by region, service, or team | |
| 212 | | **Resilience & Reliability** | Built-in replication | Observability continues even if a Parent fails | |
| 213 | | **Optimized Cost & Performance** | Distributed workloads | Prevents bottlenecks and improves resource efficiency | |
| 214 | | **Ease of Use** | Minimal setup and maintenance | Reduces complexity and operational overhead | |
| 215 | | **On-Prem Control** | Data remains within your infrastructure | Enhanced security and compliance, even when using Netdata Cloud. For a fully self-hosted control plane, see [Netdata Cloud On-Prem](https://github.com/netdata/netdata-cloud-onprem/blob/master/docs/learn.netdata.cloud/README.md) | |
| 216 | | **Comprehensive Observability** | Segmented infrastructure with unified view | Deep visibility with tailored retention, alerts, and machine learning | |
| 217 | | **Predictable Capacity Planning** | Published per-metric storage cost | Allows accurate disk and hardware sizing for Parents | |
| 218 | |
| 219 | :::tip |
| 220 | |
| 221 | Following these best practices helps you maintain a **cost-effective**, **high-performance** observability setup with Netdata. |
| 222 | |
| 223 | ::: |