Update nodes-ephemerality.md (#21471)
Co-authored-by: ilyam8 <ilya@netdata.cloud>
Kanela committed
Dec 18, 2025 at 17:28 UTC
bb7e21574dda54f9b76c3d04aba53ed51c9a9c44
1 file changed
+49
-73
docs/nodes-ephemerality.md
+49
-73
@@ -27,40 +27,13 @@ By default, Netdata treats all nodes as permanent. To mark a node as ephemeral:
27
28
1. Open the `netdata.conf` file on the target node.
29
2. Add the following configuration:
30
-
30
```ini
31
[global]
33
- is ephemeral node = yes
32
+ is ephemeral node = yes
33
```
35
-
34
3. Restart the Netdata Agent.
35
38
-This applies the `_is_ephemeral` host label, which propagates to your Parents and Netdata Cloud.
39
-
40
-<details>
41
-<summary><strong>Click to see visual representation of configuration flow</strong></summary><br/>
42
-
43
-```mermaid
44
-flowchart TD
45
- A[Node is Permanent by Default] -->|Step 1| B[Open netdata.conf on Target Node]
46
- B -->|Step 2| C[Add Configuration]
47
- C -->|Step 3| D[Restart the Node]
48
- D --> E[Node Now Marked as Ephemeral]
49
- E --> F[_is_ephemeral Label Applied]
50
- F --> G[Label Propagates to Parents and Cloud]
51
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
52
- classDef label fill: #f3e8ff, stroke: #9b59b6, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
53
- classDef subgraphStyle fill: #f8f9fa, stroke: #6c757d, stroke-width: 2px, color: #2c3e50, rx: 15, ry: 15
54
- class A step
55
- class B step
56
- class C step
57
- class D step
58
- class E label
59
- class F label
60
- class G subgraphStyle
61
-```
62
-
63
-</details>
36
+Once restarted, Netdata automatically applies the `_is_ephemeral` host label to the node. This label propagates to your Parent nodes and appears in Netdata Cloud, allowing the system to identify and handle the node as ephemeral.
37
38
## Alerts for Parent Nodes
39
@@ -71,6 +44,24 @@ Netdata v2.3.0 introduces two alerts specific to permanent nodes:
44
| `streaming_never_connected` | A permanent node has never connected to a Parent. |
45
| `streaming_disconnected` | A previously connected permanent node has disconnected. |
46
47
+## Automatic Node Instance Cleanup in Netdata Cloud
48
+
49
+Netdata Cloud automatically removes inactive nodes to keep your dashboards clean and organized.
50
+
51
+### Cleanup Rules
52
+
53
+| Node Type | Offline Duration | Description |
54
+|------------------------------|------------------|------------------------------------------------------------------------------------------------------------------------------------|
55
+| **Child nodes** | 48 hours | Nodes that connect through a Parent node. Deleted when:<br />• The child node goes offline, OR<br />• The Parent node goes offline |
56
+| **Directly connected nodes** | 60 days | Nodes claimed directly to Netdata Cloud without going through a Parent. |
57
+| **Ephemeral nodes** | Custom | Temporary nodes (containers, auto-scaling VMs) with configurable cleanup periods. See configuration below. |
58
+
59
+:::info Important
60
+
61
+If a child node is deleted and later reconnects, it's automatically recreated in Netdata Cloud. Any historical data retained on the Parent node remains accessible.
62
+
63
+:::
64
+
65
## Monitoring and Managing Node Status
66
67
### Mark Permanently Offline Nodes as Ephemeral
@@ -83,20 +74,17 @@ netdatacli mark-stale-nodes-ephemeral <node_id | machine_guid | hostname | ALL_N
74
75
This keeps historical data queryable and clears active alerts.
76
86
-<details>
87
-<summary><strong>Click to see visual representation of CLI workflow</strong></summary><br/>
88
-
77
```mermaid
78
flowchart TD
91
- A[Offline Node Detected] -->|Run CLI Command| B[Use netdatacli mark-stale-nodes-ephemeral]
92
- B --> C[Node Marked as Ephemeral]
93
- C --> D[Metrics Remain Available]
94
- C --> E[Active Alerts Cleared]
95
- C --> F{Node Reconnects?}
96
- F -->|Yes - no config| G[Reverts to Permanent]
97
- F -->|No| H[Remains Ephemeral]
98
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
99
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
79
+ A("**Offline Node Detected**") -->|Run CLI Command| B("**Use netdatacli mark-stale-nodes-ephemeral**")
80
+ B --> C("**Node Marked as Ephemeral**")
81
+ C --> D("**Metrics Remain Available**")
82
+ C --> E("**Active Alerts Cleared**")
83
+ C --> F{"**Node Reconnects?**"}
84
+ F -->|Yes - no config| G("**Reverts to Permanent**")
85
+ F -->|No| H("**Remains Ephemeral**")
86
+ classDef step fill: #4caf50, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
87
+ classDef alert fill: #ffeb3b, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
88
class A step
89
class B step
90
class C step
@@ -107,8 +95,6 @@ flowchart TD
95
class H alert
96
```
97
110
-</details>
111
-
98
### Removing Offline Nodes
99
100
To fully remove permanently offline nodes:
@@ -123,17 +109,13 @@ For detailed instructions on removing nodes from Netdata Cloud (including **offl
109
110
:::
111
126
-<details>
127
-<summary><strong>Click to see visual representation of node removal flow</strong></summary><br/>
128
-
112
```mermaid
113
flowchart TD
131
- A[Offline Node Detected] -->|Run CLI Tool| B[Execute remove-stale-node Command]
132
- B --> C[Node Removed from System]
133
- C --> D[Node No Longer Queryable]
134
- C --> E[Alerts for Node Cleared]
135
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
136
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
114
+ A("**Offline Node Detected**") -->|Run CLI Tool| B("**Execute remove-stale-node Command**")
115
+ B --> C("**Node Removed from System**")
116
+ C --> D("**Node No Longer Queryable**")
117
+ C --> E("**Alerts for Node Cleared**")
118
+ classDef step fill: #4caf50, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
119
class A step
120
class B step
121
class C step
@@ -141,8 +123,6 @@ flowchart TD
123
class E step
124
```
125
144
-</details>
145
-
126
## Automatically Removing Ephemeral Nodes
127
128
To enable automatic cleanup of ephemeral nodes:
@@ -159,28 +139,24 @@ To enable automatic cleanup of ephemeral nodes:
139
140
This removes ephemeral nodes after 24 hours of disconnection. Once all Parents purge the node, it is automatically removed from Netdata Cloud.
141
162
-<details>
163
-<summary><strong>Click to see visual representation of auto-removal process</strong></summary><br/>
164
-
142
```mermaid
143
flowchart TD
167
- A[Configure Auto-Removal in netdata.conf] --> B[Restart Parent Nodes]
168
- B --> C[Ephemeral Node Disconnects]
169
- C --> D{Wait Period Elapsed?}
170
- D -->|Yes| E[Node Automatically Removed]
171
- D -->|No| F[Node Remains in System]
172
- E --> G{All Parents Removed Node?}
173
- G -->|Yes| H[Node Removed from Cloud]
174
- classDef step fill: #e8f5e8, stroke: #27ae60, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
175
- classDef alert fill: #ffe8e8, stroke: #e74c3c, stroke-width: 2px, color: #2c3e50, rx: 10, ry: 10
144
+ A("**Configure Auto-Removal in netdata.conf**") --> B("**Restart Parent Nodes**")
145
+ B --> C("**Ephemeral Node Disconnects**")
146
+ C --> D{"**Wait Period Elapsed?**"}
147
+ D -->|Yes| E("**Node Automatically Removed**")
148
+ D -->|No| F("**Node Remains in System**")
149
+ E --> G{"**All Parents Removed Node?**"}
150
+ G -->|Yes| H("**Node Removed from Cloud**")
151
+ classDef step fill: #4caf50, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
152
+ classDef alert fill: #ffeb3b, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
153
+ classDef database fill: #2196F3, stroke: #000000, stroke-width: 3px, color: #000000, font-size: 16px
154
class A step
155
class B step
156
class C step
179
- class D step
180
- class E step
181
- class F step
182
- class G step
183
- class H step
157
+ class D alert
158
+ class E alert
159
+ class F alert
160
+ class G database
161
+ class H database
162
```
185
-
186
-</details>