@cryptotaxi247 / netdata
Code Commits Issues Pulls Sessions CI/CD
master
master
md 264 lines 7.99 KB
Rendered Raw
1 # Organize systems, metrics, and alerts
2
3 When you monitor dozens or hundreds of systems, you need powerful ways to keep everything organized. Netdata helps you structure your infrastructure with Spaces, Rooms, virtual nodes, host labels, and metric labels.
4
5 ## Choose your organization strategy
6
7 Netdata provides multiple organization methods that work together:
8
9 - **Spaces and Rooms**: Group your infrastructure and team members
10 - **Virtual nodes**: Monitor multi-component systems as separate entities
11 - **Host labels**: Tag systems by purpose, location, or any custom criteria
12 - **Metric labels**: Filter and group metrics within charts
13
14 ### Organize your infrastructure and team
15
16 <details>
17 <summary><strong>Spaces</strong> are your primary collaboration environment where you:</summary>
18
19 - Organize team members and manage access levels
20 - Connect nodes for monitoring
21 - Create a unified monitoring environment
22
23 </details>
24
25 <details>
26 <summary><strong>Rooms</strong> function as organizational units within Spaces, providing:</summary>
27
28 - Infrastructure-wide dashboards
29 - Real-time metrics visualization
30 - Focused monitoring views
31 - Flexible node grouping
32
33 </details>
34
35 <br/>
36
37 :::info
38
39 Each node belongs to exactly one Space but can be assigned to multiple Rooms within that Space.
40
41 :::
42
43 ### Set up your organization
44
45 1. **Create a Space** using the plus (+) icon in the left-most sidebar
46 2. **Invite team members** and set their access levels
47 3. **Create Rooms** to organize nodes by:
48 - Service type (Nginx, MySQL, Pulsar)
49 - Purpose (webserver, database, application)
50 - Location or infrastructure type (cloud provider, bare metal, containers)
51
52 :::tip
53
54 Most organizations need only one Space. Create multiple Rooms within that Space to organize your infrastructure effectively.
55
56 :::
57
58 Learn more in our [Spaces and Rooms documentation](/docs/netdata-cloud/organize-your-infrastructure-invite-your-team.md).
59
60 ## Virtual nodes
61
62 ### Monitor complex systems as separate entities
63
64 Virtual nodes let you split multi-component systems into distinct, monitorable units. For example, you can monitor each Windows server in your infrastructure as its own node, even when collecting metrics through a single Netdata Agent.
65
66 To create a virtual node for your Windows server:
67
68 1. Define the virtual node in `/etc/netdata/vnodes/vnodes.conf`:
69
70 ```yaml
71 - hostname: win_server1
72 guid: <value>
73 ```
74
75 :::tip
76 Generate a valid GUID using `uuidgen` on Linux or `[guid]::NewGuid()` in Windows PowerShell.
77 :::
78
79 2. Add the vnode configuration to your data collection job in `go.d/windows.conf`:
80
81 ```yaml
82 jobs:
83 - name: win_server1
84 vnode: win_server1
85 url: http://203.0.113.10:9182/metrics
86 ```
87
88 ## Host labels
89
90 ### Tag your systems for smarter monitoring
91
92 Host labels help you:
93
94 - Create alerts that adapt to each system's purpose
95 - Archive metrics with proper categorization for analysis
96 - Track ephemeral containers in Kubernetes clusters
97
98 ### Use automatic labels
99
100 Netdata automatically generates host labels when it starts, capturing:
101
102 | Label Category | Information Captured |
103 |----------------|-----------------------------------------------------|
104 | System Info | Kernel version, OS name and version |
105 | Hardware | CPU architecture, cores, frequency, RAM, disk space |
106 | Environment | Container details, Kubernetes node status |
107 | Infrastructure | Virtualization layer, Parent-child streaming status |
108
109 View your automatic labels at `http://HOST-IP:19999/api/v1/info`:
110
111 ```json
112 {
113 "host_labels": {
114 "_is_k8s_node": "false",
115 "_is_parent": "false"
116 }
117 }
118 ```
119
120 ### Create custom labels
121
122 Add your own labels to categorize systems by any criteria you need.
123
124 1. Edit your Netdata configuration:
125
126 ```bash
127 cd /etc/netdata # Replace with your Netdata config directory
128 sudo ./edit-config netdata.conf
129 ```
130
131 2. Add a `[host labels]` section:
132
133 ```text
134 [host labels]
135 type = webserver
136 location = us-seattle
137 installed = 20200218
138 ```
139
140 :::info Label naming rules
141 - Names cannot start with `_`
142 - Use only letters, numbers, dots, and dashes
143 - Values cannot contain: `!` ` ` `'` `"` `*`
144 :::
145
146 3. You can use environment variables in label values:
147
148 ```text
149 [host labels]
150 region = ${REGION}
151 rack = ${RACK:-unknown}
152 env = ${DEPLOYMENT_ENV:-production}
153 location = ${DC}-${RACK:-default}
154 ```
155
156 | Syntax | Behavior |
157 |--------|----------|
158 | `${VAR}` | Replaced with the value of `VAR`. If unset or empty, the label value becomes `[none]` |
159 | `${VAR:-default}` | Replaced with the value of `VAR`. If unset or empty, uses `default` |
160
161 Environment variables are resolved when labels are loaded or reloaded. You can mix them with literal text (e.g., `${DC}-${RACK}`).
162
163 4. Enable your labels without restarting Netdata:
164
165 ```bash
166 netdatacli reload-labels
167 ```
168
169 5. Verify your labels at `http://HOST-IP:19999/api/v1/info`
170
171 Use custom host labels such as `environment` with [Node Rule-Based Room Assignment](/docs/netdata-cloud/node-rule-based-room-assignment.md) to route Kubernetes Nodes into separate Rooms.
172
173 ### Stream labels from Child to Parent
174
175 In Parent-Child setups, host labels automatically stream from children to the parent node. Access any child's labels through the parent at:
176 `http://localhost:19999/host/CHILD_HOSTNAME/api/v1/info`
177
178 :::warning
179
180 Child node labels contain sensitive system information. Secure your streaming connections with SSL and consider using [access lists](/src/web/server/README.md#access-lists) or [restricting API access](/docs/netdata-agent/securing-netdata-agents.md#alternative-methods).
181
182 :::
183
184 ### Apply labels to alerts
185
186 Create targeted alerts based on host labels. For example, monitor disk space only on webservers:
187
188 ```text
189 template: disk_fill_rate
190 on: disk.space
191 lookup: max -1s at -30m unaligned of avail
192 calc: ($this - $avail) / (30 * 60)
193 every: 15s
194 host labels: type = webserver
195 ```
196
197 Target systems by multiple criteria:
198
199 | Target | Host Label | Use Case |
200 |-------------------|-----------------------|--------------------------------|
201 | Specific OS | `_os_name = Debian*` | Apply alerts to Debian systems |
202 | Child nodes only | `_is_child = true` | Monitor streaming children |
203 | Docker containers | `_container = docker` | Container-specific alerts |
204
205 See the [health documentation](/src/health/REFERENCE.md#alert-line-host-labels) for more possibilities.
206
207 ### Export labels with metrics
208
209 When using [metrics exporters](/src/exporting/README.md), include host labels with your exported data:
210
211 ```text
212 [exporting:global]
213 enabled = yes
214 send configured labels = yes
215 send automatic labels = no
216 ```
217
218 Configure per-connection settings:
219
220 ```text
221 [opentsdb:my_instance3]
222 enabled = yes
223 destination = localhost:4242
224 data source = sum
225 update every = 10
226 send charts matching = system.cpu
227 send configured labels = no
228 send automatic labels = yes
229 ```
230
231 ## Metric labels
232
233 ### Filter and group metrics within charts
234
235 Netdata's aggregate charts let you filter and group metrics using label name-value pairs. All go.d plugin collectors support labels at the collection job level.
236
237 Configure metric labels when collected from multiple sources. For example, label two Apache servers by service and location:
238
239 ```yaml
240 jobs:
241 - name: my_webserver1
242 url: http://host1/server-status?auto
243 labels:
244 service: "Payments"
245 location: "Atlanta"
246 - name: my_webserver2
247 url: http://host2/server-status?auto
248 labels:
249 service: "Payments"
250 location: "New York"
251 ```
252
253 :::tip
254
255 Define as many label pairs as you need across all your data collection jobs to create meaningful groupings in your dashboards.
256
257 :::
258
259 ## Next steps
260
261 1. **Start with Spaces and Rooms** to organize your infrastructure and team
262 2. **Add host labels** to categorize your systems
263 3. **Configure metric labels** for detailed filtering within charts
264 4. **Set up virtual nodes** if you monitor complex, multi-component systems