bring back old docs that were containing missing information (#17146)
* Create deployment-strategies.md * Create start-stop-restart.md
Fotis Voutsas committed
Mar 11, 2024 at 14:59 UTC
eff544439c719c2f4bfee83f3f0ef11c25042e76
2 files changed
+392
docs/deployment-guides/deployment-strategies.md
new
+239
@@ -0,0 +1,239 @@
1
+# Deployment strategies
2
+
3
+
4
+## Deployment Options Overview
5
+
6
+This section provides a quick overview of a few common deployment options. The next sections go into configuration examples and further reading.
7
+
8
+### Stand-alone Deployment
9
+
10
+To help our users have a complete experience of Netdata when they install it for the first time, a Netdata Agent with default configuration
11
+is a complete monitoring solution out of the box, having all these features enabled and available.
12
+
13
+The Agent will act as a _stand-alone_ Agent by default, and this is great to start out with for small setups and home labs. By [connecting each Agent to Cloud](https://github.com/netdata/netdata/blob/master/src/claim/README.md), you can see an overview of all your nodes, with aggregated charts and centralized alerting, without setting up a Parent.
14
+
15
+
16
+
17
+### Parent – Child Deployment
18
+
19
+An Agent connected to a Parent is called a _Child_. It will _stream_ metrics to its Parent. The Parent can then take care of storing metrics on behalf of that node (with longer retention), handle metrics queries for showing dashboards, and provide alerting.
20
+
21
+When using Cloud, it is recommended that just the Parent is connected to Cloud. Child Agents can then be configured to have short retention, in RAM instead of on Disk, and have alerting and other features disabled. Because they don't need to connect to Cloud themselves, those children can then be further secured by not allowing outbound traffic.
22
+
23
+
24
+
25
+This setup allows for leaner Child nodes and is good for setups with more than a handful of nodes. Metrics data remains accessible if the Child node is temporarily unavailable or decommissioned, although there is no failover in case the Parent becomes unavailable.
26
+
27
+
28
+### Active–Active Parent Deployment
29
+
30
+For high availability, Parents can be configured to stream data for their children between them, and keep the data sets in sync. Child Agents are configured with the addresses of both Parent Agents, but will only stream to one of them at a time. When that Parent becomes unavailable, it reconnects to another. When the first Parent becomes available again, that Parent will catch up by receiving the backlog from the second.
31
+
32
+With both Parent Agents connected to Cloud, Cloud will route queries to either Parent transparently, depending on their availability. Alerts trigger on either Parent will stream to Cloud, and Cloud will deduplicate and debounce state changes to prevent spurious notifications.
33
+
34
+
35
+
36
+
37
+## Configuration Details
38
+
39
+### Stand-alone Deployment
40
+
41
+The stand-alone setup is configured out of the box with reasonable defaults, but please consult our [configuration documentation](https://github.com/netdata/netdata/blob/master/docs/cloud/cheatsheet.md) for details, including the overview of [common configuration changes](https://github.com/netdata/netdata/blob/master/docs/configure/common-changes.md).
42
+
43
+### Parent – Child Deployment
44
+
45
+For setups involving Child and Parent Agents, the Agents need to be configured for [_streaming_](https://github.com/netdata/netdata/blob/master/src/streaming/README.md), through the configuration file `stream.conf`. This will instruct the Child to stream data to the Parent and the Parent to accept streaming connections for one or more Child Agents. To secure this connection, both need set up a shared API key (to replace the string `API_KEY` in the examples below). Additionally, the Child is configured with one or more addresses of Parent Agents (`PARENT_IP_ADDRESS`).
46
+
47
+An API key is a key created with `uuidgen` and is used for authentication and/or customization in the Parent side. I.e. a Child will stream using the API key, and a Parent is configured to accept connections from Child, but can also apply different options for children by using multiple different API keys. The easiest setup uses just one API key for all Child Agents.
48
+
49
+#### Child config
50
+
51
+As mentioned above, the recommendation is to not claim the Child to Cloud directly during your setup, avoiding establishing an [ACLK](https://github.com/netdata/netdata/blob/master/src/aclk/README.md) connection.
52
+
53
+To reduce the footprint of the Netdata Agent on your production system, some capabilities can be switched OFF on the Child and kept ON on the Parent. In this example, Machine Learning and Alerting are disabled in the Child, so that the Parent can take the load. We also use RAM instead of disk to store metrics with limited retention, covering temporary network issues.
54
+
55
+##### netdata.conf
56
+
57
+On the child node, edit `netdata.conf` by using the edit-config script: `/etc/netdata/edit-config netdata.conf` set the following parameters:
58
+
59
+```yaml
60
+[db]
61
+ # https://learn.netdata.cloud/docs/agent/database
62
+ # none = no retention, ram = some retention in ram
63
+ mode = ram
64
+ # The retention in seconds.
65
+ # This provides some tolerance to the time the child has to find a parent in
66
+ # order to transfer the data. For IoT this can be lowered to 120.
67
+ retention = 1200
68
+ # The granularity of metrics, in seconds.
69
+ # You may increase this to lower CPU resources.
70
+ update every = 1
71
+[ml]
72
+ # Disable Machine Learning
73
+ enabled = no
74
+[health]
75
+ # Disable Health Checks (Alerting)
76
+ enabled = no
77
+[web]
78
+ # Disable remote access to the local dashboard
79
+ bind to = lo
80
+[plugins]
81
+ # Uncomment the following line to disable all external plugins on extreme
82
+ # IoT cases by default.
83
+ # enable running new plugins = no
84
+```
85
+
86
+##### stream.conf
87
+
88
+To edit `stream.conf`, again use the edit-config script: `/etc/netdata/edit-config stream.conf`.
89
+
90
+Set the following parameters:
91
+
92
+```yaml
93
+[stream]
94
+ # Stream metrics to another Netdata
95
+ enabled = yes
96
+ # The IP and PORT of the parent
97
+ destination = PARENT_IP_ADDRESS:19999
98
+ # The shared API key, generated by uuidgen
99
+ api key = API_KEY
100
+```
101
+
102
+#### Parent config
103
+
104
+For the Parent, besides setting up streaming, the example will also provide an example configuration of multiple [tiers](https://github.com/netdata/netdata/blob/master/src/database/engine/README.md#tiering) of metrics [storage](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md), for 10 children, with about 2k metrics each.
105
+
106
+- 1s granularity at tier 0 for 1 week
107
+- 1m granularity at tier 1 for 1 month
108
+- 1h granularity at tier 2 for 1 year
109
+
110
+Requiring:
111
+
112
+- 25GB of disk
113
+- 3.5GB of RAM (2.5GB under pressure)
114
+
115
+##### netdata.conf
116
+
117
+On the Parent, edit `netdata.conf` with `/etc/netdata/edit-config netdata.conf` and set the following parameters:
118
+
119
+```yaml
120
+[db]
121
+ mode = dbengine
122
+ storage tiers = 3
123
+ # To allow memory pressure to offload index from ram
124
+ dbengine page descriptors in file mapped memory = yes
125
+ # storage tier 0
126
+ update every = 1
127
+ dbengine multihost disk space MB = 12000
128
+ dbengine page cache size MB = 1400
129
+ # storage tier 1
130
+ dbengine tier 1 page cache size MB = 512
131
+ dbengine tier 1 multihost disk space MB = 4096
132
+ dbengine tier 1 update every iterations = 60
133
+ dbengine tier 1 backfill = new
134
+ # storage tier 2
135
+ dbengine tier 2 page cache size MB = 128
136
+ dbengine tier 2 multihost disk space MB = 2048
137
+ dbengine tier 2 update every iterations = 60
138
+ dbengine tier 2 backfill = new
139
+[ml]
140
+ # Enabled by default
141
+ # enabled = yes
142
+[health]
143
+ # Enabled by default
144
+ # enabled = yes
145
+[web]
146
+ # Enabled by default
147
+ # bind to = *
148
+```
149
+
150
+##### stream.conf
151
+
152
+On the Parent node, edit `stream.conf` with `/etc/netdata/edit-config stream.conf`, and then set the following parameters:
153
+
154
+```yaml
155
+[API_KEY]
156
+ # Accept metrics streaming from other Agents with the specified API key
157
+ enabled = yes
158
+```
159
+
160
+### Active–Active Parent Deployment
161
+
162
+In order to setup active–active streaming between Parent 1 and Parent 2, Parent 1 needs to be instructed to stream data to Parent 2 and Parent 2 to stream data to Parent 1. The Child Agents need to be configured with the addresses of both Parent Agents. The Agent will only connect to one Parent at a time, falling back to the next if the previous failed. These examples use the same API key between Parent Agents as for connections from Child Agents.
163
+
164
+On both Netdata Parent and all Child Agents, edit `stream.conf` with `/etc/netdata/edit-config stream.conf`:
165
+
166
+##### stream.conf on Parent 1
167
+
168
+```yaml
169
+[stream]
170
+ # Stream metrics to another Netdata
171
+ enabled = yes
172
+ # The IP and PORT of Parent 2
173
+ destination = PARENT_2_IP_ADDRESS:19999
174
+ # This is the API key for the outgoing connection to Parent 2
175
+ api key = API_KEY
176
+[API_KEY]
177
+ # Accept metrics streams from Parent 2 and Child Agents
178
+ enabled = yes
179
+```
180
+
181
+##### stream.conf on Parent 2
182
+
183
+```yaml
184
+[stream]
185
+ # Stream metrics to another Netdata
186
+ enabled = yes
187
+ # The IP and PORT of Parent 1
188
+ destination = PARENT_1_IP_ADDRESS:19999
189
+ api key = API_KEY
190
+[API_KEY]
191
+ # Accept metrics streams from Parent 1 and Child Agents
192
+ enabled = yes
193
+```
194
+
195
+##### stream.conf on Child Agents
196
+
197
+```yaml
198
+[stream]
199
+ # Stream metrics to another Netdata
200
+ enabled = yes
201
+ # The IP and PORT of the parent
202
+ destination = PARENT_1_IP_ADDRESS:19999 PARENT_2_IP_ADDRESS:19999
203
+ # The shared API key, generated by uuidgen
204
+ api key = API_KEY
205
+```
206
+
207
+## Further Reading
208
+
209
+We strongly recommend the following configuration changes for production deployments:
210
+
211
+1. Understand Netdata's [security and privacy design](https://github.com/netdata/netdata/blob/master/docs/security-and-privacy-design/README.md) and
212
+ [secure your nodes](https://github.com/netdata/netdata/blob/master/docs/category-overview-pages/secure-nodes.md)
213
+
214
+ To safeguard your infrastructure and comply with your organization's security policies.
215
+
216
+2. Set up [streaming and replication](https://github.com/netdata/netdata/blob/master/src/streaming/README.md) to:
217
+
218
+ - Offload Netdata Agents running on production systems and free system resources for the production applications running on them.
219
+ - Isolate production systems from the rest of the world and improve security.
220
+ - Increase data retention.
221
+ - Make your data highly available.
222
+
223
+3. [Optimize the Netdata Agents system utilization and performance](https://github.com/netdata/netdata/blob/master/docs/guides/configure/performance.md)
224
+
225
+ To save valuable system resources, especially when running on weak IoT devices.
226
+
227
+We also suggest that you:
228
+
229
+1. [Use Netdata Cloud to access the dashboards](https://github.com/netdata/netdata/blob/master/docs/quickstart/infrastructure.md)
230
+
231
+ For increased security, user management and access to our latest tools for advanced dashboarding and troubleshooting.
232
+
233
+2. [Change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md)
234
+
235
+ To control Netdata's memory use, when you have a lot of ephemeral metrics.
236
+
237
+3. [Use host labels](https://github.com/netdata/netdata/blob/master/docs/guides/using-host-labels.md)
238
+
239
+ To organize systems, metrics, and alerts.
docs/maintenance/start-stop-restart.md
new
+153
@@ -0,0 +1,153 @@
1
+# Start, stop, or restart the Netdata Agent
2
+
3
+When you install the Netdata Agent, the [daemon](https://github.com/netdata/netdata/blob/master/src/daemon/README.md) is
4
+configured to start at boot and stop and restart/shutdown.
5
+
6
+You will most often need to _restart_ the Agent to load new or editing configuration files.
7
+[Health configuration](#reload-health-configuration) files are the only exception, as they can be reloaded without restarting
8
+the entire Agent.
9
+
10
+Stopping or restarting the Netdata Agent will cause gaps in stored metrics until the `netdata` process initiates
11
+collectors and the database engine.
12
+
13
+## Using `systemctl`, `service`, or `init.d`
14
+
15
+This is the recommended way to start, stop, or restart the Netdata daemon.
16
+
17
+- To **start** Netdata, run `sudo systemctl start netdata`.
18
+- To **stop** Netdata, run `sudo systemctl stop netdata`.
19
+- To **restart** Netdata, run `sudo systemctl restart netdata`.
20
+
21
+If the above commands fail, or you know that you're using a non-systemd system, try using the `service` command:
22
+
23
+- **service**: `sudo service netdata start`, `sudo service netdata stop`, `sudo service netdata restart`
24
+
25
+## Using `netdata`
26
+
27
+Use the `netdata` command, typically located at `/usr/sbin/netdata`, to start the Netdata daemon.
28
+
29
+```bash
30
+sudo netdata
31
+```
32
+
33
+If you start the daemon this way, close it with `sudo killall netdata`.
34
+
35
+## Using `netdatacli`
36
+
37
+The Netdata Agent also comes with a [CLI tool](https://github.com/netdata/netdata/blob/master/src/cli/README.md) capable of performing shutdowns. Start the Agent back up
38
+using your preferred method listed above.
39
+
40
+```bash
41
+sudo netdatacli shutdown-agent
42
+```
43
+
44
+## Netdata MSI installations
45
+
46
+Netdata provides an installer for Windows using WSL, on those installations by using a Windows terminal (e.g. the Command prompt or Windows Powershell) you can:
47
+
48
+- Start Netdata, by running `start-netdata`
49
+- Stop Netdata, by running `stop-netdata`
50
+- Restart Netdata, by running `restart-netdata`
51
+
52
+## Reload health configuration
53
+
54
+You do not need to restart the Netdata Agent between changes to health configuration files, such as specific health
55
+entities. Instead, use [`netdatacli`](#using-netdatacli) and the `reload-health` option to prevent gaps in metrics
56
+collection.
57
+
58
+```bash
59
+sudo netdatacli reload-health
60
+```
61
+
62
+If `netdatacli` doesn't work on your system, send a `SIGUSR2` signal to the daemon, which reloads health configuration
63
+without restarting the entire process.
64
+
65
+```bash
66
+killall -USR2 netdata
67
+```
68
+
69
+## Force stop stalled or unresponsive `netdata` processes
70
+
71
+In rare cases, the Netdata Agent may stall or not properly close sockets, preventing a new process from starting. In
72
+these cases, try the following three commands:
73
+
74
+```bash
75
+sudo systemctl stop netdata
76
+sudo killall netdata
77
+ps aux| grep netdata
78
+```
79
+
80
+The output of `ps aux` should show no `netdata` or associated processes running. You can now start the Netdata Agent
81
+again with `service netdata start`, or the appropriate method for your system.
82
+
83
+## Starting Netdata at boot
84
+
85
+In the `system` directory you can find scripts and configurations for the
86
+various distros.
87
+
88
+### systemd
89
+
90
+The installer already installs `netdata.service` if it detects a systemd system.
91
+
92
+To install `netdata.service` by hand, run:
93
+
94
+```sh
95
+# stop Netdata
96
+killall netdata
97
+
98
+# copy netdata.service to systemd
99
+cp system/netdata.service /etc/systemd/system/
100
+
101
+# let systemd know there is a new service
102
+systemctl daemon-reload
103
+
104
+# enable Netdata at boot
105
+systemctl enable netdata
106
+
107
+# start Netdata
108
+systemctl start netdata
109
+```
110
+
111
+### init.d
112
+
113
+In the system directory you can find `netdata-lsb`. Copy it to the proper place according to your distribution
114
+documentation. For Ubuntu, this can be done via running the following commands as root.
115
+
116
+```sh
117
+# copy the Netdata startup file to /etc/init.d
118
+cp system/netdata-lsb /etc/init.d/netdata
119
+
120
+# make sure it is executable
121
+chmod +x /etc/init.d/netdata
122
+
123
+# enable it
124
+update-rc.d netdata defaults
125
+```
126
+
127
+### openrc (gentoo)
128
+
129
+In the `system` directory you can find `netdata-openrc`. Copy it to the proper
130
+place according to your distribution documentation.
131
+
132
+### CentOS / Red Hat Enterprise Linux
133
+
134
+For older versions of RHEL/CentOS that don't have systemd, an init script is included in the system directory. This can
135
+be installed by running the following commands as root.
136
+
137
+```sh
138
+# copy the Netdata startup file to /etc/init.d
139
+cp system/netdata-init-d /etc/init.d/netdata
140
+
141
+# make sure it is executable
142
+chmod +x /etc/init.d/netdata
143
+
144
+# enable it
145
+chkconfig --add netdata
146
+```
147
+
148
+_There have been some recent work on the init script, see PR
149
+<https://github.com/netdata/netdata/pull/403>_
150
+
151
+### other systems
152
+
153
+You can start Netdata by running it from `/etc/rc.local` or equivalent.