Update performance.md (#14776)
Chris Akritidis committed
Mar 21, 2023 at 05:40 UTC
00621f5e5ada15fc9e8153a734bb855f35d1072c
1 file changed
+92
-100
docs/guides/configure/performance.md
+92
-100
@@ -12,6 +12,10 @@ This document describes the resources required for the various default capabilit
12
13
## Resources required by a default Netdata installation
14
15
+Netdata's performance is primarily affected by **data collection/retention** and **clients accessing data**.
16
+
17
+You can configure almost all aspects of data collection/retention, and certain aspects of clients accessing data.
18
+
19
### CPU consumption
20
21
Expect about:
@@ -26,38 +30,103 @@ As a general rule, for modern hardware and VMs, the total CPU consumption of a s
30
should be below 5 - 15% of a single core. For example, on 8 core server it will use only 0.6% - 1.8% of a total CPU capacity, depending on
31
the CPU characteristics.
32
33
+The Netdata Agent runs with the lowest possible [process scheduling policy](https://github.com/netdata/netdata/blob/master/daemon/README.md#netdata-process-scheduling-policy), which is `nice 19`, and uses the `idle` process scheduler.
34
+Together, these settings ensure that the Agent only gets CPU resources when the node has CPU resources to space. If the
35
+node reaches 100% CPU utilization, the Agent is stopped first to ensure your applications get any available resources.
36
+
37
+To reduce CPU usage you can [disable machine learning](#disable-machine-learning),
38
+[use streaming and replication](#use-streaming-and-replication),
39
+[reduce the data collection frequency](#reduce-collection-frequency), [disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors), [use a reverse proxy](#run-netdata-behind-a-proxy), and [disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard).
40
+
41
### Memory consumption
42
43
The memory footprint of Netdata is mainly influenced by the number of metrics concurrently being collected. Expect about 150MB of RAM for a typical 64-bit server collecting about 2000 to 3000 metrics.
44
33
-To estimate and control memory consumption, you can [change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md), or [use a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md).
45
+To estimate and control memory consumption, you can [disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors), [change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md), or [use a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md).
46
+
47
48
### Disk footprint and I/O
49
50
By default, Netdata should not use more than 1GB of disk space, most of which is dedicated for storing metric data and metadata. For typical installations collecting 2000 - 3000 metrics, this storage should provide a few days of high-resolution retention (per second), about a month of mid-resolution retention (per minute) and more than a year of low-resolution retention (per hour).
51
39
-Netdata spreads I/O operations across time. For typical standalone installations there should be a few write operations every 5-10 seconds of a few kilobytes each, occasionally up to 1MB.
52
+Netdata spreads I/O operations across time. For typical standalone installations there should be a few write operations every 5-10 seconds of a few kilobytes each, occasionally up to 1MB. In addition, under heavy load, collectors that require disk I/O may stop and show gaps in charts.
53
54
To configure retention, you can [change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md).
55
To control disk I/O [use a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md), avoid querying the
43
-production system using a [streaming and replication](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md), [reduce the data collection frequency](#reduce-collection-frequency).
56
+production system [using streaming and replication](#use-streaming-and-replication), [reduce the data collection frequency](#reduce-collection-frequency), and [disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors).
57
45
-## What affects Netdata's performance?
58
+## Summary of performance optimizations
59
47
-Netdata's performance is primarily affected by **data collection/retention** and **clients accessing data**.
60
+The following table summarizes the effect of each optimization on the CPU, RAM and Disk IO utilization in production.
61
49
-You can configure almost all aspects of data collection/retention, and certain aspects of clients accessing data. For
50
-example, you can't control how many users might be viewing a local Agent dashboard, [viewing an
51
-infrastructure](https://github.com/netdata/netdata/blob/master/docs/visualize/overview-infrastructure.md) in real-time with Netdata Cloud, or running [Metric
52
-Correlations](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/metric-correlations.md).
62
+Optimization | CPU | RAM | Disk IO
63
+-- | -- | -- |--
64
+[Use streaming and replication](#use-streaming-and-replication) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark:
65
+[Disable unneeded plugins or collectors](#disable-unneeded-plugins-or-collectors) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark:
66
+[Reduce data collection frequency](#reduce-collection-frequency) | :heavy_check_mark: | | :heavy_check_mark:
67
+[Change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) | | :heavy_check_mark: | :heavy_check_mark:
68
+[Use a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md) | | :heavy_check_mark: | :heavy_check_mark:
69
+[Disable machine learning](#disable-machine-learning) | :heavy_check_mark: | |
70
+[Use a reverse proxy](#run-netdata-behind-a-proxy) | :heavy_check_mark: | |
71
+[Disable/lower gzip compression for the agent dashboard](#disablelower-gzip-compression-for-the-dashboard) | :heavy_check_mark: | |
72
54
-The Netdata Agent runs with the lowest possible [process scheduling
55
-policy](https://github.com/netdata/netdata/blob/master/daemon/README.md#netdata-process-scheduling-policy), which is `nice 19`, and uses the `idle` process scheduler.
56
-Together, these settings ensure that the Agent only gets CPU resources when the node has CPU resources to space. If the
57
-node reaches 100% CPU utilization, the Agent is stopped first to ensure your applications get any available resources.
58
-In addition, under heavy load, collectors that require disk I/O may stop and show gaps in charts.
73
+## Use streaming and replication
74
+
75
+For all production environments, parent Netdata nodes outside the production infrastructure should be receiving all
76
+collected data from children Netdata nodes running on the production infrastructure, using [streaming and replication](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md).
77
+
78
+### Disable health checks on the child nodes
79
+
80
+When you set up streaming, we recommend you run your health checks on the parent. This saves resources on the children
81
+and makes it easier to configure or disable alerts and agent notifications.
82
+
83
+The parents by default run health checks for each child, as long as the child is connected (the details are in `stream.conf`).
84
+On the child nodes you should add to `netdata.conf` the following:
85
+
86
+```conf
87
+[health]
88
+ enabled = no
89
+```
90
60
-Let's walk through the best ways to improve the Netdata Agent's performance.
91
+### Use memory mode ram or save for the child nodes
92
+
93
+See [using a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md).
94
+
95
+## Disable unneeded plugins or collectors
96
+
97
+If you know that you don't need an [entire plugin or a specific
98
+collector](https://github.com/netdata/netdata/blob/master/collectors/README.md#collector-architecture-and-terminology), you can disable any of them.
99
+Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and does not consume system resources.
100
+You will only improve the Agent's performance by disabling plugins/collectors that are actively collecting metrics.
101
+
102
+Open `netdata.conf` and scroll down to the `[plugins]` section. To disable any plugin, uncomment it and set the value to
103
+`no`. For example, to explicitly keep the `proc` and `go.d` plugins enabled while disabling `python.d` and `charts.d`.
104
+
105
+```conf
106
+[plugins]
107
+ proc = yes
108
+ python.d = no
109
+ charts.d = no
110
+ go.d = yes
111
+```
112
+
113
+Disable specific collectors by opening their respective plugin configuration files, uncommenting the line for the
114
+collector, and setting its value to `no`.
115
+
116
+```bash
117
+sudo ./edit-config go.d.conf
118
+sudo ./edit-config python.d.conf
119
+sudo ./edit-config charts.d.conf
120
+```
121
+
122
+For example, to disable a few Python collectors:
123
+
124
+```conf
125
+modules:
126
+ apache: no
127
+ dockerd: no
128
+ fail2ban: no
129
+```
130
131
## Reduce collection frequency
132
@@ -105,46 +174,15 @@ run `sudo ./edit-config go.d/nginx.conf`:
174
update_every: 10
175
```
176
108
-## Disable unneeded plugins or collectors
109
-
110
-If you know that you don't need an [entire plugin or a specific
111
-collector](https://github.com/netdata/netdata/blob/master/collectors/README.md#collector-architecture-and-terminology), you can disable any of them.
112
-Keep in mind that if a plugin/collector has nothing to do, it simply shuts down and does not consume system resources.
113
-You will only improve the Agent's performance by disabling plugins/collectors that are actively collecting metrics.
114
-
115
-Open `netdata.conf` and scroll down to the `[plugins]` section. To disable any plugin, uncomment it and set the value to
116
-`no`. For example, to explicitly keep the `proc` and `go.d` plugins enabled while disabling `python.d` and `charts.d`.
117
-
118
-```conf
119
-[plugins]
120
- proc = yes
121
- python.d = no
122
- charts.d = no
123
- go.d = yes
124
-```
125
-
126
-Disable specific collectors by opening their respective plugin configuration files, uncommenting the line for the
127
-collector, and setting its value to `no`.
128
-
129
-```bash
130
-sudo ./edit-config go.d.conf
131
-sudo ./edit-config python.d.conf
132
-sudo ./edit-config charts.d.conf
133
-```
134
-
135
-For example, to disable a few Python collectors:
136
-
137
-```conf
138
-modules:
139
- apache: no
140
- dockerd: no
141
- fail2ban: no
142
-```
143
-
177
## Lower memory usage for metrics retention
178
179
See how to [change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md).
180
181
+## Use a different metric storage database
182
+
183
+Consider [using a different metric storage database](https://github.com/netdata/netdata/blob/master/database/README.md) when running Netdata on IoT devices,
184
+and for children in a parent-child set up based on [streaming and replication](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md).
185
+
186
## Disable machine learning
187
188
Automated anomaly detection may be a powerful tool, but we recommend it to only be enabled on Netdata parents
@@ -156,34 +194,12 @@ with the following:
194
enabled = no
195
```
196
159
-## Run Netdata behind Nginx
197
+## Run Netdata behind a proxy
198
161
-A dedicated web server like Nginx provides far more robustness than the Agent's internal [web server](https://github.com/netdata/netdata/blob/master/web/README.md).
199
+A dedicated web server like nginx provides more robustness than the Agent's internal [web server](https://github.com/netdata/netdata/blob/master/web/README.md).
200
Nginx can handle more concurrent connections, reuse idle connections, and use fast gzip compression to reduce payloads.
201
164
-For details on installing Nginx as a proxy for the local Agent dashboard, see our [Nginx
165
-doc](https://github.com/netdata/netdata/blob/master/docs/Running-behind-nginx.md).
166
-
167
-After you complete Nginx setup according to the doc linked above, we recommend setting `keepalive` to `1024`, and using
168
-gzip compression with the following options in the `location /` block:
169
-
170
-```conf
171
- location / {
172
- ...
173
- gzip on;
174
- gzip_proxied any;
175
- gzip_types *;
176
- }
177
-```
178
-
179
-Finally, edit `netdata.conf` with the following settings:
180
-
181
-```conf
182
-[global]
183
- bind socket to IP = 127.0.0.1
184
- disconnect idle web clients after seconds = 3600
185
- enable web responses gzip compression = no
186
-```
202
+For details on installing another web server as a proxy for the local Agent dashboard, see [reverse proxies](https://github.com/netdata/netdata/blob/master/docs/category-overview-pages/reverse-proxies.md).
203
204
## Disable/lower gzip compression for the dashboard
205
@@ -206,27 +222,3 @@ Or to lower the default compression level:
222
gzip compression level = 1
223
```
224
209
-## Disable logs
210
-
211
-If you installation is working correctly, and you're not actively auditing Netdata's logs, disable them in
212
-`netdata.conf`.
213
-
214
-```conf
215
-[logs]
216
- debug log = none
217
- error log = none
218
- access log = none
219
-```
220
-
221
-## Disable health checks
222
-
223
-If you are streaming metrics to parent nodes, we recommend you run your health checks on the parent, for all the metrics collected
224
-by the children nodes. This saves resources on the children and makes it easier to configure or disable alerts and agent notifications.
225
-
226
-The parents by default run health checks for each child, as long as it is connected (the details are in `stream.conf`).
227
-On the child nodes you should add to `netdata.conf` the following:
228
-
229
-```conf
230
-[health]
231
- enabled = no
232
-```