Improve dbengine docs and add new multihost setting (#9817)
* Clean up dbengine doc and metrics guide * Tweaks to the text * Fixes for Markos
Joel Hans committed
Aug 26, 2020 at 07:21 UTC
a305ee79d6342b8ece27bf0b43f8d88025769353
2 files changed
+33
-35
database/engine/README.md
+26
-29
@@ -1,6 +1,6 @@
1
<!--
2
title: "Database engine"
3
-description: "The highly-efficient database engine stores per-second metrics in RAM and then spills historical metrics to disk long-term storage."
3
+description: "Netdata's highly-efficient database engine use both RAM and disk for distributed, long-term storage of per-second metrics."
4
custom_edit_url: https://github.com/netdata/netdata/edit/master/database/engine/README.md
5
-->
6
@@ -26,8 +26,8 @@ To use the database engine, open `netdata.conf` and set `memory mode` to `dbengi
26
memory mode = dbengine
27
```
28
29
-To configure the database engine, look for the `page cache size` and `dbengine disk space` settings in the `[global]`
30
-section of your `netdata.conf`. The Agent ignores the `history` setting when using the database engine.
29
+To configure the database engine, look for the `page cache size` and `dbengine multihost disk space` settings in the
30
+`[global]` section of your `netdata.conf`. The Agent ignores the `history` setting when using the database engine.
31
32
```conf
33
[global]
@@ -43,51 +43,48 @@ actual page cache size will be slightly larger than this figure—see the [memor
43
section for details.
44
45
The `dbengine multihost disk space` option determines the amount of disk space in **MiB** that is dedicated to storing
46
-Netdata metric values and all related metadata describing them.
46
+Netdata metric values and all related metadata describing them. You can use the [**database engine
47
+calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to correctly set `dbengine multihost disk
48
+space` based on your metrics retention policy. The calculator gives an accurate estimate based on how many child nodes
49
+you have, how many metrics your Agent collects, and more.
50
51
### Legacy configuration
52
53
The deprecated `dbengine disk space` option determines the amount of disk space in **MiB** that is dedicated to storing
51
-Netdata metric values per legacy database engine instance (see [below](#Streaming-metrics-to-the-database-engine)).
54
+Netdata metric values per legacy database engine instance (see [details on the legacy mode](#legacy-mode) below).
55
56
```conf
57
[global]
58
dbengine disk space = 256
59
```
60
58
-Use the [**database engine calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to correctly set
59
-`dbengine disk space`(**deprecated**) based on your needs. The calculator gives an accurate estimate based on how many
60
-child nodes you have, how many metrics your Agent collects, and more.
61
-
61
### Streaming metrics to the database engine
62
64
-##### Legacy mode
65
-
66
-When streaming metrics, the Agent on the parent node used to create one instance (legacy, version <= 1.23.2) of the
67
-database engine for itself, and another instance for every child node it receives metrics from. If you had four
68
-streaming nodes, you would have five instances in total (`1 parent + 4 child nodes = 5 instances`).
63
+When using the multihost database engine, all parent and child nodes share the same `page cache size` and `dbengine
64
+multihost disk space` in a single dbengine instance. The [**database engine
65
+calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) helps you properly set `page cache size` and
66
+`dbengine multihost disk space` on your parent node to allocate enough resources based on your metrics retention policy
67
+and how many child nodes you have.
68
70
-The Agent allocated resources for each instance separately using the `dbengine disk space`(**deprecated**) setting. If
71
-`dbengine disk space`(**deprecated**) is set to the default `256`, each instance is given 256 MiB in disk space, which
72
-means the total disk space required to store all instances is, roughly, `256 MiB * 1 parent * 4 child nodes = 1280 MiB`.
69
+#### Legacy mode
70
74
-See the [database engine calculator](https://learn.netdata.cloud/docs/agent/database/calculator) to help you correctly
75
-set `dbengine disk space`(**deprecated**) and understand the total disk space required based on your streaming setup.
71
+_For Netdata Agents earlier than v1.23.2_, the Agent on the parent node uses one dbengine instance for itself, and
72
+another instance for every child node it receives metrics from. If you had four streaming nodes, you would have five
73
+instances in total (`1 parent + 4 child nodes = 5 instances`).
74
77
-##### Multi host DB mode
78
-
79
-In the newer agent versions the parent and child nodes all share `page cache size` and `dbengine multihost disk space`
80
-in a single dbengine multi-host instance.
75
+The Agent allocates resources for each instance separately using the `dbengine disk space` (**deprecated**) setting. If
76
+`dbengine disk space`(**deprecated**) is set to the default `256`, each instance is given 256 MiB in disk space, which
77
+means the total disk space required to store all instances is, roughly, `256 MiB * 1 parent * 4 child nodes = 1280 MiB`.
78
82
-##### Backward compatibility
79
+#### Backward compatibility
80
81
All existing metrics belonging to child nodes are automatically converted to legacy dbengine instances and the localhost
85
-metrics are transferred to the multi-host dbengine instance.
82
+metrics are transferred to the multihost dbengine instance.
83
87
-All new child nodes are automatically transferred to the mult-host dbengine instance and share its page cache and disk
88
-space. If you want to migrate a child node from its legacy dbengine instance to the multi-host dbengine instance you
89
-must delete the instance's directory located in `/var/cache/netdata/MACHINE_GUID/dbengine` after stopping the netdata
90
-agent.
84
+All new child nodes are automatically transferred to the multihost dbengine instance and share its page cache and disk
85
+space. If you want to migrate a child node from its legacy dbengine instance to the multihost dbengine instance, you
86
+must delete the instance's directory, which is located in `/var/cache/netdata/MACHINE_GUID/dbengine`, after stopping the
87
+Agent.
88
89
##### Information
90
docs/guides/longer-metrics-storage.md
+7
-6
@@ -43,21 +43,22 @@ When the Netdata dashboard queries for historical metrics, the database engine w
43
return relevant metrics for visualization in charts.
44
45
Now, given that the database engine uses _both_ RAM and disk, there are two other settings to consider: `page cache
46
-size` and `dbengine disk space`.
46
+size` and `dbengine multihost disk space`.
47
48
```conf
49
[global]
50
page cache size = 32
51
- dbengine disk space = 256
51
+ dbengine multihost disk space = 256
52
```
53
54
`page cache size` sets the maximum amount of RAM (in MiB) the database engine will use for caching and indexing.
55
-`dbengine disk space` sets the maximum disk space (again, in MiB) the database engine will use for storing compressed
56
-metrics. The default settings retain about two day's worth of metris on a system collecting 2,000 metrics every second.
55
+`dbengine multihost disk space` sets the maximum disk space (again, in MiB) the database engine will use for storing
56
+compressed metrics. The default settings retain about two day's worth of metris on a system collecting 2,000 metrics
57
+every second.
58
59
[**See our database engine calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to help you
59
-correctly set `dbengine disk space` based on your needs. The calculator gives an accurate estimate based on how many
60
-child nodes you have, how many metrics your Agent collects, and more.
60
+correctly set `dbengine multihost disk space` based on your needs. The calculator gives an accurate estimate based on
61
+how many child nodes you have, how many metrics your Agent collects, and more.
62
63
With the database engine active, you can back up your `/var/cache/netdata/dbengine/` folder to another location for
64
redundancy.