Add links to promote database engine calculator (#9067)
* Add links and calculator .md * Refactor to Learn * Remove erroneous paragraph
Joel Hans committed
May 18, 2020 at 13:10 UTC
223724f2186f51ea464be04df997ff2e934c8352
4 files changed
+19
-19
database/engine/README.md
+8
-2
@@ -1,8 +1,7 @@
1
<!--
2
----
2
title: "Database engine"
3
+description: "The highly-efficient database engine stores per-second metrics in RAM and then spills historical metrics to disk long-term storage."
4
custom_edit_url: https://github.com/netdata/netdata/edit/master/database/engine/README.md
5
----
5
-->
6
7
# Database engine
@@ -63,6 +62,10 @@ All DB engine instances, for localhost and all other streaming recipient nodes i
62
The above values are the default and minimum values for Page Cache size and DB engine disk space quota. Both numbers are
63
in **MiB**. All DB engine instances will allocate the configured resources separately.
64
65
+[**See our database engine calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to help you
66
+correctly set `dbengine disk space` based on your needs. The calculator gives an accurate estimate based on how many
67
+slave nodes you have, how many metrics your Agent collects, and more.
68
+
69
The `page cache size` option determines the amount of RAM in **MiB** that is dedicated to caching Netdata metric values
70
themselves as far as queries are concerned. The total page cache size will be greater since data collection itself will
71
consume additional memory as is described in the [Memory requirements](#memory-requirements) section.
@@ -108,6 +111,9 @@ and streaming recipient nodes):
111
112
An important observation is that RAM usage depends on both the `page cache size` and the `dbengine disk space` options.
113
114
+You can use our [database engine calculator](https://learn.netdata.cloud/docs/agent/database/calculator) to
115
+validate the memory requirements for your particular system(s) and configuration.
116
+
117
## File descriptor requirements
118
119
The Database Engine may keep a **significant** amount of files open per instance (e.g. per streaming slave or master
docs/getting-started.md
+3
-2
@@ -67,8 +67,9 @@ configuration keeps RAM usage low while allowing for long-term, on-disk metrics
67
You can tweak this custom _database engine_ to store a much larger dataset than your system's available RAM,
68
particularly if you allow Netdata to use slightly more RAM and disk space than the default configuration.
69
70
-Read our tutorial, [**Changing how long Netdata stores metrics**](/docs/tutorials/longer-metrics-storage.md), to learn
71
-more.
70
+Read our guide on [changing how long Netdata stores metrics](/docs/tutorials/longer-metrics-storage.md) to learn more
71
+and use our [database engine calculator](https://learn.netdata.cloud/docs/agent/database/calculator) to figure
72
+out the exact settings you'll need to store historical metrics right in the Agent's database.
73
74
**What's next?**:
75
docs/step-by-step/step-09.md
+3
-3
@@ -54,9 +54,9 @@ the database engine to use. The higher those values, the more metrics Netdata wi
54
512, respectively, the database engine should store about four day's worth of data on a system collecting 2,000 metrics
55
every second.
56
57
-> Before you make changes, we recommended you read up on the [database
58
-> engine's](/database/engine/README.md#memory-requirements) to ensure you don't overwhelm your system. Out of memory
59
-> errors are no fun!
57
+[**See our database engine calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to help you
58
+correctly set `dbengine disk space` based on your needs. The calculator gives an accurate estimate based on how many
59
+slave nodes you have, how many metrics your Agent collects, and more.
60
61
```conf
62
[global]
docs/tutorials/longer-metrics-storage.md
+5
-12
@@ -1,8 +1,7 @@
1
<!--
2
----
2
title: "Change how long Netdata stores metrics"
3
+description: "With a single configuration change, the Netdata Agent can store days, weeks, or months of metrics at its famous per-second granularity."
4
custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/tutorials/longer-metrics-storage.md
5
----
5
-->
6
7
# Change how long Netdata stores metrics
@@ -37,9 +36,6 @@ using, check out your `netdata.conf` file and look for the `memory mode` setting
36
If `memory mode` is set to anything but `dbengine`, change it and restart Netdata using the standard command for
37
restarting services on your system. You're now using the database engine!
38
40
-> Learn more about how we implemented the database engine, and our vision for its future, on our blog: [_How and why
41
-> we're bringing long-term storage to Netdata_](https://blog.netdata.cloud/posts/db-engine/).
42
-
39
What makes the database engine efficient? While it's structured like a traditional database, the database engine splits
40
data between RAM and disk. The database engine caches and indexes data on RAM to keep memory usage low, and then
41
compresses older metrics onto disk for long-term storage.
@@ -58,14 +54,11 @@ size` and `dbengine disk space`.
54
55
`page cache size` sets the maximum amount of RAM (in MiB) the database engine will use for caching and indexing.
56
`dbengine disk space` sets the maximum disk space (again, in MiB) the database engine will use for storing compressed
61
-metrics.
62
-
63
-Based on our testing, these default settings will retain about a day's worth of metrics when Netdata collects roughly
64
-4,000 metrics every second. If you increase either `page cache size` or `dbengine disk space`, Netdata will retain even
65
-more historical metrics.
57
+metrics. The default settings retain about two day's worth of metris on a system collecting 2,000 metrics every second.
58
67
-But before you change these options too dramatically, read up on the [database engine's memory
68
-footprint](/database/engine/README.md#memory-requirements).
59
+[**See our database engine calculator**](https://learn.netdata.cloud/docs/agent/database/calculator) to help you
60
+correctly set `dbengine disk space` based on your needs. The calculator gives an accurate estimate based on how many
61
+slave nodes you have, how many metrics your Agent collects, and more.
62
63
With the database engine active, you can back up your `/var/cache/netdata/dbengine/` folder to another location for
64
redundancy.