Guides and collectors reorg and cleanup part 1 (#14507)
* rm github star important * Remove some guides and rework the data collection docs * link fixes on collectors/README.md * link fixes on collectors/REFERENCE.md * Fix codacy * More codacy list formatting
Chris Akritidis committed
Feb 12, 2023 at 06:36 UTC
8b62021d1c6436a5d74eb797a719ccb152f79ee4
13 files changed
+139
-1209
collectors/README.md
+43
-25
@@ -10,45 +10,63 @@ learn_rel_path: "References/Collectors"
10
11
# Collecting metrics
12
13
-Netdata can collect metrics from hundreds of different sources, be they internal data created by the system itself, or
14
-external data created by services or applications. To see _all_ of the sources Netdata collects from, view our
15
-[list of supported collectors](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md).
13
+When Netdata starts, and with zero configuration, it auto-detects thousands of data sources and immediately collects
14
+per-second metrics.
15
17
-There are two essential points to understand about how collecting metrics works in Netdata:
16
+Netdata can immediately collect metrics from these endpoints thanks to 300+ **collectors**, which all come pre-installed
17
+when you [install Netdata](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md).
18
19
-- All collectors are **installed by default** with every installation of Netdata. You do not need to install
20
- collectors manually to collect metrics from new sources.
21
-- Upon startup, Netdata will **auto-detect** any application or service that has a
22
- [collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md), as long as both the collector
23
- and the app/service are configured correctly.
19
+All collectors are **installed by default** with every installation of Netdata. You do not need to install
20
+collectors manually to collect metrics from new sources. See our
21
+[list of supported collectors](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md).
22
25
-Most users will want to enable a new Netdata collector for their app/service. For those details, see
23
+Upon startup, Netdata will **auto-detect** any application or service that has a collector, as long as both the collector
24
+and the app/service are configured correctly. If you don't see charts for your application, see
25
our [collectors' configuration reference](https://github.com/netdata/netdata/blob/master/collectors/REFERENCE.md).
26
28
-## Take your next steps with collectors
27
+## How Netdata's metrics collectors work
28
+
29
+Every collector has two primary jobs:
30
+
31
+- Look for exposed metrics at a pre- or user-defined endpoint.
32
+- Gather exposed metrics and use additional logic to build meaningful, interactive visualizations.
33
30
-[Supported collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md)
34
+If the collector finds compatible metrics exposed on the configured endpoint, it begins a per-second collection job. The
35
+Netdata Agent gathers these metrics, sends them to the
36
+[database engine for storage](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md)
37
+, and immediately
38
+[visualizes them meaningfully](https://github.com/netdata/netdata/blob/master/docs/visualize/interact-dashboards-charts.md)
39
+on dashboards.
40
32
-[Collectors configuration reference](https://github.com/netdata/netdata/blob/master/collectors/REFERENCE.md)
41
+Each collector comes with a pre-defined configuration that matches the default setup for that application. This endpoint
42
+can be a URL and port, a socket, a file, a web page, and more. The endpoint is user-configurable, as are many other
43
+specifics of what a given collector does.
44
34
-## Guides
45
+## Collector architecture and terminology
46
36
-[Monitor Nginx or Apache web server log files with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/collect-apache-nginx-web-logs.md)
47
+- **Collectors** are the processes/programs that actually gather metrics from various sources.
48
38
-[Monitor CockroachDB metrics with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/monitor-cockroachdb.md)
49
+- **Plugins** help manage all the independent data collection processes in a variety of programming languages, based on
50
+ their purpose and performance requirements. There are three types of plugins:
51
40
-[Monitor Unbound DNS servers with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/collect-unbound-metrics.md)
52
+ - **Internal** plugins organize collectors that gather metrics from `/proc`, `/sys` and other Linux kernel sources.
53
+ They are written in `C`, and run as threads within the Netdata daemon.
54
42
-[Monitor a Hadoop cluster with Netdata](https://github.com/netdata/netdata/blob/master/docs/guides/monitor-hadoop-cluster.md)
55
+ - **External** plugins organize collectors that gather metrics from external processes, such as a MySQL database or
56
+ Nginx web server. They can be written in any language, and the `netdata` daemon spawns them as long-running
57
+ independent processes. They communicate with the daemon via pipes. All external plugins are managed by
58
+ [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md), which provides additional management options.
59
44
-## Related features
60
+- **Orchestrators** are external plugins that run and manage one or more modules. They run as independent processes.
61
+ The Go orchestrator is in active development.
62
46
-**[Dashboards](https://github.com/netdata/netdata/blob/master/web/README.md)**: Visualize your newly-collect metrics in
47
-real-time using Netdata's [built-in dashboard](https://github.com/netdata/netdata/blob/master/web/gui/README.md).
63
+ - [go.d.plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md): An orchestrator for data
64
+ collection modules written in `go`.
65
49
-**[Exporting](https://github.com/netdata/netdata/blob/master/exporting/README.md)**: Extend our
50
-built-in [database engine](https://github.com/netdata/netdata/blob/master/database/engine/README.md), which supports
51
-long-term metrics storage, by archiving metrics to external databases like Graphite, Prometheus, MongoDB, TimescaleDB,
52
-and more. It can export metrics to multiple databases simultaneously.
66
+ - [python.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md):
67
+ An orchestrator for data collection modules written in `python` v2/v3.
68
69
+ - [charts.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md):
70
+ An orchestrator for data collection modules written in`bash` v4+.
71
72
+- **Modules** are the individual programs controlled by an orchestrator to collect data from a specific application, or type of endpoint.
collectors/REFERENCE.md
+91
-116
@@ -9,76 +9,24 @@ learn_rel_path: "Setup"
9
10
# Collectors configuration reference
11
12
-Welcome to the collector configuration reference guide.
12
+The list of supported collectors can be found in [the documentation](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md),
13
+and on [our website](https://www.netdata.cloud/integrations). The documentation of each collector provides all the
14
+necessary configuration options and prerequisites for that collector. In most cases, either the charts are automatically generated
15
+without any configuration, or you just fulfil those prerequisites and [configure the collector](#configure-a-collector).
16
14
-This guide contains detailed information about enabling/disabling plugins or modules, in addition a quick reference to
15
-the internal plugins API.
17
+If the application you are interested in monitoring is not listed in our integrations,
18
+look at our [generic Prometheus collector](https://github.com/netdata/go.d.plugin/blob/master/modules/prometheus/README.md)
19
+or [write a custom collector in Go](https://github.com/netdata/go.d.plugin/blob/master/docs/how-to-write-a-module.md).
20
17
-## Netdata's collector architecture
21
+If we do support your collector but the charts described in the documentation don't appear on your dashboard, the reason will
22
+be one of the following:
23
19
-Netdata has an intricate system for organizing and managing its collectors. **Collectors** are the processes/programs
20
-that actually gather metrics from various sources. Collectors are organized by **plugins**, which help manage all the
21
-independent processes in a variety of programming languages based on their purpose and performance requirements.
22
-**Modules** are a type of collector, used primarily to connect to external applications, such as an Nginx web server or
23
-MySQL database, among many others.
24
+- The entire data collection plugin is disabled by default. Read how to [enable and disable plugins](#enable-and-disable-plugins)
25
25
-For most users, enabling individual collectors for the application/service you're interested in is far more important
26
-than knowing which plugin it uses. See our [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) to see whether your favorite app/service has
27
-a collector, and then read the documentation for that specific collector to figure out how to enable it.
26
+- The data collection plugin is enabled, but a specific data collection module is disabled. Read how to
27
+ [enable and disable a specific collection module](#enable-and-disable-a-specific-collection-module).
28
29
-There are three types of plugins:
30
-
31
-- **Internal** plugins organize collectors that gather metrics from `/proc`, `/sys` and other Linux kernel sources.
32
- They are written in `C`, and run as threads within the Netdata daemon.
33
-- **External** plugins organize collectors that gather metrics from external processes, such as a MySQL database or
34
- Nginx web server. They can be written in any language, and the `netdata` daemon spawns them as long-running
35
- independent processes. They communicate with the daemon via pipes.
36
-- **Plugin orchestrators**, which are external plugins that instead support a number of **modules**. Modules are a
37
- type of collector. We have a few plugin orchestrators available for those who want to develop their own collectors,
38
- but focus most of our efforts on the [Go plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md).
39
-
40
-## Enable, configure, and disable modules
41
-
42
-Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
43
-the default settings.
44
-
45
-However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
46
-allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
47
-metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
48
-
49
-
50
-## Troubleshoot a collector
51
-
52
-First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
53
-on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
54
-switch to the `netdata` user.
55
-
56
-```bash
57
-cd /usr/libexec/netdata/plugins.d/
58
-sudo su -s /bin/bash netdata
59
-```
60
-
61
-The next step is based on the collector's orchestrator. You can figure out which orchestrator the collector uses by
62
-
63
-uses either
64
-by viewing the [collectors list](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) and referencing the _configuration file_ field. For example, if that
65
-field contains `go.d`, that collector uses the Go orchestrator.
66
-
67
-```bash
68
-# Go orchestrator (go.d.plugin)
69
-./go.d.plugin -d -m <MODULE_NAME>
70
-
71
-# Python orchestrator (python.d.plugin)
72
-./python.d.plugin <MODULE_NAME> debug trace
73
-
74
-# Bash orchestrator (bash.d.plugin)
75
-./charts.d.plugin debug 1 <MODULE_NAME>
76
-```
77
-
78
-The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
79
-enable the collector using the details from this output, feel free to [create an issue on our
80
-GitHub](https://github.com/netdata/netdata/issues/new?assignees=&labels=bug%2Cneeds+triage&template=BUG_REPORT.yml) to get some
81
-help from our collectors experts.
29
+- Autodetection failed. Read how to [configure](#configure-a-collector) and [troubleshoot](#troubleshoot-a-collector) a collector.
30
31
## Enable and disable plugins
32
@@ -88,87 +36,114 @@ This section features a list of Netdata's plugins, with a boolean setting to ena
36
37
```conf
38
[plugins]
91
- # proc = yes
92
- # diskspace = yes
39
# timex = yes
94
- # cgroups = yes
95
- # tc = yes
40
# idlejitter = yes
41
+ # netdata monitoring = yes
42
+ # tc = yes
43
+ # diskspace = yes
44
+ # proc = yes
45
+ # cgroups = yes
46
# enable running new plugins = yes
47
# check for new plugins every = 60
48
# slabinfo = no
100
- # ioping = yes
49
# python.d = yes
50
+ # perf = yes
51
+ # ioping = yes
52
+ # fping = yes
53
+ # nfacct = yes
54
# go.d = yes
55
# apps = yes
104
- # perf = yes
56
+ # ebpf = yes
57
# charts.d = yes
58
+ # statsd = yes
59
```
60
61
By default, most plugins are enabled, so you don't need to enable them explicitly to use their collectors. To enable or
62
disable any specific plugin, remove the comment (`#`) and change the boolean setting to `yes` or `no`.
63
111
-All **external plugins** are managed by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md), which provides additional management options.
64
+## Enable and disable a specific collection module
65
113
-## Internal plugins
66
+You can enable/disable of the collection modules supported by `go.d`, `python.d` or `charts.d` individually, using the
67
+configuration file of that orchestrator. For example, you can change the behavior of the Go orchestrator, or any of its
68
+collectors, by editing `go.d.conf`.
69
115
-Each of the internal plugins runs as a thread inside the `netdata` daemon. Once this thread has started, the plugin may
116
-spawn additional threads according to its design.
70
+Use `edit-config` from your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory)
71
+to open the orchestrator primary configuration file:
72
118
-### Internal plugins API
73
+```bash
74
+cd /etc/netdata
75
+sudo ./edit-config go.d.conf
76
+```
77
120
-The internal data collection API consists of the following calls:
78
+Within this file, you can either disable the orchestrator entirely (`enabled: yes`), or find a specific collector and
79
+enable/disable it with `yes` and `no` settings. Uncomment any line you change to ensure the Netdata daemon reads it on
80
+start.
81
122
-```c
123
-collect_data() {
124
- // collect data here (one iteration)
82
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
83
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
84
126
- collected_number collected_value = collect_a_value();
85
+## Configure a collector
86
128
- // give the metrics to Netdata
87
+Most collector modules come with **auto-detection**, configured to work out-of-the-box on popular operating systems with
88
+the default settings.
89
130
- static RRDSET *st = NULL; // the chart
131
- static RRDDIM *rd = NULL; // a dimension attached to this chart
90
+However, there are cases that auto-detection fails. Usually, the reason is that the applications to be monitored do not
91
+allow Netdata to connect. In most of the cases, allowing the user `netdata` from `localhost` to connect and collect
92
+metrics, will automatically enable data collection for the application in question (it will require a Netdata restart).
93
133
- if(unlikely(!st)) {
134
- // we haven't created this chart before
135
- // create it now
136
- st = rrdset_create_localhost(
137
- "type"
138
- , "id"
139
- , "name"
140
- , "family"
141
- , "context"
142
- , "Chart Title"
143
- , "units"
144
- , "plugin-name"
145
- , "module-name"
146
- , priority
147
- , update_every
148
- , chart_type
149
- );
94
+When Netdata starts up, each collector searches for exposed metrics on the default endpoint established by that service
95
+or application's standard installation procedure. For example,
96
+the [Nginx collector](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md) searches at
97
+`http://127.0.0.1/stub_status` for exposed metrics in the correct format. If an Nginx web server is running and exposes
98
+metrics on that endpoint, the collector begins gathering them.
99
151
- // attach a metric to it
152
- rd = rrddim_add(st, "id", "name", multiplier, divider, algorithm);
153
- }
100
+However, not every node or infrastructure uses standard ports, paths, files, or naming conventions. You may need to
101
+enable or configure a collector to gather all available metrics from your systems, containers, or applications.
102
155
- // give the collected value(s) to the chart
156
- rrddim_set_by_pointer(st, rd, collected_value);
103
+First, [find the collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) you want to edit
104
+and open its documentation. Some software has collectors written in multiple languages. In these cases, you should always
105
+pick the collector written in Go.
106
158
- // signal Netdata we are done with this iteration
159
- rrdset_done(st);
160
-}
107
+Use `edit-config` from your
108
+[Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory)
109
+to open a collector's configuration file. For example, edit the Nginx collector with the following:
110
+
111
+```bash
112
+./edit-config go.d/nginx.conf
113
```
114
163
-Of course, Netdata has a lot of libraries to help you also in collecting the metrics. The best way to find your way
164
-through this, is to examine what other similar plugins do.
115
+Each configuration file describes every available option and offers examples to help you tweak Netdata's settings
116
+according to your needs. In addition, every collector's documentation shows the exact command you need to run to
117
+configure that collector. Uncomment any line you change to ensure the collector's orchestrator or the Netdata daemon
118
+read it on start.
119
+
120
+After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
121
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
122
+
123
+## Troubleshoot a collector
124
166
-## External Plugins
125
+First, navigate to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case
126
+on your system, open `netdata.conf` and look for the setting `plugins directory`. Once you're in the plugins directory,
127
+switch to the `netdata` user.
128
168
-**External plugins** use the API and are managed
169
-by [plugins.d](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
129
+```bash
130
+cd /usr/libexec/netdata/plugins.d/
131
+sudo su -s /bin/bash netdata
132
+```
133
171
-## Write a custom collector
134
+The next step is based on the collector's orchestrator.
135
173
-You can add custom collectors by following the [external plugins documentation](https://github.com/netdata/netdata/blob/master/collectors/plugins.d/README.md).
136
+```bash
137
+# Go orchestrator (go.d.plugin)
138
+./go.d.plugin -d -m <MODULE_NAME>
139
140
+# Python orchestrator (python.d.plugin)
141
+./python.d.plugin <MODULE_NAME> debug trace
142
+
143
+# Bash orchestrator (bash.d.plugin)
144
+./charts.d.plugin debug 1 <MODULE_NAME>
145
+```
146
+
147
+The output from the relevant command will provide valuable troubleshooting information. If you can't figure out how to
148
+enable the collector using the details from this output, feel free to [join our Discord server](https://discord.com/invite/mPZ6WZKKG2),
149
+to get help from our experts.
collectors/charts.d.plugin/README.md
+1
-1
@@ -4,7 +4,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/char
4
sidebar_label: "charts.d.plugin"
5
learn_status: "Published"
6
learn_topic_type: "Tasks"
7
-learn_rel_path: "Developers/Collectors"
7
+learn_rel_path: "Developers/External plugins"
8
-->
9
10
# charts.d.plugin
collectors/plugins.d/README.md
+3
-3
@@ -1,13 +1,13 @@
1
<!--
2
-title: "External plugins overview"
2
+title: "External plugins"
3
custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/plugins.d/README.md"
4
-sidebar_label: "External plugins overview"
4
+sidebar_label: "External plugins"
5
learn_status: "Published"
6
learn_topic_type: "References"
7
learn_rel_path: "Developers"
8
-->
9
10
-# External plugins overview
10
+# External plugins
11
12
`plugins.d` is the Netdata internal plugin that collects metrics
13
from external processes, thus allowing Netdata to use **external plugins**.
collectors/python.d.plugin/README.md
+1
-1
@@ -4,7 +4,7 @@ custom_edit_url: "https://github.com/netdata/netdata/edit/master/collectors/pyth
4
sidebar_label: "python.d.plugin"
5
learn_status: "Published"
6
learn_topic_type: "Tasks"
7
-learn_rel_path: "Developers/Collectors"
7
+learn_rel_path: "Developers/External plugins"
8
-->
9
10
# python.d.plugin
docs/a-github-star-is-important.md
deleted
-28
@@ -1,28 +0,0 @@
1
-<!--
2
-title: "A GitHub star is important"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/a-github-star-is-important.md
4
-sidebar_label: "A GitHub star is important"
5
-learn_status: "Published"
6
-learn_topic_type: "References"
7
-learn_rel_path: "Misc"
8
--->
9
-
10
-# A GitHub star is important
11
-
12
-**GitHub stars** allow Netdata to expand its reach, its community, especially attract people with skills willing to
13
-contribute to it.
14
-
15
-Compared to its first release, Netdata is now **twice as fast**, has all its bugs settled and a lot more functionality.
16
-This happened because a lot of people find it useful, use it daily at home and work, **rely on it** and **contribute to
17
-it**.
18
-
19
-**GitHub stars** also **motivate** us. They state that you find our work **useful**. They give us strength to continue,
20
-to work **harder** to make it even **better**.
21
-
22
-So, give Netdata a **GitHub star**, at the top right of this page.
23
-
24
-Thank you!
25
-
26
-Costa Tsaousis
27
-
28
-
docs/collect/enable-configure.md
deleted
-72
@@ -1,72 +0,0 @@
1
-<!--
2
-title: "Enable or configure a collector"
3
-description: "Every collector is highly configurable, allowing them to collect metrics from any node and any infrastructure."
4
-custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/collect/enable-configure.md"
5
-sidebar_label: "Enable or configure a collector"
6
-learn_status: "Published"
7
-learn_topic_type: "Tasks"
8
-learn_rel_path: "Setup"
9
--->
10
-
11
-# Enable or configure a collector
12
-
13
-When Netdata starts up, each collector searches for exposed metrics on the default endpoint established by that service
14
-or application's standard installation procedure. For example, the [Nginx
15
-collector](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md) searches at
16
-`http://127.0.0.1/stub_status` for exposed metrics in the correct format. If an Nginx web server is running and exposes
17
-metrics on that endpoint, the collector begins gathering them.
18
-
19
-However, not every node or infrastructure uses standard ports, paths, files, or naming conventions. You may need to
20
-enable or configure a collector to gather all available metrics from your systems, containers, or applications.
21
-
22
-## Enable a collector or its orchestrator
23
-
24
-You can enable/disable collectors individually, or enable/disable entire orchestrators, using their configuration files.
25
-For example, you can change the behavior of the Go orchestrator, or any of its collectors, by editing `go.d.conf`.
26
-
27
-Use `edit-config` from your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory) to open
28
-the orchestrator primary configuration file:
29
-
30
-```bash
31
-cd /etc/netdata
32
-sudo ./edit-config go.d.conf
33
-```
34
-
35
-Within this file, you can either disable the orchestrator entirely (`enabled: yes`), or find a specific collector and
36
-enable/disable it with `yes` and `no` settings. Uncomment any line you change to ensure the Netdata daemon reads it on
37
-start.
38
-
39
-After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
40
-method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
41
-
42
-## Configure a collector
43
-
44
-First, [find the collector](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) you want to edit and open its documentation. Some software has
45
-collectors written in multiple languages. In these cases, you should always pick the collector written in Go.
46
-
47
-Use `edit-config` from your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory) to open a
48
-collector's configuration file. For example, edit the Nginx collector with the following:
49
-
50
-```bash
51
-./edit-config go.d/nginx.conf
52
-```
53
-
54
-Each configuration file describes every available option and offers examples to help you tweak Netdata's settings
55
-according to your needs. In addition, every collector's documentation shows the exact command you need to run to
56
-configure that collector. Uncomment any line you change to ensure the collector's orchestrator or the Netdata daemon
57
-read it on start.
58
-
59
-After you make your changes, restart the Agent with `sudo systemctl restart netdata`, or the [appropriate
60
-method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
61
-
62
-## What's next?
63
-
64
-Read high-level overviews on how Netdata collects [system metrics](https://github.com/netdata/netdata/blob/master/docs/collect/system-metrics.md), [container
65
-metrics](https://github.com/netdata/netdata/blob/master/docs/collect/container-metrics.md), and [application metrics](https://github.com/netdata/netdata/blob/master/docs/collect/application-metrics.md).
66
-
67
-If you're already collecting all metrics from your systems, containers, and applications, it's time to move into
68
-Netdata's visualization features. [See an overview of your infrastructure](https://github.com/netdata/netdata/blob/master/docs/visualize/overview-infrastructure.md)
69
-using Netdata Cloud, or learn how to [interact with dashboards and
70
-charts](https://github.com/netdata/netdata/blob/master/docs/visualize/interact-dashboards-charts.md).
71
-
72
-
docs/collect/how-collectors-work.md
deleted
-82
@@ -1,82 +0,0 @@
1
-<!--
2
-title: "How Netdata's metrics collectors work"
3
-description: "When Netdata starts, and with zero configuration, it auto-detects thousands of data sources and immediately collects per-second metrics."
4
-custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/collect/how-collectors-work.md"
5
-sidebar_label: "How Netdata's metrics collectors work"
6
-learn_status: "Published"
7
-learn_topic_type: "Concepts"
8
-learn_rel_path: "Concepts"
9
--->
10
-
11
-# How Netdata's metrics collectors work
12
-
13
-When Netdata starts, and with zero configuration, it auto-detects thousands of data sources and immediately collects
14
-per-second metrics.
15
-
16
-Netdata can immediately collect metrics from these endpoints thanks to 300+ **collectors**, which all come pre-installed
17
-when you [install Netdata](https://github.com/netdata/netdata/blob/master/docs/get-started.mdx).
18
-
19
-Every collector has two primary jobs:
20
-
21
-- Look for exposed metrics at a pre- or user-defined endpoint.
22
-- Gather exposed metrics and use additional logic to build meaningful, interactive visualizations.
23
-
24
-If the collector finds compatible metrics exposed on the configured endpoint, it begins a per-second collection job. The
25
-Netdata Agent gathers these metrics, sends them to the [database engine for
26
-storage](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md), and immediately [visualizes them
27
-meaningfully](https://github.com/netdata/netdata/blob/master/docs/visualize/interact-dashboards-charts.md) on dashboards.
28
-
29
-Each collector comes with a pre-defined configuration that matches the default setup for that application. This endpoint
30
-can be a URL and port, a socket, a file, a web page, and more.
31
-
32
-For example, the [Nginx collector](https://github.com/netdata/go.d.plugin/blob/master/modules/nginx/README.md) searches
33
-at `http://127.0.0.1/stub_status`, which is the default endpoint for exposing Nginx metrics. The [web log collector for
34
-Nginx or Apache](https://github.com/netdata/go.d.plugin/blob/master/README.mdmodules/weblog) searches at
35
-`/var/log/nginx/access.log` and `/var/log/apache2/access.log`, respectively, both of which are standard locations for
36
-access log files on Linux systems.
37
-
38
-The endpoint is user-configurable, as are many other specifics of what a given collector does.
39
-
40
-## What can Netdata collect?
41
-
42
-To quickly find your answer, see our [list of supported collectors](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md).
43
-
44
-Generally, Netdata's collectors can be grouped into three types:
45
-
46
-- [Systems](https://github.com/netdata/netdata/blob/master/docs/collect/system-metrics.md): Monitor CPU, memory, disk, networking, systemd, eBPF, and much more.
47
- Every metric exposed by `/proc`, `/sys`, and other Linux kernel sources.
48
-- [Containers](https://github.com/netdata/netdata/blob/master/docs/collect/container-metrics.md): Gather metrics from container agents, like `dockerd` or `kubectl`,
49
- along with the resource usage of containers and the applications they run.
50
-- [Applications](https://github.com/netdata/netdata/blob/master/docs/collect/application-metrics.md): Collect per-second metrics from web servers, databases, logs,
51
- message brokers, APM tools, email servers, and much more.
52
-
53
-## Collector architecture and terminology
54
-
55
-**Collector** is a catch-all term for any Netdata process that gathers metrics from an endpoint.
56
-
57
-While we use _collector_ most often in documentation, release notes, and educational content, you may encounter other
58
-terms related to collecting metrics.
59
-
60
-- **Modules** are a type of collector.
61
-- **Orchestrators** are external plugins that run and manage one or more modules. They run as independent processes.
62
- The Go orchestrator is in active development.
63
- - [go.d.plugin](https://github.com/netdata/go.d.plugin/blob/master/README.md): An orchestrator for data
64
- collection modules written in `go`.
65
- - [python.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md): An orchestrator for data collection modules written in
66
- `python` v2/v3.
67
- - [charts.d.plugin](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md): An orchestrator for data collection modules written in
68
- `bash` v4+.
69
-- **External plugins** gather metrics from external processes, such as a webserver or database, and run as independent
70
- processes that communicate with the Netdata daemon via pipes.
71
-- **Internal plugins** gather metrics from `/proc`, `/sys`, and other Linux kernel sources. They are written in `C`,
72
- and run as threads within the Netdata daemon.
73
-
74
-## What's next?
75
-
76
-[Enable or configure a collector](https://github.com/netdata/netdata/blob/master/docs/collect/enable-configure.md) if the default settings are not compatible with
77
-your infrastructure.
78
-
79
-See our [collectors reference](https://github.com/netdata/netdata/blob/master/collectors/REFERENCE.md) for detailed information on Netdata's collector architecture,
80
-troubleshooting a collector, developing a custom collector, and more.
81
-
82
-
docs/guides/step-by-step/step-06.md
deleted
-129
@@ -1,129 +0,0 @@
1
-<!--
2
-title: "Step 6. Collect metrics from more services and apps"
3
-sidebar_label: "Step 6. Collect metrics from more services and apps"
4
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/step-by-step/step-06.md
5
-learn_status: "Published"
6
-learn_topic_type: "Tasks"
7
-learn_rel_path: "Guides/Step by step"
8
--->
9
-
10
-# Step 6. Collect metrics from more services and apps
11
-
12
-When Netdata _starts_, it auto-detects dozens of **data sources**, such as database servers, web servers, and more.
13
-
14
-To auto-detect and collect metrics from a source you just installed, you need to restart Netdata using `sudo systemctl
15
-restart netdata`, or the [appropriate method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
16
-
17
-However, auto-detection only works if you installed the source using its standard installation
18
-procedure. If Netdata isn't collecting metrics after a restart, your source probably isn't configured
19
-correctly.
20
-
21
-Check out the [collectors that come pre-installed with Netdata](https://github.com/netdata/netdata/blob/master/collectors/COLLECTORS.md) to find the module for the
22
-source you want to monitor.
23
-
24
-## What you'll learn in this step
25
-
26
-We'll begin with an overview on Netdata's collector architecture, and then dive into the following:
27
-
28
-- [Step 6. Collect metrics from more services and apps](#step-6-collect-metrics-from-more-services-and-apps)
29
- - [What you'll learn in this step](#what-youll-learn-in-this-step)
30
- - [Netdata's collector architecture](#netdatas-collector-architecture)
31
- - [Enable and disable plugins](#enable-and-disable-plugins)
32
- - [Example: Enable the Nginx collector](#example-enable-the-nginx-collector)
33
- - [What's next?](#whats-next)
34
-
35
-## Netdata's collector architecture
36
-
37
-Many Netdata users never have to configure collector or worry about which plugin orchestrator they want to use.
38
-
39
-But, if you want to configure collector or write a collector for your custom source, it's important to understand the
40
-underlying architecture.
41
-
42
-By default, Netdata collects a lot of metrics every second using any number of discrete collector. Collectors, in turn,
43
-are organized and manged by plugins. **Internal** plugins collect system metrics, **external** plugins collect
44
-non-system metrics, and **orchestrator** plugins group individual collectors together based on the programming language
45
-they were built in.
46
-
47
-These modules are primarily written in [Go](https://github.com/netdata/go.d.plugin/blob/master/README.md) (`go.d`) and
48
-[Python](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/README.md), although some use [Bash](https://github.com/netdata/netdata/blob/master/collectors/charts.d.plugin/README.md)
49
-(`charts.d`).
50
-
51
-## Enable and disable plugins
52
-
53
-You don't need to explicitly enable plugins to auto-detect properly configured sources, but it's useful to know how to
54
-enable or disable them.
55
-
56
-One reason you might want to _disable_ plugins is to improve Netdata's performance on low-resource systems, like
57
-ephemeral nodes or edge devices. Disabling orchestrator plugins like `python.d` can save significant resources if you're
58
-not using any of its data collector modules.
59
-
60
-You can enable or disable plugins in the `[plugin]` section of `netdata.conf`. This section features a list of all the
61
-plugins with a boolean setting (`yes` or `no`) to enable or disable them. Be sure to uncomment the line by removing the
62
-hash (`#`)!
63
-
64
-Enabled:
65
-
66
-```conf
67
-[plugins]
68
- # python.d = yes
69
-```
70
-
71
-Disabled:
72
-
73
-```conf
74
-[plugins]
75
- python.d = no
76
-```
77
-
78
-When you explicitly disable a plugin this way, it won't auto-collect metrics using its collectors.
79
-
80
-## Example: Enable the Nginx collector
81
-
82
-To help explain how the auto-detection process works, let's use an Nginx web server as an example.
83
-
84
-Even if you don't have Nginx installed on your system, we recommend you read through the following section so you can
85
-apply the process to other data sources, such as Apache, Redis, Memcached, and more.
86
-
87
-The Nginx collector, which helps Netdata collect metrics from a running Nginx web server, is part of the
88
-`python.d.plugin` external plugin _orchestrator_.
89
-
90
-In order for Netdata to auto-detect an Nginx web server, you need to enable `ngx_http_stub_status_module` and pass the
91
-`stub_status` directive in the `location` block of your Nginx configuration file.
92
-
93
-You can confirm if the `stub_status` Nginx module is already enabled or not by using following command:
94
-
95
-```sh
96
-nginx -V 2>&1 | grep -o with-http_stub_status_module
97
-```
98
-
99
-If this command returns nothing, you'll need to [enable this module](https://www.nginx.com/blog/monitoring-nginx/).
100
-
101
-Next, edit your `/etc/nginx/sites-enabled/default` file to include a `location` block with the following:
102
-
103
-```conf
104
- location /stub_status {
105
- stub_status;
106
- }
107
-```
108
-
109
-Restart Netdata using `sudo systemctl restart netdata`, or the [appropriate
110
-method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system, and Netdata will auto-detect metrics from your Nginx web
111
-server!
112
-
113
-While not necessary for most auto-detection and collection purposes, you can also configure the Nginx collector itself
114
-by editing its configuration file:
115
-
116
-```sh
117
-./edit-config python.d/nginx.conf
118
-```
119
-
120
-After configuring any source, or changing the configuration files for their respective modules, always restart Netdata.
121
-
122
-## What's next?
123
-
124
-Now that you've learned the fundamentals behind configuring data sources for auto-detection, it's time to move back to
125
-the dashboard to learn more about some of its more advanced features.
126
-
127
-[Next: Netdata's dashboard in depth →](step-07.md)
128
-
129
-
docs/guides/step-by-step/step-07.md
deleted
-121
@@ -1,121 +0,0 @@
1
-<!--
2
-title: "Step 7. Netdata's dashboard in depth"
3
-sidebar_label: "Step 7. Netdata's dashboard in depth"
4
-date: 2020-05-04
5
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/step-by-step/step-07.md
6
-learn_status: "Published"
7
-learn_topic_type: "Tasks"
8
-learn_rel_path: "Guides/Step by step"
9
--->
10
-
11
-# Step 7. Netdata's dashboard in depth
12
-
13
-Welcome to the seventh step of the Netdata guide!
14
-
15
-This step of the guide aims to get you more familiar with the features of the dashboard not previously mentioned in
16
-[step 2](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-02.md).
17
-
18
-## What you'll learn in this step
19
-
20
-In this step of the Netdata guide, you'll learn how to:
21
-
22
-- [Step 7. Netdata's dashboard in depth](#step-7-netdatas-dashboard-in-depth)
23
- - [What you'll learn in this step](#what-youll-learn-in-this-step)
24
- - [Change the dashboard's settings](#change-the-dashboards-settings)
25
- - [Check if there's an update to Netdata](#check-if-theres-an-update-to-netdata)
26
- - [Export and import a snapshot](#export-and-import-a-snapshot)
27
- - [What's next?](#whats-next)
28
-
29
-Let's get started!
30
-
31
-## Change the dashboard's settings
32
-
33
-The settings area at the top of your Netdata dashboard houses browser settings. These settings do not affect the
34
-operation of your Netdata server/daemon. They take effect immediately and are permanently saved to browser local storage
35
-(except the refresh on focus / always option).
36
-
37
-You can see the **Performance**, **Synchronization**, **Visual**, and **Locale** tabs on the dashboard settings modal.
38
-
39
-
41
-
42
-To change any setting, click on the toggle button. We recommend you spend some time reading the descriptions for each setting to understand them before making changes.
43
-
44
-Pay particular attention to the following settings, as they have dramatic impacts on the performance and appearance of
45
-your Netdata dashboard:
46
-
47
-- When to refresh the charts?
48
-- How to handle hidden charts?
49
-- Which chart refresh policy to use?
50
-- Which theme to use?
51
-- Do you need help?
52
-
53
-Some settings are applied immediately, and others are only reflected after you refresh the page.
54
-
55
-## Check if there's an update to Netdata
56
-
57
-You can always check if there is an update available from the **Update** area of your Netdata dashboard.
58
-
59
-
60
-
61
-If an update is available, you'll see a modal similar to the one above.
62
-
63
-When you use the [automatic one-line installer script](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md) attempt to update every day. If
64
-you choose to update it manually, there are [several well-documented methods](https://github.com/netdata/netdata/blob/master/packaging/installer/UPDATE.md) to achieve
65
-that. However, it is best practice for you to first go over the [changelog](https://github.com/netdata/netdata/blob/master/CHANGELOG.md).
66
-
67
-## Export and import a snapshot
68
-
69
-Netdata can export and import snapshots of the contents of your dashboard at a given time. Any Netdata agent can import
70
-a snapshot created by any other Netdata agent.
71
-
72
-Snapshot files include all the information of the dashboard, including the URL of the origin server, its unique ID, and
73
-chart data queries for the visible timeframe. While snapshots are not in real-time, and thus won't update with new
74
-metrics, you can still pan, zoom, and highlight charts as you see fit.
75
-
76
-Snapshots can be incredibly useful for diagnosing anomalies after they've already happened. Let's say Netdata triggered
77
-an alarm while you were sleeping. In the morning, you can look up the exact moment the alarm was raised, export a
78
-snapshot, and send it to a colleague for further analysis.
79
-
80
-> ❗ Know how you shouldn't go around downloading software from suspicious-looking websites? Same policy goes for loading
81
-> snapshots from untrusted or anonymous sources. Importing a snapshot loads quite a bit of data into your web browser,
82
-> and so you should always err on the side of protecting your system.
83
-
84
-To export a snapshot, click on the **export** icon.
85
-
86
-
88
-
89
-Edit the snapshot file name and select your desired compression method. Click on **Export**.
90
-
91
-When the export is complete, your browser will prompt you to save the `.snapshot` file to your machine. You can now
92
-share this file with any other Netdata user via email, Slack, or even to help describe your Netdata experience when
93
-[filing an issue](https://github.com/netdata/netdata/issues/new/choose) on GitHub.
94
-
95
-To import a snapshot, click on the **import** icon.
96
-
97
-
99
-
100
-Select the Netdata snapshot file to import. Once the file is loaded, the dashboard will update with critical information
101
-about the snapshot and the system from which it was taken. Click **import** to render it.
102
-
103
-Your Netdata dashboard will load data contained in the snapshot into charts. Because the snapshot only covers a certain
104
-period, it won't update with new metrics.
105
-
106
-An imported snapshot is also temporary. If you reload your browser tab, Netdata will remove the snapshot data and
107
-restore your real-time dashboard for your machine.
108
-
109
-## What's next?
110
-
111
-In this step of the Netdata guide, you learned how to:
112
-
113
-- Change the dashboard's settings
114
-- Check if there's an update to Netdata
115
-- Export or import a snapshot
116
-
117
-Next, you'll learn how to build your first custom dashboard!
118
-
119
-[Next: Build your first custom dashboard →](step-08.md)
120
-
121
-
docs/guides/step-by-step/step-08.md
deleted
-407
@@ -1,407 +0,0 @@
1
-<!--
2
-title: "Step 8. Build your first custom dashboard"
3
-sidebar_label: "Step 8. Build your first custom dashboard"
4
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/step-by-step/step-08.md
5
-learn_status: "Published"
6
-learn_topic_type: "Tasks"
7
-learn_rel_path: "Guides/Step by step"
8
--->
9
-
10
-# Step 8. Build your first custom dashboard
11
-
12
-In previous steps of the guide, you have learned how several sections of the Netdata dashboard worked.
13
-
14
-This step will show you how to set up a custom dashboard to fit your unique needs. If nothing else, Netdata is really,
15
-really flexible. 🤸
16
-
17
-## What you'll learn in this step
18
-
19
-In this step of the Netdata guide, you'll learn:
20
-
21
-- [Step 8. Build your first custom dashboard](#step-8-build-your-first-custom-dashboard)
22
- - [What you'll learn in this step](#what-youll-learn-in-this-step)
23
- - [Why should I create a custom dashboard?](#why-should-i-create-a-custom-dashboard)
24
- - [Create and prepare your `custom-dashboard.html` file](#create-and-prepare-your-custom-dashboardhtml-file)
25
- - [Add `dashboard.js` to your custom dashboard file](#add-dashboardjs-to-your-custom-dashboard-file)
26
- - [Add some basic styling](#add-some-basic-styling)
27
- - [Creating your dashboard's charts](#creating-your-dashboards-charts)
28
- - [The chart unique ID (required)](#the-chart-unique-id-required)
29
- - [Set chart duration](#set-chart-duration)
30
- - [Set chart size](#set-chart-size)
31
- - [Final touches](#final-touches)
32
- - [The final `custom-dashboard.html`](#the-final-custom-dashboardhtml)
33
- - [What's next?](#whats-next)
34
-
35
-Let's get on with it!
36
-
37
-## Why should I create a custom dashboard?
38
-
39
-Because it's cool!
40
-
41
-But there are way more reasons than that, most of which will prove more valuable to you.
42
-
43
-You could use custom dashboards to aggregate real-time data from multiple Netdata agents in one place. Or, you could put
44
-all the charts with metrics collected from your custom application via `statsd` and perform application performance
45
-monitoring from a single dashboard. You could even use a custom dashboard and a standalone web server to create an
46
-enriched public status page for your service, and give your users something fun to look at while they're waiting for the
47
-503 errors to clear up!
48
-
49
-Netdata's custom dashboarding capability is meant to be as flexible as your ideas. We hope you can take these
50
-fundamental ideas and turn them into something amazing.
51
-
52
-## Create and prepare your `custom-dashboard.html` file
53
-
54
-By default, Netdata stores its web server files at `/usr/share/netdata/web`. As with finding the location of your
55
-`netdata.conf` file, you can double-check this location by loading up `http://HOST:19999/netdata.conf` in your browser
56
-and finding the value of the `web files directory` option.
57
-
58
-To create your custom dashboard, create a file at `/usr/share/netdata/web/custom-dashboard.html` and copy in the
59
-following:
60
-
61
-```html
62
-<!DOCTYPE html>
63
-<html lang="en">
64
-<head>
65
- <title>My custom dashboard</title>
66
-
67
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
68
- <meta charset="utf-8">
69
- <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
70
- <meta name="viewport" content="width=device-width, initial-scale=1">
71
- <meta name="apple-mobile-web-app-capable" content="yes">
72
- <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
73
-
74
- <!-- Add dashboard.js here! -->
75
-
76
-</head>
77
-<body>
78
-
79
- <main class="container">
80
-
81
- <h1>My custom dashboard</h1>
82
-
83
- <!-- Add charts here! -->
84
-
85
- </main>
86
-
87
-</body>
88
-</html>
89
-```
90
-
91
-Try visiting `http://HOST:19999/custom-dashboard.html` in your browser.
92
-
93
-If you get a blank page with this text: `Access to file is not permitted: /usr/share/netdata/web/custom-dashboard.html`.
94
-You can fix this error by changing the dashboard file's permissions to make it owned by the `netdata` user.
95
-
96
-```bash
97
-sudo chown netdata:netdata /usr/share/netdata/web/custom-dashboard.html
98
-```
99
-
100
-Reload your browser, and you should see a blank page with the title: **Your custom dashboard**!
101
-
102
-## Add `dashboard.js` to your custom dashboard file
103
-
104
-You need to include the `dashboard.js` file of a Netdata agent to add Netdata charts. Add the following to the `<head>`
105
-of your custom dashboard page and change `HOST` according to your setup.
106
-
107
-```html
108
- <!-- Add dashboard.js here! -->
109
- <script type="text/javascript" src="http://HOST:19999/dashboard.js"></script>
110
-```
111
-
112
-When you add `dashboard.js` to any web page, it loads several JavaScript and CSS files to create and style charts. It
113
-also scans the page for elements that define charts, builds them, and refreshes with new metrics.
114
-
115
-> If you enabled SSL on your Netdata dashboard already, you'll need to use `https://` to grab the `dashboard.js` file.
116
-
117
-## Add some basic styling
118
-
119
-While not necessary, let's add some basic styling to make our dashboard look a little nicer. We're putting some
120
-basic CSS into a `<style>` tag inside of the page's `<head>` element.
121
-
122
-```html
123
- <!-- Add dashboard.js here! -->
124
- <script type="text/javascript" src="http://HOST:19999/dashboard.js"></script>
125
-
126
- <style>
127
- .wrap {
128
- max-width: 1280px;
129
- margin: 0 auto;
130
- }
131
-
132
- h1 {
133
- margin-bottom: 30px;
134
- text-align: center;
135
- }
136
-
137
- .charts {
138
- display: flex;
139
- flex-flow: row wrap;
140
- justify-content: space-around;
141
- }
142
- </style>
143
-
144
-</head>
145
-```
146
-
147
-## Creating your dashboard's charts
148
-
149
-Time to create a chart!
150
-
151
-You need to create a `<div>` for each new chart. Each `<div>` element accepts a few `data-` attributes, some of which
152
-are required and some of which are optional.
153
-
154
-Let's cover a few important ones. And while we do it, we'll create a custom dashboard that shows a few CPU-related
155
-charts on a single page.
156
-
157
-### The chart unique ID (required)
158
-
159
-You need to specify the unique ID of a chart to show it on your custom dashboard. If you forgot how to find the unique
160
-ID, head back over to [step 2](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-02.md#understand-charts-dimensions-families-and-contexts)
161
-for a re-introduction.
162
-
163
-You can then put this unique ID into a `<div>` element with the `data-netdata` attribute. Put this in the `<body>` of
164
-your custom dashboard file beneath the helpful comment.
165
-
166
-```html
167
-<body>
168
-
169
- <main class="wrap">
170
-
171
- <h1>My custom dashboard</h1>
172
-
173
- <div class="charts">
174
-
175
- <!-- Add charts here! -->
176
- <div data-netdata="system.cpu"></div>
177
-
178
- </div>
179
-
180
- </main>
181
-
182
-</body>
183
-```
184
-
185
-Reload the page, and you should see a real-time `system.cpu` chart!
186
-
187
-... and a whole lot of white space. Let's fix that by adding a few more charts.
188
-
189
-```html
190
- <!-- Add charts here! -->
191
- <div data-netdata="system.cpu"></div>
192
- <div data-netdata="apps.cpu"></div>
193
- <div data-netdata="groups.cpu"></div>
194
- <div data-netdata="users.cpu"></div>
195
-```
196
-
197
-
199
-
200
-### Set chart duration
201
-
202
-By default, these charts visualize 10 minutes of Netdata metrics. Let's get a little more granular on this dashboard. To
203
-do so, add a new `data-after=""` attribute to each chart.
204
-
205
-`data-after` takes a _relative_ number of seconds from _now_. So, by putting `-300` as the value, you're asking the
206
-custom dashboard to display the _last 5 minutes_ (`5m * 60s = 300s`) of data.
207
-
208
-```html
209
- <!-- Add charts here! -->
210
- <div data-netdata="system.cpu"
211
- data-after="-300">
212
- </div>
213
- <div data-netdata="apps.cpu"
214
- data-after="-300">
215
- </div>
216
- <div data-netdata="groups.cpu"
217
- data-after="-300">
218
- </div>
219
- <div data-netdata="users.cpu"
220
- data-after="-300">
221
- </div>
222
-```
223
-
224
-### Set chart size
225
-
226
-You can set the size of any chart using the `data-height=""` and `data-width=""` attributes. These attributes can be
227
-anything CSS accepts for width and height (e.g. percentages, pixels, em/rem, calc, and so on).
228
-
229
-Let's make the charts a little taller and allow them to fit side-by-side for a more compact view. Add
230
-`data-height="200px"` and `data-width="50%"` to each chart.
231
-
232
-```html
233
- <div data-netdata="system.cpu"
234
- data-after="-300"
235
- data-height="250px"
236
- data-width="50%"></div>
237
- <div data-netdata="apps.cpu"
238
- data-after="-300"
239
- data-height="250px"
240
- data-width="50%"></div>
241
- <div data-netdata="groups.cpu"
242
- data-after="-300"
243
- data-height="250px"
244
- data-width="50%"></div>
245
- <div data-netdata="users.cpu"
246
- data-after="-300"
247
- data-height="250px"
248
- data-width="50%"></div>
249
-```
250
-
251
-Now we're getting somewhere!
252
-
253
-
255
-
256
-## Final touches
257
-
258
-While we already have a perfectly workable dashboard, let's add some final touches to make it a little more pleasant on
259
-the eyes.
260
-
261
-First, add some extra CSS to create some vertical whitespace between the top and bottom row of charts.
262
-
263
-```html
264
- <style>
265
- ...
266
-
267
- .charts > div {
268
- margin-bottom: 6rem;
269
- }
270
- </style>
271
-```
272
-
273
-To create horizontal whitespace, change the value of `data-width="50%"` to `data-width="calc(50% - 2rem)"`.
274
-
275
-```html
276
- <div data-netdata="system.cpu"
277
- data-after="-300"
278
- data-height="250px"
279
- data-width="calc(50% - 2rem)"></div>
280
- <div data-netdata="apps.cpu"
281
- data-after="-300"
282
- data-height="250px"
283
- data-width="calc(50% - 2rem)"></div>
284
- <div data-netdata="groups.cpu"
285
- data-after="-300"
286
- data-height="250px"
287
- data-width="calc(50% - 2rem)"></div>
288
- <div data-netdata="users.cpu"
289
- data-after="-300"
290
- data-height="250px"
291
- data-width="calc(50% - 2rem)"></div>
292
-```
293
-
294
-Told you the `data-width` and `data-height` attributes can take any CSS values!
295
-
296
-Prefer a dark theme? Add this to your `<head>` _above_ where you added `dashboard.js`:
297
-
298
-```html
299
- <script>
300
- var netdataTheme = 'slate';
301
- </script>
302
-
303
- <!-- Add dashboard.js here! -->
304
- <script type="text/javascript" src="https://HOST/dashboard.js"></script>
305
-```
306
-
307
-Refresh the dashboard to give your eyes a break from all that blue light!
308
-
309
-
311
-
312
-## The final `custom-dashboard.html`
313
-
314
-In case you got lost along the way, here's the final version of the `custom-dashboard.html` file:
315
-
316
-```html
317
-<!DOCTYPE html>
318
-<html lang="en">
319
-<head>
320
- <title>My custom dashboard</title>
321
-
322
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
323
- <meta charset="utf-8">
324
- <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
325
- <meta name="viewport" content="width=device-width, initial-scale=1">
326
- <meta name="apple-mobile-web-app-capable" content="yes">
327
- <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
328
-
329
- <script>
330
- var netdataTheme = 'slate';
331
- </script>
332
-
333
- <!-- Add dashboard.js here! -->
334
- <script type="text/javascript" src="http://localhost:19999/dashboard.js"></script>
335
-
336
- <style>
337
- .wrap {
338
- max-width: 1280px;
339
- margin: 0 auto;
340
- }
341
-
342
- h1 {
343
- margin-bottom: 30px;
344
- text-align: center;
345
- }
346
-
347
- .charts {
348
- display: flex;
349
- flex-flow: row wrap;
350
- justify-content: space-around;
351
- }
352
-
353
- .charts > div {
354
- margin-bottom: 6rem;
355
- position: relative;
356
- }
357
- </style>
358
-
359
-</head>
360
-<body>
361
-
362
- <main class="wrap">
363
-
364
- <h1>My custom dashboard</h1>
365
-
366
- <div class="charts">
367
-
368
- <!-- Add charts here! -->
369
- <div data-netdata="system.cpu"
370
- data-after="-300"
371
- data-height="250px"
372
- data-width="calc(50% - 2rem)"></div>
373
- <div data-netdata="apps.cpu"
374
- data-after="-300"
375
- data-height="250px"
376
- data-width="calc(50% - 2rem)"></div>
377
- <div data-netdata="groups.cpu"
378
- data-after="-300"
379
- data-height="250px"
380
- data-width="calc(50% - 2rem)"></div>
381
- <div data-netdata="users.cpu"
382
- data-after="-300"
383
- data-height="250px"
384
- data-width="calc(50% - 2rem)"></div>
385
-
386
- </div>
387
-
388
- </main>
389
-
390
-</body>
391
-</html>
392
-```
393
-
394
-## What's next?
395
-
396
-In this guide, you learned the fundamentals of building a custom Netdata dashboard. You should now be able to add more
397
-charts to your `custom-dashboard.html`, change the charts that are already there, and size them according to your needs.
398
-
399
-Of course, the custom dashboarding features covered here are just the beginning. Be sure to read up on our [custom
400
-dashboard documentation](https://github.com/netdata/netdata/blob/master/web/gui/custom/README.md) for details on how you can use other chart libraries, pull metrics
401
-from multiple Netdata agents, and choose which dimensions a given chart shows.
402
-
403
-Next, you'll learn how to store long-term historical metrics in Netdata!
404
-
405
-[Next: Long-term metrics storage →](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-09.md)
406
-
407
-
docs/guides/step-by-step/step-09.md
deleted
-169
@@ -1,169 +0,0 @@
1
-<!--
2
-title: "Step 9. Long-term metrics storage"
3
-sidebar_label: "Step 9. Long-term metrics storage"
4
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/step-by-step/step-09.md
5
-learn_status: "Published"
6
-learn_topic_type: "Tasks"
7
-learn_rel_path: "Guides/Step by step"
8
--->
9
-
10
-# Step 9. Long-term metrics storage
11
-
12
-By default, Netdata stores metrics in a custom database we call the [database engine](https://github.com/netdata/netdata/blob/master/database/engine/README.md), which
13
-stores recent metrics in your system's RAM and "spills" historical metrics to disk. By using both RAM and disk, the
14
-database engine helps you store a much larger dataset than the amount of RAM your system has.
15
-
16
-On a system that's collecting 2,000 metrics every second, the database engine's default configuration will store about
17
-two day's worth of metrics in RAM and on disk.
18
-
19
-That's a lot of metrics. We're talking 345,600,000 individual data points. And the database engine does it with a tiny
20
-a portion of the RAM available on most systems.
21
-
22
-To store _even more_ metrics, you have two options. First, you can tweak the database engine's options to expand the RAM
23
-or disk it uses. Second, you can archive metrics to an external database. For that, we'll use MongoDB as examples.
24
-
25
-## What you'll learn in this step
26
-
27
-In this step of the Netdata guide, you'll learn how to:
28
-
29
-- [Step 9. Long-term metrics storage](#step-9-long-term-metrics-storage)
30
- - [What you'll learn in this step](#what-youll-learn-in-this-step)
31
- - [Tweak the database engine's settings](#tweak-the-database-engines-settings)
32
- - [Archive metrics to an external database](#archive-metrics-to-an-external-database)
33
- - [Archive metrics via the MongoDB exporting connector](#archive-metrics-via-the-mongodb-exporting-connector)
34
- - [What's next?](#whats-next)
35
-
36
-Let's get started!
37
-
38
-## Tweak the database engine's settings
39
-
40
-If you're using Netdata v1.18.0 or higher, and you haven't changed your `memory mode` settings before following this
41
-guide, your Netdata agent is already using the database engine.
42
-
43
-Let's look at your `netdata.conf` file again. Under the `[global]` section, you'll find three connected options.
44
-
45
-```conf
46
-[db]
47
- # mode = dbengine
48
- # dbengine page cache size MB = 32
49
- # dbengine disk space MB = 256
50
-```
51
-
52
-The `memory mode` option is set, by default, to `dbengine`. `page cache size` determines the amount of RAM, in MiB, that
53
-the database engine dedicates to caching the metrics it's collecting. `dbengine disk space` determines the amount of
54
-disk space, in MiB, that the database engine will use to store these metrics once they've been "spilled" to disk..
55
-
56
-You can uncomment and change either `page cache size` or `dbengine disk space` based on how much RAM and disk you want
57
-the database engine to use. The higher those values, the more metrics Netdata will store. If you change them to 64 and
58
-512, respectively, the database engine should store about four day's worth of data on a system collecting 2,000 metrics
59
-every second.
60
-
61
-[**See our database engine calculator**](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) to help you correctly set `dbengine disk
62
-space` based on your needs. The calculator gives an accurate estimate based on how many child nodes you have, how many
63
-metrics your Agent collects, and more.
64
-
65
-```conf
66
-[db]
67
- mode = dbengine
68
- dbengine page cache size MB = 64
69
- dbengine disk space MB = 512
70
-```
71
-
72
-After you've made your changes, restart Netdata using `sudo systemctl restart netdata`, or the [appropriate
73
-method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
74
-
75
-To confirm the database engine is working, go to your Netdata dashboard and click on the **Netdata Monitoring** menu on
76
-the right-hand side. You can find `dbengine` metrics after `queries`.
77
-
78
-
80
-
81
-## Archive metrics to an external database
82
-
83
-You can archive all the metrics collected by Netdata to **external databases**. The supported databases and services
84
-include Graphite, OpenTSDB, Prometheus, AWS Kinesis Data Streams, Google Cloud Pub/Sub, MongoDB, and the list is always
85
-growing.
86
-
87
-As we said in [step 1](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-01.md), we have only complimentary systems, not competitors! We're
88
-happy to support these archiving methods and are always working to improve them.
89
-
90
-A lot of Netdata users archive their metrics to one of these databases for long-term storage or further analysis. Since
91
-Netdata collects so many metrics every second, they can quickly overload small devices or even big servers that are
92
-aggregating metrics streaming in from other Netdata agents.
93
-
94
-We even support resampling metrics during archiving. With resampling enabled, Netdata will archive only the average or
95
-sum of every X seconds of metrics. This reduces the sheer amount of data, albeit with a little less accuracy.
96
-
97
-How you archive metrics, or if you archive metrics at all, is entirely up to you! But let's cover two easy archiving
98
-methods, MongoDB and Prometheus remote write, to get you started.
99
-
100
-### Archive metrics via the MongoDB exporting connector
101
-
102
-Begin by installing MongoDB its dependencies via the correct package manager for your system.
103
-
104
-```bash
105
-sudo apt-get install mongodb # Debian/Ubuntu
106
-sudo dnf install mongodb # Fedora
107
-sudo yum install mongodb # CentOS
108
-```
109
-
110
-Next, install the one essential dependency: v1.7.0 or higher of
111
-[libmongoc](http://mongoc.org/libmongoc/current/installing.html).
112
-
113
-```bash
114
-sudo apt-get install libmongoc-1.0-0 libmongoc-dev # Debian/Ubuntu
115
-sudo dnf install mongo-c-driver mongo-c-driver-devel # Fedora
116
-sudo yum install mongo-c-driver mongo-c-driver-devel # CentOS
117
-```
118
-
119
-Next, create a new MongoDB database and collection to store all these archived metrics. Use the `mongo` command to start
120
-the MongoDB shell, and then execute the following command:
121
-
122
-```mongodb
123
-use netdata
124
-db.createCollection("netdata_metrics")
125
-```
126
-
127
-Next, Netdata needs to be [reinstalled](https://github.com/netdata/netdata/blob/master/packaging/installer/REINSTALL.md) in order to detect that the required
128
-libraries to make this exporting connection exist. Since you most likely installed Netdata using the one-line installer
129
-script, all you have to do is run that script again. Don't worry—any configuration changes you made along the way will
130
-be retained!
131
-
132
-Now, from your Netdata config directory, initialize and edit a `exporting.conf` file to tell Netdata where to find the
133
-database you just created.
134
-
135
-```sh
136
-./edit-config exporting.conf
137
-```
138
-
139
-Add the following section to the file:
140
-
141
-```conf
142
-[mongodb:my_mongo_instance]
143
- enabled = yes
144
- destination = mongodb://localhost
145
- database = netdata
146
- collection = netdata_metrics
147
-```
148
-
149
-Restart Netdata using `sudo systemctl restart netdata`, or the [appropriate
150
-method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system, to enable the MongoDB exporting connector. Click on the
151
-**Netdata Monitoring** menu and check out the **exporting my mongo instance** sub-menu. You should start seeing these
152
-charts fill up with data about the exporting process!
153
-
154
-
155
-
156
-If you'd like to try connecting Netdata to another database, such as Prometheus or OpenTSDB, read our [exporting
157
-documentation](https://github.com/netdata/netdata/blob/master/exporting/README.md).
158
-
159
-## What's next?
160
-
161
-You're getting close to the end! In this step, you learned how to make the most of the database engine, or archive
162
-metrics to MongoDB for long-term storage.
163
-
164
-In the last step of this step-by-step guide, we'll put our sysadmin hat on and use Nginx to proxy traffic to and from
165
-our Netdata dashboard.
166
-
167
-[Next: Set up a proxy →](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-10.md)
168
-
169
-
docs/guides/step-by-step/step-99.md
deleted
-55
@@ -1,55 +0,0 @@
1
-<!--
2
-title: "You're finished"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/guides/step-by-step/step-99.md
4
-sidebar_label: "You're finished"
5
-learn_status: "Published"
6
-learn_topic_type: "Tasks"
7
-learn_rel_path: "Guides/Step by step"
8
--->
9
-
10
-# Step ∞. You're finished!
11
-
12
-Congratulations. 🎉
13
-
14
-You've completed the step-by-step Netdata guide. That means you're well on your way to becoming an expert in using
15
-our toolkit for health monitoring and performance troubleshooting.
16
-
17
-But, perhaps more importantly, also that much closer to being an expert in the _fundamental skills behind health
18
-monitoring and performance troubleshooting_, which you can take with you to any job or project.
19
-
20
-And that is the entire point of this guide, and Netdata's [documentation](https://learn.netdata.cloud) as a
21
-whole—give you every resource possible to help you build faster, more resilient systems, services, and applications.
22
-
23
-Along the way, you learned how to:
24
-
25
-- Navigate Netdata's dashboard and visually detect anomalies using its charts.
26
-- Monitor multiple systems using Netdata agents connected together with your browser and Netdata Cloud.
27
-- Edit your `netdata.conf` file to tweak Netdata to your liking.
28
-- Tune existing alarms and create entirely new ones, plus get notifications about alarms on your favorite services.
29
-- Take advantage of Netdata's auto-detection capabilities to ensure your applications/services are monitored with
30
- little to no configuration.
31
-- Use advanced features within Netdata's dashboard.
32
-- Build a custom dashboard using `dashboard.js`.
33
-- Save more historical metrics with the database engine or archive metrics to MongoDB.
34
-- Put Netdata behind a proxy to enable HTTPS and improve performance.
35
-
36
-Seems like a lot, right? Well, we hope it felt manageable and, yes, even _fun_.
37
-
38
-## What's next?
39
-
40
-Now that you're at the end of our step-by-step Netdata guide, the next steps are entirely up to you. In fact, you're
41
-just at the beginning of your journey into health monitoring and performance troubleshooting.
42
-
43
-Our documentation exists to put every Netdata resource in front of you as easily and coherently as we possibly can.
44
-Click around, search, and find new mountains to climb.
45
-
46
-If that feels like too much possibility to you, why not one of these options:
47
-
48
-- Share your experience with Netdata and this guide. Be sure to [@mention](https://twitter.com/linuxnetdata) us on
49
- Twitter!
50
-- Contribute to what we do. Browse our [open issues](https://github.com/netdata/netdata/issues) and check out out
51
- [contributions doc](https://learn.netdata.cloud/contribute/) for ideas of how you can pitch in.
52
-
53
-We can't wait to see what you monitor next! Bon voyage! ⛵
54
-
55
-