Add renamed markdown files (#14540)
Add renamed files
Chris Akritidis committed
Feb 15, 2023 at 10:59 UTC
37a918ae2bc996fc881ab60042ae5a8f434f4c52
17 files changed
+2483
docs/cloud/alerts-notifications/notifications.md
new
+152
@@ -0,0 +1,152 @@
1
+<!--
2
+title: "Cloud alert notifications"
3
+description: "Configure Netdata Cloud to send notifications to your team whenever any node on your infrastructure triggers an alert threshold."
4
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/notifications.md"
5
+sidebar_label: "Cloud alert notifications"
6
+learn_status: "Published"
7
+learn_rel_path: "Integrations/Notify/Cloud alert notifications"
8
+-->
9
+
10
+import Callout from '@site/src/components/Callout'
11
+
12
+Netdata Cloud can send centralized alert notifications to your team whenever a node enters a warning, critical, or
13
+unreachable state. By enabling notifications, you ensure no alert, on any node in your infrastructure, goes unnoticed by
14
+you or your team.
15
+
16
+Having this information centralized helps you:
17
+* Have a clear view of the health across your infrastructure, [seeing all a alerts in one place](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/view-active-alerts.md)
18
+* Easily [setup your alert notification process](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/manage-notification-methods.md):
19
+methods to use and where to use them, filtering rules, etc.
20
+* Quickly troubleshoot using [Metric Correlations](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/metric-correlations.md)
21
+or [Anomaly Advisor](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/anomaly-advisor.md)
22
+
23
+If a node is getting disconnected often or has many alerts, we protect you and your team from alert fatigue by sending
24
+you a flood protection notification. Getting one of these notifications is a good signal of health or performance issues
25
+on that node.
26
+
27
+Admins must enable alert notifications for their [Space(s)](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/manage-notification-methods.md#manage-space-notification-settings). All users in a
28
+Space can then personalize their notifications settings from within their [account
29
+menu](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/#manage-user-notification-settings).
30
+
31
+<Callout type="notice">
32
+
33
+Centralized alert notifications from Netdata Cloud is a independent process from [notifications from
34
+Netdata](https://github.com/netdata/netdata/blob/master/docs/monitor/enable-notifications.md). You can enable one or the other, or both, based on your needs. However,
35
+the alerts you see in Netdata Cloud are based on those streamed from your Netdata-monitoring nodes. If you want to tweak
36
+or add new alert that you see in Netdata Cloud, and receive via centralized alert notifications, you must
37
+[configure](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md) each node's alert watchdog.
38
+
39
+</Callout>
40
+
41
+### Alert notifications
42
+
43
+Netdata Cloud can send centralized alert notifications to your team whenever a node enters a warning, critical, or unreachable state. By enabling notifications,
44
+you ensure no alert, on any node in your infrastructure, goes unnoticed by you or your team.
45
+
46
+If a node is getting disconnected often or has many alerts, we protect you and your team from alert fatigue by sending you a flood protection notification.
47
+Getting one of these notifications is a good signal of health or performance issues on that node.
48
+
49
+Alert notifications can be delivered through different methods, these can go from an Email sent from Netdata to the use of a 3rd party tool like PagerDuty.
50
+
51
+Notification methods are classified on two main attributes:
52
+* Service level: Personal or System
53
+* Service classification: Community or Business
54
+
55
+Only administrators are able to manage the space's alert notification settings.
56
+All users in a Space can personalize their notifications settings, for Personal service level notification methods, from within their profile menu.
57
+
58
+> ⚠️ Netdata Cloud supports different notification methods and their availability will depend on the plan you are at.
59
+> For more details check [Service classification](#service-classification) or [netdata.cloud/pricing](https://www.netdata.cloud/pricing).
60
+
61
+#### Service level
62
+
63
+##### Personal
64
+
65
+The notifications methods classified as **Personal** are what we consider generic, meaning that these can't have specific rules for them set by the administrators.
66
+
67
+These notifications are sent to the destination of the channel which is a user-specific attribute, e.g. user's e-mail, and the users are the ones that will then be able to
68
+manage what specific configurations they want for the Space / Room(s) and the desired Notification level, they can achieve this from their User Profile page under
69
+**Notifications**.
70
+
71
+One example of such a notification method is the E-mail.
72
+
73
+##### System
74
+
75
+For **System** notification methods, the destination of the channel will be a target that usually isn't specific to a single user, e.g. slack channel.
76
+
77
+These notification methods allow for fine-grain rule settings to be done by administrators and more than one configuration can exist for them since. You can specify
78
+different targets depending on Rooms or Notification level settings.
79
+
80
+Some examples of such notification methods are: Webhook, PagerDuty, slack.
81
+
82
+#### Service classification
83
+
84
+##### Community
85
+
86
+Notification methods classified as Community can be used by everyone independent on the plan your space is at.
87
+These are: Email and discord
88
+
89
+##### Pro
90
+
91
+Notification methods classified as Pro are only available for **Pro** and **Business** plans
92
+These are: webhook
93
+
94
+##### Business
95
+
96
+Notification methods classified as Business are only available for **Business** plans
97
+These are: PagerDuty, slack
98
+
99
+## Flood protection
100
+
101
+If a node has too many state changes like firing too many alerts or going from reachable to unreachable, Netdata Cloud
102
+enables flood protection. As long as a node is in flood protection mode, Netdata Cloud does not send notifications about
103
+this node. Even with flood protection active, it is possible to access the node directly, either via Netdata Cloud or
104
+the local Agent dashboard at `http://NODE:19999`.
105
+
106
+## Anatomy of an alert notification
107
+
108
+Email alarm notifications show the following information:
109
+
110
+- The Space's name
111
+- The node's name
112
+- Alarm status: critical, warning, cleared
113
+- Previous alarm status
114
+- Time at which the alarm triggered
115
+- Chart context that triggered the alarm
116
+- Name and information about the triggered alarm
117
+- Alarm value
118
+- Total number of warning and critical alerts on that node
119
+- Threshold for triggering the given alarm state
120
+- Calculation or database lookups that Netdata uses to compute the value
121
+- Source of the alarm, including which file you can edit to configure this alarm on an individual node
122
+
123
+Email notifications also feature a **Go to Node** button, which takes you directly to the offending chart for that node
124
+within Cloud's embedded dashboards.
125
+
126
+Here's an example email notification for the `ram_available` chart, which is in a critical state:
127
+
128
+
129
+
130
+## What's next?
131
+
132
+Netdata Cloud's alarm notifications feature leverages the alarms configuration on each node in your infrastructure. If
133
+you'd like to tweak any of these alarms, or even add new ones based on your needs, read our [health
134
+quickstart](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md).
135
+
136
+You can also [view active alarms](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/view-active-alerts.md) in Netdata Cloud for an instant
137
+visualization of the health of your infrastructure.
138
+
139
+### Related Topics
140
+
141
+#### **Related Concepts**
142
+- [Rooms](https://github.com/netdata/netdata/blob/master/docs/cloud/war-rooms.md)
143
+- [Metric Correlations](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/metric-correlations.md)
144
+- [Anomaly Advisor](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/anomaly-advisor.md)
145
+
146
+#### Related Tasks
147
+- [View Active alarms](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/view-active-alerts.md)
148
+- [Manage notification methods](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/manage-notification-methods.md)
149
+- [Add webhook notification configuration](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/add-webhook-notification-configuration.md)
150
+- [Add Discord notification configuration](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/add-discord-notification.md)
151
+- [Add Slack notification configuration](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/add-slack-notification-configuration.md)
152
+- [Add PagerDuty notification configuration](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/add-pagerduty-notification-configuration.md)
docs/cloud/alerts-notifications/smartboard.md
new
+46
@@ -0,0 +1,46 @@
1
+<!--
2
+title: "Alerts smartboard"
3
+description: ""
4
+type: "reference"
5
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/smartboard.md"
6
+sidebar_label: "Alerts smartboard"
7
+learn_status: "Published"
8
+learn_topic_type: "Tasks"
9
+learn_rel_path: "Integrations/Notify/Cloud alert notifications"
10
+-->
11
+
12
+The Alerts view gives you a high level of availability and performance information for every node you're
13
+monitoring with Netdata Cloud. We expect it to become the "home base" for many Netdata Cloud users who want to instantly
14
+understand what's going on with their infrastructure and exactly where issues might be.
15
+
16
+The Alerts view is available entirely for free to all users and for any number of nodes.
17
+
18
+## Alerts table and filtering
19
+
20
+The Alerts view shows all active alerts in your War Room, including the alert's name, the most recent value, a
21
+timestamp of when it became active, and the relevant node.
22
+
23
+You can use the checkboxes in the filter pane on the right side of the screen to filter the alerts displayed in the
24
+table
25
+by Status, Class, Type & Componenet, Role, Operating System, or Node.
26
+
27
+Click on any of the alert names to see the alert.
28
+
29
+## View active alerts
30
+
31
+In the `Active` subtab, you can see exactly how many **critical** and **warning** alerts are active across your nodes.
32
+
33
+## View configured alerts
34
+
35
+You can view all the configured alerts on all the agents that belong to a War Room in the `Alert Configurations` subtab.
36
+From within the Alerts view, you can click the `Alert Configurations` subtab to see a high level view of the states of
37
+the alerts on the nodes within this War Room and drill down to the node level where each alert is configured with their
38
+latest status.
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
docs/cloud/alerts-notifications/view-active-alerts.md
new
+76
@@ -0,0 +1,76 @@
1
+<!--
2
+title: "View active alerts"
3
+description: >-
4
+ "Track the health of your infrastructure in one place by taking advantage of the powerful health monitoring
5
+ watchdog running on every node."
6
+type: "how-to"
7
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/view-active-alerts.md"
8
+sidebar_label: "View active alerts"
9
+learn_status: "Published"
10
+learn_topic_type: "Tasks"
11
+learn_rel_path: "Integrations/Notify/Cloud alert notifications"
12
+-->
13
+
14
+Netdata Cloud receives information about active alerts on individual nodes in your infrastructure and updates the
15
+interface based on those status changes.
16
+
17
+Netdata Cloud doesn't produce alerts itself but rather receives and aggregates alerts from each node in your
18
+infrastructure based on their configuration. Every node comes with hundreds of pre-configured alerts that have been
19
+tested by Netdata's community of DevOps engineers and SREs, but you may want to customize existing alerts or create new
20
+ones entirely.
21
+
22
+Read our doc on [health alerts](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md) to
23
+learn how to tweak existing alerts or create new
24
+health entities based on the specific needs of your infrastructure. By taking charge of alert configuration, you'll
25
+ensure Netdata Cloud always delivers the most relevant alerts about the well-being of your nodes.
26
+
27
+## View all active alerts
28
+
29
+The [Alerts Smartboard](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/smartboard.md)
30
+provides a high-level interface for viewing the number of critical or warning alerts and where they are in your
31
+infrastructure.
32
+
33
+
34
+
35
+Click on the **Alerts** tab in any War Room to open the Smartboard. Alternatively, click on any of the alert badges in
36
+the [Nodes view](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/nodes.md) to jump to the Alerts
37
+Smartboard.
38
+
39
+From here, filter active alerts using the **critical** or **warning** boxes, or hover over a box in
40
+the [nodes map](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/smartboard.md#nodes-map)
41
+to see a
42
+popup node-specific alert information.
43
+
44
+## View alerts in context with charts
45
+
46
+If you click on any of the alerts, either in a nodes map popup or the alerts table, Netdata Cloud navigates you to the
47
+single-node dashboard and scrolls to the relevant chart. Netdata Cloud also draws a highlight and the value at the
48
+moment your node triggered this alert.
49
+
50
+
51
+
52
+You can
53
+then [select this area](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md#select)
54
+with `Alt/⌘ + mouse selection` to highlight the alerted timeframe while you explore other charts for root cause
55
+analysis.
56
+
57
+Or, select the area and
58
+run [Metric Correlations](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/metric-correlations.md) to
59
+filter the single-node
60
+dashboard to only those charts most likely to be connected to the alert.
61
+
62
+## What's next?
63
+
64
+Learn more about the features of the Smartboard in
65
+its [reference](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/smartboard.md)
66
+doc. To stay notified of active alerts,
67
+enable [centralized alert notifications](https://github.com/netdata/netdata/blob/master/docs/cloud/alerts-notifications/notifications.md)
68
+from Netdata Cloud.
69
+
70
+If you're through with setting up alerts, it might be time
71
+to [invite your team](https://github.com/netdata/netdata/blob/master/docs/cloud/manage/invite-your-team.md).
72
+
73
+Check out our recommendations on organizing and
74
+using [Spaces](https://github.com/netdata/netdata/blob/master/docs/cloud/spaces.md) and
75
+[War Rooms](https://github.com/netdata/netdata/blob/master/docs/cloud/war-rooms.md) to streamline your processes once
76
+you find an alert in Netdata Cloud.
docs/cloud/cheatsheet.md
new
+232
@@ -0,0 +1,232 @@
1
+<!--
2
+title: "Netdata management and configuration cheatsheet"
3
+description: "'Connecting an Agent to the Cloud allows a Netdata Agent, running on a distributed node, to securely connect to Netdata Cloud via the encrypted Agent-Cloud link (ACLK).'"
4
+image: "/cheatsheet/cheatsheet-meta.png"
5
+sidebar_label: "Cheatsheet"
6
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/cheatsheet.md"
7
+part_of_learn: "True"
8
+learn_status: "Published"
9
+learn_topic_type: "Getting started"
10
+learn_rel_path: "Getting started"
11
+sidebar_position: "100"
12
+-->
13
+
14
+import {
15
+ OneLineInstallWget,
16
+ OneLineInstallCurl,
17
+} from '@site/src/components/OneLineInstall/';
18
+
19
+Use our management & configuration cheatsheet to simplify your interactions with Netdata, including configuration,
20
+using charts, managing the daemon, and more.
21
+
22
+## Install Netdata
23
+
24
+#### Install Netdata
25
+
26
+<OneLineInstallWget />
27
+
28
+Or, if you have cURL but not wget (such as on macOS):
29
+
30
+<OneLineInstallCurl />
31
+
32
+#### Claim a node to Netdata Cloud
33
+
34
+To do so, sign in to Netdata Cloud, click the `Claim Nodes` button, choose the `War Rooms` to add nodes to, then click `Copy` to copy the full script to your clipboard. Paste that into your node’s terminal and run it.
35
+
36
+## Metrics collection & retention
37
+
38
+You can tweak your settings in the netdata.conf file.
39
+📄 [Find your netdata.conf file](https://learn.netdata.cloud/guides/step-by-step/step-04#find-your-netdataconf-file)
40
+
41
+Open a new terminal and navigate to the netdata.conf file. Use the edit-config script to make changes: `sudo ./edit-config netdata.conf`
42
+
43
+The most popular settings to change are:
44
+
45
+#### Increase metrics retention (4GiB)
46
+
47
+```
48
+sudo ./edit-config netdata.conf
49
+```
50
+
51
+```
52
+[global]
53
+ dbengine multihost disk space = 4096
54
+```
55
+
56
+#### Reduce the collection frequency (every 5 seconds)
57
+
58
+```
59
+sudo ./edit-config netdata.conf
60
+```
61
+
62
+```
63
+[global]
64
+ update every = 5
65
+```
66
+
67
+#### Enable/disable plugins (groups of collectors)
68
+
69
+```
70
+sudo ./edit-config netdata.conf
71
+```
72
+
73
+```
74
+[plugins]
75
+ go.d = yes # enabled
76
+ node.d = no # disabled
77
+```
78
+
79
+#### Enable/disable specific collectors
80
+
81
+```
82
+sudo ./edit-config go.d.conf
83
+```
84
+
85
+> `Or python.d.conf, node.d.conf, edbpf.conf, and so on`.
86
+
87
+```
88
+modules:
89
+ activemq: no # disabled
90
+ bind: no # disabled
91
+ cockroachdb: yes # enabled
92
+```
93
+
94
+#### Edit a collector's config (example)
95
+
96
+```
97
+$ sudo ./edit-config go.d/mysql.conf
98
+$ sudo ./edit-config ebpf.conf
99
+$ sudo ./edit-config python.d/anomalies.conf
100
+```
101
+
102
+## Configuration
103
+
104
+#### The Netdata config directory: `/etc/netdata`
105
+
106
+> If you don't have such a directory:
107
+> 📄 [Find your netdata.conf file](https://learn.netdata.cloud/guides/step-by-step/step-04#find-your-netdataconf-file)
108
+> The cheatsheet assumes you’re running all commands from within the Netdata config directory!
109
+
110
+#### Edit Netdata's main config file: `$ sudo ./edit-config netdata.conf`
111
+
112
+#### Edit Netdata's other config files (examples):
113
+
114
+- `$ sudo ./edit-config apps_groups.conf`
115
+- `$ sudo ./edit-config ebpf.conf`
116
+- `$ sudo ./edit-config health.d/load.conf`
117
+- `$ sudo ./edit-config go.d/prometheus.conf`
118
+
119
+#### View the running Netdata configuration: `http://NODE:19999/netdata.conf`
120
+
121
+> Replace `NODE` with the IP address or hostname of your node. Often `localhost`.
122
+
123
+## Alarms & notifications
124
+
125
+#### Add a new alarm
126
+
127
+```
128
+sudo touch health.d/example-alarm.conf
129
+sudo ./edit-config health.d/example-alarm.conf
130
+```
131
+
132
+#### Configure a specific alarm
133
+
134
+```
135
+sudo ./edit-config health.d/example-alarm.conf
136
+```
137
+
138
+#### Silence a specific alarm
139
+
140
+```
141
+sudo ./edit-config health.d/example-alarm.conf
142
+ to: silent
143
+```
144
+
145
+#### Disable alarms and notifications
146
+
147
+```
148
+[health]
149
+ enabled = no
150
+```
151
+
152
+> After any change, reload the Netdata health configuration
153
+
154
+```
155
+netdatacli reload-health
156
+```
157
+
158
+or if that command doesn't work on your installation, use:
159
+
160
+```
161
+killall -USR2 netdata
162
+```
163
+
164
+## Manage the daemon
165
+
166
+| Intent | Action |
167
+| :-------------------------- | --------------------------------------------------------------------: |
168
+| Start Netdata | `$ sudo systemctl start netdata` |
169
+| Stop Netdata | `$ sudo systemctl stop netdata` |
170
+| Restart Netdata | `$ sudo systemctl restart netdata` |
171
+| Reload health configuration | `$ sudo netdatacli reload-health` <br></br> `$ killall -USR2 netdata` |
172
+| View error logs | `less /var/log/netdata/error.log` |
173
+
174
+## See metrics and dashboards
175
+
176
+#### Netdata Cloud: `https://app.netdata.cloud`
177
+
178
+#### Local dashboard: `https://NODE:19999`
179
+
180
+> Replace `NODE` with the IP address or hostname of your node. Often `localhost`.
181
+
182
+#### Access the Netdata API: `http://NODE:19999/api/v1/info`
183
+
184
+## Interact with charts
185
+
186
+| Intent | Action |
187
+| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
188
+| Stop a chart from updating | `click` |
189
+| Zoom | **Cloud** <br/> use the `zoom in` and `zoom out` buttons on any chart (upper right corner) <br/><br/> **Agent**<br/>`SHIFT` or `ALT` + `mouse scrollwheel` <br/> `SHIFT` or `ALT` + `two-finger pinch` (touchscreen) <br/> `SHIFT` or `ALT` + `two-finger scroll` (touchscreen) |
190
+| Zoom to a specific timeframe | **Cloud**<br/>use the `select and zoom` button on any chart and then do a `mouse selection` <br/><br/> **Agent**<br/>`SHIFT` + `mouse selection` |
191
+| Pan forward or back in time | `click` & `drag` <br/> `touch` & `drag` (touchpad/touchscreen) |
192
+| Select a certain timeframe | `ALT` + `mouse selection` <br/> WIP need to evaluate this `command?` + `mouse selection` (macOS) |
193
+| Reset to default auto refreshing state | `double click` |
194
+
195
+## Dashboards
196
+
197
+#### Disable the local dashboard
198
+
199
+Use the `edit-config` script to edit the `netdata.conf` file.
200
+
201
+```
202
+[web]
203
+mode = none
204
+```
205
+
206
+#### Change the port Netdata listens to (port 39999)
207
+
208
+```
209
+[web]
210
+default port = 39999
211
+```
212
+
213
+#### Opt out from anonymous statistics
214
+
215
+```
216
+sudo touch .opt-out-from-anonymous-statistics
217
+```
218
+
219
+## Understanding the dashboard
220
+
221
+**Charts**: A visualization displaying one or more collected/calculated metrics in a time series. Charts are generated
222
+by collectors.
223
+
224
+**Dimensions**: Any value shown on a chart, which can be raw or calculated values, such as percentages, averages,
225
+minimums, maximums, and more.
226
+
227
+**Families**: One instance of a monitored hardware or software resource that needs to be monitored and displayed
228
+separately from similar instances. Example, disks named
229
+**sda**, **sdb**, **sdc**, and so on.
230
+
231
+**Contexts**: A grouping of charts based on the types of metrics collected and visualized.
232
+**disk.io**, **disk.ops**, and **disk.backlog** are all contexts.
docs/cloud/data-privacy.md
new
+39
@@ -0,0 +1,39 @@
1
+<!--
2
+title: "Data privacy in the Netdata Cloud"
3
+description: "Keeping your data safe and secure is our priority.Netdata never stores your personal information in the Netdata Cloud."
4
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/data-privacy.md"
5
+sidebar_label: "Data privacy in the Netdata Cloud"
6
+learn_status: "Published"
7
+learn_topic_type: "Concepts"
8
+learn_rel_path: "Concepts"
9
+-->
10
+
11
+[Data privacy](https://netdata.cloud/privacy/) is very important to us. We firmly believe that your data belongs to
12
+you. This is why **we don't store any metric data in Netdata Cloud**.
13
+
14
+Your local installations of the Netdata Agent form the basis for the Netdata Cloud. All the data that you see in the web browser when using Netdata Cloud, is actually streamed directly from the Netdata Agent to the Netdata Cloud dashboard.
15
+The data passes through our systems, but it isn't stored. You can learn more about [the Agent's security design](https://github.com/netdata/netdata/blob/master/docs/netdata-security.md) in the Agent documentation.
16
+
17
+However, to be able to offer the stunning visualizations and advanced functionality of Netdata Cloud, it does store a limited number of _metadata_.
18
+
19
+## Metadata
20
+
21
+Let's look at the metadata Netdata Cloud stores using the publicly available demo server `frankfurt.my-netdata.io`:
22
+
23
+- The email address you used to sign up/or sign in
24
+- For each node connected to your Spaces in Netdata Cloud:
25
+ - Hostname (as it appears in Netdata Cloud)
26
+ - Information shown in `/api/v1/info`. For example: [https://frankfurt.my-netdata.io/api/v1/info](https://frankfurt.my-netdata.io/api/v1/info).
27
+ - The chart metadata shown in `/api/v1/charts`. For example: [https://frankfurt.my-netdata.io/api/v1/info](https://frankfurt.my-netdata.io/api/v1/info).
28
+ - Alarm configurations shown in `/api/v1/alarms?all`. For example: [https://frankfurt.my-netdata.io/api/v1/alarms?all](https://frankfurt.my-netdata.io/api/v1/alarms?all).
29
+ - Active alarms shown in `/api/v1/alarms`. For example: [https://frankfurt.my-netdata.io/api/v1/alarms](https://frankfurt.my-netdata.io/api/v1/alarms).
30
+
31
+How we use them:
32
+
33
+- The data is stored in our production database on AWS. Some of it is also used in Google BigQuery, our data lake, for analytics purposes. These analytics are crucial for our product development process.
34
+- Email is used to identify users in regards to product use and to enrich our tools with product use, such as our CRM.
35
+- This data is only available to Netdata and never to a 3rd party.
36
+
37
+## Delete all personal data
38
+
39
+To remove all personal info we have about you (email and activities) you need to delete your cloud account by logging into https://app.netdata.cloud and accessing your profile, at the bottom left of your screen.
docs/cloud/insights/anomaly-advisor.md
new
+86
@@ -0,0 +1,86 @@
1
+<!--
2
+title: "Anomaly Advisor"
3
+description: "Quickly find anomalous metrics anywhere in your infrastructure."
4
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/insights/anomaly-advisor.md"
5
+sidebar_label: "Anomaly Advisor"
6
+learn_status: "Published"
7
+learn_topic_type: "Tasks"
8
+learn_rel_path: "Operations"
9
+-->
10
+
11
+import ReactPlayer from 'react-player'
12
+
13
+The Anomaly Advisor feature lets you quickly surface potentially anomalous metrics and charts related to a particular highlight window of
14
+interest.
15
+
16
+<ReactPlayer playing true controls true url='https://user-images.githubusercontent.com/24860547/165943403-1acb9759-7446-4704-8955-c566d04ad7ab.mp4' />
17
+
18
+## Getting Started
19
+
20
+If you are running a Netdata version higher than `v1.35.0-29-nightly` you will be able to use the Anomaly Advisor out of the box with zero configuration. If you are on an earlier Netdata version you will need to first enable ML on your nodes by following the steps below.
21
+
22
+To enable the Anomaly Advisor you must first enable ML on your nodes via a small config change in `netdata.conf`. Once the anomaly detection models have trained on the Agent (with default settings this takes a couple of hours until enough data has been seen to train the models) you will then be able to enable the Anomaly Advisor feature in Netdata Cloud.
23
+
24
+### Enable ML on Netdata Agent
25
+
26
+To enable ML on your Netdata Agent, you need to edit the `[ml]` section in your `netdata.conf` to look something like the following example.
27
+
28
+```bash
29
+[ml]
30
+ enabled = yes
31
+```
32
+
33
+At a minimum you just need to set `enabled = yes` to enable ML with default params. More details about configuration can be found in the [Netdata Agent ML docs](https://learn.netdata.cloud/docs/agent/ml#configuration).
34
+
35
+**Note**: Follow [this guide](https://github.com/netdata/netdata/blob/master/docs/guides/step-by-step/step-04.md) if you are unfamiliar with making configuration changes in Netdata.
36
+
37
+When you have finished your configuration, restart Netdata with a command like `sudo systemctl restart netdata` for the config changes to take effect. You can find more info on restarting Netdata [here](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md).
38
+
39
+After a brief delay, you should see the number of `trained` dimensions start to increase on the "dimensions" chart of the "Anomaly Detection" menu on the Overview page. By default the `minimum num samples to train = 3600` parameter means at least 1 hour of data is required to train initial models, but you could set this to `900` if you want to train initial models quicker but on less data. Over time, they will retrain on up to `maximum num samples to train = 14400` (4 hours by default), but you could increase this is you wanted to train on more data.
40
+
41
+
42
+
43
+Once this line flattens out all configured metrics should have models trained and predicting anomaly scores each second, ready to be used by the new "anomalies" tab of the Anomaly Advisor.
44
+
45
+## Using Anomaly Advisor
46
+
47
+To use the Anomaly Advisor, go to the "anomalies" tab. Once you highlight a particular timeframe of interest, a selection of the most anomalous dimensions will appear below.
48
+
49
+The aim here is to surface the most anomalous metrics in the space or room for the highlighted window to try and cut down on the amount of manual searching required to get to the root cause of your issues.
50
+
51
+
52
+
53
+The "Anomaly Rate" chart shows the percentage of anomalous metrics over time per node. For example, in the following image, 3.21% of the metrics on the "ml-demo-ml-disabled" node were considered anomalous. This elevated anomaly rate could be a sign of something worth investigating.
54
+
55
+**Note**: in this example the anomaly rates for this node are actually being calculated on the parent it streams to, you can run ml on the Agent itselt or on a parent the Agent stream to. Read more about the various configuration options in the [Agent docs](https://github.com/netdata/netdata/blob/master/ml/README.md).
56
+
57
+
58
+
59
+The "Count of Anomalous Metrics" chart (collapsed by default) shows raw counts of anomalous metrics per node so may often be similar to the anomaly rate chart, apart from where nodes may have different numbers of metrics.
60
+
61
+The "Anomaly Events Detected" chart (collapsed by default) shows if the anomaly rate per node was sufficiently elevated to trigger a node level anomaly. Anomaly events will appear slightly after the anomaly rate starts to increase in the timeline, this is because a significant number of metrics in the node need to be anomalous before an anomaly event is triggered.
62
+
63
+Once you have highlighted a window of interest, you should see an ordered list of anomaly rate sparklines in the "Anomalous metrics" section like below.
64
+
65
+
66
+
67
+You can expand any sparkline chart to see the underlying raw data to see how it relates to the corresponding anomaly rate.
68
+
69
+
70
+
71
+On the upper right hand side of the page you can select which nodes to filter on if you wish to do so. The ML training status of each node is also displayed.
72
+
73
+On the lower right hand side of the page an index of anomaly rates is displayed for the highlighted timeline of interest. The index is sorted from most anomalous metric (highest anomaly rate) to least (lowest anomaly rate). Clicking on an entry in the index will scroll the rest of the page to the corresponding anomaly rate sparkline for that metric.
74
+
75
+### Usage Tips
76
+
77
+- If you are interested in a subset of specific nodes then filtering to just those nodes before highlighting tends to give better results. This is because when you highlight a region, Netdata Cloud will ask the Agents for a ranking over all metrics so if you can filter this early to just the subset of nodes you are interested in, less 'averaging' will occur and so you might be a less noisy ranking.
78
+- Ideally try and highlight close to a spike or window of interest so that the resulting ranking can narrow in more easily on the timeline you are interested in.
79
+
80
+You can read more detail on how anomaly detection in the Netdata Agent works in our [Agent docs](https://github.com/netdata/netdata/blob/master/ml/README.md).
81
+
82
+🚧 **Note**: This functionality is still **under active development** and considered experimental. We dogfood it internally and among early adopters within the Netdata community to build the feature. If you would like to get involved and help us with feedback, you can reach us through any of the following channels:
83
+- Email us at analytics-ml-team@netdata.cloud
84
+- Comment on the [beta launch post](https://community.netdata.cloud/t/anomaly-advisor-beta-launch/2717) in the Netdata community
85
+- Join us in the [🤖-ml-powered-monitoring](https://discord.gg/4eRSEUpJnc) channel of the Netdata discord.
86
+- Or open a discussion in GitHub if that's more your thing
docs/cloud/manage/sign-in.md
new
+88
@@ -0,0 +1,88 @@
1
+<!--
2
+title: "Sign in with email, Google, or GitHub"
3
+description: "Learn how signing in to Cloud works via one of our three authentication methods, plus some tips if you're having trouble signing in."
4
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/cloud/manage/sign-in.md"
5
+sidebar_label: "Sign in with email, Google, or GitHub"
6
+learn_status: "Published"
7
+learn_topic_type: "Tasks"
8
+learn_rel_path: "Operations"
9
+-->
10
+
11
+You can [sign in to Netdata](https://app.netdata.cloud/sign-in?cloudRoute=spaces?utm_source=docs&utm_content=sign_in_button_first_section) through one of three methods: email, Google, or GitHub. Email uses a
12
+time-sensitive link that authenticates your browser, and Google/GitHub both use OAuth to associate your email address
13
+with a Netdata Cloud account.
14
+
15
+No matter the method, your Netdata Cloud account is based around your email address. Netdata Cloud does not store
16
+passwords.
17
+
18
+
19
+## Email
20
+
21
+To sign in with email, visit [Netdata Cloud](https://app.netdata.cloud/sign-in?cloudRoute=spaces?utm_source=docs&utm_content=sign_in_button_email_section), enter your email address, and click
22
+the **Sign in by email** button.
23
+
24
+
25
+
26
+Click the **Verify** button in the email to begin using Netdata Cloud.
27
+
28
+To use this same Netdata Cloud account on additional devices, request another sign in email, open the email on that
29
+device, and sign in.
30
+
31
+### Don't have a Netdata Cloud account yet?
32
+
33
+If you don't have a Netdata Cloud account yet you won't need to worry about it. During the sign in process we will create one for you and make the process seamless to you.
34
+
35
+After your account is created and you sign in to Netdata, you first are asked to agree to Netdata Cloud's [Privacy
36
+Policy](https://www.netdata.cloud/privacy/) and [Terms of Use](https://www.netdata.cloud/terms/). Once you agree with these you are directed
37
+through the Netdata Cloud onboarding process, which is explained in the [Netdata Cloud
38
+quickstart](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md).
39
+
40
+### Troubleshooting
41
+
42
+You should receive your sign in email in less than a minute. The subject is **Verify your email!** and the sender is `no-reply@app.netdata.cloud` via `sendgrid.net`.
43
+
44
+If you don't see the email, try the following:
45
+
46
+- Check [Netdata Cloud status](https://status.netdata.cloud) for ongoing issues with our infrastructure.
47
+- Request another sign in email via the [sign in page](https://app.netdata.cloud/sign-in?cloudRoute=spaces?utm_source=docs&utm_content=sign_in_button_troubleshooting_section).
48
+- Check your spam folder.
49
+- In Gmail, check the **Updates** category.
50
+
51
+You may also want to add `no-reply@app.netdata.cloud` to your address book or contacts list, especially if you're using
52
+a public email service, such as Gmail. You may also want to whitelist/allowlist either the specific email or the entire
53
+`app.netdata.cloud` domain.
54
+
55
+## Google and GitHub OAuth
56
+
57
+When you use Google/GitHub OAuth, your Netdata Cloud account is associated with the email address that Netdata Cloud
58
+receives via OAuth.
59
+
60
+To sign in with Google or GitHub OAuth, visit [Netdata Cloud](https://app.netdata.cloud/sign-in?cloudRoute=spaces?utm_source=docs&utm_content=sign_in_button_google_github_section) and click the
61
+**Continue with Google/GitHub** or button. Enter your Google/GitHub username and your password. Complete two-factor
62
+authentication if you or your organization has it enabled.
63
+
64
+You are then signed in to Netdata Cloud or directed to the new-user onboarding if you have not signed up previously.
65
+
66
+## Reset a password
67
+
68
+Netdata Cloud does not store passwords and does not support password resets. All of our sign in methods do not
69
+require passwords, and use either links in emails or Google/GitHub OAuth for authentication.
70
+
71
+## Switch between sign in methods
72
+
73
+You can switch between sign in methods if the email account associated with each method is the same.
74
+
75
+For example, you first sign in via your email account, `user@example.com`, and later sign out. You later attempt to sign
76
+in via a GitHub account associated with `user@example.com`. Netdata Cloud recognizes that the two are the same and signs
77
+you in to your original account.
78
+
79
+However, if you first sign in via your `user@example.com` email account and then sign in via a Google account associated
80
+with `user2@example.com`, Netdata Cloud creates a new account and begins the onboarding process.
81
+
82
+It is not currently possible to link an account created with `user@example.com` to a Google account associated with
83
+`user2@example.com`.
84
+
85
+## What's next?
86
+
87
+If you haven't already onboarded to Netdata Cloud and connected your first nodes, visit
88
+the [Get Started section of our documentation](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md).
docs/dashboard/customize.md
new
+99
@@ -0,0 +1,99 @@
1
+<!--
2
+title: "Customize the standard dashboard"
3
+description: >-
4
+ "Netdata's preconfigured dashboard offers many customization options, such as choosing when
5
+ charts are updated, your preferred theme, and custom text to document processes, and more."
6
+type: "how-to"
7
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md"
8
+sidebar_label: "Customize the standard dashboard"
9
+learn_status: "Published"
10
+learn_topic_type: "Tasks"
11
+learn_rel_path: "Operations"
12
+-->
13
+
14
+# Customize the standard dashboard
15
+
16
+While the [Netdata dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md) comes preconfigured with hundreds of charts and
17
+thousands of metrics, you may want to alter your experience based on a particular use case or preferences.
18
+
19
+## Dashboard settings
20
+
21
+To change dashboard settings, click the on the **settings** icon 
23
+in the top panel.
24
+
25
+These settings only affect how the dashboard behaves in your browser. They take effect immediately and are permanently
26
+saved to browser local storage (except the refresh on focus / always option). Some settings are applied immediately, and
27
+others are only reflected after the dashboard is refreshed, which happens automatically.
28
+
29
+Here are a few popular settings:
30
+
31
+### Change chart legend position
32
+
33
+Find this setting under the **Visual** tab. By default, Netdata places the [legend of dimensions](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md#dimension) _below_ charts.
34
+Click this toggle to move the legend to the _right_ of charts.
35
+
36
+
37
+### Change theme
38
+
39
+Find this setting under the **Visual** tab. Choose between Dark (the default) and White.
40
+
41
+## Customize the standard dashboard info
42
+
43
+Netdata stores information about individual charts in the `dashboard_info.js` file. This file includes section and
44
+subsection headings, descriptions, colors, titles, tooltips, and other information for Netdata to render on the
45
+dashboard.
46
+
47
+One common use case for customizing the standard dashboard is adding internal "documentation" a section or specific
48
+chart that can then be read by anyone with access to that dashboard.
49
+
50
+For example, here is how `dashboard_info.js` defines the **System Overview** section.
51
+
52
+```javascript
53
+netdataDashboard.menu = {
54
+ 'system': {
55
+ title: 'System Overview',
56
+ icon: '<i class="fas fa-bookmark"></i>',
57
+ info: 'Overview of the key system metrics.'
58
+ },
59
+```
60
+
61
+If you want to customize this information, use the example `dashboard_info_custom_example.js` as a starting point.
62
+First, navigate to the web server's directory. If you're on a Linux system, this should be at `/usr/share/netdata/web/`.
63
+Copy the example file, then ensure that its permissions match the rest of the web server, which is `netdata:netdata` by
64
+default.
65
+
66
+```bash
67
+cd /usr/share/netdata/web/
68
+sudo cp dashboard_info_custom_example.js your_dashboard_info_file.js
69
+sudo chown netdata:netdata your_dashboard_info_file.js
70
+```
71
+
72
+Edit the file with customizations to the `title`, `icon`, and `info` fields. Replace the string after `fas fa-` with any
73
+icon from [Font Awesome](https://fontawesome.com/cheatsheet) to customize the icons that appear throughout the
74
+dashboard.
75
+
76
+Save the file, then navigate to your [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md) to edit `netdata.conf`. Add
77
+the following line to the `[web]` section to tell Netdata where to find your custom configuration.
78
+
79
+```conf
80
+[web]
81
+ custom dashboard_info.js = your_dashboard_info_file.js
82
+```
83
+
84
+Reload your browser tab to see your custom configuration.
85
+
86
+## What's next?
87
+
88
+If you're keen on continuing to customize your Netdata experience, check out our docs on [building new custom
89
+dashboards](https://github.com/netdata/netdata/blob/master/web/gui/custom/README.md) with HTML, CSS, and JavaScript.
90
+
91
+### Further reading & related information
92
+
93
+- Dashboard
94
+ - [How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)
95
+ - [Interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md)
96
+ - [Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)
97
+ - [Select timeframes to visualize](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md)
98
+ - [Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)
99
+ - **[Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)**
docs/dashboard/dimensions-contexts-families.md
new
+102
@@ -0,0 +1,102 @@
1
+<!--
2
+title: "Chart dimensions, contexts, and families"
3
+description: >-
4
+ "Netdata organizes charts into dimensions, contexts, and families to automatically
5
+ and meaningfully organize thousands of metrics into interactive charts."
6
+type: "explanation"
7
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md"
8
+sidebar_label: "Chart dimensions, contexts, and families"
9
+learn_status: "Published"
10
+learn_topic_type: "Concepts"
11
+learn_rel_path: "Concepts"
12
+-->
13
+
14
+# Chart dimensions, contexts, and families
15
+
16
+While Netdata's charts require no configuration and are [easy to interact with](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md),
17
+they have a lot of underlying complexity. To meaningfully organize charts out of the box based on what's happening in
18
+your nodes, Netdata uses the concepts of **dimensions**, **contexts**, and **families**.
19
+
20
+Understanding how these work will help you more easily navigate the dashboard, [write new
21
+alarms](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md), or play around with the [API](https://github.com/netdata/netdata/blob/master/web/api/README.md).
22
+
23
+For a refresher on the anatomy of a chart, see [dashboards and charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md).
24
+
25
+## Dimension
26
+
27
+A **dimension** is a value that gets shown on a chart. The value can be raw data or calculated values, such as the
28
+average (the default), minimum, or maximum. These values can then be given any type of unit. For example, CPU
29
+utilization is represented as a percentage, disk I/O as `MiB/s`, and available RAM as an absolute value in `MiB` or
30
+`GiB`.
31
+
32
+Beneath every chart (or on the right-side if you configure the dashboard) is a legend of dimensions. When there are
33
+multiple dimensions, you'll see a different entry in the legend for each dimension.
34
+
35
+The **Apps CPU Time** chart (with the [context](#context) `apps.cpu`), which visualizes CPU utilization of
36
+different types of processes/services/applications on your node, always provides a vibrant example of a chart with
37
+multiple dimensions.
38
+
39
+
41
+
42
+The chart shows 13 unique dimensions, such as `httpd` for the CPU utilization for web servers, `kernel` for anything
43
+related to the Linux kernel, and so on. In your dashboard, these specific dimensions will almost certainly be different.
44
+
45
+Dimensions can be [hidden](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md#show-and-hide-dimensions) to help you focus your
46
+attention.
47
+
48
+## Context
49
+
50
+A **context** is a way of grouping charts by the types of metrics collected and dimensions displayed. It's kind of like
51
+a machine-readable naming and organization scheme.
52
+
53
+For example, the **Apps CPU Time** has the context `apps.cpu`. A little further down on the dashboard is a similar
54
+chart, **Apps Real Memory (w/o shared)** with the context `apps.mem`. The `apps` portion of the context is the **type**,
55
+whereas anything after the `.` is specified either by the chart's developer or by the [**family**](#family).
56
+
57
+By default, a chart's type affects where it fits in the menu, while its family creates submenus.
58
+
59
+Netdata also relies on contexts for [alarm configuration](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md) (the [`on`
60
+line](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alarm-line-on)).
61
+
62
+## Family
63
+
64
+**Families** are a _single instance_ of a hardware or software resource that needs to be displayed separately from
65
+similar instances.
66
+
67
+For example, let's look at the **Disks** section, which contains a number of charts with contexts like `disk.io`,
68
+`disk.ops`, `disk.backlog`, and `disk.util`. If your node has multiple disk drives at `sda` and `sdb`, Netdata creates
69
+a separate family for each.
70
+
71
+Netdata now merges the contexts and families to create charts that are grouped by family, following a
72
+`[context].[family]` naming scheme, so that you can see the `disk.io` and `disk.ops` charts for `sda` right next to each
73
+other.
74
+
75
+Given the four example contexts, and two families of `sda` and `sdb`, Netdata will create the following charts and their
76
+names:
77
+
78
+| Context | `sda` family | `sdb` family |
79
+| :------------- | ------------------ | ------------------ |
80
+| `disk.io` | `disk_io.sda` | `disk_io.sdb` |
81
+| `disk.ops` | `disk_ops.sda` | `disk_ops.sdb` |
82
+| `disk.backlog` | `disk_backlog.sda` | `disk_backlog.sdb` |
83
+| `disk.util` | `disk_util.sda` | `disk_util.sdb` |
84
+
85
+## What's next?
86
+
87
+With an understanding of a chart's dimensions, context, and family, you're now ready to dig even deeper into Netdata's
88
+dashboard. We recommend looking into [using the timeframe selector](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md).
89
+
90
+If you feel comfortable with the [dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md) and interacting with charts, we
91
+recommend learning about [configuration](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md). While Netdata doesn't _require_ a complicated setup
92
+process or a query language to create charts, there are a lot of ways to tweak the experience to match your needs.
93
+
94
+### Further reading & related information
95
+
96
+- Dashboard
97
+ - [How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)
98
+ - [Interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md)
99
+ - **[Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)**
100
+ - [Select timeframes to visualize](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md)
101
+ - [Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)
102
+ - [Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)
docs/dashboard/how-dashboard-works.md
new
+118
@@ -0,0 +1,118 @@
1
+<!--
2
+title: "How the dashboard works"
3
+description: >-
4
+ "Learn how to navigate Netdata's preconfigured dashboard to get started
5
+ exploring, visualizing, and troubleshooting in real time."
6
+type: "explanation"
7
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md"
8
+sidebar_label: "How the dashboard works"
9
+learn_status: "Published"
10
+learn_topic_type: "Concepts"
11
+learn_rel_path: "Concepts"
12
+-->
13
+
14
+# How the dashboard works
15
+
16
+Because Netdata is a monitoring and _troubleshooting_ platform, a dashboard with real-time, meaningful, and
17
+context-aware charts is essential.
18
+
19
+As soon as you [install Netdata](https://github.com/netdata/netdata/blob/master/packaging/installer/README.md), it autodetects hardware, OS, containers, services, and
20
+applications running on your node and builds a dashboard on a single, scrollable webpage. This page features hundreds of
21
+charts, which are preconfigured to save you time from learning a query language, all stacked on top of one another. This
22
+vertical rhythm is designed to encourage exploration and help you visually identify connections between the metrics
23
+visualized in different charts.
24
+
25
+It's essential to understand the core concepts and features of Netdata's dashboard if you want to maximize your Netdata
26
+experience right after installation.
27
+
28
+## Open the dashboard
29
+
30
+Access Netdata's dashboard by navigating to `http://NODE:19999` in your browser, replacing `NODE` with either
31
+`localhost` or the hostname/IP address of a remote node.
32
+
33
+
35
+
36
+Many features of the internal web server that serves the dashboard are [configurable](https://github.com/netdata/netdata/blob/master/web/server/README.md), including
37
+the listen port, enforced TLS, and even disabling the dashboard altogether.
38
+
39
+## Sections and menus
40
+
41
+As mentioned in the introduction, Netdata automatically organizes all the metrics it collects from your node, and places
42
+them into **sections** of closely related charts.
43
+
44
+The first section on any dashboard is the **System Overview**, followed by **CPUs**, **Memory**, and so on.
45
+
46
+These sections populate the **menu**, which is on the right-hand side of the dashboard. Instead of manually scrolling up
47
+and down to explore the dashboard, it's generally faster to click on the relevant menu item to jump to that position on
48
+the dashboard.
49
+
50
+Many menu items also contain a **submenu**, with links to additional categories. For example, the **Disks** section is often separated into multiple groups based on the number of disk drives/partitions on your node, which are also known as a family.
51
+
52
+
54
+
55
+## Charts
56
+
57
+Every **chart** in the Netdata dashboard is [fully interactive](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md). Netdata
58
+synchronizes your interactions to help you understand exactly how a node behaved in any timeframe, whether that's
59
+seconds or days.
60
+
61
+A chart is an individual, interactive, always-updating graphic displaying one or more collected/calculated metrics,
62
+which are generated by [collectors](https://github.com/netdata/netdata/blob/master/collectors/README.md).
63
+
64
+
66
+
67
+Hover over any chart to temporarily pause it and see the exact metrics values presented as different dimensions. Click
68
+or tap to stop the chart from automatically updating with new metrics, thereby locking it to a single timeframe.
69
+Double-click it to resume auto-updating.
70
+
71
+Let's cover two of the most important ways to interact with charts: panning through time and zooming.
72
+
73
+To pan through time, **click and hold** (or touch and hold) on any chart, then **drag your mouse** (or finger) to the
74
+left or right. Drag to the right to pan backward through time, or drag to the left to pan forward in time. Think of it
75
+like pushing the current timeframe off the screen to see what came before or after.
76
+
77
+To zoom, press and hold `Shift`, then use your mouse's scroll wheel, or a two-finger pinch if you're using a touchpad.
78
+
79
+See [interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md) for all the possible ways to interact with the charts on
80
+your dashboard.
81
+
82
+## Alarms
83
+
84
+Many of the preconfigured charts on the Netdata dashboard also come with preconfigured alarms. Netdata sends three
85
+primary alarm states via alarms: `CLEAR`, `WARNING`, and `CRITICAL`. If an alarm moves from a `CLEAR` state to either
86
+`WARNING` or `CRITICAL`, Netdata creates a notification to let you know exactly what's going on. There are [other alarm
87
+states](https://github.com/netdata/netdata/blob/master/health/REFERENCE.md#alarm-statuses) as well.
88
+
89
+The easiest way to see alarms is by clicking on the alarm icon 
91
+in the top panel to open the alarms panel, which shows you all the active alarms. The other **All** tab shows every
92
+active alarm, and the **Log** tab shows a historical record of exactly when alarms triggered and to which state.
93
+
94
+
96
+
97
+Learn more about [viewing active alarms](https://github.com/netdata/netdata/blob/master/docs/monitor/view-active-alarms.md), [configuring
98
+alarms](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md), or [enabling a new notification
99
+method](https://github.com/netdata/netdata/blob/master/docs/monitor/enable-notifications.md).
100
+
101
+## What's next?
102
+
103
+Learn more about [interacting with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md) to quickly pan through time, zoom, and
104
+show/hide dimensions to best understand the state of your node in any timeframe. A complete understanding of [chart
105
+dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md) will also help with how Netdata
106
+organizes its dashboard and operates [alarms](https://github.com/netdata/netdata/blob/master/docs/monitor/configure-alarms.md).
107
+
108
+### Further reading & related information
109
+
110
+- Dashboard
111
+ - **[How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)**
112
+ - [Interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md)
113
+ - [Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)
114
+ - [Select timeframes to visualize](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md)
115
+ - [Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)
116
+ - [Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)
117
+- [HTTP API](https://github.com/netdata/netdata/blob/master/web/api/README.md)
118
+- [Custom dashboards](https://github.com/netdata/netdata/blob/master/web/gui/custom/README.md)
docs/dashboard/import-export-print-snapshot.md
new
+90
@@ -0,0 +1,90 @@
1
+<!--
2
+title: "Import, export, and print a snapshot"
3
+description: >-
4
+ "Snapshots can be incredibly useful for diagnosing anomalies after
5
+ they've already happened, and are interoperable with any other node
6
+ running Netdata."
7
+type: "how-to"
8
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md"
9
+sidebar_label: "Import, export, and print a snapshot"
10
+learn_status: "Published"
11
+learn_topic_type: "Tasks"
12
+learn_rel_path: "Operations"
13
+-->
14
+
15
+# Import, export, and print snapshots
16
+
17
+Netdata can export snapshots of the contents of your dashboard at a given time, which you can then import into any other
18
+node running Netdata. Or, you can create a print-ready version of your dashboard to save to PDF or actually print to
19
+paper.
20
+
21
+Snapshots can be incredibly useful for diagnosing anomalies after they've already happened. Let's say Netdata triggered a warning alarm while you were asleep. In the morning, you can [select the
22
+timeframe](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md) when the alarm triggered, export a snapshot, and send it to a
23
+
24
+colleague for further analysis.
25
+
26
+Or, send the Netdata team a snapshot of your dashboard when [filing a bug
27
+report](https://github.com/netdata/netdata/issues/new?assignees=&labels=bug%2Cneeds+triage&template=BUG_REPORT.yml) on
28
+GitHub.
29
+
30
+
32
+
33
+## Import a snapshot
34
+
35
+To import a snapshot, click on the **import** icon 
37
+in the top panel.
38
+
39
+Select the Netdata snapshot file to import. Once the file is loaded, the modal updates with information about the
40
+snapshot and the system from which it was taken. Click **Import** to begin to process.
41
+
42
+Netdata takes the data embedded inside the snapshot and re-creates a static replica on your dashboard. When the import
43
+finishes, you're free to move around and examine the charts.
44
+
45
+Some caveats and tips to keep in mind:
46
+
47
+- Only metrics in the export timeframe are available to you. If you zoom out or pan through time, you'll see the
48
+ beginning and end of the snapshot.
49
+- Charts won't update with new information, as you're looking at a static replica, not the live dashboard.
50
+- The import is only temporary. Reload your browser tab to return to your node's real-time dashboard.
51
+
52
+## Export a snapshot
53
+
54
+To export a snapshot, first pan/zoom any chart to an appropriate _visible timeframe_. The export snapshot will only
55
+contain the metrics you see in charts, so choose the most relevant timeframe.
56
+
57
+Next, click on the **export** icon 
59
+in the top panel.
60
+
61
+Select the metrics resolution to export. The default is 1-second, equal to how often Netdata collects and stores
62
+metrics. Lowering the resolution will reduce the number of data points, and thus the snapshot's overall size.
63
+
64
+Edit the snapshot file name and select your desired compression method. Click on **Export**. When the export is
65
+complete, your browser will prompt you to save the `.snapshot` file to your machine.
66
+
67
+## Print a snapshot
68
+
69
+To print a snapshot, click on the **print** icon 
71
+in the top panel.
72
+
73
+When you click **Print**, Netdata opens a new window to render every chart. This might take some time. When finished,
74
+Netdata opens a browser print dialog for you to save to PDF or print.
75
+
76
+## What's next?
77
+
78
+Now that you understand snapshots, now is a good time to delve deeper into some of the dashboard's lesser-known
79
+features, such as [customization](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md) or [building new, custom
80
+dashboards](https://github.com/netdata/netdata/blob/master/web/gui/custom/README.md).
81
+
82
+### Further reading & related information
83
+
84
+- Dashboard
85
+ - [How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)
86
+ - [Interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md)
87
+ - [Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)
88
+ - [Select timeframes to visualize](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md)
89
+ - **[Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)**
90
+ - [Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)
\ No newline at end of file
docs/dashboard/interact-charts.md
new
+201
@@ -0,0 +1,201 @@
1
+<!--
2
+title: "Interact with charts"
3
+description: "Learn how to pan, zoom, select, and customize Netdata's preconfigured charts to help you troubleshooting with real-time, per-second metrics data."
4
+type: "how-to"
5
+custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/dashboard/interact-charts.md"
6
+sidebar_label: "Interact with charts"
7
+learn_status: "Published"
8
+learn_topic_type: "Tasks"
9
+learn_rel_path: "Operations"
10
+-->
11
+
12
+# Interact with charts
13
+
14
+> ⚠️ There is a new version of charts that is currently **only** available on [Netdata Cloud](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/interact-new-charts.md). We didn't
15
+> want to keep this valuable feature from you, so after we get this into your hands on the Cloud, we will collect and implement your feedback to make sure we are providing the best possible version of the feature on the Netdata Agent dashboard as quickly as possible.
16
+
17
+While charts that update every second with new metrics are helpful for understanding the immediate state of a node, deep
18
+troubleshooting and root cause analysis begins by manipulating the default charts. To help you troubleshoot, Netdata
19
+synchronizes every chart every time you interact with one of them.
20
+
21
+Here's what synchronization looks like:
22
+
23
+
25
+
26
+Once you understand all the interactions available to you, you'll be able to quickly move around the dashboard, search
27
+for anomalies, and find root causes using per-second metrics.
28
+
29
+## Pause or stop
30
+
31
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
32
+| :---------------- | :------------- | :------------------- |
33
+| **Pause** a chart | `hover` | `n/a` |
34
+| **Stop** a chart | `click` | `tap` |
35
+
36
+By hovering over any chart, you temporarily pause it so that you can hover over a specific timeframe and see the exact
37
+values presented as dimensions. Click on the chart to lock it to this timeframe, which is useful if you want to jump to
38
+a different chart to look for possible correlations.
39
+
40
+
42
+
43
+## Pan
44
+
45
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
46
+| :---------- | :------------- | :------------------- |
47
+| **Pan** | `click + drag` | `swipe` |
48
+
49
+Drag your mouse/finger to the right to pan backward through time, or drag to the left to pan forward in time. Think of
50
+it like pushing the current timeframe off the screen to see what came before or after.
51
+
52
+## Zoom
53
+
54
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
55
+| :------------------------------- | :-------------------------- | :--------------------------------------------------- |
56
+| **Zoom** in or out | `Shift + mouse scrollwheel` | `two-finger pinch` <br />`Shift + two-finger scroll` |
57
+| **Zoom** to a specific timeframe | `Shift + mouse selection` | `n/a` |
58
+
59
+Zooming in helps you see metrics with maximum granularity, which is useful when you're trying to diagnose the root cause
60
+of an anomaly or outage. Zooming out lets you see metrics within the larger context, such as the last hour, day, or
61
+week, which is useful in understanding what "normal" looks like, or to identify long-term trends, like a slow creep in
62
+memory usage.
63
+
64
+## Select
65
+
66
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
67
+| :------------------------------ | :-------------------------------------------------------- | :------------------- |
68
+| **Select** a specific timeframe | `Alt + mouse selection` or `⌘ + mouse selection` (macOS) | `n/a` |
69
+
70
+Selecting timeframes is useful when you see an interesting spike or change in a chart and want to investigate further.
71
+
72
+Select a timeframe, then move to different charts/sections of the dashboard. Each chart shows the same selection to help
73
+you immediately identify the timeframe and look for correlations.
74
+
75
+## Reset a chart to its default state
76
+
77
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
78
+| :---------------- | :------------- | :------------------- |
79
+| **Reset** a chart | `double-click` | `n/a` |
80
+
81
+Double-check on a chart to restore it to the default auto-updating state, with a timeframe based on your browser
82
+viewport.
83
+
84
+## Resize
85
+
86
+Click-and-drag the icon on the bottom-right corner of any chart. To restore the chart to its original height,
87
+double-click the same icon.
88
+
89
+
91
+
92
+## Show and hide dimensions
93
+
94
+| Interaction | Keyboard/mouse | Touchpad/touchscreen |
95
+| :------------------------------------- | :-------------- | :------------------- |
96
+| **Show one** dimension and hide others | `click` | `tap` |
97
+| **Toggle (show/hide)** one dimension | `Shift + click` | `n/a` |
98
+
99
+Hiding dimensions simplifies the chart and can help you better discover exactly which aspect of your system might be
100
+behaving strangely.
101
+
102
+## See the context
103
+
104
+Hover your mouse over the date that appears just beneath the chart itself. A tooltip will tell you the context for that
105
+chart. Below, the context is `apps.cpu`.
106
+
107
+
109
+
110
+## See the resolution and update frequency
111
+
112
+Hover your mouse over the timestamp just to the right of the date. `resolution` is the number of seconds between each
113
+"tick" in the chart. `collection every` is how often Netdata collects and stores that metric.
114
+
115
+If the `resolution` value is higher than `collection every`, such as `resolution 5 secs, collected every 1 sec`, this
116
+means that each tick is calculating represents the average values across a 5-second period. You can zoom in to increase
117
+the resolution to `resolution 1 sec` to see the exact values.
118
+
119
+## Chart controls
120
+
121
+Many of the above interactions can also be triggered using the icons on the bottom-right corner of every chart. They
122
+are, respectively, `Pan Left`, `Reset`, `Pan Right`, `Zoom In`, and `Zoom Out`.
123
+
124
+## Chart label filtering
125
+
126
+The chart label filtering feature supports grouping by and filtering each chart based on labels (key/value pairs) applicable to the context and provides fine-grain capability on slicing the data and metrics.
127
+
128
+All metrics collected get "tagged" with labels and values, thus providing a powerful way of slicing and visualizing all metrics related to the infrastructure.
129
+
130
+The chart label filtering is currently enabled on:
131
+
132
+- All charts on the **Overview** tab
133
+- Custom dashboards
134
+
135
+
136
+
137
+
138
+The top panel on each chart displays the various filters and grouping options selected on the specific chart. These filters are specific for each chart and need to be manually configured on each chart.
139
+
140
+Additionally, the charts can be saved to a custom dashboard, new or existing, with the selected filters from the overview screen.
141
+
142
+
143
+
144
+## Custom labels for Collectors
145
+
146
+In addition to the default labels associated with a collector and metrics context (you can identify them by seeing which ones have an underscore as a prefix), there is now a new feature enabled to create custom labels. These custom labels may be needed to group your jobs or instances into various categories.
147
+
148
+These custom labels can be configured within your go.d plugins by simply associating a label key/value pair, as in the following eaxmple.
149
+
150
+```conf
151
+jobs:
152
+ - name: example_1
153
+ someOption: someValue
154
+ labels:
155
+ label1: value1
156
+ label2: value2
157
+ - name: example_2
158
+ someOption: someValue
159
+ labels:
160
+ label3: value3
161
+ label4: value4
162
+```
163
+
164
+For instance, you may be running multiple Postgres database instances within an infrastructure. Some of these may be associated with testing environments, some with staging and some with production environments. You can now associate each Postgres job / instance with a custom label. The “group by” and filtering options will then allow you to associate individual jobs by specific labels.
165
+
166
+```conf
167
+jobs:
168
+ - name: local
169
+ dsn: 'postgres://postgres:postgres@127.0.0.1:5432/postgres'
170
+ collect_databases_matching: '*'
171
+ labels:
172
+ instance_type: production
173
+ ```
174
+ 
175
+ 
176
+
177
+### Future Work
178
+
179
+We already have [configurable host labels](https://github.com/netdata/netdata/blob/master/docs/guides/using-host-labels.md) as well, which currently can’t be used to filter or group your metrics. We intend to provide the same capabilities described here with host labels, among other capabilities on other areas of the app as well
180
+
181
+## What's next?
182
+
183
+We recommend you read up on the differences between [chart dimensions, contexts, and
184
+families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md) to complete your understanding of how Netdata organizes its
185
+dashboards. Another valuable way to interact with charts is to use the [timeframe
186
+selector](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md), which helps you visualize specific moments of historical metrics.
187
+
188
+If you feel comfortable with the [dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md) and interacting with charts, we
189
+recommend moving on to learning about [configuration](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md). While Netdata doesn't _require_ a
190
+complicated setup process or a query language to create charts, there are a lot of ways to tweak the experience to match
191
+your needs.
192
+
193
+### Further reading & related information
194
+
195
+- Dashboard
196
+ - [How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)
197
+ - [Netdata Cloud · Interact with new charts](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/interact-new-charts.md)
198
+ - [Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)
199
+ - [Select timeframes to visualize](https://github.com/netdata/netdata/blob/master/docs/dashboard/visualization-date-and-time-controls.md)
200
+ - [Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)
201
+ - [Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)
docs/dashboard/reference-web-server.md
new
+282
@@ -0,0 +1,282 @@
1
+<!--
2
+title: "Web server reference"
3
+description: "The Netdata Agent's local static-threaded web server serves dashboards and real-time visualizations with security and DDoS protection."
4
+type: reference
5
+custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/dashboard/reference-web-server.md
6
+sidebar_label: "Web server reference"
7
+learn_status: "Published"
8
+learn_topic_type: "References"
9
+learn_rel_path: "References"
10
+-->
11
+
12
+# Web server reference
13
+
14
+The Netdata web server is `static-threaded`, with a fixed, configurable number of threads.
15
+
16
+All the threads are concurrently listening for web requests on the same sockets, and the kernel distributes the incoming
17
+requests to them. Each thread uses non-blocking I/O so it can serve any number of web requests in parallel.
18
+
19
+This web server respects the `keep-alive` HTTP header to serve multiple HTTP requests via the same connection.
20
+
21
+## Configuration
22
+
23
+From within your Netdata config directory (typically `/etc/netdata`), [use `edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md) to
24
+open `netdata.conf`.
25
+
26
+```
27
+sudo ./edit-config netdata.conf
28
+```
29
+
30
+Scroll down to the `[web]` section to find the following settings.
31
+
32
+## Settings
33
+
34
+| Setting | Default | Description |
35
+|:-------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
36
+| `ssl key` | `/etc/netdata/ssl/key.pem` | Declare the location of an SSL key to [enable HTTPS](#enable-httpstls-support). |
37
+| `ssl certificate` | `/etc/netdata/ssl/cert.pem` | Declare the location of an SSL certificate to [enable HTTPS](#enable-httpstls-support). |
38
+| `tls version` | `1.3` | Choose which TLS version to use. While all versions are allowed (`1` or `1.0`, `1.1`, `1.2` and `1.3`), we recommend `1.3` for the most secure encryption. If left blank, Netdata uses the highest available protocol version on your system. |
39
+| `tls ciphers` | `none` | Choose which TLS cipher to use. Options include `TLS_AES_256_GCM_SHA384`, `TLS_CHACHA20_POLY1305_SHA256`, and `TLS_AES_128_GCM_SHA256`. If left blank, Netdata uses the default cipher list for that protocol provided by your TLS implementation. |
40
+| `ses max window` | `15` | See [single exponential smoothing](https://github.com/netdata/netdata/blob/master/web/api/queries/ses/README.md). |
41
+| `des max window` | `15` | See [double exponential smoothing](https://github.com/netdata/netdata/blob/master/web/api/queries/des/README.md). |
42
+| `mode` | `static-threaded` | Turns on (`static-threaded` or off (`none`) the static-threaded web server. See the [example](#disable-the-web-server) to turn off the web server and disable the dashboard. |
43
+| `listen backlog` | `4096` | The port backlog. Check `man 2 listen`. |
44
+| `default port` | `19999` | The listen port for the static web server. |
45
+| `web files owner` | `netdata` | The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, Netdata will only serve files owned by user given in `run as user`. |
46
+| `web files group` | `netdata` | If this is set, Netdata will check if the file is owned by this group and refuse to serve the file if it's not. |
47
+| `disconnect idle clients after seconds` | `60` | The time in seconds to disconnect web clients after being totally idle. |
48
+| `timeout for first request` | `60` | How long to wait for a client to send a request before closing the socket. Prevents slow request attacks. |
49
+| `accept a streaming request every seconds` | `0` | Can be used to set a limit on how often a parent node will accept streaming requests from child nodes in a [streaming and replication setup](https://github.com/netdata/netdata/blob/master/streaming/README.md). |
50
+| `respect do not track policy` | `no` | If set to `yes`, Netdata will respect the user's browser preferences for [Do Not Track](https://www.eff.org/issues/do-not-track) (DNT) and storing cookies. If DNT is _enabled_ in the browser, and this option is set to `yes`, users will not be able to sign in to Netdata Cloud via their local Agent dashboard, and their node will not connect to any [registry](https://github.com/netdata/netdata/blob/master/registry/README.md). For certain browsers, users must disable DNT and change this option to `yes` for full functionality. |
51
+| `x-frame-options response header` | ` ` | Avoid [clickjacking attacks](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options), by ensuring that the content is not embedded into other sites. |
52
+| `allow connections from` | `localhost *` | Declare which IP addresses or full-qualified domain names (FQDNs) are allowed to connect to the web server, including the [dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md) or [HTTP API](https://github.com/netdata/netdata/blob/master/web/api/README.md). This is a global setting with higher priority to any of the ones below. |
53
+| `allow connections by dns` | `heuristic` | See the [access list examples](#access-lists) for details on using `allow` settings. |
54
+| `allow dashboard from` | `localhost *` | |
55
+| `allow dashboard by dns` | `heuristic` | |
56
+| `allow badges from` | `*` | |
57
+| `allow badges by dns` | `heuristic` | |
58
+| `allow streaming from` | `*` | |
59
+| `allow streaming by dns` | `heuristic` | |
60
+| `allow netdata.conf` | `localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.* UNKNOWN` | |
61
+| `allow netdata.conf by dns` | `no` | |
62
+| `allow management from` | `localhost` | |
63
+| `allow management by dns` | `heuristic` | |
64
+| `enable gzip compression` | `yes` | When set to `yes`, Netdata web responses will be GZIP compressed, if the web client accepts such responses. |
65
+| `gzip compression strategy` | `default` | Valid settings are `default`, `filtered`, `huffman only`, `rle` and `fixed`. |
66
+| `gzip compression level` | `3` | Valid settings are 1 (fastest) to 9 (best ratio). |
67
+| `web server threads` | ` ` | How many processor threads the web server is allowed. The default is system-specific, the minimum of `6` or the number of CPU cores. |
68
+| `web server max sockets` | ` ` | Available sockets. The default is system-specific, automatically adjusted to 50% of the max number of open files Netdata is allowed to use (via `/etc/security/limits.conf` or systemd), to allow enough file descriptors to be available for data collection. |
69
+| `custom dashboard_info.js` | ` ` | Specifies the location of a custom `dashboard.js` file. See [customizing the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md#customize-the-standard-dashboard) for details. |
70
+
71
+## Examples
72
+
73
+### Disable the web server
74
+
75
+Disable the web server by editing `netdata.conf` and setting:
76
+
77
+```
78
+[web]
79
+ mode = none
80
+```
81
+
82
+### Change the number of threads
83
+
84
+Control the number of threads and sockets with the following settings:
85
+
86
+```
87
+[web]
88
+ web server threads = 4
89
+ web server max sockets = 512
90
+```
91
+
92
+### Binding Netdata to multiple ports
93
+
94
+Netdata can bind to multiple IPs and ports, offering access to different services on each. Up to 100 sockets can be used (increase it at compile time with `CFLAGS="-DMAX_LISTEN_FDS=200" ./netdata-installer.sh ...`).
95
+
96
+The ports to bind are controlled via `[web].bind to`, like this:
97
+
98
+```
99
+[web]
100
+ default port = 19999
101
+ bind to = 127.0.0.1=dashboard^SSL=optional 10.1.1.1:19998=management|netdata.conf hostname:19997=badges [::]:19996=streaming^SSL=force localhost:19995=registry *:http=dashboard unix:/run/netdata/netdata.sock
102
+```
103
+
104
+Using the above, Netdata will bind to:
105
+
106
+- IPv4 127.0.0.1 at port 19999 (port was used from `default port`). Only the UI (dashboard) and the read API will be accessible on this port. Both HTTP and HTTPS requests will be accepted.
107
+- IPv4 10.1.1.1 at port 19998. The management API and `netdata.conf` will be accessible on this port.
108
+- All the IPs `hostname` resolves to (both IPv4 and IPv6 depending on the resolved IPs) at port 19997. Only badges will be accessible on this port.
109
+- All IPv6 IPs at port 19996. Only metric streaming requests from other Netdata agents will be accepted on this port. Only encrypted streams will be allowed (i.e. child nodes also need to be [configured for TLS](https://github.com/netdata/netdata/blob/master/streaming/README.md).
110
+- All the IPs `localhost` resolves to (both IPv4 and IPv6 depending the resolved IPs) at port 19996. This port will only accept registry API requests.
111
+- All IPv4 and IPv6 IPs at port `http` as set in `/etc/services`. Only the UI (dashboard) and the read API will be accessible on this port.
112
+- Unix domain socket `/run/netdata/netdata.sock`. All requests are serviceable on this socket. Note that in some OSs like Fedora, every service sees a different `/tmp`, so don't create a Unix socket under `/tmp`. `/run` or `/var/run` is suggested.
113
+
114
+The option `[web].default port` is used when an entries in `[web].bind to` do not specify a port.
115
+
116
+Note that the access permissions specified with the `=request type|request type|...` format are available from version 1.12 onwards.
117
+As shown in the example above, these permissions are optional, with the default being to permit all request types on the specified port.
118
+The request types are strings identical to the `allow X from` directives of the access lists, i.e. `dashboard`, `streaming`, `registry`, `netdata.conf`, `badges` and `management`.
119
+The access lists themselves and the general setting `allow connections from` in the next section are applied regardless of the ports that are configured to provide these services.
120
+The API requests are serviced as follows:
121
+
122
+- `dashboard` gives access to the UI, the read API and badges API calls.
123
+- `badges` gives access only to the badges API calls.
124
+- `management` gives access only to the management API calls.
125
+
126
+### Enable HTTPS/TLS support
127
+
128
+Since v1.16.0, Netdata supports encrypted HTTP connections to the web server, plus encryption of streaming data to a
129
+parent from its child nodes, via the TLS protocol.
130
+
131
+Inbound unix socket connections are unaffected, regardless of the TLS settings.
132
+
133
+> While Netdata uses Transport Layer Security (TLS) 1.2 to encrypt communications rather than the obsolete SSL protocol,
134
+> it's still common practice to refer to encrypted web connections as `SSL`. Many vendors, like Nginx and even Netdata
135
+> itself, use `SSL` in configuration files, whereas documentation will always refer to encrypted communications as `TLS`
136
+> or `TLS/SSL`.
137
+
138
+To enable TLS, provide the path to your certificate and private key in the `[web]` section of `netdata.conf`:
139
+
140
+```conf
141
+[web]
142
+ ssl key = /etc/netdata/ssl/key.pem
143
+ ssl certificate = /etc/netdata/ssl/cert.pem
144
+```
145
+
146
+Both files must be readable by the `netdata` user. If either of these files do not exist or are unreadable, Netdata will fall back to HTTP. For a parent-child connection, only the parent needs these settings.
147
+
148
+For test purposes, generate self-signed certificates with the following command:
149
+
150
+```bash
151
+openssl req -newkey rsa:2048 -nodes -sha512 -x509 -days 365 -keyout key.pem -out cert.pem
152
+```
153
+
154
+> If you use 4096 bits for your key and the certificate, Netdata will need more CPU to process the communication.
155
+> `rsa4096` can be up to 4 times slower than `rsa2048`, so we recommend using 2048 bits. Verify the difference
156
+> by running:
157
+>
158
+> ```sh
159
+> openssl speed rsa2048 rsa4096
160
+> ```
161
+
162
+### Select TLS version
163
+
164
+Beginning with version `v1.21.0`, specify the TLS version and the ciphers that you want to use:
165
+
166
+```conf
167
+[web]
168
+ tls version = 1.3
169
+ tls ciphers = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
170
+```
171
+
172
+If you do not specify these options, Netdata will use the highest available protocol version on your system and the default cipher list for that protocol provided by your TLS implementation.
173
+
174
+#### TLS/SSL enforcement
175
+
176
+When the certificates are defined and unless any other options are provided, a Netdata server will:
177
+
178
+- Redirect all incoming HTTP web server requests to HTTPS. Applies to the dashboard, the API, `netdata.conf` and badges.
179
+- Allow incoming child connections to use both unencrypted and encrypted communications for streaming.
180
+
181
+To change this behavior, you need to modify the `bind to` setting in the `[web]` section of `netdata.conf`. At the end of each port definition, append `^SSL=force` or `^SSL=optional`. What happens with these settings differs, depending on whether the port is used for HTTP/S requests, or for streaming.
182
+
183
+| SSL setting | HTTP requests|HTTPS requests|Unencrypted Streams|Encrypted Streams|
184
+|:---------:|:-----------:|:------------:|:-----------------:|:----------------|
185
+| none | Redirected to HTTPS|Accepted|Accepted|Accepted|
186
+| `force`| Redirected to HTTPS|Accepted|Denied|Accepted|
187
+| `optional`| Accepted|Accepted|Accepted|Accepted|
188
+
189
+Example:
190
+
191
+```
192
+[web]
193
+ bind to = *=dashboard|registry|badges|management|streaming|netdata.conf^SSL=force
194
+```
195
+
196
+For information how to configure the child to use TLS, check [securing the communication](https://github.com/netdata/netdata/blob/master/streaming/README.md#securing-streaming-communications) in the streaming documentation. There you will find additional details on the expected behavior for client and server nodes, when their respective TLS options are enabled.
197
+
198
+When we define the use of SSL in a Netdata agent for different ports, Netdata will apply the behavior specified on each port. For example, using the configuration line below:
199
+
200
+```
201
+[web]
202
+ bind to = *=dashboard|registry|badges|management|streaming|netdata.conf^SSL=force *:20000=netdata.conf^SSL=optional *:20001=dashboard|registry
203
+```
204
+
205
+Netdata will:
206
+
207
+- Force all HTTP requests to the default port to be redirected to HTTPS (same port).
208
+- Refuse unencrypted streaming connections from child nodes on the default port.
209
+- Allow both HTTP and HTTPS requests to port 20000 for `netdata.conf`
210
+- Force HTTP requests to port 20001 to be redirected to HTTPS (same port). Only allow requests for the dashboard, the read API and the registry on port 20001.
211
+
212
+#### TLS/SSL errors
213
+
214
+When you start using Netdata with TLS, you may find errors in the Netdata log, which is stored at `/var/log/netdata/error.log` by default.
215
+
216
+Most of the time, these errors are due to incompatibilities between your browser's options related to TLS/SSL protocols and Netdata's internal configuration. The most common error is `error:00000006:lib(0):func(0):EVP lib`.
217
+
218
+In the near future, Netdata will allow our users to change the internal configuration to avoid similar errors. Until then, we're recommending only the most common and safe encryption protocols listed above.
219
+
220
+### Access lists
221
+
222
+Netdata supports access lists in `netdata.conf`:
223
+
224
+```
225
+[web]
226
+ allow connections from = localhost *
227
+ allow dashboard from = localhost *
228
+ allow badges from = *
229
+ allow streaming from = *
230
+ allow netdata.conf from = localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.*
231
+ allow management from = localhost
232
+```
233
+
234
+`*` does string matches on the IPs or FQDNs of the clients.
235
+
236
+- `allow connections from` matches anyone that connects on the Netdata port(s).
237
+ So, if someone is not allowed, it will be connected and disconnected immediately, without reading even
238
+ a single byte from its connection. This is a global setting with higher priority to any of the ones below.
239
+
240
+- `allow dashboard from` receives the request and examines if it is a static dashboard file or an API call the
241
+ dashboards do.
242
+
243
+- `allow badges from` checks if the API request is for a badge. Badges are not matched by `allow dashboard from`.
244
+
245
+- `allow streaming from` checks if the child willing to stream metrics to this Netdata is allowed.
246
+ This can be controlled per API KEY and MACHINE GUID in `stream.conf`.
247
+ The setting in `netdata.conf` is checked before the ones in `stream.conf`.
248
+
249
+- `allow netdata.conf from` checks the IP to allow `http://netdata.host:19999/netdata.conf`.
250
+ The IPs listed are all the private IPv4 addresses, including link local IPv6 addresses. Keep in mind that connections to Netdata API ports are filtered by `allow connections from`. So, IPs allowed by `allow netdata.conf from` should also be allowed by `allow connections from`.
251
+
252
+- `allow management from` checks the IPs to allow API management calls. Management via the API is currently supported for [health](https://github.com/netdata/netdata/blob/master/web/api/health/README.md#health-management-api)
253
+
254
+In order to check the FQDN of the connection without opening the Netdata agent to DNS-spoofing, a reverse-dns record
255
+must be setup for the connecting host. At connection time the reverse-dns of the peer IP address is resolved, and
256
+a forward DNS resolution is made to validate the IP address against the name-pattern.
257
+
258
+Please note that this process can be expensive on a machine that is serving many connections. Each access list has an
259
+associated configuration option to turn off DNS-based patterns completely to avoid incurring this cost at run-time:
260
+
261
+```
262
+ allow connections by dns = heuristic
263
+ allow dashboard by dns = heuristic
264
+ allow badges by dns = heuristic
265
+ allow streaming by dns = heuristic
266
+ allow netdata.conf by dns = no
267
+ allow management by dns = heuristic
268
+```
269
+
270
+The three possible values for each of these options are `yes`, `no` and `heuristic`. The `heuristic` option disables
271
+the check when the pattern only contains IPv4/IPv6 addresses or `localhost`, and enables it when wildcards are
272
+present that may match DNS FQDNs.
273
+
274
+## DDoS protection
275
+
276
+If you publish your Netdata web server to the internet, you may want to apply some protection against DDoS:
277
+
278
+1. Use the `static-threaded` web server (it is the default)
279
+2. Use reasonable `[web].web server max sockets` (the default is)
280
+3. Don't use all your CPU cores for Netdata (lower `[web].web server threads`)
281
+4. Run the `netdata` process with a low process scheduling priority (the default is the lowest)
282
+5. If possible, proxy Netdata via a full featured web server (Nginx, Apache, etc)
docs/dashboard/visualization-date-and-time-controls.md
new
+125
@@ -0,0 +1,125 @@
1
+<!--
2
+title: "Visualization date and time controls"
3
+description: "Netdata's dashboard features powerful date visualization controls that include a time control (play, pause, force play), a timezone selector and a rich date and timeframe selector, with useful defaults and rich customization, to help you narrow your focus when troubleshooting issues or anomalies."
4
+type: "how-to"
5
+custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/dashboard/visualization-date-and-time-controls.md"
6
+sidebar_label: "Visualization date and time controls"
7
+learn_status: "Published"
8
+learn_topic_type: "Concepts"
9
+learn_rel_path: "Concepts"
10
+-->
11
+
12
+# Visualization date and time controls
13
+
14
+## Date and time selector
15
+
16
+### Pick timeframes to visualize
17
+
18
+While [panning through time and zooming in/out](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md) from charts it is helpful when
19
+you're looking a recent history, or want to do granular troubleshooting, what if you want to see metrics from 6 hours
20
+ago? Or 6 days?
21
+
22
+Netdata's dashboard features a **timeframe selector** to help you visualize specific timeframes in a few helpful ways.
23
+By default, it shows a certain number of minutes of historical metrics based on the your browser's viewport to ensure
24
+it's always showing per-second granularity.
25
+
26
+#### Open the timeframe selector
27
+
28
+To visualize a new timeframe, you need to open the picker, which appears just above the menu, near the top-right cover
29
+of the dashboard.
30
+
31
+
33
+
34
+The **Clear** button resets the dashboard back to its default state based on your browser viewport, and **Apply** closes
35
+the picker and shifts all charts to the selected timeframe.
36
+
37
+#### Use the Quick Selector
38
+
39
+Click any of the following options in the **Quick Selector** to choose a commonly-used timeframe.
40
+
41
+- Last 5 minutes
42
+- Last 15 minutes
43
+- Last 2 hours
44
+- Last 6 hours
45
+- Last 12 hours
46
+
47
+Click **Apply** to see metrics from your selected timeframe.
48
+
49
+#### Choose a specific interval
50
+
51
+Beneath the Quick Selector is an input field and dropdown you use in combination to select a specific timeframe of
52
+minutes, hours, days, or months. Enter a number and choose the appropriate unit of time, then click **Apply**.
53
+
54
+#### Choose multiple days
55
+
56
+Use the calendar to select multiple days. Click on a date to begin the timeframe selection, then an ending date. The
57
+timeframe begins at noon on the beginning and end dates. Click **Apply** to see your selected multi-day timeframe.
58
+
59
+## Time controls
60
+
61
+The time control provides you the following options: **Play**, **Pause** and **Force Play**.
62
+* **Play** - the content of the page will be automatically refreshed while this is in the foreground
63
+* **Pause** - the content of the page isn't refreshed due to a manual request to pause it or, for example, when your investigating data on a
64
+chart (cursor is on top of a chart)
65
+* **Force Play** - the content of the page will be automatically refreshed even if this is in the background
66
+
67
+With this, we aim to bring more clarity and allow you to distinguish if the content you are looking at is live or historical and also allow you
68
+ to always refresh the content of the page when the tabs are in the background.
69
+
70
+Main use cases for **Force Play**:
71
+* You use a terminal or deployment tools to do changes in your infra and want to see immediately, Netdata is in the background, displaying the impact
72
+of these changes
73
+* You want to have Netdata on the background, example displayed on a TV, to constantly see metrics through dashboards or to watch the alert
74
+status
75
+
76
+
78
+
79
+## Timezone selector
80
+
81
+With the timezone selector, you have the ability to change the timezone on Netdata Cloud. More often than not teams are
82
+distributed in different timezones and they need to collaborate.
83
+
84
+Our goal is to make it easier for you and your teams to troubleshoot based on your timezone preference and communicate easily
85
+with varying timezones and timeframes without the need to be concerned about their specificity.
86
+
87
+<img width="437" alt="Untitled1" src="https://user-images.githubusercontent.com/43294513/216628390-c3bd1cd2-349d-4523-b8d3-c7e68395f670.png"/>
88
+
89
+When you change the timezone all the date and time fields will be updated to be displayed according to the specified timezone, this goes from
90
+charts to alerts information and across the Netdata Cloud.
91
+
92
+## Caveats and considerations
93
+
94
+**Longer timeframes will decrease metrics granularity**. At the default timeframe, based on your browser viewport, each
95
+"tick" on charts represents one second. If you select a timeframe of 6 hours, each tick represents the _average_ value
96
+across a larger period of time.
97
+
98
+**You can only see metrics as far back in history as your metrics retention policy allows**. Netdata uses an internal
99
+time-series database (TSDB) to store as many metrics as it can within a specific amount of disk space. The default
100
+storage is 256 MiB, which should be enough for 1-3 days of historical metrics. If you navigate back to a timeframe
101
+beyond stored historical metrics, you'll see this message:
102
+
103
+
105
+
106
+At any time, [configure the internal TSDB's storage capacity](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) to expand your
107
+depth of historical metrics.
108
+
109
+## What's next?
110
+
111
+One useful next step after selecting a timeframe is [exporting the
112
+metrics](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md) into a snapshot file, which can then be shared and imported
113
+into any other Netdata dashboard.
114
+
115
+There are also many ways to [customize](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md) the standard dashboard experience, from changing
116
+the theme to editing the text that accompanies every section of charts.
117
+
118
+## Further reading & related information
119
+
120
+- Dashboard
121
+ - [How the dashboard works](https://github.com/netdata/netdata/blob/master/docs/dashboard/how-dashboard-works.md)
122
+ - [Interact with charts](https://github.com/netdata/netdata/blob/master/docs/dashboard/interact-charts.md)
123
+ - [Chart dimensions, contexts, and families](https://github.com/netdata/netdata/blob/master/docs/dashboard/dimensions-contexts-families.md)
124
+ - [Import, export, and print a snapshot](https://github.com/netdata/netdata/blob/master/docs/dashboard/import-export-print-snapshot.md)
125
+ - [Customize the standard dashboard](https://github.com/netdata/netdata/blob/master/docs/dashboard/customize.md)
docs/metrics-storage-management/enable-streaming.md
new
+158
@@ -0,0 +1,158 @@
1
+<!--
2
+title: "Enable streaming between nodes"
3
+description: >-
4
+ "With metrics streaming enabled, you can not only replicate metrics data
5
+ into a second database, but also view dashboards and trigger alarm notifications
6
+ for multiple nodes in parallel."
7
+type: "how-to"
8
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md"
9
+sidebar_label: "Enable streaming between nodes"
10
+learn_status: "Published"
11
+learn_topic_type: "Tasks"
12
+learn_rel_path: "Setup"
13
+-->
14
+
15
+# Enable streaming between nodes
16
+
17
+The simplest streaming configuration is **replication**, in which a child node streams its metrics in real time to a
18
+parent node, and both nodes retain metrics in their own databases.
19
+
20
+To configure replication, you need two nodes, each running Netdata. First you'll first enable streaming on your parent
21
+node, then enable streaming on your child node. When you're finished, you'll be able to see the child node's metrics in
22
+the parent node's dashboard, quickly switch between the two dashboards, and be able to serve [alarm
23
+notifications](https://github.com/netdata/netdata/blob/master/docs/monitor/enable-notifications.md) from either or both nodes.
24
+
25
+## Enable streaming on the parent node
26
+
27
+First, log onto the node that will act as the parent.
28
+
29
+Run `uuidgen` to create a new API key, which is a randomly-generated machine GUID the Netdata Agent uses to identify
30
+itself while initiating a streaming connection. Copy that into a separate text file for later use.
31
+
32
+> Find out how to [install `uuidgen`](https://command-not-found.com/uuidgen) on your node if you don't already have it.
33
+
34
+Next, open `stream.conf` using [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files)
35
+from within the [Netdata config directory](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md#the-netdata-config-directory).
36
+
37
+```bash
38
+cd /etc/netdata
39
+sudo ./edit-config stream.conf
40
+```
41
+
42
+Scroll down to the section beginning with `[API_KEY]`. Paste the API key you generated earlier between the brackets, so
43
+that it looks like the following:
44
+
45
+```conf
46
+[11111111-2222-3333-4444-555555555555]
47
+```
48
+
49
+Set `enabled` to `yes`, and `default memory mode` to `dbengine`. Leave all the other settings as their defaults. A
50
+simplified version of the configuration, minus the commented lines, looks like the following:
51
+
52
+```conf
53
+[11111111-2222-3333-4444-555555555555]
54
+ enabled = yes
55
+ default memory mode = dbengine
56
+```
57
+
58
+Save the file and close it, then restart Netdata with `sudo systemctl restart netdata`, or the [appropriate
59
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
60
+
61
+## Enable streaming on the child node
62
+
63
+Connect to your child node with SSH.
64
+
65
+Open `stream.conf` again. Scroll down to the `[stream]` section and set `enabled` to `yes`. Paste the IP address of your
66
+parent node at the end of the `destination` line, and paste the API key generated on the parent node onto the `api key`
67
+line.
68
+
69
+Leave all the other settings as their defaults. A simplified version of the configuration, minus the commented lines,
70
+looks like the following:
71
+
72
+```conf
73
+[stream]
74
+ enabled = yes
75
+ destination = 203.0.113.0
76
+ api key = 11111111-2222-3333-4444-555555555555
77
+```
78
+
79
+Save the file and close it, then restart Netdata with `sudo systemctl restart netdata`, or the [appropriate
80
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system.
81
+
82
+## Enable TLS/SSL on streaming (optional)
83
+
84
+While encrypting the connection between your parent and child nodes is recommended for security, it's not required to
85
+get started. If you're not interested in encryption, skip ahead to [view streamed
86
+metrics](#view-streamed-metrics-in-netdatas-dashboard).
87
+
88
+In this example, we'll use self-signed certificates.
89
+
90
+On the **parent** node, use OpenSSL to create the key and certificate, then use `chown` to make the new files readable
91
+by the `netdata` user.
92
+
93
+```bash
94
+sudo openssl req -newkey rsa:2048 -nodes -sha512 -x509 -days 365 -keyout /etc/netdata/ssl/key.pem -out /etc/netdata/ssl/cert.pem
95
+sudo chown netdata:netdata /etc/netdata/ssl/cert.pem /etc/netdata/ssl/key.pem
96
+```
97
+
98
+Next, enforce TLS/SSL on the web server. Open `netdata.conf`, scroll down to the `[web]` section, and look for the `bind
99
+to` setting. Add `^SSL=force` to turn on TLS/SSL. See the [web server
100
+reference](https://github.com/netdata/netdata/blob/master/web/server/README.md#enabling-tls-support) for other TLS/SSL options.
101
+
102
+```conf
103
+[web]
104
+ bind to = *=dashboard|registry|badges|management|streaming|netdata.conf^SSL=force
105
+```
106
+
107
+Next, connect to the **child** node and open `stream.conf`. Add `:SSL` to the end of the existing `destination` setting
108
+to connect to the parent using TLS/SSL. Uncomment the `ssl skip certificate verification` line to allow the use of
109
+self-signed certificates.
110
+
111
+```conf
112
+[stream]
113
+ enabled = yes
114
+ destination = 203.0.113.0:SSL
115
+ ssl skip certificate verification = yes
116
+ api key = 11111111-2222-3333-4444-555555555555
117
+```
118
+
119
+Restart both the parent and child nodes with `sudo systemctl restart netdata`, or the [appropriate
120
+method](https://github.com/netdata/netdata/blob/master/docs/configure/start-stop-restart.md) for your system, to stream encrypted metrics using TLS/SSL.
121
+
122
+## View streamed metrics in Netdata's dashboard
123
+
124
+At this point, the child node is streaming its metrics in real time to its parent. Open the local Agent dashboard for
125
+the parent by navigating to `http://PARENT-NODE:19999` in your browser, replacing `PARENT-NODE` with its IP address or
126
+hostname.
127
+
128
+This dashboard shows parent metrics. To see child metrics, open the left-hand sidebar with the hamburger icon
129
+
130
+in the top panel. Both nodes appear under the **Replicated Nodes** menu. Click on either of the links to switch between
131
+separate parent and child dashboards.
132
+
133
+
135
+
136
+The child dashboard is also available directly at `http://PARENT-NODE:19999/host/CHILD-HOSTNAME`, which in this example
137
+is `http://203.0.113.0:19999/host/netdata-child`.
138
+
139
+## What's next?
140
+
141
+Now that you have a basic streaming setup with replication, you may want to tweak the configuration to eliminate the
142
+child database, disable the child dashboard, or enable SSL on the streaming connection between the parent and child.
143
+
144
+See the [streaming reference
145
+doc](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/reference-streaming.md#examples) for details about
146
+other possible configurations.
147
+
148
+When using Netdata's default TSDB (`dbengine`), the parent node maintains separate, parallel databases for itself and
149
+every child node streaming to it. Each instance is sized identically based on the `dbengine multihost disk space`
150
+setting in `netdata.conf`. See our doc on [changing metrics retention](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) for
151
+details.
152
+
153
+### Related information & further reading
154
+
155
+- Streaming
156
+ - [How Netdata streams metrics](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/how-streaming-works.md)
157
+ - **[Enable streaming between nodes](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md)**
158
+ - [Streaming reference](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/reference-streaming.md)
docs/metrics-storage-management/how-streaming-works.md
new
+99
@@ -0,0 +1,99 @@
1
+<!--
2
+title: "How metrics streaming works"
3
+description: >-
4
+ "Netdata's real-time streaming allows you to replicate metrics data
5
+ across multiple nodes, or centralize all your metrics data into a single
6
+ time-series database (TSDB)."
7
+type: "explanation"
8
+custom_edit_url: "https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/how-streaming-works.md"
9
+sidebar_label: "How metrics streaming works"
10
+learn_status: "Published"
11
+learn_topic_type: "Concepts"
12
+learn_rel_path: "Concepts"
13
+-->
14
+
15
+# How metrics streaming works
16
+
17
+Each node running Netdata can stream the metrics it collects, in real time, to another node. Streaming allows you to
18
+replicate metrics data across multiple nodes, or centralize all your metrics data into a single time-series database
19
+(TSDB).
20
+
21
+When one node streams metrics to another, the node receiving metrics can visualize them on the
22
+[dashboard](https://github.com/netdata/netdata/blob/master/docs/visualize/interact-dashboards-charts.md), run health checks to [trigger
23
+alarms](https://github.com/netdata/netdata/blob/master/docs/monitor/view-active-alarms.md) and [send notifications](https://github.com/netdata/netdata/blob/master/docs/monitor/enable-notifications.md), and
24
+[export](https://github.com/netdata/netdata/blob/master/docs/export/external-databases.md) all metrics to an external TSDB. When Netdata streams metrics to another
25
+Netdata, the receiving one is able to perform everything a Netdata instance is capable of.
26
+
27
+Streaming lets you decide exactly how you want to store and maintain metrics data. While we believe Netdata's
28
+[distributed architecture](https://github.com/netdata/netdata/blob/master/docs/store/distributed-data-architecture.md) is ideal for speed and scale, streaming
29
+provides centralization options for those who want to maintain only a single TSDB instance.
30
+
31
+## Streaming basics
32
+
33
+There are three types of nodes in Netdata's streaming ecosystem.
34
+
35
+- **Parent**: A node, running Netdata, that receives streamed metric data.
36
+- **Child**: A node, running Netdata, that streams metric data to one or more parent.
37
+- **Proxy**: A node, running Netdata, that receives metric data from a child and "forwards" them on to a
38
+ separate parent node.
39
+
40
+Netdata uses API keys, which are just random GUIDs, to authorize the communication between child and parent nodes. We
41
+recommend using `uuidgen` for generating API keys, which can then be used across any number of streaming connections.
42
+Or, you can generate unique API keys for each parent-child relationship.
43
+
44
+Once the parent node authorizes the child's API key, the child can start streaming metrics.
45
+
46
+It's important to note that the streaming connection uses TCP, UDP, or Unix sockets, _not HTTP_. To proxy streaming
47
+metrics, you need to use a proxy that tunnels [OSI layer 4-7
48
+traffic](https://en.wikipedia.org/wiki/OSI_model#Layer_4:_Transport_Layer) without interfering with it, such as
49
+[SOCKS](https://en.wikipedia.org/wiki/SOCKS) or Nginx's [TCP/UDP load
50
+balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/).
51
+
52
+## Supported streaming configurations
53
+
54
+Netdata supports any combination of parent, child, and proxy nodes that you can imagine. Any node can act as both a
55
+parent, child, or proxy at the same time, sending or receiving streaming metrics from any number of other nodes.
56
+
57
+Here are a few example streaming configurations:
58
+
59
+- **Headless collector**:
60
+ - Child `A`, _without_ a database or web dashboard, streams metrics to parent `B`.
61
+ - `A` metrics are only available via the local Agent dashboard for `B`.
62
+ - `B` generates alarms for `A`.
63
+- **Replication**:
64
+ - Child `A`, _with_ a database and web dashboard, streams metrics to parent `B`.
65
+ - `A` metrics are available on both local Agent dashboards, and can be stored with the same or different metrics
66
+ retention policies.
67
+ - Both `A` and `B` generate alarms.
68
+- **Proxy**:
69
+ - Child `A`, _with or without_ a database, sends metrics to proxy `C`, also _with or without_ a database. `C` sends
70
+ metrics to parent `B`.
71
+ - Any node with a database can generate alarms.
72
+
73
+## Viewing streamed metrics
74
+
75
+Parent nodes feature a **Replicated Nodes** section in the left-hand panel, which opens with the hamburger icon
76
+
77
+in the top navigation. The parent node, plus any child nodes, appear here. Click on any of the hostnames to switch
78
+between parent and child dashboards, all served by the parent's [web server](https://github.com/netdata/netdata/blob/master/web/server/README.md).
79
+
80
+
82
+
83
+Each child dashboard is also available directly at the following URL pattern:
84
+`http://PARENT-NODE:19999/host/CHILD-HOSTNAME`.
85
+
86
+## What's next?
87
+
88
+Now that you understand the fundamentals of streaming metrics between nodes, go ahead and [enable
89
+streaming](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md) using a simple `parent-child` relationship. For all
90
+the details, see the [streaming reference](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/reference-streaming.md) doc.
91
+
92
+Take your streaming setup even further by [exporting metrics](https://github.com/netdata/netdata/blob/master/docs/export/external-databases.md) to an external TSDB.
93
+
94
+### Related information & further reading
95
+
96
+- Streaming
97
+ - **[How Netdata streams metrics](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/how-streaming-works.md)**
98
+ - [Enable streaming between nodes](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md)
99
+ - [Streaming reference](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/reference-streaming.md)
\ No newline at end of file
docs/metrics-storage-management/reference-streaming.md
new
+490
@@ -0,0 +1,490 @@
1
+<!--
2
+title: "Streaming reference"
3
+description: "Each node running Netdata can stream the metrics it collects, in real time, to another node. See all of the available settings in this reference document."
4
+type: "reference"
5
+custom_edit_url: "https://github.com/netdata/netdata/edit/master/docs/metrics-storage-management/reference-streaming.md"
6
+sidebar_label: "Streaming reference"
7
+learn_status: "Published"
8
+learn_topic_type: "References"
9
+learn_rel_path: "References/Configuration"
10
+-->
11
+
12
+# Streaming reference
13
+
14
+Each node running Netdata can stream the metrics it collects, in real time, to another node. To learn more, read about
15
+[how streaming works](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/how-streaming-works.md).
16
+
17
+For a quickstart guide for enabling a simple `parent-child` streaming relationship, see our [stream metrics between
18
+nodes](https://github.com/netdata/netdata/blob/master/docs/metrics-storage-management/enable-streaming.md) doc. All other configuration options and scenarios are
19
+covered in the sections below.
20
+
21
+## Configuration
22
+
23
+There are two files responsible for configuring Netdata's streaming capabilities: `stream.conf` and `netdata.conf`.
24
+
25
+From within your Netdata config directory (typically `/etc/netdata`), [use `edit-config`](https://github.com/netdata/netdata/blob/master/docs/configure/nodes.md) to
26
+open either `stream.conf` or `netdata.conf`.
27
+
28
+```
29
+sudo ./edit-config stream.conf
30
+sudo ./edit-config netdata.conf
31
+```
32
+
33
+## Settings
34
+
35
+As mentioned above, both `stream.conf` and `netdata.conf` contain settings relevant to streaming.
36
+
37
+### `stream.conf`
38
+
39
+The `stream.conf` file contains three sections. The `[stream]` section is for configuring child nodes.
40
+
41
+The `[API_KEY]` and `[MACHINE_GUID]` sections are both for configuring parent nodes, and share the same settings.
42
+`[API_KEY]` settings affect every child node using that key, whereas `[MACHINE_GUID]` settings affect only the child
43
+node with a matching GUID.
44
+
45
+The file `/var/lib/netdata/registry/netdata.public.unique.id` contains a random GUID that **uniquely identifies each
46
+node**. This file is automatically generated by Netdata the first time it is started and remains unaltered forever.
47
+
48
+#### `[stream]` section
49
+
50
+| Setting | Default | Description |
51
+| :---------------------------------------------- | :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
52
+| `enabled` | `no` | Whether this node streams metrics to any parent. Change to `yes` to enable streaming. |
53
+| [`destination`](#destination) | ` ` | A space-separated list of parent nodes to attempt to stream to, with the first available parent receiving metrics, using the following format: `[PROTOCOL:]HOST[%INTERFACE][:PORT][:SSL]`. [Read more →](#destination) |
54
+| `ssl skip certificate verification` | `yes` | If you want to accept self-signed or expired certificates, set to `yes` and uncomment. |
55
+| `CApath` | `/etc/ssl/certs/` | The directory where known certificates are found. Defaults to OpenSSL's default path. |
56
+| `CAfile` | `/etc/ssl/certs/cert.pem` | Add a parent node certificate to the list of known certificates in `CAPath`. |
57
+| `api key` | ` ` | The `API_KEY` to use as the child node. |
58
+| `timeout seconds` | `60` | The timeout to connect and send metrics to a parent. |
59
+| `default port` | `19999` | The port to use if `destination` does not specify one. |
60
+| [`send charts matching`](#send-charts-matching) | `*` | A space-separated list of [Netdata simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md) to filter which charts are streamed. [Read more →](#send-charts-matching) |
61
+| `buffer size bytes` | `10485760` | The size of the buffer to use when sending metrics. The default `10485760` equals a buffer of 10MB, which is good for 60 seconds of data. Increase this if you expect latencies higher than that. The buffer is flushed on reconnect. |
62
+| `reconnect delay seconds` | `5` | How long to wait until retrying to connect to the parent node. |
63
+| `initial clock resync iterations` | `60` | Sync the clock of charts for how many seconds when starting. |
64
+
65
+### `[API_KEY]` and `[MACHINE_GUID]` sections
66
+
67
+| Setting | Default | Description |
68
+| :---------------------------------------------- | :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
69
+| `enabled` | `no` | Whether this API KEY enabled or disabled. |
70
+| [`allow from`](#allow-from) | `*` | A space-separated list of [Netdata simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md) matching the IPs of nodes that will stream metrics using this API key. [Read more →](#allow-from) |
71
+| `default history` | `3600` | The default amount of child metrics history to retain when using the `save`, `map`, or `ram` memory modes. |
72
+| [`default memory mode`](#default-memory-mode) | `ram` | The [database](https://github.com/netdata/netdata/blob/master/database/README.md) to use for all nodes using this `API_KEY`. Valid settings are `dbengine`, `map`, `save`, `ram`, or `none`. [Read more →](#default-memory-mode) |
73
+| `health enabled by default` | `auto` | Whether alarms and notifications should be enabled for nodes using this `API_KEY`. `auto` enables alarms when the child is connected. `yes` enables alarms always, and `no` disables alarms. |
74
+| `default postpone alarms on connect seconds` | `60` | Postpone alarms and notifications for a period of time after the child connects. |
75
+| `default proxy enabled` | ` ` | Route metrics through a proxy. |
76
+| `default proxy destination` | ` ` | Space-separated list of `IP:PORT` for proxies. |
77
+| `default proxy api key` | ` ` | The `API_KEY` of the proxy. |
78
+| `default send charts matching` | `*` | See [`send charts matching`](#send-charts-matching). |
79
+
80
+#### `destination`
81
+
82
+A space-separated list of parent nodes to attempt to stream to, with the first available parent receiving metrics, using
83
+the following format: `[PROTOCOL:]HOST[%INTERFACE][:PORT][:SSL]`.
84
+
85
+- `PROTOCOL`: `tcp`, `udp`, or `unix`. (only tcp and unix are supported by parent nodes)
86
+- `HOST`: A IPv4, IPv6 IP, or a hostname, or a unix domain socket path. IPv6 IPs should be given with brackets
87
+ `[ip:address]`.
88
+- `INTERFACE` (IPv6 only): The network interface to use.
89
+- `PORT`: The port number or service name (`/etc/services`) to use.
90
+- `SSL`: To enable TLS/SSL encryption of the streaming connection.
91
+
92
+To enable TCP streaming to a parent node at `203.0.113.0` on port `20000` and with TLS/SSL encryption:
93
+
94
+```conf
95
+[stream]
96
+ destination = tcp:203.0.113.0:20000:SSL
97
+```
98
+
99
+#### `send charts matching`
100
+
101
+A space-separated list of [Netdata simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md) to filter which charts are streamed.
102
+
103
+The default is a single wildcard `*`, which streams all charts.
104
+
105
+To send only a few charts, list them explicitly, or list a group using a wildcard. To send _only_ the `apps.cpu` chart
106
+and charts with contexts beginning with `system.`:
107
+
108
+```conf
109
+[stream]
110
+ send charts matching = apps.cpu system.*
111
+```
112
+
113
+To send all but a few charts, use `!` to create a negative match. To send _all_ charts _but_ `apps.cpu`:
114
+
115
+```conf
116
+[stream]
117
+ send charts matching = !apps.cpu *
118
+```
119
+
120
+#### `allow from`
121
+
122
+A space-separated list of [Netdata simple patterns](https://github.com/netdata/netdata/blob/master/libnetdata/simple_pattern/README.md) matching the IPs of nodes that
123
+will stream metrics using this API key. The order is important, left to right, as the first positive or negative match is used.
124
+
125
+The default is `*`, which accepts all requests including the `API_KEY`.
126
+
127
+To allow from only a specific IP address:
128
+
129
+```conf
130
+[API_KEY]
131
+ allow from = 203.0.113.10
132
+```
133
+
134
+To allow all IPs starting with `10.*`, except `10.1.2.3`:
135
+
136
+```conf
137
+[API_KEY]
138
+ allow from = !10.1.2.3 10.*
139
+```
140
+
141
+> If you set specific IP addresses here, and also use the `allow connections` setting in the `[web]` section of
142
+> `netdata.conf`, be sure to add the IP address there so that it can access the API port.
143
+
144
+#### `default memory mode`
145
+
146
+The [database](https://github.com/netdata/netdata/blob/master/database/README.md) to use for all nodes using this `API_KEY`. Valid settings are `dbengine`, `ram`,
147
+`save`, `map`, or `none`.
148
+
149
+- `dbengine`: The default, recommended time-series database (TSDB) for Netdata. Stores recent metrics in memory, then
150
+ efficiently spills them to disk for long-term storage.
151
+- `ram`: Stores metrics _only_ in memory, which means metrics are lost when Netdata stops or restarts. Ideal for
152
+ streaming configurations that use ephemeral nodes.
153
+- `save`: Stores metrics in memory, but saves metrics to disk when Netdata stops or restarts, and loads historical
154
+ metrics on start.
155
+- `map`: Stores metrics in memory-mapped files, like swap, with constant disk write.
156
+- `none`: No database.
157
+
158
+When using `default memory mode = dbengine`, the parent node creates a separate instance of the TSDB to store metrics
159
+from child nodes. The [size of _each_ instance is configurable](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) with the `page
160
+cache size` and `dbengine multihost disk space` settings in the `[global]` section in `netdata.conf`.
161
+
162
+### `netdata.conf`
163
+
164
+| Setting | Default | Description |
165
+| :----------------------------------------- | :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
166
+| **`[global]` section** | | |
167
+| `memory mode` | `dbengine` | Determines the [database type](https://github.com/netdata/netdata/blob/master/database/README.md) to be used on that node. Other options settings include `none`, `ram`, `save`, and `map`. `none` disables the database at this host. This also disables alarms and notifications, as those can't run without a database. |
168
+| **`[web]` section** | | |
169
+| `mode` | `static-threaded` | Determines the [web server](https://github.com/netdata/netdata/blob/master/web/server/README.md) type. The other option is `none`, which disables the dashboard, API, and registry. |
170
+| `accept a streaming request every seconds` | `0` | Set a limit on how often a parent node accepts streaming requests from child nodes. `0` equals no limit. If this is set, you may see `... too busy to accept new streaming request. Will be allowed in X secs` in Netdata's `error.log`. |
171
+
172
+## Examples
173
+
174
+### Per-child settings
175
+
176
+While the `[API_KEY]` section applies settings for any child node using that key, you can also use per-child settings
177
+with the `[MACHINE_GUID]` section.
178
+
179
+For example, the metrics streamed from only the child node with `MACHINE_GUID` are saved in memory, not using the
180
+default `dbengine` as specified by the `API_KEY`, and alarms are disabled.
181
+
182
+```conf
183
+[API_KEY]
184
+ enabled = yes
185
+ default memory mode = dbengine
186
+ health enabled by default = auto
187
+ allow from = *
188
+
189
+[MACHINE_GUID]
190
+ enabled = yes
191
+ memory mode = save
192
+ health enabled = no
193
+```
194
+
195
+### Securing streaming with TLS/SSL
196
+
197
+Netdata does not activate TLS encryption by default. To encrypt streaming connections, you first need to [enable TLS
198
+support](https://github.com/netdata/netdata/blob/master/web/server/README.md#enabling-tls-support) on the parent. With encryption enabled on the receiving side, you
199
+need to instruct the child to use TLS/SSL as well. On the child's `stream.conf`, configure the destination as follows:
200
+
201
+```
202
+[stream]
203
+ destination = host:port:SSL
204
+```
205
+
206
+The word `SSL` appended to the end of the destination tells the child that connections must be encrypted.
207
+
208
+> While Netdata uses Transport Layer Security (TLS) 1.2 to encrypt communications rather than the obsolete SSL protocol,
209
+> it's still common practice to refer to encrypted web connections as `SSL`. Many vendors, like Nginx and even Netdata
210
+> itself, use `SSL` in configuration files, whereas documentation will always refer to encrypted communications as `TLS`
211
+> or `TLS/SSL`.
212
+
213
+#### Certificate verification
214
+
215
+When TLS/SSL is enabled on the child, the default behavior will be to not connect with the parent unless the server's
216
+certificate can be verified via the default chain. In case you want to avoid this check, add the following to the
217
+child's `stream.conf` file:
218
+
219
+```
220
+[stream]
221
+ ssl skip certificate verification = yes
222
+```
223
+
224
+#### Trusted certificate
225
+
226
+If you've enabled [certificate verification](#certificate-verification), you might see errors from the OpenSSL library
227
+when there's a problem with checking the certificate chain (`X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY`). More
228
+importantly, OpenSSL will reject self-signed certificates.
229
+
230
+Given these known issues, you have two options. If you trust your certificate, you can set the options `CApath` and
231
+`CAfile` to inform Netdata where your certificates, and the certificate trusted file, are stored.
232
+
233
+For more details about these options, you can read about [verify
234
+locations](https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_load_verify_locations.html).
235
+
236
+Before you changed your streaming configuration, you need to copy your trusted certificate to your child system and add
237
+the certificate to OpenSSL's list.
238
+
239
+On most Linux distributions, the `update-ca-certificates` command searches inside the `/usr/share/ca-certificates`
240
+directory for certificates. You should double-check by reading the `update-ca-certificate` manual (`man
241
+update-ca-certificate`), and then change the directory in the below commands if needed.
242
+
243
+If you have `sudo` configured on your child system, you can use that to run the following commands. If not, you'll have
244
+to log in as `root` to complete them.
245
+
246
+```
247
+# mkdir /usr/share/ca-certificates/netdata
248
+# cp parent_cert.pem /usr/share/ca-certificates/netdata/parent_cert.crt
249
+# chown -R netdata.netdata /usr/share/ca-certificates/netdata/
250
+```
251
+
252
+First, you create a new directory to store your certificates for Netdata. Next, you need to change the extension on your
253
+certificate from `.pem` to `.crt` so it's compatible with `update-ca-certificate`. Finally, you need to change
254
+permissions so the user that runs Netdata can access the directory where you copied in your certificate.
255
+
256
+Next, edit the file `/etc/ca-certificates.conf` and add the following line:
257
+
258
+```
259
+netdata/parent_cert.crt
260
+```
261
+
262
+Now you update the list of certificates running the following, again either as `sudo` or `root`:
263
+
264
+```
265
+# update-ca-certificates
266
+```
267
+
268
+> Some Linux distributions have different methods of updating the certificate list. For more details, please read this
269
+> guide on [adding trusted root certificates](https://github.com/Busindre/How-to-Add-trusted-root-certificates).
270
+
271
+Once you update your certificate list, you can set the stream parameters for Netdata to trust the parent certificate.
272
+Open `stream.conf` for editing and change the following lines:
273
+
274
+```
275
+[stream]
276
+ CApath = /etc/ssl/certs/
277
+ CAfile = /etc/ssl/certs/parent_cert.pem
278
+```
279
+
280
+With this configuration, the `CApath` option tells Netdata to search for trusted certificates inside `/etc/ssl/certs`.
281
+The `CAfile` option specifies the Netdata parent certificate is located at `/etc/ssl/certs/parent_cert.pem`. With this
282
+configuration, you can skip using the system's entire list of certificates and use Netdata's parent certificate instead.
283
+
284
+#### Expected behaviors
285
+
286
+With the introduction of TLS/SSL, the parent-child communication behaves as shown in the table below, depending on the
287
+following configurations:
288
+
289
+- **Parent TLS (Yes/No)**: Whether the `[web]` section in `netdata.conf` has `ssl key` and `ssl certificate`.
290
+- **Parent port TLS (-/force/optional)**: Depends on whether the `[web]` section `bind to` contains a `^SSL=force` or
291
+ `^SSL=optional` directive on the port(s) used for streaming.
292
+- **Child TLS (Yes/No)**: Whether the destination in the child's `stream.conf` has `:SSL` at the end.
293
+- **Child TLS Verification (yes/no)**: Value of the child's `stream.conf` `ssl skip certificate verification`
294
+ parameter (default is no).
295
+
296
+| Parent TLS enabled | Parent port SSL | Child TLS | Child SSL Ver. | Behavior |
297
+| :----------------- | :--------------- | :-------- | :------------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
298
+| No | - | No | no | Legacy behavior. The parent-child stream is unencrypted. |
299
+| Yes | force | No | no | The parent rejects the child connection. |
300
+| Yes | -/optional | No | no | The parent-child stream is unencrypted (expected situation for legacy child nodes and newer parent nodes) |
301
+| Yes | -/force/optional | Yes | no | The parent-child stream is encrypted, provided that the parent has a valid TLS/SSL certificate. Otherwise, the child refuses to connect. |
302
+| Yes | -/force/optional | Yes | yes | The parent-child stream is encrypted. |
303
+
304
+### Proxy
305
+
306
+A proxy is a node that receives metrics from a child, then streams them onward to a parent. To configure a proxy,
307
+configure it as a receiving and a sending Netdata at the same time.
308
+
309
+Netdata proxies may or may not maintain a database for the metrics passing through them. When they maintain a database,
310
+they can also run health checks (alarms and notifications) for the remote host that is streaming the metrics.
311
+
312
+In the following example, the proxy receives metrics from a child node using the `API_KEY` of
313
+`66666666-7777-8888-9999-000000000000`, then stores metrics using `dbengine`. It then uses the `API_KEY` of
314
+`11111111-2222-3333-4444-555555555555` to proxy those same metrics on to a parent node at `203.0.113.0`.
315
+
316
+```conf
317
+[stream]
318
+ enabled = yes
319
+ destination = 203.0.113.0
320
+ api key = 11111111-2222-3333-4444-555555555555
321
+
322
+[66666666-7777-8888-9999-000000000000]
323
+ enabled = yes
324
+ default memory mode = dbengine
325
+```
326
+
327
+### Ephemeral nodes
328
+
329
+Netdata can help you monitor ephemeral nodes, such as containers in an auto-scaling infrastructure, by always streaming
330
+metrics to any number of permanently-running parent nodes.
331
+
332
+On the parent, set the following in `stream.conf`:
333
+
334
+```conf
335
+[11111111-2222-3333-4444-555555555555]
336
+ # enable/disable this API key
337
+ enabled = yes
338
+
339
+ # one hour of data for each of the child nodes
340
+ default history = 3600
341
+
342
+ # do not save child metrics on disk
343
+ default memory = ram
344
+
345
+ # alarms checks, only while the child is connected
346
+ health enabled by default = auto
347
+```
348
+
349
+On the child nodes, set the following in `stream.conf`:
350
+
351
+```bash
352
+[stream]
353
+ # stream metrics to another Netdata
354
+ enabled = yes
355
+
356
+ # the IP and PORT of the parent
357
+ destination = 10.11.12.13:19999
358
+
359
+ # the API key to use
360
+ api key = 11111111-2222-3333-4444-555555555555
361
+```
362
+
363
+In addition, edit `netdata.conf` on each child node to disable the database and alarms.
364
+
365
+```bash
366
+[global]
367
+ # disable the local database
368
+ memory mode = none
369
+
370
+[health]
371
+ # disable health checks
372
+ enabled = no
373
+```
374
+
375
+## Troubleshooting
376
+
377
+Both parent and child nodes log information at `/var/log/netdata/error.log`.
378
+
379
+If the child manages to connect to the parent you will see something like (on the parent):
380
+
381
+```
382
+2017-03-09 09:38:52: netdata: INFO : STREAM [receive from [10.11.12.86]:38564]: new client connection.
383
+2017-03-09 09:38:52: netdata: INFO : STREAM xxx [10.11.12.86]:38564: receive thread created (task id 27721)
384
+2017-03-09 09:38:52: netdata: INFO : STREAM xxx [receive from [10.11.12.86]:38564]: client willing to stream metrics for host 'xxx' with machine_guid '1234567-1976-11e6-ae19-7cdd9077342a': update every = 1, history = 3600, memory mode = ram, health auto
385
+2017-03-09 09:38:52: netdata: INFO : STREAM xxx [receive from [10.11.12.86]:38564]: initializing communication...
386
+2017-03-09 09:38:52: netdata: INFO : STREAM xxx [receive from [10.11.12.86]:38564]: receiving metrics...
387
+```
388
+
389
+and something like this on the child:
390
+
391
+```
392
+2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: connecting...
393
+2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: initializing communication...
394
+2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: waiting response from remote netdata...
395
+2017-03-09 09:38:28: netdata: INFO : STREAM xxx [send to box:19999]: established communication - sending metrics...
396
+```
397
+
398
+The following sections describe the most common issues you might encounter when connecting parent and child nodes.
399
+
400
+### Slow connections between parent and child
401
+
402
+When you have a slow connection between parent and child, Netdata raises a few different errors. Most of the
403
+errors will appear in the child's `error.log`.
404
+
405
+```bash
406
+netdata ERROR : STREAM_SENDER[CHILD HOSTNAME] : STREAM CHILD HOSTNAME [send to PARENT IP:PARENT PORT]: too many data pending - buffer is X bytes long,
407
+Y unsent - we have sent Z bytes in total, W on this connection. Closing connection to flush the data.
408
+```
409
+
410
+On the parent side, you may see various error messages, most commonly the following:
411
+
412
+```
413
+netdata ERROR : STREAM_PARENT[CHILD HOSTNAME,[CHILD IP]:CHILD PORT] : read failed: end of file
414
+```
415
+
416
+Another common problem in slow connections is the child sending a partial message to the parent. In this case, the
417
+parent will write the following to its `error.log`:
418
+
419
+```
420
+ERROR : STREAM_RECEIVER[CHILD HOSTNAME,[CHILD IP]:CHILD PORT] : sent command 'B' which is not known by netdata, for host 'HOSTNAME'. Disabling it.
421
+```
422
+
423
+In this example, `B` was part of a `BEGIN` message that was cut due to connection problems.
424
+
425
+Slow connections can also cause problems when the parent misses a message and then receives a command related to the
426
+missed message. For example, a parent might miss a message containing the child's charts, and then doesn't know
427
+what to do with the `SET` message that follows. When that happens, the parent will show a message like this:
428
+
429
+```
430
+ERROR : STREAM_RECEIVER[CHILD HOSTNAME,[CHILD IP]:CHILD PORT] : requested a SET on chart 'CHART NAME' of host 'HOSTNAME', without a dimension. Disabling it.
431
+```
432
+
433
+### Child cannot connect to parent
434
+
435
+When the child can't connect to a parent for any reason (misconfiguration, networking, firewalls, parent
436
+down), you will see the following in the child's `error.log`.
437
+
438
+```
439
+ERROR : STREAM_SENDER[HOSTNAME] : Failed to connect to 'PARENT IP', port 'PARENT PORT' (errno 113, No route to host)
440
+```
441
+
442
+### 'Is this a Netdata?'
443
+
444
+This question can appear when Netdata starts the stream and receives an unexpected response. This error can appear when
445
+the parent is using SSL and the child tries to connect using plain text. You will also see this message when
446
+Netdata connects to another server that isn't Netdata. The complete error message will look like this:
447
+
448
+```
449
+ERROR : STREAM_SENDER[CHILD HOSTNAME] : STREAM child HOSTNAME [send to PARENT HOSTNAME:PARENT PORT]: server is not replying properly (is it a netdata?).
450
+```
451
+
452
+### Stream charts wrong
453
+
454
+Chart data needs to be consistent between child and parent nodes. If there are differences between chart data on
455
+a parent and a child, such as gaps in metrics collection, it most often means your child's `memory mode`
456
+does not match the parent's. To learn more about the different ways Netdata can store metrics, and thus keep chart
457
+data consistent, read our [memory mode documentation](https://github.com/netdata/netdata/blob/master/database/README.md).
458
+
459
+### Forbidding access
460
+
461
+You may see errors about "forbidding access" for a number of reasons. It could be because of a slow connection between
462
+the parent and child nodes, but it could also be due to other failures. Look in your parent's `error.log` for errors
463
+that look like this:
464
+
465
+```
466
+STREAM [receive from [child HOSTNAME]:child IP]: `MESSAGE`. Forbidding access."
467
+```
468
+
469
+`MESSAGE` will have one of the following patterns:
470
+
471
+- `request without KEY` : The message received is incomplete and the KEY value can be API, hostname, machine GUID.
472
+- `API key 'VALUE' is not valid GUID`: The UUID received from child does not have the format defined in [RFC
473
+ 4122](https://tools.ietf.org/html/rfc4122)
474
+- `machine GUID 'VALUE' is not GUID.`: This error with machine GUID is like the previous one.
475
+- `API key 'VALUE' is not allowed`: This stream has a wrong API key.
476
+- `API key 'VALUE' is not permitted from this IP`: The IP is not allowed to use STREAM with this parent.
477
+- `machine GUID 'VALUE' is not allowed.`: The GUID that is trying to send stream is not allowed.
478
+- `Machine GUID 'VALUE' is not permitted from this IP. `: The IP does not match the pattern or IP allowed to connect to
479
+ use stream.
480
+
481
+### Netdata could not create a stream
482
+
483
+The connection between parent and child is a stream. When the parent can't convert the initial connection into
484
+a stream, it will write the following message inside `error.log`:
485
+
486
+```
487
+file descriptor given is not a valid stream
488
+```
489
+
490
+After logging this error, Netdata will close the stream.