Docs: Update with go-live claiming and ACLK information (#8859) (#8960)
* Restore docs from naughty PR * Address Andrew's comments * Ini to conf * Changes based on meeting with Andrew * Tweak text around claiming * Some grammar/typo fixes * Add /var/lib/netdata to Docker instructions on README * Added a few more ACLK links per Chris Co-authored-by: Joel Hans <joel@netdata.cloud>
James Mills committed
May 12, 2020 at 00:19 UTC
30597a5bd4e9a90ef200d18e78e3fbb8a0c49d9f
4 files changed
+177
-68
README.md
+4
-1
@@ -156,11 +156,14 @@ To try Netdata in a Docker container, run this:
156
```sh
157
docker run -d --name=netdata \
158
-p 19999:19999 \
159
+ -v netdatalib:/var/lib/netdata \
160
+ -v netdatacache:/var/cache/netdata \
161
-v /etc/passwd:/host/etc/passwd:ro \
162
-v /etc/group:/host/etc/group:ro \
163
-v /proc:/host/proc:ro \
164
-v /sys:/host/sys:ro \
163
- -v /var/run/docker.sock:/var/run/docker.sock:ro \
165
+ -v /etc/os-release:/host/etc/os-release:ro \
166
+ --restart unless-stopped \
167
--cap-add SYS_PTRACE \
168
--security-opt apparmor=unconfined \
169
netdata/netdata
aclk/README.md
+83
-17
@@ -2,28 +2,31 @@
2
---
3
title: "Agent-Cloud link (ACLK)"
4
description: "The Agent-Cloud link (ACLK) is the mechanism responsible for connecting a Netdata agent to Netdata Cloud."
5
-date: 2020-04-15
5
+date: 2020-04-30
6
custom_edit_url: https://github.com/netdata/netdata/edit/master/aclk/README.md
7
---
8
-->
9
10
# Agent-cloud link (ACLK)
11
12
-The Agent-Cloud link (ACLK) is the mechanism responsible for connecting a Netdata Agent to Netdata Cloud. The ACLK uses
13
-[MQTT](https://en.wikipedia.org/wiki/MQTT) over secure websockets to first create, persist, encrypt the connection, and
14
-then enable the features found in Netdata Cloud. _No data is exchanged with Netdata Cloud until you claim a node._
12
+The Agent-Cloud link (ACLK) is the mechanism responsible for securely connecting a Netdata Agent to your web browser
13
+through Netdata Cloud. The ACLK is encrypted, safe, and _does not exchange data with Netdata Cloud until you claim a
14
+node_.
15
16
-Read our [claiming documentation](/claim/README.md) for a guide for claiming a node using the ACLK and additional
17
-troubleshooting and reference information.
16
+For a guide to claiming a node using the ACLK, plus additional troubleshooting and reference information, read our [get
17
+started with Cloud](https://learn.netdata.cloud/docs/cloud/get-started) guide or the full [claiming
18
+documentation](/claim/README.md).
19
20
## Enable and configure the ACLK
21
21
-The ACLK is enabled by default and automatically configured if the prerequisites installed correctly. You can see this
22
-in the `[cloud]` section of `netdata.conf`.
22
+The ACLK is enabled by default, with its settings automatically configured and stored in the Agent's memory. No file is
23
+created at `var/lib/netdata/cloud.d/cloud.conf` until you either claim a node or create it yourself. The default
24
+configuration uses two settings:
25
26
```conf
25
-[cloud]
26
- cloud base url = https://app.netdata.cloud
27
+[global]
28
+ enabled = yes
29
+ cloud base url = https://app.netdata.cloud
30
```
31
32
If your Agent needs to use a proxy to access the internet, you must [set up a proxy for
@@ -31,18 +34,81 @@ claiming](/claim/README.md#claim-through-a-proxy).
34
35
## Disable the ACLK
36
34
-You have two options if you prefer to disable the ACLK and not use Netdata Cloud:
37
+You have two options if you prefer to disable the ACLK and not use Netdata Cloud.
38
+
39
+### Disable at installation
40
+
41
+You can pass the `--disable-cloud` parameter to the Agent installation when using a kickstart script
42
+([kickstart.sh](/packaging/installer/methods/kickstart.md) or
43
+[kickstart-static64.sh](/packaging/installer/methods/kickstart-64.md)), or a [manual installation from
44
+Git](/packaging/installer/methods/manual.md).
45
36
-1. Pass `--disable-cloud` to `netdata-installer.sh` during installation. When you pass this parameter, the installer
37
- does not download or compile any extra libraries, and the Agent behaves as though the ACLK, and thus Netdata Cloud,
38
- does not exist. ACLK functionality is available in the Agent but remains fully inactive.
46
+When you pass this parameter, the installer does not download or compile any extra libraries. Once running, the Agent
47
+kills the thread responsible for the ACLK and claiming behavior, and behaves as though the ACLK, and thus Netdata Cloud,
48
+does not exist.
49
40
-2. Change a runtime setting in your `netdata.conf` file. This setting only stops the Agent from attempting any
41
- connection via the ACLK, but does not prevent the installer from downloading and compiling the ACLK's dependencies.
50
+### Disable at runtime
51
+
52
+You can change a runtime setting in your `cloud.conf` file to disable the ACLK. This setting only stops the Agent from
53
+attempting any connection via the ACLK, but does not prevent the installer from downloading and compiling the ACLK's
54
+dependencies.
55
+
56
+The file typically exists at `/var/lib/netdata/cloud.d/cloud.conf`, but can change if you set a prefix during
57
+installation. To disable the ACLK, open that file and change the `enabled` setting to `no`:
58
59
```conf
60
[global]
45
- netdata cloud = disable
61
+ enabled = no
62
```
63
64
+If the file at `/var/lib/netdata/cloud.d/cloud.conf` doesn't exist, you need to create it.
65
+
66
+Copy and paste the first two lines from below, which will change your prompt to `cat`.
67
+
68
+```bash
69
+cd /var/lib/netdata/cloud.d
70
+cat > cloud.conf << EOF
71
+```
72
+
73
+Copy and paste in lines 3-6, and after the final `EOF`, hit **Enter**. The final line must contain only `EOF`. Hit **Enter** again to return to your normal prompt with the newly-created file.
74
+
75
+To get your normal prompt back, the final line
76
+must contain only `EOF`.
77
+
78
+```bash
79
+[global]
80
+ enabled = no
81
+ cloud base url = https://app.netdata.cloud
82
+EOF
83
+```
84
+
85
+You also need to change the file's permissions. Use `grep "run as user" /etc/netdata/netdata.conf` to figure out which
86
+user your Agent runs as (typically `netdata`), and replace `netdata:netdata` as shown below if necessary:
87
+
88
+```bash
89
+sudo chmod 0770 cloud.conf
90
+sudo chown netdata:netdata cloud.conf
91
+```
92
+
93
+Restart your Agent to disable the ACLK.
94
+
95
+### Re-enable the ACLK
96
+
97
+If you first disable the ACLK and any Cloud functionality and then decide you would like to use Cloud, you must either
98
+reinstall Netdata with Cloud enabled or change the runtime setting in your `cloud.conf` file.
99
+
100
+If you passed `--disable-cloud` to `netdata-installer.sh` during installation, you must reinstall your Agent. Use the
101
+same method as before, but pass `--require-cloud` to the installer. When installation finishes you can [claim your
102
+node](/claim/README.md#claim-a-node).
103
+
104
+If you changed the runtime setting in your `var/lib/netdata/cloud.d/cloud.conf` file, edit the file again and change
105
+`enabled` to `yes`:
106
+
107
+```conf
108
+[global]
109
+ enabled = yes
110
+```
111
+
112
+Restart your Agent and [claim your node](/claim/README.md#claim-a-node).
113
+
114
[](<>)
claim/README.md
+87
-50
@@ -2,7 +2,7 @@
2
---
3
title: "Agent claiming"
4
description: "Agent claiming allows a Netdata Agent, running on a distributed node, to securely connect to Netdata Cloud. A Space's administrator creates a claiming token, which is used to add an Agent to their Space via the Agent-Cloud link."
5
-date: 2020-04-15
5
+date: 2020-04-30
6
custom_edit_url: https://github.com/netdata/netdata/edit/master/claim/README.md
7
---
8
-->
@@ -13,6 +13,10 @@ Agent claiming allows a Netdata Agent, running on a distributed node, to securel
13
administrator creates a **claiming token**, which is used to add an Agent to their Space via the [Agent-Cloud link
14
(ACLK)](/aclk/README.md).
15
16
+Are you just starting out with Netdata Cloud? See our [get started with
17
+Cloud](https://learn.netdata.cloud/docs/cloud/get-started) guide for a walkthrough of the process and simplified
18
+instructions.
19
+
20
Claiming nodes is a security feature in Netdata Cloud. Through the process of claiming, you demonstrate in a few ways
21
that you have administrative access to that node and the configuration settings for its Agent. By logging into the node,
22
you prove you have access, and by using the claiming script or the Netdata command line, you prove you have write access
@@ -24,41 +28,58 @@ Netdata Cloud.
28
> The claiming process ensures no third party can add your node, and then view your node's metrics, in a Cloud account,
29
> Space, or War Room that you did not authorize.
30
27
-By claiming a node, you opt-in to sending data from your Agent to Netdata Cloud via the ACLK. This data is encrypted by
28
-TLS while it is in transit. We use the the RSA keypair created during claiming to authenticate the identity of the agent
29
-when it connects to the Cloud. While the data does flow through Netdata Cloud servers on its way from Agents to the
30
-browser, we do not store or log it.
31
-
32
-## How to claim a node
31
+By claiming a node, you opt-in to sending data from your Agent to Netdata Cloud via the [ACLK](/aclk/README.md). This
32
+data is encrypted by TLS while it is in transit. We use the RSA keypair created during claiming to authenticate the
33
+identity of the Agent when it connects to the Cloud. While the data does flow through Netdata Cloud servers on its way
34
+from Agents to the browser, we do not store or log it.
35
36
You can claim a node during the Cloud onboarding process, or after you created a Space by clicking on the **USER's
35
-Space** dropdown, then **Manage Claimed Nodes**.
37
+Space** dropdown, then **Manage claimed nodes**.
38
37
-> Only the administrators of a Space in Netdata Cloud can view the claiming token and accompanying script generated by
38
-> Netdata Cloud.
39
+There are two important notes regarding claiming:
40
40
-To claim a node, copy the script given by Cloud. **You must run this script as the user running the** `netdata`
41
-**service**, which is usually the `netdata` user. You have two options: Switch to the appropriate user yourself, or run
42
-the command using `sudo` to automatically manage permissions.
41
+- _You can only claim any given node in a single Space_. You can, however, add that claimed node to multiple War Rooms
42
+ within that one Space.
43
+- You must repeat the claiming process on every node you want to add to Netdata Cloud.
44
44
-By switching to the `netdata` user:
45
+## How to claim a node
46
+
47
+To claim a node, select which War Rooms you want to add this node to with the dropdown, then copy and paste the script
48
+given by Cloud into your node's terminal. Hit **Enter**.
49
50
```bash
47
-sudo su -s /bin/bash netdata
48
-netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud
51
+sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud
52
```
53
51
-With `sudo`:
54
+The script should return `Agent was successfully claimed.`. If the claiming script returns errors, or if you don't see
55
+the node in your Space after 60 seconds, see the [troubleshooting information](#troubleshooting). If you prefer not to
56
+use root privileges via `sudo` to run the claiming script, see the next section.
57
+
58
+Repeat this process with every node you want to add to Cloud during onboarding. You can also add more nodes once you've
59
+finished onboarding.
60
+
61
+### Claim an agent without root privileges
62
+
63
+If you don't want to run the claiming script with root privileges, you can discover which user is running the Agent,
64
+switch to that user, and run the claiming script.
65
+
66
+Use `grep` to search your `netdata.conf` file, which is typically located at `/etc/netdata/netdata.conf`, for the `run
67
+as user` setting. For example:
68
69
```bash
54
-sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud
70
+grep "run as user" /etc/netdata/netdata.conf
71
+ # run as user = netdata
72
```
73
57
-Hit **Enter**. The script should return `Agent was successfully claimed.`. If the claiming script returns errors, see
58
-the [troubleshooting information](#troubleshooting).
74
+The default user is `netdata`. Yours may be different, so pay attention to the output from `grep`. Switch to that user
75
+and run the claiming script.
76
60
-> Your node may need up to 60 seconds to connect to Netdata Cloud after finishing the claiming process. Please be
61
-> patient!
77
+```bash
78
+netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud
79
+```
80
+
81
+Hit **Enter**. The script should return `Agent was successfully claimed.`. If the claiming script returns errors, or if
82
+you don't see the node in your Space after 60 seconds, see the [troubleshooting information](#troubleshooting).
83
84
### Claim an Agent running in Docker
85
@@ -68,14 +89,15 @@ and immediately claim it.
89
90
#### Running Agent containers
91
71
-Claim a _running Agent container_ by appending the script offered by Cloud to a `docker exec ...` command, replacing `netdata` with the name of your running container:
92
+Claim a _running Agent container_ by appending the script offered by Cloud to a `docker exec ...` command, replacing
93
+`netdata` with the name of your running container:
94
95
```bash
96
docker exec -it netdata netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud
97
```
98
77
-The script should return `Agent was successfully claimed.`. If the claiming script returns errors, see the
78
-[troubleshooting information](#troubleshooting).
99
+The script should return `Agent was successfully claimed.`. If the claiming script returns errors, or if
100
+you don't see the node in your Space after 60 seconds, see the [troubleshooting information](#troubleshooting).
101
102
#### New/ephemeral Agent containers
103
@@ -88,11 +110,14 @@ Cloud.
110
```bash
111
docker run -d --name=netdata \
112
-p 19999:19999 \
113
+ -v netdatalib:/var/lib/netdata \
114
+ -v netdatacache:/var/cache/netdata \
115
-v /etc/passwd:/host/etc/passwd:ro \
116
-v /etc/group:/host/etc/group:ro \
117
-v /proc:/host/proc:ro \
118
-v /sys:/host/sys:ro \
119
-v /etc/os-release:/host/etc/os-release:ro \
120
+ --restart unless-stopped \
121
--cap-add SYS_PTRACE \
122
--security-opt apparmor=unconfined \
123
netdata/netdata \
@@ -114,7 +139,7 @@ A Space's administrator can claim a node through a SOCKS5 or HTTP(S) proxy.
139
You should first configure the proxy in the `[cloud]` section of `netdata.conf`. The proxy settings you specify here
140
will also be used to tunnel the ACLK. The default `proxy` setting is `none`.
141
117
-```ini
142
+```conf
143
[cloud]
144
proxy = none
145
```
@@ -134,25 +159,23 @@ For example, a SOCKS5 proxy setting may look like the following:
159
proxy = socks5h://proxy.example.com:1080 # With a URL
160
```
161
137
-You can now move on to claiming. Be sure to switch to the `netdata` user or use `sudo` as explained in the [step
138
-above](#how-to-claim-a-node).
139
-
140
-When you claim with the `netdata-claim.sh` script, add the `-proxy=` parameter and append the same proxy setting you
141
-added to `netdata.conf`.
162
+You can now move on to claiming. When you claim with the `netdata-claim.sh` script, add the `-proxy=` parameter and
163
+append the same proxy setting you added to `netdata.conf`.
164
165
```bash
144
-netdata-claim.sh -token=MYTOKEN1234567 -rooms=room1,room2 -url=https://app.netdata.cloud -proxy=socks5h://203.0.113.0:1080
166
+sudo netdata-claim.sh -token=MYTOKEN1234567 -rooms=room1,room2 -url=https://app.netdata.cloud -proxy=socks5h://203.0.113.0:1080
167
```
168
147
-Hit **Enter**. The script should return `Agent was successfully claimed.`. If the claiming script returns errors, see
148
-the [troubleshooting information](#troubleshooting).
169
+Hit **Enter**. The script should return `Agent was successfully claimed.`. If the claiming script returns errors, or if
170
+you don't see the node in your Space after 60 seconds, see the [troubleshooting information](#troubleshooting).
171
172
### Troubleshooting
173
152
-If you're having trouble claiming a node, this may be because the ACLK cannot connect to Cloud.
174
+If you're having trouble claiming a node, this may be because the [ACLK](/aclk/README.md) cannot connect to Cloud.
175
154
-With the Netdata Agent running, visit `http://localhost:19999/api/v1/info` in your browser. The returned JSON contains
155
-four keys that will be helpful to diagnose any issues you might be having with the ACLK or claiming process.
176
+With the Netdata Agent running, visit `http://NODE:19999/api/v1/info` in your browser, replacing `NODE` with the IP
177
+address or hostname of your Agent. The returned JSON contains four keys that will be helpful to diagnose any issues you
178
+might be having with the ACLK or claiming process.
179
180
```json
181
"cloud-enabled"
@@ -169,7 +192,7 @@ If `cloud-enabled` is `false`, you probably ran the installer with `--disable-cl
192
193
Additionally, check that the `enabled` setting in `var/lib/netdata/cloud.d/cloud.conf` is set to `true`:
194
172
-```ini
195
+```conf
196
[global]
197
enabled = true
198
```
@@ -199,6 +222,12 @@ You may see one of the following error messages during installation:
222
Netdata Cloud.
223
- Unable to fetch sources for libwebsockets. The install process will continue, but you may not be able to connect
224
this node to Netdata Cloud.
225
+- Could not find cmake, which is required to build libwebsockets. The install process will continue, but you may not
226
+ be able to connect this node to Netdata Cloud.
227
+- Could not find cmake, which is required to build JSON-C. The install process will continue, but Netdata Cloud
228
+ support will be disabled.
229
+- Failed to build JSON-C. Netdata Cloud support will be disabled.
230
+- Unable to fetch sources for JSON-C. Netdata Cloud support will be disabled.
231
232
One common cause of the installer failing to build Cloud features is not having one of the following dependencies on
233
your system: `cmake` and OpenSSL, including the `devel` package.
@@ -221,7 +250,7 @@ You must [claim your node](#how-to-claim-a-node).
250
251
#### aclk-available is false
252
224
-If `aclk-available` is `false` and all other keys are `true`, your Agent is having trouble connection to the Cloud
253
+If `aclk-available` is `false` and all other keys are `true`, your Agent is having trouble connecting to the Cloud
254
through the ACLK. Please check your system's firewall.
255
256
If your Agent needs to use a proxy to access the internet, you must [set up a proxy for
@@ -239,23 +268,31 @@ Netdata library directory.
268
269
```bash
270
cd /var/lib/netdata # Replace with your Netdata library directory, if not /var/lib/netdata/
242
-rm -rf cloud.d/
271
+sudo rm -rf cloud.d/
272
```
273
245
-> You may need to use `sudo` or another method of elevating your privileges.
246
-
274
Once you delete the `cloud.d/` directory, the ACLK will not connect to Cloud the next time the Agent starts, and Cloud
248
-will then remove it from the interface.
275
+will show it as **unreachable**. You can then remove it from your War Rooms.
276
277
## Claiming reference
278
279
In the sections below, you can find reference material for the claiming script, claiming via the Agent's command line
280
tool, and details about the files found in `cloud.d`.
281
282
+### The `cloud.conf` file
283
+
284
+This section defines how and whether your Agent connects to [Netdata Cloud](https://learn.netdata.cloud/docs/cloud/)
285
+using the [ACLK](/aclk/README.md).
286
+
287
+| setting | default | info |
288
+|:-------------- |:------------------------- |:-------------------------------------------------------------------------------------------------------------------------------------- |
289
+| cloud base url | https://app.netdata.cloud | The URL for the Netdata Cloud web application. You should not change this. If you want to disable Cloud, change the `enabled` setting. |
290
+| enabled | yes | The runtime option to disable the [Agent-Cloud link](/aclk/README.md) and prevent your Agent from connecting to Netdata Cloud. |
291
+
292
### Claiming script
293
257
-A Space's administrator can claim an Agent by directly calling the `netdata-claim.sh` script **as the `netdata` user**
258
-and passing the following arguments:
294
+A Space's administrator can claim an Agent by directly calling the `netdata-claim.sh` script either with root privileges
295
+using `sudo`, or as the user running the Agent (typically `netdata`), and passing the following arguments:
296
297
```sh
298
-token=TOKEN
@@ -306,12 +343,12 @@ If need be, the user can override the Agent's defaults by providing additional a
343
344
### Claiming directory
345
309
-Netdata stores the agent claiming-related state in the Netdata library directory under `cloud.d`, e.g. in
310
-`/var/lib/netdata/cloud.d`. The user can put files in this directory to provide defaults to the `-token` and `-rooms`
311
-arguments. These files should be owned **by the `netdata` user**.
346
+Netdata stores the Agent's claiming-related state in the Netdata library directory under `cloud.d`. For a default
347
+installation, this directory exists at `/var/lib/netdata/cloud.d`. The directory and its files should be owned by the
348
+user that runs the Agent, which is typically the `netdata` user.
349
313
-The `cloud.d/token` file should contain the claiming-token and the `cloud.d/rooms` file should contain the list of
314
-war-rooms.
350
+The `cloud.d/token` file should contain the claiming-token and the `cloud.d/rooms` file should contain the list of War
351
+Rooms you added that node to.
352
353
The user can also put the Cloud endpoint's full certificate chain in `cloud.d/cloud_fullchain.pem` so that the Agent
354
can trust the endpoint if necessary.
daemon/README.md
+3
@@ -1,6 +1,7 @@
1
<!--
2
---
3
title: "Netdata daemon"
4
+date: 2020-04-29
5
custom_edit_url: https://github.com/netdata/netdata/edit/master/daemon/README.md
6
---
7
-->
@@ -184,6 +185,8 @@ The command line options of the Netdata 1.10.0 version are the following:
185
-W simple-pattern pattern string
186
Check if string matches pattern and exit.
187
188
+ -W "claim -token=TOKEN -rooms=ROOM1,ROOM2 url=https://app.netdata.cloud"
189
+ Claim the agent to the workspace rooms pointed to by TOKEN and ROOM*.
190
191
Signals netdata handles:
192