Active journal centralization guide no encryption (#16236)
* Active journal centralization guide no encryption Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud> * minor fix Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud> --------- Signed-off-by: Tasos Katsoulas <tasos@netdata.cloud>
Tasos Katsoulas committed
Oct 18, 2023 at 15:20 UTC
e3900fa838e1ab081feb8dd59a53a65b4ed6e024
1 file changed
+160
collectors/systemd-journal.plugin/active_journal_centralization_guide_no_encryption.md
new
+160
@@ -0,0 +1,160 @@
1
+# Active journal centralization without encryption
2
+
3
+This page will guide you through creating an active journal centralization setup without the use of encryption.
4
+
5
+Once you centralize your infrastructure logs to a server, Netdata will automatically detect all the logs from all
6
+servers and organize them in sources.
7
+With the setup described in this document, journal files are identified by the hostnames of the clients you pull logs.
8
+
9
+An _active_ journal server fetch logs from clients, so in this setup we will:
10
+
11
+1. configure `systemd-journal-remote` on the server, to pull journal logs.
12
+2. configure `systemd-journal-gatewayd` on the clients, to serve their logs to the micro http server.
13
+
14
+> ⚠️ **IMPORTANT**<br/>
15
+> These instructions will copy your logs to a central server, without any encryption or authorization.<br/>
16
+> DO NOT USE THIS ON NON-TRUSTED NETWORKS.
17
+
18
+## Client configuration
19
+
20
+On the clients, install `systemd-journal-gateway`.
21
+
22
+```bash
23
+# change this according to your distro
24
+sudo apt-get install systemd-journal-gateway
25
+```
26
+
27
+Optionally, if you want to change the port (the default is `19531`), edit `systemd-journal-gatewayd.socket`
28
+
29
+```bash
30
+# edit the socket file
31
+sudo systemctl edit systemd-journal-gatewayd.socket
32
+```
33
+
34
+and add the following lines into the instructed place, and choose your desired port; save and exit.
35
+
36
+```bash
37
+[Socket]
38
+ListenStream=<DESIRED_PORT>
39
+```
40
+
41
+Finally, enable it, so that it will start automatically upon receiving a connection:
42
+
43
+```bash
44
+# enable systemd-journal-remote
45
+sudo systemctl daemon-reload
46
+sudo systemctl enable --now systemd-journal-gatewayd.socket
47
+sudo systemctl enable systemd-journal-gatewayd.service
48
+sudo systemctl start systemd-journal-gatewayd.service
49
+```
50
+
51
+## Server configuration
52
+
53
+On the centralization server install `systemd-journal-remote`:
54
+
55
+```bash
56
+# change this according to your distro
57
+sudo apt-get install systemd-journal-remote
58
+```
59
+
60
+Start it once to make sure than the `systemd-journal-remote` created any necessary requirement to work as centralization
61
+server. To do that, you need to spin up a temporarily _passive_ server with http, then close it, if you won't use it
62
+also as a passive server.
63
+
64
+```bash
65
+sudo cp /lib/systemd/system/systemd-journal-remote.service /etc/systemd/system/
66
+
67
+# edit it to make sure it says:
68
+# --listen-http=-3
69
+# not:
70
+# --listen-https=-3
71
+sudo nano /etc/systemd/system/systemd-journal-remote.service
72
+
73
+# reload systemd
74
+sudo systemctl daemon-reload
75
+```
76
+
77
+Optionally, if you want to change the port (the default is `19532`), edit `systemd-journal-remote.socket`
78
+
79
+```bash
80
+# edit the socket file
81
+sudo systemctl edit systemd-journal-remote.socket
82
+```
83
+
84
+and add the following lines into the instructed place, and choose your desired port; save and exit.
85
+
86
+```bash
87
+[Socket]
88
+ListenStream=<DESIRED_PORT>
89
+```
90
+
91
+Start and (stop it, if you won't use it also as _passive_).
92
+
93
+```bash
94
+# enable systemd-journal-remote
95
+sudo systemctl start systemd-journal-remote.service
96
+sudo systemctl stop systemd-journal-remote.service
97
+```
98
+
99
+For each of your clients (endpoints that you want to fetch journal logs from) create a service that will use
100
+`systemd-journal-remote` will always fetch the logs.
101
+
102
+
103
+```bash
104
+sudo nano /etc/systemd/system/systemd-journal-endpoint-X.service
105
+```
106
+
107
+Copy the service file above, replace the Description and `TARGET_HOST`, save and exit
108
+
109
+```
110
+[Unit]
111
+Description=Fetching systemd journal logs from my endpoint X
112
+
113
+[Service]
114
+ExecStart=/usr/lib/systemd/systemd-journal-remote --url http://<TARGET_HOST>:19531/entries?follow
115
+Type=simple
116
+Restart=always
117
+User=systemd-journal-remote
118
+
119
+[Install]
120
+WantedBy=multi-user.target
121
+```
122
+
123
+Repeat the same for every host that you want to fetch journal logs.
124
+Reload the systemd daemon config, enable each service and start, like this:
125
+
126
+```bash
127
+sudo systemctl daemon-reload
128
+sudo systemctl enable systemd-journal-endpoint-X
129
+sudo systemctl start systemd-journal-endpoint-X
130
+```
131
+
132
+## Verify it works
133
+
134
+To verify the central server is receiving logs, run this on the central server:
135
+
136
+```bash
137
+sudo ls -l /var/log/journal/remote/
138
+```
139
+
140
+You should see new files from the client's hostname.
141
+
142
+Also, any of the new service files (`systemctl status systemd-journal-endpoint-X`) should show something like this:
143
+
144
+```bash
145
+● systemd-journal-client1.service - Fetching systemd journal logs from 192.168.2.146
146
+ Loaded: loaded (/etc/systemd/system/systemd-journal-client1.service; enabled; preset: disabled)
147
+ Drop-In: /usr/lib/systemd/system/service.d
148
+ └─10-timeout-abort.conf
149
+ Active: active (running) since Wed 2023-10-18 07:35:52 EEST; 23min ago
150
+ Main PID: 77959 (systemd-journal)
151
+ Tasks: 2 (limit: 6928)
152
+ Memory: 7.7M
153
+ CPU: 518ms
154
+ CGroup: /system.slice/systemd-journal-client1.service
155
+ ├─77959 /usr/lib/systemd/systemd-journal-remote --url "http://192.168.2.146:19531/entries?follow"
156
+ └─77962 curl "-HAccept: application/vnd.fdo.journal" --silent --show-error "http://192.168.2.146:19531/entries?follow"
157
+
158
+Oct 18 07:35:52 systemd-journal-server systemd[1]: Started systemd-journal-client1.service - Fetching systemd journal logs from 192.168.2.146.
159
+Oct 18 07:35:52 systemd-journal-server systemd-journal-remote[77959]: Spawning curl http://192.168.2.146:19531/entries?follow...
160
+```
\ No newline at end of file