| 1 | # Passive journal centralization without encryption |
| 2 | |
| 3 | This page will guide you through creating a passive journal centralization setup without the use of encryption. |
| 4 | |
| 5 | Once you centralize your infrastructure logs to a server, Netdata will automatically detect all the logs from all servers and organize them in sources. |
| 6 | With the setup described in this document, journal files are identified by the IPs of the clients sending the logs. Netdata will automatically do |
| 7 | reverse DNS lookups to find the names of the server and name the sources on the dashboard accordingly. |
| 8 | |
| 9 | A _passive_ journal server waits for clients to push their metrics to it, so in this setup we will: |
| 10 | |
| 11 | 1. configure `systemd-journal-remote` on the server, to listen for incoming connections. |
| 12 | 2. configure `systemd-journal-upload` on the clients, to push their logs to the server. |
| 13 | |
| 14 | :::warning |
| 15 | |
| 16 | These instructions will copy your logs to a central server, without any encryption or authorization. |
| 17 | |
| 18 | DO NOT USE THIS ON NON-TRUSTED NETWORKS. |
| 19 | |
| 20 | ::: |
| 21 | |
| 22 | ## Server configuration |
| 23 | |
| 24 | On the centralization server install `systemd-journal-remote`: |
| 25 | |
| 26 | ```bash |
| 27 | # change this according to your distro |
| 28 | sudo apt-get install systemd-journal-remote |
| 29 | ``` |
| 30 | |
| 31 | Make sure the journal transfer protocol is `http`: |
| 32 | |
| 33 | ```bash |
| 34 | sudo cp /lib/systemd/system/systemd-journal-remote.service /etc/systemd/system/ |
| 35 | |
| 36 | # edit it to make sure it says: |
| 37 | # --listen-http=-3 |
| 38 | # not: |
| 39 | # --listen-https=-3 |
| 40 | sudo nano /etc/systemd/system/systemd-journal-remote.service |
| 41 | |
| 42 | # reload systemd |
| 43 | sudo systemctl daemon-reload |
| 44 | ``` |
| 45 | |
| 46 | Optionally, if you want to change the port (the default is `19532`), edit `systemd-journal-remote.socket` |
| 47 | |
| 48 | ```bash |
| 49 | # edit the socket file |
| 50 | sudo systemctl edit systemd-journal-remote.socket |
| 51 | ``` |
| 52 | |
| 53 | and add the following lines into the instructed place, and choose your desired port; save and exit. |
| 54 | |
| 55 | ```bash |
| 56 | [Socket] |
| 57 | ListenStream=<DESIRED_PORT> |
| 58 | ``` |
| 59 | |
| 60 | Finally, enable it, so that it will start automatically upon receiving a connection: |
| 61 | |
| 62 | ```bash |
| 63 | # enable systemd-journal-remote |
| 64 | sudo systemctl enable --now systemd-journal-remote.socket |
| 65 | sudo systemctl enable systemd-journal-remote.service |
| 66 | ``` |
| 67 | |
| 68 | `systemd-journal-remote` is now listening for incoming journals from remote hosts. |
| 69 | |
| 70 | ## Client configuration |
| 71 | |
| 72 | On the clients, install `systemd-journal-remote` (it includes `systemd-journal-upload`): |
| 73 | |
| 74 | ```bash |
| 75 | # change this according to your distro |
| 76 | sudo apt-get install systemd-journal-remote |
| 77 | ``` |
| 78 | |
| 79 | Edit `/etc/systemd/journal-upload.conf` and set the IP address and the port of the server, like so: |
| 80 | |
| 81 | ```text |
| 82 | [Upload] |
| 83 | URL=http://centralization.server.ip:19532 |
| 84 | ``` |
| 85 | |
| 86 | Edit `systemd-journal-upload`, and add `Restart=always` to make sure the client will keep trying to push logs, even if the server is temporarily not there, like this: |
| 87 | |
| 88 | ```bash |
| 89 | sudo systemctl edit systemd-journal-upload |
| 90 | ``` |
| 91 | |
| 92 | At the top, add: |
| 93 | |
| 94 | ```text |
| 95 | [Service] |
| 96 | Restart=always |
| 97 | ``` |
| 98 | |
| 99 | Enable and start `systemd-journal-upload`, like this: |
| 100 | |
| 101 | ```bash |
| 102 | sudo systemctl enable systemd-journal-upload |
| 103 | sudo systemctl start systemd-journal-upload |
| 104 | ``` |
| 105 | |
| 106 | ## Verify it works |
| 107 | |
| 108 | To verify that the central server is receiving logs, run this on the central server: |
| 109 | |
| 110 | ```bash |
| 111 | sudo ls -l /var/log/journal/remote/ |
| 112 | ``` |
| 113 | |
| 114 | You should see new files from the client's IP. |
| 115 | |
| 116 | Also, `systemctl status systemd-journal-remote` should show something like this: |
| 117 | |
| 118 | ```bash |
| 119 | systemd-journal-remote.service - Journal Remote Sink Service |
| 120 | Loaded: loaded (/etc/systemd/system/systemd-journal-remote.service; indirect; preset: disabled) |
| 121 | Active: active (running) since Sun 2023-10-15 14:29:46 EEST; 2h 24min ago |
| 122 | TriggeredBy: ● systemd-journal-remote.socket |
| 123 | Docs: man:systemd-journal-remote(8) |
| 124 | man:journal-remote.conf(5) |
| 125 | Main PID: 2118153 (systemd-journal) |
| 126 | Status: "Processing requests..." |
| 127 | Tasks: 1 (limit: 154152) |
| 128 | Memory: 2.2M |
| 129 | CPU: 71ms |
| 130 | CGroup: /system.slice/systemd-journal-remote.service |
| 131 | └─2118153 /usr/lib/systemd/systemd-journal-remote --listen-http=-3 --output=/var/log/journal/remote/ |
| 132 | ``` |
| 133 | |
| 134 | Note the `status: "Processing requests..."` and the PID under `CGroup`. |
| 135 | |
| 136 | On the client `systemctl status systemd-journal-upload` should show something like this: |
| 137 | |
| 138 | ```bash |
| 139 | ● systemd-journal-upload.service - Journal Remote Upload Service |
| 140 | Loaded: loaded (/lib/systemd/system/systemd-journal-upload.service; enabled; vendor preset: disabled) |
| 141 | Drop-In: /etc/systemd/system/systemd-journal-upload.service.d |
| 142 | └─override.conf |
| 143 | Active: active (running) since Sun 2023-10-15 10:39:04 UTC; 3h 17min ago |
| 144 | Docs: man:systemd-journal-upload(8) |
| 145 | Main PID: 4169 (systemd-journal) |
| 146 | Status: "Processing input..." |
| 147 | Tasks: 1 (limit: 13868) |
| 148 | Memory: 3.5M |
| 149 | CPU: 1.081s |
| 150 | CGroup: /system.slice/systemd-journal-upload.service |
| 151 | └─4169 /lib/systemd/systemd-journal-upload --save-state |
| 152 | ``` |
| 153 | |
| 154 | Note the `Status: "Processing input..."` and the PID under `CGroup`. |