Fix issues on ebpf.plugin (#9333)
Fix missing SIGPIPE and update the ebpf.pluigin documentation.
thiagoftsm committed
Jun 16, 2020 at 12:05 UTC
95e7b62151d0087a7a5951e97c6a2154f4c1f2de
4 files changed
+111
collectors/ebpf.plugin/README.md
+82
@@ -250,3 +250,85 @@ Our [initial testing](https://github.com/netdata/netdata/issues/8195) shows the
250
nearly identical to our [apps.plugin collector](/collectors/apps.plugin/README.md), despite collecting and displaying
251
much more sophisticated metrics. You can now use the eBPF to gather deeper insights without affecting the performance of
252
your complex applications at any load.
253
+
254
+## SELinux
255
+
256
+When [SELinux](https://www.redhat.com/en/topics/linux/what-is-selinux) is enabled, it may prevent `ebpf.plugin` from
257
+starting correctly. Check the Agent's `error.log` file for errors like the ones below:
258
+
259
+```bash
260
+2020-06-14 15:32:08: ebpf.plugin ERROR : EBPF PROCESS : Cannot load program: /usr/libexec/netdata/plugins.d/pnetdata_ebpf_process.3.10.0.o (errno 13, Permission denied)
261
+2020-06-14 15:32:19: netdata ERROR : PLUGINSD[ebpf] : read failed: end of file (errno 9, Bad file descriptor)
262
+```
263
+
264
+You can also check for errors related to `ebpf.plugin` inside `/var/log/audit/audit.log`:
265
+
266
+```bash
267
+type=AVC msg=audit(1586260134.952:97): avc: denied { map_create } for pid=1387 comm="ebpf.pl" scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=bpf permissive=0
268
+type=SYSCALL msg=audit(1586260134.952:97): arch=c000003e syscall=321 success=no exit=-13 a0=0 a1=7ffe6b36f000 a2=70 a3=0 items=0 ppid=1135 pid=1387 auid=4294967295 uid=994 gid=990 euid=0 suid=0 fsuid=0 egid=990 sgid=990 fsgid=990 tty=(none) ses=4294967295 comm="ebpf_proc
269
+ess.pl" exe="/usr/libexec/netdata/plugins.d/ebpf.plugin" subj=system_u:system_r:unconfined_service_t:s0 key=(null)
270
+```
271
+
272
+If you see similar errors, you will have to adjust SELinux's policies to enable the eBPF collector.
273
+
274
+### Creation of bpf policies
275
+
276
+To enable `ebpf.plugin` to run on a distribution with SELinux enabled, it will be necessary to take the following
277
+actions.
278
+
279
+First, stop the Netdata Agent.
280
+
281
+```bash
282
+# systemctl stop netdata
283
+```
284
+
285
+Next, create a policy with the `audit.log` file you examined earlier.
286
+
287
+```bash
288
+# grep ebpf.plugin /var/log/audit/audit.log | audit2allow -M netdata_ebpf
289
+```
290
+
291
+This will create two new files: `netdata_ebpf.te` and `netdata_ebpf.mod`.
292
+
293
+Edit the `netdata_ebpf.te` file to change the options `class` and `allow`. You should have the following at the end of
294
+the `netdata_ebpf.te` file.
295
+
296
+```conf
297
+module netdata_ebpf 1.0;
298
+require {
299
+ type unconfined_service_t;
300
+ class bpf { map_create map_read map_write prog_load prog_run };
301
+}
302
+#============= unconfined_service_t ==============
303
+allow unconfined_service_t self:bpf { map_create map_read map_write prog_load prog_run };
304
+```
305
+
306
+Then compile your `netdata_ebpf.te` file with the following commands to create a binary that loads the new policies:
307
+
308
+```bash
309
+# checkmodule -M -m -o netdata_ebpf.mod netdata_ebpf.te
310
+# semodule_package -o netdata_ebpf.pp -m netdata_ebpf.mod
311
+```
312
+
313
+Finally, you can load the new policy and start the Netdata agent again:
314
+
315
+```bash
316
+# semodule -i netdata_ebpf.pp
317
+# systemctl start netdata
318
+```
319
+
320
+## Lockdown
321
+
322
+Beginning with [version 5.4](https://www.zdnet.com/article/linux-to-get-kernel-lockdown-feature/), the Linux kernel has
323
+a feature called "lockdown," which may affect `ebpf.plugin` depending how the kernel was compiled. The following table
324
+shows how the lockdown module impacts `ebpf.plugin` based on the selected options:
325
+
326
+| Enforcing kernel lockdown | Enable lockdown LSM early in init | Default lockdown mode | Can `ebpf.plugin` run with this? |
327
+|:------------------------- |:--------------------------------- |:--------------------- |:-------------------------------- |
328
+| YES | NO | NO | YES |
329
+| YES | Yes | None | YES |
330
+| YES | Yes | Integrity | YES |
331
+| YES | Yes | Confidentiality | NO |
332
+
333
+If you or your distribution compiled the kernel with the last combination, your system cannot load shared libraries
334
+required to run `ebpf.plugin`.
collectors/ebpf.plugin/ebpf.c
+1
@@ -904,6 +904,7 @@ int main(int argc, char **argv)
904
905
signal(SIGINT, ebpf_exit);
906
signal(SIGTERM, ebpf_exit);
907
+ signal(SIGPIPE, ebpf_exit);
908
909
if (ebpf_start_pthread_variables()) {
910
thread_finished++;
web/gui/dashboard.js
+14
@@ -792,6 +792,13 @@ NETDATA.unitsConversion = {
792
'gigabits/s': 1000000,
793
'terabits/s': 1000000000
794
},
795
+ 'bytes/s': {
796
+ 'bytes/s': 1,
797
+ 'kilobytes/s': 1024,
798
+ 'megabytes/s': 1024 * 1024,
799
+ 'gigabytes/s': 1024 * 1024 * 1024,
800
+ 'terabytes/s': 1024 * 1024 * 1024 * 1024
801
+ },
802
'kilobytes/s': {
803
'bytes/s': 1 / 1024,
804
'kilobytes/s': 1,
@@ -799,6 +806,13 @@ NETDATA.unitsConversion = {
806
'gigabytes/s': 1024 * 1024,
807
'terabytes/s': 1024 * 1024 * 1024
808
},
809
+ 'B/s': {
810
+ 'B/s': 1,
811
+ 'KiB/s': 1024,
812
+ 'MiB/s': 1024 * 1024,
813
+ 'GiB/s': 1024 * 1024 * 1024,
814
+ 'TiB/s': 1024 * 1024 * 1024 * 1024
815
+ },
816
'KB/s': {
817
'B/s': 1 / 1024,
818
'KB/s': 1,
web/gui/src/dashboard.js/units-conversion.js
+14
@@ -25,6 +25,13 @@ NETDATA.unitsConversion = {
25
'gigabits/s': 1000000,
26
'terabits/s': 1000000000
27
},
28
+ 'bytes/s': {
29
+ 'bytes/s': 1,
30
+ 'kilobytes/s': 1024,
31
+ 'megabytes/s': 1024 * 1024,
32
+ 'gigabytes/s': 1024 * 1024 * 1024,
33
+ 'terabytes/s': 1024 * 1024 * 1024 * 1024
34
+ },
35
'kilobytes/s': {
36
'bytes/s': 1 / 1024,
37
'kilobytes/s': 1,
@@ -32,6 +39,13 @@ NETDATA.unitsConversion = {
39
'gigabytes/s': 1024 * 1024,
40
'terabytes/s': 1024 * 1024 * 1024
41
},
42
+ 'B/s': {
43
+ 'B/s': 1,
44
+ 'KiB/s': 1024,
45
+ 'MiB/s': 1024 * 1024,
46
+ 'GiB/s': 1024 * 1024 * 1024,
47
+ 'TiB/s': 1024 * 1024 * 1024 * 1024
48
+ },
49
'KB/s': {
50
'B/s': 1 / 1024,
51
'KB/s': 1,