Apps plugin improvements2 (#18673)
* improvements * simple patterns can now be configured to run without separators; added netdata and spawn-plugins as process managers; updated documentation * cosmetic changes * fix issue in rrdlabels sanitizer * fix text_sanitizers to skip leading spaces * use quoted_strings_splitter_whitespace() instead of quoted_strings_splitter_pluginsd() * remove know extensions from executable files * detect sh -c exec ... and extract the right process name * workaround for infine loop in cgroup-network with sanitization enabled
Costa Tsaousis committed
Oct 3, 2024 at 23:00 UTC
82d91954aeab3208792255afc7f41ab25915c6c6
49 files changed
+1077
-927
CMakeLists.txt
+10
@@ -804,6 +804,15 @@ set(LIBNETDATA_FILES
804
src/libnetdata/parsers/entries.h
805
src/libnetdata/sanitizers/chart_id_and_name.c
806
src/libnetdata/sanitizers/chart_id_and_name.h
807
+ src/libnetdata/sanitizers/utf8-sanitizer.c
808
+ src/libnetdata/sanitizers/utf8-sanitizer.h
809
+ src/libnetdata/sanitizers/sanitizers.h
810
+ src/libnetdata/sanitizers/sanitizers-labels.c
811
+ src/libnetdata/sanitizers/sanitizers-labels.h
812
+ src/libnetdata/sanitizers/sanitizers-functions.c
813
+ src/libnetdata/sanitizers/sanitizers-functions.h
814
+ src/libnetdata/sanitizers/sanitizers-pluginsd.c
815
+ src/libnetdata/sanitizers/sanitizers-pluginsd.h
816
)
817
818
if(ENABLE_PLUGIN_EBPF)
@@ -1897,6 +1906,7 @@ if(ENABLE_PLUGIN_APPS)
1906
src/collectors/apps.plugin/apps_os_windows.c
1907
src/collectors/apps.plugin/apps_incremental_collection.c
1908
src/collectors/apps.plugin/apps_os_windows_nt.c
1909
+ src/collectors/apps.plugin/apps_pid_match.c
1910
)
1911
1912
add_executable(apps.plugin ${APPS_PLUGIN_FILES})
src/collectors/apps.plugin/README.md
+91
-53
@@ -4,51 +4,54 @@
4
5
## Process Aggregation and Grouping
6
7
-`apps.plugin` aggregates processes in three distinct ways to provide a more insightful
8
-breakdown of resource utilization:
7
+`apps.plugin` aggregates processes in three distinct ways to provide a more
8
+insightful breakdown of resource utilization:
9
10
- **Tree** or **Category**: Grouped by their position in the process tree.
11
- This is customizable and allows aggregation by process managers and individual
12
- processes of interest. Allows also renaming the processes for presentation purposes.
11
+ This is customizable and allows aggregation by process managers and
12
+ individual processes of interest. Allows also renaming the processes for
13
+ presentation purposes.
14
15
- **User**: Grouped by the effective user (UID) under which the processes run.
16
16
-- **Group**: Grouped by the effective group (GID) under which the processes run.
17
+- **Group**: Grouped by the effective group (GID) under which the processes
18
+ run.
19
20
## Short-Lived Process Handling
21
20
-`apps.plugin` accounts for resource utilization of both running and exited processes,
21
-capturing the impact of processes that spawn short-lived subprocesses, such as shell
22
-scripts that fork hundreds or thousands of times per second. So, although processes
23
-may spawn short lived sub-processes, `apps.plugin` will aggregate their resources
24
-utilization providing a holistic view of how resources are shared among the processes.
22
+`apps.plugin` accounts for resource utilization of both running and exited
23
+processes, capturing the impact of processes that spawn short-lived
24
+subprocesses, such as shell scripts that fork hundreds or thousands of times
25
+per second. So, although processes may spawn short-lived sub-processes,
26
+`apps.plugin` will aggregate their resources utilization providing a holistic
27
+view of how resources are shared among the processes.
28
29
## Charts sections
30
28
-To provide more valuable insights, apps.plugin aggregates individual processes in several ways.
29
-Each type of aggregation is presented as a different section on the dashboard.
31
+To provide more valuable insights, apps.plugin aggregates individual processes
32
+in several ways. Each type of aggregation is presented as a different section
33
+on the dashboard.
34
35
### Custom Process Groups (Apps)
36
33
-In this section, apps.plugin summarizes the resources consumed by all processes, grouped based
34
-on the groups provided in `/etc/netdata/apps_groups.conf`. You can edit this file using our [`edit-config`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script.
37
+In this section, apps.plugin summarizes the resources consumed by all
38
+processes, grouped based on their position in the process tree and the groups
39
+provided in `/etc/netdata/apps_groups.conf`. You can edit this file using our
40
+[`edit-config`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script.
41
36
-For this section, `apps.plugin` builds a process tree (much like `ps fax` does in Linux), and groups
37
-processes together (evaluating both child and parent processes) so that the result is always a list with
38
-a predefined set of members (of course, only process groups found running are reported).
39
-
40
-> If you find that `apps.plugin` categorizes standard applications as `other`, we would be
41
-> glad to accept pull requests improving the defaults shipped with Netdata in `apps_groups.conf`.
42
+For this section, `apps.plugin` builds a process tree (much like `ps fax` does
43
+in Linux), and groups processes together (evaluating both child and parent
44
+processes).
45
46
### By User (Users)
47
45
-In this section, apps.plugin summarizes the resources consumed by all processes, grouped by the
46
-effective user under which each process runs.
48
+In this section, apps.plugin summarizes the resources consumed by all
49
+processes, grouped by the effective user under which each process runs.
50
51
### By User Group (Groups)
52
50
-In this section, apps.plugin summarizes the resources consumed by all processes, grouped by the
51
-effective user group under which each process runs.
53
+In this section, apps.plugin summarizes the resources consumed by all
54
+processes, grouped by the effective user group under which each process runs.
55
56
## Charts
57
@@ -97,14 +100,14 @@ The above are reported:
100
101
## Performance
102
100
-`apps.plugin` is a complex piece of software and has a lot of work to do
101
-We are proud that `apps.plugin` is a lot faster compared to any other similar tool,
102
-while collecting a lot more information for the processes, however the fact is that
103
-this plugin may require more CPU resources than the `netdata` daemon itself.
103
+We are proud that `apps.plugin` is a lot faster compared to any other similar
104
+tools, while collecting a lot more information for the processes, however the
105
+fact is that this plugin needs to traverse the entire process tree on every
106
+iteration, so its resources usage may be noticable.
107
105
-Under Linux, for each process running, `apps.plugin` reads several `/proc` files
106
-per process. Doing this work per-second, especially on hosts with several thousands
107
-of processes, may increase the CPU resources consumed by the plugin.
108
+Under Linux, for each process running, `apps.plugin` reads several `/proc`
109
+files per process. Doing this work per-second, especially on hosts with several
110
+thousands of processes, may increase the CPU resources consumed by the plugin.
111
112
In such cases, you many need to lower its data collection frequency.
113
@@ -116,20 +119,23 @@ To do this, edit `/etc/netdata/netdata.conf` and find this section:
119
# command options =
120
```
121
119
-Uncomment the line `update every` and set it to a higher number. If you just set it to `2`,
120
-its CPU resources will be cut in half, and data collection will be once every 2 seconds.
122
+Uncomment the line `update every` and set it to a higher number. If you just
123
+set it to `2`, its CPU resources will be cut in half, and data collection will
124
+be once every 2 seconds.
125
126
## Configuration
127
124
-The configuration file is `/etc/netdata/apps_groups.conf`. You can edit this file using our [`edit-config`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script.
128
+The configuration file is `/etc/netdata/apps_groups.conf`. You can edit this
129
+file using our [`edit-config`](/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script.
130
131
### Configuring process managers
132
128
-`apps.plugin` needs to know the common process managers, meaning the names of the processes
129
-which spawn other processes. Process managers are used so that `apps.plugin` will automatically
130
-consider all their sub-processes important to monitor.
133
+`apps.plugin` needs to know the common process managers, the names of the processes
134
+which spawn other processes. Process managers help `apps.plugin` automatically
135
+consider all their sub-processes, important to monitor.
136
132
-Process managers are configured in `apps_groups.conf` with the prefix `managers:`, like this:
137
+Process managers are configured in `apps_groups.conf` with the prefix
138
+`managers:`, like this:
139
140
```txt
141
managers: process1 process2 process3
@@ -137,9 +143,32 @@ managers: process1 process2 process3
143
144
Multiple lines may exist, all starting with `managers:`.
145
140
-The process names given here should be exactly as the operating system sets them. In Linux these
141
-process names are limited to 15 characters. Usually the command `ps -e` or `cat /proc/{PID}/stat`
142
-states the names needed here.
146
+A line `managers: clear` will clear all managers, so that a new list can be
147
+provided.
148
+
149
+### Configuring interpreters
150
+
151
+Interpreted languages like `python`, `bash`, `sh`, `node` and more, may hide
152
+the actual name of a process.
153
+
154
+For such programs, `apps.plugin` can be instructed to check for the actual
155
+process name in one of the command line parameters of the program. When a
156
+process matches an interpreter, apps.plugin will go through all the parameters
157
+of the interpreter and find the first parameter that is an absolute filename
158
+existing on disk. When found, `apps.plugin` will name the process using
159
+the name of that filename.
160
+
161
+Interpreters are configured in `apps_groups.conf` with the prefix
162
+`interpreters:`, like this:
163
+
164
+```txt
165
+interpreters: process1 process2 process3
166
+```
167
+
168
+Multiple lines may exist, all starting with `interpreters:`.
169
+
170
+A line `interpreters: clear` will clear all interpreters, so that a new list
171
+can be provided.
172
173
### Configuring process groups and renaming processes
174
@@ -151,16 +180,20 @@ group: process1 process2 ...
180
181
Each group can be given multiple times, to add more processes to it.
182
154
-For each process given, all of its sub-processes will be grouped, not just the matched process.
183
+For each process given, all of its sub-processes will be grouped, not just the
184
+matched process.
185
+
186
+### Matching processes
187
188
The process names are the ones returned by:
189
190
- **comm**: `ps -e` or `cat /proc/{PID}/stat`
191
- **cmdline**: in case of substring mode (see below): `/proc/{PID}/cmdline`
192
161
-On Linux **comm** is limited to just a few characters. `apps.plugin` attempts to find the entire
162
-**comm** name by looking for it at the **cmdline**. When this is successful, the entire process name
163
-is available, otherwise the shortened one is used.
193
+On Linux **comm** is limited to 15 characters. `apps.plugin` attempts to find
194
+the entire **comm** name by looking for it at the **cmdline**. When this is
195
+successful, the entire process name is available, otherwise the shortened one
196
+is used.
197
198
To add process names with spaces, enclose them in quotes (single or double)
199
example: `'Plex Media Serv'` or `"my other process"`.
@@ -171,18 +204,23 @@ You can add asterisks (`*`) to provide a pattern:
204
- `name*` _prefix_ mode: will match a **comm** beginning with `name`.
205
- `*name*` _substring_ mode: will search for `name` in **cmdline**.
206
174
-Asterisks may appear in the middle of `name` (like `na*me`), without affecting what is being
175
-matched (**comm** or **cmdline**).
207
+Asterisks may appear in the middle of `name` (like `na*me`), without affecting
208
+what is being matched (**comm** or **cmdline**).
209
+
210
+To add processes with single quotes, enclose them in double quotes:
211
+`"process with this ' single quote"`.
212
177
-To add processes with single quotes, enclose them in double quotes: `"process with this ' single quote"`
213
+To add processes with double quotes, enclose them in single quotes:
214
+`'process with this " double quote'`.
215
179
-To add processes with double quotes, enclose them in single quotes: `'process with this " double quote'`
216
+The order of the entries in this list is important: the first one that matches
217
+a process is used, so follow a top-down hierarchy. Processes not matched by any
218
+row, will inherit it from their parents.
219
181
-The order of the entries in this list is important: the first one that matches a process is used, so follow a top-down hierarchy.
182
-Processes not matched by any row, will inherit it from their parents.
220
+There are a few command line options you can pass to `apps.plugin`. The list of
221
+available options can be acquired with the `--help` flag. The options can be
222
+set in the `netdata.conf` using the [`edit-config` script](/docs/netdata-agent/configuration/README.md).
223
184
-There are a few command line options you can pass to `apps.plugin`. The list of available
185
-options can be acquired with the `--help` flag. The options can be set in the `netdata.conf` using the [`edit-config` script](/docs/netdata-agent/configuration/README.md).
224
For example, to disable user and user group charts you would set:
225
226
```txt
src/collectors/apps.plugin/apps_aggregations.c
+6
-49
@@ -110,61 +110,18 @@ static inline void cleanup_exited_pids(void) {
110
}
111
}
112
113
-static struct target *matched_apps_groups_target(struct pid_stat *p, struct target *w) {
114
- if(is_process_manager(p))
115
- return NULL;
116
-
117
- p->matched_by_config = true;
118
- return w->target ? w->target : w;
119
-}
120
-
113
static struct target *get_apps_groups_target_for_pid(struct pid_stat *p) {
114
targets_assignment_counter++;
115
116
for(struct target *w = apps_groups_root_target; w ; w = w->next) {
117
if(w->type != TARGET_TYPE_APP_GROUP) continue;
118
127
- if(!w->starts_with && !w->ends_with) {
128
- if(w->ag.pattern) {
129
- if(simple_pattern_matches_string(w->ag.pattern, p->comm))
130
- return matched_apps_groups_target(p, w);
131
- }
132
- else {
133
- if(w->ag.compare == p->comm || w->ag.compare == p->comm_orig)
134
- return matched_apps_groups_target(p, w);
135
- }
136
- }
137
- else if(w->starts_with && !w->ends_with) {
138
- if(w->ag.pattern) {
139
- if(simple_pattern_matches_string(w->ag.pattern, p->comm))
140
- return matched_apps_groups_target(p, w);
141
- }
142
- else {
143
- if(string_starts_with_string(p->comm, w->ag.compare) ||
144
- (p->comm != p->comm_orig && string_starts_with_string(p->comm, w->ag.compare)))
145
- return matched_apps_groups_target(p, w);
146
- }
147
- }
148
- else if(!w->starts_with && w->ends_with) {
149
- if(w->ag.pattern) {
150
- if(simple_pattern_matches_string(w->ag.pattern, p->comm))
151
- return matched_apps_groups_target(p, w);
152
- }
153
- else {
154
- if(string_ends_with_string(p->comm, w->ag.compare) ||
155
- (p->comm != p->comm_orig && string_ends_with_string(p->comm, w->ag.compare)))
156
- return matched_apps_groups_target(p, w);
157
- }
158
- }
159
- else if(w->starts_with && w->ends_with && p->cmdline) {
160
- if(w->ag.pattern) {
161
- if(simple_pattern_matches_string(w->ag.pattern, p->cmdline))
162
- return matched_apps_groups_target(p, w);
163
- }
164
- else {
165
- if(strstr(string2str(p->cmdline), string2str(w->ag.compare)))
166
- return matched_apps_groups_target(p, w);
167
- }
119
+ if(pid_match_check(p, &w->match)) {
120
+ if(p->is_manager)
121
+ return NULL;
122
+
123
+ p->matched_by_config = true;
124
+ return w->target ? w->target : w;
125
}
126
}
127
src/collectors/apps.plugin/apps_functions.c
+1
-1
@@ -86,7 +86,7 @@ void function_processes(const char *transaction, char *function,
86
access, HTTP_ACCESS_SIGNED_ID | HTTP_ACCESS_SAME_SPACE | HTTP_ACCESS_SENSITIVE_DATA | HTTP_ACCESS_VIEW_AGENT_CONFIG) || enable_function_cmdline;
87
88
char *words[PLUGINSD_MAX_WORDS] = { NULL };
89
- size_t num_words = quoted_strings_splitter_pluginsd(function, words, PLUGINSD_MAX_WORDS);
89
+ size_t num_words = quoted_strings_splitter_whitespace(function, words, PLUGINSD_MAX_WORDS);
90
91
struct target *category = NULL, *user = NULL, *group = NULL; (void)category; (void)user; (void)group;
92
const char *process_name = NULL;
src/collectors/apps.plugin/apps_groups.conf
+25
-28
@@ -4,49 +4,46 @@
4
## Documentation at:
5
## https://github.com/netdata/netdata/blob/master/src/collectors/apps.plugin/README.md
6
##
7
-## Subprocesses of process managers are monitored.
8
-## (uncomment to edit - the default is also hardcoded into the plugin)
7
+## -----------------------------------------------------------------------------
8
+## Subprocesses of process managers are monitored individually.
9
+## (uncomment to add or edit - the default is also hardcoded into the plugin)
10
+
11
+## Clear all the managers, to set yours, otherwise append to the internal list.
12
+#managers: clear
13
14
## Linux process managers
15
#managers: init systemd containerd-shim-runc-v2 dumb-init gnome-shell docker-init
12
-#managers: openrc-run.sh crond plasmashell xfwm4
16
+#managers: spawn-plugins openrc-run.sh crond plasmashell xfwm4
17
18
## FreeBSD process managers
15
-#managers: init
19
+#managers: init spawn-plugins
20
21
## MacOS process managers
18
-#managers: launchd
22
+#managers: launchd spawn-plugins
23
24
## Windows process managers
21
-#managers: wininit services explorer System
25
+#managers: wininit services explorer System netdata
26
+
27
+## -----------------------------------------------------------------------------
28
+## Interpreters to search for the actual command name in command line.
29
+## (uncomment to add or edit - the default is also hardcoded into the plugin)
30
+
31
+## Clear all the interpreters, to set yours, otherwise append to the internal list.
32
+#interpreters: clear
33
+
34
+#interpreters: python python2 python3
35
+#interpreters: sh bash zsh
36
+#interpreters: node perl awk
37
38
## -----------------------------------------------------------------------------
39
## Processes of interest
40
+## Grouping and/or rename individual processes.
41
+## (there is no internal default for this section)
42
43
## NETDATA processes accounting
44
netdata: netdata
28
-## netdata known plugins
29
-## plugins not defined here will be accumulated into netdata, above
30
-apps.plugin: *apps.plugin*
31
-go.d.plugin: *go.d.plugin*
32
-systemd-journal.plugin: *systemd-journal.plugin*
33
-network-viewer.plugin: *network-viewer.plugin*
34
-windows-events.plugin: *windows-events.plugin*
35
-cups.plugin: *cups.plugin*
36
-perf.plugin: *perf.plugin*
37
-nfacct.plugin: *nfacct.plugin*
38
-xenstat.plugin: *xenstat.plugin*
39
-freeipmi.plugin: *freeipmi.plugin*
40
-charts.d.plugin: *charts.d.plugin*
41
-python.d.plugin: *python.d.plugin*
42
-slabinfo.plugin: *slabinfo.plugin*
43
-ebpf.plugin: *ebpf.plugin*
44
-debugfs.plugin: *debugfs.plugin*
45
-tc-qos-helper: *tc-qos-helper.sh*
46
-fping: fping
47
-ioping: ioping
48
-
49
-## agent-service-discovery
45
+
46
+## NETDATA agent-service-discovery (kubernetes)
47
agent_sd: agent_sd
48
49
## -----------------------------------------------------------------------------
src/collectors/apps.plugin/apps_incremental_collection.c
+2
-3
@@ -174,15 +174,14 @@ int read_proc_pid_cmdline(struct pid_stat *p) {
174
if(unlikely(!OS_FUNCTION(apps_os_get_pid_cmdline)(p, cmdline, sizeof(cmdline))))
175
goto cleanup;
176
177
- string_freez(p->cmdline);
178
- p->cmdline = string_strdupz(cmdline);
177
+ update_pid_cmdline(p, cmdline);
178
179
return 1;
180
181
cleanup:
182
// copy the command to the command line
183
string_freez(p->cmdline);
185
- p->cmdline = string_dup(p->comm);
184
+ p->cmdline = NULL;
185
return 0;
186
}
187
#endif
src/collectors/apps.plugin/apps_os_windows.c
+25
-18
@@ -521,15 +521,24 @@ static char *wchar_to_utf8(WCHAR *s) {
521
return utf8;
522
}
523
524
-// Convert wide string to UTF-8
525
-static STRING *wchar_to_string(WCHAR *s) {
526
- return string_strdupz(wchar_to_utf8(s));
524
+static char *ansi_to_utf8(LPCSTR str) {
525
+ static __thread WCHAR unicode[PATH_MAX];
526
+ static __thread int unicode_size = sizeof(unicode) / sizeof(*unicode);
527
+
528
+ // Step 1: Convert ANSI string (LPSTR) to wide string (UTF-16)
529
+ int wideLength = MultiByteToWideChar(CP_ACP, 0, str, -1, NULL, 0);
530
+ if (wideLength == 0 || wideLength > unicode_size)
531
+ return NULL;
532
+
533
+ MultiByteToWideChar(CP_ACP, 0, str, -1, unicode, wideLength);
534
+
535
+ return wchar_to_utf8(unicode);
536
}
537
538
// --------------------------------------------------------------------------------------------------------------------
539
540
// return a sanitized name for the process
532
-STRING *GetProcessFriendlyNameSanitized(WCHAR *path) {
541
+STRING *GetProcessFriendlyNameFromPathSanitized(WCHAR *path) {
542
static __thread uint8_t void_buf[1024 * 1024];
543
static __thread DWORD void_buf_size = sizeof(void_buf);
544
static __thread wchar_t unicode[PATH_MAX];
@@ -548,7 +557,7 @@ STRING *GetProcessFriendlyNameSanitized(WCHAR *path) {
557
wcsncpy(unicode, value, unicode_size - 1);
558
unicode[unicode_size - 1] = L'\0';
559
char *name = wchar_to_utf8(unicode);
551
- sanitize_chart_meta(name);
560
+ sanitize_apps_plugin_chart_meta(name);
561
return string_strdupz(name);
562
}
563
}
@@ -573,7 +582,7 @@ static STRING *GetNameFromCmdlineSanitized(struct pid_stat *p) {
582
char service[strlen(words[i + 1]) + sizeof(SERVICE_PREFIX)]; // sizeof() includes a null
583
strcpy(service, SERVICE_PREFIX);
584
strcpy(&service[sizeof(SERVICE_PREFIX) - 1], words[i + 1]);
576
- sanitize_chart_meta(service);
585
+ sanitize_apps_plugin_chart_meta(service);
586
return string_strdupz(service);
587
}
588
}
@@ -621,13 +630,12 @@ static void GetServiceNames(void) {
630
if(p && !p->got_service) {
631
p->got_service = true;
632
624
- size_t len = strlen(pServiceStatus[i].lpDisplayName);
625
- char buf[len + 1];
626
- memcpy(buf, pServiceStatus[i].lpDisplayName, sizeof(buf));
627
- sanitize_chart_meta(buf);
628
-
629
- string_freez(p->name);
630
- p->name = string_strdupz(buf);
633
+ char *name = ansi_to_utf8(pServiceStatus[i].lpDisplayName);
634
+ if(name) {
635
+ sanitize_apps_plugin_chart_meta(name);
636
+ string_freez(p->name);
637
+ p->name = string_strdupz(name);
638
+ }
639
}
640
}
641
@@ -695,14 +703,13 @@ void GetAllProcessesInfo(void) {
703
{
704
WCHAR *cmdline = GetProcessCommandLine(hProcess); // returns malloc'd buffer
705
if (cmdline) {
698
- string_freez(p->cmdline);
699
- p->cmdline = wchar_to_string(cmdline);
706
+ update_pid_cmdline(p, wchar_to_utf8(cmdline));
707
708
// extract the process full path from the command line
709
WCHAR *path = executable_path_from_cmdline(cmdline);
710
if(path) {
711
string_freez(p->name);
705
- p->name = GetProcessFriendlyNameSanitized(path);
712
+ p->name = GetProcessFriendlyNameFromPathSanitized(path);
713
}
714
715
free(cmdline); // free(), not freez()
@@ -713,10 +720,10 @@ void GetAllProcessesInfo(void) {
720
if (QueryFullProcessImageNameW(hProcess, 0, unicode, &unicode_size)) {
721
// put the full path name to the command into cmdline
722
if(!p->cmdline)
716
- p->cmdline = wchar_to_string(unicode);
723
+ update_pid_cmdline(p, wchar_to_utf8(unicode));
724
725
if(!p->name)
719
- p->name = GetProcessFriendlyNameSanitized(unicode);
726
+ p->name = GetProcessFriendlyNameFromPathSanitized(unicode);
727
}
728
}
729
src/collectors/apps.plugin/apps_pid.c
+160
-31
@@ -317,37 +317,165 @@ static inline void link_all_processes_to_their_parents(void) {
317
318
// --------------------------------------------------------------------------------------------------------------------
319
320
-static inline STRING *comm_from_cmdline_sanitized(char *comm, STRING *cmdline) {
321
- if(!cmdline) {
322
- sanitize_chart_meta(comm);
323
- return string_strdupz(comm);
320
+static bool is_filename(const char *s) {
321
+ if(!s || !*s) return false;
322
+
323
+#if defined(OS_WINDOWS)
324
+ if( (isalpha((uint8_t)*s) || (s[1] == ':' && s[2] == '\\')) || // windows native "x:\"
325
+ (isalpha((uint8_t)*s) || (s[1] == ':' && s[2] == '/')) || // windows native "x:/"
326
+ (*s == '\\' && s[1] == '\\' && isalpha((uint8_t)s[2]) && s[3] == '\\') || // windows native "\\x\"
327
+ (*s == '/' && s[1] == '/' && isalpha((uint8_t)s[2]) && s[3] == '/')) { // windows native "//x/"
328
+
329
+ WCHAR ws[FILENAME_MAX];
330
+ int wlen = MultiByteToWideChar(CP_UTF8, 0, s, -1, NULL, 0);
331
+ if (wlen <= 0 || (size_t)wlen > sizeof(ws) / sizeof(*ws)) {
332
+ return false; // Failed to convert UTF-8 to UTF-16
333
+ }
334
+
335
+ MultiByteToWideChar(CP_UTF8, 0, s, -1, ws, wlen);
336
+ DWORD attributes = GetFileAttributesW(ws);
337
+ if (attributes != INVALID_FILE_ATTRIBUTES)
338
+ return true;
339
}
340
+#endif
341
326
- const char *cl = string2str(cmdline);
327
- size_t len = string_strlen(cmdline);
342
+ // for: sh -c "exec /path/to/command parameters"
343
+ if(strncmp(s, "exec ", 5) == 0 && s[5]) {
344
+ s += 5;
345
+ char look_for = ' ';
346
+ if(*s == '\'') { look_for = '\''; s++; }
347
+ if(*s == '"') { look_for = '"'; s++; }
348
+ char *end = strchr(s, look_for);
349
+ if(end) *end = '\0';
350
+ }
351
329
- char buf_cmd[len + 1];
330
- // if it is enclosed in (), remove the parenthesis
331
- if(cl[0] == '(' && cl[len - 1] == ')') {
332
- memcpy(buf_cmd, &cl[1], len - 2);
333
- buf_cmd[len - 2] = '\0';
352
+ // linux, freebsd, macos, msys, cygwin
353
+ if(*s == '/') {
354
+ struct statvfs stat;
355
+ return statvfs(s, &stat) == 0;
356
}
335
- else
336
- memcpy(buf_cmd, cl, sizeof(buf_cmd));
357
338
- size_t comm_len = strlen(comm);
339
- char *start = strstr(buf_cmd, comm);
340
- if(start) {
358
+ return false;
359
+}
360
+
361
+static const char *extensions_to_strip[] = {
362
+ ".sh", // shell scripts
363
+ ".py", // python scripts
364
+ ".pl", // perl scripts
365
+ ".js", // node.js
366
+#if defined(OS_WINDOWS)
367
+ ".exe",
368
+#endif
369
+ NULL,
370
+};
371
+
372
+// strip extensions we don't want to show
373
+static void remove_extension(char *name) {
374
+ size_t name_len = strlen(name);
375
+ for(size_t i = 0; extensions_to_strip[i] != NULL; i++) {
376
+ const char *ext = extensions_to_strip[i];
377
+ size_t ext_len = strlen(ext);
378
+ if(name_len > ext_len) {
379
+ char *check = &name[name_len - ext_len];
380
+ if(strcmp(check, ext) == 0) {
381
+ *check = '\0';
382
+ break;
383
+ }
384
+ }
385
+ }
386
+}
387
+
388
+static inline STRING *comm_from_cmdline_param_sanitized(STRING *cmdline) {
389
+ if(!cmdline) return NULL;
390
+
391
+ char buf[string_strlen(cmdline) + 1];
392
+ memcpy(buf, string2str(cmdline), sizeof(buf));
393
+
394
+ char *words[100];
395
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, 100);
396
+ for(size_t word = 1; word < num_words ;word++) {
397
+ char *s = words[word];
398
+ if(is_filename(s)) {
399
+ char *name = strrchr(s, '/');
400
+
401
+#if defined(OS_WINDOWS)
402
+ if(!name)
403
+ name = strrchr(s, '\\');
404
+#endif
405
+
406
+ if(name && *name) {
407
+ name++;
408
+ remove_extension(name);
409
+ sanitize_apps_plugin_chart_meta(name);
410
+ return string_strdupz(name);
411
+ }
412
+ }
413
+ }
414
+
415
+ return NULL;
416
+}
417
+
418
+static inline STRING *comm_from_cmdline_sanitized(STRING *comm, STRING *cmdline) {
419
+ if(!cmdline) return NULL;
420
+
421
+ char buf[string_strlen(cmdline) + 1];
422
+ memcpy(buf, string2str(cmdline), sizeof(buf));
423
+
424
+ size_t comm_len = string_strlen(comm);
425
+ char *start = strstr(buf, string2str(comm));
426
+ while (start) {
427
char *end = start + comm_len;
342
- while(*end && !isspace((uint8_t)*end) && *end != '/' && *end != '\\' && *end != '"') end++;
428
+ while (*end &&
429
+ !isspace((uint8_t) *end) &&
430
+ *end != '/' && // path separator - linux
431
+ *end != '\\' && // path separator - windows
432
+ *end != '"' && // closing double quotes
433
+ *end != '\'' && // closing single quotes
434
+ *end != ')' && // sometimes process add ) at their end
435
+ *end != ':') // sometimes process add : at their end
436
+ end++;
437
+
438
*end = '\0';
439
345
- sanitize_chart_meta(start);
440
+ remove_extension(start);
441
+ sanitize_apps_plugin_chart_meta(start);
442
return string_strdupz(start);
443
}
444
349
- sanitize_chart_meta(comm);
350
- return string_strdupz(comm);
445
+ return NULL;
446
+}
447
+
448
+static void update_pid_comm_from_cmdline(struct pid_stat *p) {
449
+ bool updated = false;
450
+
451
+ STRING *new_comm = comm_from_cmdline_sanitized(p->comm, p->cmdline);
452
+ if(new_comm) {
453
+ string_freez(p->comm);
454
+ p->comm = new_comm;
455
+ updated = true;
456
+ }
457
+
458
+ if(is_process_an_interpreter(p)) {
459
+ new_comm = comm_from_cmdline_param_sanitized(p->cmdline);
460
+ if(new_comm) {
461
+ string_freez(p->comm);
462
+ p->comm = new_comm;
463
+ updated = true;
464
+ }
465
+ }
466
+
467
+ if(updated) {
468
+ p->is_manager = is_process_a_manager(p);
469
+ p->is_aggregator = is_process_an_aggregator(p);
470
+ }
471
+}
472
+
473
+void update_pid_cmdline(struct pid_stat *p, const char *cmdline) {
474
+ string_freez(p->cmdline);
475
+ p->cmdline = cmdline ? string_strdupz(cmdline) : NULL;
476
+
477
+ if(p->cmdline)
478
+ update_pid_comm_from_cmdline(p);
479
}
480
481
void update_pid_comm(struct pid_stat *p, const char *comm) {
@@ -355,10 +483,8 @@ void update_pid_comm(struct pid_stat *p, const char *comm) {
483
// no change
484
return;
485
358
-#if (PROCESSES_HAVE_CMDLINE == 1)
359
- if(likely(proc_pid_cmdline_is_needed && !p->cmdline))
360
- managed_log(p, PID_LOG_CMDLINE, read_proc_pid_cmdline(p));
361
-#endif
486
+ string_freez(p->comm_orig);
487
+ p->comm_orig = string_strdupz(comm);
488
489
// some process names have ( and ), remove the parenthesis
490
size_t len = strlen(comm);
@@ -370,14 +496,17 @@ void update_pid_comm(struct pid_stat *p, const char *comm) {
496
else
497
memcpy(buf, comm, sizeof(buf));
498
373
- string_freez(p->comm_orig);
374
- p->comm_orig = string_strdupz(comm);
499
+ sanitize_apps_plugin_chart_meta(buf);
500
+ p->comm = string_strdupz(buf);
501
+ p->is_manager = is_process_a_manager(p);
502
+ p->is_aggregator = is_process_an_aggregator(p);
503
376
- string_freez(p->comm);
377
- p->comm = comm_from_cmdline_sanitized(buf, p->cmdline);
378
-
379
- p->is_manager = is_process_manager(p);
380
- p->is_aggregator = is_process_aggregator(p);
504
+#if (PROCESSES_HAVE_CMDLINE == 1)
505
+ if(likely(proc_pid_cmdline_is_needed && !p->cmdline))
506
+ managed_log(p, PID_LOG_CMDLINE, read_proc_pid_cmdline(p));
507
+#else
508
+ update_pid_comm_from_cmdline(p);
509
+#endif
510
511
// the process changed comm, we may have to reassign it to
512
// an apps_groups.conf target.
src/collectors/apps.plugin/apps_pid_match.c
new
+90
@@ -0,0 +1,90 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#include "apps_plugin.h"
4
+
5
+bool pid_match_check(struct pid_stat *p, APPS_MATCH *match) {
6
+ if(!match->starts_with && !match->ends_with) {
7
+ if(match->pattern) {
8
+ if(simple_pattern_matches_string(match->pattern, p->comm))
9
+ return true;
10
+ }
11
+ else {
12
+ if(match->compare == p->comm || match->compare == p->comm_orig)
13
+ return true;
14
+ }
15
+ }
16
+ else if(match->starts_with && !match->ends_with) {
17
+ if(match->pattern) {
18
+ if(simple_pattern_matches_string(match->pattern, p->comm))
19
+ return true;
20
+ }
21
+ else {
22
+ if(string_starts_with_string(p->comm, match->compare) ||
23
+ (p->comm != p->comm_orig && string_starts_with_string(p->comm, match->compare)))
24
+ return true;
25
+ }
26
+ }
27
+ else if(!match->starts_with && match->ends_with) {
28
+ if(match->pattern) {
29
+ if(simple_pattern_matches_string(match->pattern, p->comm))
30
+ return true;
31
+ }
32
+ else {
33
+ if(string_ends_with_string(p->comm, match->compare) ||
34
+ (p->comm != p->comm_orig && string_ends_with_string(p->comm, match->compare)))
35
+ return true;
36
+ }
37
+ }
38
+ else if(match->starts_with && match->ends_with && p->cmdline) {
39
+ if(match->pattern) {
40
+ if(simple_pattern_matches_string(match->pattern, p->cmdline))
41
+ return true;
42
+ }
43
+ else {
44
+ if(strstr(string2str(p->cmdline), string2str(match->compare)))
45
+ return true;
46
+ }
47
+ }
48
+
49
+ return false;
50
+}
51
+
52
+APPS_MATCH pid_match_create(const char *comm) {
53
+ APPS_MATCH m = {
54
+ .starts_with = false,
55
+ .ends_with = false,
56
+ .compare = NULL,
57
+ .pattern = NULL,
58
+ };
59
+
60
+ // copy comm to make changes to it
61
+ size_t len = strlen(comm);
62
+ char buf[len + 1];
63
+ memcpy(buf, comm, sizeof(buf));
64
+
65
+ trim_all(buf);
66
+
67
+ if(buf[len - 1] == '*') {
68
+ buf[--len] = '\0';
69
+ m.starts_with = true;
70
+ }
71
+
72
+ const char *nid = buf;
73
+ if (nid[0] == '*') {
74
+ m.ends_with = true;
75
+ nid++;
76
+ }
77
+
78
+ m.compare = string_strdupz(nid);
79
+
80
+ if(strchr(nid, '*'))
81
+ m.pattern = simple_pattern_create(comm, SIMPLE_PATTERN_NO_SEPARATORS, SIMPLE_PATTERN_EXACT, true);
82
+
83
+ return m;
84
+}
85
+
86
+void pid_match_cleanup(APPS_MATCH *m) {
87
+ string_freez(m->compare);
88
+ simple_pattern_free(m->pattern);
89
+}
90
+
src/collectors/apps.plugin/apps_plugin.c
+4
@@ -118,6 +118,10 @@ static char *stock_config_dir = LIBCONFIG_DIR;
118
119
size_t pagesize;
120
121
+void sanitize_apps_plugin_chart_meta(char *buf) {
122
+ external_plugins_sanitize(buf, buf, strlen(buf) + 1);
123
+}
124
+
125
// ----------------------------------------------------------------------------
126
// update chart dimensions
127
src/collectors/apps.plugin/apps_plugin.h
+19
-12
@@ -364,24 +364,26 @@ typedef enum __attribute__((packed)) {
364
PDF_MAX
365
} PID_FIELD;
366
367
+typedef struct apps_match {
368
+ bool starts_with:1;
369
+ bool ends_with:1;
370
+ STRING *compare;
371
+ SIMPLE_PATTERN *pattern;
372
+} APPS_MATCH;
373
+
374
struct target {
375
STRING *id;
376
STRING *name;
377
STRING *clean_name;
378
379
TARGET_TYPE type;
373
- union {
374
- struct {
375
- SIMPLE_PATTERN *pattern;
376
- STRING *compare;
377
- } ag;
380
+ APPS_MATCH match;
381
#if (PROCESSES_HAVE_UID == 1)
379
- uid_t uid;
382
+ uid_t uid;
383
#endif
384
#if (PROCESSES_HAVE_GID == 1)
382
- gid_t gid;
385
+ gid_t gid;
386
#endif
384
- };
387
388
kernel_uint_t values[PDF_MAX];
389
@@ -396,8 +398,6 @@ struct target {
398
#endif
399
400
bool exposed:1; // if set, we have sent this to netdata
399
- bool ends_with:1; // if set, the compare string matches the end of the command
400
- bool starts_with:1; // if set, the compare string matches the start of the command
401
402
struct pid_on_target *root_pid; // list of aggregated pids for target debugging
403
@@ -615,6 +615,7 @@ static inline void debug_log_dummy(void) {}
615
616
#endif
617
bool managed_log(struct pid_stat *p, PID_LOG log, bool status);
618
+void sanitize_apps_plugin_chart_meta(char *buf);
619
620
// ----------------------------------------------------------------------------
621
// macro to calculate the incremental rate of a value
@@ -655,6 +656,10 @@ uint32_t file_descriptor_find_or_add(const char *name, uint32_t hash);
656
// --------------------------------------------------------------------------------------------------------------------
657
// data collection management
658
659
+bool pid_match_check(struct pid_stat *p, APPS_MATCH *match);
660
+APPS_MATCH pid_match_create(const char *comm);
661
+void pid_match_cleanup(APPS_MATCH *m);
662
+
663
bool collect_data_for_all_pids(void);
664
665
void pid_collection_started(struct pid_stat *p);
@@ -677,9 +682,11 @@ struct pid_stat *get_or_allocate_pid_entry(pid_t pid);
682
struct pid_stat *find_pid_entry(pid_t pid);
683
void del_pid_entry(pid_t pid);
684
void update_pid_comm(struct pid_stat *p, const char *comm);
685
+void update_pid_cmdline(struct pid_stat *p, const char *cmdline);
686
681
-bool is_process_manager(struct pid_stat *p);
682
-bool is_process_aggregator(struct pid_stat *p);
687
+bool is_process_a_manager(struct pid_stat *p);
688
+bool is_process_an_aggregator(struct pid_stat *p);
689
+bool is_process_an_interpreter(struct pid_stat *p);
690
691
// --------------------------------------------------------------------------------------------------------------------
692
// targets management
src/collectors/apps.plugin/apps_targets.c
+69
-72
@@ -8,9 +8,10 @@ static STRING *get_clean_name(STRING *name) {
8
char buf[string_strlen(name) + 1];
9
memcpy(buf, string2str(name), string_strlen(name) + 1);
10
netdata_fix_chart_name(buf);
11
- for (char *d = buf; *d ; d++) {
11
+
12
+ for (char *d = buf; *d ; d++)
13
if (*d == '.') *d = '_';
13
- }
14
+
15
return string_strdupz(buf);
16
}
17
@@ -34,7 +35,7 @@ struct target *find_target_by_name(struct target *base, const char *name) {
35
// Process managers and aggregators
36
37
struct comm_list {
37
- STRING *comm;
38
+ APPS_MATCH match;
39
};
40
41
struct managed_list {
@@ -46,6 +47,7 @@ struct managed_list {
47
static struct {
48
struct managed_list managers;
49
struct managed_list aggregators;
50
+ struct managed_list interpreters;
51
} tree = {
52
.managers = {
53
.array = NULL,
@@ -61,7 +63,7 @@ static struct {
63
64
static void managed_list_clear(struct managed_list *list) {
65
for(size_t c = 0; c < list->used ; c++)
64
- string_freez(list->array[c].comm);
66
+ pid_match_cleanup(&list->array[c].match);
67
68
freez(list->array);
69
list->array = NULL;
@@ -72,13 +74,13 @@ static void managed_list_clear(struct managed_list *list) {
74
static void managed_list_add(struct managed_list *list, const char *s) {
75
if(list->used >= list->size) {
76
if(!list->size)
75
- list->size = 10;
77
+ list->size = 16;
78
else
79
list->size *= 2;
80
list->array = reallocz(list->array, sizeof(*list->array) * list->size);
81
}
82
81
- list->array[list->used++].comm = string_strdupz(s);
83
+ list->array[list->used++].match = pid_match_create(s);
84
}
85
86
static STRING *KernelAggregator = NULL;
@@ -109,6 +111,12 @@ void apps_managers_and_aggregators_init(void) {
111
managed_list_add(&tree.managers, "launchd");
112
#endif
113
114
+#if defined(OS_WINDOWS)
115
+ managed_list_add(&tree.managers, "netdata");
116
+#else
117
+ managed_list_add(&tree.managers, "spawn-plugins");
118
+#endif
119
+
120
managed_list_clear(&tree.aggregators);
121
#if defined(OS_LINUX)
122
managed_list_add(&tree.aggregators, "kthread");
@@ -117,22 +125,38 @@ void apps_managers_and_aggregators_init(void) {
125
managed_list_add(&tree.aggregators, "kernel");
126
#elif defined(OS_MACOS)
127
#endif
128
+
129
+ managed_list_clear(&tree.interpreters);
130
+ managed_list_add(&tree.interpreters, "python");
131
+ managed_list_add(&tree.interpreters, "python2");
132
+ managed_list_add(&tree.interpreters, "python3");
133
+ managed_list_add(&tree.interpreters, "sh");
134
+ managed_list_add(&tree.interpreters, "bash");
135
+ managed_list_add(&tree.interpreters, "node");
136
+ managed_list_add(&tree.interpreters, "perl");
137
}
138
122
-bool is_process_manager(struct pid_stat *p) {
139
+bool is_process_a_manager(struct pid_stat *p) {
140
for(size_t c = 0; c < tree.managers.used ; c++) {
124
- if(p->comm == tree.managers.array[c].comm ||
125
- p->comm_orig == tree.managers.array[c].comm)
141
+ if(pid_match_check(p, &tree.managers.array[c].match))
142
return true;
143
}
144
145
return false;
146
}
147
132
-bool is_process_aggregator(struct pid_stat *p) {
148
+bool is_process_an_aggregator(struct pid_stat *p) {
149
for(size_t c = 0; c < tree.aggregators.used ; c++) {
134
- if(p->comm == tree.aggregators.array[c].comm ||
135
- p->comm_orig == tree.aggregators.array[c].comm)
150
+ if(pid_match_check(p, &tree.aggregators.array[c].match))
151
+ return true;
152
+ }
153
+
154
+ return false;
155
+}
156
+
157
+bool is_process_an_interpreter(struct pid_stat *p) {
158
+ for(size_t c = 0; c < tree.interpreters.used ; c++) {
159
+ if(pid_match_check(p, &tree.interpreters.array[c].match))
160
return true;
161
}
162
@@ -175,8 +199,9 @@ struct target *get_tree_target(struct pid_stat *p) {
199
200
w = callocz(sizeof(struct target), 1);
201
w->type = TARGET_TYPE_TREE;
178
- w->starts_with = w->ends_with = false;
179
- w->ag.compare = string_dup(search_for);
202
+ w->match.starts_with = w->match.ends_with = false;
203
+ w->match.compare = string_dup(search_for);
204
+ w->match.pattern = NULL;
205
w->id = search_for;
206
w->name = string_dup(search_for);
207
w->clean_name = get_clean_name(w->name);
@@ -283,42 +308,14 @@ struct target *apps_groups_root_target = NULL;
308
// find or create a new target
309
// there are targets that are just aggregated to other target (the second argument)
310
static struct target *get_apps_groups_target(const char *comm, struct target *target, const char *name) {
286
- bool ends_with = false, starts_with = false, has_asterisk_inside = false;
287
-
288
- STRING *comm_lookup = NULL;
289
- STRING *name_lookup = NULL;
290
-
291
- // extract the options from the id
292
- {
293
- size_t len = strlen(comm);
294
- char buf[len + 1];
295
- memcpy(buf, comm, sizeof(buf));
296
-
297
- if(buf[len - 1] == '*') {
298
- buf[--len] = '\0';
299
- starts_with = true;
300
- }
301
-
302
- const char *nid = buf;
303
- if (nid[0] == '*') {
304
- ends_with = true;
305
- nid++;
306
- }
307
-
308
- if(strchr(nid, '*'))
309
- has_asterisk_inside = true;
310
-
311
- comm_lookup = string_strdupz(nid);
312
- }
313
-
314
- // extract the options from the name
315
- name_lookup = string_strdupz(name);
311
+ APPS_MATCH match = pid_match_create(comm);
312
+ STRING *name_lookup = string_strdupz(name);
313
314
// find if it already exists
315
struct target *w, *last = apps_groups_root_target;
316
for(w = apps_groups_root_target ; w ; w = w->next) {
320
- if(w->id == comm_lookup) {
321
- string_freez(comm_lookup);
317
+ if(w->id == match.compare) {
318
+ pid_match_cleanup(&match);
319
string_freez(name_lookup);
320
return w;
321
}
@@ -340,23 +337,18 @@ static struct target *get_apps_groups_target(const char *comm, struct target *ta
337
338
w = callocz(sizeof(struct target), 1);
339
w->type = TARGET_TYPE_APP_GROUP;
343
- w->ag.compare = string_dup(comm_lookup);
344
- w->starts_with = starts_with;
345
- w->ends_with = ends_with;
346
- w->id = string_dup(comm_lookup);
347
-
348
- if(has_asterisk_inside)
349
- w->ag.pattern = simple_pattern_create(comm, " ", SIMPLE_PATTERN_EXACT, true);
340
+ w->match = match;
341
+ w->id = string_dup(w->match.compare);
342
343
if(unlikely(!target))
344
w->name = string_dup(name_lookup); // copy the name
345
else
354
- w->name = string_dup(comm_lookup); // copy the id
346
+ w->name = string_dup(w->id); // copy the id
347
348
// dots are used to distinguish chart type and id in streaming, so we should replace them
349
w->clean_name = get_clean_name(w->name);
350
359
- if(w->starts_with && w->ends_with)
351
+ if(w->match.starts_with && w->match.ends_with)
352
proc_pid_cmdline_is_needed = true;
353
354
w->target = target;
@@ -367,12 +359,11 @@ static struct target *get_apps_groups_target(const char *comm, struct target *ta
359
360
debug_log("ADDING TARGET ID '%s', process name '%s' (%s), aggregated on target '%s'"
361
, string2str(w->id)
370
- , string2str(w->ag.compare)
371
- , (w->starts_with && w->ends_with)?"substring":((w->starts_with)?"prefix":((w->ends_with)?"suffix":"exact"))
362
+ , string2str(w->match.compare)
363
+ , (w->match.starts_with && w->match.ends_with) ? "substring" : ((w->match.starts_with) ? "prefix" : ((w->match.ends_with) ? "suffix" : "exact"))
364
, w->target?w->target->name:w->name
365
);
366
375
- string_freez(comm_lookup);
367
string_freez(name_lookup);
368
369
return w;
@@ -399,29 +390,22 @@ int read_apps_groups_conf(const char *path, const char *file) {
390
391
size_t line, lines = procfile_lines(ff);
392
402
- bool managers_reset_done = false;
403
-
393
for(line = 0; line < lines ;line++) {
394
size_t word, words = procfile_linewords(ff, line);
395
if(!words) continue;
396
397
char *name = procfile_lineword(ff, line, 0);
409
- if(!name || !*name) continue;
398
+ if(!name || !*name || *name == '#') continue;
399
400
if(strcmp(name, "managers") == 0) {
412
- if(!managers_reset_done) {
413
- managers_reset_done = true;
401
+ if(words == 2 && strcmp(procfile_lineword(ff, line, 1), "clear") == 0)
402
managed_list_clear(&tree.managers);
415
- }
403
417
- for(word = 0; word < words ;word++) {
404
+ for(word = 1; word < words ;word++) {
405
char *s = procfile_lineword(ff, line, word);
406
if (!s || !*s) continue;
407
if (*s == '#') break;
408
422
- // is this the first word? skip it
423
- if(s == name) continue;
424
-
409
managed_list_add(&tree.managers, s);
410
}
411
@@ -429,18 +413,31 @@ int read_apps_groups_conf(const char *path, const char *file) {
413
continue;
414
}
415
416
+ if(strcmp(name, "interpreters") == 0) {
417
+ if(words == 2 && strcmp(procfile_lineword(ff, line, 1), "clear") == 0)
418
+ managed_list_clear(&tree.interpreters);
419
+
420
+ for(word = 1; word < words ;word++) {
421
+ char *s = procfile_lineword(ff, line, word);
422
+ if (!s || !*s) continue;
423
+ if (*s == '#') break;
424
+
425
+ managed_list_add(&tree.interpreters, s);
426
+ }
427
+
428
+ // done with managers, proceed to next line
429
+ continue;
430
+ }
431
+
432
// find a possibly existing target
433
struct target *w = NULL;
434
435
// loop through all words, skipping the first one (the name)
436
- for(word = 0; word < words ;word++) {
436
+ for(word = 1; word < words ;word++) {
437
char *s = procfile_lineword(ff, line, word);
438
if(!s || !*s) continue;
439
if(*s == '#') break;
440
441
- // is this the first word? skip it
442
- if(s == name) continue;
443
-
441
// add this target
442
struct target *n = get_apps_groups_target(s, w, name);
443
if(!n) {
src/collectors/cgroups.plugin/cgroup-network.c
+32
-3
@@ -146,10 +146,11 @@ static void continue_as_child(void) {
146
if (child < 0)
147
collector_error("fork() failed");
148
149
- /* Only the child returns */
149
if (child == 0)
150
+ // the child returns
151
return;
152
153
+ // here is the parent
154
for (;;) {
155
ret = waitpid(child, &status, WUNTRACED);
156
if ((ret == child) && (WIFSTOPPED(status))) {
@@ -159,9 +160,36 @@ static void continue_as_child(void) {
160
} else {
161
break;
162
}
163
+
164
+ tinysleep();
165
}
166
167
/* Return the child's exit code if possible */
168
+
169
+#ifdef __SANITIZE_ADDRESS__
170
+ /*
171
+ * With sanitization, exiting leads to an infinite loop (100% cpu) here:
172
+ *
173
+ * #0 0x00007ffff690ea8b in sched_yield () from /usr/lib/libc.so.6
174
+ * #1 0x00007ffff792c4a6 in __sanitizer::StopTheWorld (callback=<optimized out>, argument=<optimized out>) at /usr/src/debug/gcc/gcc/libsanitizer/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cpp:457
175
+ * #2 0x00007ffff793f6f9 in __lsan::LockStuffAndStopTheWorldCallback (info=<optimized out>, size=<optimized out>, data=0x7fffffffde20) at /usr/src/debug/gcc/gcc/libsanitizer/lsan/lsan_common_linux.cpp:127
176
+ * #3 0x00007ffff6977909 in dl_iterate_phdr () from /usr/lib/libc.so.6
177
+ * #4 0x00007ffff793fb24 in __lsan::LockStuffAndStopTheWorld (callback=callback@entry=0x7ffff793d9d0 <__lsan::CheckForLeaksCallback(__sanitizer::SuspendedThreadsList const&, void*)>, argument=argument@entry=0x7fffffffdea0)
178
+ * at /usr/src/debug/gcc/gcc/libsanitizer/lsan/lsan_common_linux.cpp:142
179
+ * #5 0x00007ffff793c965 in __lsan::CheckForLeaks () at /usr/src/debug/gcc/gcc/libsanitizer/lsan/lsan_common.cpp:778
180
+ * #6 0x00007ffff793cc68 in __lsan::DoLeakCheck () at /usr/src/debug/gcc/gcc/libsanitizer/lsan/lsan_common.cpp:821
181
+ * #7 0x00007ffff684e340 in __cxa_finalize () from /usr/lib/libc.so.6
182
+ * #8 0x00007ffff7838c58 in __do_global_dtors_aux () from /usr/lib/libasan.so.8
183
+ * #9 0x00007fffffffdfe0 in ?? ()
184
+ *
185
+ * Probably is something related to switching name spaces.
186
+ * So, we kill -9 self.
187
+ *
188
+ */
189
+
190
+ kill(getpid(), SIGKILL);
191
+#endif
192
+
193
if (WIFEXITED(status)) {
194
exit(WEXITSTATUS(status));
195
} else if (WIFSIGNALED(status)) {
@@ -643,11 +671,12 @@ void usage(void) {
671
exit(1);
672
}
673
646
-int main(int argc, char **argv) {
674
+int main(int argc, const char **argv) {
675
pid_t pid = 0;
676
677
clocks_init();
678
nd_log_initialize_for_external_plugins("cgroup-network");
679
+ netdata_main_spawn_server_init(NULL, argc, argv);
680
681
// since cgroup-network runs as root, prevent it from opening symbolic links
682
procfile_open_flags = O_RDONLY|O_NOFOLLOW;
@@ -707,7 +736,7 @@ int main(int argc, char **argv) {
736
if(helper) call_the_helper(pid, NULL);
737
}
738
else if(!strcmp(argv[arg], "--cgroup")) {
710
- char *cgroup = argv[arg+1];
739
+ const char *cgroup = argv[arg+1];
740
if(verify_path(cgroup) == -1) {
741
collector_error("cgroup '%s' does not exist or is not valid.", cgroup);
742
return 1;
src/collectors/ebpf.plugin/ebpf_functions.c
+1
-1
@@ -287,7 +287,7 @@ static void ebpf_function_socket_manipulation(const char *transaction,
287
ebpf_module_t *em = &ebpf_modules[EBPF_MODULE_SOCKET_IDX];
288
289
char *words[PLUGINSD_MAX_WORDS] = {NULL};
290
- size_t num_words = quoted_strings_splitter_pluginsd(function, words, PLUGINSD_MAX_WORDS);
290
+ size_t num_words = quoted_strings_splitter_whitespace(function, words, PLUGINSD_MAX_WORDS);
291
const char *name;
292
int period = -1;
293
rw_spinlock_write_lock(&ebpf_judy_pid.index.rw_spinlock);
src/collectors/freeipmi.plugin/freeipmi_plugin.c
+1
-1
@@ -1488,7 +1488,7 @@ static void freeimi_function_sensors(const char *transaction, char *function __m
1488
char function_copy[strlen(function) + 1];
1489
memcpy(function_copy, function, sizeof(function_copy));
1490
char *words[1024];
1491
- size_t num_words = quoted_strings_splitter_pluginsd(function_copy, words, 1024);
1491
+ size_t num_words = quoted_strings_splitter_whitespace(function_copy, words, 1024);
1492
for(size_t i = 1; i < num_words ;i++) {
1493
char *param = get_word(words, num_words, i);
1494
if(strcmp(param, "info") == 0) {
src/collectors/network-viewer.plugin/network-viewer.c
+1
-1
@@ -464,7 +464,7 @@ void network_viewer_function(const char *transaction, char *function __maybe_unu
464
char function_copy[strlen(function) + 1];
465
memcpy(function_copy, function, sizeof(function_copy));
466
char *words[1024];
467
- size_t num_words = quoted_strings_splitter_pluginsd(function_copy, words, 1024);
467
+ size_t num_words = quoted_strings_splitter_whitespace(function_copy, words, 1024);
468
for(size_t i = 1; i < num_words ;i++) {
469
char *param = get_word(words, num_words, i);
470
if(strcmp(param, "sockets:aggregated") == 0) {
src/collectors/systemd-journal.plugin/systemd-units.c
+1
-1
@@ -1607,7 +1607,7 @@ void function_systemd_units(const char *transaction, char *function,
1607
BUFFER *payload __maybe_unused, HTTP_ACCESS access __maybe_unused,
1608
const char *source __maybe_unused, void *data __maybe_unused) {
1609
char *words[SYSTEMD_UNITS_MAX_PARAMS] = { NULL };
1610
- size_t num_words = quoted_strings_splitter_pluginsd(function, words, SYSTEMD_UNITS_MAX_PARAMS);
1610
+ size_t num_words = quoted_strings_splitter_whitespace(function, words, SYSTEMD_UNITS_MAX_PARAMS);
1611
for(int i = 1; i < SYSTEMD_UNITS_MAX_PARAMS ;i++) {
1612
char *keyword = get_word(words, num_words, i);
1613
if(!keyword) break;
src/daemon/config/dyncfg-intercept.c
+1
-1
@@ -216,7 +216,7 @@ int dyncfg_function_intercept_cb(struct rrd_function_execute *rfe, void *data __
216
memcpy(buf, rfe->function, sizeof(buf));
217
218
char *words[20];
219
- size_t num_words = quoted_strings_splitter_pluginsd(buf, words, 20);
219
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, 20);
220
221
size_t i = 0;
222
char *config = get_word(words, num_words, i++);
src/daemon/config/dyncfg-tree.c
+1
-1
@@ -160,7 +160,7 @@ static int dyncfg_config_execute_cb(struct rrd_function_execute *rfe, void *data
160
memcpy(buf, rfe->function, sizeof(buf));
161
162
char *words[MAX_FUNCTION_PARAMETERS]; // an array of pointers for the words in this line
163
- size_t num_words = quoted_strings_splitter_pluginsd(buf, words, MAX_FUNCTION_PARAMETERS);
163
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, MAX_FUNCTION_PARAMETERS);
164
165
const char *config = get_word(words, num_words, 0);
166
const char *action = get_word(words, num_words, 1);
src/daemon/config/dyncfg-unittest.c
+2
-2
@@ -195,7 +195,7 @@ static int dyncfg_unittest_execute_cb(struct rrd_function_execute *rfe, void *da
195
memcpy(buf, rfe->function, sizeof(buf));
196
197
char *words[MAX_FUNCTION_PARAMETERS]; // an array of pointers for the words in this line
198
- size_t num_words = quoted_strings_splitter_pluginsd(buf, words, MAX_FUNCTION_PARAMETERS);
198
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, MAX_FUNCTION_PARAMETERS);
199
200
const char *config = get_word(words, num_words, 0);
201
const char *id = get_word(words, num_words, 1);
@@ -426,7 +426,7 @@ static int dyncfg_unittest_run(const char *cmd, BUFFER *wb, const char *payload,
426
memcpy(buf, cmd, sizeof(buf));
427
428
char *words[MAX_FUNCTION_PARAMETERS]; // an array of pointers for the words in this line
429
- size_t num_words = quoted_strings_splitter_pluginsd(buf, words, MAX_FUNCTION_PARAMETERS);
429
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, MAX_FUNCTION_PARAMETERS);
430
431
// const char *config = get_word(words, num_words, 0);
432
const char *id = get_word(words, num_words, 1);
src/database/rrd.h
+3
-3
@@ -270,7 +270,7 @@ void rrdr_fill_tier_gap_from_smaller_tiers(RRDDIM *rd, size_t tier, time_t now_s
270
// RRD DIMENSION - this is a metric
271
272
struct rrddim {
273
- nd_uuid_t metric_uuid; // global UUID for this metric (unique_across hosts)
273
+ nd_uuid_t metric_uuid; // global UUID for this metric (unique_across hosts)
274
275
// ------------------------------------------------------------------------
276
// dimension definition
@@ -289,10 +289,10 @@ struct rrddim {
289
// operational state members
290
291
struct rrdset *rrdset;
292
- rrd_ml_dimension_t *ml_dimension; // machine learning data about this dimension
292
+ rrd_ml_dimension_t *ml_dimension; // machine learning data about this dimension
293
294
struct {
295
- RRDMETRIC_ACQUIRED *rrdmetric; // the rrdmetric of this dimension
295
+ RRDMETRIC_ACQUIRED *rrdmetric; // the rrdmetric of this dimension
296
bool collected;
297
} rrdcontexts;
298
src/database/rrdfunctions-internals.h
-1
@@ -31,7 +31,6 @@ struct rrd_host_function {
31
struct rrd_collector *collector;
32
};
33
34
-size_t rrd_functions_sanitize(char *dst, const char *src, size_t dst_len);
34
int rrd_functions_find_by_name(RRDHOST *host, BUFFER *wb, const char *name, size_t key_length, const DICTIONARY_ITEM **item);
35
36
#endif //NETDATA_RRDFUNCTIONS_INTERNALS_H
src/database/rrdfunctions.c
-64
@@ -7,70 +7,6 @@
7
8
#define MAX_FUNCTION_LENGTH (PLUGINSD_LINE_MAX - 512) // we need some space for the rest of the line
9
10
-static unsigned char functions_allowed_chars[256] = {
11
- [0] = '\0', [1] = '_', [2] = '_', [3] = '_', [4] = '_', [5] = '_', [6] = '_', [7] = '_', [8] = '_',
12
-
13
- // control
14
- ['\t'] = ' ', ['\n'] = ' ', ['\v'] = ' ', [12] = ' ', ['\r'] = ' ',
15
-
16
- [14] = '_', [15] = '_', [16] = '_', [17] = '_', [18] = '_', [19] = '_', [20] = '_', [21] = '_',
17
- [22] = '_', [23] = '_', [24] = '_', [25] = '_', [26] = '_', [27] = '_', [28] = '_', [29] = '_',
18
- [30] = '_', [31] = '_',
19
-
20
- // symbols
21
- [' '] = ' ', ['!'] = '!', ['"'] = '\'', ['#'] = '#', ['$'] = '$', ['%'] = '%', ['&'] = '&', ['\''] = '\'',
22
- ['('] = '(', [')'] = ')', ['*'] = '*', ['+'] = '+', [','] = ',', ['-'] = '-', ['.'] = '.', ['/'] = '/',
23
-
24
- // numbers
25
- ['0'] = '0', ['1'] = '1', ['2'] = '2', ['3'] = '3', ['4'] = '4', ['5'] = '5', ['6'] = '6', ['7'] = '7',
26
- ['8'] = '8', ['9'] = '9',
27
-
28
- // symbols
29
- [':'] = ':', [';'] = ';', ['<'] = '<', ['='] = '=', ['>'] = '>', ['?'] = '?', ['@'] = '@',
30
-
31
- // capitals
32
- ['A'] = 'A', ['B'] = 'B', ['C'] = 'C', ['D'] = 'D', ['E'] = 'E', ['F'] = 'F', ['G'] = 'G', ['H'] = 'H',
33
- ['I'] = 'I', ['J'] = 'J', ['K'] = 'K', ['L'] = 'L', ['M'] = 'M', ['N'] = 'N', ['O'] = 'O', ['P'] = 'P',
34
- ['Q'] = 'Q', ['R'] = 'R', ['S'] = 'S', ['T'] = 'T', ['U'] = 'U', ['V'] = 'V', ['W'] = 'W', ['X'] = 'X',
35
- ['Y'] = 'Y', ['Z'] = 'Z',
36
-
37
- // symbols
38
- ['['] = '[', ['\\'] = '\\', [']'] = ']', ['^'] = '^', ['_'] = '_', ['`'] = '`',
39
-
40
- // lower
41
- ['a'] = 'a', ['b'] = 'b', ['c'] = 'c', ['d'] = 'd', ['e'] = 'e', ['f'] = 'f', ['g'] = 'g', ['h'] = 'h',
42
- ['i'] = 'i', ['j'] = 'j', ['k'] = 'k', ['l'] = 'l', ['m'] = 'm', ['n'] = 'n', ['o'] = 'o', ['p'] = 'p',
43
- ['q'] = 'q', ['r'] = 'r', ['s'] = 's', ['t'] = 't', ['u'] = 'u', ['v'] = 'v', ['w'] = 'w', ['x'] = 'x',
44
- ['y'] = 'y', ['z'] = 'z',
45
-
46
- // symbols
47
- ['{'] = '{', ['|'] = '|', ['}'] = '}', ['~'] = '~',
48
-
49
- // rest
50
- [127] = '_', [128] = '_', [129] = '_', [130] = '_', [131] = '_', [132] = '_', [133] = '_', [134] = '_',
51
- [135] = '_', [136] = '_', [137] = '_', [138] = '_', [139] = '_', [140] = '_', [141] = '_', [142] = '_',
52
- [143] = '_', [144] = '_', [145] = '_', [146] = '_', [147] = '_', [148] = '_', [149] = '_', [150] = '_',
53
- [151] = '_', [152] = '_', [153] = '_', [154] = '_', [155] = '_', [156] = '_', [157] = '_', [158] = '_',
54
- [159] = '_', [160] = '_', [161] = '_', [162] = '_', [163] = '_', [164] = '_', [165] = '_', [166] = '_',
55
- [167] = '_', [168] = '_', [169] = '_', [170] = '_', [171] = '_', [172] = '_', [173] = '_', [174] = '_',
56
- [175] = '_', [176] = '_', [177] = '_', [178] = '_', [179] = '_', [180] = '_', [181] = '_', [182] = '_',
57
- [183] = '_', [184] = '_', [185] = '_', [186] = '_', [187] = '_', [188] = '_', [189] = '_', [190] = '_',
58
- [191] = '_', [192] = '_', [193] = '_', [194] = '_', [195] = '_', [196] = '_', [197] = '_', [198] = '_',
59
- [199] = '_', [200] = '_', [201] = '_', [202] = '_', [203] = '_', [204] = '_', [205] = '_', [206] = '_',
60
- [207] = '_', [208] = '_', [209] = '_', [210] = '_', [211] = '_', [212] = '_', [213] = '_', [214] = '_',
61
- [215] = '_', [216] = '_', [217] = '_', [218] = '_', [219] = '_', [220] = '_', [221] = '_', [222] = '_',
62
- [223] = '_', [224] = '_', [225] = '_', [226] = '_', [227] = '_', [228] = '_', [229] = '_', [230] = '_',
63
- [231] = '_', [232] = '_', [233] = '_', [234] = '_', [235] = '_', [236] = '_', [237] = '_', [238] = '_',
64
- [239] = '_', [240] = '_', [241] = '_', [242] = '_', [243] = '_', [244] = '_', [245] = '_', [246] = '_',
65
- [247] = '_', [248] = '_', [249] = '_', [250] = '_', [251] = '_', [252] = '_', [253] = '_', [254] = '_',
66
- [255] = '_'
67
-};
68
-
69
-size_t rrd_functions_sanitize(char *dst, const char *src, size_t dst_len) {
70
- return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_len,
71
- functions_allowed_chars, true, "", NULL);
72
-}
73
-
10
// ----------------------------------------------------------------------------
11
12
// we keep a dictionary per RRDSET with these functions
src/database/rrdlabels.c
-449
@@ -88,464 +88,15 @@ static inline void STATS_MINUS_MEMORY(struct dictionary_stats *stats, size_t key
88
__atomic_fetch_sub(&stats->memory.values, (long)value_size, __ATOMIC_RELAXED);
89
}
90
91
-// ----------------------------------------------------------------------------
92
-// labels sanitization
93
-
94
-/*
95
- * All labels follow these rules:
96
- *
97
- * Character Symbol Values Names
98
- * UTF-8 characters UTF-8 yes -> _
99
- * Lower case letter [a-z] yes yes
100
- * Upper case letter [A-Z] yes -> [a-z]
101
- * Digit [0-9] yes yes
102
- * Underscore _ yes yes
103
- * Minus - yes yes
104
- * Plus + yes -> _
105
- * Colon : yes -> _
106
- * Semicolon ; -> : -> _
107
- * Equal = -> : -> _
108
- * Period . yes yes
109
- * Comma , -> . -> .
110
- * Slash / yes yes
111
- * Backslash \ -> / -> /
112
- * At @ yes -> _
113
- * Space yes -> _
114
- * Opening parenthesis ( yes -> _
115
- * Closing parenthesis ) yes -> _
116
- * anything else -> _ -> _
117
-*
118
- * The above rules should allow users to set in tags (indicative):
119
- *
120
- * 1. hostnames and domain names as-is
121
- * 2. email addresses as-is
122
- * 3. floating point numbers, converted to always use a dot as the decimal point
123
- *
124
- * Leading and trailing spaces and control characters are removed from both label
125
- * names and values.
126
- *
127
- * Multiple spaces inside the label name or the value are removed (only 1 is retained).
128
- * In names spaces are also converted to underscores.
129
- *
130
- * Names that are only underscores are rejected (they do not enter the dictionary).
131
- *
132
- * The above rules do not require any conversion to be included in JSON strings.
133
- *
134
- * Label names and values are truncated to LABELS_MAX_LENGTH (200) characters.
135
- *
136
- * When parsing, label key and value are separated by the first colon (:) found.
137
- * So label:value1:value2 is parsed as key = "label", value = "value1:value2"
138
- *
139
- * This means a label key cannot contain a colon (:) - it is converted to
140
- * underscore if it does.
141
- *
142
- */
143
-
91
#define RRDLABELS_MAX_NAME_LENGTH 200
92
#define RRDLABELS_MAX_VALUE_LENGTH 800 // 800 in bytes, up to 200 UTF-8 characters
93
147
-static unsigned char label_spaces_char_map[256];
148
-static unsigned char label_names_char_map[256];
149
-static unsigned char label_values_char_map[256] = {
150
- [0] = '\0', //
151
- [1] = '_', //
152
- [2] = '_', //
153
- [3] = '_', //
154
- [4] = '_', //
155
- [5] = '_', //
156
- [6] = '_', //
157
- [7] = '_', //
158
- [8] = '_', //
159
- [9] = '_', //
160
- [10] = '_', //
161
- [11] = '_', //
162
- [12] = '_', //
163
- [13] = '_', //
164
- [14] = '_', //
165
- [15] = '_', //
166
- [16] = '_', //
167
- [17] = '_', //
168
- [18] = '_', //
169
- [19] = '_', //
170
- [20] = '_', //
171
- [21] = '_', //
172
- [22] = '_', //
173
- [23] = '_', //
174
- [24] = '_', //
175
- [25] = '_', //
176
- [26] = '_', //
177
- [27] = '_', //
178
- [28] = '_', //
179
- [29] = '_', //
180
- [30] = '_', //
181
- [31] = '_', //
182
- [32] = ' ', // SPACE keep
183
- [33] = '_', // !
184
- [34] = '_', // "
185
- [35] = '_', // #
186
- [36] = '_', // $
187
- [37] = '_', // %
188
- [38] = '_', // &
189
- [39] = '_', // '
190
- [40] = '(', // ( keep
191
- [41] = ')', // ) keep
192
- [42] = '_', // *
193
- [43] = '+', // + keep
194
- [44] = '.', // , convert , to .
195
- [45] = '-', // - keep
196
- [46] = '.', // . keep
197
- [47] = '/', // / keep
198
- [48] = '0', // 0 keep
199
- [49] = '1', // 1 keep
200
- [50] = '2', // 2 keep
201
- [51] = '3', // 3 keep
202
- [52] = '4', // 4 keep
203
- [53] = '5', // 5 keep
204
- [54] = '6', // 6 keep
205
- [55] = '7', // 7 keep
206
- [56] = '8', // 8 keep
207
- [57] = '9', // 9 keep
208
- [58] = ':', // : keep
209
- [59] = ':', // ; convert ; to :
210
- [60] = '_', // <
211
- [61] = ':', // = convert = to :
212
- [62] = '_', // >
213
- [63] = '_', // ?
214
- [64] = '@', // @
215
- [65] = 'A', // A keep
216
- [66] = 'B', // B keep
217
- [67] = 'C', // C keep
218
- [68] = 'D', // D keep
219
- [69] = 'E', // E keep
220
- [70] = 'F', // F keep
221
- [71] = 'G', // G keep
222
- [72] = 'H', // H keep
223
- [73] = 'I', // I keep
224
- [74] = 'J', // J keep
225
- [75] = 'K', // K keep
226
- [76] = 'L', // L keep
227
- [77] = 'M', // M keep
228
- [78] = 'N', // N keep
229
- [79] = 'O', // O keep
230
- [80] = 'P', // P keep
231
- [81] = 'Q', // Q keep
232
- [82] = 'R', // R keep
233
- [83] = 'S', // S keep
234
- [84] = 'T', // T keep
235
- [85] = 'U', // U keep
236
- [86] = 'V', // V keep
237
- [87] = 'W', // W keep
238
- [88] = 'X', // X keep
239
- [89] = 'Y', // Y keep
240
- [90] = 'Z', // Z keep
241
- [91] = '[', // [ keep
242
- [92] = '/', // backslash convert \ to /
243
- [93] = ']', // ] keep
244
- [94] = '_', // ^
245
- [95] = '_', // _ keep
246
- [96] = '_', // `
247
- [97] = 'a', // a keep
248
- [98] = 'b', // b keep
249
- [99] = 'c', // c keep
250
- [100] = 'd', // d keep
251
- [101] = 'e', // e keep
252
- [102] = 'f', // f keep
253
- [103] = 'g', // g keep
254
- [104] = 'h', // h keep
255
- [105] = 'i', // i keep
256
- [106] = 'j', // j keep
257
- [107] = 'k', // k keep
258
- [108] = 'l', // l keep
259
- [109] = 'm', // m keep
260
- [110] = 'n', // n keep
261
- [111] = 'o', // o keep
262
- [112] = 'p', // p keep
263
- [113] = 'q', // q keep
264
- [114] = 'r', // r keep
265
- [115] = 's', // s keep
266
- [116] = 't', // t keep
267
- [117] = 'u', // u keep
268
- [118] = 'v', // v keep
269
- [119] = 'w', // w keep
270
- [120] = 'x', // x keep
271
- [121] = 'y', // y keep
272
- [122] = 'z', // z keep
273
- [123] = '_', // {
274
- [124] = '_', // |
275
- [125] = '_', // }
276
- [126] = '_', // ~
277
- [127] = '_', //
278
- [128] = '_', //
279
- [129] = '_', //
280
- [130] = '_', //
281
- [131] = '_', //
282
- [132] = '_', //
283
- [133] = '_', //
284
- [134] = '_', //
285
- [135] = '_', //
286
- [136] = '_', //
287
- [137] = '_', //
288
- [138] = '_', //
289
- [139] = '_', //
290
- [140] = '_', //
291
- [141] = '_', //
292
- [142] = '_', //
293
- [143] = '_', //
294
- [144] = '_', //
295
- [145] = '_', //
296
- [146] = '_', //
297
- [147] = '_', //
298
- [148] = '_', //
299
- [149] = '_', //
300
- [150] = '_', //
301
- [151] = '_', //
302
- [152] = '_', //
303
- [153] = '_', //
304
- [154] = '_', //
305
- [155] = '_', //
306
- [156] = '_', //
307
- [157] = '_', //
308
- [158] = '_', //
309
- [159] = '_', //
310
- [160] = '_', //
311
- [161] = '_', //
312
- [162] = '_', //
313
- [163] = '_', //
314
- [164] = '_', //
315
- [165] = '_', //
316
- [166] = '_', //
317
- [167] = '_', //
318
- [168] = '_', //
319
- [169] = '_', //
320
- [170] = '_', //
321
- [171] = '_', //
322
- [172] = '_', //
323
- [173] = '_', //
324
- [174] = '_', //
325
- [175] = '_', //
326
- [176] = '_', //
327
- [177] = '_', //
328
- [178] = '_', //
329
- [179] = '_', //
330
- [180] = '_', //
331
- [181] = '_', //
332
- [182] = '_', //
333
- [183] = '_', //
334
- [184] = '_', //
335
- [185] = '_', //
336
- [186] = '_', //
337
- [187] = '_', //
338
- [188] = '_', //
339
- [189] = '_', //
340
- [190] = '_', //
341
- [191] = '_', //
342
- [192] = '_', //
343
- [193] = '_', //
344
- [194] = '_', //
345
- [195] = '_', //
346
- [196] = '_', //
347
- [197] = '_', //
348
- [198] = '_', //
349
- [199] = '_', //
350
- [200] = '_', //
351
- [201] = '_', //
352
- [202] = '_', //
353
- [203] = '_', //
354
- [204] = '_', //
355
- [205] = '_', //
356
- [206] = '_', //
357
- [207] = '_', //
358
- [208] = '_', //
359
- [209] = '_', //
360
- [210] = '_', //
361
- [211] = '_', //
362
- [212] = '_', //
363
- [213] = '_', //
364
- [214] = '_', //
365
- [215] = '_', //
366
- [216] = '_', //
367
- [217] = '_', //
368
- [218] = '_', //
369
- [219] = '_', //
370
- [220] = '_', //
371
- [221] = '_', //
372
- [222] = '_', //
373
- [223] = '_', //
374
- [224] = '_', //
375
- [225] = '_', //
376
- [226] = '_', //
377
- [227] = '_', //
378
- [228] = '_', //
379
- [229] = '_', //
380
- [230] = '_', //
381
- [231] = '_', //
382
- [232] = '_', //
383
- [233] = '_', //
384
- [234] = '_', //
385
- [235] = '_', //
386
- [236] = '_', //
387
- [237] = '_', //
388
- [238] = '_', //
389
- [239] = '_', //
390
- [240] = '_', //
391
- [241] = '_', //
392
- [242] = '_', //
393
- [243] = '_', //
394
- [244] = '_', //
395
- [245] = '_', //
396
- [246] = '_', //
397
- [247] = '_', //
398
- [248] = '_', //
399
- [249] = '_', //
400
- [250] = '_', //
401
- [251] = '_', //
402
- [252] = '_', //
403
- [253] = '_', //
404
- [254] = '_', //
405
- [255] = '_' //
406
-};
407
-
408
-__attribute__((constructor)) void initialize_labels_keys_char_map(void) {
409
- // copy the values char map to the names char map
410
- size_t i;
411
- for(i = 0; i < 256 ;i++)
412
- label_names_char_map[i] = label_values_char_map[i];
413
-
414
- // apply overrides to the label names map
415
- label_names_char_map['='] = '_';
416
- label_names_char_map[':'] = '_';
417
- label_names_char_map['+'] = '_';
418
- label_names_char_map[';'] = '_';
419
- label_names_char_map['@'] = '_';
420
- label_names_char_map['('] = '_';
421
- label_names_char_map[')'] = '_';
422
- label_names_char_map[' '] = '_';
423
- label_names_char_map['\\'] = '/';
424
-
425
- // create the space map
426
- for(i = 0; i < 256 ;i++)
427
- label_spaces_char_map[i] = (isspace(i) || iscntrl(i) || !isprint(i))?1:0;
428
-
429
-}
430
-
94
__attribute__((constructor)) void initialize_label_stats(void) {
95
dictionary_stats_category_rrdlabels.memory.dict = 0;
96
dictionary_stats_category_rrdlabels.memory.index = 0;
97
dictionary_stats_category_rrdlabels.memory.values = 0;
98
}
99
437
-size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, const unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length) {
438
- if(unlikely(!src || !dst_size)) return 0;
439
-
440
- if(unlikely(!src || !*src)) {
441
- strncpyz((char *)dst, empty, dst_size);
442
- dst[dst_size - 1] = '\0';
443
- size_t len = strlen((char *)dst);
444
- if(multibyte_length) *multibyte_length = len;
445
- return len;
446
- }
447
-
448
- unsigned char *d = dst;
449
-
450
- // make room for the final string termination
451
- unsigned char *end = &d[dst_size - 1];
452
-
453
- // copy while converting, but keep only one space
454
- // we start wil last_is_space = 1 to skip leading spaces
455
- int last_is_space = 1;
456
-
457
- size_t mblen = 0;
458
-
459
- while(*src && d < end) {
460
- unsigned char c = *src;
461
-
462
- if(IS_UTF8_STARTBYTE(c) && IS_UTF8_BYTE(src[1]) && d + 2 < end) {
463
- // UTF-8 multi-byte encoded character
464
-
465
- // find how big this character is (2-4 bytes)
466
- size_t utf_character_size = 2;
467
- while(utf_character_size < 4 && src[utf_character_size] && IS_UTF8_BYTE(src[utf_character_size]) && !IS_UTF8_STARTBYTE(src[utf_character_size]))
468
- utf_character_size++;
469
-
470
- if(utf) {
471
- while(utf_character_size) {
472
- utf_character_size--;
473
- *d++ = *src++;
474
- }
475
- }
476
- else {
477
- // UTF-8 characters are not allowed.
478
- // Assume it is an underscore
479
- // and skip all except the first byte
480
- *d++ = '_';
481
- src += (utf_character_size - 1);
482
- }
483
-
484
- last_is_space = 0;
485
- mblen++;
486
- continue;
487
- }
488
-
489
- if(label_spaces_char_map[c]) {
490
- // a space character
491
-
492
- if(!last_is_space) {
493
- // add one space
494
- *d++ = char_map[c];
495
- mblen++;
496
- }
497
-
498
- last_is_space++;
499
- }
500
- else {
501
- *d++ = char_map[c];
502
- last_is_space = 0;
503
- mblen++;
504
- }
505
-
506
- src++;
507
- }
508
-
509
- // remove the last trailing space
510
- if(last_is_space && d > dst) {
511
- d--;
512
- mblen--;
513
- }
514
-
515
- // put a termination at the end of what we copied
516
- *d = '\0';
517
-
518
- // check if dst is all underscores and empty it if it is
519
- if(*dst == '_') {
520
- unsigned char *t = dst;
521
- while (*t == '_') t++;
522
- if (unlikely(*t == '\0')) {
523
- *dst = '\0';
524
- mblen = 0;
525
- }
526
- }
527
-
528
- if(unlikely(*dst == '\0')) {
529
- strncpyz((char *)dst, empty, dst_size);
530
- dst[dst_size - 1] = '\0';
531
- mblen = strlen((char *)dst);
532
- if(multibyte_length) *multibyte_length = mblen;
533
- return mblen;
534
- }
535
-
536
- if(multibyte_length) *multibyte_length = mblen;
537
-
538
- return d - dst;
539
-}
540
-
541
-static inline size_t rrdlabels_sanitize_name(char *dst, const char *src, size_t dst_size) {
542
- return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_names_char_map, 0, "", NULL);
543
-}
544
-
545
-static inline size_t rrdlabels_sanitize_value(char *dst, const char *src, size_t dst_size) {
546
- return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_values_char_map, 1, "[none]", NULL);
547
-}
548
-
100
// ----------------------------------------------------------------------------
101
// rrdlabels_create()
102
src/database/rrdlabels.h
-2
@@ -30,8 +30,6 @@ typedef enum __attribute__ ((__packed__)) rrdlabel_source {
30
31
#define RRDLABEL_FLAG_INTERNAL (RRDLABEL_FLAG_OLD | RRDLABEL_FLAG_NEW | RRDLABEL_FLAG_DONT_DELETE)
32
33
-size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, const unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length);
34
-
33
RRDLABELS *rrdlabels_create(void);
34
void rrdlabels_destroy(RRDLABELS *labels_dict);
35
void rrdlabels_add(RRDLABELS *labels, const char *name, const char *value, RRDLABEL_SRC ls);
src/libnetdata/config/dyncfg.c
+1
-1
@@ -277,7 +277,7 @@ int dyncfg_node_find_and_call(DICTIONARY *dyncfg_nodes, const char *transaction,
277
memcpy(buf, function, sizeof(buf));
278
279
char *words[MAX_FUNCTION_PARAMETERS]; // an array of pointers for the words in this line
280
- size_t num_words = quoted_strings_splitter_pluginsd(buf, words, MAX_FUNCTION_PARAMETERS);
280
+ size_t num_words = quoted_strings_splitter_whitespace(buf, words, MAX_FUNCTION_PARAMETERS);
281
282
const char *id = get_word(words, num_words, 1);
283
const char *action = get_word(words, num_words, 2);
src/libnetdata/facets/facets.c
-1
@@ -374,7 +374,6 @@ static const char *facets_key_value_id(FACET_KEY *k, FACET_VALUE *v) {
374
}
375
376
void facets_use_hashes_for_ids(FACETS *facets, bool set) {
377
- netdata_log_info("FACETS_OPTION_HASH_IDS = %s", set ? "true" : "false");
377
if(set)
378
facets->options |= FACETS_OPTION_HASH_IDS;
379
else
src/libnetdata/facets/logs_query_status.h
+1
-1
@@ -511,7 +511,7 @@ static inline bool lqs_request_parse_GET(LOGS_QUERY_STATUS *lqs, BUFFER *wb, cha
511
memcpy(func_copy, function, sizeof(func_copy));
512
513
char *words[LQS_MAX_PARAMS] = { NULL };
514
- size_t num_words = quoted_strings_splitter_pluginsd(func_copy, words, LQS_MAX_PARAMS);
514
+ size_t num_words = quoted_strings_splitter_whitespace(func_copy, words, LQS_MAX_PARAMS);
515
for(int i = 1; i < LQS_MAX_PARAMS;i++) {
516
char *keyword = get_word(words, num_words, i);
517
if(!keyword) break;
src/libnetdata/functions_evloop/functions_evloop.c
+1
-1
@@ -253,7 +253,7 @@ static void *rrd_functions_worker_globals_reader_main(void *arg) {
253
}
254
255
char *words[MAX_FUNCTION_PARAMETERS] = { NULL };
256
- size_t num_words = quoted_strings_splitter_pluginsd((char *)buffer_tostring(buffer), words, MAX_FUNCTION_PARAMETERS);
256
+ size_t num_words = quoted_strings_splitter_whitespace((char *)buffer_tostring(buffer), words, MAX_FUNCTION_PARAMETERS);
257
258
const char *keyword = get_word(words, num_words, 0);
259
src/libnetdata/libnetdata.h
+1
-1
@@ -472,7 +472,7 @@ extern const char *netdata_configured_host_prefix;
472
#include "string/string.h"
473
#include "dictionary/dictionary.h"
474
#include "dictionary/thread-cache.h"
475
-#include "sanitizers/chart_id_and_name.h"
475
+#include "sanitizers/sanitizers.h"
476
477
#if defined(HAVE_LIBBPF) && !defined(__cplusplus)
478
#include "ebpf/ebpf.h"
src/libnetdata/line_splitter/line_splitter.c
+29
-8
@@ -21,12 +21,29 @@ bool line_splitter_reconstruct_line(BUFFER *wb, void *ptr) {
21
return added > 0;
22
}
23
24
-inline int pluginsd_isspace(char c) {
24
+inline int isspace_whitespace(char c) {
25
switch(c) {
26
case ' ':
27
case '\t':
28
case '\r':
29
case '\n':
30
+ case '\f':
31
+ case '\v':
32
+ return 1;
33
+
34
+ default:
35
+ return 0;
36
+ }
37
+}
38
+
39
+inline int isspace_pluginsd(char c) {
40
+ switch(c) {
41
+ case ' ':
42
+ case '\t':
43
+ case '\r':
44
+ case '\n':
45
+ case '\f':
46
+ case '\v':
47
case '=':
48
return 1;
49
@@ -35,12 +52,14 @@ inline int pluginsd_isspace(char c) {
52
}
53
}
54
38
-inline int config_isspace(char c) {
55
+inline int isspace_config(char c) {
56
switch (c) {
57
case ' ':
58
case '\t':
59
case '\r':
60
case '\n':
61
+ case '\f':
62
+ case '\v':
63
case ',':
64
return 1;
65
@@ -49,20 +68,21 @@ inline int config_isspace(char c) {
68
}
69
}
70
52
-inline int group_by_label_isspace(char c) {
71
+inline int isspace_group_by_label(char c) {
72
if(c == ',' || c == '|')
73
return 1;
74
75
return 0;
76
}
77
59
-inline int dyncfg_id_isspace(char c) {
78
+inline int isspace_dyncfg_id(char c) {
79
if(c == ':')
80
return 1;
81
82
return 0;
83
}
84
85
+bool isspace_map_whitespace[256] = {};
86
bool isspace_map_pluginsd[256] = {};
87
bool isspace_map_config[256] = {};
88
bool isspace_map_group_by_label[256] = {};
@@ -70,9 +90,10 @@ bool isspace_dyncfg_id_map[256] = {};
90
91
__attribute__((constructor)) void initialize_is_space_arrays(void) {
92
for(int c = 0; c < 256 ; c++) {
73
- isspace_map_pluginsd[c] = pluginsd_isspace((char) c);
74
- isspace_map_config[c] = config_isspace((char) c);
75
- isspace_map_group_by_label[c] = group_by_label_isspace((char) c);
76
- isspace_dyncfg_id_map[c] = dyncfg_id_isspace((char)c);
93
+ isspace_map_whitespace[c] = isspace_whitespace((char) c);
94
+ isspace_map_pluginsd[c] = isspace_pluginsd((char) c);
95
+ isspace_map_config[c] = isspace_config((char) c);
96
+ isspace_map_group_by_label[c] = isspace_group_by_label((char) c);
97
+ isspace_dyncfg_id_map[c] = isspace_dyncfg_id((char) c);
98
}
99
}
src/libnetdata/line_splitter/line_splitter.h
+9
-4
@@ -19,11 +19,13 @@ static inline void line_splitter_reset(struct line_splitter *line) {
19
line->num_words = 0;
20
}
21
22
-int pluginsd_isspace(char c);
23
-int config_isspace(char c);
24
-int group_by_label_isspace(char c);
25
-int dyncfg_id_isspace(char c);
22
+int isspace_pluginsd(char c);
23
+int isspace_config(char c);
24
+int isspace_group_by_label(char c);
25
+int isspace_dyncfg_id(char c);
26
+int isspace_whitespace(char c);
27
28
+extern bool isspace_map_whitespace[256];
29
extern bool isspace_map_pluginsd[256];
30
extern bool isspace_map_config[256];
31
extern bool isspace_map_group_by_label[256];
@@ -103,6 +105,9 @@ static inline size_t quoted_strings_splitter(char *str, char **words, size_t max
105
return i;
106
}
107
108
+#define quoted_strings_splitter_whitespace(str, words, max_words) \
109
+ quoted_strings_splitter(str, words, max_words, isspace_map_whitespace)
110
+
111
#define quoted_strings_splitter_query_group_by_label(str, words, max_words) \
112
quoted_strings_splitter(str, words, max_words, isspace_map_group_by_label)
113
src/libnetdata/log/log.c
+18
-12
@@ -1627,30 +1627,36 @@ static void errno_annotator(BUFFER *wb, const char *key, struct log_field *lf) {
1627
static void winerror_annotator(BUFFER *wb, const char *key, struct log_field *lf) {
1628
DWORD errnum = log_field_to_uint64(lf);
1629
1630
- if(errnum == 0)
1630
+ if (errnum == 0)
1631
return;
1632
1633
char buf[1024];
1634
- DWORD size = FormatMessageA(
1634
+ wchar_t wbuf[1024];
1635
+ DWORD size = FormatMessageW(
1636
FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
1637
NULL,
1638
errnum,
1639
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
1639
- buf,
1640
- (DWORD)(sizeof(buf) - 1),
1640
+ wbuf,
1641
+ (DWORD)(sizeof(wbuf) / sizeof(wchar_t) - 1),
1642
NULL
1643
);
1643
- if(size > 0) {
1644
- // remove \r\n at the end
1645
- while(size > 0 && (buf[size - 1] == '\r' || buf[size - 1] == '\n'))
1646
- buf[--size] = '\0';
1644
+
1645
+ if (size > 0) {
1646
+ // Remove \r\n at the end
1647
+ while (size > 0 && (wbuf[size - 1] == L'\r' || wbuf[size - 1] == L'\n'))
1648
+ wbuf[--size] = L'\0';
1649
+
1650
+ // Convert wide string to UTF-8
1651
+ int utf8_size = WideCharToMultiByte(CP_UTF8, 0, wbuf, -1, buf, sizeof(buf), NULL, NULL);
1652
+ if (utf8_size == 0)
1653
+ snprintf(buf, sizeof(buf) - 1, "unknown error code");
1654
+ buf[sizeof(buf) - 1] = '\0';
1655
}
1656
else
1649
- size = snprintf(buf, sizeof(buf) - 1, "unknown error code");
1657
+ snprintf(buf, sizeof(buf) - 1, "unknown error code");
1658
1651
- buf[size] = '\0';
1652
-
1653
- if(buffer_strlen(wb))
1659
+ if (buffer_strlen(wb))
1660
buffer_fast_strcat(wb, " ", 1);
1661
1662
buffer_strcat(wb, key);
src/libnetdata/sanitizers/chart_id_and_name.c
-85
@@ -531,88 +531,3 @@ static uint8_t netdata_map_chart_ids[256] = {
531
void netdata_fix_chart_id(char *s) {
532
while ((*s = netdata_map_chart_ids[(uint8_t) *s])) s++;
533
}
534
-
535
-// --------------------------------------------------------------------------------------------------------------------
536
-
537
-/*
538
- * Sanitize Chart Meta
539
- *
540
- * We should only remove characters that somehow influence the functionality of Netdata.
541
- *
542
- * The following ASCII characters are replaced with underscores:
543
- *
544
- * comma and pipe Used in HTTP GET as list separators
545
- * colon Used in Labels as value list separators
546
- * space Used in HTTP GET as parameter separators
547
- * asterisk Used in simple patterns as wildcard
548
- * exclamation mark Used in simple patterns for negative matches
549
- * Control characters and not-printable characters (<= ' ')
550
- *
551
- * Non-ASCII UTF8 characters are retained as-is.
552
- *
553
- * When multiple consecutive characters are converted to underscores
554
- * only 1 underscore is copied to the output buffer.
555
- *
556
- */
557
-void sanitize_chart_meta(char *buf) {
558
- char *s = buf, *d = buf;
559
- bool last_was_underscore = false;
560
-
561
- // Process the input buffer
562
- while (*s) {
563
- if (!IS_UTF8_BYTE(*s)) { // ASCII character
564
- if (*s == '_') {
565
- if(!last_was_underscore)
566
- *d++ = *s++;
567
- else
568
- s++;
569
-
570
- last_was_underscore = true;
571
- }
572
- else if (*s <= ' ' || // all non-printable and control characters, including the space
573
- *s == ',' || // HTTP GET list separator
574
- *s == ':' || // labels value list separator
575
- *s == '|' || // HTTP GET list separator
576
- *s == '*' || // simple pattern wildcard
577
- *s == '!' // simple pattern negative match
578
- ) {
579
- if (!last_was_underscore)
580
- *d++ = '_';
581
-
582
- s++;
583
- last_was_underscore = true;
584
- }
585
- else {
586
- *d++ = *s++;
587
- last_was_underscore = false;
588
- }
589
- }
590
- else if (IS_UTF8_STARTBYTE(*s)) {
591
- // copy the start byte
592
- *d++ = *s++;
593
-
594
- // copy the rest of the wide character
595
- while (IS_UTF8_CONTBYTE(*s))
596
- *d++ = *s++;
597
-
598
- last_was_underscore = false;
599
- }
600
- else
601
- // an invalid UTF8 continuation byte - skip it
602
- s++;
603
- }
604
-
605
- // Null-terminate the cleaned string
606
- *d = '\0';
607
-
608
- // Remove a trailing dash (if any)
609
- if (d > buf && *(d - 1) == '_')
610
- *(d - 1) = '\0';
611
-}
612
-
613
-STRING *sanitize_chart_meta_string(STRING *s) {
614
- char buf[string_strlen(s) + 1];
615
- memcpy(buf, string2str(s), sizeof(buf));
616
- sanitize_chart_meta(buf);
617
- return string_strdupz(buf);
618
-}
src/libnetdata/sanitizers/chart_id_and_name.h
-3
@@ -8,7 +8,4 @@
8
void netdata_fix_chart_id(char *s);
9
void netdata_fix_chart_name(char *s);
10
11
-void sanitize_chart_meta(char *buf);
12
-STRING *sanitize_chart_meta_string(STRING *s);
13
-
11
#endif //NETDATA_CHART_ID_AND_NAME_H
src/libnetdata/sanitizers/sanitizers-functions.c
new
+68
@@ -0,0 +1,68 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#include "sanitizers-functions.h"
4
+
5
+static unsigned char functions_allowed_chars[256] = {
6
+ [0] = '\0', [1] = '_', [2] = '_', [3] = '_', [4] = '_', [5] = '_', [6] = '_', [7] = '_', [8] = '_',
7
+
8
+ // control characters to be treated as spaces
9
+ ['\t'] = ' ', ['\n'] = ' ', ['\v'] = ' ', ['\f'] = ' ', ['\r'] = ' ',
10
+
11
+ [14] = '_', [15] = '_', [16] = '_', [17] = '_', [18] = '_', [19] = '_', [20] = '_', [21] = '_',
12
+ [22] = '_', [23] = '_', [24] = '_', [25] = '_', [26] = '_', [27] = '_', [28] = '_', [29] = '_',
13
+ [30] = '_', [31] = '_',
14
+
15
+ // symbols
16
+ [' '] = ' ', ['!'] = '!', ['"'] = '\'', ['#'] = '#', ['$'] = '$', ['%'] = '%', ['&'] = '&', ['\''] = '\'',
17
+ ['('] = '(', [')'] = ')', ['*'] = '*', ['+'] = '+', [','] = ',', ['-'] = '-', ['.'] = '.', ['/'] = '/',
18
+
19
+ // numbers
20
+ ['0'] = '0', ['1'] = '1', ['2'] = '2', ['3'] = '3', ['4'] = '4', ['5'] = '5', ['6'] = '6', ['7'] = '7',
21
+ ['8'] = '8', ['9'] = '9',
22
+
23
+ // symbols
24
+ [':'] = ':', [';'] = ';', ['<'] = '<', ['='] = '=', ['>'] = '>', ['?'] = '?', ['@'] = '@',
25
+
26
+ // capitals
27
+ ['A'] = 'A', ['B'] = 'B', ['C'] = 'C', ['D'] = 'D', ['E'] = 'E', ['F'] = 'F', ['G'] = 'G', ['H'] = 'H',
28
+ ['I'] = 'I', ['J'] = 'J', ['K'] = 'K', ['L'] = 'L', ['M'] = 'M', ['N'] = 'N', ['O'] = 'O', ['P'] = 'P',
29
+ ['Q'] = 'Q', ['R'] = 'R', ['S'] = 'S', ['T'] = 'T', ['U'] = 'U', ['V'] = 'V', ['W'] = 'W', ['X'] = 'X',
30
+ ['Y'] = 'Y', ['Z'] = 'Z',
31
+
32
+ // symbols
33
+ ['['] = '[', ['\\'] = '\\', [']'] = ']', ['^'] = '^', ['_'] = '_', ['`'] = '`',
34
+
35
+ // lower
36
+ ['a'] = 'a', ['b'] = 'b', ['c'] = 'c', ['d'] = 'd', ['e'] = 'e', ['f'] = 'f', ['g'] = 'g', ['h'] = 'h',
37
+ ['i'] = 'i', ['j'] = 'j', ['k'] = 'k', ['l'] = 'l', ['m'] = 'm', ['n'] = 'n', ['o'] = 'o', ['p'] = 'p',
38
+ ['q'] = 'q', ['r'] = 'r', ['s'] = 's', ['t'] = 't', ['u'] = 'u', ['v'] = 'v', ['w'] = 'w', ['x'] = 'x',
39
+ ['y'] = 'y', ['z'] = 'z',
40
+
41
+ // symbols
42
+ ['{'] = '{', ['|'] = '|', ['}'] = '}', ['~'] = '~',
43
+
44
+ // rest
45
+ [127] = '_', [128] = '_', [129] = '_', [130] = '_', [131] = '_', [132] = '_', [133] = '_', [134] = '_',
46
+ [135] = '_', [136] = '_', [137] = '_', [138] = '_', [139] = '_', [140] = '_', [141] = '_', [142] = '_',
47
+ [143] = '_', [144] = '_', [145] = '_', [146] = '_', [147] = '_', [148] = '_', [149] = '_', [150] = '_',
48
+ [151] = '_', [152] = '_', [153] = '_', [154] = '_', [155] = '_', [156] = '_', [157] = '_', [158] = '_',
49
+ [159] = '_', [160] = '_', [161] = '_', [162] = '_', [163] = '_', [164] = '_', [165] = '_', [166] = '_',
50
+ [167] = '_', [168] = '_', [169] = '_', [170] = '_', [171] = '_', [172] = '_', [173] = '_', [174] = '_',
51
+ [175] = '_', [176] = '_', [177] = '_', [178] = '_', [179] = '_', [180] = '_', [181] = '_', [182] = '_',
52
+ [183] = '_', [184] = '_', [185] = '_', [186] = '_', [187] = '_', [188] = '_', [189] = '_', [190] = '_',
53
+ [191] = '_', [192] = '_', [193] = '_', [194] = '_', [195] = '_', [196] = '_', [197] = '_', [198] = '_',
54
+ [199] = '_', [200] = '_', [201] = '_', [202] = '_', [203] = '_', [204] = '_', [205] = '_', [206] = '_',
55
+ [207] = '_', [208] = '_', [209] = '_', [210] = '_', [211] = '_', [212] = '_', [213] = '_', [214] = '_',
56
+ [215] = '_', [216] = '_', [217] = '_', [218] = '_', [219] = '_', [220] = '_', [221] = '_', [222] = '_',
57
+ [223] = '_', [224] = '_', [225] = '_', [226] = '_', [227] = '_', [228] = '_', [229] = '_', [230] = '_',
58
+ [231] = '_', [232] = '_', [233] = '_', [234] = '_', [235] = '_', [236] = '_', [237] = '_', [238] = '_',
59
+ [239] = '_', [240] = '_', [241] = '_', [242] = '_', [243] = '_', [244] = '_', [245] = '_', [246] = '_',
60
+ [247] = '_', [248] = '_', [249] = '_', [250] = '_', [251] = '_', [252] = '_', [253] = '_', [254] = '_',
61
+ [255] = '_'
62
+};
63
+
64
+size_t rrd_functions_sanitize(char *dst, const char *src, size_t dst_len) {
65
+ return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_len,
66
+ functions_allowed_chars, true, "", NULL);
67
+}
68
+
src/libnetdata/sanitizers/sanitizers-functions.h
new
+10
@@ -0,0 +1,10 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#ifndef NETDATA_SANITIZERS_FUNCTIONS_H
4
+#define NETDATA_SANITIZERS_FUNCTIONS_H
5
+
6
+#include "../libnetdata.h"
7
+
8
+size_t rrd_functions_sanitize(char *dst, const char *src, size_t dst_len);
9
+
10
+#endif //NETDATA_SANITIZERS_FUNCTIONS_H
src/libnetdata/sanitizers/sanitizers-labels.c
new
+138
@@ -0,0 +1,138 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#include "sanitizers-labels.h"
4
+
5
+/*
6
+ * All labels follow these rules:
7
+ *
8
+ * Character Symbol Values Names
9
+ * UTF-8 characters UTF-8 yes -> _
10
+ * Lower case letter [a-z] yes yes
11
+ * Upper case letter [A-Z] yes -> [a-z]
12
+ * Digit [0-9] yes yes
13
+ * Underscore _ yes yes
14
+ * Minus - yes yes
15
+ * Plus + yes -> _
16
+ * Colon : yes -> _
17
+ * Semicolon ; -> : -> _
18
+ * Equal = -> : -> _
19
+ * Period . yes yes
20
+ * Comma , -> . -> .
21
+ * Slash / yes yes
22
+ * Backslash \ -> / -> /
23
+ * At @ yes -> _
24
+ * Space yes -> _
25
+ * Opening parenthesis ( yes -> _
26
+ * Closing parenthesis ) yes -> _
27
+ * anything else -> _ -> _
28
+*
29
+ * The above rules should allow users to set in tags (indicative):
30
+ *
31
+ * 1. hostnames and domain names as-is
32
+ * 2. email addresses as-is
33
+ * 3. floating point numbers, converted to always use a dot as the decimal point
34
+ *
35
+ * Leading and trailing spaces and control characters are removed from both label
36
+ * names and values.
37
+ *
38
+ * Multiple spaces inside the label name or the value are removed (only 1 is retained).
39
+ * In names spaces are also converted to underscores.
40
+ *
41
+ * Names that are only underscores are rejected (they do not enter the dictionary).
42
+ *
43
+ * The above rules do not require any conversion to be included in JSON strings.
44
+ *
45
+ * Label names and values are truncated to LABELS_MAX_LENGTH (200) characters.
46
+ *
47
+ * When parsing, label key and value are separated by the first colon (:) found.
48
+ * So label:value1:value2 is parsed as key = "label", value = "value1:value2"
49
+ *
50
+ * This means a label key cannot contain a colon (:) - it is converted to
51
+ * underscore if it does.
52
+ *
53
+ */
54
+
55
+static unsigned char label_names_char_map[256];
56
+static unsigned char label_values_char_map[256] = {
57
+ [0] = '\0', [1] = ' ', [2] = ' ', [3] = ' ', [4] = ' ', [5] = ' ', [6] = ' ', [7] = ' ', [8] = ' ',
58
+
59
+ // control characters to be treated as spaces
60
+ ['\t'] = ' ', ['\n'] = ' ', ['\v'] = ' ', ['\f'] = ' ', ['\r'] = ' ',
61
+
62
+ [14] = ' ', [15] = ' ', [16] = ' ', [17] = ' ', [18] = ' ', [19] = ' ', [20] = ' ', [21] = ' ',
63
+ [22] = ' ', [23] = ' ', [24] = ' ', [25] = ' ', [26] = ' ', [27] = ' ', [28] = ' ', [29] = ' ',
64
+ [30] = ' ', [31] = ' ',
65
+
66
+ // symbols
67
+ [' '] = ' ', ['!'] = '_', ['"'] = '_', ['#'] = '_', ['$'] = '_', ['%'] = '_', ['&'] = '_', ['\''] = '_',
68
+ ['('] = '(', [')'] = ')', ['*'] = '_', ['+'] = '+', [','] = '.', ['-'] = '-', ['.'] = '.', ['/'] = '/',
69
+
70
+ // numbers
71
+ ['0'] = '0', ['1'] = '1', ['2'] = '2', ['3'] = '3', ['4'] = '4', ['5'] = '5', ['6'] = '6', ['7'] = '7',
72
+ ['8'] = '8', ['9'] = '9',
73
+
74
+ // symbols
75
+ [':'] = ':', [';'] = ':', ['<'] = '_', ['='] = ':', ['>'] = '_', ['?'] = '_', ['@'] = '@',
76
+
77
+ // capitals
78
+ ['A'] = 'A', ['B'] = 'B', ['C'] = 'C', ['D'] = 'D', ['E'] = 'E', ['F'] = 'F', ['G'] = 'G', ['H'] = 'H',
79
+ ['I'] = 'I', ['J'] = 'J', ['K'] = 'K', ['L'] = 'L', ['M'] = 'M', ['N'] = 'N', ['O'] = 'O', ['P'] = 'P',
80
+ ['Q'] = 'Q', ['R'] = 'R', ['S'] = 'S', ['T'] = 'T', ['U'] = 'U', ['V'] = 'V', ['W'] = 'W', ['X'] = 'X',
81
+ ['Y'] = 'Y', ['Z'] = 'Z',
82
+
83
+ // symbols
84
+ ['['] = '[', ['\\'] = '/', [']'] = ']', ['^'] = '_', ['_'] = '_', ['`'] = '_',
85
+
86
+ // lower
87
+ ['a'] = 'a', ['b'] = 'b', ['c'] = 'c', ['d'] = 'd', ['e'] = 'e', ['f'] = 'f', ['g'] = 'g', ['h'] = 'h',
88
+ ['i'] = 'i', ['j'] = 'j', ['k'] = 'k', ['l'] = 'l', ['m'] = 'm', ['n'] = 'n', ['o'] = 'o', ['p'] = 'p',
89
+ ['q'] = 'q', ['r'] = 'r', ['s'] = 's', ['t'] = 't', ['u'] = 'u', ['v'] = 'v', ['w'] = 'w', ['x'] = 'x',
90
+ ['y'] = 'y', ['z'] = 'z',
91
+
92
+ // symbols
93
+ ['{'] = '_', ['|'] = '_', ['}'] = '_', ['~'] = '_',
94
+
95
+ // rest
96
+ [127] = ' ', [128] = ' ', [129] = ' ', [130] = ' ', [131] = ' ', [132] = ' ', [133] = ' ', [134] = ' ',
97
+ [135] = ' ', [136] = ' ', [137] = ' ', [138] = ' ', [139] = ' ', [140] = ' ', [141] = ' ', [142] = ' ',
98
+ [143] = ' ', [144] = ' ', [145] = ' ', [146] = ' ', [147] = ' ', [148] = ' ', [149] = ' ', [150] = ' ',
99
+ [151] = ' ', [152] = ' ', [153] = ' ', [154] = ' ', [155] = ' ', [156] = ' ', [157] = ' ', [158] = ' ',
100
+ [159] = ' ', [160] = ' ', [161] = ' ', [162] = ' ', [163] = ' ', [164] = ' ', [165] = ' ', [166] = ' ',
101
+ [167] = ' ', [168] = ' ', [169] = ' ', [170] = ' ', [171] = ' ', [172] = ' ', [173] = ' ', [174] = ' ',
102
+ [175] = ' ', [176] = ' ', [177] = ' ', [178] = ' ', [179] = ' ', [180] = ' ', [181] = ' ', [182] = ' ',
103
+ [183] = ' ', [184] = ' ', [185] = ' ', [186] = ' ', [187] = ' ', [188] = ' ', [189] = ' ', [190] = ' ',
104
+ [191] = ' ', [192] = ' ', [193] = ' ', [194] = ' ', [195] = ' ', [196] = ' ', [197] = ' ', [198] = ' ',
105
+ [199] = ' ', [200] = ' ', [201] = ' ', [202] = ' ', [203] = ' ', [204] = ' ', [205] = ' ', [206] = ' ',
106
+ [207] = ' ', [208] = ' ', [209] = ' ', [210] = ' ', [211] = ' ', [212] = ' ', [213] = ' ', [214] = ' ',
107
+ [215] = ' ', [216] = ' ', [217] = ' ', [218] = ' ', [219] = ' ', [220] = ' ', [221] = ' ', [222] = ' ',
108
+ [223] = ' ', [224] = ' ', [225] = ' ', [226] = ' ', [227] = ' ', [228] = ' ', [229] = ' ', [230] = ' ',
109
+ [231] = ' ', [232] = ' ', [233] = ' ', [234] = ' ', [235] = ' ', [236] = ' ', [237] = ' ', [238] = ' ',
110
+ [239] = ' ', [240] = ' ', [241] = ' ', [242] = ' ', [243] = ' ', [244] = ' ', [245] = ' ', [246] = ' ',
111
+ [247] = ' ', [248] = ' ', [249] = ' ', [250] = ' ', [251] = ' ', [252] = ' ', [253] = ' ', [254] = ' ',
112
+ [255] = ' '
113
+};
114
+
115
+__attribute__((constructor)) void initialize_labels_keys_char_map(void) {
116
+ // copy the values char map to the names char map
117
+ size_t i;
118
+ for(i = 0; i < 256 ;i++)
119
+ label_names_char_map[i] = label_values_char_map[i];
120
+
121
+ // apply overrides to the label names map
122
+ label_names_char_map['='] = '_';
123
+ label_names_char_map[':'] = '_';
124
+ label_names_char_map['+'] = '_';
125
+ label_names_char_map[';'] = '_';
126
+ label_names_char_map['@'] = '_';
127
+ label_names_char_map['('] = '_';
128
+ label_names_char_map[')'] = '_';
129
+ label_names_char_map['\\'] = '/';
130
+}
131
+
132
+size_t rrdlabels_sanitize_name(char *dst, const char *src, size_t dst_size) {
133
+ return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_names_char_map, 0, "", NULL);
134
+}
135
+
136
+size_t rrdlabels_sanitize_value(char *dst, const char *src, size_t dst_size) {
137
+ return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_values_char_map, 1, "[none]", NULL);
138
+}
src/libnetdata/sanitizers/sanitizers-labels.h
new
+11
@@ -0,0 +1,11 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#ifndef NETDATA_SANITIZERS_LABELS_H
4
+#define NETDATA_SANITIZERS_LABELS_H
5
+
6
+#include "../libnetdata.h"
7
+
8
+size_t rrdlabels_sanitize_name(char *dst, const char *src, size_t dst_size);
9
+size_t rrdlabels_sanitize_value(char *dst, const char *src, size_t dst_size);
10
+
11
+#endif //NETDATA_SANITIZERS_LABELS_H
src/libnetdata/sanitizers/sanitizers-pluginsd.c
new
+80
@@ -0,0 +1,80 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#include "sanitizers-pluginsd.h"
4
+
5
+/*
6
+ * Undefined and control characters become underscores
7
+ * ! -> _
8
+ * " -> _
9
+ * ' -> _
10
+ * ` -> _
11
+ * \ -> /
12
+ * : -> _
13
+ * = -> _
14
+ * | -> _
15
+ */
16
+
17
+static unsigned char external_plugins_map[256] = {
18
+ [0] = '\0', [1] = '_', [2] = '_', [3] = '_', [4] = '_', [5] = '_', [6] = '_', [7] = '_', [8] = '_',
19
+
20
+ // control characters to be treated as spaces
21
+ ['\t'] = ' ', ['\n'] = ' ', ['\v'] = ' ', ['\f'] = ' ', ['\r'] = ' ',
22
+
23
+ [14] = '_', [15] = '_', [16] = '_', [17] = '_', [18] = '_', [19] = '_', [20] = '_', [21] = '_',
24
+ [22] = '_', [23] = '_', [24] = '_', [25] = '_', [26] = '_', [27] = '_', [28] = '_', [29] = '_',
25
+ [30] = '_', [31] = '_',
26
+
27
+ // symbols
28
+ [' '] = ' ', ['!'] = '_', ['"'] = '_', ['#'] = '#', ['$'] = '$', ['%'] = '%', ['&'] = '&', ['\''] = '_',
29
+ ['('] = '(', [')'] = ')', ['*'] = '*', ['+'] = '+', [','] = ',', ['-'] = '-', ['.'] = '.', ['/'] = '/',
30
+
31
+ // numbers
32
+ ['0'] = '0', ['1'] = '1', ['2'] = '2', ['3'] = '3', ['4'] = '4', ['5'] = '5', ['6'] = '6', ['7'] = '7',
33
+ ['8'] = '8', ['9'] = '9',
34
+
35
+ // symbols
36
+ [':'] = '_', [';'] = ';', ['<'] = '<', ['='] = '_', ['>'] = '>', ['?'] = '?', ['@'] = '@',
37
+
38
+ // capitals
39
+ ['A'] = 'A', ['B'] = 'B', ['C'] = 'C', ['D'] = 'D', ['E'] = 'E', ['F'] = 'F', ['G'] = 'G', ['H'] = 'H',
40
+ ['I'] = 'I', ['J'] = 'J', ['K'] = 'K', ['L'] = 'L', ['M'] = 'M', ['N'] = 'N', ['O'] = 'O', ['P'] = 'P',
41
+ ['Q'] = 'Q', ['R'] = 'R', ['S'] = 'S', ['T'] = 'T', ['U'] = 'U', ['V'] = 'V', ['W'] = 'W', ['X'] = 'X',
42
+ ['Y'] = 'Y', ['Z'] = 'Z',
43
+
44
+ // symbols
45
+ ['['] = '[', ['\\'] = '/', [']'] = ']', ['^'] = '^', ['_'] = '_', ['`'] = '_',
46
+
47
+ // lower
48
+ ['a'] = 'a', ['b'] = 'b', ['c'] = 'c', ['d'] = 'd', ['e'] = 'e', ['f'] = 'f', ['g'] = 'g', ['h'] = 'h',
49
+ ['i'] = 'i', ['j'] = 'j', ['k'] = 'k', ['l'] = 'l', ['m'] = 'm', ['n'] = 'n', ['o'] = 'o', ['p'] = 'p',
50
+ ['q'] = 'q', ['r'] = 'r', ['s'] = 's', ['t'] = 't', ['u'] = 'u', ['v'] = 'v', ['w'] = 'w', ['x'] = 'x',
51
+ ['y'] = 'y', ['z'] = 'z',
52
+
53
+ // symbols
54
+ ['{'] = '{', ['|'] = '_', ['}'] = '}', ['~'] = '~',
55
+
56
+ // rest
57
+ [127] = '_', [128] = '_', [129] = '_', [130] = '_', [131] = '_', [132] = '_', [133] = '_', [134] = '_',
58
+ [135] = '_', [136] = '_', [137] = '_', [138] = '_', [139] = '_', [140] = '_', [141] = '_', [142] = '_',
59
+ [143] = '_', [144] = '_', [145] = '_', [146] = '_', [147] = '_', [148] = '_', [149] = '_', [150] = '_',
60
+ [151] = '_', [152] = '_', [153] = '_', [154] = '_', [155] = '_', [156] = '_', [157] = '_', [158] = '_',
61
+ [159] = '_', [160] = '_', [161] = '_', [162] = '_', [163] = '_', [164] = '_', [165] = '_', [166] = '_',
62
+ [167] = '_', [168] = '_', [169] = '_', [170] = '_', [171] = '_', [172] = '_', [173] = '_', [174] = '_',
63
+ [175] = '_', [176] = '_', [177] = '_', [178] = '_', [179] = '_', [180] = '_', [181] = '_', [182] = '_',
64
+ [183] = '_', [184] = '_', [185] = '_', [186] = '_', [187] = '_', [188] = '_', [189] = '_', [190] = '_',
65
+ [191] = '_', [192] = '_', [193] = '_', [194] = '_', [195] = '_', [196] = '_', [197] = '_', [198] = '_',
66
+ [199] = '_', [200] = '_', [201] = '_', [202] = '_', [203] = '_', [204] = '_', [205] = '_', [206] = '_',
67
+ [207] = '_', [208] = '_', [209] = '_', [210] = '_', [211] = '_', [212] = '_', [213] = '_', [214] = '_',
68
+ [215] = '_', [216] = '_', [217] = '_', [218] = '_', [219] = '_', [220] = '_', [221] = '_', [222] = '_',
69
+ [223] = '_', [224] = '_', [225] = '_', [226] = '_', [227] = '_', [228] = '_', [229] = '_', [230] = '_',
70
+ [231] = '_', [232] = '_', [233] = '_', [234] = '_', [235] = '_', [236] = '_', [237] = '_', [238] = '_',
71
+ [239] = '_', [240] = '_', [241] = '_', [242] = '_', [243] = '_', [244] = '_', [245] = '_', [246] = '_',
72
+ [247] = '_', [248] = '_', [249] = '_', [250] = '_', [251] = '_', [252] = '_', [253] = '_', [254] = '_',
73
+ [255] = '_'
74
+};
75
+
76
+size_t external_plugins_sanitize(char *dst, const char *src, size_t dst_len) {
77
+ return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_len,
78
+ external_plugins_map, true, "", NULL);
79
+}
80
+
src/libnetdata/sanitizers/sanitizers-pluginsd.h
new
+10
@@ -0,0 +1,10 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#ifndef NETDATA_SANITIZERS_PLUGINSD_H
4
+#define NETDATA_SANITIZERS_PLUGINSD_H
5
+
6
+#include "../libnetdata.h"
7
+
8
+size_t external_plugins_sanitize(char *dst, const char *src, size_t dst_len);
9
+
10
+#endif //NETDATA_SANITIZERS_PLUGINSD_H
src/libnetdata/sanitizers/sanitizers.h
new
+12
@@ -0,0 +1,12 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#ifndef NETDATA_SANITIZERS_H
4
+#define NETDATA_SANITIZERS_H
5
+
6
+#include "utf8-sanitizer.h"
7
+#include "sanitizers-labels.h"
8
+#include "sanitizers-functions.h"
9
+#include "sanitizers-pluginsd.h"
10
+#include "chart_id_and_name.h"
11
+
12
+#endif //NETDATA_SANITIZERS_H
src/libnetdata/sanitizers/utf8-sanitizer.c
new
+116
@@ -0,0 +1,116 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#include "../libnetdata.h"
4
+
5
+size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, const unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length) {
6
+ if(unlikely(!src || !dst || !dst_size)) return 0;
7
+
8
+ // skip leading spaces and invalid characters
9
+ while(src && *src && !IS_UTF8_BYTE(*src) && (isspace(*src) || iscntrl(*src) || !isprint(*src)))
10
+ src++;
11
+
12
+ if(unlikely(!src || !*src)) {
13
+ strncpyz((char *)dst, empty, dst_size);
14
+ dst[dst_size - 1] = '\0';
15
+ size_t len = strlen((char *)dst);
16
+ if(multibyte_length) *multibyte_length = len;
17
+ return len;
18
+ }
19
+
20
+ unsigned char *d = dst;
21
+
22
+ // make room for the final string termination
23
+ unsigned char *end = &dst[dst_size - 1];
24
+
25
+ // copy while converting, but keep only one space
26
+ // we start wil last_is_space = 1 to skip leading spaces
27
+ int last_is_space = 1;
28
+
29
+ size_t mblen = 0;
30
+
31
+ while(*src && d < end) {
32
+ unsigned char c = *src;
33
+
34
+ if(IS_UTF8_STARTBYTE(c) && IS_UTF8_BYTE(src[1]) && d + 2 <= end) {
35
+ // UTF-8 multi-byte encoded character
36
+
37
+ // find how big this character is (2-4 bytes)
38
+ size_t utf_character_size = 2;
39
+ while(utf_character_size < 4 &&
40
+ d + utf_character_size <= end &&
41
+ IS_UTF8_BYTE(src[utf_character_size]) &&
42
+ !IS_UTF8_STARTBYTE(src[utf_character_size]))
43
+ utf_character_size++;
44
+
45
+ if(utf) {
46
+ while(utf_character_size) {
47
+ utf_character_size--;
48
+ *d++ = *src++;
49
+ }
50
+ }
51
+ else {
52
+ // UTF-8 characters are not allowed.
53
+ // Assume it is an underscore
54
+ // and skip all except the first byte
55
+ *d++ = '_';
56
+ src += (utf_character_size - 1);
57
+ }
58
+
59
+ last_is_space = 0;
60
+ mblen++;
61
+ continue;
62
+ }
63
+
64
+ c = char_map[c];
65
+ if(c == ' ') {
66
+ // a space character
67
+
68
+ if(!last_is_space) {
69
+ // add one space
70
+ *d++ = c;
71
+ mblen++;
72
+ }
73
+
74
+ last_is_space++;
75
+ }
76
+ else {
77
+ *d++ = c;
78
+ last_is_space = 0;
79
+ mblen++;
80
+ }
81
+
82
+ src++;
83
+ }
84
+
85
+ // remove trailing spaces and underscores
86
+ while(d > dst && !IS_UTF8_BYTE(*(d - 1)) && (*(d - 1) == '_' || *(d - 1) == ' ')) {
87
+ d--;
88
+ mblen--;
89
+ }
90
+
91
+ // put a termination at the end of what we copied
92
+ *d = '\0';
93
+
94
+ // check if dst is all underscores and empty it if it is
95
+ if(*dst == '_') {
96
+ unsigned char *t = dst;
97
+ while (*t == '_') t++;
98
+ if (unlikely(*t == '\0')) {
99
+ *dst = '\0';
100
+ mblen = 0;
101
+ }
102
+ }
103
+
104
+ // check if it is empty
105
+ if(unlikely(*dst == '\0')) {
106
+ strncpyz((char *)dst, empty, dst_size);
107
+ dst[dst_size - 1] = '\0';
108
+ mblen = strlen((char *)dst);
109
+ if(multibyte_length) *multibyte_length = mblen;
110
+ return mblen;
111
+ }
112
+
113
+ if(multibyte_length) *multibyte_length = mblen;
114
+
115
+ return d - dst;
116
+}
src/libnetdata/sanitizers/utf8-sanitizer.h
new
+10
@@ -0,0 +1,10 @@
1
+// SPDX-License-Identifier: GPL-3.0-or-later
2
+
3
+#ifndef NETDATA_UTF8_SANITIZER_H
4
+#define NETDATA_UTF8_SANITIZER_H
5
+
6
+#include "../libnetdata.h"
7
+
8
+size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, const unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length);
9
+
10
+#endif //NETDATA_UTF8_SANITIZER_H
src/libnetdata/simple_pattern/simple_pattern.c
+13
-10
@@ -78,18 +78,21 @@ SIMPLE_PATTERN *simple_pattern_create(const char *list, const char *separators,
78
79
if(unlikely(!list || !*list)) return root;
80
81
- char isseparator[256] = {
82
- [' '] = 1 // space
83
- , ['\t'] = 1 // tab
84
- , ['\r'] = 1 // carriage return
85
- , ['\n'] = 1 // new line
86
- , ['\f'] = 1 // form feed
87
- , ['\v'] = 1 // vertical tab
81
+ bool isseparator[256] = {
82
+ [' '] = true // space
83
+ , ['\t'] = true // tab
84
+ , ['\r'] = true // carriage return
85
+ , ['\n'] = true // new line
86
+ , ['\f'] = true // form feed
87
+ , ['\v'] = true // vertical tab
88
};
89
90
- if (unlikely(separators && *separators)) {
91
- memset(&isseparator[0], 0, sizeof(isseparator));
92
- while(*separators) isseparator[(unsigned char)*separators++] = 1;
90
+ if (unlikely(separators == SIMPLE_PATTERN_NO_SEPARATORS))
91
+ memset(isseparator, false, sizeof(isseparator));
92
+
93
+ else if (unlikely(separators && *separators)) {
94
+ memset(isseparator, false, sizeof(isseparator));
95
+ while(*separators) isseparator[(unsigned char)*separators++] = true;
96
}
97
98
char *buf = mallocz(strlen(list) + 1);
src/libnetdata/simple_pattern/simple_pattern.h
+2
@@ -21,6 +21,8 @@ typedef enum __attribute__ ((__packed__)) {
21
struct simple_pattern;
22
typedef struct simple_pattern SIMPLE_PATTERN;
23
24
+#define SIMPLE_PATTERN_NO_SEPARATORS (const char *)(0xFFFFFFFF)
25
+
26
// create a simple_pattern from the string given
27
// default_mode is used in cases where EXACT matches, without an asterisk,
28
// should be considered PREFIX matches.
src/libnetdata/string/utf8.h
-1
@@ -5,6 +5,5 @@
5
6
#define IS_UTF8_BYTE(x) ((uint8_t)(x) & (uint8_t)0x80)
7
#define IS_UTF8_STARTBYTE(x) (IS_UTF8_BYTE(x) && ((uint8_t)(x) & (uint8_t)0x40))
8
-#define IS_UTF8_CONTBYTE(x) (IS_UTF8_BYTE(x) && !IS_UTF8_STARTBYTE(x))
8
9
#endif /* NETDATA_STRING_UTF8_H */
src/streaming/compression.c
+1
-1
@@ -32,7 +32,7 @@ void rrdpush_parse_compression_order(struct receiver_state *rpt, const char *ord
32
char *s = strdupz(order);
33
34
char *words[COMPRESSION_ALGORITHM_MAX + 100] = { NULL };
35
- size_t num_words = quoted_strings_splitter_pluginsd(s, words, COMPRESSION_ALGORITHM_MAX + 100);
35
+ size_t num_words = quoted_strings_splitter_whitespace(s, words, COMPRESSION_ALGORITHM_MAX + 100);
36
size_t slot = 0;
37
STREAM_CAPABILITIES added = STREAM_CAP_NONE;
38
for(size_t i = 0; i < num_words && slot < COMPRESSION_ALGORITHM_MAX ;i++) {
src/streaming/sender_execute.c
+1
-1
@@ -184,7 +184,7 @@ void rrdpush_sender_execute_commands(struct sender_state *s) {
184
continue;
185
}
186
187
- s->line.num_words = quoted_strings_splitter_pluginsd(start, s->line.words, PLUGINSD_MAX_WORDS);
187
+ s->line.num_words = quoted_strings_splitter_whitespace(start, s->line.words, PLUGINSD_MAX_WORDS);
188
const char *command = get_word(s->line.words, s->line.num_words, 0);
189
190
if(command && strcmp(command, PLUGINSD_CALL_FUNCTION) == 0) {