@cryptotaxi247 / netdata-1 / commits / 8fc3b351a

Allow netdata plugins to expose functions for querying more information about specific charts (#13720)

* function renames and code cleanup in popen.c; no actual code changes * netdata popen() now opens both child process stdin and stdout and returns FILE * for both * pass both input and output to parser structures * updated rrdset to call custom functions * RRDSET FUNCTION leading calls for both sync and async operation * put RRDSET functions to a separate file * added format and timeout at function definition * support for synchronous (internal plugins) and asynchronous (external plugins and children) functions * /api/v1/function endpoint * functions are now attached to the host and there is a dictionary view per chart * functions implemented at plugins.d * remove the defer until keyword hook from plugins.d when it is done * stream sender implementation of functions * sanitization of all functions so that certain characters are only allowed * strictier sanitization * common max size * 1st working plugins.d example * always init inflight dictionary * properly destroy dictionaries to avoid parallel insertion of items * add more debugging on disconnection reasons * add more debugging on disconnection reasons again * streaming receiver respects newlines * dont use the same fp for both streaming receive and send * dont free dbengine memory with internal checks * make sender proceed in the buffer * added timing info and garbage collection at plugins.d * added info about routing nodes * added info about routing nodes with delay * added more info about delays * added more info about delays again * signal sending thread to wake up * streaming version labeling and commented code to support capabilities * added functions to /api/v1/data, /api/v1/charts, /api/v1/chart, /api/v1/info * redirect top output to stdout * address coverity findings * fix resource leaks of popen * log attempts to connect to individual destinations * better messages * properly parse destinations * try to find a function from the most matching to the least matching * log added streaming destinations * rotate destinations bypassing a node in the middle that does not accept our connection * break the loops properly * use typedef to define callbacks * capabilities negotiation during streaming * functions exposed upstream based on capabilities; compression disabled per node persisting reconnects; always try to connect with all capabilities * restore functionality to lookup functions * better logging of capabilities * remove old versions from capabilities when a newer version is there * fix formatting * optimization for plugins.d rrdlabels to avoid creating and destructing dictionaries all the time * delayed health initialization for rrddim and rrdset * cleanup health initialization * fix for popen() not returning the right value * add health worker jobs for initializing rrdset and rrddim * added content type support for functions; apps.plugin permanent function to display all the processes * fixes for functions parameters parsing in apps.plugin * fix for process matching in apps.plugiin * first working function for apps.plugin * Dashboard ACL is disabled for functions; Function errors are all in JSON format * apps.plugin function processes returns json table * use json_escape_string() to escape message * fix formatting * apps.plugin exposes all its metrics to function processes * fix json formatting when filtering out some rows * reopen the internal pipe of rrdpush in case of errors * misplaced statement * do not use buffer->len * support for GLOBAL functions (functions that are not linked to a chart * added /api/v1/functions endpoint; removed format from the FUNCTIONS api; * swagger documentation about the new api end points * added plugins.d documentation about functions * never re-close a file * remove uncessesary ifdef * fixed issues identified by codacy * fix for null label value * make edit-config copy-and-paste friendly * Revert "make edit-config copy-and-paste friendly" This reverts commit 54500c0e0a97f65a0c66c4d34e966f6a9056698e. * reworked sender handshake to fix coverity findings * timeout is zero, for both send_timeout() and recv_timeout() * properly detect that parent closed the socket * support caching of function responses; limit function response to 10MB; added protection from malformed function responses * disabled excessive logging * added units to apps.plugin function processes and normalized all values to be human readable * shorter field names * fixed issues reported * fixed apps.plugin error response; tested that pluginsd can properly handle faulty responses * use double linked list macros for double linked list management * faster apps.plugin function printing by minimizing file operations * added memory percentage * fix compatibility issues with older compilers and FreeBSD * rrdpush sender code cleanup; rrhost structure cleanup from sender flags and variables; * fix letftover variable in ifdef * apps.plugin: do not call detach from the thread; exit immediately when input is broken * exclude AR charts from health * flush cleaner; prefer sender output * clarity * do not fill the cbuffer if not connected * fix * dont enabled host->sender if streaming is not enabled; send host label updates to parent; * functions are only available through ACLK * Prepared statement reports only in dev mode * fix AR chart detection * fix for streaming not being enabling itself * more cleanup of sender and receiver structures * moved read-only flags and configuration options to rrdhost->options * fixed merge with master * fix for incomplete rename * prevent service thread from working on charts that are being collected Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>

Costa Tsaousis committed Oct 5, 2022 at 14:13 UTC 8fc3b351a2e7fc96eced8f924de2e9cec9842128
67 files changed +4373 -1604
CMakeLists.txt
+2
@@ -731,6 +731,8 @@ set(RRD_PLUGIN_FILES
731 database/rrddimvar.c
732 database/rrddimvar.h
733 database/rrdfamily.c
734 + database/rrdfunctions.c
735 + database/rrdfunctions.h
736 database/rrdhost.c
737 database/rrdlabels.c
738 database/rrd.c
Makefile.am
+2
@@ -441,6 +441,8 @@ RRD_PLUGIN_FILES = \
441 database/rrd.c \
442 database/rrd.h \
443 database/rrdset.c \
444 + database/rrdfunctions.c \
445 + database/rrdfunctions.h \
446 database/rrdsetvar.c \
447 database/rrdsetvar.h \
448 database/rrdvar.c \
aclk/aclk_query.c
+1 -1
@@ -80,7 +80,7 @@ static int http_api_v2(struct aclk_query_thread *query_thr, aclk_query_t query)
80 strcpy(w->origin, "*"); // Simulate web_client_create_on_fd()
81 w->cookie1[0] = 0; // Simulate web_client_create_on_fd()
82 w->cookie2[0] = 0; // Simulate web_client_create_on_fd()
83 - w->acl = 0x1f;
83 + w->acl = WEB_CLIENT_ACL_ACLK;
84
85 buffer_strcat(log_buffer, query->data.http_api_v2.query);
86 size_t size = 0;
claim/claim.c
+5 -5
@@ -58,7 +58,7 @@ void claim_agent(char *claiming_arguments)
58 int exit_code;
59 pid_t command_pid;
60 char command_buffer[CLAIMING_COMMAND_LENGTH + 1];
61 - FILE *fp;
61 + FILE *fp_child_output, *fp_child_input;
62
63 // This is guaranteed to be set early in main via post_conf_load()
64 char *cloud_base_url = appconfig_get(&cloud_config, CONFIG_SECTION_GLOBAL, "cloud base url", NULL);
@@ -84,14 +84,14 @@ void claim_agent(char *claiming_arguments)
84 claiming_arguments);
85
86 info("Executing agent claiming command 'netdata-claim.sh'");
87 - fp = mypopen(command_buffer, &command_pid);
88 - if(!fp) {
87 + fp_child_output = netdata_popen(command_buffer, &command_pid, &fp_child_input);
88 + if(!fp_child_output) {
89 error("Cannot popen(\"%s\").", command_buffer);
90 return;
91 }
92 info("Waiting for claiming command to finish.");
93 - while (fgets(command_buffer, CLAIMING_COMMAND_LENGTH, fp) != NULL) {;}
94 - exit_code = mypclose(fp, command_pid);
93 + while (fgets(command_buffer, CLAIMING_COMMAND_LENGTH, fp_child_output) != NULL) {;}
94 + exit_code = netdata_pclose(fp_child_input, fp_child_output, command_pid);
95 info("Agent claiming command returned with code %d", exit_code);
96 if (0 == exit_code) {
97 load_claiming_state();
collectors/apps.plugin/apps_plugin.c
+737 -94
@@ -10,6 +10,11 @@
10 #include "libnetdata/libnetdata.h"
11 #include "libnetdata/required_dummies.h"
12
13 +#define APPS_PLUGIN_FUNCTIONS() do { \
14 + fprintf(stdout, PLUGINSD_KEYWORD_FUNCTION " \"processes\" 10 \"Detailed information on the currently running processes on this node\"\n"); \
15 + } while(0)
16 +
17 +
18 // ----------------------------------------------------------------------------
19 // debugging
20
@@ -191,6 +196,18 @@ struct pid_on_target {
196 struct pid_on_target *next;
197 };
198
199 +struct openfds {
200 + kernel_uint_t files;
201 + kernel_uint_t pipes;
202 + kernel_uint_t sockets;
203 + kernel_uint_t inotifies;
204 + kernel_uint_t eventfds;
205 + kernel_uint_t timerfds;
206 + kernel_uint_t signalfds;
207 + kernel_uint_t eventpolls;
208 + kernel_uint_t other;
209 +};
210 +
211 // ----------------------------------------------------------------------------
212 // target
213 //
@@ -235,24 +252,16 @@ struct target {
252
253 kernel_uint_t io_logical_bytes_read;
254 kernel_uint_t io_logical_bytes_written;
238 - // kernel_uint_t io_read_calls;
239 - // kernel_uint_t io_write_calls;
255 + kernel_uint_t io_read_calls;
256 + kernel_uint_t io_write_calls;
257 kernel_uint_t io_storage_bytes_read;
258 kernel_uint_t io_storage_bytes_written;
242 - // kernel_uint_t io_cancelled_write_bytes;
259 + kernel_uint_t io_cancelled_write_bytes;
260
261 int *target_fds;
262 int target_fds_size;
263
247 - kernel_uint_t openfiles;
248 - kernel_uint_t openpipes;
249 - kernel_uint_t opensockets;
250 - kernel_uint_t openinotifies;
251 - kernel_uint_t openeventfds;
252 - kernel_uint_t opentimerfds;
253 - kernel_uint_t opensignalfds;
254 - kernel_uint_t openeventpolls;
255 - kernel_uint_t openother;
264 + struct openfds openfds;
265
266 kernel_uint_t starttime;
267 kernel_uint_t collected_starttime;
@@ -382,22 +391,24 @@ struct pid_stat {
391
392 kernel_uint_t io_logical_bytes_read_raw;
393 kernel_uint_t io_logical_bytes_written_raw;
385 - // kernel_uint_t io_read_calls_raw;
386 - // kernel_uint_t io_write_calls_raw;
394 + kernel_uint_t io_read_calls_raw;
395 + kernel_uint_t io_write_calls_raw;
396 kernel_uint_t io_storage_bytes_read_raw;
397 kernel_uint_t io_storage_bytes_written_raw;
389 - // kernel_uint_t io_cancelled_write_bytes_raw;
398 + kernel_uint_t io_cancelled_write_bytes_raw;
399
400 kernel_uint_t io_logical_bytes_read;
401 kernel_uint_t io_logical_bytes_written;
393 - // kernel_uint_t io_read_calls;
394 - // kernel_uint_t io_write_calls;
402 + kernel_uint_t io_read_calls;
403 + kernel_uint_t io_write_calls;
404 kernel_uint_t io_storage_bytes_read;
405 kernel_uint_t io_storage_bytes_written;
397 - // kernel_uint_t io_cancelled_write_bytes;
406 + kernel_uint_t io_cancelled_write_bytes;
407
408 struct pid_fd *fds; // array of fds it uses
400 - size_t fds_size; // the size of the fds array
409 + size_t fds_size; // the size of the fds array
410 +
411 + struct openfds openfds;
412
413 int children_count; // number of processes directly referencing this
414 unsigned char keep:1; // 1 when we need to keep this process in memory even after it exited
@@ -447,8 +458,8 @@ kernel_uint_t global_uptime;
458
459 static struct pid_stat
460 *root_of_pids = NULL, // global list of all processes running
450 - **all_pids = NULL; // to avoid allocations, we pre-allocate the
451 - // the entire pid space.
461 + **all_pids = NULL; // to avoid allocations, we pre-allocate
462 + // a pointer for each pid in the entire pid space.
463
464 static size_t
465 all_pids_count = 0; // the number of processes running
@@ -961,15 +972,10 @@ static inline struct pid_stat *get_pid_entry(pid_t pid) {
972 p->fds = mallocz(sizeof(struct pid_fd) * MAX_SPARE_FDS);
973 p->fds_size = MAX_SPARE_FDS;
974 init_pid_fds(p, 0, p->fds_size);
964 -
965 - if(likely(root_of_pids))
966 - root_of_pids->prev = p;
967 -
968 - p->next = root_of_pids;
969 - root_of_pids = p;
970 -
975 p->pid = pid;
976
977 + DOUBLE_LINKED_LIST_APPEND_UNSAFE(root_of_pids, p, prev, next);
978 +
979 all_pids[pid] = p;
980 all_pids_count++;
981
@@ -986,11 +992,7 @@ static inline void del_pid_entry(pid_t pid) {
992
993 debug_log("process %d %s exited, deleting it.", pid, p->comm);
994
989 - if(root_of_pids == p)
990 - root_of_pids = p->next;
991 -
992 - if(p->next) p->next->prev = p->prev;
993 - if(p->prev) p->prev->next = p->next;
995 + DOUBLE_LINKED_LIST_REMOVE_UNSAFE(root_of_pids, p, prev, next);
996
997 // free the filename
998 #ifndef __FreeBSD__
@@ -1554,21 +1556,21 @@ static inline int read_proc_pid_io(struct pid_stat *p, void *ptr) {
1556 #else
1557 pid_incremental_rate(io, p->io_logical_bytes_read, str2kernel_uint_t(procfile_lineword(ff, 0, 1)));
1558 pid_incremental_rate(io, p->io_logical_bytes_written, str2kernel_uint_t(procfile_lineword(ff, 1, 1)));
1557 - // pid_incremental_rate(io, p->io_read_calls, str2kernel_uint_t(procfile_lineword(ff, 2, 1)));
1558 - // pid_incremental_rate(io, p->io_write_calls, str2kernel_uint_t(procfile_lineword(ff, 3, 1)));
1559 + pid_incremental_rate(io, p->io_read_calls, str2kernel_uint_t(procfile_lineword(ff, 2, 1)));
1560 + pid_incremental_rate(io, p->io_write_calls, str2kernel_uint_t(procfile_lineword(ff, 3, 1)));
1561 pid_incremental_rate(io, p->io_storage_bytes_read, str2kernel_uint_t(procfile_lineword(ff, 4, 1)));
1562 pid_incremental_rate(io, p->io_storage_bytes_written, str2kernel_uint_t(procfile_lineword(ff, 5, 1)));
1561 - // pid_incremental_rate(io, p->io_cancelled_write_bytes, str2kernel_uint_t(procfile_lineword(ff, 6, 1)));
1563 + pid_incremental_rate(io, p->io_cancelled_write_bytes, str2kernel_uint_t(procfile_lineword(ff, 6, 1)));
1564 #endif
1565
1566 if(unlikely(global_iterations_counter == 1)) {
1567 p->io_logical_bytes_read = 0;
1568 p->io_logical_bytes_written = 0;
1567 - // p->io_read_calls = 0;
1568 - // p->io_write_calls = 0;
1569 + p->io_read_calls = 0;
1570 + p->io_write_calls = 0;
1571 p->io_storage_bytes_read = 0;
1572 p->io_storage_bytes_written = 0;
1571 - // p->io_cancelled_write_bytes = 0;
1573 + p->io_cancelled_write_bytes = 0;
1574 }
1575
1576 return 1;
@@ -1577,11 +1579,11 @@ static inline int read_proc_pid_io(struct pid_stat *p, void *ptr) {
1579 cleanup:
1580 p->io_logical_bytes_read = 0;
1581 p->io_logical_bytes_written = 0;
1580 - // p->io_read_calls = 0;
1581 - // p->io_write_calls = 0;
1582 + p->io_read_calls = 0;
1583 + p->io_write_calls = 0;
1584 p->io_storage_bytes_read = 0;
1585 p->io_storage_bytes_written = 0;
1584 - // p->io_cancelled_write_bytes = 0;
1586 + p->io_cancelled_write_bytes = 0;
1587 return 0;
1588 #endif
1589 }
@@ -1888,7 +1890,7 @@ static inline int file_descriptor_find_or_add(const char *name, uint32_t hash) {
1890 else if(likely(strncmp(name, "anon_inode:", 11) == 0)) {
1891 const char *t = &name[11];
1892
1891 - if(strcmp(t, "inotify") == 0) type = FILETYPE_INOTIFY;
1893 + if(strcmp(t, "inotify") == 0) type = FILETYPE_INOTIFY;
1894 else if(strcmp(t, "[eventfd]") == 0) type = FILETYPE_EVENTFD;
1895 else if(strcmp(t, "[eventpoll]") == 0) type = FILETYPE_EVENTPOLL;
1896 else if(strcmp(t, "[timerfd]") == 0) type = FILETYPE_TIMERFD;
@@ -1943,7 +1945,6 @@ static inline void cleanup_negative_pid_fds(struct pid_stat *p) {
1945
1946 static inline void init_pid_fds(struct pid_stat *p, size_t first, size_t size) {
1947 struct pid_fd *pfd = &p->fds[first], *pfdend = &p->fds[first + size];
1946 - size_t i = first;
1948
1949 while(pfd < pfdend) {
1950 #ifndef __FreeBSD__
@@ -1951,7 +1952,6 @@ static inline void init_pid_fds(struct pid_stat *p, size_t first, size_t size) {
1952 #endif
1953 clear_pid_fd(pfd);
1954 pfd++;
1954 - i++;
1955 }
1956 }
1957
@@ -2904,24 +2904,24 @@ static size_t zero_all_targets(struct target *root) {
2904
2905 w->io_logical_bytes_read = 0;
2906 w->io_logical_bytes_written = 0;
2907 - // w->io_read_calls = 0;
2908 - // w->io_write_calls = 0;
2907 + w->io_read_calls = 0;
2908 + w->io_write_calls = 0;
2909 w->io_storage_bytes_read = 0;
2910 w->io_storage_bytes_written = 0;
2911 - // w->io_cancelled_write_bytes = 0;
2911 + w->io_cancelled_write_bytes = 0;
2912
2913 // zero file counters
2914 if(w->target_fds) {
2915 memset(w->target_fds, 0, sizeof(int) * w->target_fds_size);
2916 - w->openfiles = 0;
2917 - w->openpipes = 0;
2918 - w->opensockets = 0;
2919 - w->openinotifies = 0;
2920 - w->openeventfds = 0;
2921 - w->opentimerfds = 0;
2922 - w->opensignalfds = 0;
2923 - w->openeventpolls = 0;
2924 - w->openother = 0;
2916 + w->openfds.files = 0;
2917 + w->openfds.pipes = 0;
2918 + w->openfds.sockets = 0;
2919 + w->openfds.inotifies = 0;
2920 + w->openfds.eventfds = 0;
2921 + w->openfds.timerfds = 0;
2922 + w->openfds.signalfds = 0;
2923 + w->openfds.eventpolls = 0;
2924 + w->openfds.other = 0;
2925 }
2926
2927 w->collected_starttime = 0;
@@ -2956,60 +2956,64 @@ static inline void reallocate_target_fds(struct target *w) {
2956 }
2957 }
2958
2959 -static inline void aggregate_fd_on_target(int fd, struct target *w) {
2960 - if(unlikely(!w))
2961 - return;
2962 -
2963 - if(unlikely(w->target_fds[fd])) {
2964 - // it is already aggregated
2965 - // just increase its usage counter
2966 - w->target_fds[fd]++;
2967 - return;
2968 - }
2969 -
2970 - // increase its usage counter
2971 - // so that we will not add it again
2972 - w->target_fds[fd]++;
2973 -
2974 - switch(all_files[fd].type) {
2959 +static void aggregage_fd_type_on_openfds(FD_FILETYPE type, struct openfds *openfds) {
2960 + switch(type) {
2961 case FILETYPE_FILE:
2976 - w->openfiles++;
2962 + openfds->files++;
2963 break;
2964
2965 case FILETYPE_PIPE:
2980 - w->openpipes++;
2966 + openfds->pipes++;
2967 break;
2968
2969 case FILETYPE_SOCKET:
2984 - w->opensockets++;
2970 + openfds->sockets++;
2971 break;
2972
2973 case FILETYPE_INOTIFY:
2988 - w->openinotifies++;
2974 + openfds->inotifies++;
2975 break;
2976
2977 case FILETYPE_EVENTFD:
2992 - w->openeventfds++;
2978 + openfds->eventfds++;
2979 break;
2980
2981 case FILETYPE_TIMERFD:
2996 - w->opentimerfds++;
2982 + openfds->timerfds++;
2983 break;
2984
2985 case FILETYPE_SIGNALFD:
3000 - w->opensignalfds++;
2986 + openfds->signalfds++;
2987 break;
2988
2989 case FILETYPE_EVENTPOLL:
3004 - w->openeventpolls++;
2990 + openfds->eventpolls++;
2991 break;
2992
2993 case FILETYPE_OTHER:
3008 - w->openother++;
2994 + openfds->other++;
2995 break;
2996 }
2997 }
2998
2999 +static inline void aggregate_fd_on_target(int fd, struct target *w) {
3000 + if(unlikely(!w))
3001 + return;
3002 +
3003 + if(unlikely(w->target_fds[fd])) {
3004 + // it is already aggregated
3005 + // just increase its usage counter
3006 + w->target_fds[fd]++;
3007 + return;
3008 + }
3009 +
3010 + // increase its usage counter
3011 + // so that we will not add it again
3012 + w->target_fds[fd]++;
3013 +
3014 + aggregage_fd_type_on_openfds(all_files[fd].type, &w->openfds);
3015 +}
3016 +
3017 static inline void aggregate_pid_fds_on_targets(struct pid_stat *p) {
3018
3019 if(unlikely(!p->updated)) {
@@ -3023,6 +3027,16 @@ static inline void aggregate_pid_fds_on_targets(struct pid_stat *p) {
3027 reallocate_target_fds(u);
3028 reallocate_target_fds(g);
3029
3030 + p->openfds.files = 0;
3031 + p->openfds.pipes = 0;
3032 + p->openfds.sockets = 0;
3033 + p->openfds.inotifies = 0;
3034 + p->openfds.eventfds = 0;
3035 + p->openfds.timerfds = 0;
3036 + p->openfds.signalfds = 0;
3037 + p->openfds.eventpolls = 0;
3038 + p->openfds.other = 0;
3039 +
3040 long currentfds = 0;
3041 size_t c, size = p->fds_size;
3042 struct pid_fd *fds = p->fds;
@@ -3033,6 +3047,7 @@ static inline void aggregate_pid_fds_on_targets(struct pid_stat *p) {
3047 continue;
3048
3049 currentfds++;
3050 + aggregage_fd_type_on_openfds(all_files[fd].type, &p->openfds);
3051
3052 aggregate_fd_on_target(fd, w);
3053 aggregate_fd_on_target(fd, u);
@@ -3079,11 +3094,11 @@ static inline void aggregate_pid_on_target(struct target *w, struct pid_stat *p,
3094
3095 w->io_logical_bytes_read += p->io_logical_bytes_read;
3096 w->io_logical_bytes_written += p->io_logical_bytes_written;
3082 - // w->io_read_calls += p->io_read_calls;
3083 - // w->io_write_calls += p->io_write_calls;
3097 + w->io_read_calls += p->io_read_calls;
3098 + w->io_write_calls += p->io_write_calls;
3099 w->io_storage_bytes_read += p->io_storage_bytes_read;
3100 w->io_storage_bytes_written += p->io_storage_bytes_written;
3086 - // w->io_cancelled_write_bytes += p->io_cancelled_write_bytes;
3101 + w->io_cancelled_write_bytes += p->io_cancelled_write_bytes;
3102
3103 w->processes++;
3104 w->num_threads += p->num_threads;
@@ -3638,7 +3653,7 @@ static void send_collected_data_to_netdata(struct target *root, const char *type
3653 send_BEGIN(type, "files", dt);
3654 for (w = root; w; w = w->next) {
3655 if (unlikely(w->exposed && w->processes))
3641 - send_SET(w->name, w->openfiles);
3656 + send_SET(w->name, w->openfds.files);
3657 }
3658 if (!strcmp("apps", type)){
3659 kernel_uint_t usedfdpercentage = (kernel_uint_t) ((currentmaxfds * 100) / sysconf(_SC_OPEN_MAX));
@@ -3649,14 +3664,14 @@ static void send_collected_data_to_netdata(struct target *root, const char *type
3664 send_BEGIN(type, "sockets", dt);
3665 for (w = root; w; w = w->next) {
3666 if (unlikely(w->exposed && w->processes))
3652 - send_SET(w->name, w->opensockets);
3667 + send_SET(w->name, w->openfds.sockets);
3668 }
3669 send_END();
3670
3671 send_BEGIN(type, "pipes", dt);
3672 for (w = root; w; w = w->next) {
3673 if (unlikely(w->exposed && w->processes))
3659 - send_SET(w->name, w->openpipes);
3674 + send_SET(w->name, w->openfds.pipes);
3675 }
3676 send_END();
3677 }
@@ -3704,30 +3719,36 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3719 if(unlikely(w->exposed))
3720 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu %s\n", w->name, time_factor * RATES_DETAIL / 100, w->hidden ? "hidden" : "");
3721 }
3722 + APPS_PLUGIN_FUNCTIONS();
3723
3724 fprintf(stdout, "CHART %s.mem '' '%s Real Memory (w/o shared)' 'MiB' mem %s.mem stacked 20003 %d\n", type, title, type, update_every);
3725 for (w = root; w ; w = w->next) {
3726 if(unlikely(w->exposed))
3727 fprintf(stdout, "DIMENSION %s '' absolute %ld %ld\n", w->name, 1L, 1024L);
3728 }
3729 + APPS_PLUGIN_FUNCTIONS();
3730 +
3731
3732 fprintf(stdout, "CHART %s.vmem '' '%s Virtual Memory Size' 'MiB' mem %s.vmem stacked 20005 %d\n", type, title, type, update_every);
3733 for (w = root; w ; w = w->next) {
3734 if(unlikely(w->exposed))
3735 fprintf(stdout, "DIMENSION %s '' absolute %ld %ld\n", w->name, 1L, 1024L);
3736 }
3737 + APPS_PLUGIN_FUNCTIONS();
3738
3739 fprintf(stdout, "CHART %s.threads '' '%s Threads' 'threads' processes %s.threads stacked 20006 %d\n", type, title, type, update_every);
3740 for (w = root; w ; w = w->next) {
3741 if(unlikely(w->exposed))
3742 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3743 }
3744 + APPS_PLUGIN_FUNCTIONS();
3745
3746 fprintf(stdout, "CHART %s.processes '' '%s Processes' 'processes' processes %s.processes stacked 20007 %d\n", type, title, type, update_every);
3747 for (w = root; w ; w = w->next) {
3748 if(unlikely(w->exposed))
3749 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3750 }
3751 + APPS_PLUGIN_FUNCTIONS();
3752
3753 #ifndef __FreeBSD__
3754 fprintf(stdout, "CHART %s.uptime '' '%s Carried Over Uptime' 'seconds' processes %s.uptime line 20008 %d\n", type, title, type, update_every);
@@ -3735,6 +3756,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3756 if(unlikely(w->exposed))
3757 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3758 }
3759 + APPS_PLUGIN_FUNCTIONS();
3760
3761 if (enable_detailed_uptime_charts) {
3762 fprintf(stdout, "CHART %s.uptime_min '' '%s Minimum Uptime' 'seconds' processes %s.uptime_min line 20009 %d\n", type, title, type, update_every);
@@ -3742,18 +3764,21 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3764 if(unlikely(w->exposed))
3765 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3766 }
3767 + APPS_PLUGIN_FUNCTIONS();
3768
3769 fprintf(stdout, "CHART %s.uptime_avg '' '%s Average Uptime' 'seconds' processes %s.uptime_avg line 20010 %d\n", type, title, type, update_every);
3770 for (w = root; w ; w = w->next) {
3771 if(unlikely(w->exposed))
3772 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3773 }
3774 + APPS_PLUGIN_FUNCTIONS();
3775
3776 fprintf(stdout, "CHART %s.uptime_max '' '%s Maximum Uptime' 'seconds' processes %s.uptime_max line 20011 %d\n", type, title, type, update_every);
3777 for (w = root; w ; w = w->next) {
3778 if(unlikely(w->exposed))
3779 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3780 }
3781 + APPS_PLUGIN_FUNCTIONS();
3782 }
3783 #endif
3784
@@ -3762,12 +3787,14 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3787 if(unlikely(w->exposed))
3788 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, time_factor * RATES_DETAIL / 100LLU);
3789 }
3790 + APPS_PLUGIN_FUNCTIONS();
3791
3792 fprintf(stdout, "CHART %s.cpu_system '' '%s CPU System Time (100%% = 1 core)' 'percentage' cpu %s.cpu_system stacked 20021 %d\n", type, title, type, update_every);
3793 for (w = root; w ; w = w->next) {
3794 if(unlikely(w->exposed))
3795 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, time_factor * RATES_DETAIL / 100LLU);
3796 }
3797 + APPS_PLUGIN_FUNCTIONS();
3798
3799 if(show_guest_time) {
3800 fprintf(stdout, "CHART %s.cpu_guest '' '%s CPU Guest Time (100%% = 1 core)' 'percentage' cpu %s.cpu_guest stacked 20022 %d\n", type, title, type, update_every);
@@ -3775,6 +3802,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3802 if(unlikely(w->exposed))
3803 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, time_factor * RATES_DETAIL / 100LLU);
3804 }
3805 + APPS_PLUGIN_FUNCTIONS();
3806 }
3807
3808 #ifndef __FreeBSD__
@@ -3783,6 +3811,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3811 if(unlikely(w->exposed))
3812 fprintf(stdout, "DIMENSION %s '' absolute %ld %ld\n", w->name, 1L, 1024L);
3813 }
3814 + APPS_PLUGIN_FUNCTIONS();
3815 #endif
3816
3817 fprintf(stdout, "CHART %s.major_faults '' '%s Major Page Faults (swap read)' 'page faults/s' swap %s.major_faults stacked 20012 %d\n", type, title, type, update_every);
@@ -3790,12 +3819,14 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3819 if(unlikely(w->exposed))
3820 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, RATES_DETAIL);
3821 }
3822 + APPS_PLUGIN_FUNCTIONS();
3823
3824 fprintf(stdout, "CHART %s.minor_faults '' '%s Minor Page Faults' 'page faults/s' mem %s.minor_faults stacked 20011 %d\n", type, title, type, update_every);
3825 for (w = root; w ; w = w->next) {
3826 if(unlikely(w->exposed))
3827 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, RATES_DETAIL);
3828 }
3829 + APPS_PLUGIN_FUNCTIONS();
3830
3831 #ifdef __FreeBSD__
3832 fprintf(stdout, "CHART %s.preads '' '%s Disk Reads' 'blocks/s' disk %s.preads stacked 20002 %d\n", type, title, type, update_every);
@@ -3803,36 +3834,42 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3834 if(unlikely(w->exposed))
3835 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, RATES_DETAIL);
3836 }
3837 + APPS_PLUGIN_FUNCTIONS();
3838
3839 fprintf(stdout, "CHART %s.pwrites '' '%s Disk Writes' 'blocks/s' disk %s.pwrites stacked 20002 %d\n", type, title, type, update_every);
3840 for (w = root; w ; w = w->next) {
3841 if(unlikely(w->exposed))
3842 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, RATES_DETAIL);
3843 }
3844 + APPS_PLUGIN_FUNCTIONS();
3845 #else
3846 fprintf(stdout, "CHART %s.preads '' '%s Disk Reads' 'KiB/s' disk %s.preads stacked 20002 %d\n", type, title, type, update_every);
3847 for (w = root; w ; w = w->next) {
3848 if(unlikely(w->exposed))
3849 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, 1024LLU * RATES_DETAIL);
3850 }
3851 + APPS_PLUGIN_FUNCTIONS();
3852
3853 fprintf(stdout, "CHART %s.pwrites '' '%s Disk Writes' 'KiB/s' disk %s.pwrites stacked 20002 %d\n", type, title, type, update_every);
3854 for (w = root; w ; w = w->next) {
3855 if(unlikely(w->exposed))
3856 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, 1024LLU * RATES_DETAIL);
3857 }
3858 + APPS_PLUGIN_FUNCTIONS();
3859
3860 fprintf(stdout, "CHART %s.lreads '' '%s Disk Logical Reads' 'KiB/s' disk %s.lreads stacked 20042 %d\n", type, title, type, update_every);
3861 for (w = root; w ; w = w->next) {
3862 if(unlikely(w->exposed))
3863 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, 1024LLU * RATES_DETAIL);
3864 }
3865 + APPS_PLUGIN_FUNCTIONS();
3866
3867 fprintf(stdout, "CHART %s.lwrites '' '%s I/O Logical Writes' 'KiB/s' disk %s.lwrites stacked 20042 %d\n", type, title, type, update_every);
3868 for (w = root; w ; w = w->next) {
3869 if(unlikely(w->exposed))
3870 fprintf(stdout, "DIMENSION %s '' absolute 1 %llu\n", w->name, 1024LLU * RATES_DETAIL);
3871 }
3872 + APPS_PLUGIN_FUNCTIONS();
3873 #endif
3874
3875 if(enable_file_charts) {
@@ -3842,6 +3879,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3879 if (unlikely(w->exposed))
3880 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3881 }
3882 + APPS_PLUGIN_FUNCTIONS();
3883
3884 fprintf(stdout, "CHART %s.sockets '' '%s Open Sockets' 'open sockets' net %s.sockets stacked 20051 %d\n",
3885 type, title, type, update_every);
@@ -3849,6 +3887,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3887 if (unlikely(w->exposed))
3888 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3889 }
3890 + APPS_PLUGIN_FUNCTIONS();
3891
3892 fprintf(stdout, "CHART %s.pipes '' '%s Pipes' 'open pipes' processes %s.pipes stacked 20053 %d\n", type,
3893 title, type, update_every);
@@ -3856,6 +3895,7 @@ static void send_charts_updates_to_netdata(struct target *root, const char *type
3895 if (unlikely(w->exposed))
3896 fprintf(stdout, "DIMENSION %s '' absolute 1 1\n", w->name);
3897 }
3898 + APPS_PLUGIN_FUNCTIONS();
3899 }
3900 }
3901
@@ -4134,6 +4174,592 @@ static int check_capabilities() {
4174 }
4175 #endif
4176
4177 +netdata_mutex_t mutex = NETDATA_MUTEX_INITIALIZER;
4178 +
4179 +#define PROCESS_FILTER_CATEGORY "category:"
4180 +#define PROCESS_FILTER_USER "user:"
4181 +#define PROCESS_FILTER_GROUP "group:"
4182 +#define PROCESS_FILTER_PROCESS "process:"
4183 +#define PROCESS_FILTER_PID "pid:"
4184 +#define PROCESS_FILTER_UID "uid:"
4185 +#define PROCESS_FILTER_GID "gid:"
4186 +
4187 +static struct target *find_target_by_name(struct target *base, const char *name) {
4188 + struct target *t;
4189 + for(t = base; t ; t = t->next) {
4190 + if (strcmp(t->name, name) == 0)
4191 + return t;
4192 + }
4193 +
4194 + return NULL;
4195 +}
4196 +
4197 +static kernel_uint_t MemTotal = 0;
4198 +
4199 +static void get_MemTotal(void) {
4200 +#ifdef __FreeBSD__
4201 + // TODO - fix this for FreeBSD
4202 + return;
4203 +#else
4204 + char filename[FILENAME_MAX + 1];
4205 + snprintfz(filename, FILENAME_MAX, "%s/proc/meminfo", netdata_configured_host_prefix);
4206 +
4207 + procfile *ff = procfile_open(filename, ": \t", PROCFILE_FLAG_DEFAULT);
4208 + if(!ff)
4209 + return;
4210 +
4211 + ff = procfile_readall(ff);
4212 + if(!ff)
4213 + return;
4214 +
4215 + size_t line, lines = procfile_lines(ff);
4216 +
4217 + for(line = 0; line < lines ;line++) {
4218 + size_t words = procfile_linewords(ff, line);
4219 + if(words == 3 && strcmp(procfile_lineword(ff, line, 0), "MemTotal") == 0 && strcmp(procfile_lineword(ff, line, 2), "kB") == 0) {
4220 + kernel_uint_t n = str2ull(procfile_lineword(ff, line, 1));
4221 + if(n) MemTotal = n;
4222 + break;
4223 + }
4224 + }
4225 +
4226 + procfile_close(ff);
4227 +#endif
4228 +}
4229 +
4230 +static void apps_plugin_function_error(const char *transaction, int code, const char *msg) {
4231 + char buffer[PLUGINSD_LINE_MAX + 1];
4232 + json_escape_string(buffer, msg, PLUGINSD_LINE_MAX);
4233 +
4234 + pluginsd_function_result_begin_to_stdout(transaction, code, "application/json", now_realtime_sec());
4235 + fprintf(stdout, "{\"status\":%d,\"error_message\":\"%s\"}", code, buffer);
4236 + pluginsd_function_result_end_to_stdout();
4237 +}
4238 +
4239 +static void apps_plugin_function_processes_help(const char *transaction) {
4240 + pluginsd_function_result_begin_to_stdout(transaction, HTTP_RESP_OK, "text/plain", now_realtime_sec() + 3600);
4241 + fprintf(stdout, "%s",
4242 + "apps.plugin / processes\n"
4243 + "\n"
4244 + "Function `processes` presents all the currently running processes of the system.\n"
4245 + "\n"
4246 + "The following filters are supported:\n"
4247 + "\n"
4248 + " category:NAME\n"
4249 + " Shows only processes that are assigned the category `NAME` in apps_groups.conf\n"
4250 + "\n"
4251 + " user:NAME\n"
4252 + " Shows only processes that are running as user name `NAME`.\n"
4253 + "\n"
4254 + " group:NAME\n"
4255 + " Shows only processes that are running as group name `NAME`.\n"
4256 + "\n"
4257 + " process:NAME\n"
4258 + " Shows only processes that their Command is `NAME` or their parent's Command is `NAME`.\n"
4259 + "\n"
4260 + " pid:NUMBER\n"
4261 + " Shows only processes that their PID is `NUMBER` or their parent's PID is `NUMBER`\n"
4262 + "\n"
4263 + " uid:NUMBER\n"
4264 + " Shows only processes that their UID is `NUMBER`\n"
4265 + "\n"
4266 + " gid:NUMBER\n"
4267 + " Shows only processes that their GID is `NUMBER`\n"
4268 + "\n"
4269 + "Filters can be combined. Each filter can be given only one time.\n"
4270 + );
4271 + pluginsd_function_result_end_to_stdout();
4272 +}
4273 +
4274 +#define add_table_field(wb, key, name, visible, type, units, max, sort) do { \
4275 + if(fields_added) buffer_strcat(wb, ","); \
4276 + buffer_sprintf(wb, "\n \"%s\": {", key); \
4277 + buffer_sprintf(wb, "\n \"index\":%d,", fields_added); \
4278 + buffer_sprintf(wb, "\n \"name\":\"%s\",", name); \
4279 + buffer_sprintf(wb, "\n \"visible\":%s,", (visible)?"true":"false"); \
4280 + buffer_sprintf(wb, "\n \"type\":\"%s\",", type); \
4281 + if(units) \
4282 + buffer_sprintf(wb, "\n \"units\":\"%s\",", (char*)(units)); \
4283 + if(!isnan(max)) \
4284 + buffer_sprintf(wb, "\n \"max\":\"%f\",", (NETDATA_DOUBLE)(max)); \
4285 + buffer_sprintf(wb, "\n \"sort\":\"%s\"", sort); \
4286 + buffer_sprintf(wb, "\n }"); \
4287 + fields_added++; \
4288 + } while(0)
4289 +
4290 +static BUFFER *func_processes_fields = NULL;
4291 +
4292 +static void apps_plugin_function_processes(const char *transaction, char *function __maybe_unused, char *line_buffer __maybe_unused, int line_max __maybe_unused, int timeout __maybe_unused) {
4293 + struct pid_stat *p;
4294 +
4295 + char *words[PLUGINSD_MAX_WORDS] = { NULL };
4296 + pluginsd_split_words(function, words, PLUGINSD_MAX_WORDS, NULL, NULL, 0);
4297 +
4298 + struct target *category = NULL, *user = NULL, *group = NULL;
4299 + const char *process_name = NULL;
4300 + pid_t pid = 0;
4301 + uid_t uid = 0;
4302 + gid_t gid = 0;
4303 +
4304 + bool filter_pid = false, filter_uid = false, filter_gid = false;
4305 +
4306 + for(int i = 1; i < PLUGINSD_MAX_WORDS ;i++) {
4307 + if(!words[i]) break;
4308 +
4309 + if(!category && strncmp(words[i], PROCESS_FILTER_CATEGORY, strlen(PROCESS_FILTER_CATEGORY)) == 0) {
4310 + category = find_target_by_name(apps_groups_root_target, &words[i][strlen(PROCESS_FILTER_CATEGORY)]);
4311 + if(!category) {
4312 + apps_plugin_function_error(transaction, HTTP_RESP_BAD_REQUEST, "No category with that name found.");
4313 + return;
4314 + }
4315 + }
4316 + else if(!user && strncmp(words[i], PROCESS_FILTER_USER, strlen(PROCESS_FILTER_USER)) == 0) {
4317 + user = find_target_by_name(users_root_target, &words[i][strlen(PROCESS_FILTER_USER)]);
4318 + if(!user) {
4319 + apps_plugin_function_error(transaction, HTTP_RESP_BAD_REQUEST, "No user with that name found.");
4320 + return;
4321 + }
4322 + }
4323 + else if(strncmp(words[i], PROCESS_FILTER_GROUP, strlen(PROCESS_FILTER_GROUP)) == 0) {
4324 + group = find_target_by_name(groups_root_target, &words[i][strlen(PROCESS_FILTER_GROUP)]);
4325 + if(!group) {
4326 + apps_plugin_function_error(transaction, HTTP_RESP_BAD_REQUEST, "No group with that name found.");
4327 + return;
4328 + }
4329 + }
4330 + else if(!process_name && strncmp(words[i], PROCESS_FILTER_PROCESS, strlen(PROCESS_FILTER_PROCESS)) == 0) {
4331 + process_name = &words[i][strlen(PROCESS_FILTER_PROCESS)];
4332 + }
4333 + else if(!pid && strncmp(words[i], PROCESS_FILTER_PID, strlen(PROCESS_FILTER_PID)) == 0) {
4334 + pid = str2i(&words[i][strlen(PROCESS_FILTER_PID)]);
4335 + filter_pid = true;
4336 + }
4337 + else if(!uid && strncmp(words[i], PROCESS_FILTER_UID, strlen(PROCESS_FILTER_UID)) == 0) {
4338 + uid = str2i(&words[i][strlen(PROCESS_FILTER_UID)]);
4339 + filter_uid = true;
4340 + }
4341 + else if(!gid && strncmp(words[i], PROCESS_FILTER_GID, strlen(PROCESS_FILTER_GID)) == 0) {
4342 + gid = str2i(&words[i][strlen(PROCESS_FILTER_GID)]);
4343 + filter_gid = true;
4344 + }
4345 + else if(strcmp(words[i], "help") == 0) {
4346 + apps_plugin_function_processes_help(transaction);
4347 + return;
4348 + }
4349 + else {
4350 + char msg[PLUGINSD_LINE_MAX];
4351 + snprintfz(msg, PLUGINSD_LINE_MAX, "Invalid parameter '%s'", words[i]);
4352 + apps_plugin_function_error(transaction, HTTP_RESP_BAD_REQUEST, msg);
4353 + return;
4354 + }
4355 + }
4356 +
4357 + time_t expires = now_realtime_sec() + update_every;
4358 + pluginsd_function_result_begin_to_stdout(transaction, HTTP_RESP_OK, "application/json", expires);
4359 +
4360 + if(unlikely(!func_processes_fields)) {
4361 + func_processes_fields = buffer_create(1000);
4362 +
4363 + int fields_added = 0;
4364 +
4365 + buffer_sprintf(func_processes_fields,
4366 + "{"
4367 + "\n \"status\":%d"
4368 + ",\n \"type\":\"table\""
4369 + ",\n \"update_every\":%d"
4370 + ",\n \"columns\": {"
4371 + , HTTP_RESP_OK
4372 + , update_every
4373 + );
4374 +
4375 + // IMPORTANT!
4376 + // THE ORDER SHOULD BE THE SAME WITH THE VALUES!
4377 +
4378 + add_table_field(func_processes_fields, "Pid", "Process ID", true, "integer", NULL, NAN, "ascending");
4379 + add_table_field(func_processes_fields, "Cmd", "Process Name", true, "string", NULL, NAN, "ascending");
4380 +
4381 +#ifdef NETDATA_DEV_MODE
4382 + add_table_field(func_processes_fields, "CmdLine", "Command Line", false, "detail-string:cmd", NULL, NAN, "ascending");
4383 +#endif
4384 + add_table_field(func_processes_fields, "PPid", "Parent Process ID", false, "integer", NULL, NAN, "ascending");
4385 + add_table_field(func_processes_fields, "Category", "Category (apps_groups.conf)", true, "string", NULL, NAN, "ascending");
4386 + add_table_field(func_processes_fields, "User", "User Owner", true, "string", NULL, NAN, "ascending");
4387 + add_table_field(func_processes_fields, "Uid", "User ID", false, "integer", NULL, NAN, "ascending");
4388 + add_table_field(func_processes_fields, "Group", "Group Owner", false, "string", NULL, NAN, "ascending");
4389 + add_table_field(func_processes_fields, "Gid", "Group ID", false, "integer", NULL, NAN, "ascending");
4390 + add_table_field(func_processes_fields, "Processes", "Processes", true, "bar-with-integer", "processes", NAN, "descending");
4391 + add_table_field(func_processes_fields, "Threads", "Threads", true, "bar-with-integer", "threads", NAN, "descending");
4392 + add_table_field(func_processes_fields, "Uptime", "Uptime in seconds", true, "duration", "seconds", NAN, "descending");
4393 +
4394 + // minor page faults
4395 + add_table_field(func_processes_fields, "MinFlt", "Minor Page Faults/s", false, "bar", "page faults/s", NAN, "descending");
4396 + add_table_field(func_processes_fields, "CMinFlt", "Children Minor Page Faults/s", false, "bar", "page faults/s", NAN, "descending");
4397 + add_table_field(func_processes_fields, "TMinFlt", "Total Minor Page Faults/s", false, "bar", "page faults/s", NAN, "descending");
4398 +
4399 + // major page faults
4400 + add_table_field(func_processes_fields, "MajFlt", "Major Page Faults/s", false, "bar", "page faults/s", NAN, "descending");
4401 + add_table_field(func_processes_fields, "CMajFlt", "Children Major Page Faults/s", false, "bar", "page faults/s", NAN, "descending");
4402 + add_table_field(func_processes_fields, "TMajFlt", "Total Major Page Faults/s", true, "bar", "page faults/s", NAN, "descending");
4403 +
4404 + // CPU utilization
4405 + add_table_field(func_processes_fields, "UserCPU", "User CPU time", false, "bar-only", "%", NAN, "descending");
4406 + add_table_field(func_processes_fields, "SysCPU", "System CPU Time", false, "bar-only", "%", NAN, "descending");
4407 + add_table_field(func_processes_fields, "GuestCPU", "Guest CPU Time", false, "bar-only", "%", NAN, "descending");
4408 + add_table_field(func_processes_fields, "CUserCPU", "Children User CPU Time", false, "bar-only", "%", NAN, "descending");
4409 + add_table_field(func_processes_fields, "CSysCPU", "Children System CPU Time", false, "bar-only", "%", NAN, "descending");
4410 + add_table_field(func_processes_fields, "CGuestCPU", "Children Guest CPU Time", false, "bar-only", "%", NAN, "descending");
4411 + add_table_field(func_processes_fields, "CPU", "Total CPU Time", true, "bar-only", "%", NAN, "descending");
4412 +
4413 + // memory
4414 + add_table_field(func_processes_fields, "VMSize", "Virtual Memory Size", false, "bar", "MiB", NAN, "descending");
4415 + add_table_field(func_processes_fields, "RSS", "Resident Set Size", MemTotal?false:true, "bar", "MiB", NAN, "descending");
4416 + add_table_field(func_processes_fields, "Shared", "Shared Pages", false, "bar", "MiB", NAN, "descending");
4417 + add_table_field(func_processes_fields, "Swap", "Swap Memory", false, "bar", "MiB", NAN, "descending");
4418 +
4419 + if(MemTotal)
4420 + add_table_field(func_processes_fields, "MemPcnt", "Memory Percentage", true, "bar", "%", 100.0, "descending");
4421 +
4422 + // Logical I/O
4423 +#ifndef __FreeBSD__
4424 + add_table_field(func_processes_fields, "LReads", "Logical I/O Reads", false, "bar", "KiB/s", NAN, "descending");
4425 + add_table_field(func_processes_fields, "LWrites", "Logical I/O Writes", false, "bar", "KiB/s", NAN, "descending");
4426 +#endif
4427 +
4428 + // Physical I/O
4429 + add_table_field(func_processes_fields, "PReads", "Physical I/O Reads", true, "bar", "KiB/s", NAN, "descending");
4430 + add_table_field(func_processes_fields, "PWrites", "Physical I/O Writes", true, "bar", "KiB/s", NAN, "descending");
4431 +
4432 + // I/O calls
4433 + add_table_field(func_processes_fields, "RCalls", "I/O Read Calls", false, "bar", "calls/s", NAN, "descending");
4434 + add_table_field(func_processes_fields, "WCalls", "I/O Write Calls", false, "bar", "calls/s", NAN, "descending");
4435 +
4436 + // open file descriptors
4437 + add_table_field(func_processes_fields, "Files", "Open Files", false, "bar", "generic", NAN, "descending");
4438 + add_table_field(func_processes_fields, "Pipes", "Open Pipes", false, "bar", "generic", NAN, "descending");
4439 + add_table_field(func_processes_fields, "Sockets", "Open Sockets", false, "bar", "generic", NAN, "descending");
4440 + add_table_field(func_processes_fields, "iNotiFDs", "Open iNotify Descriptors", false, "bar", "generic", NAN, "descending");
4441 + add_table_field(func_processes_fields, "EventFDs", "Open Event Descriptors", false, "bar", "generic", NAN, "descending");
4442 + add_table_field(func_processes_fields, "TimerFDs", "Open Timer Descriptors", false, "bar", "generic", NAN, "descending");
4443 + add_table_field(func_processes_fields, "SigFDs", "Open Signal Descriptors", false, "bar", "generic", NAN, "descending");
4444 + add_table_field(func_processes_fields, "EvPollFDs", "Open Event Poll Descriptors", false, "bar", "generic", NAN, "descending");
4445 + add_table_field(func_processes_fields, "OtherFDs", "Other Open Descriptors", false, "bar", "generic", NAN, "descending");
4446 + add_table_field(func_processes_fields, "FDs", "All Open File Descriptors", true, "bar", "generic", NAN, "descending");
4447 +
4448 + buffer_strcat(
4449 + func_processes_fields,
4450 + ""
4451 + "\n },"
4452 + "\n \"default sort column\": \"CPU\","
4453 + "\n \"charts\": {"
4454 + "\n \"CPU\": {"
4455 + "\n \"name\":\"CPU Utilization\","
4456 + "\n \"type\":\"stacked-bar\","
4457 + "\n \"columns\": [ \"UserCPU\", \"SysCPU\", \"GuestCPU\", \"CUserCPU\", \"CSysCPU\", \"CGuestCPU\" ]"
4458 + "\n },"
4459 + "\n \"Memory\": {"
4460 + "\n \"name\":\"Memory\","
4461 + "\n \"type\":\"stacked-bar\","
4462 + "\n \"columns\": [ \"VMSize\", \"RSS\", \"Shared\", \"Swap\" ]"
4463 + "\n },"
4464 + );
4465 +
4466 + if(MemTotal)
4467 + buffer_strcat(
4468 + func_processes_fields,
4469 + ""
4470 + "\n \"MemoryPercent\": {"
4471 + "\n \"name\":\"Memory Percentage\","
4472 + "\n \"type\":\"stacked-bar\","
4473 + "\n \"columns\": [ \"MemPcnt\" ]"
4474 + "\n },"
4475 + );
4476 +
4477 + buffer_strcat(
4478 + func_processes_fields, ""
4479 +#ifndef __FreeBSD__
4480 + "\n \"Reads\": {"
4481 + "\n \"name\":\"I/O Reads\","
4482 + "\n \"type\":\"stacked-bar\","
4483 + "\n \"columns\": [ \"LReads\", \"PReads\" ]"
4484 + "\n },"
4485 + "\n \"Writes\": {"
4486 + "\n \"name\":\"I/O Writes\","
4487 + "\n \"type\":\"stacked-bar\","
4488 + "\n \"columns\": [ \"LWrites\", \"PWrites\" ]"
4489 + "\n },"
4490 + "\n \"LogicalIO\": {"
4491 + "\n \"name\":\"Logical I/O\","
4492 + "\n \"type\":\"stacked-bar\","
4493 + "\n \"columns\": [ \"LReads\", \"LWrites\" ]"
4494 + "\n },"
4495 +#endif
4496 + "\n \"PhysicalIO\": {"
4497 + "\n \"name\":\"Physical I/O\","
4498 + "\n \"type\":\"stacked-bar\","
4499 + "\n \"columns\": [ \"PReads\", \"PWrites\" ]"
4500 + "\n },"
4501 + "\n \"IOCalls\": {"
4502 + "\n \"name\":\"I/O Calls\","
4503 + "\n \"type\":\"stacked-bar\","
4504 + "\n \"columns\": [ \"RCalls\", \"WCalls\" ]"
4505 + "\n },"
4506 + "\n \"MinFlt\": {"
4507 + "\n \"name\":\"Minor Page Faults\","
4508 + "\n \"type\":\"stacked-bar\","
4509 + "\n \"columns\": [ \"MinFlt\", \"CMinFlt\" ]"
4510 + "\n },"
4511 + "\n \"MajFlt\": {"
4512 + "\n \"name\":\"Major Page Faults\","
4513 + "\n \"type\":\"stacked-bar\","
4514 + "\n \"columns\": [ \"MajFlt\", \"CMajFlt\" ]"
4515 + "\n },"
4516 + "\n \"Threads\": {"
4517 + "\n \"name\":\"Threads\","
4518 + "\n \"type\":\"stacked-bar\","
4519 + "\n \"columns\": [ \"Threads\" ]"
4520 + "\n },"
4521 + "\n \"Processes\": {"
4522 + "\n \"name\":\"Processes\","
4523 + "\n \"type\":\"stacked-bar\","
4524 + "\n \"columns\": [ \"Processes\" ]"
4525 + "\n },"
4526 + "\n \"FDs\": {"
4527 + "\n \"name\":\"File Descriptors\","
4528 + "\n \"type\":\"stacked-bar\","
4529 + "\n \"columns\": [ \"Files\", \"Pipes\", \"Sockets\", \"iNotiFDs\", \"EventFDs\", \"TimerFDs\", \"SigFDs\", \"EvPollFDs\", \"OtherFDs\" ]"
4530 + "\n }"
4531 + "\n },"
4532 + "\n \"group_by\": {"
4533 + "\n \"Process Tree by PID\": {"
4534 + "\n \"columns\": [ \"PPid\", \"Pid\" ],"
4535 + "\n \"order\": [ \"ascending\", \"ascending\" ]"
4536 + "\n },"
4537 + "\n \"Process Tree by Category\": {"
4538 + "\n \"columns\": [ \"Category\", \"PPid\", \"Pid\" ],"
4539 + "\n \"order\": [ \"any\", \"ascending\", \"ascending\" ]"
4540 + "\n }"
4541 + "\n },"
4542 + "\n \"data\":["
4543 + "\n"
4544 + );
4545 + }
4546 +
4547 + fwrite(buffer_tostring(func_processes_fields), buffer_strlen(func_processes_fields), 1, stdout);
4548 +
4549 + unsigned int cpu_divisor = time_factor * RATES_DETAIL / 100;
4550 + unsigned int memory_divisor = 1024;
4551 + unsigned int io_divisor = 1024 * RATES_DETAIL;
4552 +
4553 + BUFFER *wb = buffer_create(PLUGINSD_LINE_MAX);
4554 +
4555 + int rows= 0;
4556 + for(p = root_of_pids; p ; p = p->next) {
4557 + if(!p->updated)
4558 + continue;
4559 +
4560 + if(category && p->target != category)
4561 + continue;
4562 +
4563 + if(user && p->user_target != user)
4564 + continue;
4565 +
4566 + if(group && p->group_target != group)
4567 + continue;
4568 +
4569 + if(process_name && ((strcmp(p->comm, process_name) != 0 && !p->parent) || (p->parent && strcmp(p->comm, process_name) != 0 && strcmp(p->parent->comm, process_name) != 0)))
4570 + continue;
4571 +
4572 + if(filter_pid && p->pid != pid && p->ppid != pid)
4573 + continue;
4574 +
4575 + if(filter_uid && p->uid != uid)
4576 + continue;
4577 +
4578 + if(filter_gid && p->gid != gid)
4579 + continue;
4580 +
4581 + if(rows) buffer_fast_strcat(wb, ",\n", 2);
4582 + rows++;
4583 +
4584 + buffer_strcat(wb, " [");
4585 +
4586 + // IMPORTANT!
4587 + // THE ORDER SHOULD BE THE SAME WITH THE FIELDS!
4588 +
4589 + // pid
4590 + buffer_print_llu(wb, p->pid);
4591 +
4592 + // cmd
4593 + buffer_fast_strcat(wb, ",\"", 2);
4594 + buffer_strcat_jsonescape(wb, p->comm);
4595 + buffer_fast_strcat(wb, "\"", 1);
4596 +
4597 +#ifdef NETDATA_DEV_MODE
4598 + // cmdline
4599 + buffer_fast_strcat(wb, ",\"", 2);
4600 + buffer_strcat_jsonescape(wb, (p->cmdline && *p->cmdline) ? p->cmdline : p->comm);
4601 + buffer_fast_strcat(wb, "\"", 1);
4602 +#endif
4603 +
4604 + // ppid
4605 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->ppid);
4606 +
4607 + // category
4608 + buffer_fast_strcat(wb, ",\"", 2);
4609 + buffer_strcat_jsonescape(wb, p->target ? p->target->name : "-");
4610 + buffer_fast_strcat(wb, "\"", 1);
4611 +
4612 + // user
4613 + buffer_fast_strcat(wb, ",\"", 2);
4614 + buffer_strcat_jsonescape(wb, p->user_target ? p->user_target->name : "-");
4615 + buffer_fast_strcat(wb, "\"", 1);
4616 +
4617 + // uid
4618 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->uid);
4619 +
4620 + // group
4621 + buffer_fast_strcat(wb, ",\"", 2);
4622 + buffer_strcat_jsonescape(wb, p->group_target ? p->group_target->name : "-");
4623 + buffer_fast_strcat(wb, "\"", 1);
4624 +
4625 + // gid
4626 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->gid);
4627 +
4628 + // procs
4629 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->children_count);
4630 +
4631 + // threads
4632 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->num_threads);
4633 +
4634 + // uptime
4635 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->uptime);
4636 +
4637 + // minor page faults
4638 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->minflt / RATES_DETAIL);
4639 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->cminflt / RATES_DETAIL);
4640 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, (p->minflt + p->cminflt) / RATES_DETAIL);
4641 +
4642 + // major page faults
4643 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->majflt / RATES_DETAIL);
4644 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->cmajflt / RATES_DETAIL);
4645 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, (p->majflt + p->cmajflt) / RATES_DETAIL);
4646 +
4647 + // CPU utilization %
4648 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->utime) / cpu_divisor);
4649 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->stime) / cpu_divisor);
4650 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->gtime) / cpu_divisor);
4651 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->cutime) / cpu_divisor);
4652 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->cstime) / cpu_divisor);
4653 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->cgtime) / cpu_divisor);
4654 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)(p->utime + p->stime + p->gtime + p->cutime + p->cstime + p->cgtime) / cpu_divisor);
4655 +
4656 + // memory MiB
4657 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)p->status_vmsize / memory_divisor);
4658 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)p->status_vmrss / memory_divisor);
4659 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)p->status_vmshared / memory_divisor);
4660 + buffer_fast_strcat(wb, ",", 1); buffer_rrd_value(wb, (NETDATA_DOUBLE)p->status_vmswap / memory_divisor);
4661 +
4662 + if(MemTotal) {
4663 + buffer_fast_strcat(wb, ",", 1);
4664 + buffer_rrd_value(wb, (NETDATA_DOUBLE)p->status_vmrss * 100.0 / (NETDATA_DOUBLE)MemTotal);
4665 + }
4666 +
4667 + // Logical I/O
4668 +#ifndef __FreeBSD__
4669 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_logical_bytes_read / io_divisor);
4670 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_logical_bytes_written / io_divisor);
4671 +#endif
4672 +
4673 + // Physical I/O
4674 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_storage_bytes_read / io_divisor);
4675 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_storage_bytes_written / io_divisor);
4676 +
4677 + // I/O calls
4678 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_read_calls / RATES_DETAIL);
4679 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->io_write_calls / RATES_DETAIL);
4680 +
4681 + // open file descriptors
4682 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.files);
4683 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.pipes);
4684 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.sockets);
4685 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.inotifies);
4686 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.eventfds);
4687 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.timerfds);
4688 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.signalfds);
4689 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.eventpolls);
4690 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.other);
4691 + buffer_fast_strcat(wb, ",", 1); buffer_print_llu(wb, p->openfds.files + p->openfds.pipes + p->openfds.sockets + p->openfds.inotifies + p->openfds.eventfds + p->openfds.timerfds + p->openfds.signalfds + p->openfds.eventpolls + p->openfds.other);
4692 +
4693 + buffer_fast_strcat(wb, "]", 1);
4694 +
4695 + fwrite(buffer_tostring(wb), buffer_strlen(wb), 1, stdout);
4696 + buffer_flush(wb);
4697 + }
4698 +
4699 + buffer_free(wb);
4700 +
4701 + fprintf(stdout, "\n ]");
4702 + fprintf(stdout, ",\n \"expires\":%ld", expires);
4703 + fprintf(stdout, "\n}");
4704 +
4705 + pluginsd_function_result_end_to_stdout();
4706 +}
4707 +
4708 +bool apps_plugin_exit = false;
4709 +
4710 +void *reader_main(void *arg __maybe_unused) {
4711 + char buffer[PLUGINSD_LINE_MAX + 1];
4712 +
4713 + char *s = NULL;
4714 + while(!apps_plugin_exit && (s = fgets(buffer, PLUGINSD_LINE_MAX, stdin))) {
4715 +
4716 + char *words[PLUGINSD_MAX_WORDS] = { NULL };
4717 + pluginsd_split_words(buffer, words, PLUGINSD_MAX_WORDS, NULL, NULL, 0);
4718 +
4719 + if(words[0] && strcmp(words[0], PLUGINSD_KEYWORD_FUNCTION) == 0) {
4720 + char *transaction = words[1];
4721 + char *timeout_s = words[2];
4722 + char *function = words[3];
4723 +
4724 + if(!transaction || !*transaction || !timeout_s || !*timeout_s || !function || !*function) {
4725 + error("Received incomplete %s (transaction = '%s', timeout = '%s', function = '%s'). Ignoring it.",
4726 + words[0],
4727 + transaction?transaction:"(unset)",
4728 + timeout_s?timeout_s:"(unset)",
4729 + function?function:"(unset)");
4730 + }
4731 + else {
4732 + int timeout = str2i(timeout_s);
4733 + if(timeout <= 0) timeout = PLUGINS_FUNCTIONS_TIMEOUT_DEFAULT;
4734 +
4735 +// internal_error(true, "Received function '%s', transaction '%s', timeout %d", function, transaction, timeout);
4736 +
4737 + netdata_mutex_lock(&mutex);
4738 +
4739 + if(strncmp(function, "processes", strlen("processes")) == 0)
4740 + apps_plugin_function_processes(transaction, function, buffer, PLUGINSD_LINE_MAX + 1, timeout);
4741 + else
4742 + apps_plugin_function_error(transaction, HTTP_RESP_NOT_FOUND, "No function with this name found in apps.plugin.");
4743 +
4744 + fflush(stdout);
4745 + netdata_mutex_unlock(&mutex);
4746 +
4747 +// internal_error(true, "Done with function '%s', transaction '%s', timeout %d", function, transaction, timeout);
4748 + }
4749 + }
4750 + else
4751 + error("Received unknown command: %s", words[0]?words[0]:"(unset)");
4752 + }
4753 +
4754 + if(!s || feof(stdin) || ferror(stdin)) {
4755 + apps_plugin_exit = true;
4756 + error("Received error on stdin.");
4757 + }
4758 +
4759 + exit(1);
4760 + return NULL;
4761 +}
4762 +
4763 int main(int argc, char **argv) {
4764 // debug_flags = D_PROCFILE;
4765
@@ -4224,16 +4850,21 @@ int main(int argc, char **argv) {
4850 debug_log("group file: '%s'", all_group_ids.filename);
4851
4852 #if (ALL_PIDS_ARE_READ_INSTANTLY == 0)
4227 - all_pids_sortlist = callocz(sizeof(pid_t), (size_t)pid_max);
4853 + all_pids_sortlist = callocz(sizeof(pid_t), (size_t)pid_max + 1);
4854 #endif
4855
4230 - all_pids = callocz(sizeof(struct pid_stat *), (size_t) pid_max);
4856 + all_pids = callocz(sizeof(struct pid_stat *), (size_t) pid_max + 1);
4857 +
4858 + netdata_thread_t reader_thread;
4859 + netdata_thread_create(&reader_thread, "APPS_READER", NETDATA_THREAD_OPTION_DONT_LOG, reader_main, NULL);
4860 + netdata_mutex_lock(&mutex);
4861
4862 usec_t step = update_every * USEC_PER_SEC;
4863 global_iterations_counter = 1;
4864 heartbeat_t hb;
4865 heartbeat_init(&hb);
4236 - for(;1; global_iterations_counter++) {
4866 + for(; !apps_plugin_exit ; global_iterations_counter++) {
4867 + netdata_mutex_unlock(&mutex);
4868
4869 #ifdef NETDATA_PROFILING
4870 #warning "compiling for profiling"
@@ -4244,16 +4875,28 @@ int main(int argc, char **argv) {
4875 #else
4876 usec_t dt = heartbeat_next(&hb, step);
4877 #endif
4878 + netdata_mutex_lock(&mutex);
4879
4880 struct pollfd pollfd = { .fd = fileno(stdout), .events = POLLERR };
4249 - if (unlikely(poll(&pollfd, 1, 0) < 0))
4881 + if (unlikely(poll(&pollfd, 1, 0) < 0)) {
4882 + netdata_mutex_unlock(&mutex);
4883 + netdata_thread_cancel(reader_thread);
4884 fatal("Cannot check if a pipe is available");
4251 - if (unlikely(pollfd.revents & POLLERR))
4252 - fatal("Cannot write to a pipe");
4885 + }
4886 + if (unlikely(pollfd.revents & POLLERR)) {
4887 + netdata_mutex_unlock(&mutex);
4888 + netdata_thread_cancel(reader_thread);
4889 + fatal("Received error on read pipe.");
4890 + }
4891 +
4892 + if(global_iterations_counter % 10 == 0)
4893 + get_MemTotal();
4894
4895 if(!collect_data_for_all_processes()) {
4896 error("Cannot collect /proc data for running processes. Disabling apps.plugin...");
4897 printf("DISABLE\n");
4898 + netdata_mutex_unlock(&mutex);
4899 + netdata_thread_cancel(reader_thread);
4900 exit(1);
4901 }
4902
collectors/cgroups.plugin/cgroup-network.c
+6 -6
@@ -506,21 +506,21 @@ void call_the_helper(pid_t pid, const char *cgroup) {
506 info("running: %s", command);
507
508 pid_t cgroup_pid;
509 - FILE *fp;
509 + FILE *fp_child_input, *fp_child_output;
510
511 if(cgroup) {
512 - (void)mypopen_raw_default_flags(&cgroup_pid, environment, &fp, PLUGINS_DIR "/cgroup-network-helper.sh", "--cgroup", cgroup);
512 + (void)netdata_popen_raw_default_flags(&cgroup_pid, environment, &fp_child_input, &fp_child_output, PLUGINS_DIR "/cgroup-network-helper.sh", "--cgroup", cgroup);
513 }
514 else {
515 char buffer[100];
516 snprintfz(buffer, sizeof(buffer) - 1, "%d", pid);
517 - (void)mypopen_raw_default_flags(&cgroup_pid, environment, &fp, PLUGINS_DIR "/cgroup-network-helper.sh", "--pid", buffer);
517 + (void)netdata_popen_raw_default_flags(&cgroup_pid, environment, &fp_child_input, &fp_child_output, PLUGINS_DIR "/cgroup-network-helper.sh", "--pid", buffer);
518 }
519
520 - if(fp) {
520 + if(fp_child_output) {
521 char buffer[CGROUP_NETWORK_INTERFACE_MAX_LINE + 1];
522 char *s;
523 - while((s = fgets(buffer, CGROUP_NETWORK_INTERFACE_MAX_LINE, fp))) {
523 + while((s = fgets(buffer, CGROUP_NETWORK_INTERFACE_MAX_LINE, fp_child_output))) {
524 trim(s);
525
526 if(*s && *s != '\n') {
@@ -536,7 +536,7 @@ void call_the_helper(pid_t pid, const char *cgroup) {
536 }
537 }
538
539 - mypclose(fp, cgroup_pid);
539 + netdata_pclose(fp_child_input, fp_child_output, cgroup_pid);
540 }
541 else
542 error("cannot execute cgroup-network helper script: %s", command);
collectors/cgroups.plugin/sys_fs_cgroup.c
+26 -24
@@ -153,14 +153,15 @@ static enum cgroups_systemd_setting cgroups_detect_systemd(const char *exec)
153 char buf[MAXSIZE_PROC_CMDLINE];
154 char *begin, *end;
155
156 - FILE *f = mypopen(exec, &command_pid);
156 + FILE *fp_child_input;
157 + FILE *fp_child_output = netdata_popen(exec, &command_pid, &fp_child_input);
158
158 - if (!f)
159 + if (!fp_child_output)
160 return retval;
161
162 fd_set rfds;
163 struct timeval timeout;
163 - int fd = fileno(f);
164 + int fd = fileno(fp_child_output);
165 int ret = -1;
166
167 FD_ZERO(&rfds);
@@ -177,7 +178,7 @@ static enum cgroups_systemd_setting cgroups_detect_systemd(const char *exec)
178 } else if (ret == 0) {
179 info("Cannot get the output of \"%s\" within %"PRId64" seconds", exec, (int64_t)timeout.tv_sec);
180 } else {
180 - while (fgets(buf, MAXSIZE_PROC_CMDLINE, f) != NULL) {
181 + while (fgets(buf, MAXSIZE_PROC_CMDLINE, fp_child_output) != NULL) {
182 if ((begin = strstr(buf, SYSTEMD_HIERARCHY_STRING))) {
183 end = begin = begin + strlen(SYSTEMD_HIERARCHY_STRING);
184 if (!*begin)
@@ -196,7 +197,7 @@ static enum cgroups_systemd_setting cgroups_detect_systemd(const char *exec)
197 }
198 }
199
199 - if (mypclose(f, command_pid))
200 + if (netdata_pclose(fp_child_input, fp_child_output, command_pid))
201 return SYSTEMD_CGROUP_ERR;
202
203 return retval;
@@ -210,18 +211,19 @@ static enum cgroups_type cgroups_try_detect_version()
211 int cgroups2_available = 0;
212
213 // 1. check if cgroups2 available on system at all
213 - FILE *f = mypopen("grep cgroup /proc/filesystems", &command_pid);
214 - if (!f) {
214 + FILE *fp_child_input;
215 + FILE *fp_child_output = netdata_popen("grep cgroup /proc/filesystems", &command_pid, &fp_child_input);
216 + if (!fp_child_output) {
217 error("popen failed");
218 return CGROUPS_AUTODETECT_FAIL;
219 }
218 - while (fgets(buf, MAXSIZE_PROC_CMDLINE, f) != NULL) {
220 + while (fgets(buf, MAXSIZE_PROC_CMDLINE, fp_child_output) != NULL) {
221 if (strstr(buf, "cgroup2")) {
222 cgroups2_available = 1;
223 break;
224 }
225 }
224 - if(mypclose(f, command_pid))
226 + if(netdata_pclose(fp_child_input, fp_child_output, command_pid))
227 return CGROUPS_AUTODETECT_FAIL;
228
229 if(!cgroups2_available)
@@ -254,19 +256,19 @@ static enum cgroups_type cgroups_try_detect_version()
256
257 // 4. if we are unified as on Fedora (default cgroups2 only mode)
258 // check kernel command line flag that can override that setting
257 - f = fopen("/proc/cmdline", "r");
258 - if (!f) {
259 + FILE *fp = fopen("/proc/cmdline", "r");
260 + if (!fp) {
261 error("Error reading kernel boot commandline parameters");
262 return CGROUPS_AUTODETECT_FAIL;
263 }
264
263 - if (!fgets(buf, MAXSIZE_PROC_CMDLINE, f)) {
265 + if (!fgets(buf, MAXSIZE_PROC_CMDLINE, fp)) {
266 error("couldn't read all cmdline params into buffer");
265 - fclose(f);
267 + fclose(fp);
268 return CGROUPS_AUTODETECT_FAIL;
269 }
270
269 - fclose(f);
271 + fclose(fp);
272
273 if (strstr(buf, "systemd.unified_cgroup_hierarchy=0")) {
274 info("cgroups v2 (unified cgroups) is available but are disabled on this system.");
@@ -1665,16 +1667,16 @@ static inline void read_cgroup_network_interfaces(struct cgroup *cg) {
1667 }
1668
1669 debug(D_CGROUP, "executing cgroup_identifier %s --cgroup '%s' for cgroup '%s'", cgroups_network_interface_script, cgroup_identifier, cg->id);
1668 - FILE *fp;
1669 - (void)mypopen_raw_default_flags_and_environment(&cgroup_pid, &fp, cgroups_network_interface_script, "--cgroup", cgroup_identifier);
1670 - if(!fp) {
1670 + FILE *fp_child_input, *fp_child_output;
1671 + (void)netdata_popen_raw_default_flags_and_environment(&cgroup_pid, &fp_child_input, &fp_child_output, cgroups_network_interface_script, "--cgroup", cgroup_identifier);
1672 + if(!fp_child_output) {
1673 error("CGROUP: cannot popen(%s --cgroup \"%s\", \"r\").", cgroups_network_interface_script, cgroup_identifier);
1674 return;
1675 }
1676
1677 char *s;
1678 char buffer[CGROUP_NETWORK_INTERFACE_MAX_LINE + 1];
1677 - while((s = fgets(buffer, CGROUP_NETWORK_INTERFACE_MAX_LINE, fp))) {
1679 + while((s = fgets(buffer, CGROUP_NETWORK_INTERFACE_MAX_LINE, fp_child_output))) {
1680 trim(s);
1681
1682 if(*s && *s != '\n') {
@@ -1709,7 +1711,7 @@ static inline void read_cgroup_network_interfaces(struct cgroup *cg) {
1711 }
1712 }
1713
1712 - mypclose(fp, cgroup_pid);
1714 + netdata_pclose(fp_child_input, fp_child_output, cgroup_pid);
1715 // debug(D_CGROUP, "closed cgroup_identifier for cgroup '%s'", cg->id);
1716 }
1717
@@ -1871,9 +1873,9 @@ static inline void discovery_rename_cgroup(struct cgroup *cg) {
1873 debug(D_CGROUP, "executing command %s \"%s\" for cgroup '%s'", cgroups_rename_script, cg->intermediate_id, cg->chart_id);
1874 pid_t cgroup_pid;
1875
1874 - FILE *fp;
1875 - (void)mypopen_raw_default_flags_and_environment(&cgroup_pid, &fp, cgroups_rename_script, cg->id, cg->intermediate_id);
1876 - if (!fp) {
1876 + FILE *fp_child_input, *fp_child_output;
1877 + (void)netdata_popen_raw_default_flags_and_environment(&cgroup_pid, &fp_child_input, &fp_child_output, cgroups_rename_script, cg->id, cg->intermediate_id);
1878 + if (!fp_child_output) {
1879 error("CGROUP: cannot popen(%s \"%s\", \"r\").", cgroups_rename_script, cg->intermediate_id);
1880 cg->pending_renames = 0;
1881 cg->processed = 1;
@@ -1881,8 +1883,8 @@ static inline void discovery_rename_cgroup(struct cgroup *cg) {
1883 }
1884
1885 char buffer[CGROUP_CHARTID_LINE_MAX + 1];
1884 - char *new_name = fgets(buffer, CGROUP_CHARTID_LINE_MAX, fp);
1885 - int exit_code = mypclose(fp, cgroup_pid);
1886 + char *new_name = fgets(buffer, CGROUP_CHARTID_LINE_MAX, fp_child_output);
1887 + int exit_code = netdata_pclose(fp_child_input, fp_child_output, cgroup_pid);
1888
1889 switch (exit_code) {
1890 case 0:
collectors/diskspace.plugin/plugin_diskspace.c
+3
@@ -581,6 +581,7 @@ void *diskspace_slow_worker(void *ptr)
581 }
582
583 static void diskspace_main_cleanup(void *ptr) {
584 + rrd_collector_finished();
585 worker_unregister();
586
587 struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr;
@@ -612,6 +613,8 @@ void *diskspace_main(void *ptr) {
613 worker_register_job_name(WORKER_JOB_MOUNTPOINT, "mountpoint");
614 worker_register_job_name(WORKER_JOB_CLEANUP, "cleanup");
615
616 + rrd_collector_started();
617 +
618 netdata_thread_cleanup_push(diskspace_main_cleanup, ptr);
619
620 cleanup_mount_points = config_get_boolean(CONFIG_SECTION_DISKSPACE, "remove charts of unmounted disks" , cleanup_mount_points);
collectors/plugins.d/README.md
+81 -5
@@ -116,17 +116,19 @@ For example, if your plugin wants to monitor `squid`, you can search for it on p
116
117 Any program that can print a few values to its standard output can become a Netdata external plugin.
118
119 -Netdata parses 9 lines starting with:
119 +Netdata parses lines starting with:
120
121 - `CHART` - create or update a chart
122 - `DIMENSION` - add or update a dimension to the chart just created
123 +- `VARIABLE` - define a variable (to be used in health calculations)
124 +- `CLABEL` - add a label to a chart
125 +- `CLABEL_COMMIT` - commit added labels to the chart
126 +- `FUNCTION` - define a function that can be called later to execute it
127 - `BEGIN` - initialize data collection for a chart
128 - `SET` - set the value of a dimension for the initialized chart
129 - `END` - complete data collection for the initialized chart
130 - `FLUSH` - ignore the last collected values
131 - `DISABLE` - disable this plugin
128 -- `CLABEL` - add a label to a chart
129 -- `CLABEL_COMMIT` - commit added labels to the chart.
132
133 a single program can produce any number of charts with any number of dimensions each.
134
@@ -362,6 +364,80 @@ The `source` is an integer field that can have the following values:
364
365 `CLABEL_COMMIT` indicates that all labels were defined and the chart can be updated.
366
367 +#### FUNCTION
368 +
369 +> FUNCTION [GLOBAL] "name and parameters of the function" timeout "help string for users"
370 +
371 +A function can be used by users to ask for more information from the collector. Netdata maintains a registry of functions in 2 levels:
372 +
373 +- per node
374 +- per chart
375 +
376 +Both node and chart functions are exactly the same, but chart functions allow Netdata to relate functions with charts and therefore present a context sensitive menu of functions related to the chart the user is using.
377 +
378 +A function is identified by a string. The allowed characters in the function definition are:
379 +
380 +| Character | Symbol | In Functions |
381 +|-------------------|:------:|:------------:|
382 +| UTF-8 character | UTF-8 | keep |
383 +| Lower case letter | [a-z] | keep |
384 +| Upper case letter | [A-Z] | keep |
385 +| Digit | [0-9] | keep |
386 +| Underscore | _ | keep |
387 +| Comma | , | keep |
388 +| Minus | - | keep |
389 +| Period | . | keep |
390 +| Colon | : | keep |
391 +| Slash | / | keep |
392 +| Space | ' ' | keep |
393 +| Semicolon | ; | : |
394 +| Equal | = | : |
395 +| Backslash | \ | / |
396 +| Anything else | | _ |
397 +
398 +Uses can get a list of all the registered functions using the `/api/v1/functions` end point of Netdata.
399 +
400 +Users can call functions using the `/api/v1/function` end point of Netdata.
401 +Once a function is called, the plugin will receive at its standard input a command that looks like this:
402 +
403 +> FUNCTION transaction_id timeout "name and parameters of the function"
404 +
405 +The plugin is expected to parse and validate `name and parameters of the function`. Netdata allows users to edit this string, append more parameters or even change the ones the plugin originally exposed. To minimize the security risk, Netdata guarantees that only the characters shown above are accepted in function definitions, but still the plugin should carefully inspect the `name and parameters of the function` to ensure that it is valid and not harmful.
406 +
407 +If the plugin rejects the request, it should respond with this:
408 +
409 +```
410 +FUNCTION_RESULT_BEGIN transaction_id 400 application/json
411 +{
412 + "status": 400,
413 + "error_message": "description of the rejection reasons"
414 +}
415 +FUNCTION_RESULT_END
416 +```
417 +
418 +If the plugin prepares a response, it should send (via its standard output, together with the collected data, but not interleaved with them):
419 +
420 +> FUNCTION_RESULT_BEGIN transaction_id http_error_code content_type expiration
421 +
422 +Where:
423 +
424 + - `transaction_id` is the transaction id that Netdata sent for this function execution
425 + - `http_error` is the http error code Netdata should respond with, 200 is the "ok" response
426 + - `content_type` is the content type of the response
427 + - `expiration` is the absolute timestamp (number, unix epoch) this response expires
428 +
429 +Immediately after this, all text is assumed to be the response content.
430 +The content is text and line oriented. The maximum line length accepted is 15kb. Longer lines will be truncated.
431 +The type of the context itself depends on the plugin and the UI.
432 +
433 +To terminate the message, Netdata seeks a line with just this:
434 +
435 +> FUNCTION_RESULT_END
436 +
437 +This defines the end of the message. `FUNCTION_RESULT_END` should appear in a line alone, without any other text, so it is wise to add `\n` before and after it.
438 +
439 +After this line, Netdata resumes processing collected metrics from the plugin.
440 +
441 ## Data collection
442
443 data collection is defined as a series of `BEGIN` -> `SET` -> `END` lines
@@ -463,7 +539,7 @@ There are a few rules for writing plugins properly:
539 readConfiguration();
540
541 if(!verifyWeCanCollectValues()) {
466 - print "DISABLE";
542 + print("DISABLE");
543 exit(1);
544 }
545
@@ -475,7 +551,7 @@ There are a few rules for writing plugins properly:
551 var dt_since_last_run = 0;
552 var now = 0;
553
478 - FOREVER {
554 + while(true) {
555 /* find the current time in milliseconds */
556 now = currentTimeStampInMilliseconds();
557
collectors/plugins.d/plugins_d.c
+5 -121
@@ -6,117 +6,6 @@
6 char *plugin_directories[PLUGINSD_MAX_DIRECTORIES] = { NULL };
7 struct plugind *pluginsd_root = NULL;
8
9 -inline int pluginsd_space(char c) {
10 - switch(c) {
11 - case ' ':
12 - case '\t':
13 - case '\r':
14 - case '\n':
15 - case '=':
16 - return 1;
17 -
18 - default:
19 - return 0;
20 - }
21 -}
22 -
23 -inline int config_isspace(char c)
24 -{
25 - switch (c) {
26 - case ' ':
27 - case '\t':
28 - case '\r':
29 - case '\n':
30 - case ',':
31 - return 1;
32 -
33 - default:
34 - return 0;
35 - }
36 -}
37 -
38 -// split a text into words, respecting quotes
39 -inline int quoted_strings_splitter(char *str, char **words, int max_words, int (*custom_isspace)(char), char *recover_input, char **recover_location, int max_recover)
40 -{
41 - char *s = str, quote = 0;
42 - int i = 0, rec = 0;
43 - char *recover = recover_input;
44 -
45 - // skip all white space
46 - while (unlikely(custom_isspace(*s)))
47 - s++;
48 -
49 - // check for quote
50 - if (unlikely(*s == '\'' || *s == '"')) {
51 - quote = *s; // remember the quote
52 - s++; // skip the quote
53 - }
54 -
55 - // store the first word
56 - words[i++] = s;
57 -
58 - // while we have something
59 - while (likely(*s)) {
60 - // if it is escape
61 - if (unlikely(*s == '\\' && s[1])) {
62 - s += 2;
63 - continue;
64 - }
65 -
66 - // if it is quote
67 - else if (unlikely(*s == quote)) {
68 - quote = 0;
69 - if (recover && rec < max_recover) {
70 - recover_location[rec++] = s;
71 - *recover++ = *s;
72 - }
73 - *s = ' ';
74 - continue;
75 - }
76 -
77 - // if it is a space
78 - else if (unlikely(quote == 0 && custom_isspace(*s))) {
79 - // terminate the word
80 - if (recover && rec < max_recover) {
81 - if (!rec || (rec && recover_location[rec-1] != s)) {
82 - recover_location[rec++] = s;
83 - *recover++ = *s;
84 - }
85 - }
86 - *s++ = '\0';
87 -
88 - // skip all white space
89 - while (likely(custom_isspace(*s)))
90 - s++;
91 -
92 - // check for quote
93 - if (unlikely(*s == '\'' || *s == '"')) {
94 - quote = *s; // remember the quote
95 - s++; // skip the quote
96 - }
97 -
98 - // if we reached the end, stop
99 - if (unlikely(!*s))
100 - break;
101 -
102 - // store the next word
103 - if (likely(i < max_words))
104 - words[i++] = s;
105 - else
106 - break;
107 - }
108 -
109 - // anything else
110 - else
111 - s++;
112 - }
113 -
114 - // terminate the words
115 - memset(&words[i], 0, (max_words - i) * sizeof (char *));
116 -
117 - return i;
118 -}
119 -
9 inline int pluginsd_initialize_plugin_directories()
10 {
11 char plugins_dirs[(FILENAME_MAX * 2) + 1];
@@ -132,12 +21,6 @@ inline int pluginsd_initialize_plugin_directories()
21 return quoted_strings_splitter(plugins_dir_list, plugin_directories, PLUGINSD_MAX_DIRECTORIES, config_isspace, NULL, NULL, 0);
22 }
23
135 -inline int pluginsd_split_words(char *str, char **words, int max_words, char *recover_input, char **recover_location, int max_recover)
136 -{
137 - return quoted_strings_splitter(str, words, max_words, pluginsd_space, recover_input, recover_location, max_recover);
138 -}
139 -
140 -
24 static void pluginsd_worker_thread_cleanup(void *arg)
25 {
26 struct plugind *cd = (struct plugind *)arg;
@@ -238,18 +121,19 @@ void *pluginsd_worker_thread(void *arg)
121 size_t count = 0;
122
123 while (!netdata_exit) {
241 - FILE *fp = mypopen(cd->cmd, &cd->pid);
242 - if (unlikely(!fp)) {
124 + FILE *fp_child_input = NULL;
125 + FILE *fp_child_output = netdata_popen(cd->cmd, &cd->pid, &fp_child_input);
126 + if (unlikely(!fp_child_input || !fp_child_output)) {
127 error("Cannot popen(\"%s\", \"r\").", cd->cmd);
128 break;
129 }
130
131 info("connected to '%s' running on pid %d", cd->fullfilename, cd->pid);
248 - count = pluginsd_process(localhost, cd, fp, 0);
132 + count = pluginsd_process(localhost, cd, fp_child_input, fp_child_output, 0);
133 error("'%s' (pid %d) disconnected after %zu successful data collections (ENDs).", cd->fullfilename, cd->pid, count);
134 killpid(cd->pid);
135
252 - int worker_ret_code = mypclose(fp, cd->pid);
136 + int worker_ret_code = netdata_pclose(fp_child_input, fp_child_output, cd->pid);
137
138 if (likely(worker_ret_code == 0))
139 pluginsd_worker_thread_handle_success(cd);
collectors/plugins.d/plugins_d.h
+51 -22
@@ -10,22 +10,30 @@
10 #define PLUGINSD_CMD_MAX (FILENAME_MAX*2)
11 #define PLUGINSD_STOCK_PLUGINS_DIRECTORY_PATH 0
12
13 -#define PLUGINSD_KEYWORD_CHART "CHART"
14 -#define PLUGINSD_KEYWORD_DIMENSION "DIMENSION"
15 -#define PLUGINSD_KEYWORD_BEGIN "BEGIN"
16 -#define PLUGINSD_KEYWORD_END "END"
17 -#define PLUGINSD_KEYWORD_FLUSH "FLUSH"
18 -#define PLUGINSD_KEYWORD_DISABLE "DISABLE"
19 -#define PLUGINSD_KEYWORD_VARIABLE "VARIABLE"
20 -#define PLUGINSD_KEYWORD_LABEL "LABEL"
21 -#define PLUGINSD_KEYWORD_OVERWRITE "OVERWRITE"
22 -#define PLUGINSD_KEYWORD_GUID "GUID"
23 -#define PLUGINSD_KEYWORD_CONTEXT "CONTEXT"
24 -#define PLUGINSD_KEYWORD_TOMBSTONE "TOMBSTONE"
25 -#define PLUGINSD_KEYWORD_HOST "HOST"
26 -
27 -
28 -#define PLUGINSD_LINE_MAX 1024
13 +#define PLUGINSD_KEYWORD_CHART "CHART"
14 +#define PLUGINSD_KEYWORD_DIMENSION "DIMENSION"
15 +#define PLUGINSD_KEYWORD_BEGIN "BEGIN"
16 +#define PLUGINSD_KEYWORD_SET "SET"
17 +#define PLUGINSD_KEYWORD_END "END"
18 +#define PLUGINSD_KEYWORD_FLUSH "FLUSH"
19 +#define PLUGINSD_KEYWORD_DISABLE "DISABLE"
20 +#define PLUGINSD_KEYWORD_VARIABLE "VARIABLE"
21 +#define PLUGINSD_KEYWORD_LABEL "LABEL"
22 +#define PLUGINSD_KEYWORD_OVERWRITE "OVERWRITE"
23 +#define PLUGINSD_KEYWORD_CLABEL "CLABEL"
24 +#define PLUGINSD_KEYWORD_CLABEL_COMMIT "CLABEL_COMMIT"
25 +#define PLUGINSD_KEYWORD_FUNCTION "FUNCTION"
26 +#define PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN "FUNCTION_RESULT_BEGIN"
27 +#define PLUGINSD_KEYWORD_FUNCTION_RESULT_END "FUNCTION_RESULT_END"
28 +#define PLUGINSD_KEYWORD_GUID "GUID"
29 +#define PLUGINSD_KEYWORD_CONTEXT "CONTEXT"
30 +#define PLUGINSD_KEYWORD_TOMBSTONE "TOMBSTONE"
31 +#define PLUGINSD_KEYWORD_HOST "HOST"
32 +//#define PLUGINSD_KEYWORD_GAPS_REQUEST "GAPS_REQUEST" // child -> parent
33 +//#define PLUGINSD_KEYWORD_CHART_GAP "CHART_GAP" // parent <- child
34 +
35 +#define PLUGINS_FUNCTIONS_TIMEOUT_DEFAULT 10 // seconds
36 +
37 #define PLUGINSD_LINE_MAX_SSL_READ 512
38 #define PLUGINSD_MAX_WORDS 20
39
@@ -53,19 +61,40 @@ struct plugind {
61 volatile sig_atomic_t enabled; // if this is enabled or not
62
63 time_t started_t;
56 - uint32_t version;
64 + uint32_t capabilities; // follows the same principles as streaming capabilities
65 struct plugind *next;
66 };
67
68 extern struct plugind *pluginsd_root;
69
62 -extern size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp, int trust_durations);
63 -extern int pluginsd_split_words(char *str, char **words, int max_words, char *recover_string, char **recover_location, int max_recover);
70 +extern size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugin_input, FILE *fp_plugin_output, int trust_durations);
71
72 extern int pluginsd_initialize_plugin_directories();
73
67 -extern int config_isspace(char c);
68 -extern int pluginsd_space(char c);
69 -int quoted_strings_splitter(char *str, char **words, int max_words, int (*custom_isspace)(char), char *recover_input, char **recover_location, int max_recover);
74 +
75 +
76 +#define pluginsd_function_result_begin_to_buffer(wb, transaction, code, content_type, expires) \
77 + buffer_sprintf(wb \
78 + , PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN " \"%s\" %d \"%s\" %ld\n" \
79 + , (transaction) ? (transaction) : "" \
80 + , (int)(code) \
81 + , (content_type) ? (content_type) : "" \
82 + , (long int)(expires) \
83 + )
84 +
85 +#define pluginsd_function_result_end_to_buffer(wb) \
86 + buffer_strcat(wb, "\n" PLUGINSD_KEYWORD_FUNCTION_RESULT_END "\n")
87 +
88 +#define pluginsd_function_result_begin_to_stdout(transaction, code, content_type, expires) \
89 + fprintf(stdout \
90 + , PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN " \"%s\" %d \"%s\" %ld\n" \
91 + , (transaction) ? (transaction) : "" \
92 + , (int)(code) \
93 + , (content_type) ? (content_type) : "" \
94 + , (long int)(expires) \
95 + )
96 +
97 +#define pluginsd_function_result_end_to_stdout() \
98 + fprintf(stdout, "\n" PLUGINSD_KEYWORD_FUNCTION_RESULT_END "\n")
99
100 #endif /* NETDATA_PLUGINS_D_H */
collectors/plugins.d/pluginsd_parser.c
+295 -72
@@ -2,6 +2,8 @@
2
3 #include "pluginsd_parser.h"
4
5 +#define LOG_FUNCTIONS false
6 +
7 /*
8 * This is the action defined for the FLUSH command
9 */
@@ -174,29 +176,6 @@ PARSER_RC pluginsd_label_action(void *user, char *key, char *value, RRDLABEL_SRC
176 return PARSER_RC_OK;
177 }
178
177 -PARSER_RC pluginsd_clabel_action(void *user, char *key, char *value, RRDLABEL_SRC source)
178 -{
179 - if(unlikely(!((PARSER_USER_OBJECT *) user)->new_chart_labels))
180 - ((PARSER_USER_OBJECT *) user)->new_chart_labels = rrdlabels_create();
181 -
182 - rrdlabels_add(((PARSER_USER_OBJECT *)user)->new_chart_labels, key, value, source);
183 -
184 - return PARSER_RC_OK;
185 -}
186 -
187 -PARSER_RC pluginsd_clabel_commit_action(void *user, RRDHOST *host, DICTIONARY *new_chart_labels)
188 -{
189 - RRDSET *st = ((PARSER_USER_OBJECT *)user)->st;
190 - if (unlikely(!st)) {
191 - error("requested CLABEL_COMMIT on host '%s', without a BEGIN, ignoring it.", rrdhost_hostname(host));
192 - return PARSER_RC_OK;
193 - }
194 -
195 - rrdset_update_rrdlabels(st, new_chart_labels);
196 -
197 - return PARSER_RC_OK;
198 -}
199 -
179 PARSER_RC pluginsd_overwrite_action(void *user, RRDHOST *host, DICTIONARY *new_host_labels)
180 {
181 UNUSED(user);
@@ -474,6 +453,248 @@ disable:
453 return PARSER_RC_ERROR;
454 }
455
456 +// ----------------------------------------------------------------------------
457 +// execution of functions
458 +
459 +struct inflight_function {
460 + int code;
461 + int timeout;
462 + BUFFER *destination_wb;
463 + STRING *function;
464 + void (*callback)(BUFFER *wb, int code, void *callback_data);
465 + void *callback_data;
466 + usec_t timeout_ut;
467 + usec_t started_ut;
468 + usec_t sent_ut;
469 +};
470 +
471 +static void inflight_functions_insert_callback(const DICTIONARY_ITEM *item, void *func, void *parser_ptr) {
472 + struct inflight_function *pf = func;
473 +
474 + PARSER *parser = parser_ptr;
475 + FILE *fp = parser->output;
476 +
477 + // leave this code as default, so that when the dictionary is destroyed this will be sent back to the caller
478 + pf->code = HTTP_RESP_GATEWAY_TIMEOUT;
479 +
480 + // send the command to the plugin
481 + int ret = fprintf(fp, "FUNCTION %s %d \"%s\"\n",
482 + dictionary_acquired_item_name(item),
483 + pf->timeout,
484 + string2str(pf->function));
485 +
486 + pf->sent_ut = now_realtime_usec();
487 +
488 + if(ret < 0) {
489 + error("FUNCTION: failed to send function to plugin, fprintf() returned error %d", ret);
490 + rrd_call_function_error(pf->destination_wb, "Failed to communicate with collector", HTTP_RESP_BACKEND_FETCH_FAILED);
491 + }
492 + else {
493 + fflush(fp);
494 +
495 + internal_error(LOG_FUNCTIONS,
496 + "FUNCTION '%s' with transaction '%s' sent to collector (%d bytes, fd %d, in %llu usec)",
497 + string2str(pf->function), dictionary_acquired_item_name(item), ret, fileno(fp),
498 + pf->sent_ut - pf->started_ut);
499 + }
500 +}
501 +
502 +static bool inflight_functions_conflict_callback(const DICTIONARY_ITEM *item __maybe_unused, void *func __maybe_unused, void *new_func, void *parser_ptr __maybe_unused) {
503 + struct inflight_function *pf = new_func;
504 +
505 + error("PLUGINSD_PARSER: duplicate UUID on pending function '%s' detected. Ignoring the second one.", string2str(pf->function));
506 + pf->code = rrd_call_function_error(pf->destination_wb, "This request is already in progress", HTTP_RESP_BAD_REQUEST);
507 + pf->callback(pf->destination_wb, pf->code, pf->callback_data);
508 + string_freez(pf->function);
509 +
510 + return false;
511 +}
512 +static void inflight_functions_delete_callback(const DICTIONARY_ITEM *item __maybe_unused, void *func, void *parser_ptr __maybe_unused) {
513 + struct inflight_function *pf = func;
514 +
515 + internal_error(LOG_FUNCTIONS,
516 + "FUNCTION '%s' result of transaction '%s' received from collector (%zu bytes, request %llu usec, response %llu usec)",
517 + string2str(pf->function), dictionary_acquired_item_name(item),
518 + buffer_strlen(pf->destination_wb), pf->sent_ut - pf->started_ut, now_realtime_usec() - pf->sent_ut);
519 +
520 + pf->callback(pf->destination_wb, pf->code, pf->callback_data);
521 + string_freez(pf->function);
522 +}
523 +
524 +void inflight_functions_init(PARSER *parser) {
525 + parser->inflight.functions = dictionary_create(DICT_OPTION_DONT_OVERWRITE_VALUE);
526 + dictionary_register_insert_callback(parser->inflight.functions, inflight_functions_insert_callback, parser);
527 + dictionary_register_delete_callback(parser->inflight.functions, inflight_functions_delete_callback, parser);
528 + dictionary_register_conflict_callback(parser->inflight.functions, inflight_functions_conflict_callback, parser);
529 +}
530 +
531 +static void inflight_functions_garbage_collect(PARSER *parser, usec_t now) {
532 + parser->inflight.smaller_timeout = 0;
533 + struct inflight_function *pf;
534 + dfe_start_write(parser->inflight.functions, pf) {
535 + if (pf->timeout_ut < now) {
536 + internal_error(true,
537 + "FUNCTION '%s' removing expired transaction '%s', after %llu usec.",
538 + string2str(pf->function), pf_dfe.name, now - pf->started_ut);
539 +
540 + if(!buffer_strlen(pf->destination_wb) || pf->code == HTTP_RESP_OK)
541 + pf->code = rrd_call_function_error(pf->destination_wb,
542 + "Timeout waiting for collector response.",
543 + HTTP_RESP_GATEWAY_TIMEOUT);
544 +
545 + dictionary_del(parser->inflight.functions, pf_dfe.name);
546 + }
547 +
548 + else if(!parser->inflight.smaller_timeout || pf->timeout_ut < parser->inflight.smaller_timeout)
549 + parser->inflight.smaller_timeout = pf->timeout_ut;
550 + }
551 + dfe_done(pf);
552 +}
553 +
554 +// this is the function that is called from
555 +// rrd_call_function_and_wait() and rrd_call_function_async()
556 +static int pluginsd_execute_function_callback(BUFFER *destination_wb, int timeout, const char *function, void *collector_data, void (*callback)(BUFFER *wb, int code, void *callback_data), void *callback_data) {
557 + PARSER *parser = collector_data;
558 +
559 + usec_t now = now_realtime_usec();
560 +
561 + struct inflight_function tmp = {
562 + .started_ut = now,
563 + .timeout_ut = now + timeout * USEC_PER_SEC,
564 + .destination_wb = destination_wb,
565 + .timeout = timeout,
566 + .function = string_strdupz(function),
567 + .callback = callback,
568 + .callback_data = callback_data,
569 + };
570 +
571 + uuid_t uuid;
572 + uuid_generate_time(uuid);
573 +
574 + char key[UUID_STR_LEN];
575 + uuid_unparse_lower(uuid, key);
576 +
577 + dictionary_write_lock(parser->inflight.functions);
578 +
579 + // if there is any error, our dictionary callbacks will call the caller callback to notify
580 + // the caller about the error - no need for error handling here.
581 + dictionary_set(parser->inflight.functions, key, &tmp, sizeof(struct inflight_function));
582 +
583 + if(!parser->inflight.smaller_timeout || tmp.timeout_ut < parser->inflight.smaller_timeout)
584 + parser->inflight.smaller_timeout = tmp.timeout_ut;
585 +
586 + // garbage collect stale inflight functions
587 + if(parser->inflight.smaller_timeout < now)
588 + inflight_functions_garbage_collect(parser, now);
589 +
590 + dictionary_write_unlock(parser->inflight.functions);
591 +
592 + return HTTP_RESP_OK;
593 +}
594 +
595 +PARSER_RC pluginsd_function(char **words, void *user, PLUGINSD_ACTION *plugins_action __maybe_unused)
596 +{
597 + bool global = false;
598 + int i = 1;
599 + if(strcmp(words[i], "GLOBAL") == 0) {
600 + i++;
601 + global = true;
602 + }
603 +
604 + char *name = words[i++];
605 + char *timeout_s = words[i++];
606 + char *help = words[i++];
607 +
608 + RRDSET *st = (global)?NULL:((PARSER_USER_OBJECT *) user)->st;
609 + RRDHOST *host = ((PARSER_USER_OBJECT *) user)->host;
610 +
611 + if (unlikely(!host || !timeout_s || !name || !help || (!global && !st))) {
612 + error("requested a FUNCTION, without providing the required data (global = '%s', name = '%s', timeout = '%s', help = '%s'), host '%s', chart '%s'. Ignoring it.",
613 + global?"yes":"no",
614 + name?name:"(unset)",
615 + timeout_s?timeout_s:"(unset)",
616 + help?help:"(unset)",
617 + host?rrdhost_hostname(host):"(unset)",
618 + st?rrdset_id(st):"(unset)");
619 + return PARSER_RC_OK;
620 + }
621 +
622 + int timeout = PLUGINS_FUNCTIONS_TIMEOUT_DEFAULT;
623 + if (timeout_s && *timeout_s) {
624 + timeout = str2i(timeout_s);
625 + if (unlikely(timeout <= 0))
626 + timeout = PLUGINS_FUNCTIONS_TIMEOUT_DEFAULT;
627 + }
628 +
629 + PARSER *parser = ((PARSER_USER_OBJECT *) user)->parser;
630 + rrd_collector_add_function(host, st, name, timeout, help, false, pluginsd_execute_function_callback, parser);
631 +
632 + return PARSER_RC_OK;
633 +}
634 +
635 +static void pluginsd_function_result_end(struct parser *parser, void *action_data) {
636 + STRING *key = action_data;
637 + if(key)
638 + dictionary_del(parser->inflight.functions, string2str(key));
639 + string_freez(key);
640 +}
641 +
642 +PARSER_RC pluginsd_function_result_begin(char **words, void *user, PLUGINSD_ACTION *plugins_action __maybe_unused)
643 +{
644 + char *key = words[1];
645 + char *status = words[2];
646 + char *format = words[3];
647 + char *expires = words[4];
648 +
649 + if (unlikely(!key || !*key || !status || !*status || !format || !*format || !expires || !*expires)) {
650 + error("got a " PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN " without providing the required data (key = '%s', status = '%s', format = '%s', expires = '%s')."
651 + , key ? key : "(unset)"
652 + , status ? status : "(unset)"
653 + , format ? format : "(unset)"
654 + , expires ? expires : "(unset)"
655 + );
656 + }
657 +
658 + int code = (status && *status) ? str2i(status) : 0;
659 + if (code <= 0)
660 + code = HTTP_RESP_BACKEND_RESPONSE_INVALID;
661 +
662 + time_t expiration = (expires && *expires) ? str2l(expires) : 0;
663 +
664 + PARSER *parser = ((PARSER_USER_OBJECT *) user)->parser;
665 +
666 + struct inflight_function *pf = NULL;
667 +
668 + if(key && *key)
669 + pf = (struct inflight_function *)dictionary_get(parser->inflight.functions, key);
670 +
671 + if(!pf) {
672 + error("got a " PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN " for transaction '%s', but the transaction is not found.", key?key:"(unset)");
673 + }
674 + else {
675 + if(format && *format)
676 + pf->destination_wb->contenttype = functions_format_to_content_type(format);
677 +
678 + pf->code = code;
679 +
680 + pf->destination_wb->expires = expiration;
681 + if(expiration <= now_realtime_sec())
682 + buffer_no_cacheable(pf->destination_wb);
683 + else
684 + buffer_cacheable(pf->destination_wb);
685 + }
686 +
687 + parser->defer.response = (pf) ? pf->destination_wb : NULL;
688 + parser->defer.end_keyword = PLUGINSD_KEYWORD_FUNCTION_RESULT_END;
689 + parser->defer.action = pluginsd_function_result_end;
690 + parser->defer.action_data = string_strdupz(key); // it is ok is key is NULL
691 + parser->flags |= PARSER_DEFER_UNTIL_KEYWORD;
692 +
693 + return PARSER_RC_OK;
694 +}
695 +
696 +// ----------------------------------------------------------------------------
697 +
698 PARSER_RC pluginsd_variable(char **words, void *user, PLUGINSD_ACTION *plugins_action)
699 {
700 char *name = words[1];
@@ -603,55 +824,58 @@ PARSER_RC pluginsd_label(char **words, void *user, PLUGINSD_ACTION *plugins_act
824 return PARSER_RC_OK;
825 }
826
606 -PARSER_RC pluginsd_clabel(char **words, void *user, PLUGINSD_ACTION *plugins_action)
607 -{
608 - if (!words[1] || !words[2] || !words[3]) {
609 - error("Ignoring malformed or empty CHART LABEL command.");
610 - return PARSER_RC_OK;
611 - }
612 -
613 - if (plugins_action->clabel_action) {
614 - PARSER_RC rc = plugins_action->clabel_action(user, words[1], words[2], strtol(words[3], NULL, 10));
615 - return rc;
616 - }
617 -
618 - return PARSER_RC_OK;
619 -}
620 -
621 -PARSER_RC pluginsd_clabel_commit(char **words, void *user, PLUGINSD_ACTION *plugins_action)
827 +PARSER_RC pluginsd_overwrite(char **words, void *user, PLUGINSD_ACTION *plugins_action)
828 {
829 UNUSED(words);
830
831 RRDHOST *host = ((PARSER_USER_OBJECT *) user)->host;
626 - debug(D_PLUGINSD, "requested to commit chart labels");
832 + debug(D_PLUGINSD, "requested to OVERWRITE host labels");
833
834 PARSER_RC rc = PARSER_RC_OK;
835
630 - if (plugins_action->clabel_commit_action)
631 - rc = plugins_action->clabel_commit_action(user, host, ((PARSER_USER_OBJECT *)user)->new_chart_labels);
836 + if (plugins_action->overwrite_action)
837 + rc = plugins_action->overwrite_action(user, host, ((PARSER_USER_OBJECT *)user)->new_host_labels);
838
633 - rrdlabels_destroy(((PARSER_USER_OBJECT *)user)->new_chart_labels);
634 - ((PARSER_USER_OBJECT *)user)->new_chart_labels = NULL;
839 + rrdlabels_destroy(((PARSER_USER_OBJECT *)user)->new_host_labels);
840 + ((PARSER_USER_OBJECT *)user)->new_host_labels = NULL;
841
842 return rc;
843 }
844
639 -PARSER_RC pluginsd_overwrite(char **words, void *user, PLUGINSD_ACTION *plugins_action)
845 +
846 +PARSER_RC pluginsd_clabel(char **words, void *user, PLUGINSD_ACTION *plugins_action __maybe_unused)
847 +{
848 + if (!words[1] || !words[2] || !words[3]) {
849 + error("Ignoring malformed or empty CHART LABEL command.");
850 + return PARSER_RC_OK;
851 + }
852 +
853 + if(unlikely(!((PARSER_USER_OBJECT *) user)->chart_rrdlabels_linked_temporarily)) {
854 + ((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily = ((PARSER_USER_OBJECT *)user)->st->rrdlabels;
855 + rrdlabels_unmark_all(((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily);
856 + }
857 +
858 + rrdlabels_add(((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily, words[1], words[2], strtol(words[3], NULL, 10));
859 +
860 + return PARSER_RC_OK;
861 +}
862 +
863 +PARSER_RC pluginsd_clabel_commit(char **words, void *user, PLUGINSD_ACTION *plugins_action __maybe_unused)
864 {
865 UNUSED(words);
866
867 RRDHOST *host = ((PARSER_USER_OBJECT *) user)->host;
644 - debug(D_PLUGINSD, "requested to OVERWRITE host labels");
645 -
646 - PARSER_RC rc = PARSER_RC_OK;
868 + debug(D_PLUGINSD, "requested to commit chart labels");
869
648 - if (plugins_action->overwrite_action)
649 - rc = plugins_action->overwrite_action(user, host, ((PARSER_USER_OBJECT *)user)->new_host_labels);
870 + if(!((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily) {
871 + error("requested CLABEL_COMMIT on host '%s', without a BEGIN, ignoring it.", rrdhost_hostname(host));
872 + return PARSER_RC_OK;
873 + }
874
651 - rrdlabels_destroy(((PARSER_USER_OBJECT *)user)->new_host_labels);
652 - ((PARSER_USER_OBJECT *)user)->new_host_labels = NULL;
875 + rrdlabels_remove_all_unmarked(((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily);
876
654 - return rc;
877 + ((PARSER_USER_OBJECT *)user)->chart_rrdlabels_linked_temporarily = NULL;
878 + return PARSER_RC_OK;
879 }
880
881 PARSER_RC pluginsd_guid(char **words, void *user, PLUGINSD_ACTION *plugins_action)
@@ -749,26 +973,35 @@ PARSER_RC metalog_pluginsd_host(char **words, void *user, PLUGINSD_ACTION *plug
973
974 static void pluginsd_process_thread_cleanup(void *ptr) {
975 PARSER *parser = (PARSER *)ptr;
976 + rrd_collector_finished();
977 parser_destroy(parser);
978 }
979
980 // New plugins.d parser
981
757 -inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp, int trust_durations)
982 +inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugin_input, FILE *fp_plugin_output, int trust_durations)
983 {
984 int enabled = cd->enabled;
985
761 - if (!fp || !enabled) {
986 + if (!fp_plugin_input || !fp_plugin_output || !enabled) {
987 cd->enabled = 0;
988 return 0;
989 }
990
766 - if (unlikely(fileno(fp) == -1)) {
767 - error("file descriptor given is not a valid stream");
991 + if (unlikely(fileno(fp_plugin_input) == -1)) {
992 + error("input file descriptor given is not a valid stream");
993 + cd->serial_failures++;
994 + return 0;
995 + }
996 +
997 + if (unlikely(fileno(fp_plugin_output) == -1)) {
998 + error("output file descriptor given is not a valid stream");
999 cd->serial_failures++;
1000 return 0;
1001 }
771 - clearerr(fp);
1002 +
1003 + clearerr(fp_plugin_input);
1004 + clearerr(fp_plugin_output);
1005
1006 PARSER_USER_OBJECT user = {
1007 .enabled = cd->enabled,
@@ -777,25 +1010,15 @@ inline size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp, int
1010 .trust_durations = trust_durations
1011 };
1012
780 - PARSER *parser = parser_init(host, &user, fp, PARSER_INPUT_SPLIT);
1013 + // fp_plugin_output = our input; fp_plugin_input = our output
1014 + PARSER *parser = parser_init(host, &user, fp_plugin_output, fp_plugin_input, PARSER_INPUT_SPLIT);
1015 +
1016 + rrd_collector_started();
1017
1018 // this keeps the parser with its current value
1019 // so, parser needs to be allocated before pushing it
1020 netdata_thread_cleanup_push(pluginsd_process_thread_cleanup, parser);
1021
786 - parser->plugins_action->begin_action = &pluginsd_begin_action;
787 - parser->plugins_action->flush_action = &pluginsd_flush_action;
788 - parser->plugins_action->end_action = &pluginsd_end_action;
789 - parser->plugins_action->disable_action = &pluginsd_disable_action;
790 - parser->plugins_action->variable_action = &pluginsd_variable_action;
791 - parser->plugins_action->dimension_action = &pluginsd_dimension_action;
792 - parser->plugins_action->label_action = &pluginsd_label_action;
793 - parser->plugins_action->overwrite_action = &pluginsd_overwrite_action;
794 - parser->plugins_action->chart_action = &pluginsd_chart_action;
795 - parser->plugins_action->set_action = &pluginsd_set_action;
796 - parser->plugins_action->clabel_commit_action = &pluginsd_clabel_commit_action;
797 - parser->plugins_action->clabel_action = &pluginsd_clabel_action;
798 -
1022 user.parser = parser;
1023
1024 while (likely(!parser_next(parser))) {
collectors/plugins.d/pluginsd_parser.h
+4 -3
@@ -14,7 +14,7 @@ typedef struct parser_user_object {
14 struct plugind *cd;
15 int trust_durations;
16 DICTIONARY *new_host_labels;
17 - DICTIONARY *new_chart_labels;
17 + DICTIONARY *chart_rrdlabels_linked_temporarily;
18 size_t count;
19 int enabled;
20 uint8_t st_exists;
@@ -35,8 +35,9 @@ extern PARSER_RC pluginsd_dimension_action(void *user, RRDSET *st, char *id, cha
35 long multiplier, long divisor, char *options, RRD_ALGORITHM algorithm_type);
36 extern PARSER_RC pluginsd_label_action(void *user, char *key, char *value, RRDLABEL_SRC source);
37 extern PARSER_RC pluginsd_overwrite_action(void *user, RRDHOST *host, DICTIONARY *new_host_labels);
38 -extern PARSER_RC pluginsd_clabel_commit_action(void *user, RRDHOST *host, DICTIONARY *new_chart_labels);
39 -extern PARSER_RC pluginsd_clabel_action(void *user, char *key, char *value, RRDLABEL_SRC source);
38
39 +extern PARSER_RC pluginsd_function(char **words, void *user, PLUGINSD_ACTION *plugins_action);
40 +extern PARSER_RC pluginsd_function_result_begin(char **words, void *user, PLUGINSD_ACTION *plugins_action);
41 +extern void inflight_functions_init(PARSER *parser);
42
43 #endif //NETDATA_PLUGINSD_PARSER_H
collectors/tc.plugin/plugin_tc.c
+5 -5
@@ -940,21 +940,21 @@ void *tc_main(void *ptr) {
940 char *tc_script = config_get("plugin:tc", "script to run to get tc values", command);
941
942 while(!netdata_exit) {
943 - FILE *fp;
943 + FILE *fp_child_input, *fp_child_output;
944 struct tc_device *device = NULL;
945 struct tc_class *class = NULL;
946
947 snprintfz(command, TC_LINE_MAX, "exec %s %d", tc_script, localhost->rrd_update_every);
948 debug(D_TC_LOOP, "executing '%s'", command);
949
950 - fp = mypopen(command, (pid_t *)&tc_child_pid);
951 - if(unlikely(!fp)) {
950 + fp_child_output = netdata_popen(command, (pid_t *)&tc_child_pid, &fp_child_input);
951 + if(unlikely(!fp_child_output)) {
952 error("TC: Cannot popen(\"%s\", \"r\").", command);
953 goto cleanup;
954 }
955
956 char buffer[TC_LINE_MAX+1] = "";
957 - while(fgets(buffer, TC_LINE_MAX, fp) != NULL) {
957 + while(fgets(buffer, TC_LINE_MAX, fp_child_output) != NULL) {
958 if(unlikely(netdata_exit)) break;
959
960 buffer[TC_LINE_MAX] = '\0';
@@ -1163,7 +1163,7 @@ void *tc_main(void *ptr) {
1163 }
1164
1165 // fgets() failed or loop broke
1166 - int code = mypclose(fp, (pid_t)tc_child_pid);
1166 + int code = netdata_pclose(fp_child_input, fp_child_output, (pid_t)tc_child_pid);
1167 tc_child_pid = 0;
1168
1169 if(unlikely(device)) {
daemon/analytics.c
+12 -10
@@ -337,11 +337,12 @@ void analytics_alarms_notifications(void)
337
338 BUFFER *b = buffer_create(1000);
339 int cnt = 0;
340 - FILE *fp = mypopen(script, &command_pid);
341 - if (fp) {
340 + FILE *fp_child_input;
341 + FILE *fp_child_output = netdata_popen(script, &command_pid, &fp_child_input);
342 + if (fp_child_output) {
343 char line[200 + 1];
344
344 - while (fgets(line, 200, fp) != NULL) {
345 + while (fgets(line, 200, fp_child_output) != NULL) {
346 char *end = line;
347 while (*end && *end != '\n')
348 end++;
@@ -354,7 +355,7 @@ void analytics_alarms_notifications(void)
355
356 cnt++;
357 }
357 - mypclose(fp, command_pid);
358 + netdata_pclose(fp_child_input, fp_child_output, command_pid);
359 }
360 freez(script);
361
@@ -384,8 +385,8 @@ void analytics_https(void)
385 BUFFER *b = buffer_create(30);
386 #ifdef ENABLE_HTTPS
387 analytics_exporting_connectors_ssl(b);
387 - buffer_strcat(b, netdata_client_ctx && localhost->ssl.flags == NETDATA_SSL_HANDSHAKE_COMPLETE && __atomic_load_n(&localhost->rrdpush_sender_connected, __ATOMIC_SEQ_CST) ? "streaming|" : "|");
388 - buffer_strcat(b, netdata_srv_ctx ? "web" : "");
388 + buffer_strcat(b, netdata_ssl_client_ctx && rrdhost_flag_check(localhost, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED) && localhost->sender->ssl.flags == NETDATA_SSL_HANDSHAKE_COMPLETE ? "streaming|" : "|");
389 + buffer_strcat(b, netdata_ssl_srv_ctx ? "web" : "");
390 #else
391 buffer_strcat(b, "||");
392 #endif
@@ -1016,11 +1017,12 @@ void send_statistics(const char *action, const char *action_result, const char *
1017
1018 info("%s '%s' '%s' '%s'", as_script, action, action_result, action_data);
1019
1019 - FILE *fp = mypopen(command_to_run, &command_pid);
1020 - if (fp) {
1020 + FILE *fp_child_input;
1021 + FILE *fp_child_output = netdata_popen(command_to_run, &command_pid, &fp_child_input);
1022 + if (fp_child_output) {
1023 char buffer[4 + 1];
1022 - char *s = fgets(buffer, 4, fp);
1023 - int exit_code = mypclose(fp, command_pid);
1024 + char *s = fgets(buffer, 4, fp_child_output);
1025 + int exit_code = netdata_pclose(fp_child_input, fp_child_output, command_pid);
1026 if (exit_code)
1027 error("Execution of anonymous statistics script returned %d.", exit_code);
1028 if (s && strncmp(buffer, "200", 3))
daemon/main.c
+7 -6
@@ -379,10 +379,10 @@ int help(int exitcode) {
379 static void security_init(){
380 char filename[FILENAME_MAX + 1];
381 snprintfz(filename, FILENAME_MAX, "%s/ssl/key.pem",netdata_configured_user_config_dir);
382 - security_key = config_get(CONFIG_SECTION_WEB, "ssl key", filename);
382 + ssl_security_key = config_get(CONFIG_SECTION_WEB, "ssl key", filename);
383
384 snprintfz(filename, FILENAME_MAX, "%s/ssl/cert.pem",netdata_configured_user_config_dir);
385 - security_cert = config_get(CONFIG_SECTION_WEB, "ssl certificate", filename);
385 + ssl_security_cert = config_get(CONFIG_SECTION_WEB, "ssl certificate", filename);
386
387 tls_version = config_get(CONFIG_SECTION_WEB, "tls version", "1.3");
388 tls_ciphers = config_get(CONFIG_SECTION_WEB, "tls ciphers", "none");
@@ -795,12 +795,13 @@ int get_system_info(struct rrdhost_system_info *system_info) {
795
796 info("Executing %s", script);
797
798 - FILE *fp = mypopen(script, &command_pid);
799 - if(fp) {
798 + FILE *fp_child_input;
799 + FILE *fp_child_output = netdata_popen(script, &command_pid, &fp_child_input);
800 + if(fp_child_output) {
801 char line[200 + 1];
802 // Removed the double strlens, if the Coverity tainted string warning reappears I'll revert.
803 // One time init code, but I'm curious about the warning...
803 - while (fgets(line, 200, fp) != NULL) {
804 + while (fgets(line, 200, fp_child_output) != NULL) {
805 char *value=line;
806 while (*value && *value != '=') value++;
807 if (*value=='=') {
@@ -821,7 +822,7 @@ int get_system_info(struct rrdhost_system_info *system_info) {
822 }
823 }
824 }
824 - mypclose(fp, command_pid);
825 + netdata_pclose(fp_child_input, fp_child_output, command_pid);
826 }
827 freez(script);
828 return 0;
daemon/service.c
+27 -12
@@ -5,6 +5,9 @@
5 /* Run service jobs every X seconds */
6 #define SERVICE_HEARTBEAT 10
7
8 +#define TIME_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT (3600 / 2)
9 +#define ITERATIONS_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT 60
10 +
11 #define WORKER_JOB_CHILD_CHART_OBSOLETION_CHECK 1
12 #define WORKER_JOB_CLEANUP_OBSOLETE_CHARTS 2
13 #define WORKER_JOB_ARCHIVE_CHART 3
@@ -69,43 +72,53 @@ static void svc_rrddim_obsolete_to_archive(RRDDIM *rd) {
72 rrddim_free(st, rd);
73 }
74
72 -static void svc_rrdset_archive_obsolete_dimensions(RRDSET *st, bool all_dimensions) {
75 +static bool svc_rrdset_archive_obsolete_dimensions(RRDSET *st, bool all_dimensions) {
76 worker_is_busy(WORKER_JOB_ARCHIVE_CHART_DIMENSIONS);
77
78 RRDDIM *rd;
79 time_t now = now_realtime_sec();
80
78 - dfe_start_reentrant(st->rrddim_root_index, rd) {
81 + bool done_all_dimensions = true;
82 +
83 + dfe_start_write(st->rrddim_root_index, rd) {
84 if(unlikely(
85 all_dimensions ||
86 (rrddim_flag_check(rd, RRDDIM_FLAG_OBSOLETE) && (rd->last_collected_time.tv_sec + rrdset_free_obsolete_time < now))
87 )) {
88
84 - info("Removing obsolete dimension '%s' (%s) of '%s' (%s).", rrddim_name(rd), rrddim_id(rd), rrdset_name(st), rrdset_id(st));
85 - svc_rrddim_obsolete_to_archive(rd);
86 -
89 + if(dictionary_acquired_item_references(rd_dfe.item) == 1) {
90 + info("Removing obsolete dimension '%s' (%s) of '%s' (%s).", rrddim_name(rd), rrddim_id(rd), rrdset_name(st), rrdset_id(st));
91 + svc_rrddim_obsolete_to_archive(rd);
92 + }
93 + else
94 + done_all_dimensions = false;
95 }
96 + else
97 + done_all_dimensions = false;
98 }
99 dfe_done(rd);
100 +
101 + return done_all_dimensions;
102 }
103
104 static void svc_rrdset_obsolete_to_archive(RRDSET *st) {
105 worker_is_busy(WORKER_JOB_ARCHIVE_CHART);
106
107 + if(!svc_rrdset_archive_obsolete_dimensions(st, true))
108 + return;
109 +
110 rrdset_flag_set(st, RRDSET_FLAG_ARCHIVED);
111 rrdset_flag_clear(st, RRDSET_FLAG_OBSOLETE);
112
113 rrdcalc_unlink_all_rrdset_alerts(st);
114
100 - svc_rrdset_archive_obsolete_dimensions(st, true);
101 -
115 rrdsetvar_release_and_delete_all(st);
116
117 // has to be run after all dimensions are archived - or use-after-free will occur
118 rrdvar_delete_all(st->rrdvars);
119
120 if(st->rrd_memory_mode != RRD_MEMORY_MODE_DBENGINE) {
108 - if(rrdhost_flag_check(st->rrdhost, RRDHOST_FLAG_DELETE_OBSOLETE_CHARTS)) {
121 + if(rrdhost_option_check(st->rrdhost, RRDHOST_OPTION_DELETE_OBSOLETE_CHARTS)) {
122 worker_is_busy(WORKER_JOB_DELETE_CHART);
123 rrdset_delete_files(st);
124 }
@@ -148,7 +161,10 @@ static void svc_rrdset_check_obsoletion(RRDHOST *host) {
161 rrdset_foreach_read(st, host) {
162 last_entry_t = rrdset_last_entry_t(st);
163
151 - if(last_entry_t && last_entry_t < host->senders_connect_time)
164 + if(last_entry_t && last_entry_t < host->senders_connect_time && host->senders_connect_time
165 + + TIME_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT + ITERATIONS_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT * st->update_every
166 + < now_realtime_sec())
167 +
168 rrdset_is_obsolete(st);
169
170 }
@@ -175,12 +191,11 @@ static void svc_rrd_cleanup_obsolete_charts_from_all_hosts() {
191 host->senders_last_chart_command
192 && host->senders_last_chart_command + host->health_delay_up_to < now_realtime_sec()
193 )
178 - || (host->senders_connect_time + 300 < now_realtime_sec())
194 + || (host->senders_connect_time + TIME_TO_RUN_OBSOLETIONS_ON_CHILD_CONNECT < now_realtime_sec())
195 )
196 ) {
197 svc_rrdset_check_obsoletion(host);
198 host->trigger_chart_obsoletion_check = 0;
183 -
199 }
200 }
201
@@ -200,7 +215,7 @@ restart_after_removal:
215 if(rrdhost_should_be_removed(host, protected_host, now)) {
216 info("Host '%s' with machine guid '%s' is obsolete - cleaning up.", rrdhost_hostname(host), host->machine_guid);
217
203 - if (rrdhost_flag_check(host, RRDHOST_FLAG_DELETE_ORPHAN_HOST)
218 + if (rrdhost_option_check(host, RRDHOST_OPTION_DELETE_ORPHAN_HOST)
219 #ifdef ENABLE_DBENGINE
220 /* don't delete multi-host DB host files */
221 && !(host->rrd_memory_mode == RRD_MEMORY_MODE_DBENGINE && is_storage_engine_shared(host->storage_instance[0]))
daemon/signals.c
+3 -3
@@ -82,7 +82,7 @@ void signals_init(void) {
82 // This prevents zombie processes when running in a container.
83 if (getpid() == 1) {
84 info("SIGNAL: Enabling reaper");
85 - myp_init();
85 + netdata_popen_tracking_init();
86 reaper_enabled = 1;
87 } else {
88 info("SIGNAL: Not enabling reaper");
@@ -139,7 +139,7 @@ void signals_reset(void) {
139 }
140
141 if (reaper_enabled == 1)
142 - myp_free();
142 + netdata_popen_tracking_cleanup();
143 }
144
145 // reap_child reaps the child identified by pid.
@@ -198,7 +198,7 @@ static void reap_children() {
198 } else if (i.si_pid == 0) {
199 // No child exited.
200 return;
201 - } else if (myp_reap(i.si_pid) == 0) {
201 + } else if (netdata_popen_tracking_pid_shoud_be_reaped(i.si_pid) == 0) {
202 // myp managed, sleep for a short time to avoid busy wait while
203 // this is handled by myp.
204 usleep(10000);
database/engine/metadata_log/logfile.c
+2 -2
@@ -369,7 +369,7 @@ static int scan_metalog_files(struct metalog_instance *ctx)
369 .obsolete = 0,
370 .started_t = INVALID_TIME,
371 .next = NULL,
372 - .version = 0,
372 + .capabilities = 0,
373 };
374
375 struct metalog_pluginsd_state metalog_parser_state;
@@ -383,7 +383,7 @@ static int scan_metalog_files(struct metalog_instance *ctx)
383 .private = &metalog_parser_state
384 };
385
386 - PARSER *parser = parser_init(metalog_parser_object.host, &metalog_parser_object, NULL, PARSER_INPUT_SPLIT);
386 + PARSER *parser = parser_init(metalog_parser_object.host, &metalog_parser_object, NULL, NULL, PARSER_INPUT_SPLIT|PARSER_NO_ACTION_INIT);
387 parser_add_keyword(parser, PLUGINSD_KEYWORD_HOST, metalog_pluginsd_host);
388 parser_add_keyword(parser, PLUGINSD_KEYWORD_GUID, pluginsd_guid);
389 parser_add_keyword(parser, PLUGINSD_KEYWORD_CONTEXT, pluginsd_context);
database/engine/pagecache.c
+1 -1
@@ -1251,7 +1251,7 @@ void free_page_cache(struct rrdengine_instance *ctx)
1251 // Do the cleanup if we are compiling with NETDATA_INTERNAL_CHECKS
1252 // This affects the reporting of dbengine statistics which are available in real time
1253 // via the /api/v1/dbengine_stats endpoint
1254 -#ifndef NETDATA_INTERNAL_CHECKS
1254 +#ifndef NETDATA_DBENGINE_FREE
1255 if (netdata_exit)
1256 return;
1257 #endif
database/rrd.h
+75 -62
@@ -178,20 +178,21 @@ typedef enum rrddim_options {
178 // this is 8-bit
179 } RRDDIM_OPTIONS;
180
181 -#define rrddim_option_check(rd, flag) ((rd)->flags & (flag))
182 -#define rrddim_option_set(rd, flag) (rd)->flags |= (flag)
183 -#define rrddim_option_clear(rd, flag) (rd)->flags &= ~(flag)
181 +#define rrddim_option_check(rd, option) ((rd)->options & (option))
182 +#define rrddim_option_set(rd, option) (rd)->options |= (option)
183 +#define rrddim_option_clear(rd, option) (rd)->options &= ~(option)
184
185 // flags are runtime changing status flags (atomics are required to alter/access them)
186 typedef enum rrddim_flags {
187 RRDDIM_FLAG_NONE = 0,
188 + RRDDIM_FLAG_PENDING_HEALTH_INITIALIZATION = (1 << 0),
189 +
190 RRDDIM_FLAG_OBSOLETE = (1 << 2), // this is marked by the collector/module as obsolete
191 // No new values have been collected for this dimension since agent start, or it was marked RRDDIM_FLAG_OBSOLETE at
192 // least rrdset_free_obsolete_time seconds ago.
193 RRDDIM_FLAG_ARCHIVED = (1 << 3),
194 RRDDIM_FLAG_ACLK = (1 << 4),
195
194 - RRDDIM_FLAG_PENDING_FOREACH_ALARMS = (1 << 5), // set when foreach alarm has not been initialized yet
196 RRDDIM_FLAG_META_HIDDEN = (1 << 6), // Status of hidden option in the metadata database
197
198 // this is 8 bit
@@ -216,6 +217,8 @@ typedef enum rrdlabel_source {
217
218 #define RRDLABEL_FLAG_INTERNAL (RRDLABEL_FLAG_OLD | RRDLABEL_FLAG_NEW | RRDLABEL_FLAG_PERMANENT)
219
220 +extern size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length);
221 +
222 extern DICTIONARY *rrdlabels_create(void);
223 extern void rrdlabels_destroy(DICTIONARY *labels_dict);
224 extern void rrdlabels_add(DICTIONARY *dict, const char *name, const char *value, RRDLABEL_SRC ls);
@@ -491,13 +494,12 @@ typedef enum rrdset_flags {
494 // No new values have been collected for this chart since agent start, or it was marked RRDSET_FLAG_OBSOLETE at
495 // least rrdset_free_obsolete_time seconds ago.
496 RRDSET_FLAG_ARCHIVED = (1 << 15),
494 -// RRDSET_FLAG_ACLK = (1 << 16), // not used anymore
495 - RRDSET_FLAG_PENDING_FOREACH_ALARMS = (1 << 17), // contains dims with uninitialized foreach alarms
497 RRDSET_FLAG_ANOMALY_DETECTION = (1 << 18), // flag to identify anomaly detection charts.
498 RRDSET_FLAG_INDEXED_ID = (1 << 19), // the rrdset is indexed by its id
499 RRDSET_FLAG_INDEXED_NAME = (1 << 20), // the rrdset is indexed by its name
500
501 RRDSET_FLAG_ANOMALY_RATE_CHART = (1 << 21), // the rrdset is for storing anomaly rates for all dimensions
502 + RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION = (1 << 22),
503 } RRDSET_FLAGS;
504
505 #define rrdset_flag_check(st, flag) (__atomic_load_n(&((st)->flags), __ATOMIC_SEQ_CST) & (flag))
@@ -548,9 +550,6 @@ struct rrdset {
550 RRDSET_FLAGS flags; // flags
551 RRD_MEMORY_MODE rrd_memory_mode; // the db mode of this rrdset
552
551 - uuid_t hash_uuid; // hash_id for syncing with cloud
552 - // TODO - obsolete now - cleanup
553 -
553 DICTIONARY *rrddim_root_index; // dimensions index
554
555 int gap_when_lost_iterations_above; // after how many lost iterations a gap should be stored
@@ -583,6 +582,8 @@ struct rrdset {
582
583 size_t rrdlabels_last_saved_version;
584
585 + DICTIONARY *functions_view; // collector functions this rrdset supports, can be NULL
586 +
587 // ------------------------------------------------------------------------
588 // data collection - streaming to parents, temp variables
589
@@ -684,6 +685,8 @@ extern void rrdset_memory_file_update(RRDSET *st);
685 extern const char *rrdset_cache_filename(RRDSET *st);
686 extern bool rrdset_memory_load_or_create_map_save(RRDSET *st_on_file, RRD_MEMORY_MODE memory_mode);
687
688 +#include "rrdfunctions.h"
689 +
690 // ----------------------------------------------------------------------------
691 // RRDHOST flags
692 // use this for configuration flags, not for state control
@@ -691,23 +694,30 @@ extern bool rrdset_memory_load_or_create_map_save(RRDSET *st_on_file, RRD_MEMORY
694 // and may lead to missing information.
695
696 typedef enum rrdhost_flags {
694 - RRDHOST_FLAG_ORPHAN = (1 << 0), // this host is orphan (not receiving data)
695 - RRDHOST_FLAG_DELETE_OBSOLETE_CHARTS = (1 << 1), // delete files of obsolete charts
696 - RRDHOST_FLAG_DELETE_ORPHAN_HOST = (1 << 2), // delete the entire host when orphan
697 - RRDHOST_FLAG_EXPORTING_SEND = (1 << 3), // send it to external databases
698 - RRDHOST_FLAG_EXPORTING_DONT_SEND = (1 << 4), // don't send it to external databases
699 - RRDHOST_FLAG_ARCHIVED = (1 << 5), // The host is archived, no collected charts yet
700 - RRDHOST_FLAG_PENDING_FOREACH_ALARMS = (1 << 7), // contains dims with uninitialized foreach alarms
701 - RRDHOST_FLAG_STREAM_LABELS_UPDATE = (1 << 8),
702 - RRDHOST_FLAG_STREAM_LABELS_STOP = (1 << 9),
703 - RRDHOST_FLAG_ACLK_STREAM_CONTEXTS = (1 << 10), // when set, we should send ACLK stream context updates
704 - RRDHOST_FLAG_INDEXED_MACHINE_GUID = (1 << 11), // when set, we have indexed its machine guid
705 - RRDHOST_FLAG_INDEXED_HOSTNAME = (1 << 12), // when set, we have indexed its hostname
706 - RRDHOST_FLAG_STREAM_COLLECTED_METRICS = (1 << 13), // when set, rrdset_done() should push metrics to parent
707 - RRDHOST_FLAG_INITIALIZED_HEALTH = (1 << 14), // the host has initialized health structures
708 - RRDHOST_FLAG_INITIALIZED_RRDPUSH = (1 << 15), // the host has initialized rrdpush structures
709 - RRDHOST_FLAG_PENDING_OBSOLETE_CHARTS = (1 << 16), // the host has pending chart obsoletions
710 - RRDHOST_FLAG_PENDING_OBSOLETE_DIMENSIONS = (1 << 17), // the host has pending dimension obsoletions
697 + // Orphan, Archived and Obsolete flags
698 + RRDHOST_FLAG_ORPHAN = (1 << 10), // this host is orphan (not receiving data)
699 + RRDHOST_FLAG_ARCHIVED = (1 << 11), // The host is archived, no collected charts yet
700 + RRDHOST_FLAG_PENDING_OBSOLETE_CHARTS = (1 << 12), // the host has pending chart obsoletions
701 + RRDHOST_FLAG_PENDING_OBSOLETE_DIMENSIONS = (1 << 13), // the host has pending dimension obsoletions
702 +
703 + // Streaming sender
704 + RRDHOST_FLAG_RRDPUSH_SENDER_INITIALIZED = (1 << 14), // the host has initialized rrdpush structures
705 + RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN = (1 << 15), // When set, the sender thread is running
706 + RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED = (1 << 16), // When set, the host is connected to a parent
707 + RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS = (1 << 17), // when set, rrdset_done() should push metrics to parent
708 + RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS = (1 << 18), // when set, we have logged the status of metrics streaming
709 + RRDHOST_FLAG_RRDPUSH_SENDER_JOIN = (1 << 19), // When set, we want to join the sender thread
710 +
711 + // Health
712 + RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION = (1 << 20), // contains charts and dims with uninitialized variables
713 + RRDHOST_FLAG_INITIALIZED_HEALTH = (1 << 21), // the host has initialized health structures
714 +
715 + // Exporting
716 + RRDHOST_FLAG_EXPORTING_SEND = (1 << 22), // send it to external databases
717 + RRDHOST_FLAG_EXPORTING_DONT_SEND = (1 << 23), // don't send it to external databases
718 +
719 + // ACLK
720 + RRDHOST_FLAG_ACLK_STREAM_CONTEXTS = (1 << 24), // when set, we should send ACLK stream context updates
721 } RRDHOST_FLAGS;
722
723 #define rrdhost_flag_check(host, flag) (__atomic_load_n(&((host)->flags), __ATOMIC_SEQ_CST) & (flag))
@@ -721,6 +731,27 @@ typedef enum rrdhost_flags {
731 #define rrdset_debug(st, fmt, args...) debug_dummy()
732 #endif
733
734 +typedef enum {
735 + // Indexing
736 + RRDHOST_OPTION_INDEXED_MACHINE_GUID = (1 << 0), // when set, we have indexed its machine guid
737 + RRDHOST_OPTION_INDEXED_HOSTNAME = (1 << 1), // when set, we have indexed its hostname
738 +
739 + // Streaming configuration
740 + RRDHOST_OPTION_SENDER_ENABLED = (1 << 2), // set when the host is configured to send metrics to a parent
741 +
742 + // Configuration options
743 + RRDHOST_OPTION_DELETE_OBSOLETE_CHARTS = (1 << 3), // delete files of obsolete charts
744 + RRDHOST_OPTION_DELETE_ORPHAN_HOST = (1 << 4), // delete the entire host when orphan
745 +} RRDHOST_OPTIONS;
746 +
747 +#define rrdhost_option_check(host, flag) ((host)->options & (flag))
748 +#define rrdhost_option_set(host, flag) (host)->options |= flag
749 +#define rrdhost_option_clear(host, flag) (host)->options &= ~(flag)
750 +
751 +#define rrdhost_has_rrdpush_sender_enabled(host) (rrdhost_option_check(host, RRDHOST_OPTION_SENDER_ENABLED) && (host)->sender)
752 +
753 +#define rrdhost_can_send_definitions_to_parent(host) (rrdhost_has_rrdpush_sender_enabled(host) && rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED))
754 +
755 // ----------------------------------------------------------------------------
756 // Health data
757
@@ -860,7 +891,8 @@ struct rrdhost {
891
892 int32_t utc_offset; // the offset in seconds from utc
893
863 - RRDHOST_FLAGS flags; // flags about this RRDHOST
894 + RRDHOST_OPTIONS options; // configuration option for this RRDHOST (no atomics on this)
895 + RRDHOST_FLAGS flags; // runtime flags about this RRDHOST (atomics on this)
896 RRDHOST_FLAGS *exporting_flags; // array of flags for exporting connector instances
897
898 int rrd_update_every; // the update frequency of the host
@@ -873,36 +905,22 @@ struct rrdhost {
905 struct rrdhost_system_info *system_info; // information collected from the host environment
906
907 // ------------------------------------------------------------------------
876 - // streaming of data to remote hosts - rrdpush
908 + // streaming of data to remote hosts - rrdpush sender
909
878 - unsigned int rrdpush_send_enabled; // 1 when this host sends metrics to another netdata
910 char *rrdpush_send_destination; // where to send metrics to
911 char *rrdpush_send_api_key; // the api key at the receiving netdata
912 struct rrdpush_destinations *destinations; // a linked list of possible destinations
913 struct rrdpush_destinations *destination; // the current destination from the above list
914 + SIMPLE_PATTERN *rrdpush_send_charts_matching; // pattern to match the charts to be sent
915
916 // the following are state information for the threading
917 // streaming metrics from this netdata to an upstream netdata
918 struct sender_state *sender;
887 - volatile unsigned int rrdpush_sender_spawn; // 1 when the sender thread has been spawn
919 netdata_thread_t rrdpush_sender_thread; // the sender thread
920 void *dbsync_worker;
921
891 - bool rrdpush_sender_connected; // 1 when the sender is ready to push metrics
892 - int rrdpush_sender_socket; // the fd of the socket to the remote host, or -1
893 -
894 - volatile unsigned int rrdpush_sender_error_shown; // 1 when we have logged a communication error
895 - volatile unsigned int rrdpush_sender_join; // 1 when we have to join the sending thread
896 -
897 - SIMPLE_PATTERN *rrdpush_send_charts_matching; // pattern to match the charts to be sent
898 -
899 - int rrdpush_sender_pipe[2]; // collector to sender thread signaling
900 - //BUFFER *rrdpush_sender_buffer; // collector fills it, sender sends it
901 -
902 - //uint32_t stream_version; //Set the current version of the stream.
903 -
922 // ------------------------------------------------------------------------
905 - // streaming of data from remote hosts - rrdpush
923 + // streaming of data from remote hosts - rrdpush receiver
924
925 time_t senders_connect_time; // the time the last sender was connected
926 time_t senders_last_chart_command; // the time of the last CHART streaming command
@@ -916,25 +934,21 @@ struct rrdhost {
934 // ------------------------------------------------------------------------
935 // health monitoring options
936
919 - unsigned int health_enabled; // 1 when this host has health enabled
920 - time_t health_delay_up_to; // a timestamp to delay alarms processing up to
921 - STRING *health_default_exec; // the full path of the alarms notifications program
922 - STRING *health_default_recipient; // the default recipient for all alarms
923 - char *health_log_filename; // the alarms event log filename
924 - size_t health_log_entries_written; // the number of alarm events written to the alarms event log
925 - FILE *health_log_fp; // the FILE pointer to the open alarms event log file
926 - uint32_t health_default_warn_repeat_every; // the default value for the interval between repeating warning notifications
927 - uint32_t health_default_crit_repeat_every; // the default value for the interval between repeating critical notifications
937 + unsigned int health_enabled; // 1 when this host has health enabled
938 + time_t health_delay_up_to; // a timestamp to delay alarms processing up to
939 + STRING *health_default_exec; // the full path of the alarms notifications program
940 + STRING *health_default_recipient; // the default recipient for all alarms
941 + char *health_log_filename; // the alarms event log filename
942 + size_t health_log_entries_written; // the number of alarm events written to the alarms event log
943 + FILE *health_log_fp; // the FILE pointer to the open alarms event log file
944 + uint32_t health_default_warn_repeat_every; // the default value for the interval between repeating warning notifications
945 + uint32_t health_default_crit_repeat_every; // the default value for the interval between repeating critical notifications
946
947
948 // all RRDCALCs are primarily allocated and linked here
931 - // RRDCALCs may be linked to charts at any point
932 - // (charts may or may not exist when these are loaded)
949 DICTIONARY *rrdcalc_root_index;
950
951 // templates of alarms
936 - // these are used to create alarms when charts
937 - // are created or renamed, that match them
952 DICTIONARY *rrdcalctemplate_root_index;
953
954 ALARM_LOG health_log; // alarms historical events (event log)
@@ -955,6 +969,10 @@ struct rrdhost {
969 // Support for host-level labels
970 DICTIONARY *rrdlabels;
971
972 + // ------------------------------------------------------------------------
973 + // Support for functions
974 + DICTIONARY *functions; // collector functions this rrdset supports, can be NULL
975 +
976 // ------------------------------------------------------------------------
977 // indexes
978
@@ -974,11 +992,6 @@ struct rrdhost {
992 uuid_t host_uuid; // Global GUID for this host
993 uuid_t *node_id; // Cloud node_id
994
977 -#ifdef ENABLE_HTTPS
978 - struct netdata_ssl ssl; //Structure used to encrypt the connection
979 - struct netdata_ssl stream_ssl; //Structure used to encrypt the stream
980 -#endif
981 -
995 netdata_mutex_t aclk_state_lock;
996 aclk_rrdhost_state aclk_state;
997
database/rrdcontext.c
+4 -1
@@ -11,6 +11,8 @@
11 #define RRDCONTEXT_WORKER_THREAD_HEARTBEAT_USEC (1000 * USEC_PER_MS)
12 #define RRDCONTEXT_MINIMUM_ALLOWED_PRIORITY 10
13
14 +#define LOG_TRANSITIONS false
15 +
16 #define WORKER_JOB_HOSTS 1
17 #define WORKER_JOB_CHECK 2
18 #define WORKER_JOB_SEND 3
@@ -2822,7 +2824,8 @@ static bool check_if_cloud_version_changed_unsafe(RRDCONTEXT *rc, bool sending _
2824
2825 if(unlikely(id_changed || title_changed || units_changed || family_changed || chart_type_changed || priority_changed || first_time_changed || last_time_changed || deleted_changed)) {
2826
2825 - internal_error(true, "RRDCONTEXT: %s NEW VERSION '%s'%s, version %"PRIu64", title '%s'%s, units '%s'%s, family '%s'%s, chart type '%s'%s, priority %u%s, first_time_t %ld%s, last_time_t %ld%s, deleted '%s'%s, (queued for %llu ms, expected %llu ms)",
2827 + internal_error(LOG_TRANSITIONS,
2828 + "RRDCONTEXT: %s NEW VERSION '%s'%s, version %"PRIu64", title '%s'%s, units '%s'%s, family '%s'%s, chart type '%s'%s, priority %u%s, first_time_t %ld%s, last_time_t %ld%s, deleted '%s'%s, (queued for %llu ms, expected %llu ms)",
2829 sending?"SENDING":"QUEUE",
2830 string2str(rc->id), id_changed ? " (CHANGED)" : "",
2831 rc->version,
database/rrddim.c
+11 -16
@@ -27,20 +27,6 @@ struct rrddim_constructor {
27
28 };
29
30 -static void rrddim_update_rrddimvars_unsafe(RRDDIM *rd) {
31 - RRDSET *st = rd->rrdset;
32 - RRDHOST *host = st->rrdhost;
33 -
34 - if(host->health_enabled && !rrdset_is_ar_chart(st)) {
35 - rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_CALCULATED, NULL, NULL, &rd->last_stored_value, RRDVAR_FLAG_NONE);
36 - rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_COLLECTED, NULL, "_raw", &rd->last_collected_value, RRDVAR_FLAG_NONE);
37 - rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_TIME_T, NULL, "_last_collected_t", &rd->last_collected_time.tv_sec, RRDVAR_FLAG_NONE);
38 - rrddim_flag_set(rd, RRDDIM_FLAG_PENDING_FOREACH_ALARMS);
39 - rrdset_flag_set(st, RRDSET_FLAG_PENDING_FOREACH_ALARMS);
40 - rrdhost_flag_set(host, RRDHOST_FLAG_PENDING_FOREACH_ALARMS);
41 - }
42 -}
43 -
30 static void rrddim_insert_callback(const DICTIONARY_ITEM *item __maybe_unused, void *rrddim, void *constructor_data) {
31 struct rrddim_constructor *ctr = constructor_data;
32 RRDDIM *rd = rrddim;
@@ -160,7 +146,11 @@ static void rrddim_insert_callback(const DICTIONARY_ITEM *item __maybe_unused, v
146 }
147 }
148
163 - rrddim_update_rrddimvars_unsafe(rd);
149 + if(!rrdset_is_ar_chart(st)) {
150 + rrddim_flag_set(rd, RRDDIM_FLAG_PENDING_HEALTH_INITIALIZATION);
151 + rrdset_flag_set(rd->rrdset, RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION);
152 + rrdhost_flag_set(rd->rrdset->rrdhost, RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION);
153 + }
154
155 // let the chart resync
156 rrdset_flag_set(st, RRDSET_FLAG_SYNC_CLOCK);
@@ -264,7 +254,12 @@ static bool rrddim_conflict_callback(const DICTIONARY_ITEM *item __maybe_unused,
254 }
255
256 rrddim_flag_clear(rd, RRDDIM_FLAG_ARCHIVED);
267 - rrddim_update_rrddimvars_unsafe(rd);
257 +
258 + if(!rrdset_is_ar_chart(st)) {
259 + rrddim_flag_set(rd, RRDDIM_FLAG_PENDING_HEALTH_INITIALIZATION);
260 + rrdset_flag_set(rd->rrdset, RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION);
261 + rrdhost_flag_set(rd->rrdset->rrdhost, RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION);
262 + }
263 }
264
265 if(unlikely(rc))
database/rrdfunctions.c new
+758
@@ -0,0 +1,758 @@
1 +#define NETDATA_RRD_INTERNALS
2 +#include "rrd.h"
3 +
4 +#define MAX_FUNCTION_LENGTH (PLUGINSD_LINE_MAX - 512) // we need some space for the rest of the line
5 +
6 +static unsigned char functions_allowed_chars[256] = {
7 + [0] = '\0', //
8 + [1] = '_', //
9 + [2] = '_', //
10 + [3] = '_', //
11 + [4] = '_', //
12 + [5] = '_', //
13 + [6] = '_', //
14 + [7] = '_', //
15 + [8] = '_', //
16 + [9] = ' ', // Horizontal Tab
17 + [10] = ' ', // Line Feed
18 + [11] = ' ', // Vertical Tab
19 + [12] = ' ', // Form Feed
20 + [13] = ' ', // Carriage Return
21 + [14] = '_', //
22 + [15] = '_', //
23 + [16] = '_', //
24 + [17] = '_', //
25 + [18] = '_', //
26 + [19] = '_', //
27 + [20] = '_', //
28 + [21] = '_', //
29 + [22] = '_', //
30 + [23] = '_', //
31 + [24] = '_', //
32 + [25] = '_', //
33 + [26] = '_', //
34 + [27] = '_', //
35 + [28] = '_', //
36 + [29] = '_', //
37 + [30] = '_', //
38 + [31] = '_', //
39 + [32] = ' ', // SPACE keep
40 + [33] = '_', // !
41 + [34] = '_', // "
42 + [35] = '_', // #
43 + [36] = '_', // $
44 + [37] = '_', // %
45 + [38] = '_', // &
46 + [39] = '_', // '
47 + [40] = '_', // (
48 + [41] = '_', // )
49 + [42] = '_', // *
50 + [43] = '_', // +
51 + [44] = ',', // , keep
52 + [45] = '-', // - keep
53 + [46] = '.', // . keep
54 + [47] = '/', // / keep
55 + [48] = '0', // 0 keep
56 + [49] = '1', // 1 keep
57 + [50] = '2', // 2 keep
58 + [51] = '3', // 3 keep
59 + [52] = '4', // 4 keep
60 + [53] = '5', // 5 keep
61 + [54] = '6', // 6 keep
62 + [55] = '7', // 7 keep
63 + [56] = '8', // 8 keep
64 + [57] = '9', // 9 keep
65 + [58] = ':', // : keep
66 + [59] = ':', // ; convert ; to :
67 + [60] = '_', // <
68 + [61] = ':', // = convert = to :
69 + [62] = '_', // >
70 + [63] = '_', // ?
71 + [64] = '_', // @
72 + [65] = 'A', // A keep
73 + [66] = 'B', // B keep
74 + [67] = 'C', // C keep
75 + [68] = 'D', // D keep
76 + [69] = 'E', // E keep
77 + [70] = 'F', // F keep
78 + [71] = 'G', // G keep
79 + [72] = 'H', // H keep
80 + [73] = 'I', // I keep
81 + [74] = 'J', // J keep
82 + [75] = 'K', // K keep
83 + [76] = 'L', // L keep
84 + [77] = 'M', // M keep
85 + [78] = 'N', // N keep
86 + [79] = 'O', // O keep
87 + [80] = 'P', // P keep
88 + [81] = 'Q', // Q keep
89 + [82] = 'R', // R keep
90 + [83] = 'S', // S keep
91 + [84] = 'T', // T keep
92 + [85] = 'U', // U keep
93 + [86] = 'V', // V keep
94 + [87] = 'W', // W keep
95 + [88] = 'X', // X keep
96 + [89] = 'Y', // Y keep
97 + [90] = 'Z', // Z keep
98 + [91] = '_', // [
99 + [92] = '/', // backslash convert \ to /
100 + [93] = '_', // ]
101 + [94] = '_', // ^
102 + [95] = '_', // _ keep
103 + [96] = '_', // `
104 + [97] = 'a', // a keep
105 + [98] = 'b', // b keep
106 + [99] = 'c', // c keep
107 + [100] = 'd', // d keep
108 + [101] = 'e', // e keep
109 + [102] = 'f', // f keep
110 + [103] = 'g', // g keep
111 + [104] = 'h', // h keep
112 + [105] = 'i', // i keep
113 + [106] = 'j', // j keep
114 + [107] = 'k', // k keep
115 + [108] = 'l', // l keep
116 + [109] = 'm', // m keep
117 + [110] = 'n', // n keep
118 + [111] = 'o', // o keep
119 + [112] = 'p', // p keep
120 + [113] = 'q', // q keep
121 + [114] = 'r', // r keep
122 + [115] = 's', // s keep
123 + [116] = 't', // t keep
124 + [117] = 'u', // u keep
125 + [118] = 'v', // v keep
126 + [119] = 'w', // w keep
127 + [120] = 'x', // x keep
128 + [121] = 'y', // y keep
129 + [122] = 'z', // z keep
130 + [123] = '_', // {
131 + [124] = '_', // |
132 + [125] = '_', // }
133 + [126] = '_', // ~
134 + [127] = '_', //
135 + [128] = '_', //
136 + [129] = '_', //
137 + [130] = '_', //
138 + [131] = '_', //
139 + [132] = '_', //
140 + [133] = '_', //
141 + [134] = '_', //
142 + [135] = '_', //
143 + [136] = '_', //
144 + [137] = '_', //
145 + [138] = '_', //
146 + [139] = '_', //
147 + [140] = '_', //
148 + [141] = '_', //
149 + [142] = '_', //
150 + [143] = '_', //
151 + [144] = '_', //
152 + [145] = '_', //
153 + [146] = '_', //
154 + [147] = '_', //
155 + [148] = '_', //
156 + [149] = '_', //
157 + [150] = '_', //
158 + [151] = '_', //
159 + [152] = '_', //
160 + [153] = '_', //
161 + [154] = '_', //
162 + [155] = '_', //
163 + [156] = '_', //
164 + [157] = '_', //
165 + [158] = '_', //
166 + [159] = '_', //
167 + [160] = '_', //
168 + [161] = '_', //
169 + [162] = '_', //
170 + [163] = '_', //
171 + [164] = '_', //
172 + [165] = '_', //
173 + [166] = '_', //
174 + [167] = '_', //
175 + [168] = '_', //
176 + [169] = '_', //
177 + [170] = '_', //
178 + [171] = '_', //
179 + [172] = '_', //
180 + [173] = '_', //
181 + [174] = '_', //
182 + [175] = '_', //
183 + [176] = '_', //
184 + [177] = '_', //
185 + [178] = '_', //
186 + [179] = '_', //
187 + [180] = '_', //
188 + [181] = '_', //
189 + [182] = '_', //
190 + [183] = '_', //
191 + [184] = '_', //
192 + [185] = '_', //
193 + [186] = '_', //
194 + [187] = '_', //
195 + [188] = '_', //
196 + [189] = '_', //
197 + [190] = '_', //
198 + [191] = '_', //
199 + [192] = '_', //
200 + [193] = '_', //
201 + [194] = '_', //
202 + [195] = '_', //
203 + [196] = '_', //
204 + [197] = '_', //
205 + [198] = '_', //
206 + [199] = '_', //
207 + [200] = '_', //
208 + [201] = '_', //
209 + [202] = '_', //
210 + [203] = '_', //
211 + [204] = '_', //
212 + [205] = '_', //
213 + [206] = '_', //
214 + [207] = '_', //
215 + [208] = '_', //
216 + [209] = '_', //
217 + [210] = '_', //
218 + [211] = '_', //
219 + [212] = '_', //
220 + [213] = '_', //
221 + [214] = '_', //
222 + [215] = '_', //
223 + [216] = '_', //
224 + [217] = '_', //
225 + [218] = '_', //
226 + [219] = '_', //
227 + [220] = '_', //
228 + [221] = '_', //
229 + [222] = '_', //
230 + [223] = '_', //
231 + [224] = '_', //
232 + [225] = '_', //
233 + [226] = '_', //
234 + [227] = '_', //
235 + [228] = '_', //
236 + [229] = '_', //
237 + [230] = '_', //
238 + [231] = '_', //
239 + [232] = '_', //
240 + [233] = '_', //
241 + [234] = '_', //
242 + [235] = '_', //
243 + [236] = '_', //
244 + [237] = '_', //
245 + [238] = '_', //
246 + [239] = '_', //
247 + [240] = '_', //
248 + [241] = '_', //
249 + [242] = '_', //
250 + [243] = '_', //
251 + [244] = '_', //
252 + [245] = '_', //
253 + [246] = '_', //
254 + [247] = '_', //
255 + [248] = '_', //
256 + [249] = '_', //
257 + [250] = '_', //
258 + [251] = '_', //
259 + [252] = '_', //
260 + [253] = '_', //
261 + [254] = '_', //
262 + [255] = '_' //
263 +};
264 +
265 +static inline size_t sanitize_function_text(char *dst, const char *src, size_t dst_len) {
266 + return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_len,
267 + functions_allowed_chars, true, "", NULL);
268 +}
269 +
270 +// we keep a dictionary per RRDSET with these functions
271 +// the dictionary is created on demand (only when a function is added to an RRDSET)
272 +
273 +typedef enum {
274 + RRD_FUNCTION_LOCAL = (1 << 0),
275 + RRD_FUNCTION_GLOBAL = (1 << 1),
276 +
277 + // this is 8-bit
278 +} RRD_FUNCTION_OPTIONS;
279 +
280 +struct rrd_collector_function {
281 + bool sync; // when true, the function is called synchronously
282 + uint8_t options; // RRD_FUNCTION_OPTIONS
283 + STRING *help;
284 + int timeout; // the default timeout of the function
285 +
286 + int (*function)(BUFFER *wb, int timeout, const char *function, void *collector_data,
287 + function_data_ready_callback callback, void *callback_data);
288 +
289 + void *collector_data;
290 + struct rrd_collector *collector;
291 +};
292 +
293 +// Each function points to this collector structure
294 +// so that when the collector exits, all of them will
295 +// be invalidated (running == false)
296 +// The last function that is using this collector
297 +// frees the structure too (or when the collector calls
298 +// rrdset_collector_finished()).
299 +
300 +struct rrd_collector {
301 + int32_t refcount;
302 + pid_t tid;
303 + bool running;
304 +};
305 +
306 +// Each thread that adds RRDSET functions, has to call
307 +// rrdset_collector_started() and rrdset_collector_finished()
308 +// to create the collector structure.
309 +
310 +static __thread struct rrd_collector *thread_rrd_collector = NULL;
311 +
312 +static void rrd_collector_free(struct rrd_collector *rdc) {
313 + int32_t expected = 0;
314 + if(likely(!__atomic_compare_exchange_n(&rdc->refcount, &expected, -1, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST))) {
315 + // the collector is still referenced by charts.
316 + // leave it hanging there, the last chart will actually free it.
317 + return;
318 + }
319 +
320 + // we can free it now
321 + freez(rdc);
322 +}
323 +
324 +// called once per collector
325 +void rrd_collector_started(void) {
326 + if(likely(thread_rrd_collector)) return;
327 +
328 + thread_rrd_collector = callocz(1, sizeof(struct rrd_collector));
329 + thread_rrd_collector->tid = gettid();
330 + thread_rrd_collector->running = true;
331 +}
332 +
333 +// called once per collector
334 +void rrd_collector_finished(void) {
335 + if(!thread_rrd_collector)
336 + return;
337 +
338 + thread_rrd_collector->running = false;
339 + rrd_collector_free(thread_rrd_collector);
340 + thread_rrd_collector = NULL;
341 +}
342 +
343 +static struct rrd_collector *rrd_collector_acquire(void) {
344 + __atomic_add_fetch(&thread_rrd_collector->refcount, 1, __ATOMIC_SEQ_CST);
345 + return thread_rrd_collector;
346 +}
347 +
348 +static void rrd_collector_release(struct rrd_collector *rdc) {
349 + if(unlikely(!rdc)) return;
350 +
351 + int32_t refcount = __atomic_sub_fetch(&rdc->refcount, 1, __ATOMIC_SEQ_CST);
352 + if(refcount == 0 && !rdc->running)
353 + rrd_collector_free(rdc);
354 +}
355 +
356 +static void rrd_functions_insert_callback(const DICTIONARY_ITEM *item __maybe_unused, void *func __maybe_unused,
357 + void *rrdhost __maybe_unused) {
358 + struct rrd_collector_function *rdcf = func;
359 +
360 + if(!thread_rrd_collector)
361 + fatal("RRDSET_COLLECTOR: called %s() for function '%s' without calling rrd_collector_started() first.",
362 + __FUNCTION__, dictionary_acquired_item_name(item));
363 +
364 + rdcf->collector = rrd_collector_acquire();
365 +}
366 +
367 +static void rrd_functions_delete_callback(const DICTIONARY_ITEM *item __maybe_unused, void *func __maybe_unused,
368 + void *rrdhost __maybe_unused) {
369 + struct rrd_collector_function *rdcf = func;
370 + rrd_collector_release(rdcf->collector);
371 +}
372 +
373 +static bool rrd_functions_conflict_callback(const DICTIONARY_ITEM *item __maybe_unused, void *func __maybe_unused,
374 + void *new_func __maybe_unused, void *rrdhost __maybe_unused) {
375 + struct rrd_collector_function *rdcf = func;
376 + struct rrd_collector_function *new_rdcf = new_func;
377 +
378 + if(!thread_rrd_collector)
379 + fatal("RRDSET_COLLECTOR: called %s() for function '%s' without calling rrd_collector_started() first.",
380 + __FUNCTION__, dictionary_acquired_item_name(item));
381 +
382 + bool changed = false;
383 +
384 + if(rdcf->collector != thread_rrd_collector) {
385 + struct rrd_collector *old_rdc = rdcf->collector;
386 + rdcf->collector = rrd_collector_acquire();
387 + rrd_collector_release(old_rdc);
388 + changed = true;
389 + }
390 +
391 + if(rdcf->function != new_rdcf->function) {
392 + rdcf->function = new_rdcf->function;
393 + changed = true;
394 + }
395 +
396 + if(rdcf->help != new_rdcf->help) {
397 + STRING *old = rdcf->help;
398 + rdcf->help = new_rdcf->help;
399 + string_freez(old);
400 + changed = true;
401 + }
402 + else
403 + string_freez(new_rdcf->help);
404 +
405 + if(rdcf->timeout != new_rdcf->timeout) {
406 + rdcf->timeout = new_rdcf->timeout;
407 + changed = true;
408 + }
409 +
410 + if(rdcf->sync != new_rdcf->sync) {
411 + rdcf->sync = new_rdcf->sync;
412 + changed = true;
413 + }
414 +
415 + if(rdcf->collector_data != new_rdcf->collector_data) {
416 + rdcf->collector_data = new_rdcf->collector_data;
417 + changed = true;
418 + }
419 +
420 + return changed;
421 +}
422 +
423 +
424 +void rrdfunctions_init(RRDHOST *host) {
425 + if(host->functions) return;
426 +
427 + host->functions = dictionary_create(DICT_OPTION_DONT_OVERWRITE_VALUE);
428 + dictionary_register_insert_callback(host->functions, rrd_functions_insert_callback, host);
429 + dictionary_register_delete_callback(host->functions, rrd_functions_delete_callback, host);
430 + dictionary_register_conflict_callback(host->functions, rrd_functions_conflict_callback, host);
431 +}
432 +
433 +void rrdfunctions_destroy(RRDHOST *host) {
434 + dictionary_destroy(host->functions);
435 +}
436 +
437 +void rrd_collector_add_function(RRDHOST *host, RRDSET *st, const char *name, int timeout, const char *help,
438 + bool sync, function_execute_at_collector function, void *collector_data) {
439 +
440 + // RRDSET *st may be NULL in this function
441 + // to create a GLOBAL function
442 +
443 + if(st && !st->functions_view)
444 + st->functions_view = dictionary_create_view(host->functions);
445 +
446 + char key[PLUGINSD_LINE_MAX + 1];
447 + sanitize_function_text(key, name, PLUGINSD_LINE_MAX);
448 +
449 + struct rrd_collector_function tmp = {
450 + .sync = sync,
451 + .timeout = timeout,
452 + .options = (st)?RRD_FUNCTION_LOCAL:RRD_FUNCTION_GLOBAL,
453 + .function = function,
454 + .collector_data = collector_data,
455 + .help = string_strdupz(help),
456 + };
457 + const DICTIONARY_ITEM *item = dictionary_set_and_acquire_item(host->functions, key, &tmp, sizeof(tmp));
458 +
459 + if(st)
460 + dictionary_view_set(st->functions_view, key, item);
461 +
462 + dictionary_acquired_item_release(host->functions, item);
463 +}
464 +
465 +void rrd_functions_expose_rrdpush(RRDSET *st, BUFFER *wb) {
466 + if(!st->functions_view)
467 + return;
468 +
469 + struct rrd_collector_function *tmp;
470 + dfe_start_read(st->functions_view, tmp) {
471 + buffer_sprintf(wb
472 + , PLUGINSD_KEYWORD_FUNCTION " \"%s\" %d \"%s\"\n"
473 + , tmp_dfe.name
474 + , tmp->timeout
475 + , string2str(tmp->help)
476 + );
477 + }
478 + dfe_done(tmp);
479 +}
480 +
481 +struct rrd_function_call_wait {
482 + bool free_with_signal;
483 + bool data_are_ready;
484 + netdata_mutex_t mutex;
485 + pthread_cond_t cond;
486 + int code;
487 +};
488 +
489 +static void rrd_function_call_wait_free(struct rrd_function_call_wait *tmp) {
490 + pthread_cond_destroy(&tmp->cond);
491 + netdata_mutex_destroy(&tmp->mutex);
492 + freez(tmp);
493 +}
494 +
495 +struct {
496 + const char *format;
497 + uint8_t content_type;
498 +} function_formats[] = {
499 + { .format = "application/json", CT_APPLICATION_JSON },
500 + { .format = "text/plain", CT_TEXT_PLAIN },
501 + { .format = "application/xml", CT_APPLICATION_XML },
502 + { .format = "prometheus", CT_PROMETHEUS },
503 + { .format = "text", CT_TEXT_PLAIN },
504 + { .format = "txt", CT_TEXT_PLAIN },
505 + { .format = "json", CT_APPLICATION_JSON },
506 + { .format = "html", CT_TEXT_HTML },
507 + { .format = "text/html", CT_TEXT_HTML },
508 + { .format = "xml", CT_APPLICATION_XML },
509 +
510 + // terminator
511 + { .format = NULL, CT_TEXT_PLAIN },
512 +};
513 +
514 +uint8_t functions_format_to_content_type(const char *format) {
515 + if(format && *format) {
516 + for (int i = 0; function_formats[i].format; i++)
517 + if (strcmp(function_formats[i].format, format) == 0)
518 + return function_formats[i].content_type;
519 + }
520 +
521 + return CT_TEXT_PLAIN;
522 +}
523 +
524 +const char *functions_content_type_to_format(uint8_t content_type) {
525 + for (int i = 0; function_formats[i].format; i++)
526 + if (function_formats[i].content_type == content_type)
527 + return function_formats[i].format;
528 +
529 + return "text/plain";
530 +}
531 +
532 +int rrd_call_function_error(BUFFER *wb, const char *msg, int code) {
533 + char buffer[PLUGINSD_LINE_MAX];
534 + json_escape_string(buffer, msg, PLUGINSD_LINE_MAX);
535 +
536 + buffer_flush(wb);
537 + buffer_sprintf(wb, "{\"status\":%d,\"error_message\":\"%s\"}", code, buffer);
538 + wb->contenttype = CT_APPLICATION_JSON;
539 + buffer_no_cacheable(wb);
540 + return code;
541 +}
542 +
543 +static int rrd_call_function_find(RRDHOST *host, BUFFER *wb, const char *name, size_t key_length, struct rrd_collector_function **rdcf) {
544 + char buffer[MAX_FUNCTION_LENGTH + 1];
545 +
546 + strncpyz(buffer, name, MAX_FUNCTION_LENGTH);
547 + char *s = NULL;
548 +
549 + *rdcf = NULL;
550 + while(!(*rdcf) && buffer[0]) {
551 + *rdcf = dictionary_get(host->functions, buffer);
552 + if(*rdcf) break;
553 +
554 + // if s == NULL, set it to the end of the buffer
555 + // this should happen only the first time
556 + if(unlikely(!s))
557 + s = &buffer[key_length - 1];
558 +
559 + // skip a word from the end
560 + while(s >= buffer && !isspace(*s)) *s-- = '\0';
561 +
562 + // skip all spaces
563 + while(s >= buffer && isspace(*s)) *s-- = '\0';
564 + }
565 +
566 + buffer_flush(wb);
567 +
568 + if(!(*rdcf))
569 + return rrd_call_function_error(wb, "No collector is supplying this function on this host at this time.", HTTP_RESP_NOT_FOUND);
570 +
571 + if(!(*rdcf)->collector->running)
572 + return rrd_call_function_error(wb, "The collector that registered this function, is not currently running.", HTTP_RESP_BACKEND_FETCH_FAILED);
573 +
574 + return HTTP_RESP_OK;
575 +}
576 +
577 +static void rrd_call_function_signal_when_ready(BUFFER *temp_wb __maybe_unused, int code, void *callback_data) {
578 + struct rrd_function_call_wait *tmp = callback_data;
579 + bool we_should_free = false;
580 +
581 + netdata_mutex_lock(&tmp->mutex);
582 +
583 + // since we got the mutex,
584 + // the waiting thread is either in pthread_cond_timedwait()
585 + // or gave up and left.
586 +
587 + tmp->code = code;
588 + tmp->data_are_ready = true;
589 +
590 + if(tmp->free_with_signal)
591 + we_should_free = true;
592 +
593 + pthread_cond_signal(&tmp->cond);
594 +
595 + netdata_mutex_unlock(&tmp->mutex);
596 +
597 + if(we_should_free) {
598 + buffer_free(temp_wb);
599 + rrd_function_call_wait_free(tmp);
600 + }
601 +}
602 +
603 +int rrd_call_function_and_wait(RRDHOST *host, BUFFER *wb, int timeout, const char *name) {
604 + int code;
605 +
606 + struct rrd_collector_function *rdcf = NULL;
607 +
608 + char key[PLUGINSD_LINE_MAX + 1];
609 + size_t key_length = sanitize_function_text(key, name, PLUGINSD_LINE_MAX);
610 + code = rrd_call_function_find(host, wb, key, key_length, &rdcf);
611 + if(code != HTTP_RESP_OK)
612 + return code;
613 +
614 + if(timeout <= 0)
615 + timeout = rdcf->timeout;
616 +
617 + struct timespec tp;
618 + clock_gettime(CLOCK_REALTIME, &tp);
619 + tp.tv_sec += (time_t)timeout;
620 +
621 + if(rdcf->sync) {
622 + code = rdcf->function(wb, timeout, key, rdcf->collector_data, NULL, NULL);
623 + }
624 + else {
625 + struct rrd_function_call_wait *tmp = mallocz(sizeof(struct rrd_function_call_wait));
626 + tmp->free_with_signal = false;
627 + tmp->data_are_ready = false;
628 + netdata_mutex_init(&tmp->mutex);
629 + pthread_cond_init(&tmp->cond, NULL);
630 +
631 + bool we_should_free = true;
632 + BUFFER *temp_wb = buffer_create(PLUGINSD_LINE_MAX + 1); // we need it because we may give up on it
633 + temp_wb->contenttype = wb->contenttype;
634 + code = rdcf->function(temp_wb, timeout, key, rdcf->collector_data, rrd_call_function_signal_when_ready, tmp);
635 + if (code == HTTP_RESP_OK) {
636 + netdata_mutex_lock(&tmp->mutex);
637 +
638 + int rc = 0;
639 + while (rc == 0 && !tmp->data_are_ready) {
640 + // the mutex is unlocked within pthread_cond_timedwait()
641 + rc = pthread_cond_timedwait(&tmp->cond, &tmp->mutex, &tp);
642 + // the mutex is again ours
643 + }
644 +
645 + if (tmp->data_are_ready) {
646 + // we have a response
647 + buffer_fast_strcat(wb, buffer_tostring(temp_wb), buffer_strlen(temp_wb));
648 + wb->contenttype = temp_wb->contenttype;
649 + wb->expires = temp_wb->expires;
650 +
651 + if(wb->expires)
652 + buffer_cacheable(wb);
653 + else
654 + buffer_no_cacheable(wb);
655 +
656 + code = tmp->code;
657 + }
658 + else if (rc == ETIMEDOUT) {
659 + // timeout
660 + // we will go away and let the callback free the structure
661 + tmp->free_with_signal = true;
662 + we_should_free = false;
663 + code = rrd_call_function_error(wb, "Timeout while waiting for a response from the collector.", HTTP_RESP_GATEWAY_TIMEOUT);
664 + }
665 + else
666 + code = rrd_call_function_error(wb, "Failed to get the response from the collector.", HTTP_RESP_INTERNAL_SERVER_ERROR);
667 +
668 + netdata_mutex_unlock(&tmp->mutex);
669 + }
670 + else {
671 + buffer_free(temp_wb);
672 + if(!buffer_strlen(wb))
673 + rrd_call_function_error(wb, "Failed to send request to the collector.", code);
674 + }
675 +
676 + if (we_should_free)
677 + rrd_function_call_wait_free(tmp);
678 + }
679 +
680 + return code;
681 +}
682 +
683 +int rrd_call_function_async(RRDHOST *host, BUFFER *wb, int timeout, const char *name,
684 + rrd_call_function_async_callback callback, void *callback_data) {
685 + int code;
686 +
687 + struct rrd_collector_function *rdcf = NULL;
688 + char key[PLUGINSD_LINE_MAX + 1];
689 + size_t key_length = sanitize_function_text(key, name, PLUGINSD_LINE_MAX);
690 + code = rrd_call_function_find(host, wb, key, key_length, &rdcf);
691 + if(code != HTTP_RESP_OK)
692 + return code;
693 +
694 + if(timeout <= 0)
695 + timeout = rdcf->timeout;
696 +
697 + code = rdcf->function(wb, timeout, key, rdcf->collector_data, callback, callback_data);
698 +
699 + if(code != HTTP_RESP_OK) {
700 + if (!buffer_strlen(wb))
701 + rrd_call_function_error(wb, "Failed to send request to the collector.", code);
702 + }
703 +
704 + return code;
705 +}
706 +
707 +static void functions2json(DICTIONARY *functions, BUFFER *wb, const char *ident, const char *kq, const char *sq) {
708 + struct rrd_collector_function *t;
709 + dfe_start_read(functions, t) {
710 + if(!t->collector->running) continue;
711 +
712 + if(t_dfe.counter)
713 + buffer_strcat(wb, ",\n");
714 +
715 + buffer_sprintf(wb, "%s%s%s%s: {", ident, kq, t_dfe.name, kq);
716 + buffer_sprintf(wb, "\n\t%s%shelp%s: %s%s%s", ident, kq, kq, sq, string2str(t->help), sq);
717 + buffer_sprintf(wb, ",\n\t%s%stimeout%s: %d", ident, kq, kq, t->timeout);
718 + buffer_sprintf(wb, ",\n\t%s%soptions%s: \"%s%s\"", ident, kq, kq
719 + , (t->options & RRD_FUNCTION_LOCAL)?"LOCAL ":""
720 + , (t->options & RRD_FUNCTION_GLOBAL)?"GLOBAL ":""
721 + );
722 + buffer_sprintf(wb, "\n%s}", ident);
723 + }
724 + dfe_done(t);
725 + buffer_strcat(wb, "\n");
726 +}
727 +
728 +void chart_functions2json(RRDSET *st, BUFFER *wb, int tabs, const char *kq, const char *sq) {
729 + if(!st || !st->functions_view) return;
730 +
731 + char ident[tabs + 1];
732 + ident[tabs] = '\0';
733 + while(tabs) ident[--tabs] = '\t';
734 +
735 + functions2json(st->functions_view, wb, ident, kq, sq);
736 +}
737 +
738 +void host_functions2json(RRDHOST *host, BUFFER *wb, int tabs, const char *kq, const char *sq) {
739 + if(!host || !host->functions) return;
740 +
741 + char ident[tabs + 1];
742 + ident[tabs] = '\0';
743 + while(tabs) ident[--tabs] = '\t';
744 +
745 + functions2json(host->functions, wb, ident, kq, sq);
746 +}
747 +
748 +void chart_functions_to_dict(RRDSET *st, DICTIONARY *dict) {
749 + if(!st || !st->functions_view) return;
750 +
751 + struct rrd_collector_function *t;
752 + dfe_start_read(st->functions_view, t) {
753 + if(!t->collector->running) continue;
754 +
755 + dictionary_set(dict, t_dfe.name, NULL, 0);
756 + }
757 + dfe_done(t);
758 +}
database/rrdfunctions.h new
+35
@@ -0,0 +1,35 @@
1 +#ifndef NETDATA_RRDFUNCTIONS_H
2 +#define NETDATA_RRDFUNCTIONS_H 1
3 +
4 +#include "rrd.h"
5 +
6 +extern void rrdfunctions_init(RRDHOST *host);
7 +extern void rrdfunctions_destroy(RRDHOST *host);
8 +
9 +extern void rrd_collector_started(void);
10 +extern void rrd_collector_finished(void);
11 +
12 +typedef void (*function_data_ready_callback)(BUFFER *wb, int code, void *callback_data);
13 +
14 +typedef int (*function_execute_at_collector)(BUFFER *wb, int timeout, const char *function, void *collector_data,
15 + function_data_ready_callback callback, void *callback_data);
16 +
17 +extern void rrd_collector_add_function(RRDHOST *host, RRDSET *st, const char *name, int timeout, const char *help,
18 + bool sync, function_execute_at_collector function, void *collector_data);
19 +
20 +extern int rrd_call_function_and_wait(RRDHOST *host, BUFFER *wb, int timeout, const char *name);
21 +
22 +typedef void (*rrd_call_function_async_callback)(BUFFER *wb, int code, void *callback_data);
23 +extern int rrd_call_function_async(RRDHOST *host, BUFFER *wb, int timeout, const char *name, rrd_call_function_async_callback, void *callback_data);
24 +
25 +extern void rrd_functions_expose_rrdpush(RRDSET *st, BUFFER *wb);
26 +
27 +extern void chart_functions2json(RRDSET *st, BUFFER *wb, int tabs, const char *kq, const char *sq);
28 +extern void chart_functions_to_dict(RRDSET *st, DICTIONARY *dict);
29 +extern void host_functions2json(RRDHOST *host, BUFFER *wb, int tabs, const char *kq, const char *sq);
30 +
31 +extern uint8_t functions_format_to_content_type(const char *format);
32 +extern const char *functions_content_type_to_format(uint8_t content_type);
33 +extern int rrd_call_function_error(BUFFER *wb, const char *msg, int code);
34 +
35 +#endif // NETDATA_RRDFUNCTIONS_H
database/rrdhost.c
+48 -61
@@ -74,9 +74,9 @@ inline RRDHOST *rrdhost_find_by_guid(const char *guid) {
74 static inline RRDHOST *rrdhost_index_add_by_guid(RRDHOST *host) {
75 RRDHOST *ret_machine_guid = dictionary_set(rrdhost_root_index, host->machine_guid, host, sizeof(RRDHOST));
76 if(ret_machine_guid == host)
77 - rrdhost_flag_set(host, RRDHOST_FLAG_INDEXED_MACHINE_GUID);
77 + rrdhost_option_set(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID);
78 else {
79 - rrdhost_flag_clear(host, RRDHOST_FLAG_INDEXED_MACHINE_GUID);
79 + rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID);
80 error("RRDHOST: %s() host with machine guid '%s' is already indexed", __FUNCTION__, host->machine_guid);
81 }
82
@@ -84,11 +84,11 @@ static inline RRDHOST *rrdhost_index_add_by_guid(RRDHOST *host) {
84 }
85
86 static void rrdhost_index_del_by_guid(RRDHOST *host) {
87 - if(rrdhost_flag_check(host, RRDHOST_FLAG_INDEXED_MACHINE_GUID)) {
87 + if(rrdhost_option_check(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID)) {
88 if(!dictionary_del(rrdhost_root_index, host->machine_guid))
89 error("RRDHOST: %s() failed to delete machine guid '%s' from index", __FUNCTION__, host->machine_guid);
90
91 - rrdhost_flag_clear(host, RRDHOST_FLAG_INDEXED_MACHINE_GUID);
91 + rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_MACHINE_GUID);
92 }
93 }
94
@@ -107,9 +107,9 @@ static inline RRDHOST *rrdhost_index_add_hostname(RRDHOST *host) {
107
108 RRDHOST *ret_hostname = dictionary_set(rrdhost_root_index_hostname, rrdhost_hostname(host), host, sizeof(RRDHOST));
109 if(ret_hostname == host)
110 - rrdhost_flag_set(host, RRDHOST_FLAG_INDEXED_HOSTNAME);
110 + rrdhost_option_set(host, RRDHOST_OPTION_INDEXED_HOSTNAME);
111 else {
112 - rrdhost_flag_clear(host, RRDHOST_FLAG_INDEXED_HOSTNAME);
112 + rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_HOSTNAME);
113 error("RRDHOST: %s() host with hostname '%s' is already indexed", __FUNCTION__, rrdhost_hostname(host));
114 }
115
@@ -119,11 +119,11 @@ static inline RRDHOST *rrdhost_index_add_hostname(RRDHOST *host) {
119 static inline void rrdhost_index_del_hostname(RRDHOST *host) {
120 if(unlikely(!host->hostname)) return;
121
122 - if(rrdhost_flag_check(host, RRDHOST_FLAG_INDEXED_HOSTNAME)) {
122 + if(rrdhost_option_check(host, RRDHOST_OPTION_INDEXED_HOSTNAME)) {
123 if(!dictionary_del(rrdhost_root_index_hostname, rrdhost_hostname(host)))
124 error("RRDHOST: %s() failed to delete hostname '%s' from index", __FUNCTION__, rrdhost_hostname(host));
125
126 - rrdhost_flag_clear(host, RRDHOST_FLAG_INDEXED_HOSTNAME);
126 + rrdhost_option_clear(host, RRDHOST_OPTION_INDEXED_HOSTNAME);
127 }
128 }
129
@@ -200,38 +200,34 @@ void set_host_properties(RRDHOST *host, int update_every, RRD_MEMORY_MODE memory
200 // ----------------------------------------------------------------------------
201 // RRDHOST - add a host
202
203 -static void rrdhost_initialize_rrdpush(RRDHOST *host,
203 +static void rrdhost_initialize_rrdpush_sender(RRDHOST *host,
204 unsigned int rrdpush_enabled,
205 char *rrdpush_destination,
206 char *rrdpush_api_key,
207 char *rrdpush_send_charts_matching
208 ) {
209 - if(rrdhost_flag_check(host, RRDHOST_FLAG_INITIALIZED_RRDPUSH)) return;
210 - rrdhost_flag_set(host, RRDHOST_FLAG_INITIALIZED_RRDPUSH);
209 + if(rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_INITIALIZED)) return;
210
212 - sender_init(host);
213 - netdata_mutex_init(&host->receiver_lock);
211 + if(rrdpush_enabled && rrdpush_destination && *rrdpush_destination && rrdpush_api_key && *rrdpush_api_key) {
212 + rrdhost_flag_set(host, RRDHOST_FLAG_RRDPUSH_SENDER_INITIALIZED);
213
215 - host->rrdpush_send_enabled = (rrdpush_enabled && rrdpush_destination && *rrdpush_destination && rrdpush_api_key && *rrdpush_api_key) ? 1 : 0;
216 - host->rrdpush_send_destination = (host->rrdpush_send_enabled)?strdupz(rrdpush_destination):NULL;
214 + sender_init(host);
215
218 - if (host->rrdpush_send_destination)
219 - host->destinations = destinations_init(host->rrdpush_send_destination);
216 +#ifdef ENABLE_HTTPS
217 + host->sender->ssl.conn = NULL;
218 + host->sender->ssl.flags = NETDATA_SSL_START;
219 +#endif
220
221 - host->rrdpush_send_api_key = (host->rrdpush_send_enabled)?strdupz(rrdpush_api_key):NULL;
222 - host->rrdpush_send_charts_matching = simple_pattern_create(rrdpush_send_charts_matching, NULL, SIMPLE_PATTERN_EXACT);
221 + host->rrdpush_send_destination = strdupz(rrdpush_destination);
222 + rrdpush_destinations_init(host);
223
224 - host->rrdpush_sender_pipe[0] = -1;
225 - host->rrdpush_sender_pipe[1] = -1;
226 - host->rrdpush_sender_socket = -1;
224 + host->rrdpush_send_api_key = strdupz(rrdpush_api_key);
225 + host->rrdpush_send_charts_matching = simple_pattern_create(rrdpush_send_charts_matching, NULL, SIMPLE_PATTERN_EXACT);
226
228 - //host->stream_version = STREAMING_PROTOCOL_CURRENT_VERSION; Unused?
229 -#ifdef ENABLE_HTTPS
230 - host->ssl.conn = NULL;
231 - host->ssl.flags = NETDATA_SSL_START;
232 - host->stream_ssl.conn = NULL;
233 - host->stream_ssl.flags = NETDATA_SSL_START;
234 -#endif
227 + rrdhost_option_set(host, RRDHOST_OPTION_SENDER_ENABLED);
228 + }
229 + else
230 + rrdhost_option_clear(host, RRDHOST_OPTION_SENDER_ENABLED);
231 }
232
233 static void rrdhost_initialize_health(RRDHOST *host,
@@ -342,8 +338,7 @@ RRDHOST *rrdhost_create(const char *hostname,
338 int is_in_multihost = (memory_mode == RRD_MEMORY_MODE_DBENGINE && !is_legacy);
339 RRDHOST *host = callocz(1, sizeof(RRDHOST));
340
345 - strncpy(host->machine_guid, guid, GUID_LEN);
346 - host->machine_guid[GUID_LEN] = '\0';
341 + strncpyz(host->machine_guid, guid, GUID_LEN + 1);
342
343 set_host_properties(host, (update_every > 0)?update_every:1, memory_mode, registry_hostname, os,
344 tags, timezone, abbrev_timezone, utc_offset, program_name, program_version);
@@ -354,23 +349,25 @@ RRDHOST *rrdhost_create(const char *hostname,
349 host->health_enabled = ((memory_mode == RRD_MEMORY_MODE_NONE)) ? 0 : health_enabled;
350
351 if (likely(!archived)) {
352 + rrdfunctions_init(host);
353 host->rrdlabels = rrdlabels_create();
358 - rrdhost_initialize_rrdpush(
354 + rrdhost_initialize_rrdpush_sender(
355 host, rrdpush_enabled, rrdpush_destination, rrdpush_api_key, rrdpush_send_charts_matching);
356 }
357
358 netdata_rwlock_init(&host->rrdhost_rwlock);
359 netdata_mutex_init(&host->aclk_state_lock);
360 + netdata_mutex_init(&host->receiver_lock);
361
362 host->system_info = system_info;
363
364 rrdset_index_init(host);
365
366 if(config_get_boolean(CONFIG_SECTION_DB, "delete obsolete charts files", 1))
370 - rrdhost_flag_set(host, RRDHOST_FLAG_DELETE_OBSOLETE_CHARTS);
367 + rrdhost_option_set(host, RRDHOST_OPTION_DELETE_OBSOLETE_CHARTS);
368
369 if(config_get_boolean(CONFIG_SECTION_DB, "delete orphan hosts files", 1) && !is_localhost)
373 - rrdhost_flag_set(host, RRDHOST_FLAG_DELETE_ORPHAN_HOST);
370 + rrdhost_option_set(host, RRDHOST_OPTION_DELETE_ORPHAN_HOST);
371
372 char filename[FILENAME_MAX + 1];
373 if(is_localhost) {
@@ -527,7 +524,7 @@ RRDHOST *rrdhost_create(const char *hostname,
524 , host->rrd_update_every
525 , rrd_memory_mode_name(host->rrd_memory_mode)
526 , host->rrd_history_entries
530 - , host->rrdpush_send_enabled?"enabled":"disabled"
527 + , rrdhost_has_rrdpush_sender_enabled(host)?"enabled":"disabled"
528 , host->rrdpush_send_destination?host->rrdpush_send_destination:""
529 , host->rrdpush_send_api_key?host->rrdpush_send_api_key:""
530 , host->health_enabled?"enabled":"disabled"
@@ -574,10 +571,6 @@ void rrdhost_update(RRDHOST *host
571 )
572 {
573 UNUSED(guid);
577 - UNUSED(rrdpush_enabled);
578 - UNUSED(rrdpush_destination);
579 - UNUSED(rrdpush_api_key);
580 - UNUSED(rrdpush_send_charts_matching);
574
575 host->health_enabled = (mode == RRD_MEMORY_MODE_NONE) ? 0 : health_enabled;
576 //host->stream_version = STREAMING_PROTOCOL_CURRENT_VERSION; Unused?
@@ -629,20 +622,21 @@ void rrdhost_update(RRDHOST *host
622 if (rrdhost_flag_check(host, RRDHOST_FLAG_ARCHIVED)) {
623 rrdhost_flag_clear(host, RRDHOST_FLAG_ARCHIVED);
624
625 + rrdfunctions_init(host);
626 +
627 if(!host->rrdlabels)
628 host->rrdlabels = rrdlabels_create();
629
630 if (!host->rrdset_root_index)
631 rrdset_index_init(host);
632
638 - rrdhost_initialize_rrdpush(host,
633 + rrdhost_initialize_rrdpush_sender(host,
634 rrdpush_enabled,
635 rrdpush_destination,
636 rrdpush_api_key,
637 rrdpush_send_charts_matching);
638
644 - rrdhost_initialize_health(host,
645 - host == localhost);
639 + rrdhost_initialize_health(host, host == localhost);
640
641 rrd_hosts_available++;
642 ml_new_host(host);
@@ -1015,12 +1009,13 @@ void destroy_receiver_state(struct receiver_state *rpt);
1009
1010 void stop_streaming_sender(RRDHOST *host)
1011 {
1012 + rrdhost_option_clear(host, RRDHOST_OPTION_SENDER_ENABLED);
1013 +
1014 if (unlikely(!host->sender))
1015 return;
1016
1017 rrdpush_sender_thread_stop(host); // stop a possibly running thread
1018 cbuffer_free(host->sender->buffer);
1023 - buffer_free(host->sender->build);
1019 #ifdef ENABLE_COMPRESSION
1020 if (host->sender->compressor)
1021 host->sender->compressor->destroy(&host->sender->compressor);
@@ -1156,12 +1151,7 @@ void rrdhost_free(RRDHOST *host, bool force) {
1151 freez(host->varlib_dir);
1152 freez(host->rrdpush_send_api_key);
1153 freez(host->rrdpush_send_destination);
1159 - struct rrdpush_destinations *tmp_destination;
1160 - while (host->destinations) {
1161 - tmp_destination = host->destinations->next;
1162 - freez(host->destinations);
1163 - host->destinations = tmp_destination;
1164 - }
1154 + rrdpush_destinations_free(host);
1155 string_freez(host->health_default_exec);
1156 string_freez(host->health_default_recipient);
1157 freez(host->health_log_filename);
@@ -1173,6 +1163,7 @@ void rrdhost_free(RRDHOST *host, bool force) {
1163 freez(host->node_id);
1164
1165 rrdfamily_index_destroy(host);
1166 + rrdfunctions_destroy(host);
1167 rrdvariables_destroy(host->rrdvars);
1168
1169 rrdhost_destroy_rrdcontexts(host);
@@ -1318,16 +1309,17 @@ static void rrdhost_load_kubernetes_labels(void) {
1309 debug(D_RRDHOST, "Attempting to fetch external labels via %s", label_script);
1310
1311 pid_t pid;
1321 - FILE *fp = mypopen(label_script, &pid);
1322 - if(!fp) return;
1312 + FILE *fp_child_input;
1313 + FILE *fp_child_output = netdata_popen(label_script, &pid, &fp_child_input);
1314 + if(!fp_child_output) return;
1315
1316 char buffer[1000 + 1];
1325 - while (fgets(buffer, 1000, fp) != NULL)
1317 + while (fgets(buffer, 1000, fp_child_output) != NULL)
1318 rrdlabels_add_pair(localhost->rrdlabels, buffer, RRDLABEL_SRC_AUTO|RRDLABEL_SRC_K8S);
1319
1320 // Non-zero exit code means that all the script output is error messages. We've shown already any message that didn't include a ':'
1321 // Here we'll inform with an ERROR that the script failed, show whatever (if anything) was added to the list of labels, free the memory and set the return to null
1330 - int rc = mypclose(fp, pid);
1322 + int rc = netdata_pclose(fp_child_input, fp_child_output, pid);
1323 if(rc) error("%s exited abnormally. Failed to get kubernetes labels.", label_script);
1324 }
1325
@@ -1347,12 +1339,7 @@ void reload_host_labels(void) {
1339
1340 health_label_log_save(localhost);
1341
1350 -/* TODO-GAPS - fix this so that it looks properly at the state and version of the sender
1351 - if(localhost->rrdpush_send_enabled && localhost->rrdpush_sender_buffer){
1352 - localhost->labels.labels_flag |= RRDHOST_FLAG_STREAM_LABELS_UPDATE;
1353 - rrdpush_send_labels(localhost);
1354 - }
1355 -*/
1342 + rrdpush_send_host_labels(localhost);
1343 health_reload();
1344 }
1345
@@ -1385,7 +1372,7 @@ void rrdhost_cleanup_charts(RRDHOST *host) {
1372 info("Cleaning up database of host '%s'...", rrdhost_hostname(host));
1373
1374 RRDSET *st;
1388 - uint32_t rrdhost_delete_obsolete_charts = rrdhost_flag_check(host, RRDHOST_FLAG_DELETE_OBSOLETE_CHARTS);
1375 + uint32_t rrdhost_delete_obsolete_charts = rrdhost_option_check(host, RRDHOST_OPTION_DELETE_OBSOLETE_CHARTS);
1376
1377 // we get a write lock
1378 // to ensure only one thread is saving the database
@@ -1430,7 +1417,7 @@ void rrdhost_cleanup_all(void) {
1417
1418 RRDHOST *host;
1419 rrdhost_foreach_read(host) {
1433 - if (host != localhost && rrdhost_flag_check(host, RRDHOST_FLAG_DELETE_ORPHAN_HOST) && !host->receiver
1420 + if (host != localhost && rrdhost_option_check(host, RRDHOST_OPTION_DELETE_ORPHAN_HOST) && !host->receiver
1421 #ifdef ENABLE_DBENGINE
1422 /* don't delete multi-host DB host files */
1423 && !(host->rrd_memory_mode == RRD_MEMORY_MODE_DBENGINE && is_storage_engine_shared(host->storage_instance[0]))
database/rrdlabels.c
+26 -13
@@ -369,12 +369,15 @@ __attribute__((constructor)) void initialize_labels_keys_char_map(void) {
369
370 }
371
372 -static size_t rrdlabels_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, unsigned char *char_map, bool utf, const char *empty) {
372 +size_t text_sanitize(unsigned char *dst, const unsigned char *src, size_t dst_size, unsigned char *char_map, bool utf, const char *empty, size_t *multibyte_length) {
373 if(unlikely(!dst_size)) return 0;
374 +
375 if(unlikely(!src || !*src)) {
376 strncpyz((char *)dst, empty, dst_size);
377 dst[dst_size - 1] = '\0';
377 - return strlen((char *)dst);
378 + size_t len = strlen((char *)dst);
379 + if(multibyte_length) *multibyte_length = len;
380 + return len;
381 }
382
383 unsigned char *d = dst;
@@ -385,7 +388,9 @@ static size_t rrdlabels_sanitize(unsigned char *dst, const unsigned char *src, s
388 // copy while converting, but keep only one white space
389 // we start wil last_is_space = 1 to skip leading spaces
390 int last_is_space = 1;
391 +
392 size_t mblen = 0;
393 +
394 while(*src && d < end) {
395 unsigned char c = *src;
396
@@ -446,28 +451,34 @@ static size_t rrdlabels_sanitize(unsigned char *dst, const unsigned char *src, s
451 *d = '\0';
452
453 // check if dst is all underscores and empty it if it is
449 - d = dst;
450 - while(*d == '_') d++;
451 - if(unlikely(*d == '\0')) {
452 - *dst = '\0';
453 - mblen = 0;
454 + if(*dst == '_') {
455 + unsigned char *t = dst;
456 + while (*t == '_') t++;
457 + if (unlikely(*t == '\0')) {
458 + *dst = '\0';
459 + mblen = 0;
460 + }
461 }
462
463 if(unlikely(*dst == '\0')) {
464 strncpyz((char *)dst, empty, dst_size);
465 dst[dst_size - 1] = '\0';
459 - return strlen((char *)dst);
466 + mblen = strlen((char *)dst);
467 + if(multibyte_length) *multibyte_length = mblen;
468 + return mblen;
469 }
470
462 - return mblen;
471 + if(multibyte_length) *multibyte_length = mblen;
472 +
473 + return d - dst;
474 }
475
476 static inline size_t rrdlabels_sanitize_name(char *dst, const char *src, size_t dst_size) {
466 - return rrdlabels_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_names_char_map, 0, "");
477 + return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_names_char_map, 0, "", NULL);
478 }
479
480 static inline size_t rrdlabels_sanitize_value(char *dst, const char *src, size_t dst_size) {
470 - return rrdlabels_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_values_char_map, 1, "[none]");
481 + return text_sanitize((unsigned char *)dst, (const unsigned char *)src, dst_size, label_values_char_map, 1, "[none]", NULL);
482 }
483
484 // ----------------------------------------------------------------------------
@@ -1153,12 +1164,14 @@ int rrdlabels_unittest_simple_pattern() {
1164
1165 int rrdlabels_unittest_sanitize_value(const char *src, const char *expected) {
1166 char buf[RRDLABELS_MAX_VALUE_LENGTH + 1];
1156 - size_t mblen = rrdlabels_sanitize_value(buf, src, RRDLABELS_MAX_VALUE_LENGTH);
1167 + size_t len = rrdlabels_sanitize_value(buf, src, RRDLABELS_MAX_VALUE_LENGTH);
1168 + size_t expected_len = strlen(expected);
1169
1170 int err = 0;
1171 if(strcmp(buf, expected) != 0) err = 1;
1172 + if(len != expected_len) err = 1;
1173
1161 - fprintf(stderr, "%s(%s): %s, expected '%s', got '%s', mblen = %zu, bytes = %zu\n", __FUNCTION__, src, (err==1)?"FAILED":"OK", expected, buf, mblen, strlen(buf));
1174 + fprintf(stderr, "%s(%s): %s, expected '%s', got '%s', expected bytes = %zu, got bytes = %zu\n", __FUNCTION__, src, (err==1)?"FAILED":"OK", expected, buf, expected_len, strlen(buf));
1175 return err;
1176 }
1177
database/rrdset.c
+15 -20
@@ -4,6 +4,8 @@
4 #include "rrd.h"
5 #include <sched.h>
6
7 +// ----------------------------------------------------------------------------
8 +
9 void __rrdset_check_rdlock(RRDSET *st, const char *file, const char *function, const unsigned long line) {
10 debug(D_RRD_CALLS, "Checking read lock on chart '%s'", rrdset_id(st));
11
@@ -118,7 +120,7 @@ struct rrdset_constructor {
120 static void rrdset_insert_callback(const DICTIONARY_ITEM *item __maybe_unused, void *rrdset, void *constructor_data) {
121 static STRING *anomaly_rates_chart = NULL;
122
121 - if(!unlikely(!anomaly_rates_chart))
123 + if(unlikely(!anomaly_rates_chart))
124 anomaly_rates_chart = string_strdupz(ML_ANOMALY_RATES_CHART_ID);
125
126 struct rrdset_constructor *ctr = constructor_data;
@@ -179,17 +181,12 @@ static void rrdset_insert_callback(const DICTIONARY_ITEM *item __maybe_unused, v
181 // chart variables - we need this for data collection to work (collector given chart variables) - not only health
182 rrdsetvar_index_init(st);
183
182 - if(host->health_enabled) {
183 - st->green = NAN;
184 - st->red = NAN;
185 - st->rrdfamily = rrdfamily_add_and_acquire(host, rrdset_family(st));
186 - st->rrdvars = rrdvariables_create();
187 - rrddimvar_index_init(st);
188 - }
189 -
184 st->rrdlabels = rrdlabels_create();
185 rrdset_update_permanent_labels(st);
186
187 + st->green = NAN;
188 + st->red = NAN;
189 +
190 ctr->react_action = RRDSET_REACT_NEW;
191 }
192
@@ -203,6 +200,9 @@ static void rrdset_delete_callback(const DICTIONARY_ITEM *item __maybe_unused, v
200 // remove it from the name index
201 rrdset_index_del_name(host, st);
202
203 + // release the collector info
204 + dictionary_destroy(st->functions_view);
205 +
206 rrdcalc_unlink_all_rrdset_alerts(st);
207
208 // ------------------------------------------------------------------------
@@ -346,15 +346,9 @@ static void rrdset_react_callback(const DICTIONARY_ITEM *item __maybe_unused, vo
346 RRDSET *st = rrdset;
347 RRDHOST *host = st->rrdhost;
348
349 - if(host->health_enabled && (ctr->react_action & (RRDSET_REACT_NEW | RRDSET_REACT_CHART_ACTIVATED))) {
350 - rrdsetvar_add_and_leave_released(st, "last_collected_t", RRDVAR_TYPE_TIME_T, &st->last_collected_time.tv_sec, RRDVAR_FLAG_NONE);
351 - rrdsetvar_add_and_leave_released(st, "collected_total_raw", RRDVAR_TYPE_TOTAL, &st->last_collected_total, RRDVAR_FLAG_NONE);
352 - rrdsetvar_add_and_leave_released(st, "green", RRDVAR_TYPE_CALCULATED, &st->green, RRDVAR_FLAG_NONE);
353 - rrdsetvar_add_and_leave_released(st, "red", RRDVAR_TYPE_CALCULATED, &st->red, RRDVAR_FLAG_NONE);
354 - rrdsetvar_add_and_leave_released(st, "update_every", RRDVAR_TYPE_INT, &st->update_every, RRDVAR_FLAG_NONE);
355 -
356 - rrdcalc_link_matching_alerts_to_rrdset(st);
357 - rrdcalctemplate_link_matching_templates_to_rrdset(st);
349 + if((host->health_enabled && (ctr->react_action & (RRDSET_REACT_NEW | RRDSET_REACT_CHART_ACTIVATED))) && !rrdset_is_ar_chart(st)) {
350 + rrdset_flag_set(st, RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION);
351 + rrdhost_flag_set(st->rrdhost, RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION);
352 }
353
354 if(ctr->react_action & (RRDSET_REACT_CHART_ARCHIVED_TO_LIVE | RRDSET_REACT_PLUGIN_UPDATED | RRDSET_REACT_MODULE_UPDATED)) {
@@ -1093,6 +1087,7 @@ static void store_metric(RRDDIM *rd, usec_t point_end_time_ut, NETDATA_DOUBLE n,
1087 rrdcontext_collected_rrddim(rd);
1088 }
1089
1090 +// caching of dimensions rrdset_done() and rrdset_done_interpolate() loop through
1091 struct rda_item {
1092 const DICTIONARY_ITEM *item;
1093 RRDDIM *rd;
@@ -1492,7 +1487,7 @@ void rrdset_done(RRDSET *st) {
1487 after_first_database_work:
1488 st->counter_done++;
1489
1495 - if(unlikely(st->rrdhost->rrdpush_send_enabled))
1490 + if(unlikely(rrdhost_has_rrdpush_sender_enabled(st->rrdhost)))
1491 rrdset_done_push(st);
1492
1493 size_t rda_slots = dictionary_entries(st->rrddim_root_index);
@@ -1771,7 +1766,7 @@ after_first_database_work:
1766 rrdset_done_interpolate(
1767 st
1768 , rda_base
1774 - ,rda_slots
1769 + , rda_slots
1770 , update_every_ut
1771 , last_stored_ut
1772 , next_store_ut
database/sqlite/sqlite_aclk_node.c
+1 -2
@@ -86,8 +86,7 @@ void sql_build_node_info(struct aclk_database_worker_config *wc, struct aclk_dat
86 char *host_version = NULL;
87 if (host != localhost) {
88 netdata_mutex_lock(&host->receiver_lock);
89 - host_version =
90 - strdupz(host->receiver && host->receiver->program_version ? host->receiver->program_version : "unknown");
89 + host_version = strdupz(host->receiver && host->receiver->program_version ? host->receiver->program_version : "unknown");
90 netdata_mutex_unlock(&host->receiver_lock);
91 }
92
database/sqlite/sqlite_functions.c
+2 -2
@@ -134,7 +134,7 @@ static void add_stmt_to_list(sqlite3_stmt *res)
134 static void release_statement(void *statement)
135 {
136 int rc;
137 -#ifdef NETDATA_INTERNAL_CHECKS
137 +#ifdef NETDATA_DEV_MODE
138 info("Thread %d: Cleaning prepared statement on %p", gettid(), statement);
139 #endif
140 if (unlikely(rc = sqlite3_finalize((sqlite3_stmt *) statement) != SQLITE_OK))
@@ -155,7 +155,7 @@ int prepare_statement(sqlite3 *database, char *query, sqlite3_stmt **statement)
155 if (likely(rc == SQLITE_OK)) {
156 if (likely(key)) {
157 ret = pthread_setspecific(*key, *statement);
158 -#ifdef NETDATA_INTERNAL_CHECKS
158 +#ifdef NETDATA_DEV_MODE
159 info("Thread %d: Using key %u on statement %p", gettid(), keys_used, *statement);
160 #endif
161 }
exporting/exporting_engine.c
+1 -1
@@ -7,7 +7,7 @@ static struct engine *engine = NULL;
7 void analytics_exporting_connectors_ssl(BUFFER *b)
8 {
9 #ifdef ENABLE_HTTPS
10 - if (netdata_exporting_ctx) {
10 + if (netdata_ssl_exporting_ctx) {
11 for (struct instance *instance = engine->instance_root; instance; instance = instance->next) {
12 struct simple_connector_data *connector_specific_data = instance->connector_specific_data;
13 if (connector_specific_data->flags == NETDATA_SSL_HANDSHAKE_COMPLETE) {
exporting/send_data.c
+3 -3
@@ -313,7 +313,7 @@ void simple_connector_worker(void *instance_p)
313 if (unlikely(sock == -1)) {
314 size_t reconnects = 0;
315
316 - sock = connect_to_one_of(
316 + sock = connect_to_one_of_urls(
317 instance->config.destination,
318 connector_specific_config->default_port,
319 &timeout,
@@ -322,12 +322,12 @@ void simple_connector_worker(void *instance_p)
322 CONNECTED_TO_MAX);
323 #ifdef ENABLE_HTTPS
324 if (exporting_tls_is_enabled(instance->config.type, options) && sock != -1) {
325 - if (netdata_exporting_ctx) {
325 + if (netdata_ssl_exporting_ctx) {
326 if (sock_delnonblock(sock) < 0)
327 error("Exporting cannot remove the non-blocking flag from socket %d", sock);
328
329 if (connector_specific_data->conn == NULL) {
330 - connector_specific_data->conn = SSL_new(netdata_exporting_ctx);
330 + connector_specific_data->conn = SSL_new(netdata_ssl_exporting_ctx);
331 if (connector_specific_data->conn == NULL) {
332 error("Failed to allocate SSL structure to socket %d.", sock);
333 connector_specific_data->flags = NETDATA_SSL_NO_HANDSHAKE;
health/health.c
+53 -26
@@ -2,6 +2,22 @@
2
3 #include "health.h"
4
5 +#define WORKER_HEALTH_JOB_RRD_LOCK 0
6 +#define WORKER_HEALTH_JOB_HOST_LOCK 1
7 +#define WORKER_HEALTH_JOB_DB_QUERY 2
8 +#define WORKER_HEALTH_JOB_CALC_EVAL 3
9 +#define WORKER_HEALTH_JOB_WARNING_EVAL 4
10 +#define WORKER_HEALTH_JOB_CRITICAL_EVAL 5
11 +#define WORKER_HEALTH_JOB_ALARM_LOG_ENTRY 6
12 +#define WORKER_HEALTH_JOB_ALARM_LOG_PROCESS 7
13 +#define WORKER_HEALTH_JOB_DELAYED_INIT_RRDSET 8
14 +#define WORKER_HEALTH_JOB_DELAYED_INIT_RRDDIM 9
15 +
16 +#if WORKER_UTILIZATION_MAX_JOB_TYPES < 10
17 +#error WORKER_UTILIZATION_MAX_JOB_TYPES has to be at least 10
18 +#endif
19 +
20 +
21 unsigned int default_health_enabled = 1;
22 char *silencers_filename;
23
@@ -659,22 +675,48 @@ static int update_disabled_silenced(RRDHOST *host, RRDCALC *rc) {
675 return 0;
676 }
677
662 -// Create alarms for dimensions that have been added to charts
663 -// since the previous iteration.
664 -static void health_execute_pending_updates(RRDHOST *host) {
678 +static void health_execute_delayed_initializations(RRDHOST *host) {
679 RRDSET *st;
680
667 - if (!rrdhost_flag_check(host, RRDHOST_FLAG_PENDING_FOREACH_ALARMS))
668 - return;
681 + if (!rrdhost_flag_check(host, RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION)) return;
682 + rrdhost_flag_clear(host, RRDHOST_FLAG_PENDING_HEALTH_INITIALIZATION);
683
684 rrdset_foreach_reentrant(st, host) {
671 - if(!rrdset_flag_check(st, RRDSET_FLAG_PENDING_FOREACH_ALARMS))
685 + if(!rrdset_flag_check(st, RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION)) continue;
686 + rrdset_flag_clear(st, RRDSET_FLAG_PENDING_HEALTH_INITIALIZATION);
687 +
688 + if(unlikely(rrdset_is_ar_chart(st)))
689 continue;
690
691 + worker_is_busy(WORKER_HEALTH_JOB_DELAYED_INIT_RRDSET);
692 +
693 + if(!st->rrdfamily)
694 + st->rrdfamily = rrdfamily_add_and_acquire(host, rrdset_family(st));
695 +
696 + if(!st->rrdvars)
697 + st->rrdvars = rrdvariables_create();
698 +
699 + rrddimvar_index_init(st);
700 +
701 + rrdsetvar_add_and_leave_released(st, "last_collected_t", RRDVAR_TYPE_TIME_T, &st->last_collected_time.tv_sec, RRDVAR_FLAG_NONE);
702 + rrdsetvar_add_and_leave_released(st, "collected_total_raw", RRDVAR_TYPE_TOTAL, &st->last_collected_total, RRDVAR_FLAG_NONE);
703 + rrdsetvar_add_and_leave_released(st, "green", RRDVAR_TYPE_CALCULATED, &st->green, RRDVAR_FLAG_NONE);
704 + rrdsetvar_add_and_leave_released(st, "red", RRDVAR_TYPE_CALCULATED, &st->red, RRDVAR_FLAG_NONE);
705 + rrdsetvar_add_and_leave_released(st, "update_every", RRDVAR_TYPE_INT, &st->update_every, RRDVAR_FLAG_NONE);
706 +
707 + rrdcalc_link_matching_alerts_to_rrdset(st);
708 + rrdcalctemplate_link_matching_templates_to_rrdset(st);
709 +
710 RRDDIM *rd;
711 rrddim_foreach_read(rd, st) {
676 - if(!rrddim_flag_check(rd, RRDDIM_FLAG_PENDING_FOREACH_ALARMS))
677 - continue;
712 + if(!rrddim_flag_check(rd, RRDDIM_FLAG_PENDING_HEALTH_INITIALIZATION)) continue;
713 + rrddim_flag_clear(rd, RRDDIM_FLAG_PENDING_HEALTH_INITIALIZATION);
714 +
715 + worker_is_busy(WORKER_HEALTH_JOB_DELAYED_INIT_RRDDIM);
716 +
717 + rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_CALCULATED, NULL, NULL, &rd->last_stored_value, RRDVAR_FLAG_NONE);
718 + rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_COLLECTED, NULL, "_raw", &rd->last_collected_value, RRDVAR_FLAG_NONE);
719 + rrddimvar_add_and_leave_released(rd, RRDVAR_TYPE_TIME_T, NULL, "_last_collected_t", &rd->last_collected_time.tv_sec, RRDVAR_FLAG_NONE);
720
721 RRDCALCTEMPLATE *rt;
722 foreach_rrdcalctemplate_read(host, rt) {
@@ -685,14 +727,10 @@ static void health_execute_pending_updates(RRDHOST *host) {
727 rrdcalctemplate_check_rrddim_conditions_and_link(rt, st, rd, host);
728 }
729 foreach_rrdcalctemplate_done(rt);
688 -
689 - rrddim_flag_clear(rd, RRDDIM_FLAG_PENDING_FOREACH_ALARMS);
730 }
731 rrddim_foreach_done(rd);
692 - rrdset_flag_clear(st, RRDSET_FLAG_PENDING_FOREACH_ALARMS);
732 }
733 rrdset_foreach_done(st);
695 - rrdhost_flag_clear(host, RRDHOST_FLAG_PENDING_FOREACH_ALARMS);
734 }
735
736 /**
@@ -705,19 +743,6 @@ static void health_execute_pending_updates(RRDHOST *host) {
743 * @return It always returns NULL
744 */
745
708 -#define WORKER_HEALTH_JOB_RRD_LOCK 0
709 -#define WORKER_HEALTH_JOB_HOST_LOCK 1
710 -#define WORKER_HEALTH_JOB_DB_QUERY 2
711 -#define WORKER_HEALTH_JOB_CALC_EVAL 3
712 -#define WORKER_HEALTH_JOB_WARNING_EVAL 4
713 -#define WORKER_HEALTH_JOB_CRITICAL_EVAL 5
714 -#define WORKER_HEALTH_JOB_ALARM_LOG_ENTRY 6
715 -#define WORKER_HEALTH_JOB_ALARM_LOG_PROCESS 7
716 -
717 -#if WORKER_UTILIZATION_MAX_JOB_TYPES < 8
718 -#error WORKER_UTILIZATION_MAX_JOB_TYPES has to be at least 8
719 -#endif
720 -
746 void *health_main(void *ptr) {
747 worker_register("HEALTH");
748 worker_register_job_name(WORKER_HEALTH_JOB_RRD_LOCK, "rrd lock");
@@ -728,6 +753,8 @@ void *health_main(void *ptr) {
753 worker_register_job_name(WORKER_HEALTH_JOB_CRITICAL_EVAL, "critical eval");
754 worker_register_job_name(WORKER_HEALTH_JOB_ALARM_LOG_ENTRY, "alarm log entry");
755 worker_register_job_name(WORKER_HEALTH_JOB_ALARM_LOG_PROCESS, "alarm log process");
756 + worker_register_job_name(WORKER_HEALTH_JOB_DELAYED_INIT_RRDSET, "rrdset init");
757 + worker_register_job_name(WORKER_HEALTH_JOB_DELAYED_INIT_RRDDIM, "rrddim init");
758
759 netdata_thread_cleanup_push(health_main_cleanup, ptr);
760
@@ -809,7 +836,7 @@ void *health_main(void *ptr) {
836 if(likely(!host->health_log_fp) && (loop == 1 || loop % cleanup_sql_every_loop == 0))
837 sql_health_alarm_log_cleanup(host);
838
812 - health_execute_pending_updates(host);
839 + health_execute_delayed_initializations(host);
840
841 worker_is_busy(WORKER_HEALTH_JOB_HOST_LOCK);
842
libnetdata/buffer/buffer.c
+48
@@ -148,6 +148,54 @@ void buffer_print_ll(BUFFER *wb, long long value)
148 buffer_print_llu(wb, value);
149 }
150
151 +static unsigned char bits03_to_hex[16] = {
152 + [0] = '0',
153 + [1] = '1',
154 + [2] = '2',
155 + [3] = '3',
156 + [4] = '4',
157 + [5] = '5',
158 + [6] = '6',
159 + [7] = '7',
160 + [8] = '8',
161 + [9] = '9',
162 + [10] = 'A',
163 + [11] = 'B',
164 + [12] = 'C',
165 + [13] = 'D',
166 + [14] = 'E',
167 + [15] = 'F'
168 +};
169 +
170 +void buffer_print_llu_hex(BUFFER *wb, unsigned long long value)
171 +{
172 + unsigned char buffer[sizeof(unsigned long long) * 2 + 2 + 1]; // 8 bytes * 2 + '0x' + '\0'
173 + unsigned char *e = &buffer[sizeof(unsigned long long) * 2 + 2];
174 + unsigned char *p = e;
175 +
176 + *p-- = '\0';
177 + *p-- = bits03_to_hex[value & 0xF];
178 + value >>= 4;
179 + if(value) {
180 + *p-- = bits03_to_hex[value & 0xF];
181 + value >>= 4;
182 +
183 + while(value) {
184 + *p-- = bits03_to_hex[value & 0xF];
185 + value >>= 4;
186 +
187 + if(value) {
188 + *p-- = bits03_to_hex[value & 0xF];
189 + value >>= 4;
190 + }
191 + }
192 + }
193 + *p-- = 'x';
194 + *p = '0';
195 +
196 + buffer_fast_strcat(wb, (char *)p, e - p);
197 +}
198 +
199 void buffer_fast_strcat(BUFFER *wb, const char *txt, size_t len) {
200 if(unlikely(!txt || !*txt)) return;
201
libnetdata/buffer/buffer.h
+1
@@ -79,6 +79,7 @@ extern char *print_number_llu_r_smart(char *str, unsigned long long uvalue);
79
80 extern void buffer_print_llu(BUFFER *wb, unsigned long long uvalue);
81 extern void buffer_print_ll(BUFFER *wb, long long value);
82 +extern void buffer_print_llu_hex(BUFFER *wb, unsigned long long value);
83
84 static inline void buffer_need_bytes(BUFFER *buffer, size_t needed_free_size) {
85 if(unlikely(buffer->size - buffer->len < needed_free_size))
libnetdata/circular_buffer/circular_buffer.c
+5
@@ -89,3 +89,8 @@ size_t cbuffer_next_unsafe(struct circular_buffer *buf, char **start) {
89 }
90 return buf->size - buf->read;
91 }
92 +
93 +void cbuffer_flush(struct circular_buffer*buf) {
94 + buf->write = 0;
95 + buf->read = 0;
96 +}
\ No newline at end of file
libnetdata/circular_buffer/circular_buffer.h
+1
@@ -14,5 +14,6 @@ extern int cbuffer_add_unsafe(struct circular_buffer *buf, const char *d, size_t
14 extern void cbuffer_remove_unsafe(struct circular_buffer *buf, size_t num);
15 extern size_t cbuffer_next_unsafe(struct circular_buffer *buf, char **start);
16 extern size_t cbuffer_available_size_unsafe(struct circular_buffer *buf);
17 +extern void cbuffer_flush(struct circular_buffer*buf);
18
19 #endif
libnetdata/config/appconfig.c
+4 -4
@@ -32,8 +32,8 @@ _CONNECTOR_INSTANCE *add_connector_instance(struct section *connector, struct se
32 local_ci = callocz(1, sizeof(struct _connector_instance));
33 local_ci->instance = instance;
34 local_ci->connector = connector;
35 - strncpy(local_ci->instance_name, instance->name, CONFIG_MAX_NAME);
36 - strncpy(local_ci->connector_name, connector->name, CONFIG_MAX_NAME);
35 + strncpyz(local_ci->instance_name, instance->name, CONFIG_MAX_NAME);
36 + strncpyz(local_ci->connector_name, connector->name, CONFIG_MAX_NAME);
37 local_ci->next = global_connector_instance;
38 global_connector_instance = local_ci;
39
@@ -686,14 +686,14 @@ int appconfig_load(struct config *root, char *filename, int overwrite_used, cons
686 int rc;
687 rc = is_valid_connector(s, 0);
688 if (likely(rc)) {
689 - strncpy(working_connector, s, CONFIG_MAX_NAME);
689 + strncpyz(working_connector, s, CONFIG_MAX_NAME);
690 s = s + rc + 1;
691 if (unlikely(!(*s))) {
692 _connectors++;
693 sprintf(buffer, "instance_%d", _connectors);
694 s = buffer;
695 }
696 - strncpy(working_instance, s, CONFIG_MAX_NAME);
696 + strncpyz(working_instance, s, CONFIG_MAX_NAME);
697 working_connector_section = NULL;
698 if (unlikely(appconfig_section_find(root, working_instance))) {
699 error("Instance (%s) already exists", working_instance);
libnetdata/dictionary/dictionary.c
+23 -4
@@ -249,6 +249,9 @@ void dictionary_register_conflict_callback(DICTIONARY *dict, bool (*conflict_cal
249 if(unlikely(is_view_dictionary(dict)))
250 fatal("DICTIONARY: called %s() on a view.", __FUNCTION__ );
251
252 + internal_error(!(dict->options & DICT_OPTION_DONT_OVERWRITE_VALUE), "DICTIONARY: registering conflict callback without DICT_OPTION_DONT_OVERWRITE_VALUE");
253 + dict->options |= DICT_OPTION_DONT_OVERWRITE_VALUE;
254 +
255 dictionary_hooks_allocate(dict);
256 dict->hooks->conflict_callback = conflict_callback;
257 dict->hooks->conflict_callback_data = data;
@@ -457,6 +460,10 @@ static inline REFCOUNT DICTIONARY_ITEM_REFCOUNT_GET(DICTIONARY *dict, DICTIONARY
460 return (REFCOUNT)__atomic_load_n(&item->refcount, __ATOMIC_SEQ_CST);
461 }
462
463 +static inline REFCOUNT DICTIONARY_ITEM_REFCOUNT_GET_SOLE(DICTIONARY_ITEM *item) {
464 + return (REFCOUNT)__atomic_load_n(&item->refcount, __ATOMIC_SEQ_CST);
465 +}
466 +
467 // ----------------------------------------------------------------------------
468 // callbacks execution
469
@@ -617,6 +624,12 @@ static void ll_recursive_unlock(DICTIONARY *dict, char rw) {
624 }
625 }
626
627 +void dictionary_write_lock(DICTIONARY *dict) {
628 + ll_recursive_lock(dict, DICTIONARY_LOCK_WRITE);
629 +}
630 +void dictionary_write_unlock(DICTIONARY *dict) {
631 + ll_recursive_unlock(dict, DICTIONARY_LOCK_WRITE);
632 +}
633
634 static inline void dictionary_index_lock_rdlock(DICTIONARY *dict) {
635 if(unlikely(is_dictionary_single_threaded(dict)))
@@ -1494,7 +1507,9 @@ static bool dictionary_free_all_resources(DICTIONARY *dict, size_t *mem, bool fo
1507 #endif
1508
1509 // destroy the index
1510 + dictionary_index_lock_wrlock(dict);
1511 index_size += hashtable_destroy_unsafe(dict);
1512 + dictionary_index_lock_unlock(dict);
1513
1514 ll_recursive_lock(dict, DICTIONARY_LOCK_WRITE);
1515 DICTIONARY_ITEM *item = dict->items.list;
@@ -1797,6 +1812,7 @@ size_t dictionary_destroy(DICTIONARY *dict) {
1812
1813 if(!dict) return 0;
1814
1815 + dict_flag_set(dict, DICT_FLAG_DESTROYED);
1816 DICTIONARY_STATS_DICT_DESTRUCTIONS_PLUS1(dict);
1817
1818 size_t referenced_items = dictionary_referenced_items(dict);
@@ -1936,20 +1952,23 @@ void dictionary_acquired_item_release(DICTIONARY *dict, DICT_ITEM_CONST DICTIONA
1952 // get the name/value of an item
1953
1954 const char *dictionary_acquired_item_name(DICT_ITEM_CONST DICTIONARY_ITEM *item) {
1939 - api_internal_check(NULL, item, true, false);
1955 return item_get_name(item);
1956 }
1957
1958 void *dictionary_acquired_item_value(DICT_ITEM_CONST DICTIONARY_ITEM *item) {
1944 - // we allow the item to be NULL here
1945 - api_internal_check(NULL, item, true, true);
1946 -
1959 if(likely(item))
1960 return item->shared->value;
1961
1962 return NULL;
1963 }
1964
1965 +size_t dictionary_acquired_item_references(DICT_ITEM_CONST DICTIONARY_ITEM *item) {
1966 + if(likely(item))
1967 + return DICTIONARY_ITEM_REFCOUNT_GET_SOLE(item);
1968 +
1969 + return 0;
1970 +}
1971 +
1972 // ----------------------------------------------------------------------------
1973 // DEL an item
1974
libnetdata/dictionary/dictionary.h
+5 -1
@@ -180,7 +180,7 @@ extern DICT_ITEM_CONST DICTIONARY_ITEM *dictionary_set_and_acquire_item_advanced
180 #define dictionary_view_set_and_acquire_item(dict, name, master_item) dictionary_view_set_and_acquire_item_advanced(dict, name, -1, master_item)
181 extern DICT_ITEM_CONST DICTIONARY_ITEM *dictionary_view_set_and_acquire_item_advanced(DICTIONARY *dict, const char *name, ssize_t name_len, DICTIONARY_ITEM *master_item);
182 #define dictionary_view_set(dict, name, master_item) dictionary_view_set_advanced(dict, name, -1, master_item)
183 -extern void *dictionary_view_set_advanced(DICTIONARY *dict, const char *name, ssize_t name_len, DICTIONARY_ITEM *master_item);
183 +extern void *dictionary_view_set_advanced(DICTIONARY *dict, const char *name, ssize_t name_len, DICT_ITEM_CONST DICTIONARY_ITEM *master_item);
184
185 // ----------------------------------------------------------------------------
186 // Get an item from the dictionary
@@ -211,6 +211,7 @@ extern DICT_ITEM_CONST DICTIONARY_ITEM *dictionary_acquired_item_dup(DICTIONARY
211 extern const char *dictionary_acquired_item_name(DICT_ITEM_CONST DICTIONARY_ITEM *item);
212 extern void *dictionary_acquired_item_value(DICT_ITEM_CONST DICTIONARY_ITEM *item);
213
214 +extern size_t dictionary_acquired_item_references(DICT_ITEM_CONST DICTIONARY_ITEM *item);
215
216 // ----------------------------------------------------------------------------
217 // Traverse (walk through) the items of the dictionary.
@@ -252,6 +253,9 @@ int dictionary_sorted_walkthrough_rw(DICTIONARY *dict, char rw, int (*callback)(
253 #define DICTIONARY_LOCK_WRITE 'w'
254 #define DICTIONARY_LOCK_REENTRANT 'z'
255
256 +extern void dictionary_write_lock(DICTIONARY *dict);
257 +extern void dictionary_write_unlock(DICTIONARY *dict);
258 +
259 typedef DICTFE_CONST struct dictionary_foreach {
260 DICTIONARY *dict; // the dictionary upon we work
261
libnetdata/inlined.h
+20 -21
@@ -42,24 +42,12 @@ static inline uint32_t simple_uhash(const char *name) {
42 return hval;
43 }
44
45 -static inline int simple_hash_strcmp(const char *name, const char *b, uint32_t *hash) {
46 - unsigned char *s = (unsigned char *) name;
47 - uint32_t hval = 0x811c9dc5;
48 - int ret = 0;
49 - while (*s) {
50 - if(!ret) ret = *s - *b++;
51 - hval *= 16777619;
52 - hval ^= (uint32_t) *s++;
53 - }
54 - *hash = hval;
55 - return ret;
56 -}
57 -
45 static inline int str2i(const char *s) {
46 int n = 0;
60 - char c, negative = (*s == '-');
47 + char c, negative = (char)(*s == '-');
48 + const char *e = &s[30]; // max number of character to iterate
49
62 - for(c = (negative)?*(++s):*s; c >= '0' && c <= '9' ; c = *(++s)) {
50 + for(c = (char)((negative)?*(++s):*s); c >= '0' && c <= '9' && s < e ; c = *(++s)) {
51 n *= 10;
52 n += c - '0';
53 }
@@ -73,8 +61,9 @@ static inline int str2i(const char *s) {
61 static inline long str2l(const char *s) {
62 long n = 0;
63 char c, negative = (*s == '-');
64 + const char *e = &s[30]; // max number of character to iterate
65
77 - for(c = (negative)?*(++s):*s; c >= '0' && c <= '9' ; c = *(++s)) {
66 + for(c = (negative)?*(++s):*s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
67 n *= 10;
68 n += c - '0';
69 }
@@ -88,7 +77,9 @@ static inline long str2l(const char *s) {
77 static inline uint32_t str2uint32_t(const char *s) {
78 uint32_t n = 0;
79 char c;
91 - for(c = *s; c >= '0' && c <= '9' ; c = *(++s)) {
80 + const char *e = &s[30]; // max number of character to iterate
81 +
82 + for(c = *s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
83 n *= 10;
84 n += c - '0';
85 }
@@ -98,7 +89,9 @@ static inline uint32_t str2uint32_t(const char *s) {
89 static inline uint64_t str2uint64_t(const char *s) {
90 uint64_t n = 0;
91 char c;
101 - for(c = *s; c >= '0' && c <= '9' ; c = *(++s)) {
92 + const char *e = &s[30]; // max number of character to iterate
93 +
94 + for(c = *s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
95 n *= 10;
96 n += c - '0';
97 }
@@ -108,7 +101,9 @@ static inline uint64_t str2uint64_t(const char *s) {
101 static inline unsigned long str2ul(const char *s) {
102 unsigned long n = 0;
103 char c;
111 - for(c = *s; c >= '0' && c <= '9' ; c = *(++s)) {
104 + const char *e = &s[30]; // max number of character to iterate
105 +
106 + for(c = *s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
107 n *= 10;
108 n += c - '0';
109 }
@@ -118,7 +113,9 @@ static inline unsigned long str2ul(const char *s) {
113 static inline unsigned long long str2ull(const char *s) {
114 unsigned long long n = 0;
115 char c;
121 - for(c = *s; c >= '0' && c <= '9' ; c = *(++s)) {
116 + const char *e = &s[30]; // max number of character to iterate
117 +
118 + for(c = *s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
119 n *= 10;
120 n += c - '0';
121 }
@@ -137,7 +134,9 @@ static inline long long str2ll(const char *s, char **endptr) {
134
135 long long n = 0;
136 char c;
140 - for(c = *s; c >= '0' && c <= '9' ; c = *(++s)) {
137 + const char *e = &s[30]; // max number of character to iterate
138 +
139 + for(c = *s; c >= '0' && c <= '9' && s < e ; c = *(++s)) {
140 n *= 10;
141 n += c - '0';
142 }
libnetdata/libnetdata.c
+133
@@ -1535,6 +1535,121 @@ char *find_and_replace(const char *src, const char *find, const char *replace, c
1535 return value;
1536 }
1537
1538 +inline int pluginsd_space(char c) {
1539 + switch(c) {
1540 + case ' ':
1541 + case '\t':
1542 + case '\r':
1543 + case '\n':
1544 + case '=':
1545 + return 1;
1546 +
1547 + default:
1548 + return 0;
1549 + }
1550 +}
1551 +
1552 +inline int config_isspace(char c)
1553 +{
1554 + switch (c) {
1555 + case ' ':
1556 + case '\t':
1557 + case '\r':
1558 + case '\n':
1559 + case ',':
1560 + return 1;
1561 +
1562 + default:
1563 + return 0;
1564 + }
1565 +}
1566 +
1567 +// split a text into words, respecting quotes
1568 +inline int quoted_strings_splitter(char *str, char **words, int max_words, int (*custom_isspace)(char), char *recover_input, char **recover_location, int max_recover)
1569 +{
1570 + char *s = str, quote = 0;
1571 + int i = 0, rec = 0;
1572 + char *recover = recover_input;
1573 +
1574 + // skip all white space
1575 + while (unlikely(custom_isspace(*s)))
1576 + s++;
1577 +
1578 + // check for quote
1579 + if (unlikely(*s == '\'' || *s == '"')) {
1580 + quote = *s; // remember the quote
1581 + s++; // skip the quote
1582 + }
1583 +
1584 + // store the first word
1585 + words[i++] = s;
1586 +
1587 + // while we have something
1588 + while (likely(*s)) {
1589 + // if it is escape
1590 + if (unlikely(*s == '\\' && s[1])) {
1591 + s += 2;
1592 + continue;
1593 + }
1594 +
1595 + // if it is quote
1596 + else if (unlikely(*s == quote)) {
1597 + quote = 0;
1598 + if (recover && rec < max_recover) {
1599 + recover_location[rec++] = s;
1600 + *recover++ = *s;
1601 + }
1602 + *s = ' ';
1603 + continue;
1604 + }
1605 +
1606 + // if it is a space
1607 + else if (unlikely(quote == 0 && custom_isspace(*s))) {
1608 + // terminate the word
1609 + if (recover && rec < max_recover) {
1610 + if (!rec || recover_location[rec-1] != s) {
1611 + recover_location[rec++] = s;
1612 + *recover++ = *s;
1613 + }
1614 + }
1615 + *s++ = '\0';
1616 +
1617 + // skip all white space
1618 + while (likely(custom_isspace(*s)))
1619 + s++;
1620 +
1621 + // check for quote
1622 + if (unlikely(*s == '\'' || *s == '"')) {
1623 + quote = *s; // remember the quote
1624 + s++; // skip the quote
1625 + }
1626 +
1627 + // if we reached the end, stop
1628 + if (unlikely(!*s))
1629 + break;
1630 +
1631 + // store the next word
1632 + if (likely(i < max_words))
1633 + words[i++] = s;
1634 + else
1635 + break;
1636 + }
1637 +
1638 + // anything else
1639 + else
1640 + s++;
1641 + }
1642 +
1643 + // terminate the words
1644 + memset(&words[i], 0, (max_words - i) * sizeof (char *));
1645 +
1646 + return i;
1647 +}
1648 +
1649 +inline int pluginsd_split_words(char *str, char **words, int max_words, char *recover_input, char **recover_location, int max_recover)
1650 +{
1651 + return quoted_strings_splitter(str, words, max_words, pluginsd_space, recover_input, recover_location, max_recover);
1652 +}
1653
1654 bool bitmap256_get_bit(BITMAP256 *ptr, uint8_t idx) {
1655 if (unlikely(!ptr))
@@ -1550,3 +1665,21 @@ void bitmap256_set_bit(BITMAP256 *ptr, uint8_t idx, bool value) {
1665 else
1666 ptr->data[idx / 64] &= ~(1ULL << (idx % 64));
1667 }
1668 +
1669 +bool run_command_and_copy_output_to_stdout(const char *command, int max_line_length) {
1670 + pid_t pid;
1671 + FILE *fp = netdata_popen(command, &pid, NULL);
1672 +
1673 + if(fp) {
1674 + char buffer[max_line_length + 1];
1675 + while (fgets(buffer, max_line_length, fp))
1676 + fprintf(stdout, "%s", buffer);
1677 + }
1678 + else {
1679 + error("Failed to execute command '%s'.", command);
1680 + return false;
1681 + }
1682 +
1683 + netdata_pclose(NULL, fp, pid);
1684 + return true;
1685 +}
libnetdata/libnetdata.h
+9
@@ -388,6 +388,15 @@ typedef struct bitmap256 {
388 extern bool bitmap256_get_bit(BITMAP256 *ptr, uint8_t idx);
389 extern void bitmap256_set_bit(BITMAP256 *ptr, uint8_t idx, bool value);
390
391 +#define COMPRESSION_MAX_MSG_SIZE 0x4000
392 +#define PLUGINSD_LINE_MAX (COMPRESSION_MAX_MSG_SIZE - 1024)
393 +extern int config_isspace(char c);
394 +extern int pluginsd_space(char c);
395 +int quoted_strings_splitter(char *str, char **words, int max_words, int (*custom_isspace)(char), char *recover_input, char **recover_location, int max_recover);
396 +extern int pluginsd_split_words(char *str, char **words, int max_words, char *recover_string, char **recover_location, int max_recover);
397 +
398 +extern bool run_command_and_copy_output_to_stdout(const char *command, int max_line_length);
399 +
400 extern void netdata_cleanup_and_exit(int ret) NORETURN;
401 extern void send_statistics(const char *action, const char *action_result, const char *action_data);
402 extern char *netdata_configured_host_prefix;
libnetdata/popen/popen.c
+244 -193
@@ -2,81 +2,129 @@
2
3 #include "../libnetdata.h"
4
5 -static pthread_mutex_t myp_lock;
6 -static int myp_tracking = 0;
5 +// ----------------------------------------------------------------------------
6 +// popen with tracking
7
8 -struct mypopen {
8 +static pthread_mutex_t netdata_popen_tracking_mutex;
9 +static bool netdata_popen_tracking_enabled = false;
10 +
11 +struct netdata_popen {
12 pid_t pid;
10 - struct mypopen *next;
11 - struct mypopen *prev;
13 + struct netdata_popen *next;
14 + struct netdata_popen *prev;
15 };
16
14 -static struct mypopen *mypopen_root = NULL;
17 +static struct netdata_popen *netdata_popen_root = NULL;
18
19 // myp_add_lock takes the lock if we're tracking.
17 -static void myp_add_lock(void) {
18 - if (myp_tracking == 0)
20 +static void netdata_popen_tracking_lock(void) {
21 + if(!netdata_popen_tracking_enabled)
22 return;
23
21 - netdata_mutex_lock(&myp_lock);
24 + netdata_mutex_lock(&netdata_popen_tracking_mutex);
25 }
26
27 // myp_add_unlock release the lock if we're tracking.
25 -static void myp_add_unlock(void) {
26 - if (myp_tracking == 0)
28 +static void netdata_popen_tracking_unlock(void) {
29 + if(!netdata_popen_tracking_enabled)
30 return;
31
29 - netdata_mutex_unlock(&myp_lock);
32 + netdata_mutex_unlock(&netdata_popen_tracking_mutex);
33 }
34
35 // myp_add_locked adds pid if we're tracking.
36 // myp_add_lock must have been called previously.
34 -static void myp_add_locked(pid_t pid) {
35 - struct mypopen *mp;
36 -
37 - if (myp_tracking == 0)
37 +static void netdata_popen_tracking_add_pid_unsafe(pid_t pid) {
38 + if(!netdata_popen_tracking_enabled)
39 return;
40
40 - mp = mallocz(sizeof(struct mypopen));
41 + struct netdata_popen *mp;
42 +
43 + mp = mallocz(sizeof(struct netdata_popen));
44 mp->pid = pid;
45
43 - mp->next = mypopen_root;
44 - mp->prev = NULL;
45 - if (mypopen_root != NULL)
46 - mypopen_root->prev = mp;
47 - mypopen_root = mp;
48 - netdata_mutex_unlock(&myp_lock);
46 + DOUBLE_LINKED_LIST_PREPEND_UNSAFE(netdata_popen_root, mp, prev, next);
47 }
48
49 // myp_del deletes pid if we're tracking.
52 -static void myp_del(pid_t pid) {
53 - struct mypopen *mp;
54 -
55 - if (myp_tracking == 0)
50 +static void netdata_popen_tracking_del_pid(pid_t pid) {
51 + if(!netdata_popen_tracking_enabled)
52 return;
53
58 - netdata_mutex_lock(&myp_lock);
59 - for (mp = mypopen_root; mp != NULL; mp = mp->next) {
60 - if (mp->pid == pid) {
61 - if (mp->next != NULL)
62 - mp->next->prev = mp->prev;
63 - if (mp->prev != NULL)
64 - mp->prev->next = mp->next;
65 - if (mypopen_root == mp)
66 - mypopen_root = mp->next;
67 - freez(mp);
54 + struct netdata_popen *mp;
55 +
56 + netdata_mutex_lock(&netdata_popen_tracking_mutex);
57 +
58 + DOUBLE_LINKED_LIST_FOREACH_FORWARD(netdata_popen_root, mp, prev, next) {
59 + if(unlikely(mp->pid == pid))
60 break;
69 - }
61 }
62
72 - if (mp == NULL)
63 + if(mp) {
64 + DOUBLE_LINKED_LIST_REMOVE_UNSAFE(netdata_popen_root, mp, prev, next);
65 + freez(mp);
66 + }
67 + else
68 error("Cannot find pid %d.", pid);
69
75 - netdata_mutex_unlock(&myp_lock);
70 + netdata_mutex_unlock(&netdata_popen_tracking_mutex);
71 }
72
78 -#define PIPE_READ 0
79 -#define PIPE_WRITE 1
73 +// netdata_popen_tracking_init() should be called by apps which act as init
74 +// (pid 1) so that processes created by mypopen and mypopene
75 +// are tracked. This enables the reaper to ignore processes
76 +// which will be handled internally, by calling myp_reap, to
77 +// avoid issues with already reaped processes during wait calls.
78 +//
79 +// Callers should call myp_free() to clean up resources.
80 +void netdata_popen_tracking_init(void) {
81 + info("process tracking enabled.");
82 + netdata_popen_tracking_enabled = true;
83 +
84 + if (netdata_mutex_init(&netdata_popen_tracking_mutex) != 0)
85 + fatal("netdata_popen_tracking_init() mutex init failed.");
86 +}
87 +
88 +// myp_free cleans up any resources allocated for process
89 +// tracking.
90 +void netdata_popen_tracking_cleanup(void) {
91 + if(!netdata_popen_tracking_enabled)
92 + return;
93 +
94 + netdata_mutex_lock(&netdata_popen_tracking_mutex);
95 + netdata_popen_tracking_enabled = false;
96 +
97 + while(netdata_popen_root) {
98 + struct netdata_popen *mp = netdata_popen_root;
99 + DOUBLE_LINKED_LIST_REMOVE_UNSAFE(netdata_popen_root, mp, prev, next);
100 + freez(mp);
101 + }
102 +
103 + netdata_mutex_unlock(&netdata_popen_tracking_mutex);
104 +}
105 +
106 +// myp_reap returns 1 if pid should be reaped, 0 otherwise.
107 +int netdata_popen_tracking_pid_shoud_be_reaped(pid_t pid) {
108 + if(!netdata_popen_tracking_enabled)
109 + return 0;
110 +
111 + netdata_mutex_lock(&netdata_popen_tracking_mutex);
112 +
113 + int ret = 1;
114 + struct netdata_popen *mp;
115 + DOUBLE_LINKED_LIST_FOREACH_FORWARD(netdata_popen_root, mp, prev, next) {
116 + if(unlikely(mp->pid == pid)) {
117 + ret = 0;
118 + break;
119 + }
120 + }
121 +
122 + netdata_mutex_unlock(&netdata_popen_tracking_mutex);
123 + return ret;
124 +}
125 +
126 +// ----------------------------------------------------------------------------
127 +// helpers
128
129 static inline void convert_argv_to_string(char *dst, size_t size, const char *spawn_argv[]) {
130 int i;
@@ -89,118 +137,185 @@ static inline void convert_argv_to_string(char *dst, size_t size, const char *sp
137 }
138 }
139
140 +// ----------------------------------------------------------------------------
141 +// the core of netdata popen
142 +
143 /*
144 * Returns -1 on failure, 0 on success. When POPEN_FLAG_CREATE_PIPE is set, on success set the FILE *fp pointer.
145 */
95 -static int custom_popene(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp, const char *command, const char *spawn_argv[]) {
146 +#define PIPE_READ 0
147 +#define PIPE_WRITE 1
148 +
149 +static int popene_internal(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp_child_stdin, FILE **fpp_child_stdout, const char *command, const char *spawn_argv[]) {
150 // create a string to be logged about the command we are running
151 char command_to_be_logged[2048];
152 convert_argv_to_string(command_to_be_logged, sizeof(command_to_be_logged), spawn_argv);
153 // info("custom_popene() running command: %s", command_to_be_logged);
154
101 - FILE *fp = NULL;
102 - int ret = 0; // success by default
103 - int pipefd[2], error;
155 + int ret = 0; // success by default
156 + int attr_rc = 1; // failure by default
157 +
158 + FILE *fp_child_stdin = NULL, *fp_child_stdout = NULL;
159 + int pipefd_stdin[2] = { -1, -1 };
160 + int pipefd_stdout[2] = { -1, -1 };
161 +
162 pid_t pid;
163 posix_spawnattr_t attr;
164 posix_spawn_file_actions_t fa;
165
108 - if (flags & POPEN_FLAG_CREATE_PIPE) {
109 - if (pipe(pipefd) == -1)
110 - return -1;
111 - if ((fp = fdopen(pipefd[PIPE_READ], "r")) == NULL) {
112 - goto error_after_pipe;
166 + int stdin_fd_to_exclude_from_closing = -1;
167 + int stdout_fd_to_exclude_from_closing = -1;
168 +
169 + if(posix_spawn_file_actions_init(&fa)) {
170 + error("POPEN: posix_spawn_file_actions_init() failed.");
171 + return -1;
172 + }
173 +
174 + if(fpp_child_stdin) {
175 + if (pipe(pipefd_stdin) == -1) {
176 + error("POPEN: stdin pipe() failed");
177 + ret = -1;
178 + goto cleanup_and_return;
179 + }
180 +
181 + if ((fp_child_stdin = fdopen(pipefd_stdin[PIPE_WRITE], "w")) == NULL) {
182 + error("POPEN: fdopen() stdin failed");
183 + ret = -1;
184 + goto cleanup_and_return;
185 + }
186 +
187 + if(posix_spawn_file_actions_adddup2(&fa, pipefd_stdin[PIPE_READ], STDIN_FILENO)) {
188 + error("POPEN: posix_spawn_file_actions_adddup2() on stdin failed.");
189 + ret = -1;
190 + goto cleanup_and_return;
191 + }
192 + }
193 + else {
194 + if (posix_spawn_file_actions_addopen(&fa, STDIN_FILENO, "/dev/null", O_RDONLY, 0)) {
195 + error("POPEN: posix_spawn_file_actions_addopen() on stdin to /dev/null failed.");
196 + // this is not a fatal error
197 + stdin_fd_to_exclude_from_closing = STDIN_FILENO;
198 }
199 }
200
116 - if (flags & POPEN_FLAG_CLOSE_FD) {
117 - // Mark all files to be closed by the exec() stage of posix_spawn()
118 - int i;
119 - for (i = (int) (sysconf(_SC_OPEN_MAX) - 1); i >= 0; i--) {
120 - if (i != STDIN_FILENO && i != STDERR_FILENO)
121 - (void) fcntl(i, F_SETFD, FD_CLOEXEC);
201 + if (fpp_child_stdout) {
202 + if (pipe(pipefd_stdout) == -1) {
203 + error("POPEN: stdout pipe() failed");
204 + ret = -1;
205 + goto cleanup_and_return;
206 + }
207 +
208 + if ((fp_child_stdout = fdopen(pipefd_stdout[PIPE_READ], "r")) == NULL) {
209 + error("POPEN: fdopen() stdout failed");
210 + ret = -1;
211 + goto cleanup_and_return;
212 + }
213 +
214 + if(posix_spawn_file_actions_adddup2(&fa, pipefd_stdout[PIPE_WRITE], STDOUT_FILENO)) {
215 + error("POPEN: posix_spawn_file_actions_adddup2() on stdout failed.");
216 + ret = -1;
217 + goto cleanup_and_return;
218 + }
219 + }
220 + else {
221 + if (posix_spawn_file_actions_addopen(&fa, STDOUT_FILENO, "/dev/null", O_WRONLY, 0)) {
222 + error("POPEN: posix_spawn_file_actions_addopen() on stdout to /dev/null failed.");
223 + // this is not a fatal error
224 + stdout_fd_to_exclude_from_closing = STDOUT_FILENO;
225 }
226 }
227
125 - if (!posix_spawn_file_actions_init(&fa)) {
126 - if (flags & POPEN_FLAG_CREATE_PIPE) {
127 - // move the pipe to stdout in the child
128 - if (posix_spawn_file_actions_adddup2(&fa, pipefd[PIPE_WRITE], STDOUT_FILENO)) {
129 - error("posix_spawn_file_actions_adddup2() failed");
130 - goto error_after_posix_spawn_file_actions_init;
131 - }
132 - } else {
133 - // set stdout to /dev/null
134 - if (posix_spawn_file_actions_addopen(&fa, STDOUT_FILENO, "/dev/null", O_WRONLY, 0)) {
135 - error("posix_spawn_file_actions_addopen() failed");
136 - // this is not a fatal error
137 - }
228 + if(flags & POPEN_FLAG_CLOSE_FD) {
229 + // Mark all files to be closed by the exec() stage of posix_spawn()
230 + for(int i = (int)(sysconf(_SC_OPEN_MAX) - 1); i >= 0; i--) {
231 + if(likely(i != STDERR_FILENO && i != stdin_fd_to_exclude_from_closing && i != stdout_fd_to_exclude_from_closing))
232 + (void)fcntl(i, F_SETFD, FD_CLOEXEC);
233 }
139 - } else {
140 - error("posix_spawn_file_actions_init() failed.");
141 - goto error_after_pipe;
234 }
143 - if (!(error = posix_spawnattr_init(&attr))) {
235 +
236 + attr_rc = posix_spawnattr_init(&attr);
237 + if(attr_rc) {
238 + // failed
239 + error("POPEN: posix_spawnattr_init() failed.");
240 + }
241 + else {
242 + // success
243 // reset all signals in the child
145 - sigset_t mask;
244
245 if (posix_spawnattr_setflags(&attr, POSIX_SPAWN_SETSIGMASK | POSIX_SPAWN_SETSIGDEF))
148 - error("posix_spawnattr_setflags() failed.");
246 + error("POPEN: posix_spawnattr_setflags() failed.");
247 +
248 + sigset_t mask;
249 sigemptyset(&mask);
250 +
251 if (posix_spawnattr_setsigmask(&attr, &mask))
151 - error("posix_spawnattr_setsigmask() failed.");
152 - } else {
153 - error("posix_spawnattr_init() failed.");
252 + error("POPEN: posix_spawnattr_setsigmask() failed.");
253 }
254
255 // Take the lock while we fork to ensure we don't race with SIGCHLD
256 // delivery on a process which exits quickly.
158 - myp_add_lock();
257 + netdata_popen_tracking_lock();
258 if (!posix_spawn(&pid, command, &fa, &attr, (char * const*)spawn_argv, env)) {
259 + // success
260 *pidptr = pid;
161 - myp_add_locked(pid);
162 - debug(D_CHILDS, "Spawned command: \"%s\" on pid %d from parent pid %d.", command_to_be_logged, pid, getpid());
163 - } else {
164 - myp_add_unlock();
165 - error("Failed to spawn command: \"%s\" from parent pid %d.", command_to_be_logged, getpid());
166 - if (flags & POPEN_FLAG_CREATE_PIPE) {
167 - fclose(fp);
168 - }
169 - ret = -1;
261 + netdata_popen_tracking_add_pid_unsafe(pid);
262 + netdata_popen_tracking_unlock();
263 }
171 - if (flags & POPEN_FLAG_CREATE_PIPE) {
172 - close(pipefd[PIPE_WRITE]);
173 - if (0 == ret) // on success set FILE * pointer
174 - if(fpp) *fpp = fp;
264 + else {
265 + // failure
266 + netdata_popen_tracking_unlock();
267 + error("POPEN: failed to spawn command: \"%s\" from parent pid %d.", command_to_be_logged, getpid());
268 + ret = -1;
269 + goto cleanup_and_return;
270 }
271
177 - if (!error) {
272 + // the normal cleanup will run
273 + // but ret == 0 at this point
274 +
275 +cleanup_and_return:
276 + if(!attr_rc) {
277 // posix_spawnattr_init() succeeded
278 if (posix_spawnattr_destroy(&attr))
180 - error("posix_spawnattr_destroy");
279 + error("POPEN: posix_spawnattr_destroy() failed");
280 }
182 - if (posix_spawn_file_actions_destroy(&fa))
183 - error("posix_spawn_file_actions_destroy");
281
185 - return ret;
186 -
187 -error_after_posix_spawn_file_actions_init:
282 if (posix_spawn_file_actions_destroy(&fa))
189 - error("posix_spawn_file_actions_destroy");
283 + error("POPEN: posix_spawn_file_actions_destroy() failed");
284 +
285 + // the child end - close it
286 + if(pipefd_stdin[PIPE_READ] != -1)
287 + close(pipefd_stdin[PIPE_READ]);
288 +
289 + // our end
290 + if(ret == -1 || !fpp_child_stdin) {
291 + if (fp_child_stdin)
292 + fclose(fp_child_stdin);
293 + else if (pipefd_stdin[PIPE_WRITE] != -1)
294 + close(pipefd_stdin[PIPE_WRITE]);
295 + }
296 + else {
297 + *fpp_child_stdin = fp_child_stdin;
298 + }
299
191 -error_after_pipe:
192 - if (flags & POPEN_FLAG_CREATE_PIPE) {
193 - if (fp)
194 - fclose(fp);
195 - else
196 - close(pipefd[PIPE_READ]);
300 + // the child end - close it
301 + if (pipefd_stdout[PIPE_WRITE] != -1)
302 + close(pipefd_stdout[PIPE_WRITE]);
303
198 - close(pipefd[PIPE_WRITE]);
304 + // our end
305 + if (ret == -1 || !fpp_child_stdout) {
306 + if (fp_child_stdout)
307 + fclose(fp_child_stdout);
308 + else if (pipefd_stdout[PIPE_READ] != -1)
309 + close(pipefd_stdout[PIPE_READ]);
310 }
200 - return -1;
311 + else {
312 + *fpp_child_stdout = fp_child_stdout;
313 + }
314 +
315 + return ret;
316 }
317
203 -int custom_popene_variadic_internal_dont_use_directly(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp, const char *command, ...) {
318 +int netdata_popene_variadic_internal_dont_use_directly(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp_child_input, FILE **fpp_child_output, const char *command, ...) {
319 // convert the variable list arguments into what posix_spawn() needs
320 // all arguments are expected strings
321 va_list args;
@@ -232,88 +347,34 @@ int custom_popene_variadic_internal_dont_use_directly(volatile pid_t *pidptr, ch
347 va_end(args);
348 }
349
235 - return custom_popene(pidptr, env, flags, fpp, command, spawn_argv);
350 + return popene_internal(pidptr, env, flags, fpp_child_input, fpp_child_output, command, spawn_argv);
351 }
352
353 // See man environ
354 extern char **environ;
355
241 -// myp_init should be called by apps which act as init
242 -// (pid 1) so that processes created by mypopen and mypopene
243 -// are tracked. This enables the reaper to ignore processes
244 -// which will be handled internally, by calling myp_reap, to
245 -// avoid issues with already reaped processes during wait calls.
246 -//
247 -// Callers should call myp_free() to clean up resources.
248 -void myp_init(void) {
249 - info("process tracking enabled.");
250 - myp_tracking = 1;
251 -
252 - if (netdata_mutex_init(&myp_lock) != 0) {
253 - fatal("myp_init() mutex init failed.");
254 - }
255 -}
256 -
257 -// myp_free cleans up any resources allocated for process
258 -// tracking.
259 -void myp_free(void) {
260 - struct mypopen *mp, *next;
261 -
262 - if (myp_tracking == 0)
263 - return;
264 -
265 - netdata_mutex_lock(&myp_lock);
266 - for (mp = mypopen_root; mp != NULL; mp = next) {
267 - next = mp->next;
268 - freez(mp);
269 - }
270 -
271 - mypopen_root = NULL;
272 - myp_tracking = 0;
273 - netdata_mutex_unlock(&myp_lock);
274 -}
275 -
276 -// myp_reap returns 1 if pid should be reaped, 0 otherwise.
277 -int myp_reap(pid_t pid) {
278 - struct mypopen *mp;
279 -
280 - if (myp_tracking == 0)
281 - return 0;
282 -
283 - netdata_mutex_lock(&myp_lock);
284 - for (mp = mypopen_root; mp != NULL; mp = mp->next) {
285 - if (mp->pid == pid) {
286 - netdata_mutex_unlock(&myp_lock);
287 - return 0;
288 - }
289 - }
290 - netdata_mutex_unlock(&myp_lock);
291 -
292 - return 1;
293 -}
294 -
295 -FILE *mypopen(const char *command, volatile pid_t *pidptr) {
296 - FILE *fp = NULL;
356 +FILE *netdata_popen(const char *command, volatile pid_t *pidptr, FILE **fpp_child_input) {
357 + FILE *fp_child_output = NULL;
358 const char *spawn_argv[] = {
359 "sh",
360 "-c",
361 command,
362 NULL
363 };
303 - (void)custom_popene(pidptr, environ, POPEN_FLAG_CREATE_PIPE|POPEN_FLAG_CLOSE_FD, &fp, "/bin/sh", spawn_argv);
304 - return fp;
364 + (void)popene_internal(pidptr, environ, POPEN_FLAG_CLOSE_FD, fpp_child_input, &fp_child_output, "/bin/sh", spawn_argv);
365 + return fp_child_output;
366 }
367
307 -FILE *mypopene(const char *command, volatile pid_t *pidptr, char **env) {
308 - FILE *fp = NULL;
368 +FILE *netdata_popene(const char *command, volatile pid_t *pidptr, char **env, FILE **fpp_child_input) {
369 + FILE *fp_child_output = NULL;
370 const char *spawn_argv[] = {
371 "sh",
372 "-c",
373 command,
374 NULL
375 };
315 - (void)custom_popene( pidptr, env, POPEN_FLAG_CREATE_PIPE|POPEN_FLAG_CLOSE_FD, &fp, "/bin/sh", spawn_argv);
316 - return fp;
376 + (void)popene_internal(pidptr, env, POPEN_FLAG_CLOSE_FD, fpp_child_input, &fp_child_output, "/bin/sh", spawn_argv);
377 + return fp_child_output;
378 }
379
380 // returns 0 on success, -1 on failure
@@ -324,29 +385,25 @@ int netdata_spawn(const char *command, volatile pid_t *pidptr) {
385 command,
386 NULL
387 };
327 - return custom_popene( pidptr, environ, POPEN_FLAG_NONE, NULL, "/bin/sh", spawn_argv);
388 + return popene_internal(pidptr, environ, POPEN_FLAG_NONE, NULL, NULL, "/bin/sh", spawn_argv);
389 }
390
330 -int custom_pclose(FILE *fp, pid_t pid) {
391 +int netdata_pclose(FILE *fp_child_input, FILE *fp_child_output, pid_t pid) {
392 int ret;
393 siginfo_t info;
394
334 - debug(D_EXIT, "Request to mypclose() on pid %d", pid);
395 + debug(D_EXIT, "Request to netdata_pclose() on pid %d", pid);
396
336 - if (fp) {
337 - // close the pipe fd
338 - // this is required in musl
339 - // without it the childs do not exit
340 - close(fileno(fp));
397 + if (fp_child_input)
398 + fclose(fp_child_input);
399
342 - // close the pipe file pointer
343 - fclose(fp);
344 - }
400 + if (fp_child_output)
401 + fclose(fp_child_output);
402
403 errno = 0;
404
405 ret = waitid(P_PID, (id_t) pid, &info, WEXITED);
349 - myp_del(pid);
406 + netdata_popen_tracking_del_pid(pid);
407
408 if (ret != -1) {
409 switch (info.si_code) {
@@ -392,12 +449,6 @@ int custom_pclose(FILE *fp, pid_t pid) {
449 return 0;
450 }
451
395 -int mypclose(FILE *fp, pid_t pid)
396 -{
397 - return custom_pclose(fp, pid);
398 -}
399 -
400 -int netdata_spawn_waitpid(pid_t pid)
401 -{
402 - return custom_pclose(NULL, pid);
452 +int netdata_spawn_waitpid(pid_t pid) {
453 + return netdata_pclose(NULL, NULL, pid);
454 }
libnetdata/popen/popen.h
+13 -13
@@ -10,29 +10,29 @@
10
11 /* custom_popene_variadic_internal_dont_use_directly flag definitions */
12 #define POPEN_FLAG_NONE 0
13 -#define POPEN_FLAG_CREATE_PIPE 1 // Create a pipe like popen() when set, otherwise set stdout to /dev/null
14 -#define POPEN_FLAG_CLOSE_FD 2 // Close all file descriptors other than STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO
13 +#define POPEN_FLAG_CLOSE_FD (1 << 0) // Close all file descriptors other than STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO
14
15 // the flags to be used by default
17 -#define POPEN_FLAGS_DEFAULT (POPEN_FLAG_CREATE_PIPE|POPEN_FLAG_CLOSE_FD)
16 +#define POPEN_FLAGS_DEFAULT (POPEN_FLAG_CLOSE_FD)
17
18 // mypopen_raw is the interface to use instead of custom_popene_variadic_internal_dont_use_directly()
19 // mypopen_raw will add the terminating NULL at the arguments list
20 // we append the parameter 'command' twice - this is because the underlying call needs the command to execute and the argv[0] to pass to it
22 -#define mypopen_raw_default_flags_and_environment(pidptr, fpp, command, args...) custom_popene_variadic_internal_dont_use_directly(pidptr, environ, POPEN_FLAGS_DEFAULT, fpp, command, command, ##args, NULL)
23 -#define mypopen_raw_default_flags(pidptr, env, fpp, command, args...) custom_popene_variadic_internal_dont_use_directly(pidptr, env, POPEN_FLAGS_DEFAULT, fpp, command, command, ##args, NULL)
24 -#define mypopen_raw(pidptr, env, flags, fpp, command, args...) custom_popene_variadic_internal_dont_use_directly(pidptr, env, flags, fpp, command, command, ##args, NULL)
21 +#define netdata_popen_raw_default_flags_and_environment(pidptr, fpp_child_input, fpp_child_output, command, args...) netdata_popene_variadic_internal_dont_use_directly(pidptr, environ, POPEN_FLAGS_DEFAULT, fpp_child_input, fpp_child_output, command, command, ##args, NULL)
22 +#define netdata_popen_raw_default_flags(pidptr, env, fpp_child_input, fpp_child_output, command, args...) netdata_popene_variadic_internal_dont_use_directly(pidptr, env, POPEN_FLAGS_DEFAULT, fpp_child_input, fpp_child_output, command, command, ##args, NULL)
23 +#define netdata_popen_raw(pidptr, env, flags, fpp_child_input, fpp_child_output, command, args...) netdata_popene_variadic_internal_dont_use_directly(pidptr, env, flags, fpp_child_input, fpp_child_output, command, command, ##args, NULL)
24
26 -extern int custom_popene_variadic_internal_dont_use_directly(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp, const char *command, ...);
25 +extern FILE *netdata_popen(const char *command, volatile pid_t *pidptr, FILE **fp_child_input);
26 +extern FILE *netdata_popene(const char *command, volatile pid_t *pidptr, char **env, FILE **fp_child_input);
27 +extern int netdata_popene_variadic_internal_dont_use_directly(volatile pid_t *pidptr, char **env, uint8_t flags, FILE **fpp_child_input, FILE **fpp_child_output, const char *command, ...);
28 +extern int netdata_pclose(FILE *fp_child_input, FILE *fp_child_output, pid_t pid);
29
28 -extern FILE *mypopen(const char *command, volatile pid_t *pidptr);
29 -extern FILE *mypopene(const char *command, volatile pid_t *pidptr, char **env);
30 -extern int mypclose(FILE *fp, pid_t pid);
30 extern int netdata_spawn(const char *command, volatile pid_t *pidptr);
31 extern int netdata_spawn_waitpid(pid_t pid);
33 -extern void myp_init(void);
34 -extern void myp_free(void);
35 -extern int myp_reap(pid_t pid);
32 +
33 +extern void netdata_popen_tracking_init(void);
34 +extern void netdata_popen_tracking_cleanup(void);
35 +extern int netdata_popen_tracking_pid_shoud_be_reaped(pid_t pid);
36
37 extern void signals_unblock(void);
38 extern void signals_reset(void);
libnetdata/required_dummies.h
+1
@@ -34,6 +34,7 @@ int health_variable_lookup(STRING *variable, struct rrdcalc *rc, NETDATA_DOUBLE
34 #endif
35
36 void rrdset_thread_rda_free(void){};
37 +void sender_thread_buffer_free(void){};
38
39 // required by get_system_cpus()
40 char *netdata_configured_host_prefix = "";
libnetdata/socket/security.c
+23 -22
@@ -2,14 +2,14 @@
2
3 #ifdef ENABLE_HTTPS
4
5 -SSL_CTX *netdata_exporting_ctx=NULL;
6 -SSL_CTX *netdata_client_ctx=NULL;
7 -SSL_CTX *netdata_srv_ctx=NULL;
8 -const char *security_key=NULL;
9 -const char *security_cert=NULL;
5 +SSL_CTX *netdata_ssl_exporting_ctx =NULL;
6 +SSL_CTX *netdata_ssl_client_ctx =NULL;
7 +SSL_CTX *netdata_ssl_srv_ctx =NULL;
8 +const char *ssl_security_key =NULL;
9 +const char *ssl_security_cert =NULL;
10 const char *tls_version=NULL;
11 const char *tls_ciphers=NULL;
12 -int netdata_validate_server = NETDATA_SSL_VALID_CERTIFICATE;
12 +int netdata_ssl_validate_server = NETDATA_SSL_VALID_CERTIFICATE;
13
14 /**
15 * Info Callback
@@ -161,7 +161,7 @@ static SSL_CTX * security_initialize_openssl_server() {
161 return NULL;
162 }
163
164 - SSL_CTX_use_certificate_file(ctx, security_cert, SSL_FILETYPE_PEM);
164 + SSL_CTX_use_certificate_file(ctx, ssl_security_cert, SSL_FILETYPE_PEM);
165 #else
166 ctx = SSL_CTX_new(TLS_server_method());
167 if (!ctx) {
@@ -169,11 +169,11 @@ static SSL_CTX * security_initialize_openssl_server() {
169 return NULL;
170 }
171
172 - SSL_CTX_use_certificate_chain_file(ctx, security_cert);
172 + SSL_CTX_use_certificate_chain_file(ctx, ssl_security_cert);
173 #endif
174 security_openssl_common_options(ctx, 0);
175
176 - SSL_CTX_use_PrivateKey_file(ctx,security_key,SSL_FILETYPE_PEM);
176 + SSL_CTX_use_PrivateKey_file(ctx, ssl_security_key,SSL_FILETYPE_PEM);
177
178 if (!SSL_CTX_check_private_key(ctx)) {
179 ERR_error_string_n(ERR_get_error(),lerror,sizeof(lerror));
@@ -207,24 +207,25 @@ void security_start_ssl(int selector) {
207 switch (selector) {
208 case NETDATA_SSL_CONTEXT_SERVER: {
209 struct stat statbuf;
210 - if (stat(security_key, &statbuf) || stat(security_cert, &statbuf)) {
210 + if (stat(ssl_security_key, &statbuf) || stat(ssl_security_cert, &statbuf)) {
211 info("To use encryption it is necessary to set \"ssl certificate\" and \"ssl key\" in [web] !\n");
212 return;
213 }
214
215 - netdata_srv_ctx = security_initialize_openssl_server();
216 - SSL_CTX_set_mode(netdata_srv_ctx, SSL_MODE_ENABLE_PARTIAL_WRITE);
215 + netdata_ssl_srv_ctx = security_initialize_openssl_server();
216 + SSL_CTX_set_mode(netdata_ssl_srv_ctx, SSL_MODE_ENABLE_PARTIAL_WRITE);
217 break;
218 }
219 case NETDATA_SSL_CONTEXT_STREAMING: {
220 - netdata_client_ctx = security_initialize_openssl_client();
220 + netdata_ssl_client_ctx = security_initialize_openssl_client();
221 //This is necessary for the stream, because it is working sometimes with nonblock socket.
222 //It returns the bitmask after to change, there is not any description of errors in the documentation
223 - SSL_CTX_set_mode(netdata_client_ctx, SSL_MODE_ENABLE_PARTIAL_WRITE |SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER |SSL_MODE_AUTO_RETRY);
223 + SSL_CTX_set_mode(
224 + netdata_ssl_client_ctx, SSL_MODE_ENABLE_PARTIAL_WRITE |SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER |SSL_MODE_AUTO_RETRY);
225 break;
226 }
227 case NETDATA_SSL_CONTEXT_EXPORTING: {
227 - netdata_exporting_ctx = security_initialize_openssl_client();
228 + netdata_ssl_exporting_ctx = security_initialize_openssl_client();
229 break;
230 }
231 }
@@ -237,16 +238,16 @@ void security_start_ssl(int selector) {
238 */
239 void security_clean_openssl()
240 {
240 - if (netdata_srv_ctx) {
241 - SSL_CTX_free(netdata_srv_ctx);
241 + if (netdata_ssl_srv_ctx) {
242 + SSL_CTX_free(netdata_ssl_srv_ctx);
243 }
244
244 - if (netdata_client_ctx) {
245 - SSL_CTX_free(netdata_client_ctx);
245 + if (netdata_ssl_client_ctx) {
246 + SSL_CTX_free(netdata_ssl_client_ctx);
247 }
248
248 - if (netdata_exporting_ctx) {
249 - SSL_CTX_free(netdata_exporting_ctx);
249 + if (netdata_ssl_exporting_ctx) {
250 + SSL_CTX_free(netdata_ssl_exporting_ctx);
251 }
252
253 #if OPENSSL_VERSION_NUMBER < OPENSSL_VERSION_110
@@ -355,7 +356,7 @@ int security_test_certificate(SSL *ssl) {
356 *
357 * @return It returns 0 on success and -1 otherwise.
358 */
358 -int security_location_for_context(SSL_CTX *ctx, char *file, char *path) {
359 +int ssl_security_location_for_context(SSL_CTX *ctx, char *file, char *path) {
360 struct stat statbuf;
361 if (stat(file, &statbuf)) {
362 info("Netdata does not have the parent's SSL certificate, so it will use the default OpenSSL configuration to validate certificates!");
libnetdata/socket/security.h
+7 -7
@@ -42,15 +42,15 @@ struct netdata_ssl{
42 uint32_t flags; //The flags for SSL connection
43 };
44
45 -extern SSL_CTX *netdata_exporting_ctx;
46 -extern SSL_CTX *netdata_client_ctx;
47 -extern SSL_CTX *netdata_srv_ctx;
48 -extern const char *security_key;
49 -extern const char *security_cert;
45 +extern SSL_CTX *netdata_ssl_exporting_ctx;
46 +extern SSL_CTX *netdata_ssl_client_ctx;
47 +extern SSL_CTX *netdata_ssl_srv_ctx;
48 +extern const char *ssl_security_key;
49 +extern const char *ssl_security_cert;
50 extern const char *tls_version;
51 extern const char *tls_ciphers;
52 -extern int netdata_validate_server;
53 -extern int security_location_for_context(SSL_CTX *ctx,char *file,char *path);
52 +extern int netdata_ssl_validate_server;
53 +extern int ssl_security_location_for_context(SSL_CTX *ctx,char *file,char *path);
54
55 void security_openssl_library();
56 void security_clean_openssl();
libnetdata/socket/socket.c
+77 -24
@@ -779,6 +779,10 @@ int connect_to_this(const char *definition, int default_port, struct timeval *ti
779 char *path = host + 5;
780 return connect_to_unix(path, timeout);
781 }
782 + else if(*host == '/') {
783 + char *path = host;
784 + return connect_to_unix(path, timeout);
785 + }
786
787 char *e = host;
788 if(*e == '[') {
@@ -826,41 +830,92 @@ int connect_to_this(const char *definition, int default_port, struct timeval *ti
830 return connect_to_this_ip46(protocol, socktype, host, scope_id, service, timeout);
831 }
832
829 -int connect_to_one_of(const char *destination, int default_port, struct timeval *timeout, size_t *reconnects_counter, char *connected_to, size_t connected_to_size) {
830 - int sock = -1;
831 -
833 +void foreach_entry_in_connection_string(const char *destination, bool (*callback)(char *entry, void *data), void *data) {
834 const char *s = destination;
835 while(*s) {
836 const char *e = s;
837
836 - // skip path, moving both s(tart) and e(nd)
837 - if(*e == '/')
838 - while(!isspace(*e) && *e != ',') s = ++e;
839 -
838 // skip separators, moving both s(tart) and e(nd)
839 while(isspace(*e) || *e == ',') s = ++e;
840
841 // move e(nd) to the first separator
844 - while(*e && !isspace(*e) && *e != ',' && *e != '/') e++;
842 + while(*e && !isspace(*e) && *e != ',') e++;
843
844 // is there anything?
845 if(!*s || s == e) break;
846
847 char buf[e - s + 1];
848 strncpyz(buf, s, e - s);
851 - if(reconnects_counter) *reconnects_counter += 1;
852 - sock = connect_to_this(buf, default_port, timeout);
853 - if(sock != -1) {
854 - if(connected_to && connected_to_size) {
855 - strncpy(connected_to, buf, connected_to_size);
856 - connected_to[connected_to_size - 1] = '\0';
857 - }
858 - break;
859 - }
849 +
850 + if(callback(buf, data)) break;
851 +
852 s = e;
853 }
854 +}
855
863 - return sock;
856 +struct connect_to_one_of_data {
857 + int default_port;
858 + struct timeval *timeout;
859 + size_t *reconnects_counter;
860 + char *connected_to;
861 + size_t connected_to_size;
862 + int sock;
863 +};
864 +
865 +static bool connect_to_one_of_callback(char *entry, void *data) {
866 + struct connect_to_one_of_data *t = data;
867 +
868 + if(t->reconnects_counter)
869 + t->reconnects_counter++;
870 +
871 + t->sock = connect_to_this(entry, t->default_port, t->timeout);
872 + if(t->sock != -1) {
873 + if(t->connected_to && t->connected_to_size) {
874 + strncpyz(t->connected_to, entry, t->connected_to_size);
875 + t->connected_to[t->connected_to_size - 1] = '\0';
876 + }
877 +
878 + return true;
879 + }
880 +
881 + return false;
882 +}
883 +
884 +int connect_to_one_of(const char *destination, int default_port, struct timeval *timeout, size_t *reconnects_counter, char *connected_to, size_t connected_to_size) {
885 + struct connect_to_one_of_data t = {
886 + .default_port = default_port,
887 + .timeout = timeout,
888 + .reconnects_counter = reconnects_counter,
889 + .connected_to = connected_to,
890 + .connected_to_size = connected_to_size,
891 + .sock = -1,
892 + };
893 +
894 + foreach_entry_in_connection_string(destination, connect_to_one_of_callback, &t);
895 +
896 + return t.sock;
897 +}
898 +
899 +static bool connect_to_one_of_urls_callback(char *entry, void *data) {
900 + char *s = strchr(entry, '/');
901 + if(s) *s = '\0';
902 +
903 + return connect_to_one_of_callback(entry, data);
904 +}
905 +
906 +int connect_to_one_of_urls(const char *destination, int default_port, struct timeval *timeout, size_t *reconnects_counter, char *connected_to, size_t connected_to_size) {
907 + struct connect_to_one_of_data t = {
908 + .default_port = default_port,
909 + .timeout = timeout,
910 + .reconnects_counter = reconnects_counter,
911 + .connected_to = connected_to,
912 + .connected_to_size = connected_to_size,
913 + .sock = -1,
914 + };
915 +
916 + foreach_entry_in_connection_string(destination, connect_to_one_of_urls_callback, &t);
917 +
918 + return t.sock;
919 }
920
921
@@ -1087,12 +1142,11 @@ int accept_socket(int fd, int flags, char *client_ip, size_t ipsize, char *clien
1142 if (getnameinfo((struct sockaddr *)&sadr, addrlen, client_ip, (socklen_t)ipsize,
1143 client_port, (socklen_t)portsize, NI_NUMERICHOST | NI_NUMERICSERV) != 0) {
1144 error("LISTENER: cannot getnameinfo() on received client connection.");
1090 - strncpyz(client_ip, "UNKNOWN", ipsize - 1);
1091 - strncpyz(client_port, "UNKNOWN", portsize - 1);
1145 + strncpyz(client_ip, "UNKNOWN", ipsize);
1146 + strncpyz(client_port, "UNKNOWN", portsize);
1147 }
1148 if (!strcmp(client_ip, "127.0.0.1") || !strcmp(client_ip, "::1")) {
1094 - strncpy(client_ip, "localhost", ipsize);
1095 - client_ip[ipsize - 1] = '\0';
1149 + strncpyz(client_ip, "localhost", ipsize);
1150 }
1151
1152 #ifdef __FreeBSD__
@@ -1107,8 +1161,7 @@ int accept_socket(int fd, int flags, char *client_ip, size_t ipsize, char *clien
1161 case AF_UNIX:
1162 debug(D_LISTENER, "New UNIX domain web client from %s on socket %d.", client_ip, fd);
1163 // set the port - certain versions of libc return garbage on unix sockets
1110 - strncpy(client_port, "UNIX", portsize);
1111 - client_port[portsize - 1] = '\0';
1164 + strncpyz(client_port, "UNIX", portsize);
1165 break;
1166
1167 case AF_INET:
libnetdata/socket/socket.h
+17 -11
@@ -10,19 +10,22 @@
10 #endif
11
12 typedef enum web_client_acl {
13 - WEB_CLIENT_ACL_NONE = 0,
14 - WEB_CLIENT_ACL_NOCHECK = 0,
15 - WEB_CLIENT_ACL_DASHBOARD = 1 << 0,
16 - WEB_CLIENT_ACL_REGISTRY = 1 << 1,
17 - WEB_CLIENT_ACL_BADGE = 1 << 2,
18 - WEB_CLIENT_ACL_MGMT = 1 << 3,
19 - WEB_CLIENT_ACL_STREAMING = 1 << 4,
20 - WEB_CLIENT_ACL_NETDATACONF = 1 << 5,
13 + WEB_CLIENT_ACL_NONE = 0,
14 + WEB_CLIENT_ACL_NOCHECK = 0,
15 + WEB_CLIENT_ACL_DASHBOARD = 1 << 0,
16 + WEB_CLIENT_ACL_REGISTRY = 1 << 1,
17 + WEB_CLIENT_ACL_BADGE = 1 << 2,
18 + WEB_CLIENT_ACL_MGMT = 1 << 3,
19 + WEB_CLIENT_ACL_STREAMING = 1 << 4,
20 + WEB_CLIENT_ACL_NETDATACONF = 1 << 5,
21 WEB_CLIENT_ACL_SSL_OPTIONAL = 1 << 6,
22 - WEB_CLIENT_ACL_SSL_FORCE = 1 << 7,
23 - WEB_CLIENT_ACL_SSL_DEFAULT = 1 << 8
22 + WEB_CLIENT_ACL_SSL_FORCE = 1 << 7,
23 + WEB_CLIENT_ACL_SSL_DEFAULT = 1 << 8,
24 + WEB_CLIENT_ACL_ACLK = 1 << 9,
25 } WEB_CLIENT_ACL;
26
27 +#define WEB_CLIENT_ACL_ALL 0xFFFF
28 +
29 #define web_client_can_access_dashboard(w) ((w)->acl & WEB_CLIENT_ACL_DASHBOARD)
30 #define web_client_can_access_registry(w) ((w)->acl & WEB_CLIENT_ACL_REGISTRY)
31 #define web_client_can_access_badges(w) ((w)->acl & WEB_CLIENT_ACL_BADGE)
@@ -54,9 +57,12 @@ extern char *strdup_client_description(int family, const char *protocol, const c
57 extern int listen_sockets_setup(LISTEN_SOCKETS *sockets);
58 extern void listen_sockets_close(LISTEN_SOCKETS *sockets);
59
60 +extern void foreach_entry_in_connection_string(const char *destination, bool (*callback)(char *entry, void *data), void *data);
61 +extern int connect_to_this_ip46(int protocol, int socktype, const char *host, uint32_t scope_id, const char *service, struct timeval *timeout);
62 extern int connect_to_this(const char *definition, int default_port, struct timeval *timeout);
63 extern int connect_to_one_of(const char *destination, int default_port, struct timeval *timeout, size_t *reconnects_counter, char *connected_to, size_t connected_to_size);
59 -int connect_to_this_ip46(int protocol, int socktype, const char *host, uint32_t scope_id, const char *service, struct timeval *timeout);
64 +extern int connect_to_one_of_urls(const char *destination, int default_port, struct timeval *timeout, size_t *reconnects_counter, char *connected_to, size_t connected_to_size);
65 +
66
67 #ifdef ENABLE_HTTPS
68 extern ssize_t recv_timeout(struct netdata_ssl *ssl,int sockfd, void *buf, size_t len, int flags, int timeout);
libnetdata/threads/threads.c
+2
@@ -107,6 +107,7 @@ void netdata_threads_init_after_fork(size_t stacksize) {
107 // netdata_thread_create
108
109 extern void rrdset_thread_rda_free(void);
110 +extern void sender_thread_buffer_free(void);
111
112 static void thread_cleanup(void *ptr) {
113 if(netdata_thread != ptr) {
@@ -117,6 +118,7 @@ static void thread_cleanup(void *ptr) {
118 if(!(netdata_thread->options & NETDATA_THREAD_OPTION_DONT_LOG_CLEANUP))
119 info("thread with task id %d finished", gettid());
120
121 + sender_thread_buffer_free();
122 rrdset_thread_rda_free();
123 thread_cache_destroy();
124
parser/parser.c
+81 -25
@@ -1,10 +1,11 @@
1 // SPDX-License-Identifier: GPL-3.0-or-later
2
3 #include "parser.h"
4 +#include "collectors/plugins.d/pluginsd_parser.h"
5
5 -static inline int find_keyword(char *str, char *keyword, int max_size, int (*custom_isspace)(char))
6 +inline int find_first_keyword(const char *str, char *keyword, int max_size, int (*custom_isspace)(char))
7 {
7 - char *s = str, *keyword_start;
8 + const char *s = str, *keyword_start;
9
10 while (unlikely(custom_isspace(*s))) s++;
11 keyword_start = s;
@@ -28,7 +29,7 @@ static inline int find_keyword(char *str, char *keyword, int max_size, int (*cus
29 *
30 */
31
31 -PARSER *parser_init(RRDHOST *host, void *user, void *input, PARSER_INPUT_TYPE flags)
32 +PARSER *parser_init(RRDHOST *host, void *user, void *input, void *output, PARSER_INPUT_TYPE flags)
33 {
34 PARSER *parser;
35
@@ -36,9 +37,11 @@ PARSER *parser_init(RRDHOST *host, void *user, void *input, PARSER_INPUT_TYPE fl
37 parser->plugins_action = callocz(1, sizeof(PLUGINSD_ACTION));
38 parser->user = user;
39 parser->input = input;
40 + parser->output = output;
41 parser->flags = flags;
42 parser->host = host;
43 parser->worker_job_next_id = WORKER_PARSER_FIRST_JOB;
44 + inflight_functions_init(parser);
45
46 #ifdef ENABLE_HTTPS
47 parser->bytesleft = 0;
@@ -46,18 +49,34 @@ PARSER *parser_init(RRDHOST *host, void *user, void *input, PARSER_INPUT_TYPE fl
49 #endif
50
51 if (unlikely(!(flags & PARSER_NO_PARSE_INIT))) {
49 - parser_add_keyword(parser, PLUGINSD_KEYWORD_FLUSH, pluginsd_flush);
50 - parser_add_keyword(parser, PLUGINSD_KEYWORD_CHART, pluginsd_chart);
51 - parser_add_keyword(parser, PLUGINSD_KEYWORD_DIMENSION, pluginsd_dimension);
52 - parser_add_keyword(parser, PLUGINSD_KEYWORD_DISABLE, pluginsd_disable);
53 - parser_add_keyword(parser, PLUGINSD_KEYWORD_VARIABLE, pluginsd_variable);
54 - parser_add_keyword(parser, PLUGINSD_KEYWORD_LABEL, pluginsd_label);
55 - parser_add_keyword(parser, PLUGINSD_KEYWORD_OVERWRITE, pluginsd_overwrite);
56 - parser_add_keyword(parser, PLUGINSD_KEYWORD_END, pluginsd_end);
57 - parser_add_keyword(parser, "CLABEL_COMMIT", pluginsd_clabel_commit);
58 - parser_add_keyword(parser, "CLABEL", pluginsd_clabel);
59 - parser_add_keyword(parser, PLUGINSD_KEYWORD_BEGIN, pluginsd_begin);
60 - parser_add_keyword(parser, "SET", pluginsd_set);
52 + parser_add_keyword(parser, PLUGINSD_KEYWORD_FLUSH, pluginsd_flush);
53 + parser_add_keyword(parser, PLUGINSD_KEYWORD_CHART, pluginsd_chart);
54 + parser_add_keyword(parser, PLUGINSD_KEYWORD_DIMENSION, pluginsd_dimension);
55 + parser_add_keyword(parser, PLUGINSD_KEYWORD_DISABLE, pluginsd_disable);
56 + parser_add_keyword(parser, PLUGINSD_KEYWORD_VARIABLE, pluginsd_variable);
57 + parser_add_keyword(parser, PLUGINSD_KEYWORD_LABEL, pluginsd_label);
58 + parser_add_keyword(parser, PLUGINSD_KEYWORD_OVERWRITE, pluginsd_overwrite);
59 + parser_add_keyword(parser, PLUGINSD_KEYWORD_END, pluginsd_end);
60 + parser_add_keyword(parser, PLUGINSD_KEYWORD_CLABEL_COMMIT, pluginsd_clabel_commit);
61 + parser_add_keyword(parser, PLUGINSD_KEYWORD_CLABEL, pluginsd_clabel);
62 + parser_add_keyword(parser, PLUGINSD_KEYWORD_BEGIN, pluginsd_begin);
63 + parser_add_keyword(parser, PLUGINSD_KEYWORD_SET, pluginsd_set);
64 + parser_add_keyword(parser, PLUGINSD_KEYWORD_FUNCTION, pluginsd_function);
65 + parser_add_keyword(parser, PLUGINSD_KEYWORD_FUNCTION_RESULT_BEGIN, pluginsd_function_result_begin);
66 + //parser_add_keyword(parser, PLUGINSD_KEYWORD_GAPS_REQUEST, pluginsd_gaps_request);
67 + }
68 +
69 + if(unlikely(!(flags & PARSER_NO_ACTION_INIT))) {
70 + parser->plugins_action->begin_action = &pluginsd_begin_action;
71 + parser->plugins_action->flush_action = &pluginsd_flush_action;
72 + parser->plugins_action->end_action = &pluginsd_end_action;
73 + parser->plugins_action->disable_action = &pluginsd_disable_action;
74 + parser->plugins_action->variable_action = &pluginsd_variable_action;
75 + parser->plugins_action->dimension_action = &pluginsd_dimension_action;
76 + parser->plugins_action->label_action = &pluginsd_label_action;
77 + parser->plugins_action->overwrite_action = &pluginsd_overwrite_action;
78 + parser->plugins_action->chart_action = &pluginsd_chart_action;
79 + parser->plugins_action->set_action = &pluginsd_set_action;
80 }
81
82 return parser;
@@ -155,6 +174,8 @@ void parser_destroy(PARSER *parser)
174 if (unlikely(!parser))
175 return;
176
177 + dictionary_destroy(parser->inflight.functions);
178 +
179 PARSER_KEYWORD *tmp_keyword, *tmp_keyword_next;
180 PARSER_DATA *tmp_parser_data, *tmp_parser_data_next;
181
@@ -237,32 +258,65 @@ int parser_next(PARSER *parser)
258
259 inline int parser_action(PARSER *parser, char *input)
260 {
240 - PARSER_RC rc = PARSER_RC_OK;
261 + PARSER_RC rc = PARSER_RC_OK;
262 char *words[PLUGINSD_MAX_WORDS] = { NULL };
263 char command[PLUGINSD_LINE_MAX + 1];
264 keyword_function action_function;
265 keyword_function *action_function_list = NULL;
266
246 - if (unlikely(!parser))
267 + if (unlikely(!parser)) {
268 + internal_error(true, "parser is NULL");
269 return 1;
270 + }
271 +
272 parser->recover_location[0] = 0x0;
273
274 // if not direct input check if we have reprocessed this
275 if (unlikely(!input && parser->flags & PARSER_INPUT_PROCESSED))
276 return 0;
277
254 - PARSER_KEYWORD *tmp_keyword = parser->keyword;
278 + PARSER_KEYWORD *tmp_keyword = parser->keyword;
279 if (unlikely(!tmp_keyword)) {
280 + internal_error(true, "called without a keyword");
281 return 1;
282 }
283
284 if (unlikely(!input))
285 input = parser->buffer;
286
262 - if (unlikely(!find_keyword(input, command, PLUGINSD_LINE_MAX, pluginsd_space)))
287 + if(unlikely(parser->flags & PARSER_DEFER_UNTIL_KEYWORD)) {
288 + bool has_keyword = find_first_keyword(input, command, PLUGINSD_LINE_MAX, pluginsd_space);
289 +
290 + if(!has_keyword || strcmp(command, parser->defer.end_keyword) != 0) {
291 + if(parser->defer.response) {
292 + buffer_strcat(parser->defer.response, input);
293 + if(buffer_strlen(parser->defer.response) > 10 * 1024 * 1024) {
294 + // more than 10MB of data
295 + // a bad plugin that did not send the end_keyword
296 + internal_error(true, "Deferred response is too big (%zu bytes). Stopping this plugin.", buffer_strlen(parser->defer.response));
297 + return 1;
298 + }
299 + }
300 + return 0;
301 + }
302 + else {
303 + // call the action
304 + parser->defer.action(parser, parser->defer.action_data);
305 +
306 + // empty everything
307 + parser->defer.action = NULL;
308 + parser->defer.action_data = NULL;
309 + parser->defer.end_keyword = NULL;
310 + parser->defer.response = NULL;
311 + parser->flags &= ~PARSER_DEFER_UNTIL_KEYWORD;
312 + }
313 + return 0;
314 + }
315 +
316 + if (unlikely(!find_first_keyword(input, command, PLUGINSD_LINE_MAX, pluginsd_space)))
317 return 0;
318
265 - if ((parser->flags & PARSER_INPUT_ORIGINAL) == PARSER_INPUT_ORIGINAL)
319 + if ((parser->flags & PARSER_INPUT_KEEP_ORIGINAL) == PARSER_INPUT_KEEP_ORIGINAL)
320 pluginsd_split_words(input, words, PLUGINSD_MAX_WORDS, parser->recover_input, parser->recover_location, PARSER_MAX_RECOVER_KEYWORDS);
321 else
322 pluginsd_split_words(input, words, PLUGINSD_MAX_WORDS, NULL, NULL, 0);
@@ -285,16 +339,17 @@ inline int parser_action(PARSER *parser, char *input)
339 rc = parser->unknown_function(words, parser->user, NULL);
340 else
341 rc = PARSER_RC_ERROR;
288 -#ifdef NETDATA_INTERNAL_CHECKS
289 - error("Unknown keyword [%s]", input);
290 -#endif
342 +
343 + internal_error(rc != PARSER_RC_OK, "Unknown keyword [%s]", input);
344 }
345 else {
346 worker_is_busy(worker_job_id);
347 while ((action_function = *action_function_list) != NULL) {
348 rc = action_function(words, parser->user, parser->plugins_action);
296 - if (unlikely(rc == PARSER_RC_ERROR || rc == PARSER_RC_STOP))
297 - break;
349 + if (unlikely(rc == PARSER_RC_ERROR || rc == PARSER_RC_STOP)) {
350 + internal_error(true, "action_function() failed with rc = %u", rc);
351 + break;
352 + }
353 action_function_list++;
354 }
355 worker_is_idle();
@@ -303,6 +358,7 @@ inline int parser_action(PARSER *parser, char *input)
358 if (likely(input == parser->buffer))
359 parser->flags |= PARSER_INPUT_PROCESSED;
360
361 + internal_error(rc == PARSER_RC_ERROR, "parser_action() failed.");
362 return (rc == PARSER_RC_ERROR);
363 }
364
parser/parser.h
+24 -9
@@ -32,8 +32,6 @@ typedef struct pluginsd_action {
32 PARSER_RC (*variable_action)(void *user, RRDHOST *host, RRDSET *st, char *name, int global, NETDATA_DOUBLE value);
33 PARSER_RC (*label_action)(void *user, char *key, char *value, RRDLABEL_SRC source);
34 PARSER_RC (*overwrite_action)(void *user, RRDHOST *host, DICTIONARY *new_labels);
35 - PARSER_RC (*clabel_action)(void *user, char *key, char *value, RRDLABEL_SRC source);
36 - PARSER_RC (*clabel_commit_action)(void *user, RRDHOST *host, DICTIONARY *new_labels);
35
36 PARSER_RC (*guid_action)(void *user, uuid_t *uuid);
37 PARSER_RC (*context_action)(void *user, uuid_t *uuid);
@@ -43,11 +41,12 @@ typedef struct pluginsd_action {
41 } PLUGINSD_ACTION;
42
43 typedef enum parser_input_type {
46 - PARSER_INPUT_SPLIT = 1 << 1,
47 - PARSER_INPUT_ORIGINAL = 1 << 2,
48 - PARSER_INPUT_PROCESSED = 1 << 3,
49 - PARSER_NO_PARSE_INIT = 1 << 4,
50 - PARSER_NO_ACTION_INIT = 1 << 5,
44 + PARSER_INPUT_SPLIT = (1 << 1),
45 + PARSER_INPUT_KEEP_ORIGINAL = (1 << 2),
46 + PARSER_INPUT_PROCESSED = (1 << 3),
47 + PARSER_NO_PARSE_INIT = (1 << 4),
48 + PARSER_NO_ACTION_INIT = (1 << 5),
49 + PARSER_DEFER_UNTIL_KEYWORD = (1 << 6),
50 } PARSER_INPUT_TYPE;
51
52 #define PARSER_INPUT_FULL (PARSER_INPUT_SPLIT|PARSER_INPUT_ORIGINAL)
@@ -73,6 +72,7 @@ typedef struct parser {
72 uint8_t version; // Parser version
73 RRDHOST *host;
74 void *input; // Input source e.g. stream
75 + void *output; // Stream to send commands to plugin
76 PARSER_DATA *data; // extra input
77 PARSER_KEYWORD *keyword; // List of parse keywords and functions
78 PLUGINSD_ACTION *plugins_action;
@@ -90,9 +90,24 @@ typedef struct parser {
90 char tmpbuffer[PLUGINSD_LINE_MAX];
91 char *readfrom;
92 #endif
93 +
94 + struct {
95 + const char *end_keyword;
96 + BUFFER *response;
97 + void (*action)(struct parser *parser, void *action_data);
98 + void *action_data;
99 + } defer;
100 +
101 + struct {
102 + DICTIONARY *functions;
103 + usec_t smaller_timeout;
104 + } inflight;
105 +
106 } PARSER;
107
95 -PARSER *parser_init(RRDHOST *host, void *user, void *input, PARSER_INPUT_TYPE flags);
108 +extern int find_first_keyword(const char *str, char *keyword, int max_size, int (*custom_isspace)(char));
109 +
110 +PARSER *parser_init(RRDHOST *host, void *user, void *input, void *output, PARSER_INPUT_TYPE flags);
111 int parser_add_keyword(PARSER *working_parser, char *keyword, keyword_function func);
112 int parser_next(PARSER *working_parser);
113 int parser_action(PARSER *working_parser, char *input);
@@ -100,7 +115,7 @@ int parser_push(PARSER *working_parser, char *line);
115 void parser_destroy(PARSER *working_parser);
116 int parser_recover_input(PARSER *working_parser);
117
103 -extern size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp, int trust_durations);
118 +extern size_t pluginsd_process(RRDHOST *host, struct plugind *cd, FILE *fp_plugin_input, FILE *fp_plugin_output, int trust_durations);
119
120 extern PARSER_RC pluginsd_set(char **words, void *user, PLUGINSD_ACTION *plugins_action);
121 extern PARSER_RC pluginsd_begin(char **words, void *user, PLUGINSD_ACTION *plugins_action);
streaming/compression.c
+6 -6
@@ -66,8 +66,8 @@ static size_t lz4_compressor_compress(struct compressor_state *state, const char
66 if(unlikely(!state || !size || !out))
67 return 0;
68
69 - if(unlikely(size > LZ4_MAX_MSG_SIZE)) {
70 - error("%s: Compression Failed - Message size %lu above compression buffer limit: %d", STREAM_COMPRESSION_MSG, (long unsigned int) size, LZ4_MAX_MSG_SIZE);
69 + if(unlikely(size > COMPRESSION_MAX_MSG_SIZE)) {
70 + error("%s: Compression Failed - Message size %lu above compression buffer limit: %d", STREAM_COMPRESSION_MSG, (long unsigned int)size, COMPRESSION_MAX_MSG_SIZE);
71 return 0;
72 }
73
@@ -103,7 +103,7 @@ static size_t lz4_compressor_compress(struct compressor_state *state, const char
103
104 // update the next writing position of the ring buffer
105 state->data->input_ring_buffer_pos += size;
106 - if(unlikely(state->data->input_ring_buffer_pos >= state->data->input_ring_buffer_size - LZ4_MAX_MSG_SIZE))
106 + if(unlikely(state->data->input_ring_buffer_pos >= state->data->input_ring_buffer_size - COMPRESSION_MAX_MSG_SIZE))
107 state->data->input_ring_buffer_pos = 0;
108
109 // update the signature header
@@ -128,7 +128,7 @@ struct compressor_state *create_compressor()
128
129 state->data = callocz(1, sizeof(struct compressor_data));
130 state->data->stream = LZ4_createStream();
131 - state->data->input_ring_buffer_size = LZ4_DECODER_RING_BUFFER_SIZE(LZ4_MAX_MSG_SIZE * 2);
131 + state->data->input_ring_buffer_size = LZ4_DECODER_RING_BUFFER_SIZE(COMPRESSION_MAX_MSG_SIZE * 2);
132 state->data->input_ring_buffer = callocz(1, state->data->input_ring_buffer_size);
133 state->compression_result_buffer_size = 0;
134 state->reset(state);
@@ -280,7 +280,7 @@ static size_t lz4_decompressor_decompress(struct decompressor_state *state)
280
281 state->out_buffer = state->data->stream_buffer + state->data->stream_buffer_pos;
282 state->data->stream_buffer_pos += decompressed_size;
283 - if (state->data->stream_buffer_pos >= state->data->stream_buffer_size - LZ4_MAX_MSG_SIZE)
283 + if (state->data->stream_buffer_pos >= state->data->stream_buffer_size - COMPRESSION_MAX_MSG_SIZE)
284 state->data->stream_buffer_pos = 0;
285 state->out_buffer_len = decompressed_size;
286 state->out_buffer_pos = 0;
@@ -358,7 +358,7 @@ struct decompressor_state *create_decompressor()
358 state->data = callocz(1, sizeof(struct decompressor_data));
359 fatal_assert(state->data);
360 state->data->stream = LZ4_createStreamDecode();
361 - state->data->stream_buffer_size = LZ4_decoderRingBufferSize(LZ4_MAX_MSG_SIZE);
361 + state->data->stream_buffer_size = LZ4_decoderRingBufferSize(COMPRESSION_MAX_MSG_SIZE);
362 state->data->stream_buffer = mallocz(state->data->stream_buffer_size);
363 fatal_assert(state->data->stream_buffer);
364 state->reset(state);
streaming/receiver.c
+125 -81
@@ -72,9 +72,8 @@ PARSER_RC streaming_timestamp(char **words, void *user, PLUGINSD_ACTION *plugins
72 time_t remote_time = 0;
73 RRDHOST *host = ((PARSER_USER_OBJECT *)user)->host;
74 struct plugind *cd = ((PARSER_USER_OBJECT *)user)->cd;
75 - if (cd->version < VERSION_GAP_FILLING ) {
76 - error("STREAM %s from %s: Child negotiated version %u but sent TIMESTAMP!", rrdhost_hostname(host), cd->cmd,
77 - cd->version);
75 + if (!(cd->capabilities & STREAM_CAP_GAP_FILLING)) {
76 + error("STREAM %s from %s: Child negotiated version %u but sent TIMESTAMP!", rrdhost_hostname(host), cd->cmd, cd->capabilities);
77 return PARSER_RC_OK; // Ignore error and continue stream
78 }
79 if (remote_time_txt && *remote_time_txt) {
@@ -111,8 +110,8 @@ PARSER_RC streaming_timestamp(char **words, void *user, PLUGINSD_ACTION *plugins
110 (int64_t)remote_time);
111 int ret;
112 #ifdef ENABLE_HTTPS
114 - SSL *conn = host->stream_ssl.conn ;
115 - if(conn && !host->stream_ssl.flags) {
113 + SSL *conn = host->receiver->ssl.conn ;
114 + if(conn && !host->receiver->ssl.flags) {
115 ret = SSL_write(conn, message, strlen(message));
116 } else {
117 ret = send(host->receiver->fd, message, strlen(message), MSG_DONTWAIT);
@@ -292,8 +291,10 @@ static int receiver_read(struct receiver_state *r, FILE *fp) {
291 size_t available = sizeof(r->read_buffer) - r->read_len;
292 if (available) {
293 size_t len = r->decompressor->get(r->decompressor, r->read_buffer + r->read_len, available);
295 - if (!len)
294 + if (!len) {
295 + internal_error(true, "decompressor returned zero length");
296 return 1;
297 + }
298
299 r->read_len += len;
300 }
@@ -301,8 +302,10 @@ static int receiver_read(struct receiver_state *r, FILE *fp) {
302 }
303
304 int ret = 0;
304 - if (read_stream(r, fp, r->read_buffer + r->read_len, sizeof(r->read_buffer) - r->read_len - 1, &ret))
305 + if (read_stream(r, fp, r->read_buffer + r->read_len, sizeof(r->read_buffer) - r->read_len - 1, &ret)) {
306 + internal_error(true, "read_stream() failed (1).");
307 return 1;
308 + }
309
310 worker_set_metric(WORKER_RECEIVER_JOB_BYTES_READ, ret);
311
@@ -320,8 +323,10 @@ static int receiver_read(struct receiver_state *r, FILE *fp) {
323 // we're unable to decompress incomplete block
324 char compressed[bytes_to_read];
325 do {
323 - if (read_stream(r, fp, compressed, bytes_to_read, &ret))
326 + if (read_stream(r, fp, compressed, bytes_to_read, &ret)) {
327 + internal_error(true, "read_stream() failed (2).");
328 return 1;
329 + }
330
331 worker_set_metric(WORKER_RECEIVER_JOB_BYTES_READ, ret);
332
@@ -334,8 +339,10 @@ static int receiver_read(struct receiver_state *r, FILE *fp) {
339
340 // Decompress
341 size_t bytes_to_parse = r->decompressor->decompress(r->decompressor);
337 - if (!bytes_to_parse)
342 + if (!bytes_to_parse) {
343 + internal_error(true, "no bytes to parse.");
344 return 1;
345 + }
346
347 // Fill read buffer with decompressed data
348 r->read_len = r->decompressor->get(r->decompressor, r->read_buffer, sizeof(r->read_buffer));
@@ -347,30 +354,56 @@ static int receiver_read(struct receiver_state *r, FILE *fp) {
354 /* Produce a full line if one exists, statefully return where we start next time.
355 * When we hit the end of the buffer with a partial line move it to the beginning for the next fill.
356 */
350 -static char *receiver_next_line(struct receiver_state *r, int *pos) {
351 - int start = *pos, scan = *pos;
352 - if (scan >= r->read_len) {
357 +static char *receiver_next_line(struct receiver_state *r, char *buffer, size_t buffer_length, size_t *pos) {
358 + size_t start = *pos;
359 +
360 + char *ss = &r->read_buffer[start];
361 + char *se = &r->read_buffer[r->read_len];
362 + char *ds = buffer;
363 + char *de = &buffer[buffer_length - 2];
364 +
365 + if(ss >= se) {
366 r->read_len = 0;
367 return NULL;
368 }
356 - while (scan < r->read_len && r->read_buffer[scan] != '\n')
357 - scan++;
358 - if (scan < r->read_len && r->read_buffer[scan] == '\n') {
359 - *pos = scan+1;
360 - r->read_buffer[scan] = 0;
361 - return &r->read_buffer[start];
369 +
370 + // copy all bytes to buffer
371 + while(ss < se && ds < de && *ss != '\n')
372 + *ds++ = *ss++;
373 +
374 + // if we have a newline, return the buffer
375 + if(ss < se && ds < de && *ss == '\n') {
376 + // newline found in the r->read_buffer
377 +
378 + *ds++ = *ss++; // copy the newline too
379 + *ds = '\0';
380 +
381 + *pos = ss - r->read_buffer;
382 + return buffer;
383 }
384 +
385 + // if the destination is full, oops!
386 + if(ds == de) {
387 + error("STREAM: received line exceeds %d bytes. Truncating it.", PLUGINSD_LINE_MAX);
388 + *ds = '\0';
389 + *pos = ss - r->read_buffer;
390 + return buffer;
391 + }
392 +
393 + // no newline found in the r->read_buffer
394 + // move everything to the beginning
395 memmove(r->read_buffer, &r->read_buffer[start], r->read_len - start);
364 - r->read_len -= start;
396 + r->read_len -= (int)start;
397 return NULL;
398 }
399
400 static void streaming_parser_thread_cleanup(void *ptr) {
401 PARSER *parser = (PARSER *)ptr;
402 + rrd_collector_finished();
403 parser_destroy(parser);
404 }
405
373 -size_t streaming_parser(struct receiver_state *rpt, struct plugind *cd, FILE *fp) {
406 +size_t streaming_parser(struct receiver_state *rpt, struct plugind *cd, FILE *fp_in, FILE *fp_out) {
407 size_t result;
408
409 PARSER_USER_OBJECT user = {
@@ -381,7 +414,9 @@ size_t streaming_parser(struct receiver_state *rpt, struct plugind *cd, FILE *fp
414 .trust_durations = 1
415 };
416
384 - PARSER *parser = parser_init(rpt->host, &user, fp, PARSER_INPUT_SPLIT);
417 + PARSER *parser = parser_init(rpt->host, &user, fp_in, fp_out, PARSER_INPUT_SPLIT);
418 +
419 + rrd_collector_started();
420
421 // this keeps the parser with its current value
422 // so, parser needs to be allocated before pushing it
@@ -390,19 +425,6 @@ size_t streaming_parser(struct receiver_state *rpt, struct plugind *cd, FILE *fp
425 parser_add_keyword(parser, "TIMESTAMP", streaming_timestamp);
426 parser_add_keyword(parser, "CLAIMED_ID", streaming_claimed_id);
427
393 - parser->plugins_action->begin_action = &pluginsd_begin_action;
394 - parser->plugins_action->flush_action = &pluginsd_flush_action;
395 - parser->plugins_action->end_action = &pluginsd_end_action;
396 - parser->plugins_action->disable_action = &pluginsd_disable_action;
397 - parser->plugins_action->variable_action = &pluginsd_variable_action;
398 - parser->plugins_action->dimension_action = &pluginsd_dimension_action;
399 - parser->plugins_action->label_action = &pluginsd_label_action;
400 - parser->plugins_action->overwrite_action = &pluginsd_overwrite_action;
401 - parser->plugins_action->chart_action = &pluginsd_chart_action;
402 - parser->plugins_action->set_action = &pluginsd_set_action;
403 - parser->plugins_action->clabel_commit_action = &pluginsd_clabel_commit_action;
404 - parser->plugins_action->clabel_action = &pluginsd_clabel_action;
405 -
428 user.parser = parser;
429
430 #ifdef ENABLE_COMPRESSION
@@ -410,15 +432,26 @@ size_t streaming_parser(struct receiver_state *rpt, struct plugind *cd, FILE *fp
432 rpt->decompressor->reset(rpt->decompressor);
433 #endif
434
413 - do{
414 - if (receiver_read(rpt, fp))
415 - break;
416 - int pos = 0;
417 - char *line;
418 - while ((line = receiver_next_line(rpt, &pos))) {
419 - if (unlikely(netdata_exit || rpt->shutdown || parser_action(parser, line)))
435 + char buffer[PLUGINSD_LINE_MAX + 2];
436 + do {
437 + if(receiver_read(rpt, fp_in)) break;
438 +
439 + size_t pos = 0;
440 + while(receiver_next_line(rpt, buffer, PLUGINSD_LINE_MAX + 2, &pos)) {
441 + if(unlikely(netdata_exit)) {
442 + internal_error(true, "exiting...");
443 + goto done;
444 + }
445 + if(unlikely(rpt->shutdown)) {
446 + internal_error(true, "parser shutdown...");
447 goto done;
448 + }
449 + if (unlikely(parser_action(parser, buffer))) {
450 + internal_error(true, "parser_action() failed...");
451 + goto done;
452 + }
453 }
454 +
455 rpt->last_msg_t = now_realtime_sec();
456 }
457 while(!netdata_exit);
@@ -495,8 +528,6 @@ static int rrdpush_receive(struct receiver_state *rpt)
528 char initial_response[HTTP_HEADER_SIZE + 1];
529 snprintfz(initial_response, HTTP_HEADER_SIZE, "%s", START_STREAMING_ERROR_SAME_LOCALHOST);
530 #ifdef ENABLE_HTTPS
498 - rpt->host->stream_ssl.conn = rpt->ssl.conn;
499 - rpt->host->stream_ssl.flags = rpt->ssl.flags;
531 if(send_timeout(&rpt->ssl, rpt->fd, initial_response, strlen(initial_response), 0, 60) != (ssize_t)strlen(initial_response)) {
532 #else
533 if(send_timeout(rpt->fd, initial_response, strlen(initial_response), 0, 60) != strlen(initial_response)) {
@@ -611,7 +642,7 @@ static int rrdpush_receive(struct receiver_state *rpt)
642 .obsolete = 0,
643 .started_t = now_realtime_sec(),
644 .next = NULL,
614 - .version = 0,
645 + .capabilities = 0,
646 };
647
648 // put the client IP and port into the buffers used by plugins.d
@@ -620,32 +651,31 @@ static int rrdpush_receive(struct receiver_state *rpt)
651 snprintfz(cd.fullfilename, FILENAME_MAX, "%s:%s", rpt->client_ip, rpt->client_port);
652 snprintfz(cd.cmd, PLUGINSD_CMD_MAX, "%s:%s", rpt->client_ip, rpt->client_port);
653
623 - info("STREAM %s [receive from [%s]:%s]: initializing communication...", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port);
624 - char initial_response[HTTP_HEADER_SIZE];
625 - if (rpt->stream_version > 1) {
626 - if(rpt->stream_version >= STREAM_VERSION_COMPRESSION){
654 #ifdef ENABLE_COMPRESSION
628 - if(!rpt->rrdpush_compression)
629 - rpt->stream_version = STREAM_VERSION_CLABELS;
630 -#else
631 - if(STREAMING_PROTOCOL_CURRENT_VERSION < rpt->stream_version) {
632 - rpt->stream_version = STREAMING_PROTOCOL_CURRENT_VERSION;
633 - }
655 + if (stream_has_capability(rpt, STREAM_CAP_COMPRESSION)) {
656 + if (!rpt->rrdpush_compression)
657 + rpt->capabilities &= ~STREAM_CAP_COMPRESSION;
658 + }
659 #endif
635 - }
636 - info("STREAM %s [receive from [%s]:%s]: Netdata is using the stream version %u.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->stream_version);
637 - sprintf(initial_response, "%s%u", START_STREAMING_PROMPT_VN, rpt->stream_version);
638 - } else if (rpt->stream_version == 1) {
639 - info("STREAM %s [receive from [%s]:%s]: Netdata is using the stream version %u.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->stream_version);
660 +
661 + info("STREAM %s [receive from [%s]:%s]: initializing communication...", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port);
662 + char initial_response[HTTP_HEADER_SIZE];
663 + if (stream_has_capability(rpt, STREAM_CAP_VCAPS)) {
664 + log_receiver_capabilities(rpt);
665 + sprintf(initial_response, "%s%u", START_STREAMING_PROMPT_VN, rpt->capabilities);
666 + }
667 + else if (stream_has_capability(rpt, STREAM_CAP_VN)) {
668 + log_receiver_capabilities(rpt);
669 + sprintf(initial_response, "%s%d", START_STREAMING_PROMPT_VN, stream_capabilities_to_vn(rpt->capabilities));
670 + } else if (stream_has_capability(rpt, STREAM_CAP_V2)) {
671 + log_receiver_capabilities(rpt);
672 sprintf(initial_response, "%s", START_STREAMING_PROMPT_V2);
641 - } else {
642 - info("STREAM %s [receive from [%s]:%s]: Netdata is using first stream protocol.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port);
643 - sprintf(initial_response, "%s", START_STREAMING_PROMPT);
673 + } else { // stream_has_capability(rpt, STREAM_CAP_V1)
674 + log_receiver_capabilities(rpt);
675 + sprintf(initial_response, "%s", START_STREAMING_PROMPT_V1);
676 }
677 debug(D_STREAM, "Initial response to %s: %s", rpt->client_ip, initial_response);
646 - #ifdef ENABLE_HTTPS
647 - rpt->host->stream_ssl.conn = rpt->ssl.conn;
648 - rpt->host->stream_ssl.flags = rpt->ssl.flags;
678 +#ifdef ENABLE_HTTPS
679 if(send_timeout(&rpt->ssl, rpt->fd, initial_response, strlen(initial_response), 0, 60) != (ssize_t)strlen(initial_response)) {
680 #else
681 if(send_timeout(rpt->fd, initial_response, strlen(initial_response), 0, 60) != strlen(initial_response)) {
@@ -667,11 +697,33 @@ static int rrdpush_receive(struct receiver_state *rpt)
697 error("STREAM %s [receive from [%s]:%s]: cannot set timeout for socket %d", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->fd);
698
699 // convert the socket to a FILE *
670 - FILE *fp = fdopen(rpt->fd, "r");
671 - if(!fp) {
700 + // It seems that the same FILE * cannot be used for both reading and writing.
701 + // (reads and writes seem to interfere with each other, with undefined results).
702 +
703 + int fd_in = rpt->fd;
704 + int fd_out = fcntl(rpt->fd, F_DUPFD_CLOEXEC, 0);
705 + if(fd_out == -1) {
706 + error("STREAM %s [receive from [%s]:%s]: failed to duplicate FD %d.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->fd);
707 log_stream_connection(rpt->client_ip, rpt->client_port, rpt->key, rpt->host->machine_guid, rrdhost_hostname(rpt->host), "FAILED - SOCKET ERROR");
673 - error("STREAM %s [receive from [%s]:%s]: failed to get a FILE for FD %d.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->fd);
674 - close(rpt->fd);
708 + close(fd_in);
709 + return 0;
710 + }
711 +
712 + FILE *fp_out = fdopen(fd_out, "w");
713 + if(!fp_out) {
714 + error("STREAM %s [receive from [%s]:%s]: failed to get a FILE pointer for fd_out %d.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->fd);
715 + log_stream_connection(rpt->client_ip, rpt->client_port, rpt->key, rpt->host->machine_guid, rrdhost_hostname(rpt->host), "FAILED - SOCKET ERROR");
716 + close(fd_in);
717 + close(fd_out);
718 + return 0;
719 + }
720 +
721 + FILE *fp_in = fdopen(fd_in, "r");
722 + if(!fp_in) {
723 + error("STREAM %s [receive from [%s]:%s]: failed to get a FILE pointer for fd_in %d.", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, rpt->fd);
724 + log_stream_connection(rpt->client_ip, rpt->client_port, rpt->key, rpt->host->machine_guid, rrdhost_hostname(rpt->host), "FAILED - SOCKET ERROR");
725 + close(fd_in);
726 + fclose(fp_out);
727 return 0;
728 }
729
@@ -686,15 +738,6 @@ static int rrdpush_receive(struct receiver_state *rpt)
738 */
739
740 // rpt->host->connected_senders++;
689 - if(rpt->stream_version > 0) {
690 - rrdhost_flag_set(rpt->host, RRDHOST_FLAG_STREAM_LABELS_UPDATE);
691 - rrdhost_flag_clear(rpt->host, RRDHOST_FLAG_STREAM_LABELS_STOP);
692 - }
693 - else {
694 - rrdhost_flag_set(rpt->host, RRDHOST_FLAG_STREAM_LABELS_STOP);
695 - rrdhost_flag_clear(rpt->host, RRDHOST_FLAG_STREAM_LABELS_UPDATE);
696 - }
697 -
741 if(health_enabled != CONFIG_BOOLEAN_NO) {
742 if(alarms_delay > 0) {
743 rpt->host->health_delay_up_to = now_realtime_sec() + alarms_delay;
@@ -713,7 +756,7 @@ static int rrdpush_receive(struct receiver_state *rpt)
756 info("STREAM %s [receive from [%s]:%s]: receiving metrics...", rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port);
757 log_stream_connection(rpt->client_ip, rpt->client_port, rpt->key, rpt->host->machine_guid, rrdhost_hostname(rpt->host), "CONNECTED");
758
716 - cd.version = rpt->stream_version;
759 + cd.capabilities = rpt->capabilities;
760
761 #ifdef ENABLE_ACLK
762 // in case we have cloud connection we inform cloud
@@ -724,7 +767,7 @@ static int rrdpush_receive(struct receiver_state *rpt)
767
768 rrdcontext_host_child_connected(rpt->host);
769
727 - size_t count = streaming_parser(rpt, &cd, fp);
770 + size_t count = streaming_parser(rpt, &cd, fp_in, fp_out);
771
772 log_stream_connection(rpt->client_ip, rpt->client_port, rpt->key, rpt->host->machine_guid, rpt->hostname,
773 "DISCONNECTED");
@@ -762,7 +805,8 @@ static int rrdpush_receive(struct receiver_state *rpt)
805 }
806
807 // cleanup
765 - fclose(fp);
808 + fclose(fp_in);
809 + fclose(fp_out);
810 return (int)count;
811 }
812
streaming/rrdpush.c
+225 -151
@@ -11,8 +11,8 @@
11 * 1. a random data collection thread, calling rrdset_done_push()
12 * this is called for each chart.
13 *
14 - * the output of this work is kept in a BUFFER in RRDHOST
15 - * the sender thread is signalled via a pipe (also in RRDHOST)
14 + * the output of this work is kept in a thread BUFFER
15 + * the sender thread is signalled via a pipe (in RRDHOST)
16 *
17 * 2. a sender thread running at the sending netdata
18 * this is spawned automatically on the first chart to be pushed
@@ -101,9 +101,9 @@ int rrdpush_init() {
101 bool invalid_certificate = appconfig_get_boolean(&stream_config, CONFIG_SECTION_STREAM, "ssl skip certificate verification", CONFIG_BOOLEAN_NO);
102
103 if(invalid_certificate == CONFIG_BOOLEAN_YES){
104 - if(netdata_validate_server == NETDATA_SSL_VALID_CERTIFICATE){
104 + if(netdata_ssl_validate_server == NETDATA_SSL_VALID_CERTIFICATE){
105 info("Netdata is configured to accept invalid SSL certificate.");
106 - netdata_validate_server = NETDATA_SSL_INVALID_CERTIFICATE;
106 + netdata_ssl_validate_server = NETDATA_SSL_INVALID_CERTIFICATE;
107 }
108 }
109
@@ -130,40 +130,35 @@ unsigned int remote_clock_resync_iterations = 60;
130
131
132 static inline bool should_send_chart_matching(RRDSET *st) {
133 - RRDSET_FLAGS flags = rrdset_flag_check(st, RRDSET_FLAG_UPSTREAM_SEND|RRDSET_FLAG_UPSTREAM_IGNORE);
133 + // get all the flags we need to check, with one atomic operation
134 + RRDSET_FLAGS flags = rrdset_flag_check(st,
135 + RRDSET_FLAG_UPSTREAM_SEND
136 + |RRDSET_FLAG_UPSTREAM_IGNORE
137 + |RRDSET_FLAG_ANOMALY_RATE_CHART
138 + |RRDSET_FLAG_ANOMALY_DETECTION);
139
140 if(unlikely(!flags)) {
141 RRDHOST *host = st->rrdhost;
142
143 // Do not stream anomaly rates charts.
139 - if (unlikely(rrdset_is_ar_chart(st))) {
140 - rrdset_flag_clear(st, RRDSET_FLAG_UPSTREAM_SEND);
144 + if (unlikely(flags & RRDSET_FLAG_ANOMALY_RATE_CHART))
145 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_IGNORE);
142 - flags = RRDSET_FLAG_UPSTREAM_IGNORE;
143 - }
144 - else if (rrdset_flag_check(st, RRDSET_FLAG_ANOMALY_DETECTION)) {
145 - if(ml_streaming_enabled()) {
146 - rrdset_flag_clear(st, RRDSET_FLAG_UPSTREAM_IGNORE);
146 +
147 + else if (flags & RRDSET_FLAG_ANOMALY_DETECTION) {
148 + if(ml_streaming_enabled())
149 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_SEND);
148 - flags = RRDSET_FLAG_UPSTREAM_SEND;
149 - }
150 - else {
151 - rrdset_flag_clear(st, RRDSET_FLAG_UPSTREAM_SEND);
150 + else
151 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_IGNORE);
153 - flags = RRDSET_FLAG_UPSTREAM_IGNORE;
154 - }
152 }
153 else if(simple_pattern_matches(host->rrdpush_send_charts_matching, rrdset_id(st)) ||
157 - simple_pattern_matches(host->rrdpush_send_charts_matching, rrdset_name(st))) {
158 - rrdset_flag_clear(st, RRDSET_FLAG_UPSTREAM_IGNORE);
154 + simple_pattern_matches(host->rrdpush_send_charts_matching, rrdset_name(st)))
155 +
156 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_SEND);
160 - flags = RRDSET_FLAG_UPSTREAM_SEND;
161 - }
162 - else {
163 - rrdset_flag_clear(st, RRDSET_FLAG_UPSTREAM_SEND);
157 + else
158 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_IGNORE);
165 - flags = RRDSET_FLAG_UPSTREAM_IGNORE;
166 - }
159 +
160 + // get the flags again, to know how to respond
161 + flags = rrdset_flag_check(st, RRDSET_FLAG_UPSTREAM_SEND|RRDSET_FLAG_UPSTREAM_IGNORE);
162 }
163
164 return flags & RRDSET_FLAG_UPSTREAM_SEND;
@@ -196,16 +191,17 @@ static int send_clabels_callback(const char *name, const char *value, RRDLABEL_S
191 buffer_sprintf(wb, "CLABEL \"%s\" \"%s\" %d\n", name, value, ls);
192 return 1;
193 }
199 -void rrdpush_send_clabels(RRDHOST *host, RRDSET *st) {
194 +
195 +static void rrdpush_send_clabels(BUFFER *wb, RRDSET *st) {
196 if (st->rrdlabels) {
201 - if(rrdlabels_walkthrough_read(st->rrdlabels, send_clabels_callback, host->sender->build) > 0)
202 - buffer_sprintf(host->sender->build,"CLABEL_COMMIT\n");
197 + if(rrdlabels_walkthrough_read(st->rrdlabels, send_clabels_callback, wb) > 0)
198 + buffer_sprintf(wb, "CLABEL_COMMIT\n");
199 }
200 }
201
202 // Send the current chart definition.
203 // Assumes that collector thread has already called sender_start for mutex / buffer state.
208 -static inline void rrdpush_send_chart_definition(RRDSET *st) {
204 +static inline void rrdpush_send_chart_definition(BUFFER *wb, RRDSET *st) {
205 RRDHOST *host = st->rrdhost;
206
207 rrdset_flag_set(st, RRDSET_FLAG_UPSTREAM_EXPOSED);
@@ -225,7 +221,7 @@ static inline void rrdpush_send_chart_definition(RRDSET *st) {
221
222 // send the chart
223 buffer_sprintf(
228 - host->sender->build
224 + wb
225 , "CHART \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" \"%s\" %ld %d \"%s %s %s %s\" \"%s\" \"%s\"\n"
226 , rrdset_id(st)
227 , name
@@ -245,14 +241,14 @@ static inline void rrdpush_send_chart_definition(RRDSET *st) {
241 );
242
243 // send the chart labels
248 - if (host->sender->version >= STREAM_VERSION_CLABELS)
249 - rrdpush_send_clabels(host, st);
244 + if (stream_has_capability(host->sender, STREAM_CAP_CLABELS))
245 + rrdpush_send_clabels(wb, st);
246
247 // send the dimensions
248 RRDDIM *rd;
249 rrddim_foreach_read(rd, st) {
250 buffer_sprintf(
255 - host->sender->build
251 + wb
252 , "DIMENSION \"%s\" \"%s\" \"%s\" " COLLECTED_NUMBER_FORMAT " " COLLECTED_NUMBER_FORMAT " \"%s %s %s\"\n"
253 , rrddim_id(rd)
254 , rrddim_name(rd)
@@ -267,30 +263,30 @@ static inline void rrdpush_send_chart_definition(RRDSET *st) {
263 }
264 rrddim_foreach_done(rd);
265
266 + // send the chart functions
267 + if(stream_has_capability(host->sender, STREAM_CAP_FUNCTIONS))
268 + rrd_functions_expose_rrdpush(st, wb);
269 +
270 // send the chart local custom variables
271 - rrdsetvar_print_to_streaming_custom_chart_variables(st, host->sender->build);
271 + rrdsetvar_print_to_streaming_custom_chart_variables(st, wb);
272
273 st->upstream_resync_time = st->last_collected_time.tv_sec + (remote_clock_resync_iterations * st->update_every);
274 }
275
276 // sends the current chart dimensions
277 -static inline bool rrdpush_send_chart_metrics_nolock(RRDSET *st, struct sender_state *s) {
278 - RRDHOST *host = st->rrdhost;
279 - BUFFER *wb = host->sender->build;
280 -
277 +static inline void rrdpush_send_chart_metrics(BUFFER *wb, RRDSET *st, struct sender_state *s) {
278 buffer_fast_strcat(wb, "BEGIN \"", 7);
279 buffer_fast_strcat(wb, rrdset_id(st), string_strlen(st->id));
280 buffer_fast_strcat(wb, "\" ", 2);
281 buffer_print_llu(wb, (st->last_collected_time.tv_sec > st->upstream_resync_time)?st->usec_since_last_update:0);
282
286 - if (s->version >= VERSION_GAP_FILLING) {
283 + if (stream_has_capability(s, STREAM_CAP_GAP_FILLING)) {
284 buffer_fast_strcat(wb, " ", 1);
285 buffer_print_ll(wb, st->last_collected_time.tv_sec);
286 }
287
288 buffer_fast_strcat(wb, "\n", 1);
289
293 - size_t count_of_dimensions_written = 0;
290 RRDDIM *rd;
291 rrddim_foreach_read(rd, st) {
292 if(unlikely(!rd->updated))
@@ -302,7 +298,6 @@ static inline bool rrdpush_send_chart_metrics_nolock(RRDSET *st, struct sender_s
298 buffer_fast_strcat(wb, "\" = ", 4);
299 buffer_print_ll(wb, rd->collected_value);
300 buffer_fast_strcat(wb, "\n", 1);
305 - count_of_dimensions_written++;
301 }
302 else {
303 internal_error(true, "host '%s', chart '%s', dimension '%s' flag 'exposed' is updated but not exposed", rrdhost_hostname(st->rrdhost), rrdset_id(st), rrddim_id(rd));
@@ -312,8 +307,6 @@ static inline bool rrdpush_send_chart_metrics_nolock(RRDSET *st, struct sender_s
307 }
308 rrddim_foreach_done(rd);
309 buffer_fast_strcat(wb, "END\n", 4);
315 -
316 - return count_of_dimensions_written != 0;
310 }
311
312 static void rrdpush_sender_thread_spawn(RRDHOST *host);
@@ -322,12 +315,12 @@ static void rrdpush_sender_thread_spawn(RRDHOST *host);
315 bool rrdset_push_chart_definition_now(RRDSET *st) {
316 RRDHOST *host = st->rrdhost;
317
325 - if(unlikely(!host->rrdpush_send_enabled || !should_send_chart_matching(st)))
318 + if(unlikely(!rrdhost_can_send_definitions_to_parent(host) || !should_send_chart_matching(st)))
319 return false;
320
328 - sender_start(host->sender);
329 - rrdpush_send_chart_definition(st);
330 - sender_commit(host->sender);
321 + BUFFER *wb = sender_start(host->sender);
322 + rrdpush_send_chart_definition(wb, st);
323 + sender_commit(host->sender, wb);
324
325 return true;
326 }
@@ -365,44 +358,44 @@ bool rrdpush_incremental_transmission_of_chart_definitions(RRDHOST *host, DICTFE
358 }
359
360 void rrdset_done_push(RRDSET *st) {
368 - if(unlikely(!should_send_chart_matching(st)))
369 - return;
370 -
361 RRDHOST *host = st->rrdhost;
362
373 - // Handle non-connected case
374 - if(unlikely(!__atomic_load_n(&host->rrdpush_sender_connected, __ATOMIC_SEQ_CST)
375 - || !rrdhost_flag_check(host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS))) {
363 + // fetch the flags we need to check with one atomic operation
364 + RRDHOST_FLAGS flags = rrdhost_flag_check(host,
365 + RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS
366 + | RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS
367 + | RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN
368 + );
369 +
370 + // check if we are not connected
371 + if(unlikely(!(flags & RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS))) {
372
377 - if(unlikely(host->rrdpush_send_enabled && !host->rrdpush_sender_spawn))
373 + if(unlikely(!(flags & RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN)))
374 rrdpush_sender_thread_spawn(host);
375
380 - if(unlikely(!host->rrdpush_sender_error_shown))
376 + if(unlikely(!(flags & RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS))) {
377 + rrdhost_flag_set(host, RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS);
378 error("STREAM %s [send]: not ready - collected metrics are not sent to parent.", rrdhost_hostname(host));
382 -
383 - host->rrdpush_sender_error_shown = 1;
379 + }
380
381 return;
382 }
387 - else if(unlikely(host->rrdpush_sender_error_shown)) {
383 + else if(unlikely(flags & RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS)) {
384 info("STREAM %s [send]: sending metrics to parent...", rrdhost_hostname(host));
389 - host->rrdpush_sender_error_shown = 0;
385 + rrdhost_flag_clear(host, RRDHOST_FLAG_RRDPUSH_SENDER_LOGGED_STATUS);
386 }
387
392 - sender_start(host->sender);
388 + if(unlikely(!should_send_chart_matching(st)))
389 + return;
390 +
391 + BUFFER *wb = sender_start(host->sender);
392
393 if(unlikely(need_to_send_chart_definition(st)))
395 - rrdpush_send_chart_definition(st);
394 + rrdpush_send_chart_definition(wb, st);
395
397 - if(likely(rrdpush_send_chart_metrics_nolock(st, host->sender))) {
398 - // signal the sender there are more data
399 - if (host->rrdpush_sender_pipe[PIPE_WRITE] != -1 && write(host->rrdpush_sender_pipe[PIPE_WRITE], " ", 1) == -1)
400 - error("STREAM %s [send]: cannot write to internal pipe", rrdhost_hostname(host));
396 + rrdpush_send_chart_metrics(wb, st, host->sender);
397
402 - sender_commit(host->sender);
403 - }
404 - else
405 - sender_cancel(host->sender);
398 + sender_commit(host->sender, wb);
399 }
400
401 // labels
@@ -411,45 +404,38 @@ static int send_labels_callback(const char *name, const char *value, RRDLABEL_SR
404 buffer_sprintf(wb, "LABEL \"%s\" = %d \"%s\"\n", name, ls, value);
405 return 1;
406 }
414 -void rrdpush_send_labels(RRDHOST *host) {
415 - if (!host->rrdlabels || !rrdhost_flag_check(host, RRDHOST_FLAG_STREAM_LABELS_UPDATE) || (rrdhost_flag_check(host, RRDHOST_FLAG_STREAM_LABELS_STOP)))
407 +void rrdpush_send_host_labels(RRDHOST *host) {
408 + if(unlikely(!rrdhost_can_send_definitions_to_parent(host)
409 + || !stream_has_capability(host->sender, STREAM_CAP_HLABELS)))
410 return;
411
418 - sender_start(host->sender);
419 -
420 - rrdlabels_walkthrough_read(host->rrdlabels, send_labels_callback, host->sender->build);
421 - buffer_sprintf(host->sender->build, "OVERWRITE %s\n", "labels");
422 - sender_commit(host->sender);
412 + BUFFER *wb = sender_start(host->sender);
413
424 - if(host->rrdpush_sender_pipe[PIPE_WRITE] != -1 && write(host->rrdpush_sender_pipe[PIPE_WRITE], " ", 1) == -1)
425 - error("STREAM %s [send]: cannot write to internal pipe", rrdhost_hostname(host));
414 + rrdlabels_walkthrough_read(host->rrdlabels, send_labels_callback, wb);
415 + buffer_sprintf(wb, "OVERWRITE %s\n", "labels");
416
427 - rrdhost_flag_clear(host, RRDHOST_FLAG_STREAM_LABELS_UPDATE);
417 + sender_commit(host->sender, wb);
418 }
419
420 void rrdpush_claimed_id(RRDHOST *host)
421 {
432 - if(unlikely(!host->rrdpush_send_enabled || !__atomic_load_n(&host->rrdpush_sender_connected, __ATOMIC_SEQ_CST)))
433 - return;
434 -
435 - if(host->sender->version < STREAM_VERSION_CLAIM)
422 + if(!stream_has_capability(host->sender, STREAM_CAP_CLAIM))
423 return;
424
438 - sender_start(host->sender);
425 + if(unlikely(!rrdhost_can_send_definitions_to_parent(host)))
426 + return;
427 +
428 + BUFFER *wb = sender_start(host->sender);
429 rrdhost_aclk_state_lock(host);
430
441 - buffer_sprintf(host->sender->build, "CLAIMED_ID %s %s\n", host->machine_guid, (host->aclk_state.claimed_id ? host->aclk_state.claimed_id : "NULL") );
431 + buffer_sprintf(wb, "CLAIMED_ID %s %s\n", host->machine_guid, (host->aclk_state.claimed_id ? host->aclk_state.claimed_id : "NULL") );
432
433 rrdhost_aclk_state_unlock(host);
444 - sender_commit(host->sender);
445 -
446 - // signal the sender there are more data
447 - if(host->rrdpush_sender_pipe[PIPE_WRITE] != -1 && write(host->rrdpush_sender_pipe[PIPE_WRITE], " ", 1) == -1)
448 - error("STREAM %s [send]: cannot write to internal pipe", rrdhost_hostname(host));
434 + sender_commit(host->sender, wb);
435 }
436
437 int connect_to_one_of_destinations(
452 - struct rrdpush_destinations *destinations,
438 + RRDHOST *host,
439 int default_port,
440 struct timeval *timeout,
441 size_t *reconnects_counter,
@@ -459,28 +445,44 @@ int connect_to_one_of_destinations(
445 {
446 int sock = -1;
447
462 - for (struct rrdpush_destinations *d = destinations; d; d = d->next) {
463 - if (d->disabled_no_proper_reply) {
464 - d->disabled_no_proper_reply = 0;
465 - continue;
466 - } else if (d->disabled_because_of_localhost) {
467 - continue;
468 - } else if (d->disabled_already_streaming && (d->disabled_already_streaming + 30 > now_realtime_sec())) {
469 - continue;
470 - } else if (d->disabled_because_of_denied_access) {
471 - d->disabled_because_of_denied_access = 0;
448 + for (struct rrdpush_destinations *d = host->destinations; d; d = d->next) {
449 + time_t now = now_realtime_sec();
450 +
451 + if(d->postpone_reconnection_until > now) {
452 + info(
453 + "STREAM %s: skipping destination '%s' (default port: %d) due to last error (code: %d, %s), will retry it in %d seconds",
454 + rrdhost_hostname(host),
455 + string2str(d->destination),
456 + default_port,
457 + d->last_handshake, d->last_error?d->last_error:"unset reason description",
458 + (int)(d->postpone_reconnection_until - now));
459 +
460 continue;
461 }
462
463 + info(
464 + "STREAM %s: attempting to connect to '%s' (default port: %d)...",
465 + rrdhost_hostname(host),
466 + string2str(d->destination),
467 + default_port);
468 +
469 if (reconnects_counter)
470 *reconnects_counter += 1;
477 - sock = connect_to_this(d->destination, default_port, timeout);
471 +
472 + sock = connect_to_this(string2str(d->destination), default_port, timeout);
473 +
474 if (sock != -1) {
479 - if (connected_to && connected_to_size) {
480 - strncpy(connected_to, d->destination, connected_to_size);
481 - connected_to[connected_to_size - 1] = '\0';
482 - }
475 + if (connected_to && connected_to_size)
476 + strncpyz(connected_to, string2str(d->destination), connected_to_size);
477 +
478 *destination = d;
479 +
480 + // move the current item to the end of the list
481 + // without this, this destination will break the loop again and again
482 + // not advancing the destinations to find one that may work
483 + DOUBLE_LINKED_LIST_REMOVE_UNSAFE(host->destinations, d, prev, next);
484 + DOUBLE_LINKED_LIST_APPEND_UNSAFE(host->destinations, d, prev, next);
485 +
486 break;
487 }
488 }
@@ -488,44 +490,51 @@ int connect_to_one_of_destinations(
490 return sock;
491 }
492
491 -struct rrdpush_destinations *destinations_init(const char *dests) {
492 - const char *s = dests;
493 - struct rrdpush_destinations *destinations = NULL, *prev = NULL;
494 - while(*s) {
495 - const char *e = s;
496 -
497 - // skip path, moving both s(tart) and e(nd)
498 - if(*e == '/')
499 - while(!isspace(*e) && *e != ',') s = ++e;
500 -
501 - // skip separators, moving both s(tart) and e(nd)
502 - while(isspace(*e) || *e == ',') s = ++e;
503 -
504 - // move e(nd) to the first separator
505 - while(*e && !isspace(*e) && *e != ',' && *e != '/') e++;
506 -
507 - // is there anything?
508 - if(!*s || s == e) break;
509 -
510 - char buf[e - s + 1];
511 - strncpyz(buf, s, e - s);
512 - struct rrdpush_destinations *d = callocz(1, sizeof(struct rrdpush_destinations));
513 - strncpyz(d->destination, buf, sizeof(d->destination)-1);
514 - d->disabled_no_proper_reply = 0;
515 - d->disabled_because_of_localhost = 0;
516 - d->disabled_already_streaming = 0;
517 - d->disabled_because_of_denied_access = 0;
518 - d->next = NULL;
519 - if (!destinations) {
520 - destinations = d;
521 - } else {
522 - prev->next = d;
523 - }
524 - prev = d;
493 +struct destinations_init_tmp {
494 + RRDHOST *host;
495 + struct rrdpush_destinations *list;
496 + int count;
497 +};
498 +
499 +bool destinations_init_add_one(char *entry, void *data) {
500 + struct destinations_init_tmp *t = data;
501 +
502 + struct rrdpush_destinations *d = callocz(1, sizeof(struct rrdpush_destinations));
503 + d->destination = string_strdupz(entry);
504 +
505 + DOUBLE_LINKED_LIST_APPEND_UNSAFE(t->list, d, prev, next);
506 +
507 + t->count++;
508 + info("STREAM: added streaming destination No %d: '%s' to host '%s'", t->count, string2str(d->destination), rrdhost_hostname(t->host));
509
526 - s = e;
510 + return false; // we return false, so that we will get all defined destinations
511 +}
512 +
513 +void rrdpush_destinations_init(RRDHOST *host) {
514 + if(!host->rrdpush_send_destination) return;
515 +
516 + rrdpush_destinations_free(host);
517 +
518 + struct destinations_init_tmp t = {
519 + .host = host,
520 + .list = NULL,
521 + .count = 0,
522 + };
523 +
524 + foreach_entry_in_connection_string(host->rrdpush_send_destination, destinations_init_add_one, &t);
525 +
526 + host->destinations = t.list;
527 +}
528 +
529 +void rrdpush_destinations_free(RRDHOST *host) {
530 + while (host->destinations) {
531 + struct rrdpush_destinations *tmp = host->destinations;
532 + DOUBLE_LINKED_LIST_REMOVE_UNSAFE(host->destinations, tmp, prev, next);
533 + string_freez(tmp->destination);
534 + freez(tmp);
535 }
528 - return destinations;
536 +
537 + host->destinations = NULL;
538 }
539
540 // ----------------------------------------------------------------------------
@@ -541,11 +550,13 @@ void rrdpush_sender_thread_stop(RRDHOST *host) {
550 netdata_mutex_lock(&host->sender->mutex);
551 netdata_thread_t thr = 0;
552
544 - if(host->rrdpush_sender_spawn) {
553 + if(rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN)) {
554 + rrdhost_flag_clear(host, RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN);
555 +
556 info("STREAM %s [send]: signaling sending thread to stop...", rrdhost_hostname(host));
557
558 // signal the thread that we want to join it
548 - host->rrdpush_sender_join = 1;
559 + rrdhost_flag_set(host, RRDHOST_FLAG_RRDPUSH_SENDER_JOIN);
560
561 // copy the thread id, so that we will be waiting for the right one
562 // even if a new one has been spawn
@@ -577,15 +588,16 @@ void log_stream_connection(const char *client_ip, const char *client_port, const
588 static void rrdpush_sender_thread_spawn(RRDHOST *host) {
589 netdata_mutex_lock(&host->sender->mutex);
590
580 - if(!host->rrdpush_sender_spawn) {
591 + if(!rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN)) {
592 char tag[NETDATA_THREAD_TAG_MAX + 1];
593 snprintfz(tag, NETDATA_THREAD_TAG_MAX, "STREAM_SENDER[%s]", rrdhost_hostname(host));
594
595 if(netdata_thread_create(&host->rrdpush_sender_thread, tag, NETDATA_THREAD_OPTION_JOINABLE, rrdpush_sender_thread, (void *) host->sender))
596 error("STREAM %s [send]: failed to create new thread for client.", rrdhost_hostname(host));
597 else
587 - host->rrdpush_sender_spawn = 1;
598 + rrdhost_flag_set(host, RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN);
599 }
600 +
601 netdata_mutex_unlock(&host->sender->mutex);
602 }
603
@@ -654,7 +666,7 @@ int rrdpush_receiver_thread_spawn(struct web_client *w, char *url) {
666 else if(!strcmp(name, "tags"))
667 tags = value;
668 else if(!strcmp(name, "ver"))
657 - stream_version = MIN((uint32_t) strtoul(value, NULL, 0), STREAMING_PROTOCOL_CURRENT_VERSION);
669 + stream_version = convert_stream_version_to_capabilities(strtoul(value, NULL, 0));
670 else {
671 // An old Netdata child does not have a compatible streaming protocol, map to something sane.
672 if (!strcmp(name, "NETDATA_SYSTEM_OS_NAME"))
@@ -670,7 +682,7 @@ int rrdpush_receiver_thread_spawn(struct web_client *w, char *url) {
682 else if (!strcmp(name, "NETDATA_SYSTEM_OS_DETECTION"))
683 name = "NETDATA_HOST_OS_DETECTION";
684 else if(!strcmp(name, "NETDATA_PROTOCOL_VERSION") && stream_version == UINT_MAX) {
673 - stream_version = 1;
685 + stream_version = convert_stream_version_to_capabilities(1);
686 }
687
688 if (unlikely(rrdhost_set_system_info_variable(system_info, name, value))) {
@@ -681,7 +693,7 @@ int rrdpush_receiver_thread_spawn(struct web_client *w, char *url) {
693 }
694
695 if (stream_version == UINT_MAX)
684 - stream_version = 0;
696 + stream_version = convert_stream_version_to_capabilities(0);
697
698 if(!key || !*key) {
699 rrdhost_system_info_free(system_info);
@@ -857,7 +869,7 @@ int rrdpush_receiver_thread_spawn(struct web_client *w, char *url) {
869 rpt->client_port = strdupz(w->client_port);
870 rpt->update_every = update_every;
871 rpt->system_info = system_info;
860 - rpt->stream_version = stream_version;
872 + rpt->capabilities = stream_version;
873 #ifdef ENABLE_HTTPS
874 rpt->ssl.conn = w->ssl.conn;
875 rpt->ssl.flags = w->ssl.flags;
@@ -901,3 +913,65 @@ int rrdpush_receiver_thread_spawn(struct web_client *w, char *url) {
913 buffer_flush(w->response.data);
914 return 200;
915 }
916 +
917 +static void stream_capabilities_to_string(BUFFER *wb, STREAM_CAPABILITIES caps) {
918 + if(caps & STREAM_CAP_V1) buffer_strcat(wb, "V1 ");
919 + if(caps & STREAM_CAP_V2) buffer_strcat(wb, "V2 ");
920 + if(caps & STREAM_CAP_VN) buffer_strcat(wb, "VN ");
921 + if(caps & STREAM_CAP_VCAPS) buffer_strcat(wb, "VCAPS ");
922 + if(caps & STREAM_CAP_HLABELS) buffer_strcat(wb, "HLABELS ");
923 + if(caps & STREAM_CAP_CLAIM) buffer_strcat(wb, "CLAIM ");
924 + if(caps & STREAM_CAP_CLABELS) buffer_strcat(wb, "CLABELS ");
925 + if(caps & STREAM_CAP_COMPRESSION) buffer_strcat(wb, "COMPRESSION ");
926 + if(caps & STREAM_CAP_FUNCTIONS) buffer_strcat(wb, "FUNCTIONS ");
927 + if(caps & STREAM_CAP_GAP_FILLING) buffer_strcat(wb, "GAP_FILLING ");
928 +}
929 +
930 +void log_receiver_capabilities(struct receiver_state *rpt) {
931 + BUFFER *wb = buffer_create(100);
932 + stream_capabilities_to_string(wb, rpt->capabilities);
933 +
934 + info("STREAM %s [receive from [%s]:%s]: established link with negotiated capabilities: %s",
935 + rrdhost_hostname(rpt->host), rpt->client_ip, rpt->client_port, buffer_tostring(wb));
936 +
937 + buffer_free(wb);
938 +}
939 +
940 +void log_sender_capabilities(struct sender_state *s) {
941 + BUFFER *wb = buffer_create(100);
942 + stream_capabilities_to_string(wb, s->capabilities);
943 +
944 + info("STREAM %s [send to %s]: established link with negotiated capabilities: %s",
945 + rrdhost_hostname(s->host), s->connected_to, buffer_tostring(wb));
946 +
947 + buffer_free(wb);
948 +}
949 +
950 +STREAM_CAPABILITIES convert_stream_version_to_capabilities(int32_t version) {
951 + STREAM_CAPABILITIES caps = 0;
952 +
953 + if(version <= 1) caps = STREAM_CAP_V1;
954 + else if(version < STREAM_OLD_VERSION_CLAIM) caps = STREAM_CAP_V2 | STREAM_CAP_HLABELS;
955 + else if(version <= STREAM_OLD_VERSION_CLAIM) caps = STREAM_CAP_VN | STREAM_CAP_HLABELS | STREAM_CAP_CLAIM;
956 + else if(version <= STREAM_OLD_VERSION_CLABELS) caps = STREAM_CAP_VN | STREAM_CAP_HLABELS | STREAM_CAP_CLAIM | STREAM_CAP_CLABELS;
957 + else if(version <= STREAM_OLD_VERSION_COMPRESSION) caps = STREAM_CAP_VN | STREAM_CAP_HLABELS | STREAM_CAP_CLAIM | STREAM_CAP_CLABELS | STREAM_HAS_COMPRESSION;
958 + else caps = version;
959 +
960 + if(caps & STREAM_CAP_VCAPS)
961 + caps &= ~(STREAM_CAP_V1|STREAM_CAP_V2|STREAM_CAP_VN);
962 +
963 + if(caps & STREAM_CAP_VN)
964 + caps &= ~(STREAM_CAP_V1|STREAM_CAP_V2);
965 +
966 + if(caps & STREAM_CAP_V2)
967 + caps &= ~(STREAM_CAP_V1);
968 +
969 + return caps & STREAM_OUR_CAPABILITIES;
970 +}
971 +
972 +int32_t stream_capabilities_to_vn(uint32_t caps) {
973 + if(caps & STREAM_CAP_COMPRESSION) return STREAM_OLD_VERSION_COMPRESSION;
974 + if(caps & STREAM_CAP_CLABELS) return STREAM_OLD_VERSION_CLABELS;
975 + return STREAM_OLD_VERSION_CLAIM; // if(caps & STREAM_CAP_CLAIM)
976 +}
977 +
streaming/rrdpush.h
+101 -33
@@ -10,32 +10,79 @@
10
11 #define CONNECTED_TO_SIZE 100
12
13 -#define STREAM_VERSION_CLAIM 3
14 -#define STREAM_VERSION_CLABELS 4
15 -#define STREAM_VERSION_COMPRESSION 5
16 -#define VERSION_GAP_FILLING 6
13 +// ----------------------------------------------------------------------------
14 +// obsolete versions - do not use anymore
15 +
16 +#define STREAM_OLD_VERSION_CLAIM 3
17 +#define STREAM_OLD_VERSION_CLABELS 4
18 +#define STREAM_OLD_VERSION_COMPRESSION 5 // this is production
19 +
20 +// ----------------------------------------------------------------------------
21 +// capabilities negotiation
22 +
23 +typedef enum {
24 + // do not use the first 3 bits
25 + STREAM_CAP_V1 = (1 << 3), // v1 = the oldest protocol
26 + STREAM_CAP_V2 = (1 << 4), // v2 = the second version of the protocol (with host labels)
27 + STREAM_CAP_VN = (1 << 5), // version negotiation supported (for versions 3, 4, 5 of the protocol)
28 + // v3 = claiming supported
29 + // v4 = chart labels supported
30 + // v5 = lz4 compression supported
31 + STREAM_CAP_VCAPS = (1 << 6), // capabilities negotiation supported
32 + STREAM_CAP_HLABELS = (1 << 7), // host labels supported
33 + STREAM_CAP_CLAIM = (1 << 8), // claiming supported
34 + STREAM_CAP_CLABELS = (1 << 9), // chart labels supported
35 + STREAM_CAP_COMPRESSION = (1 << 10), // lz4 compression supported
36 + STREAM_CAP_FUNCTIONS = (1 << 11), // plugin functions supported
37 + STREAM_CAP_GAP_FILLING = (1 << 12), // gap filling supported
38 +
39 + // this must be signed int, so don't use the last bit
40 + // needed for negotiating errors between parent and child
41 +} STREAM_CAPABILITIES;
42
43 #ifdef ENABLE_COMPRESSION
19 -#define STREAMING_PROTOCOL_CURRENT_VERSION (uint32_t)(STREAM_VERSION_COMPRESSION)
44 +#define STREAM_HAS_COMPRESSION STREAM_CAP_COMPRESSION
45 #else
21 -#define STREAMING_PROTOCOL_CURRENT_VERSION (uint32_t)(STREAM_VERSION_CLABELS)
46 +#define STREAM_HAS_COMPRESSION 0
47 #endif //ENABLE_COMPRESSION
48
49 +#define STREAM_OUR_CAPABILITIES (STREAM_CAP_V1 | STREAM_CAP_V2 | STREAM_CAP_VN | STREAM_CAP_VCAPS | STREAM_CAP_HLABELS | STREAM_CAP_CLAIM | STREAM_CAP_CLABELS | STREAM_HAS_COMPRESSION | STREAM_CAP_FUNCTIONS)
50 +
51 +#define stream_has_capability(rpt, capability) ((rpt) && ((rpt)->capabilities & (capability)))
52 +
53 +// ----------------------------------------------------------------------------
54 +// stream handshake
55 +
56 +#define HTTP_HEADER_SIZE 8192
57 +
58 #define STREAMING_PROTOCOL_VERSION "1.1"
25 -#define START_STREAMING_PROMPT "Hit me baby, push them over..."
26 -#define START_STREAMING_PROMPT_V2 "Hit me baby, push them over and bring the host labels..."
59 +#define START_STREAMING_PROMPT_V1 "Hit me baby, push them over..."
60 +#define START_STREAMING_PROMPT_V2 "Hit me baby, push them over and bring the host labels..."
61 #define START_STREAMING_PROMPT_VN "Hit me baby, push them over with the version="
62
63 #define START_STREAMING_ERROR_SAME_LOCALHOST "Don't hit me baby, you are trying to stream my localhost back"
64 #define START_STREAMING_ERROR_ALREADY_STREAMING "This GUID is already streaming to this server"
65 #define START_STREAMING_ERROR_NOT_PERMITTED "You are not permitted to access this. Check the logs for more info."
66
33 -#define HTTP_HEADER_SIZE 8192
34 -
67 typedef enum {
36 - RRDPUSH_MULTIPLE_CONNECTIONS_ALLOW,
37 - RRDPUSH_MULTIPLE_CONNECTIONS_DENY_NEW
38 -} RRDPUSH_MULTIPLE_CONNECTIONS_STRATEGY;
68 + STREAM_HANDSHAKE_OK_V5 = 5, // COMPRESSION
69 + STREAM_HANDSHAKE_OK_V4 = 4, // CLABELS
70 + STREAM_HANDSHAKE_OK_V3 = 3, // CLAIM
71 + STREAM_HANDSHAKE_OK_V2 = 2, // HLABELS
72 + STREAM_HANDSHAKE_OK_V1 = 1,
73 + STREAM_HANDSHAKE_ERROR_BAD_HANDSHAKE = -1,
74 + STREAM_HANDSHAKE_ERROR_LOCALHOST = -2,
75 + STREAM_HANDSHAKE_ERROR_ALREADY_CONNECTED = -3,
76 + STREAM_HANDSHAKE_ERROR_DENIED = -4,
77 + STREAM_HANDSHAKE_ERROR_SEND_TIMEOUT = -5,
78 + STREAM_HANDSHAKE_ERROR_RECEIVE_TIMEOUT = -6,
79 + STREAM_HANDSHAKE_ERROR_INVALID_CERTIFICATE = -7,
80 + STREAM_HANDSHAKE_ERROR_SSL_ERROR = -8,
81 + STREAM_HANDSHAKE_ERROR_CANT_CONNECT = -9
82 +} STREAM_HANDSHAKE;
83 +
84 +
85 +// ----------------------------------------------------------------------------
86
87 typedef struct {
88 char *os_name;
@@ -46,7 +93,6 @@ typedef struct {
93 } stream_encoded_t;
94
95 #ifdef ENABLE_COMPRESSION
49 -#define LZ4_MAX_MSG_SIZE 0x4000
96 struct compressor_state {
97 char *compression_result_buffer;
98 size_t compression_result_buffer_size;
@@ -81,11 +127,17 @@ struct decompressor_state {
127 // Thread-local storage
128 // Metric transmission: collector threads asynchronously fill the buffer, sender thread uses it.
129
130 +typedef enum {
131 + SENDER_FLAG_OVERFLOW = (1 << 0), // The buffer has been overflown
132 + SENDER_FLAG_COMPRESSION = (1 << 1), // The stream needs to have and has compression
133 +} SENDER_FLAGS;
134 +
135 struct sender_state {
136 RRDHOST *host;
86 - pid_t task_id;
87 - unsigned int overflow:1;
88 - int timeout, default_port;
137 + pid_t tid; // the thread id of the sender, from gettid()
138 + SENDER_FLAGS flags;
139 + int timeout;
140 + int default_port;
141 usec_t reconnect_delay;
142 char connected_to[CONNECTED_TO_SIZE + 1]; // We don't know which proxy we connect to, passed back from socket.c
143 size_t begin;
@@ -99,14 +151,19 @@ struct sender_state {
151 // the lazy creation of the sender thread - both cases (buffer access and thread creation) are guarded here.
152 netdata_mutex_t mutex;
153 struct circular_buffer *buffer;
102 - BUFFER *build;
103 - char read_buffer[512];
154 + char read_buffer[PLUGINSD_LINE_MAX + 1];
155 int read_len;
105 - int32_t version;
156 + STREAM_CAPABILITIES capabilities;
157 +
158 + int rrdpush_sender_pipe[2]; // collector to sender thread signaling
159 + int rrdpush_sender_socket;
160 +
161 #ifdef ENABLE_COMPRESSION
107 - unsigned int rrdpush_compression;
162 struct compressor_state *compressor;
163 #endif
164 +#ifdef ENABLE_HTTPS
165 + struct netdata_ssl ssl; // Structure used to encrypt the connection
166 +#endif
167 };
168
169 struct receiver_state {
@@ -128,9 +185,9 @@ struct receiver_state {
185 char *program_version;
186 struct rrdhost_system_info *system_info;
187 int update_every;
131 - uint32_t stream_version;
188 + STREAM_CAPABILITIES capabilities;
189 time_t last_msg_t;
133 - char read_buffer[1024]; // Need to allow RRD_ID_LENGTH_MAX * 4 + the other fields
190 + char read_buffer[PLUGINSD_LINE_MAX + 1];
191 int read_len;
192 unsigned int shutdown:1; // Tell the thread to exit
193 unsigned int exited; // Indicates that the thread has exited (NOT A BITFIELD!)
@@ -144,11 +201,13 @@ struct receiver_state {
201 };
202
203 struct rrdpush_destinations {
147 - char destination[CONNECTED_TO_SIZE + 1];
148 - int disabled_no_proper_reply;
149 - int disabled_because_of_localhost;
150 - time_t disabled_already_streaming;
151 - int disabled_because_of_denied_access;
204 + STRING *destination;
205 +
206 + const char *last_error;
207 + time_t postpone_reconnection_until;
208 + STREAM_HANDSHAKE last_handshake;
209 +
210 + struct rrdpush_destinations *prev;
211 struct rrdpush_destinations *next;
212 };
213
@@ -161,10 +220,12 @@ extern char *default_rrdpush_api_key;
220 extern char *default_rrdpush_send_charts_matching;
221 extern unsigned int remote_clock_resync_iterations;
222
223 +extern void rrdpush_destinations_init(RRDHOST *host);
224 +extern void rrdpush_destinations_free(RRDHOST *host);
225 +
226 extern void sender_init(RRDHOST *parent);
165 -extern struct rrdpush_destinations *destinations_init(const char *destinations);
166 -void sender_start(struct sender_state *s);
167 -void sender_commit(struct sender_state *s);
227 +BUFFER *sender_start(struct sender_state *s);
228 +void sender_commit(struct sender_state *s, BUFFER *wb);
229 void sender_cancel(struct sender_state *s);
230 extern int rrdpush_init();
231 extern int configured_as_parent();
@@ -172,7 +233,7 @@ extern void rrdset_done_push(RRDSET *st);
233 extern bool rrdset_push_chart_definition_now(RRDSET *st);
234 extern bool rrdpush_incremental_transmission_of_chart_definitions(RRDHOST *host, DICTFE *dictfe, bool restart, bool stop);
235 extern void *rrdpush_sender_thread(void *ptr);
175 -extern void rrdpush_send_labels(RRDHOST *host);
236 +extern void rrdpush_send_host_labels(RRDHOST *host);
237 extern void rrdpush_claimed_id(RRDHOST *host);
238
239 extern int rrdpush_receiver_thread_spawn(struct web_client *w, char *url);
@@ -181,7 +242,7 @@ extern void rrdpush_sender_thread_stop(RRDHOST *host);
242 extern void rrdpush_sender_send_this_host_variable_now(RRDHOST *host, const RRDVAR_ACQUIRED *rva);
243 extern void log_stream_connection(const char *client_ip, const char *client_port, const char *api_key, const char *machine_guid, const char *host, const char *msg);
244 extern int connect_to_one_of_destinations(
184 - struct rrdpush_destinations *destinations,
245 + RRDHOST *host,
246 int default_port,
247 struct timeval *timeout,
248 size_t *reconnects_counter,
@@ -189,10 +250,17 @@ extern int connect_to_one_of_destinations(
250 size_t connected_to_size,
251 struct rrdpush_destinations **destination);
252
253 +extern void rrdpush_signal_sender_to_wake_up(struct sender_state *s);
254 +
255 #ifdef ENABLE_COMPRESSION
256 struct compressor_state *create_compressor();
257 struct decompressor_state *create_decompressor();
258 size_t is_compressed_data(const char *data, size_t data_size);
259 #endif
260
261 +extern void log_receiver_capabilities(struct receiver_state *rpt);
262 +extern void log_sender_capabilities(struct sender_state *s);
263 +extern STREAM_CAPABILITIES convert_stream_version_to_capabilities(int32_t version);
264 +extern int32_t stream_capabilities_to_vn(uint32_t caps);
265 +
266 #endif //NETDATA_RRDPUSH_H
streaming/sender.c
+651 -367
@@ -1,6 +1,7 @@
1 // SPDX-License-Identifier: GPL-3.0-or-later
2
3 #include "rrdpush.h"
4 +#include "parser/parser.h"
5
6 #define WORKER_SENDER_JOB_CONNECT 0
7 #define WORKER_SENDER_JOB_PIPE_READ 1
@@ -30,15 +31,31 @@ extern int netdata_use_ssl_on_stream;
31 extern char *netdata_ssl_ca_path;
32 extern char *netdata_ssl_ca_file;
33
34 +static __thread BUFFER *sender_thread_buffer = NULL;
35 +static __thread bool sender_thread_buffer_used = false;
36 +
37 +void sender_thread_buffer_free(void) {
38 + if(sender_thread_buffer) {
39 + buffer_free(sender_thread_buffer);
40 + sender_thread_buffer = NULL;
41 + }
42 +}
43 +
44 // Collector thread starting a transmission
34 -void sender_start(struct sender_state *s) {
35 - netdata_mutex_lock(&s->mutex);
36 - buffer_flush(s->build);
45 +BUFFER *sender_start(struct sender_state *s __maybe_unused) {
46 + if(!sender_thread_buffer)
47 + sender_thread_buffer = buffer_create(1024);
48 +
49 + if(sender_thread_buffer_used)
50 + fatal("STREAMING: thread buffer is used multiple times concurrently.");
51 +
52 + sender_thread_buffer_used = true;
53 + buffer_flush(sender_thread_buffer);
54 + return sender_thread_buffer;
55 }
56
39 -void sender_cancel(struct sender_state *s) {
40 - buffer_flush(s->build);
41 - netdata_mutex_unlock(&s->mutex);
57 +void sender_cancel(struct sender_state *s __maybe_unused) {
58 + sender_thread_buffer_used = false;
59 }
60
61 static inline void rrdpush_sender_thread_close_socket(RRDHOST *host);
@@ -52,81 +69,101 @@ static inline void rrdpush_sender_thread_close_socket(RRDHOST *host);
69 static inline void deactivate_compression(struct sender_state *s) {
70 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_NO_COMPRESSION);
71 error("STREAM_COMPRESSION: Compression returned error, disabling it.");
55 - default_compression_enabled = 0;
56 - s->rrdpush_compression = 0;
57 - s->version = STREAM_VERSION_CLABELS;
72 + s->flags &= ~SENDER_FLAG_COMPRESSION;
73 error("STREAM %s [send to %s]: Restarting connection without compression.", rrdhost_hostname(s->host), s->connected_to);
74 rrdpush_sender_thread_close_socket(s->host);
75 }
76 #endif
77
78 // Collector thread finishing a transmission
64 -void sender_commit(struct sender_state *s) {
65 - char *src = (char *)buffer_tostring(s->host->sender->build);
66 - size_t src_len = s->host->sender->build->len;
79 +void sender_commit(struct sender_state *s, BUFFER *wb) {
80 +
81 + if(unlikely(wb != sender_thread_buffer))
82 + fatal("STREAMING: sender is trying to commit a buffer that is not this thread's buffer.");
83 +
84 + if(unlikely(!sender_thread_buffer_used))
85 + fatal("STREAMING: sender is committing a buffer twice.");
86 +
87 + sender_thread_buffer_used = false;
88 +
89 + char *src = (char *)buffer_tostring(wb);
90 + size_t src_len = buffer_strlen(wb);
91 +
92 + if(unlikely(!src || !src_len))
93 + return;
94 +
95 + netdata_mutex_lock(&s->mutex);
96 +
97 #ifdef ENABLE_COMPRESSION
68 - if (src && src_len) {
69 - if (s->compressor && s->rrdpush_compression) {
70 - while(src_len) {
71 - size_t size_to_compress = src_len;
72 -
73 - if(size_to_compress > LZ4_MAX_MSG_SIZE) {
74 - // we need to find the last newline
75 - // so that the decompressor will have a whole line to work with
76 -
77 - const char *t = &src[LZ4_MAX_MSG_SIZE - 1];
78 - while(t-- > src)
79 - if(*t == '\n')
80 - break;
81 -
82 - if(t == src)
83 - size_to_compress = LZ4_MAX_MSG_SIZE;
84 - else
85 - size_to_compress = t - src + 1;
86 - }
98 + if (s->flags & SENDER_FLAG_COMPRESSION && s->compressor) {
99 + while(src_len) {
100 + size_t size_to_compress = src_len;
101 +
102 + if(size_to_compress > COMPRESSION_MAX_MSG_SIZE) {
103 + // we need to find the last newline
104 + // so that the decompressor will have a whole line to work with
105 +
106 + const char *t = &src[COMPRESSION_MAX_MSG_SIZE - 1];
107 + while(t-- > src)
108 + if(*t == '\n')
109 + break;
110 +
111 + if(t == src)
112 + size_to_compress = COMPRESSION_MAX_MSG_SIZE;
113 + else
114 + size_to_compress = t - src + 1;
115 + }
116 +
117 + char *dst;
118 + size_t dst_len = s->compressor->compress(s->compressor, src, size_to_compress, &dst);
119 + if (!dst_len) {
120 + error("STREAM %s [send to %s]: compression failed. Resetting compressor and re-trying",
121 + rrdhost_hostname(s->host), s->connected_to);
122 +
123 + s->compressor->reset(s->compressor);
124 + dst_len = s->compressor->compress(s->compressor, src, size_to_compress, &dst);
125 + if(!dst_len) {
126 + error("STREAM %s [send to %s]: compression failed again. Deactivating compression",
127 + rrdhost_hostname(s->host), s->connected_to);
128
88 - char *dst;
89 - size_t dst_len = s->compressor->compress(s->compressor, src, size_to_compress, &dst);
90 - if (!dst_len) {
129 deactivate_compression(s);
92 - buffer_flush(s->build);
130 netdata_mutex_unlock(&s->mutex);
131 return;
132 }
133 + }
134
97 - if(cbuffer_add_unsafe(s->host->sender->buffer, dst, dst_len))
98 - s->overflow = 1;
135 + if(cbuffer_add_unsafe(s->host->sender->buffer, dst, dst_len))
136 + s->flags |= SENDER_FLAG_OVERFLOW;
137
100 - src = src + size_to_compress;
101 - src_len -= size_to_compress;
102 - }
138 + src = src + size_to_compress;
139 + src_len -= size_to_compress;
140 }
104 - else if(cbuffer_add_unsafe(s->host->sender->buffer, src, src_len))
105 - s->overflow = 1;
141 }
142 + else if(cbuffer_add_unsafe(s->host->sender->buffer, src, src_len))
143 + s->flags |= SENDER_FLAG_OVERFLOW;
144 #else
145 if(cbuffer_add_unsafe(s->host->sender->buffer, src, src_len))
109 - s->overflow = 1;
146 + s->flags |= SENDER_FLAG_OVERFLOW;
147 #endif
148
112 - buffer_flush(s->build);
149 netdata_mutex_unlock(&s->mutex);
150 + rrdpush_signal_sender_to_wake_up(s);
151 }
152
153
154 static inline void rrdpush_sender_thread_close_socket(RRDHOST *host) {
118 - rrdhost_flag_clear(host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS);
119 - __atomic_clear(&host->rrdpush_sender_connected, __ATOMIC_SEQ_CST);
155 + rrdhost_flag_clear(host, RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS);
156 + rrdhost_flag_clear(host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED);
157
121 - if(host->rrdpush_sender_socket != -1) {
122 - close(host->rrdpush_sender_socket);
123 - host->rrdpush_sender_socket = -1;
158 + if(host->sender->rrdpush_sender_socket != -1) {
159 + close(host->sender->rrdpush_sender_socket);
160 + host->sender->rrdpush_sender_socket = -1;
161 }
162 }
163
127 -static inline void rrdpush_sender_add_host_variable_to_buffer_nolock(RRDHOST *host, const RRDVAR_ACQUIRED *rva) {
164 +static inline void rrdpush_sender_add_host_variable_to_buffer(BUFFER *wb, const RRDVAR_ACQUIRED *rva) {
165 buffer_sprintf(
129 - host->sender->build
166 + wb
167 , "VARIABLE HOST %s = " NETDATA_DOUBLE_FORMAT "\n"
168 , rrdvar_name(rva)
169 , rrdvar2number(rva)
@@ -136,36 +173,41 @@ static inline void rrdpush_sender_add_host_variable_to_buffer_nolock(RRDHOST *ho
173 }
174
175 void rrdpush_sender_send_this_host_variable_now(RRDHOST *host, const RRDVAR_ACQUIRED *rva) {
139 - if(host->rrdpush_send_enabled && host->rrdpush_sender_spawn && __atomic_load_n(&host->rrdpush_sender_connected, __ATOMIC_SEQ_CST)) {
140 - sender_start(host->sender);
141 - rrdpush_sender_add_host_variable_to_buffer_nolock(host, rva);
142 - sender_commit(host->sender);
176 + if(rrdhost_can_send_definitions_to_parent(host)) {
177 + BUFFER *wb = sender_start(host->sender);
178 + rrdpush_sender_add_host_variable_to_buffer(wb, rva);
179 + sender_commit(host->sender, wb);
180 }
181 }
182
183 +struct custom_host_variables_callback {
184 + BUFFER *wb;
185 +};
186
147 -static int rrdpush_sender_thread_custom_host_variables_callback(const DICTIONARY_ITEM *item __maybe_unused, void *rrdvar_ptr __maybe_unused, void *host_ptr) {
187 +static int rrdpush_sender_thread_custom_host_variables_callback(const DICTIONARY_ITEM *item __maybe_unused, void *rrdvar_ptr __maybe_unused, void *struct_ptr) {
188 const RRDVAR_ACQUIRED *rv = (const RRDVAR_ACQUIRED *)item;
149 - RRDHOST *host = (RRDHOST *)host_ptr;
189 + struct custom_host_variables_callback *tmp = struct_ptr;
190 + BUFFER *wb = tmp->wb;
191
192 if(unlikely(rrdvar_flags(rv) & RRDVAR_FLAG_CUSTOM_HOST_VAR && rrdvar_type(rv) == RRDVAR_TYPE_CALCULATED)) {
152 - rrdpush_sender_add_host_variable_to_buffer_nolock(host, rv);
153 -
154 - // return 1, so that the traversal will return the number of variables sent
193 + rrdpush_sender_add_host_variable_to_buffer(wb, rv);
194 return 1;
195 }
157 -
158 - // returning a negative number will break the traversal
196 return 0;
197 }
198
199 static void rrdpush_sender_thread_send_custom_host_variables(RRDHOST *host) {
163 - sender_start(host->sender);
164 - int ret = rrdvar_walkthrough_read(host->rrdvars, rrdpush_sender_thread_custom_host_variables_callback, host);
165 - (void)ret;
166 - sender_commit(host->sender);
167 -
168 - debug(D_STREAM, "RRDVAR sent %d VARIABLES", ret);
200 + if(rrdhost_can_send_definitions_to_parent(host)) {
201 + BUFFER *wb = sender_start(host->sender);
202 + struct custom_host_variables_callback tmp = {
203 + .wb = wb
204 + };
205 + int ret = rrdvar_walkthrough_read(host->rrdvars, rrdpush_sender_thread_custom_host_variables_callback, &tmp);
206 + (void)ret;
207 + sender_commit(host->sender, wb);
208 +
209 + debug(D_STREAM, "RRDVAR sent %d VARIABLES", ret);
210 + }
211 }
212
213 // resets all the chart, so that their definitions
@@ -187,23 +229,13 @@ static void rrdpush_sender_thread_reset_all_charts(RRDHOST *host) {
229
230 static inline void rrdpush_sender_thread_data_flush(RRDHOST *host) {
231 netdata_mutex_lock(&host->sender->mutex);
190 -
191 - size_t len = cbuffer_next_unsafe(host->sender->buffer, NULL);
192 - if (len)
193 - error("STREAM %s [send]: discarding %zu bytes of metrics already in the buffer.", rrdhost_hostname(host), len);
194 -
195 - cbuffer_remove_unsafe(host->sender->buffer, len);
232 + cbuffer_flush(host->sender->buffer);
233 netdata_mutex_unlock(&host->sender->mutex);
234
235 rrdpush_sender_thread_reset_all_charts(host);
236 rrdpush_sender_thread_send_custom_host_variables(host);
237 }
238
202 -static inline void rrdpush_set_flags_to_newest_stream(RRDHOST *host) {
203 - rrdhost_flag_set(host, RRDHOST_FLAG_STREAM_LABELS_UPDATE);
204 - rrdhost_flag_clear(host, RRDHOST_FLAG_STREAM_LABELS_STOP);
205 -}
206 -
239 void rrdpush_encode_variable(stream_encoded_t *se, RRDHOST *host)
240 {
241 se->os_name = (host->system_info->host_os_name)?url_encode(host->system_info->host_os_name):"";
@@ -231,52 +263,123 @@ void rrdpush_clean_encoded(stream_encoded_t *se)
263 freez(se->kernel_version);
264 }
265
234 -static inline long int parse_stream_version_for_errors(char *http)
235 -{
236 - if (!memcmp(http, START_STREAMING_ERROR_SAME_LOCALHOST, sizeof(START_STREAMING_ERROR_SAME_LOCALHOST)))
237 - return -2;
238 - else if (!memcmp(http, START_STREAMING_ERROR_ALREADY_STREAMING, sizeof(START_STREAMING_ERROR_ALREADY_STREAMING)))
239 - return -3;
240 - else if (!memcmp(http, START_STREAMING_ERROR_NOT_PERMITTED, sizeof(START_STREAMING_ERROR_NOT_PERMITTED)))
241 - return -4;
242 - else
243 - return -1;
244 -}
266 +struct {
267 + const char *response;
268 + size_t length;
269 + int32_t version;
270 + bool dynamic;
271 + const char *error;
272 + int worker_job_id;
273 + time_t postpone_reconnect_seconds;
274 +} stream_responses[] = {
275 + {
276 + .response = START_STREAMING_PROMPT_VN,
277 + .length = sizeof(START_STREAMING_PROMPT_VN) - 1,
278 + .version = STREAM_HANDSHAKE_OK_V3, // and above
279 + .dynamic = true, // dynamic = we will parse the version / capabilities
280 + .error = NULL,
281 + .worker_job_id = 0,
282 + .postpone_reconnect_seconds = 0,
283 + },
284 + {
285 + .response = START_STREAMING_PROMPT_V2,
286 + .length = sizeof(START_STREAMING_PROMPT_V2) - 1,
287 + .version = STREAM_HANDSHAKE_OK_V2,
288 + .dynamic = false,
289 + .error = NULL,
290 + .worker_job_id = 0,
291 + .postpone_reconnect_seconds = 0,
292 + },
293 + {
294 + .response = START_STREAMING_PROMPT_V1,
295 + .length = sizeof(START_STREAMING_PROMPT_V1) - 1,
296 + .version = STREAM_HANDSHAKE_OK_V1,
297 + .dynamic = false,
298 + .error = NULL,
299 + .worker_job_id = 0,
300 + .postpone_reconnect_seconds = 0,
301 + },
302 + {
303 + .response = START_STREAMING_ERROR_SAME_LOCALHOST,
304 + .length = sizeof(START_STREAMING_ERROR_SAME_LOCALHOST) - 1,
305 + .version = STREAM_HANDSHAKE_ERROR_LOCALHOST,
306 + .dynamic = false,
307 + .error = "remote server rejected this stream, the host we are trying to stream is its localhost",
308 + .worker_job_id = WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE,
309 + .postpone_reconnect_seconds = 60 * 60, // the IP may change, try it every hour
310 + },
311 + {
312 + .response = START_STREAMING_ERROR_ALREADY_STREAMING,
313 + .length = sizeof(START_STREAMING_ERROR_ALREADY_STREAMING) - 1,
314 + .version = STREAM_HANDSHAKE_ERROR_ALREADY_CONNECTED,
315 + .dynamic = false,
316 + .error = "remote server rejected this stream, the host we are trying to stream is already streamed to it",
317 + .worker_job_id = WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE,
318 + .postpone_reconnect_seconds = 1 * 60, // 1 minute
319 + },
320 + {
321 + .response = START_STREAMING_ERROR_NOT_PERMITTED,
322 + .length = sizeof(START_STREAMING_ERROR_NOT_PERMITTED) - 1,
323 + .version = STREAM_HANDSHAKE_ERROR_DENIED,
324 + .dynamic = false,
325 + .error = "remote server denied access, probably we don't have the right API key?",
326 + .worker_job_id = WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE,
327 + .postpone_reconnect_seconds = 1 * 60, // 1 minute
328 + },
329 +
330 + // terminator
331 + {
332 + .response = NULL,
333 + .length = 0,
334 + .version = STREAM_HANDSHAKE_ERROR_BAD_HANDSHAKE,
335 + .dynamic = false,
336 + .error = "remote node response is not understood, is it Netdata?",
337 + .worker_job_id = WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE,
338 + .postpone_reconnect_seconds = 1 * 60, // 1 minute
339 + }
340 +};
341
246 -static inline long int parse_stream_version(RRDHOST *host, char *http)
247 -{
248 - long int stream_version = -1;
249 - int answer = -1;
250 - char *stream_version_start = strchr(http, '=');
251 - if (stream_version_start) {
252 - stream_version_start++;
253 - stream_version = strtol(stream_version_start, NULL, 10);
254 - answer = memcmp(http, START_STREAMING_PROMPT_VN, (size_t)(stream_version_start - http));
255 - if (!answer) {
256 - rrdpush_set_flags_to_newest_stream(host);
342 +static inline bool rrdpush_sender_validate_response(RRDHOST *host, struct sender_state *s, char *http, size_t http_length) {
343 + int32_t version = STREAM_HANDSHAKE_ERROR_BAD_HANDSHAKE;
344 +
345 + int i;
346 + for(i = 0; stream_responses[i].response ; i++) {
347 + if(stream_responses[i].dynamic &&
348 + http_length > stream_responses[i].length && http_length < (stream_responses[i].length + 30) &&
349 + strncmp(http, stream_responses[i].response, stream_responses[i].length) == 0) {
350 +
351 + version = str2i(&http[stream_responses[i].length]);
352 + break;
353 }
258 - } else {
259 - answer = memcmp(http, START_STREAMING_PROMPT_V2, strlen(START_STREAMING_PROMPT_V2));
260 - if (!answer) {
261 - stream_version = 1;
262 - rrdpush_set_flags_to_newest_stream(host);
263 - } else {
264 - answer = memcmp(http, START_STREAMING_PROMPT, strlen(START_STREAMING_PROMPT));
265 - if (!answer) {
266 - stream_version = 0;
267 - rrdhost_flag_set(host, RRDHOST_FLAG_STREAM_LABELS_STOP);
268 - rrdhost_flag_clear(host, RRDHOST_FLAG_STREAM_LABELS_UPDATE);
269 - }
270 - else {
271 - stream_version = parse_stream_version_for_errors(http);
272 - }
354 + else if(http_length == stream_responses[i].length && strcmp(http, stream_responses[i].response) == 0) {
355 + version = stream_responses[i].version;
356 +
357 + break;
358 }
359 }
275 - return stream_version;
360 + const char *error = stream_responses[i].error;
361 + int worker_job_id = stream_responses[i].worker_job_id;
362 + time_t delay = stream_responses[i].postpone_reconnect_seconds;
363 +
364 + if(version >= STREAM_HANDSHAKE_OK_V1) {
365 + host->destination->last_error = NULL;
366 + host->destination->last_handshake = version;
367 + host->destination->postpone_reconnection_until = 0;
368 + s->capabilities = convert_stream_version_to_capabilities(version);
369 + return true;
370 + }
371 +
372 + error("STREAM %s [send to %s]: %s.", rrdhost_hostname(host), s->connected_to, error);
373 +
374 + worker_is_busy(worker_job_id);
375 + rrdpush_sender_thread_close_socket(host);
376 + host->destination->last_error = error;
377 + host->destination->last_handshake = version;
378 + host->destination->postpone_reconnection_until = now_realtime_sec() + delay;
379 + return false;
380 }
381
278 -static int rrdpush_sender_thread_connect_to_parent(RRDHOST *host, int default_port, int timeout,
279 - struct sender_state *s) {
382 +static bool rrdpush_sender_thread_connect_to_parent(RRDHOST *host, int default_port, int timeout, struct sender_state *s) {
383
384 struct timeval tv = {
385 .tv_sec = timeout,
@@ -286,11 +389,8 @@ static int rrdpush_sender_thread_connect_to_parent(RRDHOST *host, int default_po
389 // make sure the socket is closed
390 rrdpush_sender_thread_close_socket(host);
391
289 - debug(D_STREAM, "STREAM: Attempting to connect...");
290 - info("STREAM %s [send to %s]: connecting...", rrdhost_hostname(host), host->rrdpush_send_destination);
291 -
292 - host->rrdpush_sender_socket = connect_to_one_of_destinations(
293 - host->destinations
392 + s->rrdpush_sender_socket = connect_to_one_of_destinations(
393 + host
394 , default_port
395 , &tv
396 , &s->reconnects_counter
@@ -299,48 +399,50 @@ static int rrdpush_sender_thread_connect_to_parent(RRDHOST *host, int default_po
399 , &host->destination
400 );
401
302 - if(unlikely(host->rrdpush_sender_socket == -1)) {
303 - error("STREAM %s [send to %s]: failed to connect", rrdhost_hostname(host), host->rrdpush_send_destination);
304 - return 0;
402 + if(unlikely(s->rrdpush_sender_socket == -1)) {
403 + error("STREAM %s [send to %s]: could not connect to parent node at this time.", rrdhost_hostname(host), host->rrdpush_send_destination);
404 + return false;
405 }
406
407 info("STREAM %s [send to %s]: initializing communication...", rrdhost_hostname(host), s->connected_to);
408
409 #ifdef ENABLE_HTTPS
310 - if( netdata_client_ctx ){
311 - host->ssl.flags = NETDATA_SSL_START;
312 - if (!host->ssl.conn){
313 - host->ssl.conn = SSL_new(netdata_client_ctx);
314 - if(!host->ssl.conn){
410 + if(netdata_ssl_client_ctx){
411 + host->sender->ssl.flags = NETDATA_SSL_START;
412 + if (!host->sender->ssl.conn){
413 + host->sender->ssl.conn = SSL_new(netdata_ssl_client_ctx);
414 + if(!host->sender->ssl.conn){
415 error("Failed to allocate SSL structure.");
316 - host->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
416 + host->sender->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
417 }
418 }
419 else{
320 - SSL_clear(host->ssl.conn);
420 + SSL_clear(host->sender->ssl.conn);
421 }
422
323 - if (host->ssl.conn)
423 + if (host->sender->ssl.conn)
424 {
325 - if (SSL_set_fd(host->ssl.conn, host->rrdpush_sender_socket) != 1) {
326 - error("Failed to set the socket to the SSL on socket fd %d.", host->rrdpush_sender_socket);
327 - host->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
425 + if (SSL_set_fd(host->sender->ssl.conn, s->rrdpush_sender_socket) != 1) {
426 + error("Failed to set the socket to the SSL on socket fd %d.", s->rrdpush_sender_socket);
427 + host->sender->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
428 } else{
329 - host->ssl.flags = NETDATA_SSL_HANDSHAKE_COMPLETE;
429 + host->sender->ssl.flags = NETDATA_SSL_HANDSHAKE_COMPLETE;
430 }
431 }
432 }
433 else {
334 - host->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
434 + host->sender->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
435 }
436 #endif
437
438 + // reset our capabilities to default
439 + s->capabilities = STREAM_OUR_CAPABILITIES;
440 +
441 #ifdef ENABLE_COMPRESSION
339 -// Negotiate stream VERSION_CLABELS if stream compression is not supported
340 -s->rrdpush_compression = (default_compression_enabled && (s->version >= STREAM_VERSION_COMPRESSION));
341 -if(!s->rrdpush_compression)
342 - s->version = STREAM_VERSION_CLABELS;
343 -#endif //ENABLE_COMPRESSION
442 + // If we don't want compression, remove it from our capabilities
443 + if(!(s->flags & SENDER_FLAG_COMPRESSION) && stream_has_capability(s, STREAM_CAP_COMPRESSION))
444 + s->capabilities &= ~STREAM_CAP_COMPRESSION;
445 +#endif // ENABLE_COMPRESSION
446
447 /* TODO: During the implementation of #7265 switch the set of variables to HOST_* and CONTAINER_* if the
448 version negotiation resulted in a high enough version.
@@ -365,7 +467,7 @@ if(!s->rrdpush_compression)
467 "&ml_enabled=%d"
468 "&mc_version=%d"
469 "&tags=%s"
368 - "&ver=%d"
470 + "&ver=%u"
471 "&NETDATA_INSTANCE_CLOUD_TYPE=%s"
472 "&NETDATA_INSTANCE_CLOUD_INSTANCE_TYPE=%s"
473 "&NETDATA_INSTANCE_CLOUD_INSTANCE_REGION=%s"
@@ -411,7 +513,7 @@ if(!s->rrdpush_compression)
513 , host->system_info->ml_enabled
514 , host->system_info->mc_version
515 , rrdhost_tags(host)
414 - , s->version
516 + , s->capabilities
517 , (host->system_info->cloud_provider_type) ? host->system_info->cloud_provider_type : ""
518 , (host->system_info->cloud_instance_type) ? host->system_info->cloud_instance_type : ""
519 , (host->system_info->cloud_instance_region) ? host->system_info->cloud_instance_region : ""
@@ -447,131 +549,123 @@ if(!s->rrdpush_compression)
549 rrdpush_clean_encoded(&se);
550
551 #ifdef ENABLE_HTTPS
450 - if (!host->ssl.flags) {
552 + if (!host->sender->ssl.flags) {
553 ERR_clear_error();
452 - SSL_set_connect_state(host->ssl.conn);
453 - int err = SSL_connect(host->ssl.conn);
554 + SSL_set_connect_state(host->sender->ssl.conn);
555 + int err = SSL_connect(host->sender->ssl.conn);
556 if (err != 1){
455 - err = SSL_get_error(host->ssl.conn, err);
456 - error("SSL cannot connect with the server: %s ",ERR_error_string((long)SSL_get_error(host->ssl.conn,err),NULL));
557 + err = SSL_get_error(host->sender->ssl.conn, err);
558 + error("SSL cannot connect with the server: %s ",ERR_error_string((long)SSL_get_error(host->sender->ssl.conn,err),NULL));
559 if (netdata_use_ssl_on_stream == NETDATA_SSL_FORCE) {
560 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_SSL_ERROR);
561 rrdpush_sender_thread_close_socket(host);
460 - if (host->destination->next)
461 - host->destination->disabled_no_proper_reply = 1;
462 - return 0;
463 - }else {
464 - host->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
562 + host->destination->last_error = "SSL error";
563 + host->destination->last_handshake = STREAM_HANDSHAKE_ERROR_SSL_ERROR;
564 + host->destination->postpone_reconnection_until = now_realtime_sec() + 5 * 60;
565 + return false;
566 + }
567 + else {
568 + host->sender->ssl.flags = NETDATA_SSL_NO_HANDSHAKE;
569 }
570 }
571 else {
572 if (netdata_use_ssl_on_stream == NETDATA_SSL_FORCE) {
469 - if (netdata_validate_server == NETDATA_SSL_VALID_CERTIFICATE) {
470 - if ( security_test_certificate(host->ssl.conn)) {
573 + if (netdata_ssl_validate_server == NETDATA_SSL_VALID_CERTIFICATE) {
574 + if ( security_test_certificate(host->sender->ssl.conn)) {
575 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_SSL_ERROR);
576 error("Closing the stream connection, because the server SSL certificate is not valid.");
577 rrdpush_sender_thread_close_socket(host);
474 - if (host->destination->next)
475 - host->destination->disabled_no_proper_reply = 1;
476 - return 0;
578 + host->destination->last_error = "invalid SSL certificate";
579 + host->destination->last_handshake = STREAM_HANDSHAKE_ERROR_INVALID_CERTIFICATE;
580 + host->destination->postpone_reconnection_until = now_realtime_sec() + 5 * 60;
581 + return false;
582 }
583 }
584 }
585 }
586 }
482 - if(send_timeout(&host->ssl,host->rrdpush_sender_socket, http, strlen(http), 0, timeout) == -1) {
483 -#else
484 - if(send_timeout(host->rrdpush_sender_socket, http, strlen(http), 0, timeout) == -1) {
587 #endif
588 +
589 + ssize_t bytes;
590 +
591 + bytes = send_timeout(
592 +#ifdef ENABLE_HTTPS
593 + &host->sender->ssl,
594 +#endif
595 + s->rrdpush_sender_socket,
596 + http,
597 + strlen(http),
598 + 0,
599 + timeout);
600 +
601 + if(bytes <= 0) { // timeout is 0
602 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_TIMEOUT);
487 - error("STREAM %s [send to %s]: failed to send HTTP header to remote netdata.", rrdhost_hostname(host), s->connected_to);
603 rrdpush_sender_thread_close_socket(host);
489 - return 0;
604 + error("STREAM %s [send to %s]: failed to send HTTP header to remote netdata.", rrdhost_hostname(host), s->connected_to);
605 + host->destination->last_error = "timeout while sending request";
606 + host->destination->last_handshake = STREAM_HANDSHAKE_ERROR_SEND_TIMEOUT;
607 + host->destination->postpone_reconnection_until = now_realtime_sec() + 1 * 60;
608 + return false;
609 }
610
611 info("STREAM %s [send to %s]: waiting response from remote netdata...", rrdhost_hostname(host), s->connected_to);
612
494 - ssize_t received;
613 + bytes = recv_timeout(
614 #ifdef ENABLE_HTTPS
496 - received = recv_timeout(&host->ssl,host->rrdpush_sender_socket, http, HTTP_HEADER_SIZE, 0, timeout);
497 - if(received == -1) {
498 -#else
499 - received = recv_timeout(host->rrdpush_sender_socket, http, HTTP_HEADER_SIZE, 0, timeout);
500 - if(received == -1) {
615 + &host->sender->ssl,
616 #endif
617 + s->rrdpush_sender_socket,
618 + http,
619 + HTTP_HEADER_SIZE,
620 + 0,
621 + timeout);
622 +
623 + if(bytes <= 0) { // timeout is 0
624 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_TIMEOUT);
503 - error("STREAM %s [send to %s]: remote netdata does not respond.", rrdhost_hostname(host), s->connected_to);
625 rrdpush_sender_thread_close_socket(host);
505 - return 0;
626 + error("STREAM %s [send to %s]: remote netdata does not respond.", rrdhost_hostname(host), s->connected_to);
627 + host->destination->last_error = "timeout while expecting first response";
628 + host->destination->last_handshake = STREAM_HANDSHAKE_ERROR_RECEIVE_TIMEOUT;
629 + host->destination->postpone_reconnection_until = now_realtime_sec() + 30;
630 + return false;
631 }
632
508 - http[received] = '\0';
633 + http[bytes] = '\0';
634 debug(D_STREAM, "Response to sender from far end: %s", http);
510 - int32_t version = (int32_t)parse_stream_version(host, http);
511 - if(version == -1) {
512 - worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE);
513 - error("STREAM %s [send to %s]: server is not replying properly (is it a netdata?).", rrdhost_hostname(host), s->connected_to);
514 - rrdpush_sender_thread_close_socket(host);
515 - //catch other reject reasons and force to check other destinations
516 - if (host->destination->next)
517 - host->destination->disabled_no_proper_reply = 1;
518 - return 0;
519 - }
520 - else if(version == -2) {
521 - error("STREAM %s [send to %s]: remote server is the localhost for [%s].", rrdhost_hostname(host), s->connected_to, rrdhost_hostname(host));
522 - rrdpush_sender_thread_close_socket(host);
523 - host->destination->disabled_because_of_localhost = 1;
524 - return 0;
525 - }
526 - else if(version == -3) {
527 - error("STREAM %s [send to %s]: remote server already receives metrics for [%s].", rrdhost_hostname(host), s->connected_to, rrdhost_hostname(host));
528 - rrdpush_sender_thread_close_socket(host);
529 - host->destination->disabled_already_streaming = now_realtime_sec();
530 - return 0;
531 - }
532 - else if(version == -4) {
533 - error("STREAM %s [send to %s]: remote server denied access for [%s].", rrdhost_hostname(host), s->connected_to, rrdhost_hostname(host));
534 - rrdpush_sender_thread_close_socket(host);
535 - if (host->destination->next)
536 - host->destination->disabled_because_of_denied_access = 1;
537 - return 0;
538 - }
539 - s->version = version;
635 + if(!rrdpush_sender_validate_response(host, s, http, bytes))
636 + return false;
637
638 #ifdef ENABLE_COMPRESSION
542 - s->rrdpush_compression = (s->rrdpush_compression && (s->version >= STREAM_VERSION_COMPRESSION));
543 - if(s->rrdpush_compression)
544 - {
545 - // parent supports compression
639 + // if the stream does not have compression capability,
640 + // shut it down for us too.
641 + // FIXME - this means that if there are multiple parents and one of them does not support compression
642 + // we are going to shut it down for all of them eventually...
643 + if(!stream_has_capability(s, STREAM_CAP_COMPRESSION))
644 + s->flags &= ~SENDER_FLAG_COMPRESSION;
645 +
646 + if(s->flags & SENDER_FLAG_COMPRESSION) {
647 if(s->compressor)
648 s->compressor->reset(s->compressor);
649 }
549 - else {
550 - //parent does not support compression or has compression disabled
551 - debug(D_STREAM, "Stream is uncompressed! One of the agents (%s <-> %s) does not support compression OR compression is disabled.", s->connected_to, rrdhost_hostname(s->host));
552 - infoerr("Stream is uncompressed! One of the agents (%s <-> %s) does not support compression OR compression is disabled.", s->connected_to, rrdhost_hostname(s->host));
553 - s->version = STREAM_VERSION_CLABELS;
554 - }
555 -#endif //ENABLE_COMPRESSION
650 + else
651 + info("STREAM %s [send to %s]: compression is disabled on this connection.", rrdhost_hostname(host), s->connected_to);
652
653 +#endif //ENABLE_COMPRESSION
654
558 - info("STREAM %s [send to %s]: established communication with a parent using protocol version %d"
559 - , rrdhost_hostname(host)
560 - , s->connected_to
561 - , s->version);
655 + log_sender_capabilities(s);
656
563 - if(sock_setnonblock(host->rrdpush_sender_socket) < 0)
657 + if(sock_setnonblock(s->rrdpush_sender_socket) < 0)
658 error("STREAM %s [send to %s]: cannot set non-blocking mode for socket.", rrdhost_hostname(host), s->connected_to);
659
566 - if(sock_enlarge_out(host->rrdpush_sender_socket) < 0)
660 + if(sock_enlarge_out(s->rrdpush_sender_socket) < 0)
661 error("STREAM %s [send to %s]: cannot enlarge the socket buffer.", rrdhost_hostname(host), s->connected_to);
662
569 - debug(D_STREAM, "STREAM: Connected on fd %d...", host->rrdpush_sender_socket);
663 + debug(D_STREAM, "STREAM: Connected on fd %d...", s->rrdpush_sender_socket);
664
571 - return 1;
665 + return true;
666 }
667
574 -static void attempt_to_connect(struct sender_state *state)
668 +static bool attempt_to_connect(struct sender_state *state)
669 {
670 state->send_attempts = 0;
671
@@ -591,44 +685,46 @@ static void attempt_to_connect(struct sender_state *state)
685 state->sent_bytes_on_this_connection = 0;
686
687 // let the data collection threads know we are ready
594 - __atomic_test_and_set(&state->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST);
688 + rrdhost_flag_set(state->host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED);
689 +
690 + return true;
691 }
596 - else {
597 - // increase the failed connections counter
598 - state->not_connected_loops++;
692
600 - // reset the number of bytes sent
601 - state->sent_bytes_on_this_connection = 0;
693 + // we couldn't connect
694
603 - // slow re-connection on repeating errors
604 - sleep_usec(USEC_PER_SEC * state->reconnect_delay); // seconds
605 - }
695 + // increase the failed connections counter
696 + state->not_connected_loops++;
697 +
698 + // reset the number of bytes sent
699 + state->sent_bytes_on_this_connection = 0;
700 +
701 + // slow re-connection on repeating errors
702 + sleep_usec(USEC_PER_SEC * state->reconnect_delay); // seconds
703 +
704 + return false;
705 }
706
707 // TCP window is open and we have data to transmit.
708 static ssize_t attempt_to_send(struct sender_state *s) {
709 ssize_t ret = 0;
710
612 - rrdpush_send_labels(s->host);
613 -
711 #ifdef NETDATA_INTERNAL_CHECKS
712 struct circular_buffer *cb = s->buffer;
713 #endif
714
618 - netdata_thread_disable_cancelability();
715 netdata_mutex_lock(&s->mutex);
716 char *chunk;
717 size_t outstanding = cbuffer_next_unsafe(s->buffer, &chunk);
718 debug(D_STREAM, "STREAM: Sending data. Buffer r=%zu w=%zu s=%zu, next chunk=%zu", cb->read, cb->write, cb->size, outstanding);
719
720 #ifdef ENABLE_HTTPS
625 - SSL *conn = s->host->ssl.conn ;
626 - if(conn && !s->host->ssl.flags)
721 + SSL *conn = s->host->sender->ssl.conn ;
722 + if(conn && s->host->sender->ssl.flags == NETDATA_SSL_HANDSHAKE_COMPLETE)
723 ret = SSL_write(conn, chunk, outstanding);
724 else
629 - ret = send(s->host->rrdpush_sender_socket, chunk, outstanding, MSG_DONTWAIT);
725 + ret = send(s->rrdpush_sender_socket, chunk, outstanding, MSG_DONTWAIT);
726 #else
631 - ret = send(s->host->rrdpush_sender_socket, chunk, outstanding, MSG_DONTWAIT);
727 + ret = send(s->rrdpush_sender_socket, chunk, outstanding, MSG_DONTWAIT);
728 #endif
729
730 if (likely(ret > 0)) {
@@ -650,7 +746,6 @@ static ssize_t attempt_to_send(struct sender_state *s) {
746 debug(D_STREAM, "STREAM: send() returned 0 -> no error but no transmission");
747
748 netdata_mutex_unlock(&s->mutex);
653 - netdata_thread_enable_cancelability();
749
750 return ret;
751 }
@@ -659,15 +754,15 @@ static ssize_t attempt_read(struct sender_state *s) {
754 ssize_t ret = 0;
755
756 #ifdef ENABLE_HTTPS
662 - if (s->host->ssl.conn && !s->host->stream_ssl.flags) {
757 + if (s->host->sender->ssl.conn && s->host->sender->ssl.flags == NETDATA_SSL_HANDSHAKE_COMPLETE) {
758 ERR_clear_error();
759 int desired = sizeof(s->read_buffer) - s->read_len - 1;
665 - ret = SSL_read(s->host->ssl.conn, s->read_buffer, desired);
760 + ret = SSL_read(s->host->sender->ssl.conn, s->read_buffer, desired);
761 if (ret > 0 ) {
762 s->read_len += ret;
763 return ret;
764 }
670 - int sslerrno = SSL_get_error(s->host->ssl.conn, desired);
765 + int sslerrno = SSL_get_error(s->host->sender->ssl.conn, desired);
766 if (sslerrno == SSL_ERROR_WANT_READ || sslerrno == SSL_ERROR_WANT_WRITE)
767 return ret;
768
@@ -676,49 +771,127 @@ static ssize_t attempt_read(struct sender_state *s) {
771 char buf[256];
772 while ((err = ERR_get_error()) != 0) {
773 ERR_error_string_n(err, buf, sizeof(buf));
679 - error("STREAM %s [send to %s] ssl error: %s", rrdhost_hostname(s->host), s->connected_to, buf);
774 + error("STREAM %s [send to %s] SSL error: %s", rrdhost_hostname(s->host), s->connected_to, buf);
775 }
681 - error("Restarting connection");
776 rrdpush_sender_thread_close_socket(s->host);
777 return ret;
778 }
779 #endif
686 - ret = recv(s->host->rrdpush_sender_socket, s->read_buffer + s->read_len, sizeof(s->read_buffer) - s->read_len - 1,MSG_DONTWAIT);
780 + ret = recv(s->rrdpush_sender_socket, s->read_buffer + s->read_len, sizeof(s->read_buffer) - s->read_len - 1,MSG_DONTWAIT);
781 if (ret > 0) {
782 s->read_len += ret;
783 return ret;
784 }
785
692 - debug(D_STREAM, "Socket was POLLIN, but req %zu bytes gave %zd", sizeof(s->read_buffer) - s->read_len - 1, ret);
693 -
786 if (ret < 0 && (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR))
787 return ret;
788
697 - if (ret == 0) {
789 + if (ret == 0 || errno == ECONNRESET) {
790 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_PARENT_CLOSED);
699 - error("STREAM %s [send to %s]: connection closed by far end. Restarting connection", rrdhost_hostname(s->host), s->connected_to);
791 + error("STREAM %s [send to %s]: connection closed by far end.", rrdhost_hostname(s->host), s->connected_to);
792 }
793 else {
794 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_RECEIVE_ERROR);
703 - error("STREAM %s [send to %s]: error during receive (%zd). Restarting connection", rrdhost_hostname(s->host), s->connected_to, ret);
795 + error("STREAM %s [send to %s]: error during receive (%zd) - closing connection.", rrdhost_hostname(s->host), s->connected_to, ret);
796 }
797 rrdpush_sender_thread_close_socket(s->host);
798
799 return ret;
800 }
801
802 +struct inflight_stream_function {
803 + struct sender_state *sender;
804 + STRING *transaction;
805 + usec_t received_ut;
806 +};
807 +
808 +void stream_execute_function_callback(BUFFER *func_wb, int code, void *data) {
809 + struct inflight_stream_function *tmp = data;
810 +
811 + struct sender_state *s = tmp->sender;
812 +
813 + if(rrdhost_can_send_definitions_to_parent(s->host)) {
814 + BUFFER *wb = sender_start(s);
815 +
816 + pluginsd_function_result_begin_to_buffer(wb
817 + , string2str(tmp->transaction)
818 + , code
819 + , functions_content_type_to_format(func_wb->contenttype)
820 + , func_wb->expires);
821 +
822 + buffer_fast_strcat(wb, buffer_tostring(func_wb), buffer_strlen(func_wb));
823 + pluginsd_function_result_end_to_buffer(wb);
824 +
825 + sender_commit(s, wb);
826 +
827 + internal_error(true, "STREAM %s [send to %s] FUNCTION transaction %s sending back response (%zu bytes, %llu usec).",
828 + rrdhost_hostname(s->host), s->connected_to,
829 + string2str(tmp->transaction),
830 + buffer_strlen(func_wb),
831 + now_realtime_usec() - tmp->received_ut);
832 + }
833 + string_freez(tmp->transaction);
834 + buffer_free(func_wb);
835 + freez(tmp);
836 +}
837 +
838 // This is just a placeholder until the gap filling state machine is inserted
839 void execute_commands(struct sender_state *s) {
840 char *start = s->read_buffer, *end = &s->read_buffer[s->read_len], *newline;
841 *end = 0;
714 - while( start<end && (newline=strchr(start, '\n')) ) {
715 - *newline = 0;
716 - info("STREAM %s [send to %s] received command over connection: %s", rrdhost_hostname(s->host), s->connected_to, start);
717 - start = newline+1;
842 + while( start < end && (newline = strchr(start, '\n')) ) {
843 + *newline = '\0';
844 +
845 + log_access("STREAM: %d from '%s' for host '%s': %s",
846 + gettid(), s->connected_to, rrdhost_hostname(s->host), start);
847 +
848 + internal_error(true, "STREAM %s [send to %s] received command over connection: %s", rrdhost_hostname(s->host), s->connected_to, start);
849 +
850 + char *words[PLUGINSD_MAX_WORDS] = { NULL };
851 + pluginsd_split_words(start, words, PLUGINSD_MAX_WORDS, NULL, NULL, 0);
852 +
853 + if(words[0] && strcmp(words[0], PLUGINSD_KEYWORD_FUNCTION) == 0) {
854 + char *transaction = words[1];
855 + char *timeout_s = words[2];
856 + char *function = words[3];
857 +
858 + if(!transaction || !*transaction || !timeout_s || !*timeout_s || !function || !*function) {
859 + error("STREAM %s [send to %s] %s execution command is incomplete (transaction = '%s', timeout = '%s', function = '%s'). Ignoring it.",
860 + rrdhost_hostname(s->host), s->connected_to,
861 + words[0],
862 + transaction?transaction:"(unset)",
863 + timeout_s?timeout_s:"(unset)",
864 + function?function:"(unset)");
865 + }
866 + else {
867 + int timeout = str2i(timeout_s);
868 + if(timeout <= 0) timeout = PLUGINS_FUNCTIONS_TIMEOUT_DEFAULT;
869 +
870 + struct inflight_stream_function *tmp = callocz(1, sizeof(struct inflight_stream_function));
871 + tmp->received_ut = now_realtime_usec();
872 + tmp->sender = s;
873 + tmp->transaction = string_strdupz(transaction);
874 + BUFFER *wb = buffer_create(PLUGINSD_LINE_MAX + 1);
875 +
876 + int code = rrd_call_function_async(s->host, wb, timeout, function, stream_execute_function_callback, tmp);
877 + if(code != HTTP_RESP_OK) {
878 + rrd_call_function_error(wb, "Failed to route request to collector", code);
879 + stream_execute_function_callback(wb, code, tmp);
880 + }
881 + }
882 + }
883 + else
884 + error("STREAM %s [send to %s] received unknown command over connection: %s", rrdhost_hostname(s->host), s->connected_to, words[0]?words[0]:"(unset)");
885 +
886 + start = newline + 1;
887 }
719 - if (start<end) {
888 + if (start < end) {
889 memmove(s->read_buffer, start, end-start);
721 - s->read_len = end-start;
890 + s->read_len = end - start;
891 + }
892 + else {
893 + s->read_buffer[0] = '\0';
894 + s->read_len = 0;
895 }
896 }
897
@@ -731,6 +904,7 @@ struct rrdpush_sender_thread_data {
904 SENDING_DEFINITIONS_CONTINUE,
905 SENDING_DEFINITIONS_DONE,
906 } sending_definitions_status;
907 + char *pipe_buffer;
908 };
909
910 static size_t cbuffer_available_bytes_with_lock(struct rrdpush_sender_thread_data *thread_data) {
@@ -742,7 +916,7 @@ static size_t cbuffer_available_bytes_with_lock(struct rrdpush_sender_thread_dat
916
917 static void rrdpush_queue_incremental_definitions(struct rrdpush_sender_thread_data *thread_data) {
918
745 - while(__atomic_load_n(&thread_data->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST)
919 + while(rrdhost_flag_check(thread_data->host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED)
920 && thread_data->sending_definitions_status != SENDING_DEFINITIONS_DONE
921 && cbuffer_available_bytes_with_lock(thread_data) > (thread_data->sender_state->buffer->max_size / 2)) {
922
@@ -762,6 +936,61 @@ static void rrdpush_queue_incremental_definitions(struct rrdpush_sender_thread_d
936 }
937 }
938
939 +static bool rrdpush_sender_pipe_close(RRDHOST *host, int *pipe_fds, bool reopen) {
940 + static netdata_mutex_t mutex = NETDATA_MUTEX_INITIALIZER;
941 +
942 + bool ret = true;
943 +
944 + netdata_mutex_lock(&mutex);
945 +
946 + int new_pipe_fds[2];
947 + if(reopen) {
948 + if(pipe(new_pipe_fds) != 0) {
949 + error("STREAM %s [send]: cannot create required pipe.", rrdhost_hostname(host));
950 + new_pipe_fds[PIPE_READ] = -1;
951 + new_pipe_fds[PIPE_WRITE] = -1;
952 + ret = false;
953 + }
954 + }
955 +
956 + int old_pipe_fds[2];
957 + old_pipe_fds[PIPE_READ] = pipe_fds[PIPE_READ];
958 + old_pipe_fds[PIPE_WRITE] = pipe_fds[PIPE_WRITE];
959 +
960 + if(reopen) {
961 + pipe_fds[PIPE_READ] = new_pipe_fds[PIPE_READ];
962 + pipe_fds[PIPE_WRITE] = new_pipe_fds[PIPE_WRITE];
963 + }
964 + else {
965 + pipe_fds[PIPE_READ] = -1;
966 + pipe_fds[PIPE_WRITE] = -1;
967 + }
968 +
969 + if(old_pipe_fds[PIPE_READ] > 2)
970 + close(old_pipe_fds[PIPE_READ]);
971 +
972 + if(old_pipe_fds[PIPE_WRITE] > 2)
973 + close(old_pipe_fds[PIPE_WRITE]);
974 +
975 + netdata_mutex_unlock(&mutex);
976 + return ret;
977 +}
978 +
979 +void rrdpush_signal_sender_to_wake_up(struct sender_state *s) {
980 + if(unlikely(s->tid == gettid()))
981 + return;
982 +
983 + RRDHOST *host = s->host;
984 +
985 + int pipe_fd = s->rrdpush_sender_pipe[PIPE_WRITE];
986 +
987 + // signal the sender there are more data
988 + if (pipe_fd != -1 && write(pipe_fd, " ", 1) == -1) {
989 + error("STREAM %s [send]: cannot write to internal pipe.", rrdhost_hostname(host));
990 + rrdpush_sender_pipe_close(host, s->rrdpush_sender_pipe, true);
991 + }
992 +}
993 +
994 static void rrdpush_sender_thread_cleanup_callback(void *ptr) {
995 struct rrdpush_sender_thread_data *data = ptr;
996 worker_unregister();
@@ -775,29 +1004,20 @@ static void rrdpush_sender_thread_cleanup_callback(void *ptr) {
1004 info("STREAM %s [send]: sending thread cleans up...", rrdhost_hostname(host));
1005
1006 rrdpush_sender_thread_close_socket(host);
1007 + rrdpush_sender_pipe_close(host, host->sender->rrdpush_sender_pipe, false);
1008
779 - // close the pipe
780 - if(host->rrdpush_sender_pipe[PIPE_READ] != -1) {
781 - close(host->rrdpush_sender_pipe[PIPE_READ]);
782 - host->rrdpush_sender_pipe[PIPE_READ] = -1;
783 - }
784 -
785 - if(host->rrdpush_sender_pipe[PIPE_WRITE] != -1) {
786 - close(host->rrdpush_sender_pipe[PIPE_WRITE]);
787 - host->rrdpush_sender_pipe[PIPE_WRITE] = -1;
788 - }
789 -
790 - if(!host->rrdpush_sender_join) {
1009 + if(!rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_JOIN)) {
1010 info("STREAM %s [send]: sending thread detaches itself.", rrdhost_hostname(host));
1011 netdata_thread_detach(netdata_thread_self());
1012 }
1013
795 - host->rrdpush_sender_spawn = 0;
1014 + rrdhost_flag_clear(host, RRDHOST_FLAG_RRDPUSH_SENDER_SPAWN);
1015
1016 info("STREAM %s [send]: sending thread now exits.", rrdhost_hostname(host));
1017
1018 netdata_mutex_unlock(&host->sender->mutex);
1019
1020 + freez(data->pipe_buffer);
1021 freez(data);
1022 }
1023
@@ -809,118 +1029,140 @@ void sender_init(RRDHOST *parent)
1029 parent->sender = callocz(1, sizeof(*parent->sender));
1030 parent->sender->host = parent;
1031 parent->sender->buffer = cbuffer_new(1024, 1024*1024);
812 - parent->sender->build = buffer_create(1);
1032 + parent->sender->capabilities = STREAM_OUR_CAPABILITIES;
1033 +
1034 + parent->sender->rrdpush_sender_pipe[PIPE_READ] = -1;
1035 + parent->sender->rrdpush_sender_pipe[PIPE_WRITE] = -1;
1036 + parent->sender->rrdpush_sender_socket = -1;
1037 +
1038 #ifdef ENABLE_COMPRESSION
814 - parent->sender->rrdpush_compression = default_compression_enabled;
815 - if (default_compression_enabled)
1039 + if(default_compression_enabled) {
1040 + parent->sender->flags |= SENDER_FLAG_COMPRESSION;
1041 parent->sender->compressor = create_compressor();
1042 + }
1043 #endif
1044 +
1045 netdata_mutex_init(&parent->sender->mutex);
1046 }
1047
1048 void *rrdpush_sender_thread(void *ptr) {
1049 + worker_register("STREAMSND");
1050 + worker_register_job_name(WORKER_SENDER_JOB_CONNECT, "connect");
1051 + worker_register_job_name(WORKER_SENDER_JOB_PIPE_READ, "pipe read");
1052 + worker_register_job_name(WORKER_SENDER_JOB_SOCKET_RECEIVE, "receive");
1053 + worker_register_job_name(WORKER_SENDER_JOB_EXECUTE, "execute");
1054 + worker_register_job_name(WORKER_SENDER_JOB_SOCKET_SEND, "send");
1055 +
1056 + // disconnection reasons
1057 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_TIMEOUT, "disconnect timeout");
1058 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_POLL_ERROR, "disconnect poll error");
1059 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SOCKER_ERROR, "disconnect socket error");
1060 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_OVERFLOW, "disconnect overflow");
1061 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SSL_ERROR, "disconnect ssl error");
1062 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_PARENT_CLOSED, "disconnect parent closed");
1063 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_RECEIVE_ERROR, "disconnect receive error");
1064 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SEND_ERROR, "disconnect send error");
1065 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_NO_COMPRESSION, "disconnect no compression");
1066 + worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE, "disconnect bad handshake");
1067 +
1068 + worker_register_job_custom_metric(WORKER_SENDER_JOB_BUFFER_RATIO, "used buffer ratio", "%", WORKER_METRIC_ABSOLUTE);
1069 + worker_register_job_custom_metric(WORKER_SENDER_JOB_BYTES_RECEIVED, "bytes received", "bytes/s", WORKER_METRIC_INCREMENTAL);
1070 + worker_register_job_custom_metric(WORKER_SENDER_JOB_BYTES_SENT, "bytes sent", "bytes/s", WORKER_METRIC_INCREMENTAL);
1071 +
1072 struct sender_state *s = ptr;
823 - s->task_id = gettid();
1073 + s->tid = gettid();
1074
825 - if(!s->host->rrdpush_send_enabled || !s->host->rrdpush_send_destination ||
1075 + if(!rrdhost_has_rrdpush_sender_enabled(s->host) || !s->host->rrdpush_send_destination ||
1076 !*s->host->rrdpush_send_destination || !s->host->rrdpush_send_api_key ||
1077 !*s->host->rrdpush_send_api_key) {
1078 error("STREAM %s [send]: thread created (task id %d), but host has streaming disabled.",
829 - rrdhost_hostname(s->host), s->task_id);
1079 + rrdhost_hostname(s->host), s->tid);
1080 return NULL;
1081 }
1082
1083 #ifdef ENABLE_HTTPS
1084 if (netdata_use_ssl_on_stream & NETDATA_SSL_FORCE ){
1085 security_start_ssl(NETDATA_SSL_CONTEXT_STREAMING);
836 - security_location_for_context(netdata_client_ctx, netdata_ssl_ca_file, netdata_ssl_ca_path);
1086 + ssl_security_location_for_context(netdata_ssl_client_ctx, netdata_ssl_ca_file, netdata_ssl_ca_path);
1087 }
1088 #endif
1089
840 - info("STREAM %s [send]: thread created (task id %d)", rrdhost_hostname(s->host), s->task_id);
1090 + info("STREAM %s [send]: thread created (task id %d)", rrdhost_hostname(s->host), s->tid);
1091 +
1092 + s->timeout = (int)appconfig_get_number(
1093 + &stream_config, CONFIG_SECTION_STREAM, "timeout seconds", 60);
1094 +
1095 + s->default_port = (int)appconfig_get_number(
1096 + &stream_config, CONFIG_SECTION_STREAM, "default port", 19999);
1097 +
1098 + s->buffer->max_size = (size_t)appconfig_get_number(
1099 + &stream_config, CONFIG_SECTION_STREAM, "buffer size bytes", 1024 * 1024 * 10);
1100 +
1101 + s->reconnect_delay = (unsigned int)appconfig_get_number(
1102 + &stream_config, CONFIG_SECTION_STREAM, "reconnect delay seconds", 5);
1103
842 - s->timeout = (int)appconfig_get_number(&stream_config, CONFIG_SECTION_STREAM, "timeout seconds", 60);
843 - s->default_port = (int)appconfig_get_number(&stream_config, CONFIG_SECTION_STREAM, "default port", 19999);
844 - s->buffer->max_size =
845 - (size_t)appconfig_get_number(&stream_config, CONFIG_SECTION_STREAM, "buffer size bytes", 1024 * 1024 * 10);
846 - s->reconnect_delay =
847 - (unsigned int)appconfig_get_number(&stream_config, CONFIG_SECTION_STREAM, "reconnect delay seconds", 5);
1104 remote_clock_resync_iterations = (unsigned int)appconfig_get_number(
1105 &stream_config, CONFIG_SECTION_STREAM,
1106 "initial clock resync iterations",
1107 remote_clock_resync_iterations); // TODO: REMOVE FOR SLEW / GAPFILLING
1108
1109 // initialize rrdpush globals
854 - rrdhost_flag_clear(s->host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS);
855 - __atomic_clear(&s->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST);
856 - if(pipe(s->host->rrdpush_sender_pipe) == -1) {
857 - error("STREAM %s [send]: cannot create required pipe. DISABLING STREAMING THREAD", rrdhost_hostname(s->host));
858 - return NULL;
859 - }
860 - s->version = STREAMING_PROTOCOL_CURRENT_VERSION;
861 -
862 - enum {
863 - Collector,
864 - Socket
865 - };
866 - struct pollfd fds[2];
867 - fds[Collector].fd = s->host->rrdpush_sender_pipe[PIPE_READ];
868 - fds[Collector].events = POLLIN;
1110 + rrdhost_flag_clear(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS);
1111 + rrdhost_flag_clear(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED);
1112
870 - worker_register("STREAMSND");
871 - worker_register_job_name(WORKER_SENDER_JOB_CONNECT, "connect");
872 - worker_register_job_name(WORKER_SENDER_JOB_PIPE_READ, "pipe read");
873 - worker_register_job_name(WORKER_SENDER_JOB_SOCKET_RECEIVE, "receive");
874 - worker_register_job_name(WORKER_SENDER_JOB_EXECUTE, "execute");
875 - worker_register_job_name(WORKER_SENDER_JOB_SOCKET_SEND, "send");
876 -
877 - // disconnection reasons
878 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_TIMEOUT, "disconnect timeout");
879 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_POLL_ERROR, "disconnect poll error");
880 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SOCKER_ERROR, "disconnect socket error");
881 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_OVERFLOW, "disconnect overflow");
882 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SSL_ERROR, "disconnect ssl error");
883 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_PARENT_CLOSED, "disconnect parent closed");
884 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_RECEIVE_ERROR, "disconnect receive error");
885 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_SEND_ERROR, "disconnect send error");
886 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_NO_COMPRESSION, "disconnect no compression");
887 - worker_register_job_name(WORKER_SENDER_JOB_DISCONNECT_BAD_HANDSHAKE, "disconnect bad handshake");
1113 + int pipe_buffer_size = 10 * 1024;
1114 +#ifdef F_GETPIPE_SZ
1115 + pipe_buffer_size = fcntl(s->rrdpush_sender_pipe[PIPE_READ], F_GETPIPE_SZ);
1116 +#endif
1117 + if(pipe_buffer_size < 10 * 1024)
1118 + pipe_buffer_size = 10 * 1024;
1119
889 - worker_register_job_custom_metric(WORKER_SENDER_JOB_BUFFER_RATIO, "used buffer ratio", "%", WORKER_METRIC_ABSOLUTE);
890 - worker_register_job_custom_metric(WORKER_SENDER_JOB_BYTES_RECEIVED, "bytes received", "bytes/s", WORKER_METRIC_INCREMENTAL);
891 - worker_register_job_custom_metric(WORKER_SENDER_JOB_BYTES_SENT, "bytes sent", "bytes/s", WORKER_METRIC_INCREMENTAL);
1120 + if(!rrdpush_sender_pipe_close(s->host, s->rrdpush_sender_pipe, true)) {
1121 + error("STREAM %s [send]: cannot create inter-thread communication pipe. Disabling streaming.",
1122 + rrdhost_hostname(s->host));
1123 + return NULL;
1124 + }
1125
1126 struct rrdpush_sender_thread_data *thread_data = callocz(1, sizeof(struct rrdpush_sender_thread_data));
1127 + thread_data->pipe_buffer = mallocz(pipe_buffer_size);
1128 thread_data->sender_state = s;
1129 thread_data->host = s->host;
1130 thread_data->sending_definitions_status = SENDING_DEFINITIONS_RESTART;
1131
1132 + // reset our cleanup flags
1133 + rrdhost_flag_clear(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_JOIN);
1134 +
1135 netdata_thread_cleanup_push(rrdpush_sender_thread_cleanup_callback, thread_data);
1136
900 - for(; s->host->rrdpush_send_enabled && !netdata_exit ;) {
1137 + for(; rrdhost_has_rrdpush_sender_enabled(s->host) && !netdata_exit ;) {
1138 // check for outstanding cancellation requests
1139 netdata_thread_testcancel();
1140
1141 // The connection attempt blocks (after which we use the socket in nonblocking)
905 - if(unlikely(s->host->rrdpush_sender_socket == -1)) {
1142 + if(unlikely(s->rrdpush_sender_socket == -1)) {
1143 worker_is_busy(WORKER_SENDER_JOB_CONNECT);
1144 thread_data->sending_definitions_status = SENDING_DEFINITIONS_RESTART;
908 - rrdhost_flag_clear(s->host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS);
909 - s->overflow = 0;
1145 + rrdhost_flag_clear(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS);
1146 + s->flags &= ~SENDER_FLAG_OVERFLOW;
1147 s->read_len = 0;
1148 s->buffer->read = 0;
1149 s->buffer->write = 0;
913 - attempt_to_connect(s);
914 - if (s->version >= VERSION_GAP_FILLING) {
1150 +
1151 + if(unlikely(!attempt_to_connect(s)))
1152 + continue;
1153 +
1154 + if (stream_has_capability(s, STREAM_CAP_GAP_FILLING)) {
1155 time_t now = now_realtime_sec();
916 - sender_start(s);
917 - buffer_sprintf(s->build, "TIMESTAMP %"PRId64"", (int64_t)now);
918 - sender_commit(s);
1156 + BUFFER *wb = sender_start(s);
1157 + buffer_sprintf(wb, "TIMESTAMP %"PRId64"", (int64_t)now);
1158 + sender_commit(s, wb);
1159 }
1160 +
1161 rrdpush_claimed_id(s->host);
1162 + rrdpush_send_host_labels(s->host);
1163
1164 // TO PUSH METRICS WITH DEFINITIONS:
923 - //if(unlikely(s->host->rrdpush_sender_socket != -1 && __atomic_load_n(&s->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST))) {
1165 + //if(unlikely(s->rrdpush_sender_socket != -1 && __atomic_load_n(&s->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST))) {
1166 // thread_data->sending_definitions_status = SENDING_DEFINITIONS_DONE;
1167 // rrdhost_flag_set(s->host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS);
1168 //}
@@ -939,13 +1181,6 @@ void *rrdpush_sender_thread(void *ptr) {
1181 if(unlikely(thread_data->sending_definitions_status != SENDING_DEFINITIONS_DONE))
1182 rrdpush_queue_incremental_definitions(thread_data);
1183
942 - worker_is_idle();
943 -
944 - // Wait until buffer opens in the socket or a rrdset_done_push wakes us
945 - fds[Collector].revents = 0;
946 - fds[Socket].revents = 0;
947 - fds[Socket].fd = s->host->rrdpush_sender_socket;
948 -
1184 netdata_mutex_lock(&s->mutex);
1185 size_t outstanding = cbuffer_next_unsafe(s->host->sender->buffer, NULL);
1186 size_t available = cbuffer_available_size_unsafe(s->host->sender->buffer);
@@ -953,50 +1188,89 @@ void *rrdpush_sender_thread(void *ptr) {
1188
1189 worker_set_metric(WORKER_SENDER_JOB_BUFFER_RATIO, (NETDATA_DOUBLE)(s->host->sender->buffer->max_size - available) * 100.0 / (NETDATA_DOUBLE)s->host->sender->buffer->max_size);
1190
956 - if(outstanding) {
1191 + if(outstanding)
1192 s->send_attempts++;
958 - fds[Socket].events = POLLIN | POLLOUT;
959 - }
1193 else {
961 - fds[Socket].events = POLLIN;
962 -
1194 if(unlikely(thread_data->sending_definitions_status == SENDING_DEFINITIONS_DONE
964 - && __atomic_load_n(&s->host->rrdpush_sender_connected, __ATOMIC_SEQ_CST)
965 - && !rrdhost_flag_check(s->host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS)
966 - )) {
1195 + && rrdhost_flag_check(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_CONNECTED)
1196 + && !rrdhost_flag_check(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS)
1197 + )) {
1198 // let the data collection threads know we are ready to push metrics
968 - rrdhost_flag_set(s->host, RRDHOST_FLAG_STREAM_COLLECTED_METRICS);
1199 + rrdhost_flag_set(s->host, RRDHOST_FLAG_RRDPUSH_SENDER_READY_4_METRICS);
1200 info("STREAM %s [send to %s]: enabling metrics streaming...", rrdhost_hostname(s->host), s->connected_to);
1201 }
1202 }
1203
973 - int retval = poll(fds, 2, 1000);
1204 + if(unlikely(s->rrdpush_sender_pipe[PIPE_READ] == -1)) {
1205 + if(!rrdpush_sender_pipe_close(s->host, s->rrdpush_sender_pipe, true)) {
1206 + error("STREAM %s [send]: cannot create inter-thread communication pipe. Disabling streaming.",
1207 + rrdhost_hostname(s->host));
1208 + rrdpush_sender_thread_close_socket(s->host);
1209 + break;
1210 + }
1211 + }
1212 +
1213 + worker_is_idle();
1214 +
1215 + // Wait until buffer opens in the socket or a rrdset_done_push wakes us
1216 + enum {
1217 + Collector = 0,
1218 + Socket = 1,
1219 + };
1220 + struct pollfd fds[2] = {
1221 + [Collector] = {
1222 + .fd = s->rrdpush_sender_pipe[PIPE_READ],
1223 + .events = POLLIN,
1224 + .revents = 0,
1225 + },
1226 + [Socket] = {
1227 + .fd = s->rrdpush_sender_socket,
1228 + .events = POLLIN | (outstanding ? POLLOUT : 0 ),
1229 + .revents = 0,
1230 + }
1231 + };
1232 + int poll_rc = poll(fds, 2, 1000);
1233 +
1234 debug(D_STREAM, "STREAM: poll() finished collector=%d socket=%d (current chunk %zu bytes)...",
1235 fds[Collector].revents, fds[Socket].revents, outstanding);
1236
1237 if(unlikely(netdata_exit)) break;
1238
1239 + internal_error(fds[Collector].fd != s->rrdpush_sender_pipe[PIPE_READ],
1240 + "STREAM %s [send to %s]: pipe changed after poll().", rrdhost_hostname(s->host), s->connected_to);
1241 +
1242 + internal_error(fds[Socket].fd != s->rrdpush_sender_socket,
1243 + "STREAM %s [send to %s]: socket changed after poll().", rrdhost_hostname(s->host), s->connected_to);
1244 +
1245 // Spurious wake-ups without error - loop again
980 - if (retval == 0 || ((retval == -1) && (errno == EAGAIN || errno == EINTR))) {
1246 + if (poll_rc == 0 || ((poll_rc == -1) && (errno == EAGAIN || errno == EINTR))) {
1247 debug(D_STREAM, "Spurious wakeup");
1248 continue;
1249 }
1250
1251 // Only errors from poll() are internal, but try restarting the connection
986 - if(unlikely(retval == -1)) {
1252 + if(unlikely(poll_rc == -1)) {
1253 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_POLL_ERROR);
1254 error("STREAM %s [send to %s]: failed to poll(). Closing socket.", rrdhost_hostname(s->host), s->connected_to);
1255 + rrdpush_sender_pipe_close(s->host, s->rrdpush_sender_pipe, true);
1256 rrdpush_sender_thread_close_socket(s->host);
1257 continue;
1258 }
1259
1260 + // If we have data and have seen the TCP window open then try to close it by a transmission.
1261 + if(likely(outstanding && (fds[Socket].revents & POLLOUT))) {
1262 + worker_is_busy(WORKER_SENDER_JOB_SOCKET_SEND);
1263 + ssize_t bytes = attempt_to_send(s);
1264 + if(bytes > 0)
1265 + worker_set_metric(WORKER_SENDER_JOB_BYTES_SENT, bytes);
1266 + }
1267 +
1268 // If the collector woke us up then empty the pipe to remove the signal
994 - if (fds[Collector].revents & POLLIN || fds[Collector].revents & POLLPRI) {
1269 + if (fds[Collector].revents & (POLLIN|POLLPRI)) {
1270 worker_is_busy(WORKER_SENDER_JOB_PIPE_READ);
1271 debug(D_STREAM, "STREAM: Data added to send buffer (current buffer chunk %zu bytes)...", outstanding);
1272
998 - char buffer[10000 + 1];
999 - if (read(s->host->rrdpush_sender_pipe[PIPE_READ], buffer, 10000) == -1)
1273 + if (read(fds[Collector].fd, thread_data->pipe_buffer, pipe_buffer_size) == -1)
1274 error("STREAM %s [send to %s]: cannot read from internal pipe.", rrdhost_hostname(s->host), s->connected_to);
1275 }
1276
@@ -1013,33 +1287,43 @@ void *rrdpush_sender_thread(void *ptr) {
1287 execute_commands(s);
1288 }
1289
1016 - // If we have data and have seen the TCP window open then try to close it by a transmission.
1017 - if(likely(outstanding && fds[Socket].revents & POLLOUT)) {
1018 - worker_is_busy(WORKER_SENDER_JOB_SOCKET_SEND);
1019 - ssize_t bytes = attempt_to_send(s);
1020 - if(bytes > 0)
1021 - worker_set_metric(WORKER_SENDER_JOB_BYTES_SENT, bytes);
1290 + if(unlikely(fds[Collector].revents & (POLLERR|POLLHUP|POLLNVAL))) {
1291 + char *error = NULL;
1292 +
1293 + if (unlikely(fds[Collector].revents & POLLERR))
1294 + error = "pipe reports errors (POLLERR)";
1295 + else if (unlikely(fds[Collector].revents & POLLHUP))
1296 + error = "pipe closed (POLLHUP)";
1297 + else if (unlikely(fds[Collector].revents & POLLNVAL))
1298 + error = "pipe is invalid (POLLNVAL)";
1299 +
1300 + if(error) {
1301 + rrdpush_sender_pipe_close(s->host, s->rrdpush_sender_pipe, true);
1302 + error("STREAM %s [send to %s]: restarting internal pipe: %s.",
1303 + rrdhost_hostname(s->host), s->connected_to, error);
1304 + }
1305 }
1306
1024 - // TODO-GAPS - why do we only check this on the socket, not the pipe?
1025 - if(outstanding) {
1307 + if(unlikely(fds[Socket].revents & (POLLERR|POLLHUP|POLLNVAL))) {
1308 char *error = NULL;
1309 +
1310 if (unlikely(fds[Socket].revents & POLLERR))
1311 error = "socket reports errors (POLLERR)";
1312 else if (unlikely(fds[Socket].revents & POLLHUP))
1313 error = "connection closed by remote end (POLLHUP)";
1314 else if (unlikely(fds[Socket].revents & POLLNVAL))
1315 error = "connection is invalid (POLLNVAL)";
1316 +
1317 if(unlikely(error)) {
1318 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_SOCKER_ERROR);
1035 - error("STREAM %s [send to %s]: restart stream because %s - %zu bytes transmitted.", rrdhost_hostname(s->host),
1036 - s->connected_to, error, s->sent_bytes_on_this_connection);
1319 + error("STREAM %s [send to %s]: restarting connection: %s - %zu bytes transmitted.",
1320 + rrdhost_hostname(s->host), s->connected_to, error, s->sent_bytes_on_this_connection);
1321 rrdpush_sender_thread_close_socket(s->host);
1322 }
1323 }
1324
1325 // protection from overflow
1042 - if(unlikely(s->overflow)) {
1326 + if(unlikely(s->flags & SENDER_FLAG_OVERFLOW)) {
1327 worker_is_busy(WORKER_SENDER_JOB_DISCONNECT_OVERFLOW);
1328 errno = 0;
1329 error("STREAM %s [send to %s]: buffer full (allocated %zu bytes) after sending %zu bytes. Restarting connection",
web/api/formatters/json_wrapper.c
+19
@@ -175,6 +175,25 @@ void rrdr_json_wrapper_begin(RRDR *r, BUFFER *wb, uint32_t format, RRDR_OPTIONS
175 buffer_strcat(wb, "],\n");
176 }
177
178 + // functions
179 + {
180 + DICTIONARY *funcs = dictionary_create(DICT_OPTION_SINGLE_THREADED|DICT_OPTION_DONT_OVERWRITE_VALUE);
181 + for (i = 0, rd = temp_rd ? temp_rd : r->st->dimensions; rd; rd = rd->next) {
182 + chart_functions_to_dict(rd->rrdset, funcs);
183 + }
184 +
185 + buffer_sprintf(wb, " %sfunctions%s: [", kq, kq);
186 + void *t; (void)t;
187 + dfe_start_read(funcs, t) {
188 + const char *comma = "";
189 + if(t_dfe.counter) comma = ", ";
190 + buffer_sprintf(wb, "%s%s%s%s", comma, sq, t_dfe.name, sq);
191 + }
192 + dfe_done(t);
193 + dictionary_destroy(funcs);
194 + buffer_strcat(wb, "],\n");
195 + }
196 +
197 // Composite charts
198 if (context_mode && temp_rd) {
199 buffer_sprintf(
web/api/formatters/rrdset2json.c
+4 -1
@@ -143,8 +143,11 @@ void rrdset2json(RRDSET *st, BUFFER *wb, size_t *dimensions_count, size_t *memor
143 }
144 buffer_strcat(wb, ",\n\t\t\t\"chart_labels\": {\n");
145 chart_labels2json(st, wb, 2);
146 - buffer_strcat(wb, "\t\t\t}\n");
146 + buffer_strcat(wb, "\t\t\t}");
147
148 + buffer_strcat(wb, ",\n\t\t\t\"functions\": {\n");
149 + chart_functions2json(st, wb, 4, "\"", "\"");
150 + buffer_strcat(wb, "\t\t\t}");
151
152 buffer_sprintf(wb,
153 "\n\t\t}"
web/api/netdata-swagger.json
+62
@@ -1629,6 +1629,68 @@
1629 }
1630 }
1631 },
1632 + "/function": {
1633 + "get": {
1634 + "summary": "Execute a collector function.",
1635 + "parameters": [
1636 + {
1637 + "name": "function",
1638 + "in": "query",
1639 + "description": "The name of the function, as returned by the collector.",
1640 + "required": true,
1641 + "allowEmptyValue": false,
1642 + "schema": {
1643 + "type": "string"
1644 + }
1645 + },
1646 + {
1647 + "name": "timeout",
1648 + "in": "query",
1649 + "description": "The timeout in seconds to wait for the function to complete.",
1650 + "required": false,
1651 + "schema": {
1652 + "type": "number",
1653 + "format": "integer",
1654 + "default": 10
1655 + }
1656 + }
1657 + ],
1658 + "responses": {
1659 + "200": {
1660 + "description": "The collector function has been executed successfully. Each collector may return a different type of content."
1661 + },
1662 + "400": {
1663 + "description": "The request was rejected by the collector."
1664 + },
1665 + "404": {
1666 + "description": "The requested function is not found."
1667 + },
1668 + "500": {
1669 + "description": "Other internal error, getting this error means there is a bug in Netdata."
1670 + },
1671 + "503": {
1672 + "description": "The collector to execute the function is not currently available."
1673 + },
1674 + "504": {
1675 + "description": "Timeout while waiting for the collector to execute the function."
1676 + },
1677 + "591": {
1678 + "description": "The collector sent a response, but it was invalid or corrupted."
1679 + }
1680 + }
1681 + }
1682 + },
1683 + "/functions": {
1684 + "get": {
1685 + "summary": "Get a list of all registered collector functions.",
1686 + "description": "Collector functions are programs that can be executed on demand.",
1687 + "responses": {
1688 + "200": {
1689 + "description": "A JSON object containing one object per supported function."
1690 + }
1691 + }
1692 + }
1693 + },
1694 "/weights": {
1695 "get": {
1696 "summary": "Analyze all the metrics using an algorithm and score them accordingly",
web/api/netdata-swagger.yaml
+41
@@ -1351,6 +1351,47 @@ paths:
1351 that correlated the metrics did not produce any result.
1352 "504":
1353 description: Timeout - the query took too long and has been cancelled.
1354 + /function:
1355 + get:
1356 + summary: "Execute a collector function."
1357 + parameters:
1358 + - name: function
1359 + in: query
1360 + description: The name of the function, as returned by the collector.
1361 + required: true
1362 + allowEmptyValue: false
1363 + schema:
1364 + type: string
1365 + - name: timeout
1366 + in: query
1367 + description: The timeout in seconds to wait for the function to complete.
1368 + required: false
1369 + schema:
1370 + type: number
1371 + format: integer
1372 + default: 10
1373 + responses:
1374 + "200":
1375 + description: The collector function has been executed successfully. Each collector may return a different type of content.
1376 + "400":
1377 + description: The request was rejected by the collector.
1378 + "404":
1379 + description: The requested function is not found.
1380 + "500":
1381 + description: Other internal error, getting this error means there is a bug in Netdata.
1382 + "503":
1383 + description: The collector to execute the function is not currently available.
1384 + "504":
1385 + description: Timeout while waiting for the collector to execute the function.
1386 + "591":
1387 + description: The collector sent a response, but it was invalid or corrupted.
1388 + /functions:
1389 + get:
1390 + summary: Get a list of all registered collector functions.
1391 + description: Collector functions are programs that can be executed on demand.
1392 + responses:
1393 + "200":
1394 + description: A JSON object containing one object per supported function.
1395 /weights:
1396 get:
1397 summary: "Analyze all the metrics using an algorithm and score them accordingly"
web/api/web_api_v1.c
+85 -24
@@ -1200,6 +1200,10 @@ inline int web_client_api_request_v1_info_fill_buffer(RRDHOST *host, BUFFER *wb)
1200 host_labels2json(host, wb, 2);
1201 buffer_strcat(wb, "\t},\n");
1202
1203 + buffer_strcat(wb, "\t\"functions\": {\n");
1204 + host_functions2json(host, wb, 2, "\"", "\"");
1205 + buffer_strcat(wb, "\t},\n");
1206 +
1207 buffer_strcat(wb, "\t\"collectors\": [");
1208 chartcollectors2json(host, wb);
1209 buffer_strcat(wb, "\n\t],\n");
@@ -1250,7 +1254,7 @@ inline int web_client_api_request_v1_info_fill_buffer(RRDHOST *host, BUFFER *wb)
1254 #ifdef ENABLE_COMPRESSION
1255 if(host->sender){
1256 buffer_strcat(wb, "\t\"stream-compression\": ");
1253 - buffer_strcat(wb, (host->sender->rrdpush_compression ? "true" : "false"));
1257 + buffer_strcat(wb, (host->sender->flags & SENDER_FLAG_COMPRESSION) ? "true" : "false");
1258 buffer_strcat(wb, ",\n");
1259 }else{
1260 buffer_strcat(wb, "\t\"stream-compression\": null,\n");
@@ -1483,6 +1487,53 @@ int web_client_api_request_v1_weights(RRDHOST *host, struct web_client *w, char
1487 return web_client_api_request_v1_weights_internal(host, w, url, WEIGHTS_METHOD_ANOMALY_RATE, WEIGHTS_FORMAT_CONTEXTS);
1488 }
1489
1490 +int web_client_api_request_v1_function(RRDHOST *host, struct web_client *w, char *url) {
1491 + if (!netdata_ready)
1492 + return HTTP_RESP_BACKEND_FETCH_FAILED;
1493 +
1494 + int timeout = 0;
1495 + const char *function = NULL;
1496 +
1497 + while (url) {
1498 + char *value = mystrsep(&url, "&");
1499 + if (!value || !*value)
1500 + continue;
1501 +
1502 + char *name = mystrsep(&value, "=");
1503 + if (!name || !*name)
1504 + continue;
1505 +
1506 + if (!strcmp(name, "function"))
1507 + function = value;
1508 +
1509 + else if (!strcmp(name, "timeout"))
1510 + timeout = (int) strtoul(value, NULL, 0);
1511 + }
1512 +
1513 + BUFFER *wb = w->response.data;
1514 + buffer_flush(wb);
1515 + wb->contenttype = CT_APPLICATION_JSON;
1516 + buffer_no_cacheable(wb);
1517 +
1518 + return rrd_call_function_and_wait(host, wb, timeout, function);
1519 +}
1520 +
1521 +int web_client_api_request_v1_functions(RRDHOST *host, struct web_client *w, char *url __maybe_unused) {
1522 + if (!netdata_ready)
1523 + return HTTP_RESP_BACKEND_FETCH_FAILED;
1524 +
1525 + BUFFER *wb = w->response.data;
1526 + buffer_flush(wb);
1527 + wb->contenttype = CT_APPLICATION_JSON;
1528 + buffer_no_cacheable(wb);
1529 +
1530 + buffer_strcat(wb, "{\n");
1531 + host_functions2json(host, wb, 1, "\"", "\"");
1532 + buffer_strcat(wb, "}");
1533 +
1534 + return HTTP_RESP_OK;
1535 +}
1536 +
1537 #ifndef ENABLE_DBENGINE
1538 int web_client_api_request_v1_dbengine_stats(RRDHOST *host, struct web_client *w, char *url) {
1539 return HTTP_RESP_NOT_FOUND;
@@ -1585,47 +1636,57 @@ int web_client_api_request_v1_dbengine_stats(RRDHOST *host __maybe_unused, struc
1636 }
1637 #endif
1638
1639 +#ifdef NETDATA_DEV_MODE
1640 +#define ACL_DEV_OPEN_ACCESS WEB_CLIENT_ACL_DASHBOARD
1641 +#else
1642 +#define ACL_DEV_OPEN_ACCESS 0
1643 +#endif
1644 +
1645 static struct api_command {
1646 const char *command;
1647 uint32_t hash;
1648 WEB_CLIENT_ACL acl;
1649 int (*callback)(RRDHOST *host, struct web_client *w, char *url);
1650 } api_commands[] = {
1594 - { "info", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_info },
1595 - { "data", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_data },
1596 - { "chart", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_chart },
1597 - { "charts", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_charts },
1598 - { "context", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_context },
1599 - { "contexts", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_contexts },
1600 - { "archivedcharts", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_archivedcharts },
1651 + { "info", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_info },
1652 + { "data", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_data },
1653 + { "chart", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_chart },
1654 + { "charts", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_charts },
1655 + { "context", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_context },
1656 + { "contexts", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_contexts },
1657 + { "archivedcharts", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_archivedcharts },
1658
1659 // registry checks the ACL by itself, so we allow everything
1603 - { "registry", 0, WEB_CLIENT_ACL_NOCHECK, web_client_api_request_v1_registry },
1660 + { "registry", 0, WEB_CLIENT_ACL_NOCHECK, web_client_api_request_v1_registry },
1661
1662 // badges can be fetched with both dashboard and badge permissions
1606 - { "badge.svg", 0, WEB_CLIENT_ACL_DASHBOARD|WEB_CLIENT_ACL_BADGE, web_client_api_request_v1_badge },
1663 + { "badge.svg", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_BADGE | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_badge },
1664
1608 - { "alarms", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_alarms },
1609 - { "alarms_values", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_alarms_values },
1610 - { "alarm_log", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_alarm_log },
1611 - { "alarm_variables", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_alarm_variables },
1612 - { "alarm_count", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_alarm_count },
1613 - { "allmetrics", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_allmetrics },
1665 + { "alarms", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_alarms },
1666 + { "alarms_values", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_alarms_values },
1667 + { "alarm_log", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_alarm_log },
1668 + { "alarm_variables", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_alarm_variables },
1669 + { "alarm_count", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_alarm_count },
1670 + { "allmetrics", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_allmetrics },
1671
1672 #if defined(ENABLE_ML)
1616 - { "ml_info", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_ml_info },
1617 - { "ml_models", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_ml_models },
1673 + { "anomaly_events", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_anomaly_events },
1674 + { "anomaly_event_info", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_anomaly_event_info },
1675 + { "ml_info", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_ml_info },
1676 #endif
1677
1620 - { "manage/health", 0, WEB_CLIENT_ACL_MGMT, web_client_api_request_v1_mgmt_health },
1621 - { "aclk", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_aclk_state },
1622 - { "metric_correlations", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_metric_correlations },
1623 - { "weights", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_weights },
1678 + { "manage/health", 0, WEB_CLIENT_ACL_MGMT | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_mgmt_health },
1679 + { "aclk", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_aclk_state },
1680 + { "metric_correlations", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_metric_correlations },
1681 + { "weights", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_weights },
1682 +
1683 + { "function", 0, WEB_CLIENT_ACL_ACLK | ACL_DEV_OPEN_ACCESS, web_client_api_request_v1_function },
1684 + { "functions", 0, WEB_CLIENT_ACL_ACLK | ACL_DEV_OPEN_ACCESS, web_client_api_request_v1_functions },
1685
1625 - { "dbengine_stats", 0, WEB_CLIENT_ACL_DASHBOARD, web_client_api_request_v1_dbengine_stats },
1686 + { "dbengine_stats", 0, WEB_CLIENT_ACL_DASHBOARD | WEB_CLIENT_ACL_ACLK, web_client_api_request_v1_dbengine_stats },
1687
1688 // terminator
1628 - { NULL, 0, WEB_CLIENT_ACL_NONE, NULL },
1689 + { NULL, 0, WEB_CLIENT_ACL_NONE, NULL },
1690 };
1691
1692 inline int web_client_api_request_v1(RRDHOST *host, struct web_client *w, char *url) {
web/server/static/static-threaded.c
+2 -2
@@ -196,7 +196,7 @@ static void *web_server_add_callback(POLLINFO *pi, short int *events, void *data
196 }
197
198 #ifdef ENABLE_HTTPS
199 - if ((!web_client_check_unix(w)) && ( netdata_srv_ctx )) {
199 + if ((!web_client_check_unix(w)) && (netdata_ssl_srv_ctx)) {
200 if( sock_delnonblock(w->ifd) < 0 ){
201 error("Web server cannot remove the non-blocking flag from socket %d",w->ifd);
202 }
@@ -218,7 +218,7 @@ static void *web_server_add_callback(POLLINFO *pi, short int *events, void *data
218 //The next two ifs are not together because I am reusing SSL structure
219 if (!w->ssl.conn)
220 {
221 - w->ssl.conn = SSL_new(netdata_srv_ctx);
221 + w->ssl.conn = SSL_new(netdata_ssl_srv_ctx);
222 if ( w->ssl.conn ) {
223 SSL_set_accept_state(w->ssl.conn);
224 } else {
web/server/web_client.c
+4 -4
@@ -1018,7 +1018,7 @@ static inline HTTP_VALIDATION http_request_validate(struct web_client *w) {
1018 // TODO -- ideally we we should avoid copying buffers around
1019 snprintfz(w->last_url, NETDATA_WEB_REQUEST_URL_SIZE, "%s%s", w->decoded_url, w->decoded_query_string);
1020 #ifdef ENABLE_HTTPS
1021 - if ( (!web_client_check_unix(w)) && (netdata_srv_ctx) ) {
1021 + if ( (!web_client_check_unix(w)) && (netdata_ssl_srv_ctx) ) {
1022 if ((w->ssl.conn) && ((w->ssl.flags & NETDATA_SSL_NO_HANDSHAKE) && (web_client_is_using_ssl_force(w) || web_client_is_using_ssl_default(w)) && (w->mode != WEB_CLIENT_MODE_STREAM)) ) {
1023 w->header_parse_tries = 0;
1024 w->header_parse_last_size = 0;
@@ -1054,7 +1054,7 @@ static inline ssize_t web_client_send_data(struct web_client *w,const void *buf,
1054 {
1055 ssize_t bytes;
1056 #ifdef ENABLE_HTTPS
1057 - if ( (!web_client_check_unix(w)) && (netdata_srv_ctx) ) {
1057 + if ( (!web_client_check_unix(w)) && (netdata_ssl_srv_ctx) ) {
1058 if ( ( w->ssl.conn ) && ( !w->ssl.flags ) ){
1059 bytes = SSL_write(w->ssl.conn,buf, len) ;
1060 } else {
@@ -1211,7 +1211,7 @@ static inline void web_client_send_http_header(struct web_client *w) {
1211 size_t count = 0;
1212 ssize_t bytes;
1213 #ifdef ENABLE_HTTPS
1214 - if ( (!web_client_check_unix(w)) && (netdata_srv_ctx) ) {
1214 + if ( (!web_client_check_unix(w)) && (netdata_ssl_srv_ctx) ) {
1215 if ( ( w->ssl.conn ) && ( !w->ssl.flags ) ){
1216 while((bytes = SSL_write(w->ssl.conn, buffer_tostring(w->response.header_output), buffer_strlen(w->response.header_output))) < 0) {
1217 count++;
@@ -1915,7 +1915,7 @@ ssize_t web_client_receive(struct web_client *w)
1915 buffer_need_bytes(w->response.data, NETDATA_WEB_REQUEST_RECEIVE_SIZE);
1916
1917 #ifdef ENABLE_HTTPS
1918 - if ( (!web_client_check_unix(w)) && (netdata_srv_ctx) ) {
1918 + if ( (!web_client_check_unix(w)) && (netdata_ssl_srv_ctx) ) {
1919 if ( ( w->ssl.conn ) && (!w->ssl.flags)) {
1920 bytes = SSL_read(w->ssl.conn, &w->response.data->buffer[w->response.data->len], (size_t) (left - 1));
1921 }else {
web/server/web_client.h
+1
@@ -27,6 +27,7 @@ extern int web_enable_gzip, web_gzip_level, web_gzip_strategy;
27 #define HTTP_RESP_INTERNAL_SERVER_ERROR 500
28 #define HTTP_RESP_BACKEND_FETCH_FAILED 503
29 #define HTTP_RESP_GATEWAY_TIMEOUT 504
30 +#define HTTP_RESP_BACKEND_RESPONSE_INVALID 591
31
32 extern int respect_web_browser_do_not_track_policy;
33 extern char *web_x_frame_options;
web/server/web_client_cache.c
+2 -2
@@ -9,7 +9,7 @@
9 #ifdef ENABLE_HTTPS
10
11 static void web_client_reuse_ssl(struct web_client *w) {
12 - if (netdata_srv_ctx) {
12 + if (netdata_ssl_srv_ctx) {
13 if (w->ssl.conn) {
14 SSL_clear(w->ssl.conn);
15 }
@@ -48,7 +48,7 @@ static void web_client_free(struct web_client *w) {
48 buffer_free(w->response.data);
49 freez(w->user_agent);
50 #ifdef ENABLE_HTTPS
51 - if ((!web_client_check_unix(w)) && ( netdata_srv_ctx )) {
51 + if ((!web_client_check_unix(w)) && (netdata_ssl_srv_ctx)) {
52 if (w->ssl.conn) {
53 SSL_free(w->ssl.conn);
54 w->ssl.conn = NULL;