@cryptotaxi247 / netdata-1 / commits / b2b3c1825

ACLK agent 1 (#7894)

* - Add initial mqtt support * [WIP] Agent cloud link - Setup main mqtt thread to connect to a broker using V5 of the MQTT protocol (TBD) - Send alarms to "netdata/alarm" - Add error checks to handle connection failures - Add params for Broker, port Maximum concurrent sent / recev messages - Dummy function to check claiming status - Generic mqtt_send command to publish message to a base topic , sub topic It will end up in the form base_topic/sub_topic - Add host/port in the connection failure error message * Test libmosquitto libs * connect to broker locally (assume localhost:1883) * subscribe to channel netdata/command * Test try a reload command to trigger health reload * publish alerts to netdata/alarm * - Fix compile issues * - Use sleep_usec instead of usleep * - Delay reconnection on failure due to misconfiguration (high cpu usage) * - Remove the TLS connection config * - Fix NETDATA_MQTT_INITIALIZATION_SLEEP_WAIT to use seconds * - Gather ACLK related code under aclk folder - Add aclk_ functions for abstract layer - Moved low level libs intergration in mqtt.c * - Add README.md file with initial comment * - Clean MQTT v5 * - Code cleanup * - Remove alarm log for now - Remove the heart beat * - Remove message properties for V5 * - Remove message properties for V5 (header) * Fixed the netdata target to use a local static version of libmosquitto. The installer does not yet have steps to pull and build the local library. cd project_root git clone ssh://git@github.com/netdata/mosquitto mosquitto/ (cd mosquitto/lib && make) # Ignore the cpp error This will leave mosquitto/lib/libmosquitto.a for the build process to use. * - Fix compile issues with older < 1.6 libmosquitto lib * - Enable alarm events to check it works - Re arrange includes - Rework topic to be agent/guid/. Actual id will be returned by the is_agent_claimed * - Add initial metadata info - Added helper function in web_api - Added a debug command (info) * Update the claiming state to retrieve the claimed id. * - Use define for constants like command and metadata topics - Function to wait for initialization of the ACLK link - New aclk_subscribe command with QOS parameter for the mqtt subscription - Use the is_agent_claimed function to get the real claim id and use it to build the topics that will be used for the cloud communication - Change in netdata-claim.sh.in to write the claim id without a trailing \n * - Use define for constants like command and metadata topics - Function to wait for initialization of the ACLK link - New aclk_subscribe command with QOS parameter for the mqtt subscription - Use the is_agent_claimed function to get the real claim id and use it to build the topics that will be used for the cloud communication - Change in netdata-claim.sh.in to write the claim id without a trailing \n * - Remove the alarm log for now - Add code (but disabled) to send charts * - Use dummy anon, anon as username and password for testing purposes * - Use client id anon as well * Testing without TLS * Switching TLS back on to fix docker environment. * - Added query processing An incoming URL now calls web_client_api_request_v1_data to handle a request and push the results back to the "data" topic - Move the above processing from the message callback to the query handle loop - Added helper "pause" , "resume" commands to stop and resume query processing to stress test loading the queue with queries before executing them - Changed the endpoint topics to "meta", and "cmd" (previously metadata and command) * make info message follow protocol * move metadata msg generation into new func * move metadata msg generation into new func * - Add metadata to the responses - Add hook to queue chart changes on creation and dimensions - Changed the queue mechanism to include delay for X seconds - Add delayed submittion of charts to the cloud so that all DIMs are defined to avoid resubmission * - Add additional data info for aclk_queue command * - Use web_clinet_api_request_v1 to handle the incoming request This will handle all requests coming from the cloud * - Cleanup and aclk_query structure - Add msg_id parameter - Enable the incoming JSON request - Enable the outgoing JSON response * - Added new thread to handle query processing - Add lock and cond wait to wakeup thread when queries are submitted - Cleanup on the main init function * - Add wait time on agent init, to allow for chart, alarms and other definitions to be completed. - During the wait time, no queries will be queued * - Send metadata on query thread init - New generic create header function for the JSON response - Pack info and charts into one message - Modified chart to remove entries (test) - Modified charts mod to remove entries e.g alarms and volatile info - Change input to aclk_update_chart (RRDHOST / instead of hostname) * - When a request fails, add to the payload - We may need to handle in a different key - Error check in json parsing * - Add dummy aclk_update_alarm command * - Move incoming request JSON parsing code away from mqtt.c - Added #ifdef ACLK_ENABLE so that we can have code merged but disabled by default - Added version in incoming and outgoing JSON dict * - Disable code if ACLK_ENABLE is not defined - Remove references to the mqtt (mosquitto) lib - Add dummy stubs in mqtt.c for completeness if ACLK_ENABLE is not defined * - Disable challenge sample code for now * - Remove libmosquitto from makefile * - Fix spaces in Makefile.am - Remove ifdef to avoid warning from LGTM * - Remove for now the code that builds an along log test message to send to the cloud * - Add check for ACLK_ENABLE definition and avoid calling the chart update functions * - Remove commented code * - Move source files to the correct place (ACLK_PLUGIN_FILES) * - Remove include file thats not needed * - Remove include file thats not needed - Add improved checks for load_claiming_state() * - Fix error message. Used error() that also logs errno and message * - Fix some codacy issues * - Fix more codacy issues, code cleanup * - Revert code to address codacy warnings * - Revert spaces added in a previous commit by mistake * clean up if/else nest * print error if fopen fails * minor - error already logs errno * - Fix version formatting * - Cleanup all ACLK related compiler warnings - Re-arrange include files - Removed unused defines * - More compilation warnings fixed - Bug with thread creation fixed * - Add condition to skip compilation of the ACLK code entirely. Add env variable ACLK="yes" to enable * - Add condition to skip the libmosquitto * - Change feature flag from ACLK_ENABLE to ENABLE_ACLK in accordance with the rest of ENABLE_xx flags - Typo in info message fix Co-authored-by: Andrew Moss <1043609+amoss@users.noreply.github.com> Co-authored-by: Timo <6674623+underhood@users.noreply.github.com>

Stelios Fragkakis committed Feb 6, 2020 at 17:58 UTC b2b3c182548fe81e6d1c9a599b2571dabfdabcaa
19 files changed +1587 -32
CMakeLists.txt
+10 -2
@@ -339,8 +339,7 @@ set(LIBNETDATA_FILES
339 add_library(libnetdata OBJECT ${LIBNETDATA_FILES})
340
341 set(APPS_PLUGIN_FILES
342 - collectors/apps.plugin/apps_plugin.c
343 - )
342 + collectors/apps.plugin/apps_plugin.c)
343
344 set(CHECKS_PLUGIN_FILES
345 collectors/checks.plugin/plugin_checks.c
@@ -606,6 +605,14 @@ set(CLAIM_PLUGIN_FILES
605 claim/claim.h
606 )
607
608 +set(ACLK_PLUGIN_FILES
609 + aclk/agent_cloud_link.c
610 + aclk/agent_cloud_link.h
611 + aclk/agent_cloud_link.c
612 + aclk/mqtt.c
613 + aclk/mqtt.h
614 + )
615 +
616 set(EXPORTING_ENGINE_FILES
617 exporting/exporting_engine.c
618 exporting/exporting_engine.h
@@ -673,6 +680,7 @@ set(NETDATA_FILES
680 ${STREAMING_PLUGIN_FILES}
681 ${WEB_PLUGIN_FILES}
682 ${CLAIM_PLUGIN_FILES}
683 + ${ACLK_PLUGIN_FILES}
684 )
685
686 set(NETDATACLI_FILES
Makefile.am
+25
@@ -100,6 +100,7 @@ SUBDIRS += \
100 streaming \
101 web \
102 claim \
103 + aclk \
104 $(NULL)
105
106
@@ -108,6 +109,7 @@ AM_CFLAGS = \
109 $(OPTIONAL_NFACCT_CFLAGS) \
110 $(OPTIONAL_ZLIB_CFLAGS) \
111 $(OPTIONAL_UUID_CFLAGS) \
112 + $(OPTIONAL_MQTT_CFLAGS) \
113 $(OPTIONAL_LIBCAP_LIBS) \
114 $(OPTIONAL_IPMIMONITORING_CFLAGS) \
115 $(OPTIONAL_CUPS_CFLAGS) \
@@ -456,6 +458,13 @@ CLAIM_PLUGIN_FILES = \
458 claim/claim.h \
459 $(NULL)
460
461 +ACLK_PLUGIN_FILES = \
462 + aclk/agent_cloud_link.c \
463 + aclk/agent_cloud_link.h \
464 + aclk/mqtt.c \
465 + aclk/mqtt.h \
466 + $(NULL)
467 +
468 EXPORTING_ENGINE_FILES = \
469 exporting/exporting_engine.c \
470 exporting/exporting_engine.h \
@@ -526,6 +535,12 @@ NETDATA_FILES = \
535 $(CLAIM_PLUGIN_FILES) \
536 $(NULL)
537
538 +if ENABLE_ACLK
539 + NETDATA_FILES += \
540 + $(ACLK_PLUGIN_FILES) \
541 + $(NULL)
542 +endif
543 +
544 if FREEBSD
545 NETDATA_FILES += \
546 $(FREEBSD_PLUGIN_FILES) \
@@ -559,6 +574,7 @@ NETDATA_COMMON_LIBS = \
574 $(OPTIONAL_ZLIB_LIBS) \
575 $(OPTIONAL_SSL_LIBS) \
576 $(OPTIONAL_UUID_LIBS) \
577 + $(OPTIONAL_MQTT_LIBS) \
578 $(OPTIONAL_UV_LIBS) \
579 $(OPTIONAL_LZ4_LIBS) \
580 $(OPTIONAL_JUDY_LIBS) \
@@ -575,9 +591,18 @@ NETDATACLI_FILES = \
591
592 sbin_PROGRAMS += netdata
593 netdata_SOURCES = $(NETDATA_FILES)
594 +
595 +if ENABLE_ACLK
596 netdata_LDADD = \
597 + mosquitto/lib/libmosquitto.a \
598 $(NETDATA_COMMON_LIBS) \
599 $(NULL)
600 +else
601 +netdata_LDADD = \
602 + $(NETDATA_COMMON_LIBS) \
603 + $(NULL)
604 +endif
605 +
606 if ENABLE_CXX_LINKER
607 netdata_LINK = $(CXXLD) $(CXXFLAGS) $(LDFLAGS) -o $@
608 else
aclk/Makefile.am new
+18
@@ -0,0 +1,18 @@
1 +# SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +AUTOMAKE_OPTIONS = subdir-objects
4 +MAINTAINERCLEANFILES = $(srcdir)/Makefile.in
5 +
6 +CLEANFILES = \
7 + $(NULL)
8 +
9 +include $(top_srcdir)/build/subst.inc
10 +SUFFIXES = .in
11 +
12 +sbin_SCRIPTS = \
13 + $(NULL)
14 +
15 +dist_noinst_DATA = \
16 + README.md \
17 + $(NULL)
18 +
aclk/README.md new
+1
@@ -0,0 +1 @@
1 +This is the agent cloud link (ACLK) information file
\ No newline at end of file
aclk/agent_cloud_link.c new
+995
@@ -0,0 +1,995 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#include "libnetdata/libnetdata.h"
4 +#include "agent_cloud_link.h"
5 +
6 +// Read from the config file -- new section [agent_cloud_link]
7 +// Defaults are supplied
8 +int aclk_recv_maximum = 0; // default 20
9 +int aclk_send_maximum = 0; // default 20
10 +
11 +int aclk_port = 0; // default 1883
12 +char *aclk_hostname = NULL; //default localhost
13 +int aclk_subscribed = 0;
14 +
15 +int aclk_metadata_submitted = 0;
16 +int waiting_init = 1;
17 +int cmdpause = 0; // Used to pause query processing
18 +
19 +BUFFER *aclk_buffer = NULL;
20 +char *global_base_topic = NULL;
21 +
22 +int cloud_to_agent_parse(JSON_ENTRY *e)
23 +{
24 + struct aclk_request *data = e->callback_data;
25 +
26 + switch(e->type) {
27 + case JSON_OBJECT:
28 + e->callback_function = cloud_to_agent_parse;
29 + break;
30 + case JSON_ARRAY:
31 + e->callback_function = cloud_to_agent_parse;
32 + break;
33 + case JSON_STRING:
34 + if (!strcmp(e->name, ACLK_JSON_IN_MSGID)) {
35 + data->msg_id = strdupz(e->data.string);
36 + break;
37 + }
38 + if (!strcmp(e->name, ACLK_JSON_IN_TYPE)) {
39 + data->type_id = strdupz(e->data.string);
40 + break;
41 + }
42 + if (!strcmp(e->name, ACLK_JSON_IN_TOPIC)) {
43 + data->topic = strdupz(e->data.string);
44 + break;
45 + }
46 + if (!strcmp(e->name, ACLK_JSON_IN_URL)) {
47 + data->url = strdupz(e->data.string);
48 + break;
49 + }
50 + break;
51 + case JSON_NUMBER:
52 + if (!strcmp(e->name, ACLK_JSON_IN_VERSION)) {
53 + data->version = atol(e->data.string);
54 + break;
55 + }
56 + break;
57 +
58 + case JSON_BOOLEAN:
59 + break;
60 +
61 + case JSON_NULL:
62 + break;
63 + }
64 + return 0;
65 +}
66 +
67 +//char *send_http_request(char *host, char *port, char *url, BUFFER *b)
68 +//{
69 +// struct timeval timeout = { .tv_sec = 30, .tv_usec = 0 };
70 +//
71 +// buffer_flush(b);
72 +// buffer_sprintf(
73 +// b,
74 +// "GET %s HTTP/1.1\r\nHost: %s\r\nAccept: plain/text\r\nAccept-Language: en-us\r\nUser-Agent: Netdata/rocks\r\n\r\n",
75 +// url, host);
76 +// int sock = connect_to_this_ip46(IPPROTO_TCP, SOCK_STREAM, host, 0, "443", &timeout);
77 +//
78 +// if (unlikely(sock == -1)) {
79 +// error("Handshake failed");
80 +// return NULL;
81 +// }
82 +//
83 +// SSL_CTX *ctx = security_initialize_openssl_client();
84 +// // Certificate chain: not updating the stores - do we need private CA roots?
85 +// // Calls to SSL_CTX_load_verify_locations would go here.
86 +// SSL *ssl = SSL_new(ctx);
87 +// SSL_set_fd(ssl, sock);
88 +// int err = SSL_connect(ssl);
89 +// SSL_write(ssl, b->buffer, b->len); // Timeout options?
90 +// int bytes_read = SSL_read(ssl, b->buffer, b->len);
91 +// SSL_shutdown(ssl);
92 +// close(sock);
93 +//}
94 +
95 +// Set when we have connection up and running from the connection callback
96 +int aclk_connection_initialized = 0;
97 +
98 +static netdata_mutex_t aclk_mutex = NETDATA_MUTEX_INITIALIZER;
99 +static netdata_mutex_t query_mutex = NETDATA_MUTEX_INITIALIZER;
100 +
101 +#define ACLK_LOCK netdata_mutex_lock(&aclk_mutex)
102 +#define ACLK_UNLOCK netdata_mutex_unlock(&aclk_mutex)
103 +
104 +#define QUERY_LOCK netdata_mutex_lock(&query_mutex)
105 +#define QUERY_UNLOCK netdata_mutex_unlock(&query_mutex)
106 +
107 +pthread_cond_t query_cond_wait = PTHREAD_COND_INITIALIZER;
108 +pthread_mutex_t query_lock_wait = PTHREAD_MUTEX_INITIALIZER;
109 +
110 +#define QUERY_THREAD_LOCK pthread_mutex_lock(&query_lock_wait);
111 +#define QUERY_THREAD_UNLOCK pthread_mutex_unlock(&query_lock_wait)
112 +#define QUERY_THREAD_WAKEUP pthread_cond_signal(&query_cond_wait)
113 +
114 +
115 +struct aclk_query {
116 + time_t created;
117 + time_t run_after; // Delay run until after this time
118 + char *topic; // Topic to respond to
119 + char *data; // Internal data (NULL if request from the cloud)
120 + char *msg_id; // msg_id generated by the cloud (NULL if internal)
121 + char *query; // The actual query
122 + u_char deleted; // Mark deleted for garbage collect
123 + struct aclk_query *next;
124 +};
125 +
126 +struct aclk_query_queue {
127 + struct aclk_query *aclk_query_head;
128 + struct aclk_query *aclk_query_tail;
129 + u_int64_t count;
130 +} aclk_queue = { .aclk_query_head = NULL, .aclk_query_tail = NULL, .count = 0 };
131 +
132 +/*
133 + * Free a query structure when done
134 + */
135 +
136 +void aclk_query_free(struct aclk_query *this_query)
137 +{
138 + if (unlikely(!this_query))
139 + return;
140 +
141 + freez(this_query->topic);
142 + freez(this_query->query);
143 + if (this_query->data)
144 + freez(this_query->data);
145 + if (this_query->msg_id)
146 + freez(this_query->msg_id);
147 + freez(this_query);
148 + return;
149 +}
150 +
151 +// Returns the entry after which we need to create a new entry to run at the specified time
152 +// If NULL is returned we need to add to HEAD
153 +// Called with locked entries
154 +
155 +struct aclk_query *aclk_query_find_position(time_t time_to_run)
156 +{
157 + struct aclk_query *tmp_query, *last_query;
158 +
159 + last_query = NULL;
160 + tmp_query = aclk_queue.aclk_query_head;
161 +
162 + while (tmp_query) {
163 + if (tmp_query->run_after > time_to_run)
164 + return last_query;
165 + last_query = tmp_query;
166 + tmp_query = tmp_query->next;
167 + }
168 + return last_query;
169 +}
170 +
171 +// Need to have a lock before calling this
172 +struct aclk_query *aclk_query_find(char *topic, char *data, char *msg_id, char *query)
173 +{
174 + struct aclk_query *tmp_query;
175 +
176 + tmp_query = aclk_queue.aclk_query_head;
177 +
178 + while (tmp_query) {
179 + if (likely(!tmp_query->deleted)) {
180 + if (strcmp(tmp_query->topic, topic) == 0 && (strcmp(tmp_query->query, query) == 0)) {
181 + if ((!data || (data && strcmp(data, tmp_query->data) == 0)) &&
182 + (!msg_id || (msg_id && strcmp(msg_id, tmp_query->msg_id) == 0)))
183 + return tmp_query;
184 + }
185 + }
186 + tmp_query = tmp_query->next;
187 + }
188 + return NULL;
189 +}
190 +
191 +/*
192 + * Add a query to execute, the result will be send to the specified topic
193 + */
194 +
195 +int aclk_queue_query(char *topic, char *data, char *msg_id, char *query, int run_after, int internal)
196 +{
197 + struct aclk_query *new_query, *tmp_query;
198 +
199 + // Ignore all commands while we wait for the agent to initialize
200 + if (unlikely(waiting_init))
201 + return 0;
202 +
203 + run_after = now_realtime_sec() + run_after;
204 +
205 + QUERY_LOCK;
206 + tmp_query = aclk_query_find(topic, data, msg_id, query);
207 + if (unlikely(tmp_query)) {
208 + if (tmp_query->run_after == run_after) {
209 + QUERY_UNLOCK;
210 + QUERY_THREAD_WAKEUP;
211 + return 0;
212 + }
213 + tmp_query->deleted = 1;
214 + }
215 +
216 + new_query = callocz(1, sizeof(struct aclk_query));
217 + if (internal) {
218 + new_query->topic = strdupz(topic);
219 + new_query->query = strdupz(query);
220 + } else {
221 + new_query->topic = topic;
222 + new_query->query = query;
223 + new_query->msg_id = msg_id;
224 + }
225 +
226 + if (data)
227 + new_query->data = strdupz(data);
228 +
229 + new_query->next = NULL;
230 + new_query->created = now_realtime_sec();
231 + new_query->run_after = run_after;
232 +
233 + info("Added query (%s) (%s)", topic, query);
234 +
235 + tmp_query = aclk_query_find_position(run_after);
236 +
237 + if (tmp_query) {
238 + new_query->next = tmp_query->next;
239 + tmp_query->next = new_query;
240 + if (tmp_query == aclk_queue.aclk_query_tail)
241 + aclk_queue.aclk_query_tail = new_query;
242 + aclk_queue.count++;
243 + QUERY_UNLOCK;
244 + QUERY_THREAD_WAKEUP;
245 + return 0;
246 + }
247 +
248 + new_query->next = aclk_queue.aclk_query_head;
249 + aclk_queue.aclk_query_head = new_query;
250 + aclk_queue.count++;
251 +
252 + QUERY_UNLOCK;
253 + QUERY_THREAD_WAKEUP;
254 + return 0;
255 +
256 +// if (likely(aclk_queue.aclk_query_tail)) {
257 +// aclk_queue.aclk_query_tail->next = new_query;
258 +// aclk_queue.aclk_query_tail = new_query;
259 +// aclk_queue.count++;
260 +// QUERY_UNLOCK;
261 +// return 0;
262 +// }
263 +//
264 +// if (likely(!aclk_queue.aclk_query_head)) {
265 +// aclk_queue.aclk_query_head = new_query;
266 +// aclk_queue.aclk_query_tail = new_query;
267 +// aclk_queue.count++;
268 +// QUERY_UNLOCK;
269 +// return 0;
270 +// }
271 +// QUERY_UNLOCK;
272 +// return 0;
273 +}
274 +
275 +inline int aclk_submit_request(struct aclk_request *request)
276 +{
277 + return aclk_queue_query(request->topic, NULL, request->msg_id, request->url, 0, 0);
278 +}
279 +
280 +/*
281 + * Get the next query to process - NULL if nothing there
282 + * The caller needs to free memory by calling aclk_query_free()
283 + *
284 + * topic
285 + * query
286 + * The structure itself
287 + *
288 + */
289 +struct aclk_query *aclk_queue_pop()
290 +{
291 + struct aclk_query *this_query;
292 +
293 + QUERY_LOCK;
294 +
295 + if (likely(!aclk_queue.aclk_query_head)) {
296 + QUERY_UNLOCK;
297 + return NULL;
298 + }
299 +
300 + this_query = aclk_queue.aclk_query_head;
301 +
302 + if (this_query->run_after > now_realtime_sec()) {
303 + info("Query %s will run in %ld seconds", this_query->query, this_query->run_after - now_realtime_sec());
304 + QUERY_UNLOCK;
305 + return NULL;
306 + }
307 +
308 + aclk_queue.count--;
309 + aclk_queue.aclk_query_head = aclk_queue.aclk_query_head->next;
310 +
311 + if (likely(!aclk_queue.aclk_query_head)) {
312 + aclk_queue.aclk_query_tail = NULL;
313 + }
314 +
315 + QUERY_UNLOCK;
316 + return this_query;
317 +}
318 +
319 +// This will give the base topic that the agent will publish messages.
320 +// subtopics will be sent under the base topic e.g. base_topic/subtopic
321 +// This is called by aclk_init(), to compute the base topic once and have
322 +// it stored internally.
323 +// Need to check if additional logic should be added to make sure that there
324 +// is enough information to determine the base topic at init time
325 +
326 +// TODO: Locking may be needed, depends on the calculation of the base topic and also if we need to switch
327 +// that on the fly
328 +
329 +char *get_publish_base_topic(PUBLISH_TOPIC_ACTION action)
330 +{
331 + static char *topic = NULL;
332 +
333 + if (unlikely(!is_agent_claimed()))
334 + return NULL;
335 +
336 + ACLK_LOCK;
337 +
338 + if (unlikely(action == PUBLICH_TOPIC_FREE)) {
339 + if (likely(topic)) {
340 + freez(topic);
341 + topic = NULL;
342 + }
343 +
344 + ACLK_UNLOCK;
345 +
346 + return NULL;
347 + }
348 +
349 + if (unlikely(action == PUBLICH_TOPIC_REBUILD)) {
350 + ACLK_UNLOCK;
351 + get_publish_base_topic(PUBLICH_TOPIC_FREE);
352 + return get_publish_base_topic(PUBLICH_TOPIC_GET);
353 + }
354 +
355 + if (unlikely(!topic)) {
356 + char tmp_topic[ACLK_MAX_TOPIC + 1];
357 +
358 + sprintf(tmp_topic, ACLK_TOPIC_STRUCTURE, is_agent_claimed());
359 + topic = strdupz(tmp_topic);
360 + }
361 +
362 + ACLK_UNLOCK;
363 + return topic;
364 +}
365 +
366 +char *get_topic(char *sub_topic, char *final_topic, int max_size)
367 +{
368 + if (unlikely(!global_base_topic))
369 + global_base_topic = GET_PUBLISH_BASE_TOPIC;
370 +
371 + if (unlikely(!global_base_topic))
372 + return sub_topic;
373 +
374 + snprintfz(final_topic, max_size, "%s/%s", global_base_topic, sub_topic);
375 +
376 + return final_topic;
377 +}
378 +
379 +// Wait for ACLK connection to be established
380 +int aclk_wait_for_initialization()
381 +{
382 + if (unlikely(!aclk_connection_initialized)) {
383 + time_t now = now_realtime_sec();
384 +
385 + while (!aclk_connection_initialized && (now_realtime_sec() - now) < ACLK_INITIALIZATION_WAIT) {
386 + sleep_usec(USEC_PER_SEC * ACLK_INITIALIZATION_SLEEP_WAIT);
387 + _link_event_loop(0);
388 + }
389 +
390 + if (unlikely(!aclk_connection_initialized)) {
391 + error("ACLK connection cannot be established");
392 + return 1;
393 + }
394 + }
395 + return 0;
396 +}
397 +
398 +/*
399 + * This function will fetch the next pending command and process it
400 + *
401 + */
402 +int aclk_process_query()
403 +{
404 + struct aclk_query *this_query;
405 + static u_int64_t query_count = 0;
406 + //int rc;
407 +
408 + if (unlikely(cmdpause))
409 + return 0;
410 +
411 + if (!aclk_connection_initialized)
412 + return 0;
413 +
414 + this_query = aclk_queue_pop();
415 + if (likely(!this_query)) {
416 + //info("No pending queries");
417 + return 0;
418 + }
419 +
420 + if (unlikely(this_query->deleted)) {
421 + info("Garbage collect query %s:%s", this_query->topic, this_query->query);
422 + aclk_query_free(this_query);
423 + return 1;
424 + }
425 +
426 + query_count++;
427 + info(
428 + "Query #%d (%s) (%s) in queue %d seconds", (int) query_count, this_query->topic, this_query->query,
429 + (int) (now_realtime_sec() - this_query->created));
430 +
431 + if (strncmp((char *)this_query->query, "/api/v1/", 8) == 0) {
432 + struct web_client *w = (struct web_client *)callocz(1, sizeof(struct web_client));
433 + w->response.data = buffer_create(NETDATA_WEB_RESPONSE_INITIAL_SIZE);
434 + strcpy(w->origin, "*"); // Simulate web_client_create_on_fd()
435 + w->cookie1[0] = 0; // Simulate web_client_create_on_fd()
436 + w->cookie2[0] = 0; // Simulate web_client_create_on_fd()
437 + w->acl = 0x1f;
438 +
439 + char *mysep = strchr(this_query->query, '?');
440 + if (mysep) {
441 + strncpyz(w->decoded_query_string, mysep, NETDATA_WEB_REQUEST_URL_SIZE);
442 + *mysep = '\0';
443 + } else
444 + strncpyz(w->decoded_query_string, this_query->query, NETDATA_WEB_REQUEST_URL_SIZE);
445 +
446 + mysep = strrchr(this_query->query, '/');
447 +
448 + // TODO: ignore return code for now
449 + web_client_api_request_v1(localhost, w, mysep ? mysep + 1 : "noop");
450 +
451 + //TODO: handle bad response perhaps in a different way. For now it does to the payload
452 + //if (rc == HTTP_RESP_OK || 1) {
453 + buffer_flush(aclk_buffer);
454 +
455 + aclk_create_metadata_message(aclk_buffer, mysep ? mysep + 1 : "noop", this_query->msg_id, w->response.data);
456 + aclk_buffer->contenttype = CT_APPLICATION_JSON;
457 + aclk_send_message(this_query->topic, aclk_buffer->buffer);
458 + //} else
459 + // error("Query RESP: %s", w->response.data->buffer);
460 +
461 + buffer_free(w->response.data);
462 + freez(w);
463 + aclk_query_free(this_query);
464 + return 1;
465 + }
466 +
467 + if (strcmp((char *)this_query->topic, "_chart") == 0) {
468 + aclk_send_single_chart(this_query->data, this_query->query);
469 + }
470 +
471 + aclk_query_free(this_query);
472 +
473 + return 1;
474 +}
475 +
476 +// Launch a query processing thread
477 +
478 +/*
479 + * Process all pending queries
480 + * Return 0 if no queries were processed, 1 otherwise
481 + *
482 + */
483 +
484 +int aclk_process_queries()
485 +{
486 + if (unlikely(cmdpause))
487 + return 0;
488 +
489 + // Return if no queries pending
490 + if (likely(!aclk_queue.count))
491 + return 0;
492 +
493 + info("Processing %d queries", (int ) aclk_queue.count);
494 +
495 + while (aclk_process_query()) {
496 + //rc = _link_event_loop(0);
497 + };
498 +
499 + return 1;
500 +}
501 +
502 +static void aclk_query_thread_cleanup(void *ptr)
503 +{
504 + struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr;
505 + static_thread->enabled = NETDATA_MAIN_THREAD_EXITING;
506 +
507 + info("cleaning up...");
508 +
509 + static_thread->enabled = NETDATA_MAIN_THREAD_EXITED;
510 +}
511 +
512 +/**
513 + * MAin query processing thread
514 + *
515 + */
516 +void *aclk_query_main_thread(void *ptr)
517 +{
518 + netdata_thread_cleanup_push(aclk_query_thread_cleanup, ptr);
519 +
520 + while (!netdata_exit) {
521 +
522 + QUERY_THREAD_LOCK;
523 +
524 + if (unlikely(!aclk_metadata_submitted)) {
525 + aclk_send_metadata();
526 + aclk_metadata_submitted = 1;
527 + }
528 +
529 + if (unlikely(pthread_cond_wait(&query_cond_wait, &query_lock_wait)))
530 + sleep_usec(USEC_PER_SEC * 1);
531 +
532 + if (likely(aclk_connection_initialized && !netdata_exit)) {
533 + while (aclk_process_queries()) {
534 + // Sleep for a few ms and retry maybe we have something to process
535 + // before going to sleep
536 + // TODO: This needs improvement to avoid missed queries
537 + sleep_usec(USEC_PER_MS * 100);
538 + }
539 + }
540 +
541 + QUERY_THREAD_UNLOCK;
542 +
543 + } // forever
544 + info("Shutting down query processing thread");
545 + netdata_thread_cleanup_pop(1);
546 + return NULL;
547 +}
548 +
549 +// Thread cleanup
550 +static void aclk_main_cleanup(void *ptr)
551 +{
552 + struct netdata_static_thread *static_thread = (struct netdata_static_thread *)ptr;
553 + static_thread->enabled = NETDATA_MAIN_THREAD_EXITING;
554 +
555 + info("cleaning up...");
556 +
557 + QUERY_THREAD_WAKEUP;
558 +
559 + static_thread->enabled = NETDATA_MAIN_THREAD_EXITED;
560 +}
561 +
562 +/**
563 + * Main agent cloud link thread
564 + *
565 + * This thread will simply call the main event loop that handles
566 + * pending requests - both inbound and outbound
567 + *
568 + * @param ptr is a pointer to the netdata_static_thread structure.
569 + *
570 + * @return It always returns NULL
571 + */
572 +void *aclk_main(void *ptr)
573 +{
574 + //netdata_thread_t *query_thread;
575 + struct netdata_static_thread query_thread;
576 +
577 + memset(&query_thread, 0, sizeof(query_thread));
578 +
579 + netdata_thread_cleanup_push(aclk_main_cleanup, ptr);
580 +
581 + if (unlikely(!aclk_buffer))
582 + aclk_buffer = buffer_create(NETDATA_WEB_RESPONSE_INITIAL_SIZE);
583 +
584 + assert(aclk_buffer != NULL);
585 +
586 + //netdata_thread_cleanup_push(aclk_query_thread_cleanup, ptr);
587 + //netdata_thread_create(&query_thread.thread , "ACLKQ", NETDATA_THREAD_OPTION_DEFAULT, aclk_query_main_thread, &query_thread);
588 + info("Waiting for netdata to be ready");
589 + while (!netdata_ready) {
590 + sleep_usec(USEC_PER_MS * 300);
591 + }
592 + info("Waiting %d seconds for the agent to initialize", ACLK_STARTUP_WAIT);
593 + sleep_usec(USEC_PER_SEC * ACLK_STARTUP_WAIT);
594 +
595 + // Ok mark we are ready to accept incoming requests
596 + waiting_init = 0;
597 +
598 + while (!netdata_exit) {
599 + // TODO: This may change when we have enough info from the claiming itself to avoid wasting 60 seconds
600 + // TODO: Handle the unclaim command as well -- we may need to shutdown the connection
601 + if (likely(!is_agent_claimed())) {
602 + sleep_usec(USEC_PER_SEC * 60);
603 + info("Checking agent claiming status");
604 + continue;
605 + }
606 +
607 + if (unlikely(!aclk_connection_initialized)) {
608 + static int initializing = 0;
609 +
610 + if (likely(initializing)) {
611 + _link_event_loop(ACLK_LOOP_TIMEOUT * 1000);
612 + continue;
613 + }
614 + initializing = 1;
615 + info("Initializing connection");
616 + //send_http_request(aclk_hostname, "443", "/auth/challenge?id=blah", aclk_buffer);
617 + if (unlikely(aclk_init(ACLK_INIT))) {
618 + // TODO: TBD how to handle. We are claimed and we cant init the connection. For now keep trying.
619 + sleep_usec(USEC_PER_SEC * 60);
620 + continue;
621 + } else {
622 + sleep_usec(USEC_PER_SEC * 1);
623 + }
624 + _link_event_loop(ACLK_LOOP_TIMEOUT * 1000);
625 + continue;
626 + }
627 +
628 + if (unlikely(!aclk_subscribed)) {
629 + aclk_subscribed = !aclk_subscribe(ACLK_COMMAND_TOPIC, 2);
630 + }
631 + if (unlikely(!query_thread.thread)) {
632 + query_thread.thread = mallocz(sizeof(netdata_thread_t));
633 + netdata_thread_create(
634 + query_thread.thread, "ACLKQ", NETDATA_THREAD_OPTION_DEFAULT, aclk_query_main_thread, &query_thread);
635 + }
636 +
637 + //TODO: Check if there is a return code
638 + _link_event_loop(ACLK_LOOP_TIMEOUT * 1000);
639 +
640 + } // forever
641 + aclk_shutdown();
642 +
643 + netdata_thread_cleanup_pop(1);
644 + return NULL;
645 +}
646 +
647 +/*
648 + * Send a message to the cloud, using a base topic and sib_topic
649 + * The final topic will be in the form <base_topic>/<sub_topic>
650 + * If base_topic is missing then the global_base_topic will be used (if available)
651 + *
652 + */
653 +int aclk_send_message(char *sub_topic, char *message)
654 +{
655 + int rc;
656 + static int skip_due_to_shutdown = 0;
657 + char topic[ACLK_MAX_TOPIC + 1];
658 + char *final_topic;
659 +
660 + if (!aclk_connection_initialized)
661 + return 0;
662 +
663 + if (unlikely(netdata_exit)) {
664 + if (unlikely(!aclk_connection_initialized))
665 + return 1;
666 +
667 + ++skip_due_to_shutdown;
668 + if (unlikely(!(skip_due_to_shutdown % 100)))
669 + info("%d messages not sent -- shutdown in progress", skip_due_to_shutdown);
670 + return 1;
671 + }
672 +
673 + if (unlikely(!message))
674 + return 0;
675 +
676 + if (unlikely(aclk_wait_for_initialization()))
677 + return 1;
678 +
679 + final_topic = get_topic(sub_topic, topic, ACLK_MAX_TOPIC);
680 +
681 + ACLK_LOCK;
682 + rc = _link_send_message(final_topic, message);
683 + ACLK_UNLOCK;
684 +
685 + // TODO: Add better handling -- error will flood the logfile here
686 + if (unlikely(rc))
687 + error("Failed to send message, error code %d (%s)", rc, _link_strerror(rc));
688 +
689 + return rc;
690 +}
691 +
692 +/*
693 + * Subscribe to a topic in the cloud
694 + * The final subscription will be in the form
695 + * /agent/claim_id/<sub_topic>
696 + */
697 +int aclk_subscribe(char *sub_topic, int qos)
698 +{
699 + int rc;
700 + //static char *global_base_topic = NULL;
701 + char topic[ACLK_MAX_TOPIC + 1];
702 + char *final_topic;
703 +
704 + if (!aclk_connection_initialized)
705 + return 0;
706 +
707 + if (unlikely(netdata_exit)) {
708 + return 1;
709 + }
710 +
711 + if (unlikely(aclk_wait_for_initialization()))
712 + return 1;
713 +
714 + final_topic = get_topic(sub_topic, topic, ACLK_MAX_TOPIC);
715 +
716 + ACLK_LOCK;
717 + rc = _link_subscribe(final_topic, qos);
718 + ACLK_UNLOCK;
719 +
720 + // TODO: Add better handling -- error will flood the logfile here
721 + if (unlikely(rc))
722 + error("Failed to send message, error code %d (%s)", rc, _link_strerror(rc));
723 +
724 + return rc;
725 +}
726 +
727 +// This is called from a callback when the link goes up
728 +void aclk_connect(void *ptr)
729 +{
730 + (void) ptr;
731 + info("Connection detected");
732 + return;
733 +}
734 +
735 +// This is called from a callback when the link goes down
736 +void aclk_disconnect(void *ptr)
737 +{
738 + (void) ptr;
739 + info("Disconnect detected");
740 + aclk_subscribed = 0;
741 + aclk_metadata_submitted = 0;
742 +}
743 +
744 +void aclk_shutdown()
745 +{
746 + info("Shutdown initiated");
747 + aclk_connection_initialized = 0;
748 + _link_shutdown();
749 + info("Shutdown complete");
750 +}
751 +
752 +int aclk_init(ACLK_INIT_ACTION action)
753 +{
754 + (void) action;
755 +
756 + static int init = 0;
757 + int rc;
758 +
759 + if (likely(init))
760 + return 0;
761 +
762 + aclk_send_maximum = config_get_number(CONFIG_SECTION_ACLK, "agent cloud link send maximum", 20);
763 + aclk_recv_maximum = config_get_number(CONFIG_SECTION_ACLK, "agent cloud link receive maximum", 20);
764 +
765 + aclk_hostname = config_get(CONFIG_SECTION_ACLK, "agent cloud link hostname", "localhost");
766 + aclk_port = config_get_number(CONFIG_SECTION_ACLK, "agent cloud link port", 1883);
767 +
768 + info("Maximum parallel outgoing messages %d", aclk_send_maximum);
769 + info("Maximum parallel incoming messages %d", aclk_recv_maximum);
770 +
771 + // This will setup the base publish topic internally
772 + //get_publish_base_topic(PUBLICH_TOPIC_GET);
773 +
774 + // initialize the low level link to the cloud
775 + rc = _link_lib_init(aclk_hostname, aclk_port, aclk_connect, aclk_disconnect);
776 + if (unlikely(rc)) {
777 + error("Failed to initialize the agent cloud link library");
778 + return 1;
779 + }
780 + global_base_topic = GET_PUBLISH_BASE_TOPIC;
781 + init = 1;
782 +
783 + return 0;
784 +}
785 +
786 +// Use this to disable encoding of quotes and newlines so that
787 +// MQTT subscriber can display more readable data on screen
788 +
789 +void aclk_create_header(BUFFER *dest, char *type, char *msg_id)
790 +{
791 + uuid_t uuid;
792 + char uuid_str[36 + 1];
793 +
794 + if (unlikely(!msg_id)) {
795 + uuid_generate(uuid);
796 + uuid_unparse(uuid, uuid_str);
797 + msg_id = uuid_str;
798 + }
799 +
800 + buffer_sprintf(
801 + dest,
802 + "\t{\"type\": \"%s\",\n"
803 + "\t\"msg-id\": \"%s\",\n"
804 + "\t\"version\": %s,\n"
805 + "\t\"payload\": ",
806 + type, msg_id, ACLK_VERSION);
807 +}
808 +
809 +#define EYE_FRIENDLY 1
810 +
811 +// encapsulate contents into metadata message as per ACLK documentation
812 +void aclk_create_metadata_message(BUFFER *dest, char *type, char *msg_id, BUFFER *contents)
813 +{
814 +#ifndef EYE_FRIENDLY
815 + char *tmp_buffer = mallocz(NETDATA_WEB_RESPONSE_INITIAL_SIZE);
816 + char *src, *dst;
817 +#endif
818 +
819 + buffer_sprintf(
820 + dest,
821 + "\t{\"type\": \"%s\",\n"
822 + "\t\"msg-id\": \"%s\",\n"
823 + "\t\"payload\": %s\n\t}",
824 + type, msg_id ? msg_id : "", contents->buffer);
825 +
826 +#ifndef EYE_FRIENDLY
827 + //TODO: this is the initial escaping, It will expanded
828 + src = dest->buffer;
829 + dst = tmp_buffer;
830 + while (*src) {
831 + switch (*src) {
832 + case '0x0a':
833 + case '\n':
834 + *dst++ = '\\';
835 + *dst++ = 'n';
836 + break;
837 + case '\"':
838 + *dst++ = '\\';
839 + *dst++ = '\"';
840 + break;
841 + case '\'':
842 + *dst++ = '\\';
843 + *dst++ = '\"';
844 + break;
845 + default:
846 + *dst++ = *src;
847 + }
848 + src++;
849 + }
850 + *dst = '\0';
851 +
852 + buffer_flush(dest);
853 + buffer_sprintf(dest, "%s", tmp_buffer);
854 +
855 + freez(tmp_buffer);
856 +#endif
857 + return;
858 +}
859 +
860 +//TODO: this has been changed in the latest specs. We need to pack the data in one MQTT
861 +//message with a payload and has a list of json objects
862 +int aclk_send_alarm_metadata()
863 +{
864 + //TODO: improve locking on the buffer -- same lock is used for the message send
865 + //improve error handling
866 + ACLK_LOCK;
867 + buffer_flush(aclk_buffer);
868 + // Alarms configuration
869 + aclk_create_header(aclk_buffer, "alarms", NULL);
870 + health_alarms2json(localhost, aclk_buffer, 1);
871 + buffer_sprintf(aclk_buffer,"\n}");
872 + ACLK_UNLOCK;
873 + aclk_send_message(ACLK_ALARMS_TOPIC, aclk_buffer->buffer);
874 +
875 + // Alarms log
876 + ACLK_LOCK;
877 + buffer_flush(aclk_buffer);
878 + aclk_create_header(aclk_buffer, "alarms_log", NULL);
879 + health_alarm_log2json(localhost, aclk_buffer, 0);
880 + buffer_sprintf(aclk_buffer,"\n}");
881 + ACLK_UNLOCK;
882 + aclk_send_message(ACLK_ALARMS_TOPIC, aclk_buffer->buffer);
883 +
884 + return 0;
885 +}
886 +
887 +
888 +// Send info metadata message to the cloud if the link is established
889 +// or on request
890 +int aclk_send_metadata()
891 +{
892 + ACLK_LOCK;
893 +
894 + buffer_flush(aclk_buffer);
895 +
896 + aclk_create_header(aclk_buffer, "connect", NULL);
897 + buffer_sprintf(aclk_buffer,"{\n\t \"info\" : ");
898 + web_client_api_request_v1_info_fill_buffer(localhost, aclk_buffer);
899 + buffer_sprintf(aclk_buffer,", \n\t \"charts\" : ");
900 + charts2json(localhost, aclk_buffer);
901 + buffer_sprintf(aclk_buffer,"\n}\n}");
902 + aclk_buffer->contenttype = CT_APPLICATION_JSON;
903 +
904 + ACLK_UNLOCK;
905 +
906 + aclk_send_message(ACLK_METADATA_TOPIC, aclk_buffer->buffer);
907 +
908 + aclk_send_alarm_metadata();
909 +
910 + return 0;
911 +}
912 +
913 +//rrd_stats_api_v1_chart(RRDSET *st, BUFFER *buf)
914 +
915 +int aclk_send_single_chart(char *hostname, char *chart)
916 +{
917 + RRDHOST *target_host;
918 + ACLK_LOCK;
919 +
920 + buffer_flush(aclk_buffer);
921 +
922 + target_host = rrdhost_find_by_hostname(hostname, 0);
923 + if (!target_host)
924 + return 1;
925 +
926 + RRDSET *st = rrdset_find(target_host, chart);
927 +
928 + if (!st)
929 + st = rrdset_find_byname(target_host, chart);
930 +
931 + if (!st) {
932 + info("FAILED to find chart %s", chart);
933 + return 1;
934 + }
935 +
936 + aclk_buffer->contenttype = CT_APPLICATION_JSON;
937 +
938 + buffer_flush(aclk_buffer);
939 +
940 + aclk_create_header(aclk_buffer, "chart", NULL);
941 +
942 + rrdset2json(st, aclk_buffer, NULL, NULL);
943 + buffer_sprintf(aclk_buffer,"\n}\n}");
944 +
945 +
946 + ACLK_UNLOCK;
947 + aclk_send_message(ACLK_METADATA_TOPIC, aclk_buffer->buffer);
948 + return 0;
949 +}
950 +
951 +int aclk_update_chart(RRDHOST *host, char *chart_name)
952 +{
953 + (void) host;
954 + (void) chart_name;
955 +#ifndef ENABLE_ACLK
956 + return 0;
957 +#else
958 + if (host != localhost)
959 + return 0;
960 +
961 + aclk_queue_query("_chart", host->hostname, NULL, chart_name, 2, 1);
962 + return 0;
963 +#endif
964 +}
965 +
966 +int aclk_update_alarm(RRDHOST *host, char *alarm_name)
967 +{
968 + if (host != localhost)
969 + return 0;
970 +
971 + aclk_queue_query("_alarm", host->hostname, NULL, alarm_name, 2, 1);
972 + return 0;
973 +}
974 +
975 +
976 +//TODO: add and check the incoming type e.g http
977 +int aclk_handle_cloud_request(char *payload)
978 +{
979 + struct aclk_request cloud_to_agent = { .msg_id = NULL, .topic = NULL, .url = NULL, .version = 1};
980 +
981 + int rc = json_parse(payload, &cloud_to_agent, cloud_to_agent_parse);
982 +
983 + if (unlikely(JSON_OK != rc)) {
984 + error("Malformed json request (%s)", payload);
985 + return 1;
986 + }
987 +
988 + if (unlikely(!cloud_to_agent.url || !cloud_to_agent.topic)) {
989 + return 1;
990 + }
991 +
992 + aclk_submit_request(&cloud_to_agent);
993 +
994 + return 0;
995 +}
aclk/agent_cloud_link.h new
+103
@@ -0,0 +1,103 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#ifndef NETDATA_AGENT_CLOUD_LINK_H
4 +#define NETDATA_AGENT_CLOUD_LINK_H
5 +
6 +#include "../daemon/common.h"
7 +#include "mqtt.h"
8 +
9 +#define ACLK_JSON_IN_MSGID "msg-id"
10 +#define ACLK_JSON_IN_TYPE "type"
11 +#define ACLK_JSON_IN_VERSION "version"
12 +#define ACLK_JSON_IN_TOPIC "callback-topic"
13 +#define ACLK_JSON_IN_URL "payload"
14 +
15 +
16 +#define ACLK_MSG_TYPE_CHART "chart"
17 +#define ACLK_ALARMS_TOPIC "alarms"
18 +#define ACLK_METADATA_TOPIC "meta"
19 +#define ACLK_COMMAND_TOPIC "cmd"
20 +#define ACLK_TOPIC_STRUCTURE "/agent/%s"
21 +
22 +#define ACLK_STARTUP_WAIT 30 // Seconds to wait before establishing initialization process
23 +#define ACLK_INITIALIZATION_WAIT 60 // Wait for link to initialize in seconds (per msg)
24 +#define ACLK_INITIALIZATION_SLEEP_WAIT 1 // Wait time @ spin lock for MQTT initialization in seconds
25 +#define ACLK_QOS 1
26 +#define ACLK_PING_INTERVAL 60
27 +#define ACLK_LOOP_TIMEOUT 5 // seconds to wait for operations in the library loop
28 +
29 +#define ACLK_MAX_TOPIC 255
30 +
31 +#define ACLK_RECONNECT_DELAY 1 // reconnect delay -- with backoff stragegy fow now
32 +#define ACLK_MAX_RECONNECT_DELAY 120
33 +#define ACLK_VERSION "1"
34 +
35 +#define CONFIG_SECTION_ACLK "agent_cloud_link"
36 +
37 +struct aclk_request {
38 + char *type_id;
39 + char *msg_id;
40 + char *topic;
41 + char *url;
42 + int version;
43 +};
44 +
45 +
46 +typedef enum publish_topic_action {
47 + PUBLICH_TOPIC_GET,
48 + PUBLICH_TOPIC_FREE,
49 + PUBLICH_TOPIC_REBUILD
50 +} PUBLISH_TOPIC_ACTION;
51 +
52 +typedef enum aclk_init_action {
53 + ACLK_INIT,
54 + ACLK_REINIT
55 +} ACLK_INIT_ACTION;
56 +
57 +
58 +#define GET_PUBLISH_BASE_TOPIC get_publish_base_topic(0)
59 +#define FREE_PUBLISH_BASE_TOPIC get_publish_base_topic(1)
60 +#define REBUILD_PUBLISH_BASE_TOPIC get_publish_base_topic(2)
61 +
62 +void *aclk_main(void *ptr);
63 +
64 +#define NETDATA_ACLK_HOOK \
65 + { \
66 + .name = "AgentCloudLink", \
67 + .config_section = NULL, \
68 + .config_name = NULL, \
69 + .enabled = 1, \
70 + .thread = NULL, \
71 + .init_routine = NULL, \
72 + .start_routine = aclk_main \
73 + },
74 +
75 +extern int aclk_send_message(char *sub_topic, char *message);
76 +
77 +int aclk_init();
78 +char *get_base_topic();
79 +
80 +extern char *is_agent_claimed(void);
81 +
82 +// callbacks for agent cloud link
83 +int aclk_subscribe(char *topic, int qos);
84 +void aclk_shutdown();
85 +//void aclk_message_callback(struct mosquitto *moqs, void *obj, const struct mosquitto_message *msg);
86 +
87 +int cloud_to_agent_parse(JSON_ENTRY *e);
88 +void aclk_disconnect(void *conn);
89 +void aclk_connect(void *conn);
90 +void aclk_create_metadata_message(BUFFER *dest, char *type, char *msg_id, BUFFER *contents);
91 +int aclk_send_metadata();
92 +int aclk_wait_for_initialization();
93 +//int aclk_send_charts(RRDHOST *host, BUFFER *wb);
94 +int aclk_send_single_chart(char *host, char *chart);
95 +int aclk_queue_query(char *token, char *data, char *msg_type, char *query, int run_after, int internal);
96 +struct aclk_query *aclk_query_find(char *token, char *data, char *msg_id, char *query);
97 +//void aclk_rrdset2json(RRDSET *st, BUFFER *wb, char *hostname, int is_slave);
98 +int aclk_update_chart(RRDHOST *host, char *chart_name);
99 +int aclk_update_alarm(RRDHOST *host, char *alarm_name);
100 +void aclk_create_header(BUFFER *dest, char *type, char *msg_id);
101 +int aclk_handle_cloud_request(char *payload);
102 +int aclk_submit_request(struct aclk_request *);
103 +#endif //NETDATA_AGENT_CLOUD_LINK_H
aclk/mqtt.c new
+318
@@ -0,0 +1,318 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#include <libnetdata/json/json.h>
4 +#include "../daemon/common.h"
5 +#include "mqtt.h"
6 +
7 +void (*_on_connect)(void *ptr) = NULL;
8 +void (*_on_disconnect)(void *ptr) = NULL;
9 +extern int cmdpause;
10 +
11 +
12 +#ifndef ENABLE_ACLK
13 +
14 +inline const char *_link_strerror(int rc)
15 +{
16 + (void) rc;
17 + return "no error";
18 +}
19 +
20 +int _link_event_loop(int timeout)
21 +{
22 + (void) timeout;
23 + return 0;
24 +}
25 +
26 +int _link_send_message(char *topic, char *message)
27 +{
28 + (void) topic;
29 + (void) message;
30 + return 0;
31 +}
32 +
33 +int _link_subscribe(char *topic, int qos)
34 +{
35 + (void) topic;
36 + (void) qos;
37 + return 0;
38 +}
39 +
40 +void _link_shutdown()
41 +{
42 + return;
43 +}
44 +
45 +int _link_lib_init(char *aclk_hostname, int aclk_port, void (*on_connect)(void *), void (*on_disconnect)(void *))
46 +{
47 + (void) aclk_hostname;
48 + (void) aclk_port;
49 + (void) on_connect;
50 + (void) on_disconnect;
51 + return 0;
52 +}
53 +
54 +#else
55 +/*
56 + * Just report the library info in the logfile for reference when issues arise
57 + *
58 + */
59 +
60 +struct mosquitto *mosq = NULL;
61 +
62 +// Get a string description of the error
63 +
64 +inline const char *_link_strerror(int rc)
65 +{
66 + return mosquitto_strerror(rc);
67 +}
68 +
69 +
70 +void mqtt_message_callback(
71 + struct mosquitto *mosq, void *obj, const struct mosquitto_message *msg)
72 +{
73 + (void) mosq;
74 + (void) obj;
75 +
76 + // TODO: handle commands in a more efficient way, if we have many
77 +
78 + if (strcmp((char *)msg->payload, "pause") == 0) {
79 + cmdpause = 1;
80 + return;
81 + }
82 +
83 + if (strcmp((char *)msg->payload, "resume") == 0) {
84 + cmdpause = 0;
85 + return;
86 + }
87 +
88 + if (strcmp((char *)msg->payload, "reload") == 0) {
89 + error_log_limit_unlimited();
90 + info("Reloading health configuration");
91 + health_reload();
92 + error_log_limit_reset();
93 + return;
94 + }
95 +
96 + if (strcmp((char *)msg->payload, "info") == 0) {
97 + aclk_send_metadata();
98 + return;
99 + }
100 +
101 + aclk_handle_cloud_request(msg->payload);
102 +
103 + //info("Received type=[%s], msg-id=[%s], topic=[%s], url=[%s]",cloud_to_agent.type_id, cloud_to_agent.msg_id, cloud_to_agent.topic, cloud_to_agent.url);
104 +
105 +}
106 +
107 +void connect_callback(struct mosquitto *mosq, void *obj, int rc)
108 +{
109 + (void) obj;
110 + (void) rc;
111 +
112 + info("Connection to cloud estabilished");
113 +
114 + aclk_connection_initialized = 1;
115 + _on_connect((void *) mosq);
116 +
117 + return;
118 +}
119 +
120 +
121 +void disconnect_callback(struct mosquitto *mosq, void *obj, int rc)
122 +{
123 + (void) obj;
124 + (void) rc;
125 +
126 + info("Connection to cloud failed");
127 + // TODO: Keep the connection "alive" for now. The library will reconnect.
128 +
129 + //mqtt_connection_initialized = 0;
130 + _on_disconnect((void *) mosq);
131 + //sleep_usec(USEC_PER_SEC * 5);
132 + return;
133 +}
134 +
135 +
136 +void _show_mqtt_info()
137 +{
138 + int libmosq_major, libmosq_minor, libmosq_revision, libmosq_version;
139 + libmosq_version = mosquitto_lib_version(&libmosq_major, &libmosq_minor, &libmosq_revision);
140 +
141 + info("Detected libmosquitto library version %d, %d.%d.%d",libmosq_version, libmosq_major, libmosq_minor, libmosq_revision);
142 +}
143 +
144 +int _link_lib_init(char *aclk_hostname, int aclk_port, void (*on_connect)(void *), void (*on_disconnect)(void *))
145 +{
146 + int rc;
147 + int libmosq_major, libmosq_minor, libmosq_revision, libmosq_version;
148 + char *ca_crt;
149 + char *server_crt;
150 + char *server_key;
151 +
152 + // show library info so can have in in the logfile
153 + libmosq_version = mosquitto_lib_version(&libmosq_major, &libmosq_minor, &libmosq_revision);
154 + ca_crt = config_get(CONFIG_SECTION_ACLK, "agent cloud link cert", "*");
155 + server_crt = config_get(CONFIG_SECTION_ACLK, "agent cloud link server cert", "*");
156 + server_key = config_get(CONFIG_SECTION_ACLK, "agent cloud link server key", "*");
157 +
158 +
159 + if (ca_crt[0] == '*') {
160 + freez(ca_crt);
161 + ca_crt = NULL;
162 + }
163 +
164 + if (server_crt[0] == '*') {
165 + freez(server_crt);
166 + server_crt = NULL;
167 + }
168 +
169 + if (server_key[0] == '*') {
170 + freez(server_key);
171 + server_key = NULL;
172 + }
173 +
174 + info(
175 + "Detected libmosquitto library version %d, %d.%d.%d", libmosq_version, libmosq_major, libmosq_minor,
176 + libmosq_revision);
177 +
178 + rc = mosquitto_lib_init();
179 + if (unlikely(rc != MOSQ_ERR_SUCCESS)) {
180 + error("Failed to initialize MQTT (libmosquitto library)");
181 + return 1;
182 + }
183 +
184 + mosq = mosquitto_new("anon", true, NULL);
185 + if (unlikely(!mosq)) {
186 + mosquitto_lib_cleanup();
187 + error("MQTT new structure -- %s", mosquitto_strerror(errno));
188 + return 1;
189 + }
190 +
191 + _on_connect = on_connect;
192 + _on_disconnect = on_disconnect;
193 +
194 + mosquitto_connect_callback_set(mosq, connect_callback);
195 + mosquitto_disconnect_callback_set(mosq, disconnect_callback);
196 +
197 + mosquitto_username_pw_set(mosq, "anon", "anon");
198 +
199 + rc = mosquitto_threaded_set(mosq, 1);
200 + if (unlikely(rc != MOSQ_ERR_SUCCESS))
201 + error("Failed to tune the thread model for libmoquitto (%s)", mosquitto_strerror(rc));
202 +
203 +#if defined(LIBMOSQUITTO_VERSION_NUMBER) >= 1006000
204 + rc = mosquitto_int_option(mosq, MQTT_PROTOCOL_V311, 0);
205 + if (unlikely(rc != MOSQ_ERR_SUCCESS))
206 + error("MQTT protocol specification rc = %d (%s)", rc, mosquitto_strerror(rc));
207 +
208 + rc = mosquitto_int_option(mosq, MOSQ_OPT_SEND_MAXIMUM, 1);
209 + info("MQTT in flight messages set to 1 -- %s", mosquitto_strerror(rc));
210 +#endif
211 +
212 + rc = mosquitto_reconnect_delay_set(mosq, ACLK_RECONNECT_DELAY, ACLK_MAX_RECONNECT_DELAY, 1);
213 +
214 + if (unlikely(rc != MOSQ_ERR_SUCCESS))
215 + error("Failed to set the reconnect delay (%d) (%s)", rc, mosquitto_strerror(rc));
216 +
217 + mosquitto_tls_set(mosq, ca_crt, NULL, server_crt, server_key, NULL);
218 +
219 + rc = mosquitto_connect_async(mosq, aclk_hostname, aclk_port, ACLK_PING_INTERVAL);
220 +
221 + if (unlikely(rc != MOSQ_ERR_SUCCESS))
222 + error("Connect %s MQTT status = %d (%s)", aclk_hostname, rc, mosquitto_strerror(rc));
223 + else
224 + info("Establishing MQTT link to %s", aclk_hostname);
225 +
226 + return rc;
227 +}
228 +
229 +int _link_event_loop(int timeout)
230 +{
231 + int rc;
232 +
233 + rc = mosquitto_loop(mosq, timeout, 1);
234 +
235 + if (unlikely(rc != MOSQ_ERR_SUCCESS)) {
236 + errno = 0;
237 + error("Loop error code %d (%s)", rc, mosquitto_strerror(rc));
238 + rc = mosquitto_reconnect(mosq);
239 + if (unlikely(rc != MOSQ_ERR_SUCCESS)) {
240 + error("Reconnect loop error code %d (%s)", rc, mosquitto_strerror(rc));
241 + }
242 + // TBD: Using delay
243 + sleep_usec(USEC_PER_SEC * 10);
244 + }
245 + return rc;
246 +}
247 +
248 +void _link_shutdown()
249 +{
250 + int rc;
251 +
252 + rc = mosquitto_disconnect(mosq);
253 + switch (rc) {
254 + case MOSQ_ERR_SUCCESS:
255 + info("MQTT disconnected from broker");
256 + break;
257 + default:
258 + info("MQTT invalid structure");
259 + break;
260 + };
261 +
262 + mosquitto_destroy(mosq);
263 + mosq = NULL;
264 + return;
265 +}
266 +
267 +
268 +int _link_subscribe(char *topic, int qos)
269 +{
270 + int rc;
271 +
272 + if (unlikely(!mosq))
273 + return 1;
274 +
275 + mosquitto_message_callback_set(mosq, mqtt_message_callback);
276 +
277 + rc = mosquitto_subscribe(mosq, NULL, topic, qos);
278 + if (unlikely(rc)) {
279 + errno = 0;
280 + error("Failed to register subscription %d (%s)", rc, mosquitto_strerror(rc));
281 + return 1;
282 + }
283 +
284 + return 0;
285 +}
286 +
287 +
288 +/*
289 + * Send a message to the cloud to specific topic
290 + *
291 + */
292 +
293 +int _link_send_message(char *topic, char *message)
294 +{
295 + int rc;
296 +
297 + rc = mosquitto_pub_topic_check(topic);
298 +
299 + if (unlikely(rc != MOSQ_ERR_SUCCESS))
300 + return rc;
301 +
302 + int msg_len = strlen(message);
303 +
304 + // TODO: handle encoding validation -- the message should be UFT8 encoded by the sender
305 + //rc = mosquitto_validate_utf8(message, msg_len);
306 + //if (unlikely(rc != MOSQ_ERR_SUCCESS))
307 + // return rc;
308 +
309 + rc = mosquitto_publish(mosq, NULL, topic, msg_len, message, ACLK_QOS, 0);
310 +
311 + // TODO: Add better handling -- error will flood the logfile here
312 + if (unlikely(rc != MOSQ_ERR_SUCCESS)) {
313 + error("MQTT message failed : %s", mosquitto_strerror(rc));
314 + }
315 +
316 + return rc;
317 +}
318 +#endif
\ No newline at end of file
aclk/mqtt.h new
+21
@@ -0,0 +1,21 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +#ifndef NETDATA_MQTT_H
4 +#define NETDATA_MQTT_H
5 +
6 +#ifdef ENABLE_ACLK
7 +#include "mosquitto/lib/mosquitto.h"
8 +#endif
9 +
10 +void _show_mqtt_info();
11 +int _link_event_loop(int timeout);
12 +void _link_shutdown();
13 +int _link_lib_init(char *aclk_hostname, int aclk_port, void (*on_connect)(void *), void (*on_disconnect)(void *));
14 +int _link_subscribe(char *topic, int qos);
15 +int _link_send_message(char *topic, char *message);
16 +const char *_link_strerror(int rc);
17 +
18 +extern int aclk_connection_initialized;
19 +int aclk_handle_cloud_request(char *);
20 +
21 +#endif //NETDATA_MQTT_H
claim/claim.c
+31 -15
@@ -21,13 +21,12 @@ static char *claiming_errors[] = {
21 "internal server error" // 12
22 };
23
24 -#define AGENT_UNCLAIMED 0
25 -#define AGENT_CLAIMED 1
26 -static uint8_t claiming_status = AGENT_UNCLAIMED;
24
28 -uint8_t is_agent_claimed(void)
25 +static char *claimed_id = NULL;
26 +
27 +char *is_agent_claimed(void)
28 {
30 - return (AGENT_CLAIMED == claiming_status);
29 + return claimed_id;
30 }
31
32 #define CLAIMING_COMMAND_LENGTH 16384
@@ -64,8 +63,7 @@ void claim_agent(char *claiming_arguments)
63 exit_code = mypclose(fp, command_pid);
64 info("Agent claiming command returned with code %d", exit_code);
65 if (0 == exit_code) {
67 - claiming_status = AGENT_CLAIMED;
68 - info("Agent successfully claimed.");
66 + load_claiming_state();
67 return;
68 }
69 if (exit_code < 0) {
@@ -85,19 +83,37 @@ void claim_agent(char *claiming_arguments)
83
84 void load_claiming_state(void)
85 {
88 - info("The claiming feature is under development and still subject to change before the next release");
89 - return;
86 + if (claimed_id != NULL) {
87 + freez(claimed_id);
88 + claimed_id = NULL;
89 + }
90
91 char filename[FILENAME_MAX + 1];
92 struct stat statbuf;
93
94 - snprintfz(filename, FILENAME_MAX, "%s/claim.d/is_claimed", netdata_configured_user_config_dir);
94 + snprintfz(filename, FILENAME_MAX, "%s/claim.d/claimed_id", netdata_configured_user_config_dir);
95 +
96 // check if the file exists
97 if (lstat(filename, &statbuf) != 0) {
97 - info("File '%s' was not found. Setting state to AGENT_UNCLAIMED.", filename);
98 - claiming_status = AGENT_UNCLAIMED;
99 - } else {
100 - info("File '%s' was found. Setting state to AGENT_CLAIMED.", filename);
101 - claiming_status = AGENT_CLAIMED;
98 + info("lstat on File '%s' failed reason=\"%s\". Setting state to AGENT_UNCLAIMED.", filename, strerror(errno));
99 + return;
100 }
101 + if (unlikely(statbuf.st_size == 0)) {
102 + info("File '%s' has no contents. Setting state to AGENT_UNCLAIMED.", filename);
103 + return;
104 + }
105 +
106 + FILE *f = fopen(filename, "rt");
107 + if (unlikely(f == NULL)) {
108 + error("File '%s' cannot be opened. Setting state to AGENT_UNCLAIMED.", filename);
109 + return;
110 + }
111 +
112 + claimed_id = callocz(1, statbuf.st_size + 1);
113 + size_t bytes_read = fread(claimed_id, 1, statbuf.st_size, f);
114 + claimed_id[bytes_read] = 0;
115 + info("File '%s' was found. Setting state to AGENT_CLAIMED.", filename);
116 + fclose(f);
117 +
118 + snprintfz(filename, FILENAME_MAX, "%s/claim.d/private.pem", netdata_configured_user_config_dir);
119 }
claim/claim.h
+1 -1
@@ -8,7 +8,7 @@
8 extern char *claiming_pending_arguments;
9
10 void claim_agent(char *claiming_arguments);
11 -uint8_t is_agent_claimed(void);
11 +char *is_agent_claimed(void);
12 void load_claiming_state(void);
13
14 #endif //NETDATA_CLAIM_H
claim/netdata-claim.sh.in
+2 -2
@@ -211,6 +211,6 @@ if [ "${HTTP_STATUS_CODE}" -ne 200 ] ; then
211 fi
212
213 rm -f "${CLAIMING_DIR}/tmpout.txt"
214 -touch "${CLAIMING_DIR}/is_claimed"
214 +echo -n "${ID}" >"${CLAIMING_DIR}/claimed_id"
215 rm -f "${CLAIMING_DIR}/token"
216 -echo >&2 "Agent was successfully claimed."
\ No newline at end of file
216 +echo >&2 "Agent was successfully claimed."
configure.ac
+15 -1
@@ -342,7 +342,6 @@ AC_DEFINE([NETDATA_WITH_UUID], [1], [uuid usability])
342 OPTIONAL_UUID_CFLAGS="${UUID_CFLAGS}"
343 OPTIONAL_UUID_LIBS="${UUID_LIBS}"
344
345 -
345 # -----------------------------------------------------------------------------
346 # OpenSSL Cryptography and SSL/TLS Toolkit
347
@@ -408,6 +407,19 @@ fi
407 AC_MSG_RESULT([${enable_https}])
408 AM_CONDITIONAL([ENABLE_HTTPS], [test "${enable_https}" = "yes"])
409
410 +# -----------------------------------------------------------------------------
411 +# ACLK
412 +AC_MSG_CHECKING([if netdata ACLK should be enabled])
413 +if test "${ACLK}" = "yes"; then
414 + enable_aclk="yes"
415 + AC_DEFINE([ENABLE_ACLK], [1], [netdata ACLK])
416 + CFLAGS="${CFLAGS} -DENABLE_ACLK"
417 +else
418 + enable_aclk="no"
419 +fi
420 +AC_MSG_RESULT([${enable_aclk}])
421 +AM_CONDITIONAL([ENABLE_ACLK], [test "${enable_aclk}" = "yes"])
422 +
423 # -----------------------------------------------------------------------------
424 # Exporting engine
425 AC_MSG_CHECKING([if netdata exporting engine should be used])
@@ -1146,6 +1158,7 @@ AC_SUBST([OPTIONAL_ZLIB_CFLAGS])
1158 AC_SUBST([OPTIONAL_ZLIB_LIBS])
1159 AC_SUBST([OPTIONAL_UUID_CFLAGS])
1160 AC_SUBST([OPTIONAL_UUID_LIBS])
1161 +AC_SUBST([OPTIONAL_MQTT_LIBS])
1162 AC_SUBST([OPTIONAL_LIBCAP_CFLAGS])
1163 AC_SUBST([OPTIONAL_LIBCAP_LIBS])
1164 AC_SUBST([OPTIONAL_IPMIMONITORING_CFLAGS])
@@ -1284,6 +1297,7 @@ AC_CONFIG_FILES([
1297 web/server/Makefile
1298 web/server/static/Makefile
1299 claim/Makefile
1300 + aclk/Makefile
1301 ])
1302 AC_OUTPUT
1303
daemon/common.h
+3
@@ -63,6 +63,9 @@
63 // netdata agent claiming
64 #include "claim/claim.h"
65
66 +// netdata agent cloud link
67 +#include "aclk/agent_cloud_link.h"
68 +
69 // the netdata deamon
70 #include "daemon.h"
71 #include "main.h"
daemon/main.c
+5
@@ -73,6 +73,10 @@ struct netdata_static_thread static_threads[] = {
73 NETDATA_PLUGIN_HOOK_IDLEJITTER
74 NETDATA_PLUGIN_HOOK_STATSD
75
76 +#ifdef ENABLE_ACLK
77 + NETDATA_ACLK_HOOK
78 +#endif
79 +
80 // common plugins for all systems
81 {"BACKENDS", NULL, NULL, 1, NULL, NULL, backends_main},
82 #ifdef ENABLE_EXPORTING
@@ -785,6 +789,7 @@ int get_system_info(struct rrdhost_system_info *system_info) {
789
790 void send_statistics( const char *action, const char *action_result, const char *action_data) {
791 static char *as_script;
792 +
793 if (netdata_anonymous_statistics_enabled == -1) {
794 char *optout_file = mallocz(sizeof(char) * (strlen(netdata_configured_user_config_dir) +strlen(".opt-out-from-anonymous-statistics") + 2));
795 sprintf(optout_file, "%s/%s", netdata_configured_user_config_dir, ".opt-out-from-anonymous-statistics");
database/rrddim.c
+21 -1
@@ -183,6 +183,9 @@ void rrdcalc_link_to_rrddim(RRDDIM *rd, RRDSET *st, RRDHOST *host) {
183 }
184 }
185 }
186 +#ifdef ENABLE_ACLK
187 + aclk_update_chart(st->rrdhost, st->id);
188 +#endif
189 }
190
191 RRDDIM *rrddim_add_custom(RRDSET *st, const char *id, const char *name, collected_number multiplier, collected_number divisor, RRD_ALGORITHM algorithm, RRD_MEMORY_MODE memory_mode) {
@@ -424,7 +427,9 @@ RRDDIM *rrddim_add_custom(RRDSET *st, const char *id, const char *name, collecte
427 }
428 }
429 rrdset_unlock(st);
427 -
430 +#ifdef ENABLE_ACLK
431 + aclk_update_chart(host, st->id);
432 +#endif
433 return(rd);
434 }
435
@@ -478,6 +483,9 @@ void rrddim_free(RRDSET *st, RRDDIM *rd)
483 freez(rd);
484 break;
485 }
486 +#ifdef ENABLE_ACLK
487 + aclk_update_chart(st->rrdhost, st->id);
488 +#endif
489 }
490
491
@@ -496,6 +504,9 @@ int rrddim_hide(RRDSET *st, const char *id) {
504 }
505
506 rrddim_flag_set(rd, RRDDIM_FLAG_HIDDEN);
507 +#ifdef ENABLE_ACLK
508 + aclk_update_chart(st->rrdhost, st->id);
509 +#endif
510 return 0;
511 }
512
@@ -510,6 +521,9 @@ int rrddim_unhide(RRDSET *st, const char *id) {
521 }
522
523 rrddim_flag_clear(rd, RRDDIM_FLAG_HIDDEN);
524 +#ifdef ENABLE_ACLK
525 + aclk_update_chart(st->rrdhost, st->id);
526 +#endif
527 return 0;
528 }
529
@@ -518,12 +532,18 @@ inline void rrddim_is_obsolete(RRDSET *st, RRDDIM *rd) {
532
533 rrddim_flag_set(rd, RRDDIM_FLAG_OBSOLETE);
534 rrdset_flag_set(st, RRDSET_FLAG_OBSOLETE_DIMENSIONS);
535 +#ifdef ENABLE_ACLK
536 + aclk_update_chart(st->rrdhost, st->id);
537 +#endif
538 }
539
540 inline void rrddim_isnot_obsolete(RRDSET *st __maybe_unused, RRDDIM *rd) {
541 debug(D_RRD_CALLS, "rrddim_isnot_obsolete() for chart %s, dimension %s", st->name, rd->name);
542
543 rrddim_flag_clear(rd, RRDDIM_FLAG_OBSOLETE);
544 +#ifdef ENABLE_ACLK
545 + aclk_update_chart(st->rrdhost, st->id);
546 +#endif
547 }
548
549 // ----------------------------------------------------------------------------
database/rrdset.c
+3 -1
@@ -761,7 +761,9 @@ RRDSET *rrdset_create_custom(
761 rrdhost_cleanup_obsolete_charts(host);
762
763 rrdhost_unlock(host);
764 -
764 +#ifdef ENABLE_ACLK
765 + aclk_update_chart(host, st->id);
766 +#endif
767 return(st);
768 }
769
health/health_log.c
-1
@@ -101,7 +101,6 @@ inline void health_label_log_save(RRDHOST *host) {
101
102 inline void health_alarm_log_save(RRDHOST *host, ALARM_ENTRY *ae) {
103 health_log_rotate(host);
104 -
104 if(likely(host->health_log_fp)) {
105 if(unlikely(fprintf(host->health_log_fp
106 , "%c\t%s"
web/api/web_api_v1.c
+14 -7
@@ -793,13 +793,8 @@ inline void host_labels2json(RRDHOST *host, BUFFER *wb, size_t indentation) {
793 rrdhost_unlock(host);
794 }
795
796 -inline int web_client_api_request_v1_info(RRDHOST *host, struct web_client *w, char *url) {
797 - (void)url;
798 - if (!netdata_ready) return HTTP_RESP_BACKEND_FETCH_FAILED;
799 - BUFFER *wb = w->response.data;
800 - buffer_flush(wb);
801 - wb->contenttype = CT_APPLICATION_JSON;
802 -
796 +inline int web_client_api_request_v1_info_fill_buffer(RRDHOST *host, BUFFER *wb)
797 +{
798 buffer_strcat(wb, "{\n");
799 buffer_sprintf(wb, "\t\"version\": \"%s\",\n", host->program_version);
800 buffer_sprintf(wb, "\t\"uid\": \"%s\",\n", host->machine_guid);
@@ -849,6 +844,18 @@ inline int web_client_api_request_v1_info(RRDHOST *host, struct web_client *w, c
844 buffer_strcat(wb, "\n\t]\n");
845
846 buffer_strcat(wb, "}");
847 + return 0;
848 +}
849 +
850 +inline int web_client_api_request_v1_info(RRDHOST *host, struct web_client *w, char *url) {
851 + (void)url;
852 + if (!netdata_ready) return HTTP_RESP_BACKEND_FETCH_FAILED;
853 + BUFFER *wb = w->response.data;
854 + buffer_flush(wb);
855 + wb->contenttype = CT_APPLICATION_JSON;
856 +
857 + web_client_api_request_v1_info_fill_buffer(host, wb);
858 +
859 buffer_no_cacheable(wb);
860 return HTTP_RESP_OK;
861 }
web/api/web_api_v1.h
+1 -1
@@ -23,7 +23,7 @@ extern int web_client_api_request_v1_data(RRDHOST *host, struct web_client *w, c
23 extern int web_client_api_request_v1_registry(RRDHOST *host, struct web_client *w, char *url);
24 extern int web_client_api_request_v1_info(RRDHOST *host, struct web_client *w, char *url);
25 extern int web_client_api_request_v1(RRDHOST *host, struct web_client *w, char *url);
26 -
26 +extern int web_client_api_request_v1_info_fill_buffer(RRDHOST *host, BUFFER *wb);
27 extern void host_labels2json(RRDHOST *host, BUFFER *wb, size_t indentation);
28
29 extern void web_client_api_v1_init(void);