Create a template for all dimensions (#6560)
* health_connection: Comments inside Health Config To try to understand better what is necessary to change and where it is necessary to change anything inside the health, I commented the functions inside this file" " * health_connection: Comments about Health in other files This commit brings the rest of the comments that were missed for health" * health_connection: Comments on health_log I had to append more comments on health_log * health_connection: Create a new variable New variable is created to work with foreach * health_connection: Fix new option and doc The first implementation of the 'foreach' had a problem, this fixes the error. This commit also brings the updates for the documentation * health_connection: Understanding health This commit is to save the place that I am working, it has the map to understand all the alam process * health_connection: Update map I changed the position of the error message to identify the correct place to add new alarms * health_connection: End of simple alarm This commit finishes what is necessary to bring the same lookup for different dimensions in one unique line * health_connection: Documentation and template steps This commit brings the documentation missed for template and comments to help in the next step of apply a template to create an alarm. * health_connection: Restoring After some tests, it was detected that the alarms were not working as expected * health_connection: Fix bug and bring dimension to template This commit brings a fix for an old Netdata bug, before this the Netdata always tried to create a new entry in an index with the same id raising an error. It also brings the possibility to use 'foreach' in template * health_connection: Fix cmake compilation There was a problem with cmake compilation fixed by this commit * health_connection: shell script Finilize the shell script to test the PR * health_connection: Remove debug message During the development, I used some messages to understand the code this commit removes the last message * health_connection: Fix bugs This commits fix bugs reported by tests * health_connection: Alarm working This commit brings the necessary change for the alarms work, but it is missing the unlink from the newest list * health_connection: Template code written This commit finishes the creation of alarm from template, but it was not tested yet. * health_connection: Remove comments I am removing the comments from this PR to bring back late * health_connection: Remove lines Another commit to restore the files before they to be commented * health_connection: New alarm and remove messages I am bringing a new alarm to test template with SP and removing comments used during the development * health_connection: Functional test review After to review the functional test script, it was necessary to small adjust to test all the features available with the new version * health_connection: Free structure I am moving the free list for the correct place, the previous place was not safe * health_connection: ShellCheck This commit fixes the problems with shellcheck * health_connection: FIx hash This commit fix the hash calculation that was using wrong input * health_connection: Fix message error The system was showing a wronge message, because when we have foreach the alarm created with templated is added in a second stage to the index * health_connection: Fix documentation In this commit I am fixing the grammar of the previous doc and bringing two examples * health_connection: Fix examples This commit fix the last two examples that was brought in this PR * health_connection: Fix example doc When I brought the correct grammar in the last commit, I lost a mark * health_connection: Grammar fix Fixing grammar of the documentation * health_connection: Memory leak This commit fixes the memory leak that was present in the PR * health_connection: Reload This commit fix the problem that the alarms were not linked after to receive a SIGUSR2 * health_connection: False Positive from codacy Codacy was given a false positive, I changed the function to avoid it. * health_connection: dead code Remove dead code from the code. * health_connection: Memory Leak Remove memory leak when clean simple pattern * health_connection: Script format With this commit I am formatting the last message to return for the default color on terminal * health_connection: Script format 2 With this commit I am formatting the last message to return for the default color on terminal * health_connection: Script format 3 With this commit I am formatting the error message to return for the default color on terminal
thiagoftsm committed
Sep 27, 2019 at 12:24 UTC
e3471fa5727bcf286dd3b52ec0cdecd8fdf7067e
20 files changed
+636
-91
.gitignore
+1
@@ -170,6 +170,7 @@ sitespeed-result/
170
tests/acls/acl.sh
171
tests/urls/request.sh
172
tests/alarm_repetition/alarm.sh
173
+tests/template_dimension/template_dim.sh
174
175
# tests and temp files
176
python.d/python-modules-installer.sh
database/rrd.h
+3
@@ -697,6 +697,7 @@ struct rrdhost {
697
// RRDCALCs may be linked to charts at any point
698
// (charts may or may not exist when these are loaded)
699
RRDCALC *alarms;
700
+ RRDCALC *alarms_with_foreach;
701
avl_tree_lock alarms_idx_health_log;
702
avl_tree_lock alarms_idx_name;
703
@@ -709,6 +710,7 @@ struct rrdhost {
710
// these are used to create alarms when charts
711
// are created or renamed, that match them
712
RRDCALCTEMPLATE *templates;
713
+ RRDCALCTEMPLATE *alarms_template_with_foreach;
714
715
716
// ------------------------------------------------------------------------
@@ -1008,6 +1010,7 @@ static inline time_t rrdset_slot2time(RRDSET *st, size_t slot) {
1010
// ----------------------------------------------------------------------------
1011
// RRD DIMENSION functions
1012
1013
+extern void rrdcalc_link_to_rrddim(RRDDIM *rd, RRDSET *st, RRDHOST *host);
1014
extern RRDDIM *rrddim_add_custom(RRDSET *st, const char *id, const char *name, collected_number multiplier, collected_number divisor, RRD_ALGORITHM algorithm, RRD_MEMORY_MODE memory_mode);
1015
#define rrddim_add(st, id, name, multiplier, divisor, algorithm) rrddim_add_custom(st, id, name, multiplier, divisor, algorithm, (st)->rrd_memory_mode)
1016
database/rrdcalc.c
+169
-22
@@ -255,6 +255,53 @@ inline uint32_t rrdcalc_get_unique_id(RRDHOST *host, const char *chart, const ch
255
return host->health_log.next_alarm_id++;
256
}
257
258
+/**
259
+ * Alarm name with dimension
260
+ *
261
+ * Change the name of the current alarm appending a new diagram.
262
+ *
263
+ * @param name the alarm name
264
+ * @param namelen is the length of the previous vector.
265
+ * @param dim the dimension of the chart.
266
+ * @param dimlen is the length of the previous vector.
267
+ *
268
+ * @return It returns the new name on success and the old otherwise
269
+ */
270
+char *alarm_name_with_dim(char *name, size_t namelen, const char *dim, size_t dimlen) {
271
+ char *newname,*move;
272
+
273
+ newname = malloc(namelen + dimlen + 2);
274
+ if(newname) {
275
+ move = newname;
276
+ memcpy(move, name, namelen);
277
+ move += namelen;
278
+
279
+ *move++ = '_';
280
+ memcpy(move, dim, dimlen);
281
+ move += dimlen;
282
+ *move = '\0';
283
+ } else {
284
+ newname = name;
285
+ }
286
+
287
+ return newname;
288
+}
289
+
290
+/**
291
+ * Remove pipe comma
292
+ *
293
+ * Remove the pipes and commas converting to space.
294
+ *
295
+ * @param str the string to change.
296
+ */
297
+void dimension_remove_pipe_comma(char *str) {
298
+ while(*str) {
299
+ if(*str == '|' || *str == ',') *str = ' ';
300
+
301
+ str++;
302
+ }
303
+}
304
+
305
inline void rrdcalc_add_to_host(RRDHOST *host, RRDCALC *rc) {
306
rrdhost_check_rdlock(host);
307
@@ -282,24 +329,39 @@ inline void rrdcalc_add_to_host(RRDHOST *host, RRDCALC *rc) {
329
rc->critical->rrdcalc = rc;
330
}
331
285
- // link it to the host
286
- if(likely(host->alarms)) {
287
- // append it
288
- RRDCALC *t;
289
- for(t = host->alarms; t && t->next ; t = t->next) ;
290
- t->next = rc;
291
- }
292
- else {
293
- host->alarms = rc;
294
- }
332
+ if(!rc->foreachdim) {
333
+ // link it to the host alarms list
334
+ if(likely(host->alarms)) {
335
+ // append it
336
+ RRDCALC *t;
337
+ for(t = host->alarms; t && t->next ; t = t->next) ;
338
+ t->next = rc;
339
+ }
340
+ else {
341
+ host->alarms = rc;
342
+ }
343
296
- // link it to its chart
297
- RRDSET *st;
298
- rrdset_foreach_read(st, host) {
299
- if(rrdcalc_is_matching_this_rrdset(rc, st)) {
300
- rrdsetcalc_link(st, rc);
301
- break;
344
+ // link it to its chart
345
+ RRDSET *st;
346
+ rrdset_foreach_read(st, host) {
347
+ if(rrdcalc_is_matching_this_rrdset(rc, st)) {
348
+ rrdsetcalc_link(st, rc);
349
+ break;
350
+ }
351
}
352
+ } else {
353
+ //link it case there is a foreach
354
+ if(likely(host->alarms_with_foreach)) {
355
+ // append it
356
+ RRDCALC *t;
357
+ for(t = host->alarms_with_foreach; t && t->next ; t = t->next) ;
358
+ t->next = rc;
359
+ }
360
+ else {
361
+ host->alarms_with_foreach = rc;
362
+ }
363
+
364
+ //I am not linking this alarm direct to the host here, this will be done when the children is created
365
}
366
}
367
@@ -311,13 +373,19 @@ inline RRDCALC *rrdcalc_create_from_template(RRDHOST *host, RRDCALCTEMPLATE *rt,
373
374
RRDCALC *rc = callocz(1, sizeof(RRDCALC));
375
rc->next_event_id = 1;
314
- rc->id = rrdcalc_get_unique_id(host, chart, rt->name, &rc->next_event_id);
376
rc->name = strdupz(rt->name);
377
rc->hash = simple_hash(rc->name);
378
rc->chart = strdupz(chart);
379
rc->hash_chart = simple_hash(rc->chart);
380
381
+ rc->id = rrdcalc_get_unique_id(host, rc->chart, rc->name, &rc->next_event_id);
382
+
383
if(rt->dimensions) rc->dimensions = strdupz(rt->dimensions);
384
+ if(rt->foreachdim) {
385
+ rc->foreachdim = strdupz(rt->foreachdim);
386
+ rc->spdim = health_pattern_from_foreach(rc->foreachdim);
387
+ }
388
+ rc->foreachcounter = rt->foreachcounter;
389
390
rc->green = rt->green;
391
rc->red = rt->red;
@@ -361,7 +429,7 @@ inline RRDCALC *rrdcalc_create_from_template(RRDHOST *host, RRDCALCTEMPLATE *rt,
429
error("Health alarm '%s.%s': failed to re-parse critical expression '%s'", chart, rt->name, rt->critical->source);
430
}
431
364
- debug(D_HEALTH, "Health runtime added alarm '%s.%s': exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
432
+ debug(D_HEALTH, "Health runtime added alarm '%s.%s': exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', for each dimension '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
433
(rc->chart)?rc->chart:"NOCHART",
434
rc->name,
435
(rc->exec)?rc->exec:"DEFAULT",
@@ -373,6 +441,7 @@ inline RRDCALC *rrdcalc_create_from_template(RRDHOST *host, RRDCALCTEMPLATE *rt,
441
rc->before,
442
rc->options,
443
(rc->dimensions)?rc->dimensions:"NONE",
444
+ (rc->foreachdim)?rc->foreachdim:"NONE",
445
rc->update_every,
446
(rc->calculation)?rc->calculation->parsed_as:"NONE",
447
(rc->warning)?rc->warning->parsed_as:"NONE",
@@ -387,18 +456,94 @@ inline RRDCALC *rrdcalc_create_from_template(RRDHOST *host, RRDCALCTEMPLATE *rt,
456
);
457
458
rrdcalc_add_to_host(host, rc);
390
- RRDCALC *rdcmp = (RRDCALC *) avl_insert_lock(&(host)->alarms_idx_health_log,(avl *)rc);
391
- if (rdcmp != rc) {
392
- error("Cannot insert the alarm index ID %s",rc->name);
459
+ if(!rt->foreachdim) {
460
+ RRDCALC *rdcmp = (RRDCALC *) avl_insert_lock(&(host)->alarms_idx_health_log,(avl *)rc);
461
+ if (rdcmp != rc) {
462
+ error("Cannot insert the alarm index ID %s",rc->name);
463
+ }
464
}
465
466
return rc;
467
}
468
469
+/**
470
+ * Create from RRDCALC
471
+ *
472
+ * Create a new alarm using another alarm as template.
473
+ *
474
+ * @param rc is the alarm that will be used as source
475
+ * @param host is the host structure.
476
+ * @param name is the newest chart name.
477
+ * @param dimension is the current dimension
478
+ * @param foreachdim the whole list of dimension
479
+ *
480
+ * @return it returns the new alarm changed.
481
+ */
482
+inline RRDCALC *rrdcalc_create_from_rrdcalc(RRDCALC *rc, RRDHOST *host, const char *name, const char *dimension) {
483
+ RRDCALC *newrc = callocz(1, sizeof(RRDCALC));
484
+
485
+ newrc->next_event_id = 1;
486
+ newrc->id = rrdcalc_get_unique_id(host, rc->chart, name, &rc->next_event_id);
487
+ newrc->name = (char *)name;
488
+ newrc->hash = simple_hash(newrc->name);
489
+ newrc->chart = strdupz(rc->chart);
490
+ newrc->hash_chart = simple_hash(rc->chart);
491
+
492
+ newrc->dimensions = strdupz(dimension);
493
+ newrc->foreachdim = NULL;
494
+ rc->foreachcounter++;
495
+ newrc->foreachcounter = rc->foreachcounter;
496
+
497
+ newrc->green = rc->green;
498
+ newrc->red = rc->red;
499
+ newrc->value = NAN;
500
+ newrc->old_value = NAN;
501
+
502
+ newrc->delay_up_duration = rc->delay_up_duration;
503
+ newrc->delay_down_duration = rc->delay_down_duration;
504
+ newrc->delay_max_duration = rc->delay_max_duration;
505
+ newrc->delay_multiplier = rc->delay_multiplier;
506
+
507
+ newrc->last_repeat = 0;
508
+ newrc->warn_repeat_every = rc->warn_repeat_every;
509
+ newrc->crit_repeat_every = rc->crit_repeat_every;
510
+
511
+ newrc->group = rc->group;
512
+ newrc->after = rc->after;
513
+ newrc->before = rc->before;
514
+ newrc->update_every = rc->update_every;
515
+ newrc->options = rc->options;
516
+
517
+ if(rc->exec) newrc->exec = strdupz(rc->exec);
518
+ if(rc->recipient) newrc->recipient = strdupz(rc->recipient);
519
+ if(rc->source) newrc->source = strdupz(rc->source);
520
+ if(rc->units) newrc->units = strdupz(rc->units);
521
+ if(rc->info) newrc->info = strdupz(rc->info);
522
+
523
+ if(rc->calculation) {
524
+ newrc->calculation = expression_parse(rc->calculation->source, NULL, NULL);
525
+ if(!newrc->calculation)
526
+ error("Health alarm '%s.%s': failed to parse calculation expression '%s'", rc->chart, rc->name, rc->calculation->source);
527
+ }
528
+
529
+ if(rc->warning) {
530
+ newrc->warning = expression_parse(rc->warning->source, NULL, NULL);
531
+ if(!newrc->warning)
532
+ error("Health alarm '%s.%s': failed to re-parse warning expression '%s'", rc->chart, rc->name, rc->warning->source);
533
+ }
534
+
535
+ if(rc->critical) {
536
+ newrc->critical = expression_parse(rc->critical->source, NULL, NULL);
537
+ if(!newrc->critical)
538
+ error("Health alarm '%s.%s': failed to re-parse critical expression '%s'", rc->chart, rc->name, rc->critical->source);
539
+ }
540
+
541
+ return newrc;
542
+}
543
+
544
void rrdcalc_free(RRDCALC *rc) {
545
if(unlikely(!rc)) return;
546
401
-
547
expression_free(rc->calculation);
548
expression_free(rc->warning);
549
expression_free(rc->critical);
@@ -407,11 +552,13 @@ void rrdcalc_free(RRDCALC *rc) {
552
freez(rc->chart);
553
freez(rc->family);
554
freez(rc->dimensions);
555
+ freez(rc->foreachdim);
556
freez(rc->exec);
557
freez(rc->recipient);
558
freez(rc->source);
559
freez(rc->units);
560
freez(rc->info);
561
+ simple_pattern_free(rc->spdim);
562
freez(rc);
563
}
564
database/rrdcalc.h
+9
-2
@@ -37,7 +37,7 @@ struct rrdcalc {
37
uint32_t next_event_id; // the next event id that will be used for this alarm
38
39
char *name; // the name of this alarm
40
- uint32_t hash;
40
+ uint32_t hash; // the hash of the alarm name
41
42
char *exec; // the command to execute when this alarm switches state
43
char *recipient; // the recipient of the alarm (the first parameter to exec)
@@ -59,7 +59,11 @@ struct rrdcalc {
59
// database lookup settings
60
61
char *dimensions; // the chart dimensions
62
- RRDR_GROUPING group; // grouping method: average, max, etc.
62
+ char *foreachdim; // the group of dimensions that the `foreach` will be applied.
63
+ SIMPLE_PATTERN *spdim; // used if and only if there is a simple pattern for the chart.
64
+ int foreachcounter; // the number of alarms created with foreachdim, this also works as an id of the
65
+ // children
66
+ RRDR_GROUPING group; // grouping method: average, max, etc.
67
int before; // ending point in time-series
68
int after; // starting point in time-series
69
uint32_t options; // calculation options
@@ -148,7 +152,10 @@ extern void rrdcalc_unlink_and_free(RRDHOST *host, RRDCALC *rc);
152
extern int rrdcalc_exists(RRDHOST *host, const char *chart, const char *name, uint32_t hash_chart, uint32_t hash_name);
153
extern uint32_t rrdcalc_get_unique_id(RRDHOST *host, const char *chart, const char *name, uint32_t *next_event_id);
154
extern RRDCALC *rrdcalc_create_from_template(RRDHOST *host, RRDCALCTEMPLATE *rt, const char *chart);
155
+extern RRDCALC *rrdcalc_create_from_rrdcalc(RRDCALC *rc, RRDHOST *host, const char *name, const char *dimension);
156
extern void rrdcalc_add_to_host(RRDHOST *host, RRDCALC *rc);
157
+extern void dimension_remove_pipe_comma(char *str);
158
+extern char *alarm_name_with_dim(char *name, size_t namelen, const char *dim, size_t dimlen);
159
160
static inline int rrdcalc_isrepeating(RRDCALC *rc) {
161
if (unlikely(rc->warn_repeat_every > 0 || rc->crit_repeat_every > 0)) {
database/rrdcalctemplate.c
+24
-12
@@ -5,23 +5,35 @@
5
6
// ----------------------------------------------------------------------------
7
// RRDCALCTEMPLATE management
8
+/**
9
+ * RRDCALC TEMPLATE LINK MATCHING
10
+ *
11
+ * @param rt is the template used to create the chart.
12
+ * @param st is the chart where the alarm will be attached.
13
+ */
14
+void rrdcalctemplate_link_matching_test(RRDCALCTEMPLATE *rt, RRDSET *st, RRDHOST *host ) {
15
+ if(rt->hash_context == st->hash_context && !strcmp(rt->context, st->context)
16
+ && (!rt->family_pattern || simple_pattern_matches(rt->family_pattern, st->family))) {
17
+ RRDCALC *rc = rrdcalc_create_from_template(host, rt, st->id);
18
+ if(unlikely(!rc))
19
+ info("Health tried to create alarm from template '%s' on chart '%s' of host '%s', but it failed", rt->name, st->id, host->hostname);
20
+#ifdef NETDATA_INTERNAL_CHECKS
21
+ else if(rc->rrdset != st && !rc->foreachdim) //When we have a template with foreadhdim, the child will be added to the index late
22
+ error("Health alarm '%s.%s' should be linked to chart '%s', but it is not", rc->chart?rc->chart:"NOCHART", rc->name, st->id);
23
+#endif
24
+ }
25
+}
26
27
void rrdcalctemplate_link_matching(RRDSET *st) {
28
RRDHOST *host = st->rrdhost;
29
RRDCALCTEMPLATE *rt;
30
31
for(rt = host->templates; rt ; rt = rt->next) {
14
- if(rt->hash_context == st->hash_context && !strcmp(rt->context, st->context)
15
- && (!rt->family_pattern || simple_pattern_matches(rt->family_pattern, st->family))) {
16
- RRDCALC *rc = rrdcalc_create_from_template(host, rt, st->id);
17
- if(unlikely(!rc))
18
- info("Health tried to create alarm from template '%s' on chart '%s' of host '%s', but it failed", rt->name, st->id, host->hostname);
32
+ rrdcalctemplate_link_matching_test(rt, st, host);
33
+ }
34
20
-#ifdef NETDATA_INTERNAL_CHECKS
21
- else if(rc->rrdset != st)
22
- error("Health alarm '%s.%s' should be linked to chart '%s', but it is not", rc->chart?rc->chart:"NOCHART", rc->name, st->id);
23
-#endif
24
- }
35
+ for(rt = host->alarms_template_with_foreach; rt ; rt = rt->next) {
36
+ rrdcalctemplate_link_matching_test(rt, st, host);
37
}
38
}
39
@@ -43,6 +55,8 @@ inline void rrdcalctemplate_free(RRDCALCTEMPLATE *rt) {
55
freez(rt->units);
56
freez(rt->info);
57
freez(rt->dimensions);
58
+ freez(rt->foreachdim);
59
+ simple_pattern_free(rt->spdim);
60
freez(rt);
61
}
62
@@ -67,5 +81,3 @@ inline void rrdcalctemplate_unlink_and_free(RRDHOST *host, RRDCALCTEMPLATE *rt)
81
82
rrdcalctemplate_free(rt);
83
}
70
-
71
-
database/rrdcalctemplate.h
+6
-2
@@ -35,7 +35,11 @@ struct rrdcalctemplate {
35
// database lookup settings
36
37
char *dimensions; // the chart dimensions
38
- RRDR_GROUPING group; // grouping method: average, max, etc.
38
+ char *foreachdim; // the group of dimensions that the lookup will be applied.
39
+ SIMPLE_PATTERN *spdim; // used if and only if there is a simple pattern for the chart.
40
+ int foreachcounter; // the number of alarms created with foreachdim, this also works as an id of the
41
+ // children
42
+ RRDR_GROUPING group; // grouping method: average, max, etc.
43
int before; // ending point in time-series
44
int after; // starting point in time-series
45
uint32_t options; // calculation options
@@ -70,5 +74,5 @@ extern void rrdcalctemplate_link_matching(RRDSET *st);
74
75
extern void rrdcalctemplate_free(RRDCALCTEMPLATE *rt);
76
extern void rrdcalctemplate_unlink_and_free(RRDHOST *host, RRDCALCTEMPLATE *rt);
73
-
77
+extern void rrdcalctemplate_create_alarms(RRDHOST *host, RRDCALCTEMPLATE *rt, RRDSET *st);
78
#endif //NETDATA_RRDCALCTEMPLATE_H
database/rrddim.c
+35
@@ -156,6 +156,35 @@ static time_t rrddim_query_oldest_time(RRDDIM *rd) {
156
// ----------------------------------------------------------------------------
157
// RRDDIM create a dimension
158
159
+void rrdcalc_link_to_rrddim(RRDDIM *rd, RRDSET *st, RRDHOST *host) {
160
+ RRDCALC *rrdc;
161
+ for (rrdc = host->alarms_with_foreach; rrdc ; rrdc = rrdc->next) {
162
+ if (simple_pattern_matches(rrdc->spdim, rd->id) || simple_pattern_matches(rrdc->spdim, rd->name)) {
163
+ if (!strcmp(rrdc->chart, st->name)) {
164
+ char *usename = alarm_name_with_dim(rrdc->name, strlen(rrdc->name), rd->name, strlen(rd->name));
165
+ if (usename) {
166
+ if(rrdcalc_exists(host, st->name, usename, 0, 0)){
167
+ freez(usename);
168
+ continue;
169
+ }
170
+
171
+ RRDCALC *child = rrdcalc_create_from_rrdcalc(rrdc, host, usename, rd->name);
172
+ if (child) {
173
+ rrdcalc_add_to_host(host, child);
174
+ RRDCALC *rdcmp = (RRDCALC *) avl_insert_lock(&(host)->alarms_idx_health_log,(avl *)child);
175
+ if (rdcmp != child) {
176
+ error("Cannot insert the alarm index ID %s",child->name);
177
+ }
178
+ } else {
179
+ error("Cannot allocate a new alarm.");
180
+ rrdc->foreachcounter--;
181
+ }
182
+ }
183
+ }
184
+ }
185
+ }
186
+}
187
+
188
RRDDIM *rrddim_add_custom(RRDSET *st, const char *id, const char *name, collected_number multiplier, collected_number divisor, RRD_ALGORITHM algorithm, RRD_MEMORY_MODE memory_mode) {
189
rrdset_wrlock(st);
190
@@ -371,6 +400,12 @@ RRDDIM *rrddim_add_custom(RRDSET *st, const char *id, const char *name, collecte
400
if(unlikely(rrddim_index_add(st, rd) != rd))
401
error("RRDDIM: INTERNAL ERROR: attempt to index duplicate dimension '%s' on chart '%s'", rd->id, st->id);
402
403
+ if(host->alarms_with_foreach || host->alarms_template_with_foreach) {
404
+ rrdhost_wrlock(host);
405
+ rrdcalc_link_to_rrddim(rd, st, host);
406
+
407
+ rrdhost_unlock(host);
408
+ }
409
rrdset_unlock(st);
410
return(rd);
411
}
database/rrdhost.c
+14
@@ -598,9 +598,23 @@ void rrdhost_free(RRDHOST *host) {
598
while(host->alarms)
599
rrdcalc_unlink_and_free(host, host->alarms);
600
601
+ RRDCALC *rc,*nc;
602
+ for(rc = host->alarms_with_foreach; rc ; rc = nc) {
603
+ nc = rc->next;
604
+ rrdcalc_free(rc);
605
+ }
606
+ host->alarms_with_foreach = NULL;
607
+
608
while(host->templates)
609
rrdcalctemplate_unlink_and_free(host, host->templates);
610
611
+ RRDCALCTEMPLATE *rt,*next;
612
+ for(rt = host->alarms_template_with_foreach; rt ; rt = next) {
613
+ next = rt->next;
614
+ rrdcalctemplate_free(rt);
615
+ }
616
+ host->alarms_template_with_foreach = NULL;
617
+
618
debug(D_RRD_CALLS, "RRDHOST: Cleaning up remaining host variables for host '%s'", host->hostname);
619
rrdvar_free_remaining_variables(host, &host->rrdvar_root_index);
620
health/README.md
+43
-1
@@ -163,7 +163,7 @@ This line makes a database lookup to find a value. This result of this lookup is
163
The format is:
164
165
```
166
-lookup: METHOD AFTER [at BEFORE] [every DURATION] [OPTIONS] [of DIMENSIONS]
166
+lookup: METHOD AFTER [at BEFORE] [every DURATION] [OPTIONS] [of DIMENSIONS] [foreach DIMENSIONS]
167
```
168
169
Everything is the same with [badges](../web/api/badges/). In short:
@@ -190,6 +190,11 @@ Everything is the same with [badges](../web/api/badges/). In short:
190
have spaces in their names). This accepts Netdata simple patterns and the `match-ids` and
191
`match-names` options affect the searches for dimensions.
192
193
+- `foreach DIMENSIONS` is optional, will always be the last parameter, and uses the same `,`/`|`
194
+ rules as the `of` parameter. Each dimension you specify in `foreach` will use the same rule
195
+ to trigger an alarm. If you set both `of` and `foreach`, Netdata will ignore the `of` parameter
196
+ and replace it with one of the dimensions you gave to `foreach`.
197
+
198
The result of the lookup will be available as `$this` and `$NAME` in expressions.
199
The timestamps of the timeframe evaluated by the database lookup is available as variables
200
`$after` and `$before` (both are unix timestamps).
@@ -660,6 +665,43 @@ Note that the drops chart does not exist if a network interface has never droppe
665
When Netdata detects a dropped packet, it will add the chart and it will automatically attach this
666
alarm to it.
667
668
+### Example 5
669
+
670
+Check if user or system dimension is using more than 50% of cpu:
671
+
672
+```
673
+ alarm: dim_template
674
+ on: system.cpu
675
+ os: linux
676
+lookup: average -3s percentage foreach system,user
677
+ units: %
678
+ every: 10s
679
+ warn: $this > 50
680
+ crit: $this > 80
681
+```
682
+
683
+The `lookup` line will calculate the average CPU usage from system and user in the last 3 seconds. Because we have
684
+the foreach in the `lookup` line, Netdata will create two independent alarms called `dim_template_system`
685
+and `dim_template_user` that will have all the other parameters shared among them.
686
+
687
+### Example 6
688
+
689
+Check if all dimensions are using more than 50% of cpu:
690
+
691
+```
692
+ alarm: dim_template
693
+ on: system.cpu
694
+ os: linux
695
+lookup: average -3s percentage foreach *
696
+ units: %
697
+ every: 10s
698
+ warn: $this > 50
699
+ crit: $this > 80
700
+```
701
+
702
+The `lookup` line will calculate the average of CPU usage from system and user in the last 3 seconds. In this case
703
+Netdata will create alarms for all dimensions of the chart.
704
+
705
## Troubleshooting
706
707
You can compile Netdata with [debugging](../daemon#debugging) and then set in `netdata.conf`:
health/health.c
+23
@@ -113,9 +113,23 @@ void health_reload_host(RRDHOST *host) {
113
while(host->templates)
114
rrdcalctemplate_unlink_and_free(host, host->templates);
115
116
+ RRDCALCTEMPLATE *rt,*next;
117
+ for(rt = host->alarms_template_with_foreach; rt ; rt = next) {
118
+ next = rt->next;
119
+ rrdcalctemplate_free(rt);
120
+ }
121
+ host->alarms_template_with_foreach = NULL;
122
+
123
while(host->alarms)
124
rrdcalc_unlink_and_free(host, host->alarms);
125
126
+ RRDCALC *rc,*nc;
127
+ for(rc = host->alarms_with_foreach; rc ; rc = nc) {
128
+ nc = rc->next;
129
+ rrdcalc_free(rc);
130
+ }
131
+ host->alarms_with_foreach = NULL;
132
+
133
rrdhost_unlock(host);
134
135
// invalidate all previous entries in the alarm log
@@ -139,9 +153,17 @@ void health_reload_host(RRDHOST *host) {
153
health_readdir(host, user_path, stock_path, NULL);
154
155
// link the loaded alarms to their charts
156
+ RRDDIM *rd;
157
rrdset_foreach_write(st, host) {
158
rrdsetcalc_link_matching(st);
159
rrdcalctemplate_link_matching(st);
160
+
161
+ //This loop must be the last, because ` rrdcalctemplate_link_matching` will create alarms related to it.
162
+ rrdset_rdlock(st);
163
+ rrddim_foreach_read(rd, st) {
164
+ rrdcalc_link_to_rrddim(rd, st, host);
165
+ }
166
+ rrdset_unlock(st);
167
}
168
169
rrdhost_unlock(host);
@@ -888,6 +910,7 @@ void *health_main(void *ptr) {
910
}
911
}
912
}
913
+
914
if(unlikely(repeat_every > 0 && (rc->last_repeat + repeat_every) <= now)) {
915
rc->last_repeat = now;
916
ALARM_ENTRY *ae = health_create_alarm_entry(
health/health.h
+3
@@ -48,6 +48,7 @@ extern unsigned int default_health_enabled;
48
#define HEALTH_INFO_KEY "info"
49
#define HEALTH_DELAY_KEY "delay"
50
#define HEALTH_OPTIONS_KEY "options"
51
+#define HEALTH_FOREACH_KEY "foreach"
52
53
#define HEALTH_SILENCERS_MAX_FILE_LEN 10000
54
@@ -106,4 +107,6 @@ extern void health_alarm_log_free_one_nochecks_nounlink(ALARM_ENTRY *ae);
107
108
extern void *health_cmdapi_thread(void *ptr);
109
110
+extern SIMPLE_PATTERN *health_pattern_from_foreach(char *s);
111
+
112
#endif //NETDATA_HEALTH_H
health/health_config.c
+131
-52
@@ -46,7 +46,7 @@ static inline int rrdcalc_add_alarm_from_config(RRDHOST *host, RRDCALC *rc) {
46
47
rc->id = rrdcalc_get_unique_id(host, rc->chart, rc->name, &rc->next_event_id);
48
49
- debug(D_HEALTH, "Health configuration adding alarm '%s.%s' (%u): exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
49
+ debug(D_HEALTH, "Health configuration adding alarm '%s.%s' (%u): exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', for each dimension '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
50
rc->chart?rc->chart:"NOCHART",
51
rc->name,
52
rc->id,
@@ -59,6 +59,7 @@ static inline int rrdcalc_add_alarm_from_config(RRDHOST *host, RRDCALC *rc) {
59
rc->before,
60
rc->options,
61
(rc->dimensions)?rc->dimensions:"NONE",
62
+ (rc->foreachdim)?rc->foreachdim:"NONE",
63
rc->update_every,
64
(rc->calculation)?rc->calculation->parsed_as:"NONE",
65
(rc->warning)?rc->warning->parsed_as:"NONE",
@@ -73,6 +74,7 @@ static inline int rrdcalc_add_alarm_from_config(RRDHOST *host, RRDCALC *rc) {
74
);
75
76
rrdcalc_add_to_host(host, rc);
77
+
78
return 1;
79
}
80
@@ -93,48 +95,70 @@ static inline int rrdcalctemplate_add_template_from_config(RRDHOST *host, RRDCAL
95
}
96
97
RRDCALCTEMPLATE *t, *last = NULL;
96
- for (t = host->templates; t ; last = t, t = t->next) {
97
- if(unlikely(t->hash_name == rt->hash_name
98
- && !strcmp(t->name, rt->name)
99
- && !strcmp(t->family_match?t->family_match:"*", rt->family_match?rt->family_match:"*")
100
- )) {
101
- error("Health configuration template '%s' already exists for host '%s'.", rt->name, host->hostname);
102
- return 0;
98
+ if(!rt->foreachdim) {
99
+ for (t = host->templates; t ; last = t, t = t->next) {
100
+ if(unlikely(t->hash_name == rt->hash_name
101
+ && !strcmp(t->name, rt->name)
102
+ && !strcmp(t->family_match?t->family_match:"*", rt->family_match?rt->family_match:"*")
103
+ )) {
104
+ error("Health configuration template '%s' already exists for host '%s'.", rt->name, host->hostname);
105
+ return 0;
106
+ }
107
+ }
108
+
109
+ if(likely(last)) {
110
+ last->next = rt;
111
+ }
112
+ else {
113
+ rt->next = host->templates;
114
+ host->templates = rt;
115
+ }
116
+ } else {
117
+ for (t = host->alarms_template_with_foreach; t ; last = t, t = t->next) {
118
+ if(unlikely(t->hash_name == rt->hash_name
119
+ && !strcmp(t->name, rt->name)
120
+ && !strcmp(t->family_match?t->family_match:"*", rt->family_match?rt->family_match:"*")
121
+ )) {
122
+ error("Health configuration template '%s' already exists for host '%s'.", rt->name, host->hostname);
123
+ return 0;
124
+ }
125
+ }
126
+
127
+ if(likely(last)) {
128
+ last->next = rt;
129
+ }
130
+ else {
131
+ rt->next = host->alarms_template_with_foreach;
132
+ host->alarms_template_with_foreach = rt;
133
}
134
}
135
106
- debug(D_HEALTH, "Health configuration adding template '%s': context '%s', exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
107
- rt->name,
108
- (rt->context)?rt->context:"NONE",
109
- (rt->exec)?rt->exec:"DEFAULT",
110
- (rt->recipient)?rt->recipient:"DEFAULT",
111
- rt->green,
112
- rt->red,
113
- (int)rt->group,
114
- rt->after,
115
- rt->before,
116
- rt->options,
117
- (rt->dimensions)?rt->dimensions:"NONE",
118
- rt->update_every,
119
- (rt->calculation)?rt->calculation->parsed_as:"NONE",
120
- (rt->warning)?rt->warning->parsed_as:"NONE",
121
- (rt->critical)?rt->critical->parsed_as:"NONE",
122
- rt->source,
123
- rt->delay_up_duration,
124
- rt->delay_down_duration,
125
- rt->delay_max_duration,
126
- rt->delay_multiplier,
127
- rt->warn_repeat_every,
128
- rt->crit_repeat_every
136
+ debug(D_HEALTH, "Health configuration adding template '%s': context '%s', exec '%s', recipient '%s', green " CALCULATED_NUMBER_FORMAT_AUTO ", red " CALCULATED_NUMBER_FORMAT_AUTO ", lookup: group %d, after %d, before %d, options %u, dimensions '%s', for each dimension '%s', update every %d, calculation '%s', warning '%s', critical '%s', source '%s', delay up %d, delay down %d, delay max %d, delay_multiplier %f, warn_repeat_every %u, crit_repeat_every %u",
137
+ rt->name,
138
+ (rt->context)?rt->context:"NONE",
139
+ (rt->exec)?rt->exec:"DEFAULT",
140
+ (rt->recipient)?rt->recipient:"DEFAULT",
141
+ rt->green,
142
+ rt->red,
143
+ (int)rt->group,
144
+ rt->after,
145
+ rt->before,
146
+ rt->options,
147
+ (rt->dimensions)?rt->dimensions:"NONE",
148
+ (rt->foreachdim)?rt->foreachdim:"NONE",
149
+ rt->update_every,
150
+ (rt->calculation)?rt->calculation->parsed_as:"NONE",
151
+ (rt->warning)?rt->warning->parsed_as:"NONE",
152
+ (rt->critical)?rt->critical->parsed_as:"NONE",
153
+ rt->source,
154
+ rt->delay_up_duration,
155
+ rt->delay_down_duration,
156
+ rt->delay_max_duration,
157
+ rt->delay_multiplier,
158
+ rt->warn_repeat_every,
159
+ rt->crit_repeat_every
160
);
161
131
- if(likely(last)) {
132
- last->next = rt;
133
- }
134
- else {
135
- rt->next = host->templates;
136
- host->templates = rt;
137
- }
162
163
return 1;
164
}
@@ -291,16 +315,37 @@ static inline int health_parse_repeat(
315
return 1;
316
}
317
318
+/**
319
+ * Health pattern from Foreach
320
+ *
321
+ * Create a new simple pattern using the user input
322
+ *
323
+ * @param s the string that will be used to create the simple pattern.
324
+ */
325
+SIMPLE_PATTERN *health_pattern_from_foreach(char *s) {
326
+ char *convert= strdupz(s);
327
+ SIMPLE_PATTERN *val = NULL;
328
+ if(convert) {
329
+ dimension_remove_pipe_comma(convert);
330
+ val = simple_pattern_create(convert, NULL, SIMPLE_PATTERN_EXACT);
331
+
332
+ freez(convert);
333
+ }
334
+
335
+ return val;
336
+}
337
338
static inline int health_parse_db_lookup(
339
size_t line, const char *filename, char *string,
340
RRDR_GROUPING *group_method, int *after, int *before, int *every,
298
- uint32_t *options, char **dimensions
341
+ uint32_t *options, char **dimensions, char **foreachdim
342
) {
343
debug(D_HEALTH, "Health configuration parsing database lookup %zu@%s: %s", line, filename, string);
344
345
if(*dimensions) freez(*dimensions);
346
+ if(*foreachdim) freez(*foreachdim);
347
*dimensions = NULL;
348
+ *foreachdim = NULL;
349
*after = 0;
350
*before = 0;
351
*every = 0;
@@ -387,8 +432,22 @@ static inline int health_parse_db_lookup(
432
*options |= RRDR_OPTION_MATCH_NAMES;
433
}
434
else if(!strcasecmp(key, "of")) {
390
- if(*s && strcasecmp(s, "all") != 0)
435
+ char *find = NULL;
436
+ if(*s && strcasecmp(s, "all") != 0) {
437
+ find = strcasestr(s, " foreach");
438
+ if(find) {
439
+ *find = '\0';
440
+ }
441
*dimensions = strdupz(s);
442
+ }
443
+
444
+ if(!find) {
445
+ break;
446
+ }
447
+ s = ++find;
448
+ }
449
+ else if(!strcasecmp(key, HEALTH_FOREACH_KEY )) {
450
+ *foreachdim = strdupz(s);
451
break;
452
}
453
else {
@@ -521,8 +580,12 @@ static int health_readfile(const char *filename, void *data) {
580
uint32_t hash = simple_uhash(key);
581
582
if(hash == hash_alarm && !strcasecmp(key, HEALTH_ALARM_KEY)) {
524
- if (rc && (ignore_this || !rrdcalc_add_alarm_from_config(host, rc)))
525
- rrdcalc_free(rc);
583
+ if(rc) {
584
+ if(ignore_this || !rrdcalc_add_alarm_from_config(host, rc)) {
585
+ rrdcalc_free(rc);
586
+ }
587
+ // health_add_alarms_loop(host, rc, ignore_this) ;
588
+ }
589
590
if(rt) {
591
if (ignore_this || !rrdcalctemplate_add_template_from_config(host, rt))
@@ -552,14 +615,18 @@ static int health_readfile(const char *filename, void *data) {
615
}
616
else if(hash == hash_template && !strcasecmp(key, HEALTH_TEMPLATE_KEY)) {
617
if(rc) {
555
- if(ignore_this || !rrdcalc_add_alarm_from_config(host, rc))
618
+// health_add_alarms_loop(host, rc, ignore_this) ;
619
+ if(ignore_this || !rrdcalc_add_alarm_from_config(host, rc)) {
620
rrdcalc_free(rc);
621
+ }
622
623
rc = NULL;
624
}
625
561
- if(rt && (ignore_this || !rrdcalctemplate_add_template_from_config(host, rt)))
562
- rrdcalctemplate_free(rt);
626
+ if(rt) {
627
+ if(ignore_this || !rrdcalctemplate_add_template_from_config(host, rt))
628
+ rrdcalctemplate_free(rt);
629
+ }
630
631
rt = callocz(1, sizeof(RRDCALCTEMPLATE));
632
rt->name = strdupz(value);
@@ -622,8 +689,10 @@ static int health_readfile(const char *filename, void *data) {
689
}
690
else if(hash == hash_lookup && !strcasecmp(key, HEALTH_LOOKUP_KEY)) {
691
health_parse_db_lookup(line, filename, value, &rc->group, &rc->after, &rc->before,
625
- &rc->update_every,
626
- &rc->options, &rc->dimensions);
692
+ &rc->update_every, &rc->options, &rc->dimensions, &rc->foreachdim);
693
+ if(rc->foreachdim) {
694
+ rc->spdim = health_pattern_from_foreach(rc->foreachdim);
695
+ }
696
}
697
else if(hash == hash_every && !strcasecmp(key, HEALTH_EVERY_KEY)) {
698
if(!config_parse_duration(value, &rc->update_every))
@@ -752,7 +821,10 @@ static int health_readfile(const char *filename, void *data) {
821
}
822
else if(hash == hash_lookup && !strcasecmp(key, HEALTH_LOOKUP_KEY)) {
823
health_parse_db_lookup(line, filename, value, &rt->group, &rt->after, &rt->before,
755
- &rt->update_every, &rt->options, &rt->dimensions);
824
+ &rt->update_every, &rt->options, &rt->dimensions, &rt->foreachdim);
825
+ if(rt->foreachdim) {
826
+ rt->spdim = health_pattern_from_foreach(rt->foreachdim);
827
+ }
828
}
829
else if(hash == hash_every && !strcasecmp(key, HEALTH_EVERY_KEY)) {
830
if(!config_parse_duration(value, &rt->update_every))
@@ -866,11 +938,17 @@ static int health_readfile(const char *filename, void *data) {
938
}
939
}
940
869
- if(rc && (ignore_this || !rrdcalc_add_alarm_from_config(host, rc)))
870
- rrdcalc_free(rc);
941
+ if(rc) {
942
+ //health_add_alarms_loop(host, rc, ignore_this) ;
943
+ if(ignore_this || !rrdcalc_add_alarm_from_config(host, rc)) {
944
+ rrdcalc_free(rc);
945
+ }
946
+ }
947
872
- if(rt && (ignore_this || !rrdcalctemplate_add_template_from_config(host, rt)))
873
- rrdcalctemplate_free(rt);
948
+ if(rt) {
949
+ if(ignore_this || !rrdcalctemplate_add_template_from_config(host, rt))
950
+ rrdcalctemplate_free(rt);
951
+ }
952
953
fclose(fp);
954
return 1;
@@ -881,5 +959,6 @@ void health_readdir(RRDHOST *host, const char *user_path, const char *stock_path
959
debug(D_HEALTH, "CONFIG health is not enabled for host '%s'", host->hostname);
960
return;
961
}
962
+
963
recursive_config_double_dir_load(user_path, stock_path, subpath, health_readfile, (void *) host, 0);
964
}
tests/Makefile.am
+3
@@ -8,6 +8,7 @@ CLEANFILES = \
8
acls/acl.sh \
9
urls/request.sh \
10
alarm_repetition/alarm.sh \
11
+ template_dimension/template_dim.sh \
12
$(NULL)
13
14
include $(top_srcdir)/build/subst.inc
@@ -28,6 +29,7 @@ dist_noinst_DATA = \
29
acls/acl.sh.in \
30
urls/request.sh.in \
31
alarm_repetition/alarm.sh.in \
32
+ template_dimension/template_dim.sh.in \
33
$(NULL)
34
35
dist_plugins_SCRIPTS = \
@@ -35,6 +37,7 @@ dist_plugins_SCRIPTS = \
37
acls/acl.sh \
38
urls/request.sh \
39
alarm_repetition/alarm.sh \
40
+ template_dimension/template_dim.sh \
41
$(NULL)
42
43
dist_noinst_SCRIPTS = \
tests/template_dimension/system_cpu.conf.alarm_foreach
new
+8
@@ -0,0 +1,8 @@
1
+ alarm: dev_dim_template
2
+ on: system.cpu
3
+ os: linux
4
+lookup: sum -3s at 0 every 3 percentage foreach system,user,nice
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
tests/template_dimension/system_cpu.conf.alarm_foreach_sp
new
+8
@@ -0,0 +1,8 @@
1
+ alarm: dev_dim_template
2
+ on: system.cpu
3
+ os: linux
4
+lookup: sum -3s at 0 every 3 percentage foreach *
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
tests/template_dimension/system_cpu.conf.template_alarm
new
+26
@@ -0,0 +1,26 @@
1
+template: dev_dim_template_system
2
+ on: system.cpu
3
+ os: linux
4
+ lookup: sum -3s at 0 every 3 percentage of system
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
9
+
10
+template: dev_dim_template_user
11
+ on: system.cpu
12
+ os: linux
13
+ lookup: sum -3s at 0 every 3 percentage of user
14
+ units: %
15
+ every: 1s
16
+ warn: $this > 1
17
+ crit: $this > 4
18
+
19
+template: dev_dim_template_nice
20
+ on: system.cpu
21
+ os: linux
22
+ lookup: sum -3s at 0 every 3 percentage of nice
23
+ units: %
24
+ every: 1s
25
+ warn: $this > 1
26
+ crit: $this > 4
tests/template_dimension/system_cpu.conf.template_foreach
new
+8
@@ -0,0 +1,8 @@
1
+template: dev_dim_template
2
+ on: system.cpu
3
+ os: linux
4
+ lookup: sum -3s at 0 every 3 percentage foreach system,user,nice
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
tests/template_dimension/system_cpu.conf.template_foreach_sp
new
+8
@@ -0,0 +1,8 @@
1
+ template: dev_dim_template
2
+ on: system.cpu
3
+ os: linux
4
+ lookup: sum -3s at 0 every 3 percentage foreach *
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
tests/template_dimension/system_cpu.conf.unique_alarm
new
+26
@@ -0,0 +1,26 @@
1
+ alarm: dev_dim_template_system
2
+ on: system.cpu
3
+ os: linux
4
+lookup: sum -3s at 0 every 3 percentage of system
5
+ units: %
6
+ every: 1s
7
+ warn: $this > 1
8
+ crit: $this > 4
9
+
10
+ alarm: dev_dim_template_user
11
+ on: system.cpu
12
+ os: linux
13
+lookup: sum -3s at 0 every 3 percentage of user
14
+ units: %
15
+ every: 1s
16
+ warn: $this > 1
17
+ crit: $this > 4
18
+
19
+ alarm: dev_dim_template_nice
20
+ on: system.cpu
21
+ os: linux
22
+lookup: sum -3s at 0 every 3 percentage of nice
23
+ units: %
24
+ every: 1s
25
+ warn: $this > 1
26
+ crit: $this > 4
tests/template_dimension/template_dim.sh.in
new
+88
@@ -0,0 +1,88 @@
1
+#!/bin/bash
2
+
3
+#The health directory to put the alarms
4
+HEALTHDIR="@configdir_POST@/health.d/"
5
+
6
+#the current time
7
+OUTDIR="alarms"
8
+QUERY="/api/v1/alarms?all"
9
+MURL="http://localhost:19999$QUERY"
10
+
11
+#error messages
12
+RED='\033[0;31m'
13
+GREEN='\033[0;32m'
14
+NOCOLOR='\033[0m'
15
+
16
+ALARMTEST="dev_dim_template"
17
+
18
+change_alarm_file() {
19
+ if [ -f "$1" ]; then
20
+ rm "$1"
21
+ fi
22
+
23
+ #copy keeping the permissions
24
+ cp -a "$2" "$3"
25
+}
26
+
27
+netdata_test_download() {
28
+ grep "HTTP/1.1 200 OK" "$1" 2>/dev/null 1>/dev/null
29
+ TEST="$?"
30
+ if [ "$TEST" -ne "0" ]; then
31
+ echo -e "${RED} Error to get the alarm log. ${NOCOLOR}"
32
+ exit 1
33
+ fi
34
+
35
+ TOTALARM=$(grep "$ALARMTEST" "$2" | grep name | cut -d: -f2 | grep -c "$ALARMTEST")
36
+
37
+ if [ "$TOTALARM" -ne "$3" ]; then
38
+ echo -e "${RED} The number of actives alarms with the name $SYSTEMALARM is wrong ${NOCOLOR}"
39
+ exit 1
40
+ fi
41
+}
42
+
43
+get_the_logs() {
44
+ curl -v -k --create-dirs -o "$OUTDIR/$1.out" "$MURL" 2> "$OUTDIR/$1.err"
45
+ netdata_test_download "$OUTDIR/$1.err" "$OUTDIR/$1.out" "$2"
46
+}
47
+
48
+process_data() {
49
+ netdata -D &
50
+ NETDATAPID=$!
51
+ echo -e "${NOCOLOR}Sleeping during 15 seconds to create alarms"
52
+ sleep 15
53
+ kill $NETDATAPID
54
+ get_the_logs "$1" "$2"
55
+}
56
+
57
+mkdir "$OUTDIR"
58
+CREATEDIR="$?"
59
+if [ "$CREATEDIR" -ne "0" ]; then
60
+ echo -e "${RED}Cannot create the output directory, it already exists. The test will overwrite previous results. ${NOCOLOR}"
61
+fi
62
+
63
+if [ -n "$1" ]; then
64
+ MURL="$1$QUERY"
65
+fi
66
+
67
+change_alarm_file "./0" "system_cpu.conf.unique_alarm" "$HEALTHDIR/dim_double_without_template.conf"
68
+process_data "double_without_template" 3 "$HEALTHDIR/dim_double_without_template.conf"
69
+
70
+change_alarm_file "$HEALTHDIR/dim_double_without_template.conf" "system_cpu.conf.alarm_foreach" "$HEALTHDIR/dim_foreach_without_template.conf"
71
+process_data "foreach_without_template" 3 "$HEALTHDIR/dim_foreach_without_template.conf"
72
+
73
+change_alarm_file "$HEALTHDIR/dim_foreach_without_template.conf" "system_cpu.conf.alarm_foreach_sp" "$HEALTHDIR/dim_foreach_without_template_sp.conf"
74
+process_data "foreach_without_template" 10 "$HEALTHDIR/dim_foreach_without_template_sp.conf"
75
+
76
+change_alarm_file "$HEALTHDIR/dim_foreach_without_template_sp.conf" "system_cpu.conf.template_alarm" "$HEALTHDIR/dim_double_with_template.conf"
77
+process_data "double_with_template" 3 "$HEALTHDIR/dim_double_with_template.conf"
78
+
79
+change_alarm_file "$HEALTHDIR/dim_double_with_template.conf" "system_cpu.conf.template_foreach" "$HEALTHDIR/dim_foreach_with_template.conf"
80
+process_data "foreach_with_template" 3 "$HEALTHDIR/dim_foreach_with_template.conf"
81
+
82
+change_alarm_file "$HEALTHDIR/dim_foreach_with_template.conf" "system_cpu.conf.template_foreach_sp" "$HEALTHDIR/dim_foreach_with_template_sp.conf"
83
+process_data "foreach_with_template" 10 "$HEALTHDIR/dim_foreach_with_template_sp.conf"
84
+
85
+rm "$HEALTHDIR/dim_foreach_with_template_sp.conf"
86
+rm -rf "$OUTDIR"
87
+
88
+echo -e "${GREEN} all the tests were sucessful ${NOCOLOR}"