@cryptotaxi247 / netdata-1 / commits / 464695b41

Add chart filtering parameter to the allmetrics API query (#12820)

* Add chart filtering in the allmetrics API call * Fix compilation warnings * Remove unnecessary function * Update the documentation * Apply suggestions from code review * Check for filter instead of filter_string * Do not check both - chart id and name for prometheus and shell formats * Fix unit tests Co-authored-by: Ilya Mashchenko <ilya@netdata.cloud>

Vladimir Kobal committed May 5, 2022 at 19:32 UTC 464695b41069aa9ac39c5daecdd15b1d89b75bdd
12 files changed +166 -89
database/sqlite/sqlite_aclk_alert.c
+1
@@ -198,6 +198,7 @@ bind_fail:
198 #else
199 UNUSED(host);
200 UNUSED(ae);
201 + UNUSED(skip_filter);
202 #endif
203 return 0;
204 }
exporting/README.md
+78 -56
@@ -28,61 +28,82 @@ X seconds (though, it can send them per second if you need it to).
28
29 ## Features
30
31 -1. The exporting engine uses a number of connectors to send Netdata metrics to external time-series databases. See our
32 - [list of supported databases](/docs/export/external-databases.md#supported-databases) for information on which
33 - connector to enable and configure for your database of choice.
34 -
35 - - [**AWS Kinesis Data Streams**](/exporting/aws_kinesis/README.md): Metrics are sent to the service in `JSON`
36 - format.
37 - - [**Google Cloud Pub/Sub Service**](/exporting/pubsub/README.md): Metrics are sent to the service in `JSON`
38 - format.
39 - - [**Graphite**](/exporting/graphite/README.md): A plaintext interface. Metrics are sent to the database server as
40 - `prefix.hostname.chart.dimension`. `prefix` is configured below, `hostname` is the hostname of the machine (can
41 - also be configured). Learn more in our guide to [export and visualize Netdata metrics in
42 - Graphite](/docs/guides/export/export-netdata-metrics-graphite.md).
43 - - [**JSON** document databases](/exporting/json/README.md)
44 - - [**OpenTSDB**](/exporting/opentsdb/README.md): Use a plaintext or HTTP interfaces. Metrics are sent to
45 - OpenTSDB as `prefix.chart.dimension` with tag `host=hostname`.
46 - - [**MongoDB**](/exporting/mongodb/README.md): Metrics are sent to the database in `JSON` format.
47 - - [**Prometheus**](/exporting/prometheus/README.md): Use an existing Prometheus installation to scrape metrics
48 - from node using the Netdata API.
49 - - [**Prometheus remote write**](/exporting/prometheus/remote_write/README.md). A binary snappy-compressed protocol
50 - buffer encoding over HTTP. Supports many [storage
51 - providers](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).
52 - - [**TimescaleDB**](/exporting/TIMESCALE.md): Use a community-built connector that takes JSON streams from a
53 - Netdata client and writes them to a TimescaleDB table.
54 -
55 -2. Netdata can filter metrics (at the chart level), to send only a subset of the collected metrics.
56 -
57 -3. Netdata supports three modes of operation for all exporting connectors:
58 -
59 - - `as-collected` sends to external databases the metrics as they are collected, in the units they are collected.
60 - So, counters are sent as counters and gauges are sent as gauges, much like all data collectors do. For example,
61 - to calculate CPU utilization in this format, you need to know how to convert kernel ticks to percentage.
62 -
63 - - `average` sends to external databases normalized metrics from the Netdata database. In this mode, all metrics
64 - are sent as gauges, in the units Netdata uses. This abstracts data collection and simplifies visualization, but
65 - you will not be able to copy and paste queries from other sources to convert units. For example, CPU utilization
66 - percentage is calculated by Netdata, so Netdata will convert ticks to percentage and send the average percentage
67 - to the external database.
68 -
69 - - `sum` or `volume`: the sum of the interpolated values shown on the Netdata graphs is sent to the external
70 - database. So, if Netdata is configured to send data to the database every 10 seconds, the sum of the 10 values
71 - shown on the Netdata charts will be used.
72 -
73 - Time-series databases suggest to collect the raw values (`as-collected`). If you plan to invest on building your
74 - monitoring around a time-series database and you already know (or you will invest in learning) how to convert units
75 - and normalize the metrics in Grafana or other visualization tools, we suggest to use `as-collected`.
76 -
77 - If, on the other hand, you just need long term archiving of Netdata metrics and you plan to mainly work with
78 - Netdata, we suggest to use `average`. It decouples visualization from data collection, so it will generally be a lot
79 - simpler. Furthermore, if you use `average`, the charts shown in the external service will match exactly what you
80 - see in Netdata, which is not necessarily true for the other modes of operation.
81 -
82 -4. This code is smart enough, not to slow down Netdata, independently of the speed of the external database server. You
83 - should keep in mind though that many exporting connector instances can consume a lot of CPU resources if they run
84 - their batches at the same time. You can set different update intervals for every exporting connector instance, but
85 - even in that case they can occasionally synchronize their batches for a moment.
31 +### Integration
32 +
33 +The exporting engine uses a number of connectors to send Netdata metrics to external time-series databases. See our
34 +[list of supported databases](/docs/export/external-databases.md#supported-databases) for information on which
35 +connector to enable and configure for your database of choice.
36 +
37 +- [**AWS Kinesis Data Streams**](/exporting/aws_kinesis/README.md): Metrics are sent to the service in `JSON`
38 + format.
39 +- [**Google Cloud Pub/Sub Service**](/exporting/pubsub/README.md): Metrics are sent to the service in `JSON`
40 + format.
41 +- [**Graphite**](/exporting/graphite/README.md): A plaintext interface. Metrics are sent to the database server as
42 + `prefix.hostname.chart.dimension`. `prefix` is configured below, `hostname` is the hostname of the machine (can
43 + also be configured). Learn more in our guide to [export and visualize Netdata metrics in
44 + Graphite](/docs/guides/export/export-netdata-metrics-graphite.md).
45 +- [**JSON** document databases](/exporting/json/README.md)
46 +- [**OpenTSDB**](/exporting/opentsdb/README.md): Use a plaintext or HTTP interfaces. Metrics are sent to
47 + OpenTSDB as `prefix.chart.dimension` with tag `host=hostname`.
48 +- [**MongoDB**](/exporting/mongodb/README.md): Metrics are sent to the database in `JSON` format.
49 +- [**Prometheus**](/exporting/prometheus/README.md): Use an existing Prometheus installation to scrape metrics
50 + from node using the Netdata API.
51 +- [**Prometheus remote write**](/exporting/prometheus/remote_write/README.md). A binary snappy-compressed protocol
52 + buffer encoding over HTTP. Supports many [storage
53 + providers](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).
54 +- [**TimescaleDB**](/exporting/TIMESCALE.md): Use a community-built connector that takes JSON streams from a
55 + Netdata client and writes them to a TimescaleDB table.
56 +
57 +### Chart filtering
58 +
59 +Netdata can filter metrics, to send only a subset of the collected metrics. You can use the
60 +configuration file
61 +
62 +```txt
63 +[prometheus:exporter]
64 + send charts matching = system.*
65 +```
66 +
67 +or the URL parameter `filter` in the `allmetrics` API call.
68 +
69 +```txt
70 +http://localhost:19999/api/v1/allmetrics?format=shell&filter=system.*
71 +```
72 +
73 +### Operation modes
74 +
75 +Netdata supports three modes of operation for all exporting connectors:
76 +
77 +- `as-collected` sends to external databases the metrics as they are collected, in the units they are collected.
78 + So, counters are sent as counters and gauges are sent as gauges, much like all data collectors do. For example,
79 + to calculate CPU utilization in this format, you need to know how to convert kernel ticks to percentage.
80 +
81 +- `average` sends to external databases normalized metrics from the Netdata database. In this mode, all metrics
82 + are sent as gauges, in the units Netdata uses. This abstracts data collection and simplifies visualization, but
83 + you will not be able to copy and paste queries from other sources to convert units. For example, CPU utilization
84 + percentage is calculated by Netdata, so Netdata will convert ticks to percentage and send the average percentage
85 + to the external database.
86 +
87 +- `sum` or `volume`: the sum of the interpolated values shown on the Netdata graphs is sent to the external
88 + database. So, if Netdata is configured to send data to the database every 10 seconds, the sum of the 10 values
89 + shown on the Netdata charts will be used.
90 +
91 +Time-series databases suggest to collect the raw values (`as-collected`). If you plan to invest on building your
92 +monitoring around a time-series database and you already know (or you will invest in learning) how to convert units
93 +and normalize the metrics in Grafana or other visualization tools, we suggest to use `as-collected`.
94 +
95 +If, on the other hand, you just need long term archiving of Netdata metrics and you plan to mainly work with
96 +Netdata, we suggest to use `average`. It decouples visualization from data collection, so it will generally be a lot
97 +simpler. Furthermore, if you use `average`, the charts shown in the external service will match exactly what you
98 +see in Netdata, which is not necessarily true for the other modes of operation.
99 +
100 +### Independent operation
101 +
102 +This code is smart enough, not to slow down Netdata, independently of the speed of the external database server.
103 +
104 +> ❗ You should keep in mind though that many exporting connector instances can consume a lot of CPU resources if they
105 +> run their batches at the same time. You can set different update intervals for every exporting connector instance,
106 +> but even in that case they can occasionally synchronize their batches for a moment.
107
108 ## Configuration
109
@@ -252,7 +273,8 @@ Configure individual connectors and override any global settings with the follow
273 within each pattern). The patterns are checked against both chart id and chart name. A pattern starting with `!`
274 gives a negative match. So to match all charts named `apps.*` except charts ending in `*reads`, use `!*reads
275 apps.*` (so, the order is important: the first pattern matching the chart id or the chart name will be used -
255 - positive or negative).
276 + positive or negative). There is also a URL parameter `filter` that can be used while querying `allmetrics`. The URL
277 + parameter has a higher priority than the configuration option.
278
279 - `send names instead of ids = yes | no` controls the metric names Netdata should send to the external database.
280 Netdata supports names and IDs for charts and dimensions. Usually IDs are unique identifiers as read by the system
exporting/prometheus/prometheus.c
+36 -17
@@ -12,9 +12,10 @@
12 *
13 * @param instance an instance data structure.
14 * @param st a chart.
15 + * @param filter a simple pattern to match against.
16 * @return Returns 1 if the chart can be sent, 0 otherwise.
17 */
17 -inline int can_send_rrdset(struct instance *instance, RRDSET *st)
18 +inline int can_send_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN *filter)
19 {
20 #ifdef NETDATA_INTERNAL_CHECKS
21 RRDHOST *host = st->rrdhost;
@@ -27,19 +28,29 @@ inline int can_send_rrdset(struct instance *instance, RRDSET *st)
28 if (unlikely(rrdset_flag_check(st, RRDSET_FLAG_EXPORTING_IGNORE)))
29 return 0;
30
30 - if (unlikely(!rrdset_flag_check(st, RRDSET_FLAG_EXPORTING_SEND))) {
31 - // we have not checked this chart
32 - if (simple_pattern_matches(instance->config.charts_pattern, st->id) ||
33 - simple_pattern_matches(instance->config.charts_pattern, st->name))
34 - rrdset_flag_set(st, RRDSET_FLAG_EXPORTING_SEND);
35 - else {
36 - rrdset_flag_set(st, RRDSET_FLAG_EXPORTING_IGNORE);
37 - debug(
38 - D_EXPORTING,
39 - "EXPORTING: not sending chart '%s' of host '%s', because it is disabled for exporting.",
40 - st->id,
41 - host->hostname);
42 - return 0;
31 + if (filter) {
32 + if (instance->config.options & EXPORTING_OPTION_SEND_NAMES) {
33 + if (!simple_pattern_matches(filter, st->name))
34 + return 0;
35 + } else {
36 + if (!simple_pattern_matches(filter, st->id))
37 + return 0;
38 + }
39 + } else {
40 + if (unlikely(!rrdset_flag_check(st, RRDSET_FLAG_EXPORTING_SEND))) {
41 + // we have not checked this chart
42 + if (simple_pattern_matches(instance->config.charts_pattern, st->id) ||
43 + simple_pattern_matches(instance->config.charts_pattern, st->name))
44 + rrdset_flag_set(st, RRDSET_FLAG_EXPORTING_SEND);
45 + else {
46 + rrdset_flag_set(st, RRDSET_FLAG_EXPORTING_IGNORE);
47 + debug(
48 + D_EXPORTING,
49 + "EXPORTING: not sending chart '%s' of host '%s', because it is disabled for exporting.",
50 + st->id,
51 + host->hostname);
52 + return 0;
53 + }
54 }
55 }
56
@@ -480,6 +491,7 @@ static void generate_as_collected_prom_metric(BUFFER *wb, struct gen_parameters
491 *
492 * @param instance an instance data structure.
493 * @param host a data collecting host.
494 + * @param filter_string a simple pattern filter.
495 * @param wb the buffer to fill with metrics.
496 * @param prefix a prefix for every metric.
497 * @param exporting_options options to configure what data is exported.
@@ -489,12 +501,14 @@ static void generate_as_collected_prom_metric(BUFFER *wb, struct gen_parameters
501 static void rrd_stats_api_v1_charts_allmetrics_prometheus(
502 struct instance *instance,
503 RRDHOST *host,
504 + const char *filter_string,
505 BUFFER *wb,
506 const char *prefix,
507 EXPORTING_OPTIONS exporting_options,
508 int allhosts,
509 PROMETHEUS_OUTPUT_OPTIONS output_options)
510 {
511 + SIMPLE_PATTERN *filter = simple_pattern_create(filter_string, NULL, SIMPLE_PATTERN_EXACT);
512 rrdhost_rdlock(host);
513
514 char hostname[PROMETHEUS_ELEMENT_MAX + 1];
@@ -592,7 +606,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
606 rrdset_foreach_read(st, host)
607 {
608
595 - if (likely(can_send_rrdset(instance, st))) {
609 + if (likely(can_send_rrdset(instance, st, filter))) {
610 rrdset_rdlock(st);
611
612 char chart[PROMETHEUS_ELEMENT_MAX + 1];
@@ -777,6 +791,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(
791 }
792
793 rrdhost_unlock(host);
794 + simple_pattern_free(filter);
795 }
796
797 /**
@@ -850,6 +865,7 @@ static inline time_t prometheus_preparation(
865 * Write metrics and auxiliary information for one host to a buffer.
866 *
867 * @param host a data collecting host.
868 + * @param filter_string a simple pattern filter.
869 * @param wb the buffer to write to.
870 * @param server the name of a Prometheus server.
871 * @param prefix a prefix for every metric.
@@ -858,6 +874,7 @@ static inline time_t prometheus_preparation(
874 */
875 void rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
876 RRDHOST *host,
877 + const char *filter_string,
878 BUFFER *wb,
879 const char *server,
880 const char *prefix,
@@ -880,13 +897,14 @@ void rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
897 output_options);
898
899 rrd_stats_api_v1_charts_allmetrics_prometheus(
883 - prometheus_exporter_instance, host, wb, prefix, exporting_options, 0, output_options);
900 + prometheus_exporter_instance, host, filter_string, wb, prefix, exporting_options, 0, output_options);
901 }
902
903 /**
904 * Write metrics and auxiliary information for all hosts to a buffer.
905 *
906 * @param host a data collecting host.
907 + * @param filter_string a simple pattern filter.
908 * @param wb the buffer to write to.
909 * @param server the name of a Prometheus server.
910 * @param prefix a prefix for every metric.
@@ -895,6 +913,7 @@ void rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
913 */
914 void rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(
915 RRDHOST *host,
916 + const char *filter_string,
917 BUFFER *wb,
918 const char *server,
919 const char *prefix,
@@ -920,7 +939,7 @@ void rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(
939 rrdhost_foreach_read(host)
940 {
941 rrd_stats_api_v1_charts_allmetrics_prometheus(
923 - prometheus_exporter_instance, host, wb, prefix, exporting_options, 1, output_options);
942 + prometheus_exporter_instance, host, filter_string, wb, prefix, exporting_options, 1, output_options);
943 }
944 rrd_unlock();
945 }
exporting/prometheus/prometheus.h
+3 -3
@@ -23,13 +23,13 @@ typedef enum prometheus_output_flags {
23 } PROMETHEUS_OUTPUT_OPTIONS;
24
25 extern void rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
26 - RRDHOST *host, BUFFER *wb, const char *server, const char *prefix,
26 + RRDHOST *host, const char *filter_string, BUFFER *wb, const char *server, const char *prefix,
27 EXPORTING_OPTIONS exporting_options, PROMETHEUS_OUTPUT_OPTIONS output_options);
28 extern void rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(
29 - RRDHOST *host, BUFFER *wb, const char *server, const char *prefix,
29 + RRDHOST *host, const char *filter_string, BUFFER *wb, const char *server, const char *prefix,
30 EXPORTING_OPTIONS exporting_options, PROMETHEUS_OUTPUT_OPTIONS output_options);
31
32 -int can_send_rrdset(struct instance *instance, RRDSET *st);
32 +int can_send_rrdset(struct instance *instance, RRDSET *st, SIMPLE_PATTERN *filter);
33 size_t prometheus_name_copy(char *d, const char *s, size_t usable);
34 size_t prometheus_label_copy(char *d, const char *s, size_t usable);
35 char *prometheus_units_copy(char *d, const char *s, size_t usable, int showoldunits);
exporting/tests/test_exporting_engine.c
+7 -7
@@ -988,21 +988,21 @@ static void test_can_send_rrdset(void **state)
988 {
989 (void)*state;
990
991 - assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root), 1);
991 + assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root, NULL), 1);
992
993 rrdset_flag_set(localhost->rrdset_root, RRDSET_FLAG_EXPORTING_IGNORE);
994 - assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root), 0);
994 + assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root, NULL), 0);
995 rrdset_flag_clear(localhost->rrdset_root, RRDSET_FLAG_EXPORTING_IGNORE);
996
997 // TODO: test with a denying simple pattern
998
999 rrdset_flag_set(localhost->rrdset_root, RRDSET_FLAG_OBSOLETE);
1000 - assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root), 0);
1000 + assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root, NULL), 0);
1001 rrdset_flag_clear(localhost->rrdset_root, RRDSET_FLAG_OBSOLETE);
1002
1003 localhost->rrdset_root->rrd_memory_mode = RRD_MEMORY_MODE_NONE;
1004 prometheus_exporter_instance->config.options |= EXPORTING_SOURCE_DATA_AVERAGE;
1005 - assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root), 0);
1005 + assert_int_equal(can_send_rrdset(prometheus_exporter_instance, localhost->rrdset_root, NULL), 0);
1006 }
1007
1008 static void test_prometheus_name_copy(void **state)
@@ -1067,7 +1067,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(void **state)
1067 expect_function_call(__wrap_exporting_calculate_value_from_stored_data);
1068 will_return(__wrap_exporting_calculate_value_from_stored_data, pack_storage_number(27, SN_DEFAULT_FLAGS));
1069
1070 - rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(localhost, buffer, "test_server", "test_prefix", 0, 0);
1070 + rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(localhost, NULL, buffer, "test_server", "test_prefix", 0, 0);
1071
1072 assert_string_equal(
1073 buffer_tostring(buffer),
@@ -1085,7 +1085,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(void **state)
1085 will_return(__wrap_exporting_calculate_value_from_stored_data, pack_storage_number(27, SN_DEFAULT_FLAGS));
1086
1087 rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
1088 - localhost, buffer, "test_server", "test_prefix", 0, PROMETHEUS_OUTPUT_NAMES | PROMETHEUS_OUTPUT_TYPES);
1088 + localhost, NULL, buffer, "test_server", "test_prefix", 0, PROMETHEUS_OUTPUT_NAMES | PROMETHEUS_OUTPUT_TYPES);
1089
1090 assert_string_equal(
1091 buffer_tostring(buffer),
@@ -1103,7 +1103,7 @@ static void rrd_stats_api_v1_charts_allmetrics_prometheus(void **state)
1103 expect_function_call(__wrap_exporting_calculate_value_from_stored_data);
1104 will_return(__wrap_exporting_calculate_value_from_stored_data, pack_storage_number(27, SN_DEFAULT_FLAGS));
1105
1106 - rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(localhost, buffer, "test_server", "test_prefix", 0, 0);
1106 + rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(localhost, NULL, buffer, "test_server", "test_prefix", 0, 0);
1107
1108 assert_string_equal(
1109 buffer_tostring(buffer),
web/api/exporters/allmetrics.c
+8 -2
@@ -19,6 +19,7 @@ struct prometheus_output_options {
19
20 inline int web_client_api_request_v1_allmetrics(RRDHOST *host, struct web_client *w, char *url) {
21 int format = ALLMETRICS_SHELL;
22 + const char *filter = NULL;
23 const char *prometheus_server = w->client_ip;
24
25 uint32_t prometheus_exporting_options;
@@ -57,6 +58,9 @@ inline int web_client_api_request_v1_allmetrics(RRDHOST *host, struct web_client
58 else
59 format = 0;
60 }
61 + else if(!strcmp(name, "filter")) {
62 + filter = value;
63 + }
64 else if(!strcmp(name, "server")) {
65 prometheus_server = value;
66 }
@@ -87,18 +91,19 @@ inline int web_client_api_request_v1_allmetrics(RRDHOST *host, struct web_client
91 switch(format) {
92 case ALLMETRICS_JSON:
93 w->response.data->contenttype = CT_APPLICATION_JSON;
90 - rrd_stats_api_v1_charts_allmetrics_json(host, w->response.data);
94 + rrd_stats_api_v1_charts_allmetrics_json(host, filter, w->response.data);
95 return HTTP_RESP_OK;
96
97 case ALLMETRICS_SHELL:
98 w->response.data->contenttype = CT_TEXT_PLAIN;
95 - rrd_stats_api_v1_charts_allmetrics_shell(host, w->response.data);
99 + rrd_stats_api_v1_charts_allmetrics_shell(host, filter, w->response.data);
100 return HTTP_RESP_OK;
101
102 case ALLMETRICS_PROMETHEUS:
103 w->response.data->contenttype = CT_PROMETHEUS;
104 rrd_stats_api_v1_charts_allmetrics_prometheus_single_host(
105 host
106 + , filter
107 , w->response.data
108 , prometheus_server
109 , prometheus_prefix
@@ -111,6 +116,7 @@ inline int web_client_api_request_v1_allmetrics(RRDHOST *host, struct web_client
116 w->response.data->contenttype = CT_PROMETHEUS;
117 rrd_stats_api_v1_charts_allmetrics_prometheus_all_hosts(
118 host
119 + , filter
120 , w->response.data
121 , prometheus_server
122 , prometheus_prefix
web/api/exporters/shell/allmetrics_shell.c
+12 -2
@@ -22,13 +22,17 @@ static inline size_t shell_name_copy(char *d, const char *s, size_t usable) {
22
23 #define SHELL_ELEMENT_MAX 100
24
25 -void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, BUFFER *wb) {
25 +void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, const char *filter_string, BUFFER *wb) {
26 analytics_log_shell();
27 + SIMPLE_PATTERN *filter = simple_pattern_create(filter_string, NULL, SIMPLE_PATTERN_EXACT);
28 rrdhost_rdlock(host);
29
30 // for each chart
31 RRDSET *st;
32 rrdset_foreach_read(st, host) {
33 + if (filter && !simple_pattern_matches(filter, st->name))
34 + continue;
35 +
36 calculated_number total = 0.0;
37 char chart[SHELL_ELEMENT_MAX + 1];
38 shell_name_copy(chart, st->name?st->name:st->id, SHELL_ELEMENT_MAX);
@@ -88,12 +92,14 @@ void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, BUFFER *wb) {
92 }
93
94 rrdhost_unlock(host);
95 + simple_pattern_free(filter);
96 }
97
98 // ----------------------------------------------------------------------------
99
95 -void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, BUFFER *wb) {
100 +void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, const char *filter_string, BUFFER *wb) {
101 analytics_log_json();
102 + SIMPLE_PATTERN *filter = simple_pattern_create(filter_string, NULL, SIMPLE_PATTERN_EXACT);
103 rrdhost_rdlock(host);
104
105 buffer_strcat(wb, "{");
@@ -104,6 +110,9 @@ void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, BUFFER *wb) {
110 // for each chart
111 RRDSET *st;
112 rrdset_foreach_read(st, host) {
113 + if (filter && !(simple_pattern_matches(filter, st->id) || simple_pattern_matches(filter, st->name)))
114 + continue;
115 +
116 if(rrdset_is_available_for_viewers(st)) {
117 rrdset_rdlock(st);
118
@@ -160,5 +169,6 @@ void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, BUFFER *wb) {
169
170 buffer_strcat(wb, "\n}");
171 rrdhost_unlock(host);
172 + simple_pattern_free(filter);
173 }
174
web/api/exporters/shell/allmetrics_shell.h
+2 -2
@@ -15,7 +15,7 @@
15 #define ALLMETRICS_JSON 3
16 #define ALLMETRICS_PROMETHEUS_ALL_HOSTS 4
17
18 -extern void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, BUFFER *wb);
19 -extern void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, BUFFER *wb);
18 +extern void rrd_stats_api_v1_charts_allmetrics_json(RRDHOST *host, const char *filter_string, BUFFER *wb);
19 +extern void rrd_stats_api_v1_charts_allmetrics_shell(RRDHOST *host, const char *filter_string, BUFFER *wb);
20
21 #endif //NETDATA_API_ALLMETRICS_SHELL_H
web/api/netdata-swagger.json
+10
@@ -711,6 +711,16 @@
711 "default": "shell"
712 }
713 },
714 + {
715 + "name": "filter",
716 + "in": "query",
717 + "description": "Allows to filter charts out using simple patterns.",
718 + "required": false,
719 + "schema": {
720 + "type": "string",
721 + "format": "any text"
722 + }
723 + },
724 {
725 "name": "variables",
726 "in": "query",
web/api/netdata-swagger.yaml
+7
@@ -593,6 +593,13 @@ paths:
593 - prometheus_all_hosts
594 - json
595 default: shell
596 + - name: filter
597 + in: query
598 + description: Allows to filter charts out using simple patterns.
599 + required: false
600 + schema:
601 + type: string
602 + format: any text
603 - name: variables
604 in: query
605 description: When enabled, netdata will expose various system
web/api/tests/valid_urls.c
+1
@@ -16,6 +16,7 @@ void free_temporary_host(RRDHOST *host)
16 void *__wrap_free_temporary_host(RRDHOST *host)
17 {
18 (void) host;
19 + return NULL;
20 }
21
22
web/api/tests/web_api.c
+1
@@ -16,6 +16,7 @@ void free_temporary_host(RRDHOST *host)
16 void *__wrap_free_temporary_host(RRDHOST *host)
17 {
18 (void) host;
19 + return NULL;
20 }
21
22 RRDHOST *sql_create_host_by_uuid(char *hostname)