remove python.d/zscores (#18897)
Ilya Mashchenko committed
Oct 30, 2024 at 15:02 UTC
a5e5e2bf0b69f209f0820b67c2468f04f8e26f32
7 files changed
-674
CMakeLists.txt
-2
@@ -3096,7 +3096,6 @@ if(ENABLE_PLUGIN_PYTHON)
3096
src/collectors/python.d.plugin/haproxy/haproxy.conf
3097
src/collectors/python.d.plugin/pandas/pandas.conf
3098
src/collectors/python.d.plugin/traefik/traefik.conf
3099
- src/collectors/python.d.plugin/zscores/zscores.conf
3099
COMPONENT plugin-pythond
3100
DESTINATION usr/lib/netdata/conf.d/python.d)
3101
@@ -3106,7 +3105,6 @@ if(ENABLE_PLUGIN_PYTHON)
3105
src/collectors/python.d.plugin/haproxy/haproxy.chart.py
3106
src/collectors/python.d.plugin/pandas/pandas.chart.py
3107
src/collectors/python.d.plugin/traefik/traefik.chart.py
3109
- src/collectors/python.d.plugin/zscores/zscores.chart.py
3108
COMPONENT plugin-pythond
3109
DESTINATION usr/libexec/netdata/python.d)
3110
src/collectors/python.d.plugin/python.d.conf
-1
@@ -34,7 +34,6 @@ go_expvar: no
34
# smartd_log: yes
35
# traefik: yes
36
# varnish: yes
37
-# zscores: no
37
38
39
## Disabled for existing installations.
src/collectors/python.d.plugin/zscores/README.md
deleted
-1
@@ -1 +0,0 @@
1
-integrations/python.d_zscores.md
\ No newline at end of file
src/collectors/python.d.plugin/zscores/integrations/python.d_zscores.md
deleted
-229
@@ -1,229 +0,0 @@
1
-<!--startmeta
2
-custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/collectors/python.d.plugin/zscores/README.md"
3
-meta_yaml: "https://github.com/netdata/netdata/edit/master/src/collectors/python.d.plugin/zscores/metadata.yaml"
4
-sidebar_label: "python.d zscores"
5
-learn_status: "Published"
6
-learn_rel_path: "Collecting Metrics/Other"
7
-most_popular: False
8
-message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE"
9
-endmeta-->
10
-
11
-# python.d zscores
12
-
13
-Plugin: python.d.plugin
14
-Module: zscores
15
-
16
-<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />
17
-
18
-## Overview
19
-
20
-By using smoothed, rolling [Z-Scores](https://en.wikipedia.org/wiki/Standard_score) for selected metrics or charts you can narrow down your focus and shorten root cause analysis.
21
-
22
-
23
-This collector uses the [Netdata rest api](https://github.com/netdata/netdata/blob/master/src/web/api/README.md) to get the `mean` and `stddev`
24
-for each dimension on specified charts over a time range (defined by `train_secs` and `offset_secs`).
25
-
26
-For each dimension it will calculate a Z-Score as `z = (x - mean) / stddev` (clipped at `z_clip`). Scores are then smoothed over
27
-time (`z_smooth_n`) and, if `mode: 'per_chart'`, aggregated across dimensions to a smoothed, rolling chart level Z-Score at each time step.
28
-
29
-
30
-This collector is supported on all platforms.
31
-
32
-This collector supports collecting metrics from multiple instances of this integration, including remote instances.
33
-
34
-
35
-### Default Behavior
36
-
37
-#### Auto-Detection
38
-
39
-This integration doesn't support auto-detection.
40
-
41
-#### Limits
42
-
43
-The default configuration for this integration does not impose any limits on data collection.
44
-
45
-#### Performance Impact
46
-
47
-The default configuration for this integration is not expected to impose a significant performance impact on the system.
48
-
49
-
50
-## Metrics
51
-
52
-Metrics grouped by *scope*.
53
-
54
-The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.
55
-
56
-
57
-
58
-### Per python.d zscores instance
59
-
60
-These metrics refer to the entire monitored application.
61
-
62
-This scope has no labels.
63
-
64
-Metrics:
65
-
66
-| Metric | Dimensions | Unit |
67
-|:------|:----------|:----|
68
-| zscores.z | a dimension per chart or dimension | z |
69
-| zscores.3stddev | a dimension per chart or dimension | count |
70
-
71
-
72
-
73
-## Alerts
74
-
75
-There are no alerts configured by default for this integration.
76
-
77
-
78
-## Setup
79
-
80
-### Prerequisites
81
-
82
-#### Python Requirements
83
-
84
-This collector will only work with Python 3 and requires the below packages be installed.
85
-
86
-```bash
87
-# become netdata user
88
-sudo su -s /bin/bash netdata
89
-# install required packages
90
-pip3 install numpy pandas requests netdata-pandas==0.0.38
91
-```
92
-
93
-
94
-
95
-### Configuration
96
-
97
-#### File
98
-
99
-The configuration file name for this integration is `python.d/zscores.conf`.
100
-
101
-
102
-You can edit the configuration file using the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#edit-a-configuration-file-using-edit-config) script from the
103
-Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).
104
-
105
-```bash
106
-cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
107
-sudo ./edit-config python.d/zscores.conf
108
-```
109
-#### Options
110
-
111
-There are 2 sections:
112
-
113
-* Global variables
114
-* One or more JOBS that can define multiple different instances to monitor.
115
-
116
-The following options can be defined globally: priority, penalty, autodetection_retry, update_every, but can also be defined per JOB to override the global values.
117
-
118
-Additionally, the following collapsed table contains all the options that can be configured inside a JOB definition.
119
-
120
-Every configuration JOB starts with a `job_name` value which will appear in the dashboard, unless a `name` parameter is specified.
121
-
122
-
123
-<details open><summary>Config options</summary>
124
-
125
-| Name | Description | Default | Required |
126
-|:----|:-----------|:-------|:--------:|
127
-| charts_regex | what charts to pull data for - A regex like `system\..*/` or `system\..*/apps.cpu/apps.mem` etc. | system\..* | yes |
128
-| train_secs | length of time (in seconds) to base calculations off for mean and stddev. | 14400 | yes |
129
-| offset_secs | offset (in seconds) preceding latest data to ignore when calculating mean and stddev. | 300 | yes |
130
-| train_every_n | recalculate the mean and stddev every n steps of the collector. | 900 | yes |
131
-| z_smooth_n | smooth the z score (to reduce sensitivity to spikes) by averaging it over last n values. | 15 | yes |
132
-| z_clip | cap absolute value of zscore (before smoothing) for better stability. | 10 | yes |
133
-| z_abs | set z_abs: 'true' to make all zscores be absolute values only. | true | yes |
134
-| burn_in | burn in period in which to initially calculate mean and stddev on every step. | 2 | yes |
135
-| mode | mode can be to get a zscore 'per_dim' or 'per_chart'. | per_chart | yes |
136
-| per_chart_agg | per_chart_agg is how you aggregate from dimension to chart when mode='per_chart'. | mean | yes |
137
-| update_every | Sets the default data collection frequency. | 5 | no |
138
-| priority | Controls the order of charts at the netdata dashboard. | 60000 | no |
139
-| autodetection_retry | Sets the job re-check interval in seconds. | 0 | no |
140
-| penalty | Indicates whether to apply penalty to update_every in case of failures. | yes | no |
141
-
142
-</details>
143
-
144
-#### Examples
145
-
146
-##### Default
147
-
148
-Default configuration.
149
-
150
-```yaml
151
-local:
152
- name: 'local'
153
- host: '127.0.0.1:19999'
154
- charts_regex: 'system\..*'
155
- charts_to_exclude: 'system.uptime'
156
- train_secs: 14400
157
- offset_secs: 300
158
- train_every_n: 900
159
- z_smooth_n: 15
160
- z_clip: 10
161
- z_abs: 'true'
162
- burn_in: 2
163
- mode: 'per_chart'
164
- per_chart_agg: 'mean'
165
-
166
-```
167
-
168
-
169
-## Troubleshooting
170
-
171
-### Debug Mode
172
-
173
-
174
-To troubleshoot issues with the `zscores` collector, run the `python.d.plugin` with the debug option enabled. The output
175
-should give you clues as to why the collector isn't working.
176
-
177
-- Navigate to the `plugins.d` directory, usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case on
178
- your system, open `netdata.conf` and look for the `plugins` setting under `[directories]`.
179
-
180
- ```bash
181
- cd /usr/libexec/netdata/plugins.d/
182
- ```
183
-
184
-- Switch to the `netdata` user.
185
-
186
- ```bash
187
- sudo -u netdata -s
188
- ```
189
-
190
-- Run the `python.d.plugin` to debug the collector:
191
-
192
- ```bash
193
- ./python.d.plugin zscores debug trace
194
- ```
195
-
196
-### Getting Logs
197
-
198
-If you're encountering problems with the `zscores` collector, follow these steps to retrieve logs and identify potential issues:
199
-
200
-- **Run the command** specific to your system (systemd, non-systemd, or Docker container).
201
-- **Examine the output** for any warnings or error messages that might indicate issues. These messages should provide clues about the root cause of the problem.
202
-
203
-#### System with systemd
204
-
205
-Use the following command to view logs generated since the last Netdata service restart:
206
-
207
-```bash
208
-journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep zscores
209
-```
210
-
211
-#### System without systemd
212
-
213
-Locate the collector log file, typically at `/var/log/netdata/collector.log`, and use `grep` to filter for collector's name:
214
-
215
-```bash
216
-grep zscores /var/log/netdata/collector.log
217
-```
218
-
219
-**Note**: This method shows logs from all restarts. Focus on the **latest entries** for troubleshooting current issues.
220
-
221
-#### Docker Container
222
-
223
-If your Netdata runs in a Docker container named "netdata" (replace if different), use this command:
224
-
225
-```bash
226
-docker logs netdata 2>&1 | grep zscores
227
-```
228
-
229
-
src/collectors/python.d.plugin/zscores/metadata.yaml
deleted
-187
@@ -1,187 +0,0 @@
1
-plugin_name: python.d.plugin
2
-modules:
3
- - meta:
4
- plugin_name: python.d.plugin
5
- module_name: zscores
6
- monitored_instance:
7
- name: python.d zscores
8
- link: https://en.wikipedia.org/wiki/Standard_score
9
- categories:
10
- - data-collection.other
11
- icon_filename: ""
12
- related_resources:
13
- integrations:
14
- list: []
15
- info_provided_to_referring_integrations:
16
- description: ""
17
- keywords:
18
- - zscore
19
- - z-score
20
- - standard score
21
- - standard deviation
22
- - anomaly detection
23
- - statistical anomaly detection
24
- most_popular: false
25
- overview:
26
- data_collection:
27
- metrics_description: |
28
- By using smoothed, rolling [Z-Scores](https://en.wikipedia.org/wiki/Standard_score) for selected metrics or charts you can narrow down your focus and shorten root cause analysis.
29
- method_description: |
30
- This collector uses the [Netdata rest api](/src/web/api/README.md) to get the `mean` and `stddev`
31
- for each dimension on specified charts over a time range (defined by `train_secs` and `offset_secs`).
32
-
33
- For each dimension it will calculate a Z-Score as `z = (x - mean) / stddev` (clipped at `z_clip`). Scores are then smoothed over
34
- time (`z_smooth_n`) and, if `mode: 'per_chart'`, aggregated across dimensions to a smoothed, rolling chart level Z-Score at each time step.
35
- supported_platforms:
36
- include: []
37
- exclude: []
38
- multi_instance: true
39
- additional_permissions:
40
- description: ""
41
- default_behavior:
42
- auto_detection:
43
- description: ""
44
- limits:
45
- description: ""
46
- performance_impact:
47
- description: ""
48
- setup:
49
- prerequisites:
50
- list:
51
- - title: Python Requirements
52
- description: |
53
- This collector will only work with Python 3 and requires the below packages be installed.
54
-
55
- ```bash
56
- # become netdata user
57
- sudo su -s /bin/bash netdata
58
- # install required packages
59
- pip3 install numpy pandas requests netdata-pandas==0.0.38
60
- ```
61
- configuration:
62
- file:
63
- name: python.d/zscores.conf
64
- description: ""
65
- options:
66
- description: |
67
- There are 2 sections:
68
-
69
- * Global variables
70
- * One or more JOBS that can define multiple different instances to monitor.
71
-
72
- The following options can be defined globally: priority, penalty, autodetection_retry, update_every, but can also be defined per JOB to override the global values.
73
-
74
- Additionally, the following collapsed table contains all the options that can be configured inside a JOB definition.
75
-
76
- Every configuration JOB starts with a `job_name` value which will appear in the dashboard, unless a `name` parameter is specified.
77
- folding:
78
- title: "Config options"
79
- enabled: true
80
- list:
81
- - name: charts_regex
82
- description: what charts to pull data for - A regex like `system\..*|` or `system\..*|apps.cpu|apps.mem` etc.
83
- default_value: "system\\..*"
84
- required: true
85
- - name: train_secs
86
- description: length of time (in seconds) to base calculations off for mean and stddev.
87
- default_value: 14400
88
- required: true
89
- - name: offset_secs
90
- description: offset (in seconds) preceding latest data to ignore when calculating mean and stddev.
91
- default_value: 300
92
- required: true
93
- - name: train_every_n
94
- description: recalculate the mean and stddev every n steps of the collector.
95
- default_value: 900
96
- required: true
97
- - name: z_smooth_n
98
- description: smooth the z score (to reduce sensitivity to spikes) by averaging it over last n values.
99
- default_value: 15
100
- required: true
101
- - name: z_clip
102
- description: cap absolute value of zscore (before smoothing) for better stability.
103
- default_value: 10
104
- required: true
105
- - name: z_abs
106
- description: "set z_abs: 'true' to make all zscores be absolute values only."
107
- default_value: "true"
108
- required: true
109
- - name: burn_in
110
- description: burn in period in which to initially calculate mean and stddev on every step.
111
- default_value: 2
112
- required: true
113
- - name: mode
114
- description: mode can be to get a zscore 'per_dim' or 'per_chart'.
115
- default_value: per_chart
116
- required: true
117
- - name: per_chart_agg
118
- description: per_chart_agg is how you aggregate from dimension to chart when mode='per_chart'.
119
- default_value: mean
120
- required: true
121
- - name: update_every
122
- description: Sets the default data collection frequency.
123
- default_value: 5
124
- required: false
125
- - name: priority
126
- description: Controls the order of charts at the netdata dashboard.
127
- default_value: 60000
128
- required: false
129
- - name: autodetection_retry
130
- description: Sets the job re-check interval in seconds.
131
- default_value: 0
132
- required: false
133
- - name: penalty
134
- description: Indicates whether to apply penalty to update_every in case of failures.
135
- default_value: yes
136
- required: false
137
- examples:
138
- folding:
139
- enabled: true
140
- title: "Config"
141
- list:
142
- - name: Default
143
- description: Default configuration.
144
- folding:
145
- enabled: false
146
- config: |
147
- local:
148
- name: 'local'
149
- host: '127.0.0.1:19999'
150
- charts_regex: 'system\..*'
151
- charts_to_exclude: 'system.uptime'
152
- train_secs: 14400
153
- offset_secs: 300
154
- train_every_n: 900
155
- z_smooth_n: 15
156
- z_clip: 10
157
- z_abs: 'true'
158
- burn_in: 2
159
- mode: 'per_chart'
160
- per_chart_agg: 'mean'
161
- troubleshooting:
162
- problems:
163
- list: []
164
- alerts: []
165
- metrics:
166
- folding:
167
- title: Metrics
168
- enabled: false
169
- description: ""
170
- availability: []
171
- scopes:
172
- - name: global
173
- description: "These metrics refer to the entire monitored application."
174
- labels: []
175
- metrics:
176
- - name: zscores.z
177
- description: Z Score
178
- unit: "z"
179
- chart_type: line
180
- dimensions:
181
- - name: a dimension per chart or dimension
182
- - name: zscores.3stddev
183
- description: Z Score >3
184
- unit: "count"
185
- chart_type: stacked
186
- dimensions:
187
- - name: a dimension per chart or dimension
src/collectors/python.d.plugin/zscores/zscores.chart.py
deleted
-146
@@ -1,146 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: zscores netdata python.d module
3
-# Author: andrewm4894
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-from datetime import datetime
7
-import re
8
-
9
-import requests
10
-import numpy as np
11
-import pandas as pd
12
-
13
-from bases.FrameworkServices.SimpleService import SimpleService
14
-from netdata_pandas.data import get_data, get_allmetrics
15
-
16
-priority = 60000
17
-update_every = 5
18
-disabled_by_default = True
19
-
20
-ORDER = [
21
- 'z',
22
- '3stddev'
23
-]
24
-
25
-CHARTS = {
26
- 'z': {
27
- 'options': ['z', 'Z Score', 'z', 'Z Score', 'zscores.z', 'line'],
28
- 'lines': []
29
- },
30
- '3stddev': {
31
- 'options': ['3stddev', 'Z Score >3', 'count', '3 Stddev', 'zscores.3stddev', 'stacked'],
32
- 'lines': []
33
- },
34
-}
35
-
36
-
37
-class Service(SimpleService):
38
- def __init__(self, configuration=None, name=None):
39
- SimpleService.__init__(self, configuration=configuration, name=name)
40
- self.host = self.configuration.get('host', '127.0.0.1:19999')
41
- self.charts_regex = re.compile(self.configuration.get('charts_regex', 'system.*'))
42
- self.charts_to_exclude = self.configuration.get('charts_to_exclude', '').split(',')
43
- self.charts_in_scope = [
44
- c for c in
45
- list(filter(self.charts_regex.match,
46
- requests.get(f'http://{self.host}/api/v1/charts').json()['charts'].keys()))
47
- if c not in self.charts_to_exclude
48
- ]
49
- self.train_secs = self.configuration.get('train_secs', 14400)
50
- self.offset_secs = self.configuration.get('offset_secs', 300)
51
- self.train_every_n = self.configuration.get('train_every_n', 900)
52
- self.z_smooth_n = self.configuration.get('z_smooth_n', 15)
53
- self.z_clip = self.configuration.get('z_clip', 10)
54
- self.z_abs = bool(self.configuration.get('z_abs', True))
55
- self.burn_in = self.configuration.get('burn_in', 2)
56
- self.mode = self.configuration.get('mode', 'per_chart')
57
- self.per_chart_agg = self.configuration.get('per_chart_agg', 'mean')
58
- self.order = ORDER
59
- self.definitions = CHARTS
60
- self.collected_dims = {'z': set(), '3stddev': set()}
61
- self.df_mean = pd.DataFrame()
62
- self.df_std = pd.DataFrame()
63
- self.df_z_history = pd.DataFrame()
64
-
65
- def check(self):
66
- _ = get_allmetrics(self.host, self.charts_in_scope, wide=True, col_sep='.')
67
- return True
68
-
69
- def validate_charts(self, chart, data, algorithm='absolute', multiplier=1, divisor=1):
70
- """If dimension not in chart then add it.
71
- """
72
- for dim in data:
73
- if dim not in self.collected_dims[chart]:
74
- self.collected_dims[chart].add(dim)
75
- self.charts[chart].add_dimension([dim, dim, algorithm, multiplier, divisor])
76
-
77
- for dim in list(self.collected_dims[chart]):
78
- if dim not in data:
79
- self.collected_dims[chart].remove(dim)
80
- self.charts[chart].del_dimension(dim, hide=False)
81
-
82
- def train_model(self):
83
- """Calculate the mean and stddev for all relevant metrics and store them for use in calulcating zscore at each timestep.
84
- """
85
- before = int(datetime.now().timestamp()) - self.offset_secs
86
- after = before - self.train_secs
87
-
88
- self.df_mean = get_data(
89
- self.host, self.charts_in_scope, after, before, points=10, group='average', col_sep='.'
90
- ).mean().to_frame().rename(columns={0: "mean"})
91
-
92
- self.df_std = get_data(
93
- self.host, self.charts_in_scope, after, before, points=10, group='stddev', col_sep='.'
94
- ).mean().to_frame().rename(columns={0: "std"})
95
-
96
- def create_data(self, df_allmetrics):
97
- """Use x, mean, stddev to generate z scores and 3stddev flags via some pandas manipulation.
98
- Returning two dictionaries of dimensions and measures, one for each chart.
99
-
100
- :param df_allmetrics <pd.DataFrame>: pandas dataframe with latest data from api/v1/allmetrics.
101
- :return: (<dict>,<dict>) tuple of dictionaries, one for zscores and the other for a flag if abs(z)>3.
102
- """
103
- # calculate clipped z score for each available metric
104
- df_z = pd.concat([self.df_mean, self.df_std, df_allmetrics], axis=1, join='inner')
105
- df_z['z'] = ((df_z['value'] - df_z['mean']) / df_z['std']).clip(-self.z_clip, self.z_clip).fillna(0) * 100
106
- if self.z_abs:
107
- df_z['z'] = df_z['z'].abs()
108
-
109
- # append last z_smooth_n rows of zscores to history table in wide format
110
- self.df_z_history = self.df_z_history.append(
111
- df_z[['z']].reset_index().pivot_table(values='z', columns='index'), sort=True
112
- ).tail(self.z_smooth_n)
113
-
114
- # get average zscore for last z_smooth_n for each metric
115
- df_z_smooth = self.df_z_history.melt(value_name='z').groupby('index')['z'].mean().to_frame()
116
- df_z_smooth['3stddev'] = np.where(abs(df_z_smooth['z']) > 300, 1, 0)
117
- data_z = df_z_smooth['z'].add_suffix('_z').to_dict()
118
-
119
- # aggregate to chart level if specified
120
- if self.mode == 'per_chart':
121
- df_z_smooth['chart'] = ['.'.join(x[0:2]) + '_z' for x in df_z_smooth.index.str.split('.').to_list()]
122
- if self.per_chart_agg == 'absmax':
123
- data_z = \
124
- list(df_z_smooth.groupby('chart').agg({'z': lambda x: max(x, key=abs)})['z'].to_dict().values())[0]
125
- else:
126
- data_z = list(df_z_smooth.groupby('chart').agg({'z': [self.per_chart_agg]})['z'].to_dict().values())[0]
127
-
128
- data_3stddev = {}
129
- for k in data_z:
130
- data_3stddev[k.replace('_z', '')] = 1 if abs(data_z[k]) > 300 else 0
131
-
132
- return data_z, data_3stddev
133
-
134
- def get_data(self):
135
-
136
- if self.runs_counter <= self.burn_in or self.runs_counter % self.train_every_n == 0:
137
- self.train_model()
138
-
139
- data_z, data_3stddev = self.create_data(
140
- get_allmetrics(self.host, self.charts_in_scope, wide=True, col_sep='.').transpose())
141
- data = {**data_z, **data_3stddev}
142
-
143
- self.validate_charts('z', data_z, divisor=100)
144
- self.validate_charts('3stddev', data_3stddev)
145
-
146
- return data
src/collectors/python.d.plugin/zscores/zscores.conf
deleted
-108
@@ -1,108 +0,0 @@
1
-# netdata python.d.plugin configuration for example
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-update_every: 5
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, example also supports the following:
63
-#
64
-# - none
65
-#
66
-# ----------------------------------------------------------------------
67
-# AUTO-DETECTION JOBS
68
-# only one of them will run (they have the same name)
69
-
70
-local:
71
- name: 'local'
72
-
73
- # what host to pull data from
74
- host: '127.0.0.1:19999'
75
-
76
- # what charts to pull data for - A regex like 'system\..*|' or 'system\..*|apps.cpu|apps.mem' etc.
77
- charts_regex: 'system\..*'
78
-
79
- # Charts to exclude, useful if you would like to exclude some specific charts.
80
- # Note: should be a ',' separated string like 'chart.name,chart.name'.
81
- charts_to_exclude: 'system.uptime'
82
-
83
- # length of time to base calculations off for mean and stddev
84
- train_secs: 14400 # use last 4 hours to work out the mean and stddev for the zscore
85
-
86
- # offset preceding latest data to ignore when calculating mean and stddev
87
- offset_secs: 300 # ignore last 5 minutes of data when calculating the mean and stddev
88
-
89
- # recalculate the mean and stddev every n steps of the collector
90
- train_every_n: 900 # recalculate mean and stddev every 15 minutes
91
-
92
- # smooth the z score by averaging it over last n values
93
- z_smooth_n: 15 # take a rolling average of the last 15 zscore values to reduce sensitivity to temporary 'spikes'
94
-
95
- # cap absolute value of zscore (before smoothing) for better stability
96
- z_clip: 10 # cap each zscore at 10 so as to avoid really large individual zscores swamping any rolling average
97
-
98
- # set z_abs: 'true' to make all zscores be absolute values only.
99
- z_abs: 'true'
100
-
101
- # burn in period in which to initially calculate mean and stddev on every step
102
- burn_in: 2 # on startup of the collector continually update the mean and stddev in case any gaps or initial calculations fail to return
103
-
104
- # mode can be to get a zscore 'per_dim' or 'per_chart'
105
- mode: 'per_chart' # 'per_chart' means individual dimension level smoothed zscores will be aggregated to one zscore per chart per time step
106
-
107
- # per_chart_agg is how you aggregate from dimension to chart when mode='per_chart'
108
- per_chart_agg: 'mean' # 'absmax' will take the max absolute value across all dimensions but will maintain the sign. 'mean' will just average.