@cryptotaxi247 / netdata
Code Commits Issues Pulls Sessions CI/CD
master
master
conf 151 lines 5.33 KB
Raw
1 # you can disable an alarm notification by setting the 'to' line to: silent
2
3 # -----------------------------------------------------------------------------
4
5 # True (1) if the node is experiencing a configuration-related error, false (0) otherwise.
6
7 template: kubelet_node_config_error
8 on: k8s_kubelet.kubelet_node_config_error
9 class: Errors
10 type: Kubernetes
11 component: Kubelet
12 calc: $experiencing_error
13 units: bool
14 every: 10s
15 warn: $this == 1
16 delay: down 1m multiplier 1.5 max 2h
17 summary: Kubelet node config error
18 info: The node is experiencing a configuration-related error (0: false, 1: true)
19 to: sysadmin
20
21 # Failed Token() requests to the alternate token source
22
23 template: kubelet_token_requests
24 on: k8s_kubelet.kubelet_token_requests
25 class: Errors
26 type: Kubernetes
27 component: Kubelet
28 lookup: sum -10s of failed
29 units: requests
30 every: 10s
31 warn: $this > 0
32 delay: down 1m multiplier 1.5 max 2h
33 summary: Kubelet failed token requests
34 info: Number of failed Token() requests to the alternate token source
35 to: sysadmin
36
37 # Docker and runtime operation errors
38
39 template: kubelet_operations_error
40 on: k8s_kubelet.kubelet_operations_errors
41 class: Errors
42 type: Kubernetes
43 component: Kubelet
44 lookup: sum -1m
45 units: errors
46 every: 10s
47 warn: $this > (($status >= $WARNING) ? (0) : (20))
48 delay: up 30s down 1m multiplier 1.5 max 2h
49 summary: Kubelet runtime errors
50 info: Number of Docker or runtime operation errors
51 to: sysadmin
52
53 # -----------------------------------------------------------------------------
54
55 # Pod Lifecycle Event Generator Relisting Latency
56
57 # 1. calculate the pleg relisting latency for 1m (quantile 0.5, quantile 0.9, quantile 0.99)
58 # 2. do the same for the last 10s
59 # 3. raise an alarm if the later is:
60 # - 2x the first for quantile 0.5
61 # - 4x the first for quantile 0.9
62 # - 8x the first for quantile 0.99
63 #
64 # we assume the minimum latency is 1000 microseconds
65
66 # quantile 0.5
67
68 template: kubelet_1m_pleg_relist_latency_quantile_05
69 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
70 class: Latency
71 type: Kubernetes
72 component: Kubelet
73 lookup: average -1m unaligned of 0.5
74 units: microseconds
75 every: 10s
76 info: average Pod Lifecycle Event Generator relisting latency over the last minute (quantile 0.5)
77
78 template: kubelet_10s_pleg_relist_latency_quantile_05
79 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
80 class: Latency
81 type: Kubernetes
82 component: Kubelet
83 lookup: average -10s unaligned of 0.5
84 calc: $this * 100 / (($kubelet_1m_pleg_relist_latency_quantile_05 < 1000)?(1000):($kubelet_1m_pleg_relist_latency_quantile_05))
85 every: 10s
86 units: %
87 warn: $this > (($status >= $WARNING)?(100):(200))
88 crit: $this > (($status >= $WARNING)?(200):(400))
89 delay: down 1m multiplier 1.5 max 2h
90 summary: Kubelet relisting latency (quantile 0.5)
91 info: Ratio of average Pod Lifecycle Event Generator relisting latency over the last 10 seconds, \
92 compared to the last minute (quantile 0.5)
93 to: sysadmin
94
95 # quantile 0.9
96
97 template: kubelet_1m_pleg_relist_latency_quantile_09
98 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
99 class: Latency
100 type: Kubernetes
101 component: Kubelet
102 lookup: average -1m unaligned of 0.9
103 units: microseconds
104 every: 10s
105 info: average Pod Lifecycle Event Generator relisting latency over the last minute (quantile 0.9)
106
107 template: kubelet_10s_pleg_relist_latency_quantile_09
108 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
109 class: Latency
110 type: Kubernetes
111 component: Kubelet
112 lookup: average -10s unaligned of 0.9
113 calc: $this * 100 / (($kubelet_1m_pleg_relist_latency_quantile_09 < 1000)?(1000):($kubelet_1m_pleg_relist_latency_quantile_09))
114 every: 10s
115 units: %
116 warn: $this > (($status >= $WARNING)?(200):(400))
117 crit: $this > (($status >= $WARNING)?(400):(800))
118 delay: down 1m multiplier 1.5 max 2h
119 summary: Kubelet relisting latency (quantile 0.9)
120 info: Ratio of average Pod Lifecycle Event Generator relisting latency over the last 10 seconds, \
121 compared to the last minute (quantile 0.9)
122 to: sysadmin
123
124 # quantile 0.99
125
126 template: kubelet_1m_pleg_relist_latency_quantile_099
127 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
128 class: Latency
129 type: Kubernetes
130 component: Kubelet
131 lookup: average -1m unaligned of 0.99
132 units: microseconds
133 every: 10s
134 info: average Pod Lifecycle Event Generator relisting latency over the last minute (quantile 0.99)
135
136 template: kubelet_10s_pleg_relist_latency_quantile_099
137 on: k8s_kubelet.kubelet_pleg_relist_latency_microseconds
138 class: Latency
139 type: Kubernetes
140 component: Kubelet
141 lookup: average -10s unaligned of 0.99
142 calc: $this * 100 / (($kubelet_1m_pleg_relist_latency_quantile_099 < 1000)?(1000):($kubelet_1m_pleg_relist_latency_quantile_099))
143 every: 10s
144 units: %
145 warn: $this > (($status >= $WARNING)?(400):(800))
146 crit: $this > (($status >= $WARNING)?(800):(1200))
147 delay: down 1m multiplier 1.5 max 2h
148 summary: Kubelet relisting latency (quantile 0.99)
149 info: Ratio of average Pod Lifecycle Event Generator relisting latency over the last 10 seconds, \
150 compared to the last minute (quantile 0.99)
151 to: sysadmin