chore(python.d): remove python.d/* announced in v1.34.0 deprecation notice (#12796)
Ilya Mashchenko committed
May 2, 2022 at 18:15 UTC
90c2fcb838dc711e6706713510b0d07916d94829
70 files changed
+351
-9307
collectors/COLLECTORS.md
+341
-368
@@ -66,278 +66,253 @@ configure any of these collectors according to your setup and infrastructure.
66
67
### Generic
68
69
-- [Prometheus endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus): Gathers
70
- metrics from any number of Prometheus endpoints, with support to autodetect more than 600 services and applications.
69
+- [Prometheus endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus): Gathers
70
+ metrics from any number of Prometheus endpoints, with support to autodetect more than 600 services and applications.
71
72
### APM (application performance monitoring)
73
74
-- [Go applications](/collectors/python.d.plugin/go_expvar/README.md): Monitor any Go application that exposes its
75
- metrics with the `expvar` package from the Go standard library.
76
-- [Java Spring Boot 2
77
- applications](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/springboot2/) (Go version):
78
- Monitor running Java Spring Boot 2 applications that expose their metrics with the use of the Spring Boot Actuator.
79
-- [Java Spring Boot 2 applications](/collectors/python.d.plugin/springboot/README.md) (Python version): Monitor
80
- running Java Spring Boot applications that expose their metrics with the use of the Spring Boot Actuator.
81
-- [statsd](/collectors/statsd.plugin/README.md): Implement a high performance `statsd` server for Netdata.
82
-- [phpDaemon](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpdaemon/): Collect worker
83
- statistics (total, active, idle), and uptime for web and network applications.
84
-- [uWSGI](/collectors/python.d.plugin/uwsgi/README.md): Monitor performance metrics exposed by the uWSGI Stats
85
- Server.
74
+- [Go applications](/collectors/python.d.plugin/go_expvar/README.md): Monitor any Go application that exposes its
75
+ metrics with the `expvar` package from the Go standard library.
76
+- [Java Spring Boot 2
77
+ applications](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/springboot2/) (Go version):
78
+ Monitor running Java Spring Boot 2 applications that expose their metrics with the use of the Spring Boot Actuator.
79
+- [Java Spring Boot 2 applications](/collectors/python.d.plugin/springboot/README.md) (Python version): Monitor
80
+ running Java Spring Boot applications that expose their metrics with the use of the Spring Boot Actuator.
81
+- [statsd](/collectors/statsd.plugin/README.md): Implement a high performance `statsd` server for Netdata.
82
+- [phpDaemon](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpdaemon/): Collect worker
83
+ statistics (total, active, idle), and uptime for web and network applications.
84
+- [uWSGI](/collectors/python.d.plugin/uwsgi/README.md): Monitor performance metrics exposed by the uWSGI Stats
85
+ Server.
86
87
### Containers and VMs
88
89
-- [Docker containers](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Docker
90
- containers using the cgroups collector plugin.
91
-- [DockerD](/collectors/python.d.plugin/dockerd/README.md): Collect container health statistics.
92
-- [Docker Engine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/docker_engine/): Collect
93
- runtime statistics from the `docker` daemon using the `metrics-address` feature.
94
-- [Docker Hub](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dockerhub/): Collect statistics
95
- about Docker repositories, such as pulls, starts, status, time since last update, and more.
96
-- [Libvirt](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Libvirt containers
97
- using the cgroups collector plugin.
98
-- [LXC](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXC containers using
99
- the cgroups collector plugin.
100
-- [LXD](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXD containers using
101
- the cgroups collector plugin.
102
-- [systemd-nspawn](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual
103
- systemd-nspawn containers using the cgroups collector plugin.
104
-- [vCenter Server Appliance](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vcsa/): Monitor
105
- appliance system, components, and software update health statuses via the Health API.
106
-- [vSphere](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vsphere/): Collect host and virtual
107
- machine performance metrics.
108
-- [Xen/XCP-ng](/collectors/xenstat.plugin/README.md): Collect XenServer and XCP-ng metrics using `libxenstat`.
89
+- [Docker containers](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Docker
90
+ containers using the cgroups collector plugin.
91
+- [DockerD](/collectors/python.d.plugin/dockerd/README.md): Collect container health statistics.
92
+- [Docker Engine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/docker_engine/): Collect
93
+ runtime statistics from the `docker` daemon using the `metrics-address` feature.
94
+- [Docker Hub](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dockerhub/): Collect statistics
95
+ about Docker repositories, such as pulls, starts, status, time since last update, and more.
96
+- [Libvirt](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual Libvirt containers
97
+ using the cgroups collector plugin.
98
+- [LXC](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXC containers using
99
+ the cgroups collector plugin.
100
+- [LXD](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual LXD containers using
101
+ the cgroups collector plugin.
102
+- [systemd-nspawn](/collectors/cgroups.plugin/README.md): Monitor the health and performance of individual
103
+ systemd-nspawn containers using the cgroups collector plugin.
104
+- [vCenter Server Appliance](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vcsa/): Monitor
105
+ appliance system, components, and software update health statuses via the Health API.
106
+- [vSphere](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vsphere/): Collect host and virtual
107
+ machine performance metrics.
108
+- [Xen/XCP-ng](/collectors/xenstat.plugin/README.md): Collect XenServer and XCP-ng metrics using `libxenstat`.
109
110
### Data stores
111
112
-- [CockroachDB](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/cockroachdb/): Monitor various
113
- database components using `_status/vars` endpoint.
114
-- [Consul](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/consul/): Capture service and unbound
115
- checks status (passing, warning, critical, maintenance).
116
-- [Couchbase](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/couchbase/): Gather per-bucket
117
- metrics from any number of instances of the distributed JSON document database.
118
-- [CouchDB](/collectors/python.d.plugin/couchdb/README.md): Monitor database health and performance metrics
119
- (reads/writes, HTTP traffic, replication status, etc).
120
-- [MongoDB](/collectors/python.d.plugin/mongodb/README.md): Collect memory-caching system performance metrics and
121
- reads the server's response to `stats` command (stats interface).
122
-- [MySQL](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/mysql/): Collect database global,
123
- replication and per user statistics.
124
-- [OracleDB](/collectors/python.d.plugin/oracledb/README.md): Monitor database performance and health metrics.
125
-- [Pika](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pika/): Gather metric, such as clients,
126
- memory usage, queries, and more from the Redis interface-compatible database.
127
-- [Postgres](/collectors/python.d.plugin/postgres/README.md): Collect database health and performance metrics.
128
-- [ProxySQL](/collectors/python.d.plugin/proxysql/README.md): Monitor database backend and frontend performance
129
- metrics.
130
-- [Redis (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/redis/): Monitor status from any
131
- number of database instances by reading the server's response to the `INFO ALL` command.
132
-- [Redis (Python)](/collectors/python.d.plugin/redis/README.md): Monitor database status by reading the server's response to
133
- the `INFO` command.
134
-- [RethinkDB](/collectors/python.d.plugin/rethinkdbs/README.md): Collect database server and cluster statistics.
135
-- [Riak KV](/collectors/python.d.plugin/riakkv/README.md): Collect database stats from the `/stats` endpoint.
136
-- [Zookeeper](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/zookeeper/): Monitor application
137
- health metrics reading the server's response to the `mntr` command.
138
-- [Memcached](/collectors/python.d.plugin/memcached/README.md): Collect memory-caching system performance metrics.
112
+- [CockroachDB](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/cockroachdb/): Monitor various
113
+ database components using `_status/vars` endpoint.
114
+- [Consul](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/consul/): Capture service and unbound
115
+ checks status (passing, warning, critical, maintenance).
116
+- [Couchbase](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/couchbase/): Gather per-bucket
117
+ metrics from any number of instances of the distributed JSON document database.
118
+- [CouchDB](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/couchdb): Monitor database health and
119
+ performance metrics
120
+ (reads/writes, HTTP traffic, replication status, etc).
121
+- [MongoDB](/collectors/python.d.plugin/mongodb/README.md): Collect memory-caching system performance metrics and
122
+ reads the server's response to `stats` command (stats interface).
123
+- [MySQL](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/mysql/): Collect database global,
124
+ replication and per user statistics.
125
+- [OracleDB](/collectors/python.d.plugin/oracledb/README.md): Monitor database performance and health metrics.
126
+- [Pika](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pika/): Gather metric, such as clients,
127
+ memory usage, queries, and more from the Redis interface-compatible database.
128
+- [Postgres](/collectors/python.d.plugin/postgres/README.md): Collect database health and performance metrics.
129
+- [ProxySQL](/collectors/python.d.plugin/proxysql/README.md): Monitor database backend and frontend performance
130
+ metrics.
131
+- [Redis](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/redis/): Monitor status from any
132
+ number of database instances by reading the server's response to the `INFO ALL` command.
133
+- [RethinkDB](/collectors/python.d.plugin/rethinkdbs/README.md): Collect database server and cluster statistics.
134
+- [Riak KV](/collectors/python.d.plugin/riakkv/README.md): Collect database stats from the `/stats` endpoint.
135
+- [Zookeeper](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/zookeeper/): Monitor application
136
+ health metrics reading the server's response to the `mntr` command.
137
+- [Memcached](/collectors/python.d.plugin/memcached/README.md): Collect memory-caching system performance metrics.
138
139
### Distributed computing
140
142
-- [BOINC](/collectors/python.d.plugin/boinc/README.md): Monitor the total number of tasks, open tasks, and task
143
- states for the distributed computing client.
144
-- [Gearman](/collectors/python.d.plugin/gearman/README.md): Collect application summary (queued, running) and per-job
145
- worker statistics (queued, idle, running).
141
+- [BOINC](/collectors/python.d.plugin/boinc/README.md): Monitor the total number of tasks, open tasks, and task
142
+ states for the distributed computing client.
143
+- [Gearman](/collectors/python.d.plugin/gearman/README.md): Collect application summary (queued, running) and per-job
144
+ worker statistics (queued, idle, running).
145
146
### Email
147
149
-- [Dovecot](/collectors/python.d.plugin/dovecot/README.md): Collect email server performance metrics by reading the
150
- server's response to the `EXPORT global` command.
151
-- [EXIM](/collectors/python.d.plugin/exim/README.md): Uses the `exim` tool to monitor the queue length of a
152
- mail/message transfer agent (MTA).
153
-- [Postfix](/collectors/python.d.plugin/postfix/README.md): Uses the `postqueue` tool to monitor the queue length of a
154
- mail/message transfer agent (MTA).
148
+- [Dovecot](/collectors/python.d.plugin/dovecot/README.md): Collect email server performance metrics by reading the
149
+ server's response to the `EXPORT global` command.
150
+- [EXIM](/collectors/python.d.plugin/exim/README.md): Uses the `exim` tool to monitor the queue length of a
151
+ mail/message transfer agent (MTA).
152
+- [Postfix](/collectors/python.d.plugin/postfix/README.md): Uses the `postqueue` tool to monitor the queue length of a
153
+ mail/message transfer agent (MTA).
154
155
### Kubernetes
156
158
-- [Kubelet](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubelet/): Monitor one or more
159
- instances of the Kubelet agent and collects metrics on number of pods/containers running, volume of Docker
160
- operations, and more.
161
-- [kube-proxy](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubeproxy/): Collect
162
- metrics, such as syncing proxy rules and REST client requests, from one or more instances of `kube-proxy`.
163
-- [Service discovery](https://github.com/netdata/agent-service-discovery/): Find what services are running on a
164
- cluster's pods, converts that into configuration files, and exports them so they can be monitored by Netdata.
157
+- [Kubelet](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubelet/): Monitor one or more
158
+ instances of the Kubelet agent and collects metrics on number of pods/containers running, volume of Docker
159
+ operations, and more.
160
+- [kube-proxy](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/k8s_kubeproxy/): Collect
161
+ metrics, such as syncing proxy rules and REST client requests, from one or more instances of `kube-proxy`.
162
+- [Service discovery](https://github.com/netdata/agent-service-discovery/): Find what services are running on a
163
+ cluster's pods, converts that into configuration files, and exports them so they can be monitored by Netdata.
164
165
### Logs
166
168
-- [Fluentd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/fluentd/): Gather application
169
- plugins metrics from an endpoint provided by `in_monitor plugin`.
170
-- [Logstash](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/logstash/): Monitor JVM threads,
171
- memory usage, garbage collection statistics, and more.
172
-- [OpenVPN status logs](/collectors/python.d.plugin/ovpn_status_log/README.md): Parse server log files and provide summary
173
- (client, traffic) metrics.
174
-- [Squid web server logs](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/squidlog/): Tail Squid
175
- access logs to return the volume of requests, types of requests, bandwidth, and much more.
176
-- [Web server logs (Go version for Apache,
177
- NGINX)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/): Tail access logs and provide
178
- very detailed web server performance statistics. This module is able to parse 200k+ rows in less than half a second.
179
-- [Web server logs (Python version for Apache, NGINX, Squid)](/collectors/python.d.plugin/web_log/README.md): Tail access log
180
- file and collect web server/caching proxy metrics.
167
+- [Fluentd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/fluentd/): Gather application
168
+ plugins metrics from an endpoint provided by `in_monitor plugin`.
169
+- [Logstash](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/logstash/): Monitor JVM threads,
170
+ memory usage, garbage collection statistics, and more.
171
+- [OpenVPN status logs](/collectors/python.d.plugin/ovpn_status_log/README.md): Parse server log files and provide
172
+ summary
173
+ (client, traffic) metrics.
174
+- [Squid web server logs](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/squidlog/): Tail Squid
175
+ access logs to return the volume of requests, types of requests, bandwidth, and much more.
176
+- [Web server logs (Go version for Apache,
177
+ NGINX)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/): Tail access logs and provide
178
+ very detailed web server performance statistics. This module is able to parse 200k+ rows in less than half a second.
179
+- [Web server logs (Apache, NGINX)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog): Tail
180
+ access log
181
+ file and collect web server/caching proxy metrics.
182
183
### Messaging
184
184
-- [ActiveMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/activemq/): Collect message broker
185
- queues and topics statistics using the ActiveMQ Console API.
186
-- [Beanstalk](/collectors/python.d.plugin/beanstalk/README.md): Collect server and tube-level statistics, such as CPU
187
- usage, jobs rates, commands, and more.
188
-- [Pulsar](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pulsar/): Collect summary,
189
- namespaces, and topics performance statistics.
190
-- [RabbitMQ (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/rabbitmq/): Collect message
191
- broker overview, system and per virtual host metrics.
192
-- [RabbitMQ (Python)](/collectors/python.d.plugin/rabbitmq/README.md): Collect message broker global and per virtual
193
- host metrics.
194
-- [VerneMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vernemq/): Monitor MQTT broker
195
- health and performance metrics. It collects all available info for both MQTTv3 and v5 communication
185
+- [ActiveMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/activemq/): Collect message broker
186
+ queues and topics statistics using the ActiveMQ Console API.
187
+- [Beanstalk](/collectors/python.d.plugin/beanstalk/README.md): Collect server and tube-level statistics, such as CPU
188
+ usage, jobs rates, commands, and more.
189
+- [Pulsar](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pulsar/): Collect summary,
190
+ namespaces, and topics performance statistics.
191
+- [RabbitMQ (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/rabbitmq/): Collect message
192
+ broker overview, system and per virtual host metrics.
193
+- [RabbitMQ (Python)](/collectors/python.d.plugin/rabbitmq/README.md): Collect message broker global and per virtual
194
+ host metrics.
195
+- [VerneMQ](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/vernemq/): Monitor MQTT broker
196
+ health and performance metrics. It collects all available info for both MQTTv3 and v5 communication
197
198
### Network
199
199
-- [Bind 9](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/bind/): Collect nameserver summary
200
- performance statistics via a web interface (`statistics-channels` feature).
201
-- [Chrony](/collectors/python.d.plugin/chrony/README.md): Monitor the precision and statistics of a local `chronyd`
202
- server.
203
-- [CoreDNS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/coredns/): Measure DNS query round
204
- trip time.
205
-- [Dnsmasq](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq_dhcp/): Automatically
206
- detects all configured `Dnsmasq` DHCP ranges and Monitor their utilization.
207
-- [DNSdist (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsdist/): Collect
208
- load-balancer performance and health metrics.
209
-- [DNSdist (Python)](/collectors/python.d.plugin/dnsdist/README.md): Collect load-balancer performance and health
210
- metrics.
211
-- [Dnsmasq DNS Forwarder](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq/): Gather
212
- queries, entries, operations, and events for the lightweight DNS forwarder.
213
-- [dns_query](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsquery/): Monitor the round
214
- trip time for DNS queries in milliseconds.
215
-- [DNS Query Time](/collectors/python.d.plugin/dns_query_time/README.md): Measure DNS query round trip time.
216
-- [Freeradius (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/freeradius/): Collect
217
- server authentication and accounting statistics from the `status server`.
218
-- [Freeradius (Python)](/collectors/python.d.plugin/freeradius/README.md): Collect server authentication and
219
- accounting statistics from the `status server` using the `radclient` tool.
220
-- [Libreswan](/collectors/charts.d.plugin/libreswan/README.md): Collect bytes-in, bytes-out, and uptime metrics.
221
-- [Icecast](/collectors/python.d.plugin/icecast/README.md): Monitor the number of listeners for active sources.
222
-- [ISC Bind (RDNC)](/collectors/python.d.plugin/bind_rndc/README.md): Collect nameserver summary performance
223
- statistics using the `rndc` tool.
224
-- [ISC DHCP (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/isc_dhcpd): Reads a
225
- `dhcpd.leases` file and collects metrics on total active leases, pool active leases, and pool utilization.
226
-- [ISC DHCP (Python)](/collectors/python.d.plugin/isc_dhcpd/README.md): Reads `dhcpd.leases` file and reports DHCP
227
- pools utilization and leases statistics (total number, leases per pool).
228
-- [OpenLDAP](/collectors/python.d.plugin/openldap/README.md): Provides statistics information from the OpenLDAP
229
- (`slapd`) server.
230
-- [NSD](/collectors/python.d.plugin/nsd/README.md): Monitor nameserver performance metrics using the `nsd-control`
231
- tool.
232
-- [NTP daemon](/collectors/python.d.plugin/ntpd/README.md): Monitor the system variables of the local `ntpd` daemon
233
- (optionally including variables of the polled peers) using the NTP Control Message Protocol via a UDP socket.
234
-- [OpenSIPS](/collectors/charts.d.plugin/opensips/README.md): Collect server health and performance metrics using the
235
- `opensipsctl` tool.
236
-- [OpenVPN](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/openvpn/): Gather server summary
237
- (client, traffic) and per user metrics (traffic, connection time) stats using `management-interface`.
238
-- [Pi-hole](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pihole/): Monitor basic (DNS
239
- queries, clients, blocklist) and extended (top clients, top permitted, and blocked domains) statistics using the PHP
240
- API.
241
-- [PowerDNS Authoritative Server
242
- (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns): Monitor one or more instances
243
- of the nameserver software to collect questions, events, and latency metrics.
244
-- [PowerDNS Recursor (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns_recursor):
245
- Gather incoming/outgoing questions, drops, timeouts, and cache usage from any number of DNS recursor instances.
246
-- [PowerDNS (Python)](/collectors/python.d.plugin/powerdns/README.md): Monitor authoritative server and recursor
247
- statistics.
248
-- [RetroShare](/collectors/python.d.plugin/retroshare/README.md): Monitor application bandwidth, peers, and DHT
249
- metrics.
250
-- [Tor](/collectors/python.d.plugin/tor/README.md): Capture traffic usage statistics using the Tor control port.
251
-- [Unbound](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/unbound/): Collect DNS resolver
252
- summary and extended system and per thread metrics via the `remote-control` interface.
200
+- [Bind 9](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/bind/): Collect nameserver summary
201
+ performance statistics via a web interface (`statistics-channels` feature).
202
+- [Chrony](/collectors/python.d.plugin/chrony/README.md): Monitor the precision and statistics of a local `chronyd`
203
+ server.
204
+- [CoreDNS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/coredns/): Measure DNS query round
205
+ trip time.
206
+- [Dnsmasq](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq_dhcp/): Automatically
207
+ detects all configured `Dnsmasq` DHCP ranges and Monitor their utilization.
208
+- [DNSdist](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsdist/): Collect
209
+ load-balancer performance and health metrics.
210
+- [Dnsmasq DNS Forwarder](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsmasq/): Gather
211
+ queries, entries, operations, and events for the lightweight DNS forwarder.
212
+- [DNS Query Time](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/dnsquery/): Monitor the round
213
+ trip time for DNS queries in milliseconds.
214
+- [Freeradius](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/freeradius/): Collect
215
+ server authentication and accounting statistics from the `status server`.
216
+- [Libreswan](/collectors/charts.d.plugin/libreswan/README.md): Collect bytes-in, bytes-out, and uptime metrics.
217
+- [Icecast](/collectors/python.d.plugin/icecast/README.md): Monitor the number of listeners for active sources.
218
+- [ISC Bind (RDNC)](/collectors/python.d.plugin/bind_rndc/README.md): Collect nameserver summary performance
219
+ statistics using the `rndc` tool.
220
+- [ISC DHCP](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/isc_dhcpd): Reads a
221
+ `dhcpd.leases` file and collects metrics on total active leases, pool active leases, and pool utilization.
222
+- [OpenLDAP](/collectors/python.d.plugin/openldap/README.md): Provides statistics information from the OpenLDAP
223
+ (`slapd`) server.
224
+- [NSD](/collectors/python.d.plugin/nsd/README.md): Monitor nameserver performance metrics using the `nsd-control`
225
+ tool.
226
+- [NTP daemon](/collectors/python.d.plugin/ntpd/README.md): Monitor the system variables of the local `ntpd` daemon
227
+ (optionally including variables of the polled peers) using the NTP Control Message Protocol via a UDP socket.
228
+- [OpenSIPS](/collectors/charts.d.plugin/opensips/README.md): Collect server health and performance metrics using the
229
+ `opensipsctl` tool.
230
+- [OpenVPN](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/openvpn/): Gather server summary
231
+ (client, traffic) and per user metrics (traffic, connection time) stats using `management-interface`.
232
+- [Pi-hole](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/pihole/): Monitor basic (DNS
233
+ queries, clients, blocklist) and extended (top clients, top permitted, and blocked domains) statistics using the PHP
234
+ API.
235
+- [PowerDNS Authoritative Server](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns):
236
+ Monitor one or more instances of the nameserver software to collect questions, events, and latency metrics.
237
+- [PowerDNS Recursor](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/powerdns_recursor):
238
+ Gather incoming/outgoing questions, drops, timeouts, and cache usage from any number of DNS recursor instances.
239
+- [RetroShare](/collectors/python.d.plugin/retroshare/README.md): Monitor application bandwidth, peers, and DHT
240
+ metrics.
241
+- [Tor](/collectors/python.d.plugin/tor/README.md): Capture traffic usage statistics using the Tor control port.
242
+- [Unbound](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/unbound/): Collect DNS resolver
243
+ summary and extended system and per thread metrics via the `remote-control` interface.
244
245
### Provisioning
246
256
-- [Puppet](/collectors/python.d.plugin/puppet/README.md): Monitor the status of Puppet Server and Puppet DB.
247
+- [Puppet](/collectors/python.d.plugin/puppet/README.md): Monitor the status of Puppet Server and Puppet DB.
248
249
### Remote devices
250
260
-- [AM2320](/collectors/python.d.plugin/am2320/README.md): Monitor sensor temperature and humidity.
261
-- [Access point](/collectors/charts.d.plugin/ap/README.md): Monitor client, traffic and signal metrics using the `aw`
262
- tool.
263
-- [APC UPS](/collectors/charts.d.plugin/apcupsd/README.md): Capture status information using the `apcaccess` tool.
264
-- [Energi Core (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/energid): Monitor
265
- blockchain indexes, memory usage, network usage, and transactions of wallet instances.
266
-- [Energi Core (Python)](/collectors/python.d.plugin/energid/README.md): Monitor blockchain, memory, network, and
267
- unspent transactions statistics.
268
-- [UPS/PDU](/collectors/charts.d.plugin/nut/README.md): Read the status of UPS/PDU devices using the `upsc` tool.
269
-- [SNMP devices](/collectors/node.d.plugin/snmp/README.md): Gather data using the SNMP protocol.
270
-- [1-Wire sensors](/collectors/python.d.plugin/w1sensor/README.md): Monitor sensor temperature.
251
+- [AM2320](/collectors/python.d.plugin/am2320/README.md): Monitor sensor temperature and humidity.
252
+- [Access point](/collectors/charts.d.plugin/ap/README.md): Monitor client, traffic and signal metrics using the `aw`
253
+ tool.
254
+- [APC UPS](/collectors/charts.d.plugin/apcupsd/README.md): Capture status information using the `apcaccess` tool.
255
+- [Energi Core](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/energid): Monitor
256
+ blockchain indexes, memory usage, network usage, and transactions of wallet instances.
257
+- [UPS/PDU](/collectors/charts.d.plugin/nut/README.md): Read the status of UPS/PDU devices using the `upsc` tool.
258
+- [SNMP devices](/collectors/node.d.plugin/snmp/README.md): Gather data using the SNMP protocol.
259
+- [1-Wire sensors](/collectors/python.d.plugin/w1sensor/README.md): Monitor sensor temperature.
260
261
### Search
262
274
-- [Elasticsearch (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/elasticsearch): Collect
275
- dozens of metrics on search engine performance from local nodes and local indices. Includes cluster health and
276
- statistics.
277
-- [Elasticsearch (Python)](/collectors/python.d.plugin/elasticsearch/README.md): Collect search engine performance and
278
- health statistics. Optionally collects per-index metrics.
279
-- [Solr](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/solr/): Collect application search
280
- requests, search errors, update requests, and update errors statistics.
263
+- [Elasticsearch](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/elasticsearch): Collect
264
+ dozens of metrics on search engine performance from local nodes and local indices. Includes cluster health and
265
+ statistics.
266
+- [Solr](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/solr/): Collect application search
267
+ requests, search errors, update requests, and update errors statistics.
268
269
### Storage
270
284
-- [Ceph](/collectors/python.d.plugin/ceph/README.md): Monitor the Ceph cluster usage and server data consumption.
285
-- [HDFS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/hdfs/): Monitor health and performance
286
- metrics for filesystem datanodes and namenodes.
287
-- [IPFS](/collectors/python.d.plugin/ipfs/README.md): Collect file system bandwidth, peers, and repo metrics.
288
-- [Scaleio](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/scaleio/): Monitor storage system,
289
- storage pools, and SDCS health and performance metrics via VxFlex OS Gateway API.
290
-- [Samba](/collectors/python.d.plugin/samba/README.md): Collect file sharing metrics using the `smbstatus` tool.
271
+- [Ceph](/collectors/python.d.plugin/ceph/README.md): Monitor the Ceph cluster usage and server data consumption.
272
+- [HDFS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/hdfs/): Monitor health and performance
273
+ metrics for filesystem datanodes and namenodes.
274
+- [IPFS](/collectors/python.d.plugin/ipfs/README.md): Collect file system bandwidth, peers, and repo metrics.
275
+- [Scaleio](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/scaleio/): Monitor storage system,
276
+ storage pools, and SDCS health and performance metrics via VxFlex OS Gateway API.
277
+- [Samba](/collectors/python.d.plugin/samba/README.md): Collect file sharing metrics using the `smbstatus` tool.
278
279
### Web
280
294
-- [Apache (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/apache/): Collect Apache web
295
- server performance metrics via the `server-status?auto` endpoint.
296
-- [Apache (Python)](/collectors/python.d.plugin/apache/README.md): Collect Apache web server performance metrics via
297
- the `server-status?auto` endpoint.
298
-- [HAProxy](/collectors/python.d.plugin/haproxy/README.md): Collect frontend, backend, and health metrics.
299
-- [HTTP endpoints (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/httpcheck/): Monitor
300
- any HTTP endpoint's availability and response time.
301
-- [HTTP endpoints (Python)](/collectors/python.d.plugin/httpcheck/README.md): Monitor any HTTP endpoint's
302
- availability and response time.
303
-- [Lighttpd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd/): Collect web server
304
- performance metrics using the `server-status?auto` endpoint.
305
-- [Lighttpd2](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd2/): Collect web server
306
- performance metrics using the `server-status?format=plain` endpoint.
307
-- [Litespeed](/collectors/python.d.plugin/litespeed/README.md): Collect web server data (network, connection,
308
- requests, cache) by reading `.rtreport*` files.
309
-- [Nginx (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginx/): Monitor web server
310
- status information by gathering metrics via `ngx_http_stub_status_module`.
311
-- [Nginx (Python)](/collectors/python.d.plugin/nginx/README.md): Monitor web server status information by gathering
312
- metrics via `ngx_http_stub_status_module`.
313
-- [Nginx Plus](/collectors/python.d.plugin/nginx_plus/README.md): Collect global and per-server zone, upstream, and
314
- cache metrics.
315
-- [Nginx VTS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginxvts/): Gathers metrics from
316
- any Nginx deployment with the _virtual host traffic status module_ enabled, including metrics on uptime, memory
317
- usage, and cache, and more.
318
-- [PHP-FPM (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpfpm/): Collect application
319
- summary and processes health metrics by scraping the status page (`/status?full`).
320
-- [PHP-FPM (Python)](/collectors/python.d.plugin/phpfpm/README.md): Collect application summary and processes health
321
- metrics by scraping the status page (`/status?full`).
322
-- [TCP endpoints (Go)](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/portcheck/): Monitor any
323
- TCP endpoint's availability and response time.
324
-- [TCP endpoints (Python)](/collectors/python.d.plugin/portcheck/README.md): Monitor any TCP endpoint's availability
325
- and response time.
326
-- [Spigot Minecraft servers](/collectors/python.d.plugin/spigotmc/README.md): Monitor average ticket rate and number
327
- of users.
328
-- [Squid](/collectors/python.d.plugin/squid/README.md): Monitor client and server bandwidth/requests by gathering
329
- data from the Cache Manager component.
330
-- [Tengine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/tengine/): Monitor web server
331
- statistics using information provided by `ngx_http_reqstat_module`.
332
-- [Tomcat](/collectors/python.d.plugin/tomcat/README.md): Collect web server performance metrics from the Manager App
333
- (`/manager/status?XML=true`).
334
-- [Traefik](/collectors/python.d.plugin/traefik/README.md): Uses Traefik's Health API to provide statistics.
335
-- [Varnish](/collectors/python.d.plugin/varnish/README.md): Provides HTTP accelerator global, backends (VBE), and
336
- disks (SMF) statistics using the `varnishstat` tool.
337
-- [x509 check](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/x509check/): Monitor certificate
338
- expiration time.
339
-- [Whois domain expiry](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/whoisquery/): Checks the
340
- remaining time until a given domain is expired.
281
+- [Apache](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/apache/): Collect Apache web
282
+ server performance metrics via the `server-status?auto` endpoint.
283
+- [HAProxy](/collectors/python.d.plugin/haproxy/README.md): Collect frontend, backend, and health metrics.
284
+- [HTTP endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/httpcheck/): Monitor
285
+ any HTTP endpoint's availability and response time.
286
+- [Lighttpd](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd/): Collect web server
287
+ performance metrics using the `server-status?auto` endpoint.
288
+- [Lighttpd2](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/lighttpd2/): Collect web server
289
+ performance metrics using the `server-status?format=plain` endpoint.
290
+- [Litespeed](/collectors/python.d.plugin/litespeed/README.md): Collect web server data (network, connection,
291
+ requests, cache) by reading `.rtreport*` files.
292
+- [Nginx](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginx/): Monitor web server
293
+ status information by gathering metrics via `ngx_http_stub_status_module`.
294
+- [Nginx VTS](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginxvts/): Gathers metrics from
295
+ any Nginx deployment with the _virtual host traffic status module_ enabled, including metrics on uptime, memory
296
+ usage, and cache, and more.
297
+- [PHP-FPM](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpfpm/): Collect application
298
+ summary and processes health metrics by scraping the status page (`/status?full`).
299
+- [TCP endpoints](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/portcheck/): Monitor any
300
+ TCP endpoint's availability and response time.
301
+- [Spigot Minecraft servers](/collectors/python.d.plugin/spigotmc/README.md): Monitor average ticket rate and number
302
+ of users.
303
+- [Squid](/collectors/python.d.plugin/squid/README.md): Monitor client and server bandwidth/requests by gathering
304
+ data from the Cache Manager component.
305
+- [Tengine](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/tengine/): Monitor web server
306
+ statistics using information provided by `ngx_http_reqstat_module`.
307
+- [Tomcat](/collectors/python.d.plugin/tomcat/README.md): Collect web server performance metrics from the Manager App
308
+ (`/manager/status?XML=true`).
309
+- [Traefik](/collectors/python.d.plugin/traefik/README.md): Uses Traefik's Health API to provide statistics.
310
+- [Varnish](/collectors/python.d.plugin/varnish/README.md): Provides HTTP accelerator global, backends (VBE), and
311
+ disks (SMF) statistics using the `varnishstat` tool.
312
+- [x509 check](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/x509check/): Monitor certificate
313
+ expiration time.
314
+- [Whois domain expiry](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/whoisquery/): Checks the
315
+ remaining time until a given domain is expired.
316
317
## System collectors
318
@@ -346,158 +321,158 @@ The Netdata Agent can collect these system- and hardware-level metrics using a v
321
322
### Applications
323
349
-- [Fail2ban](/collectors/python.d.plugin/fail2ban/README.md): Parses configuration files to detect all jails, then
350
- uses log files to report ban rates and volume of banned IPs.
351
-- [Monit](/collectors/python.d.plugin/monit/README.md): Monitor statuses of targets (service-checks) using the XML
352
- stats interface.
353
-- [WMI (Windows Management Instrumentation)
354
- exporter](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/wmi/): Collect CPU, memory,
355
- network, disk, OS, system, and log-in metrics scraping `wmi_exporter`.
324
+- [Fail2ban](/collectors/python.d.plugin/fail2ban/README.md): Parses configuration files to detect all jails, then
325
+ uses log files to report ban rates and volume of banned IPs.
326
+- [Monit](/collectors/python.d.plugin/monit/README.md): Monitor statuses of targets (service-checks) using the XML
327
+ stats interface.
328
+- [WMI (Windows Management Instrumentation)
329
+ exporter](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/wmi/): Collect CPU, memory,
330
+ network, disk, OS, system, and log-in metrics scraping `wmi_exporter`.
331
332
### Disks and filesystems
333
359
-- [BCACHE](/collectors/proc.plugin/README.md): Monitor BCACHE statistics with the the `proc.plugin` collector.
360
-- [Block devices](/collectors/proc.plugin/README.md): Gather metrics about the health and performance of block
361
- devices using the the `proc.plugin` collector.
362
-- [Btrfs](/collectors/proc.plugin/README.md): Monitors Btrfs filesystems with the the `proc.plugin` collector.
363
-- [Device mapper](/collectors/proc.plugin/README.md): Gather metrics about the Linux device mapper with the proc
364
- collector.
365
-- [Disk space](/collectors/diskspace.plugin/README.md): Collect disk space usage metrics on Linux mount points.
366
-- [Clock synchronization](/collectors/timex.plugin/README.md): Collect the system clock synchronization status on Linux.
367
-- [Files and directories](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/filecheck): Gather
368
- metrics about the existence, modification time, and size of files or directories.
369
-- [ioping.plugin](/collectors/ioping.plugin/README.md): Measure disk read/write latency.
370
-- [NFS file servers and clients](/collectors/proc.plugin/README.md): Gather operations, utilization, and space usage
371
- using the the `proc.plugin` collector.
372
-- [RAID arrays](/collectors/proc.plugin/README.md): Collect health, disk status, operation status, and more with the
373
- the `proc.plugin` collector.
374
-- [Veritas Volume Manager](/collectors/proc.plugin/README.md): Gather metrics about the Veritas Volume Manager (VVM).
375
-- [ZFS](/collectors/proc.plugin/README.md): Monitor bandwidth and utilization of ZFS disks/partitions using the proc
376
- collector.
334
+- [BCACHE](/collectors/proc.plugin/README.md): Monitor BCACHE statistics with the the `proc.plugin` collector.
335
+- [Block devices](/collectors/proc.plugin/README.md): Gather metrics about the health and performance of block
336
+ devices using the the `proc.plugin` collector.
337
+- [Btrfs](/collectors/proc.plugin/README.md): Monitors Btrfs filesystems with the the `proc.plugin` collector.
338
+- [Device mapper](/collectors/proc.plugin/README.md): Gather metrics about the Linux device mapper with the proc
339
+ collector.
340
+- [Disk space](/collectors/diskspace.plugin/README.md): Collect disk space usage metrics on Linux mount points.
341
+- [Clock synchronization](/collectors/timex.plugin/README.md): Collect the system clock synchronization status on Linux.
342
+- [Files and directories](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/filecheck): Gather
343
+ metrics about the existence, modification time, and size of files or directories.
344
+- [ioping.plugin](/collectors/ioping.plugin/README.md): Measure disk read/write latency.
345
+- [NFS file servers and clients](/collectors/proc.plugin/README.md): Gather operations, utilization, and space usage
346
+ using the the `proc.plugin` collector.
347
+- [RAID arrays](/collectors/proc.plugin/README.md): Collect health, disk status, operation status, and more with the
348
+ the `proc.plugin` collector.
349
+- [Veritas Volume Manager](/collectors/proc.plugin/README.md): Gather metrics about the Veritas Volume Manager (VVM).
350
+- [ZFS](/collectors/proc.plugin/README.md): Monitor bandwidth and utilization of ZFS disks/partitions using the proc
351
+ collector.
352
353
### eBPF
354
380
-- [Files](/collectors/ebpf.plugin/README.md): Provides information about how often a system calls kernel
381
- functions related to file descriptors using the eBPF collector.
382
-- [Virtual file system (VFS)](/collectors/ebpf.plugin/README.md): Monitor IO, errors, deleted objects, and
383
- more for kernel virtual file systems (VFS) using the eBPF collector.
384
-- [Processes](/collectors/ebpf.plugin/README.md): Monitor threads, task exits, and errors using the eBPF collector.
355
+- [Files](/collectors/ebpf.plugin/README.md): Provides information about how often a system calls kernel
356
+ functions related to file descriptors using the eBPF collector.
357
+- [Virtual file system (VFS)](/collectors/ebpf.plugin/README.md): Monitor IO, errors, deleted objects, and
358
+ more for kernel virtual file systems (VFS) using the eBPF collector.
359
+- [Processes](/collectors/ebpf.plugin/README.md): Monitor threads, task exits, and errors using the eBPF collector.
360
361
### Hardware
362
388
-- [Adaptec RAID](/collectors/python.d.plugin/adaptec_raid/README.md): Monitor logical and physical devices health
389
- metrics using the `arcconf` tool.
390
-- [CUPS](/collectors/cups.plugin/README.md): Monitor CUPS.
391
-- [FreeIPMI](/collectors/freeipmi.plugin/README.md): Uses `libipmimonitoring-dev` or `libipmimonitoring-devel` to
392
- monitor the number of sensors, temperatures, voltages, currents, and more.
393
-- [Hard drive temperature](/collectors/python.d.plugin/hddtemp/README.md): Monitor the temperature of storage
394
- devices.
395
-- [HP Smart Storage Arrays](/collectors/python.d.plugin/hpssa/README.md): Monitor controller, cache module, logical
396
- and physical drive state, and temperature using the `ssacli` tool.
397
-- [MegaRAID controllers](/collectors/python.d.plugin/megacli/README.md): Collect adapter, physical drives, and
398
- battery stats using the `megacli` tool.
399
-- [NVIDIA GPU](/collectors/python.d.plugin/nvidia_smi/README.md): Monitor performance metrics (memory usage, fan
400
- speed, pcie bandwidth utilization, temperature, and more) using the `nvidia-smi` tool.
401
-- [Sensors](/collectors/python.d.plugin/sensors/README.md): Reads system sensors information (temperature, voltage,
402
- electric current, power, and more) from `/sys/devices/`.
403
-- [S.M.A.R.T](/collectors/python.d.plugin/smartd_log/README.md): Reads SMART Disk Monitoring daemon logs.
363
+- [Adaptec RAID](/collectors/python.d.plugin/adaptec_raid/README.md): Monitor logical and physical devices health
364
+ metrics using the `arcconf` tool.
365
+- [CUPS](/collectors/cups.plugin/README.md): Monitor CUPS.
366
+- [FreeIPMI](/collectors/freeipmi.plugin/README.md): Uses `libipmimonitoring-dev` or `libipmimonitoring-devel` to
367
+ monitor the number of sensors, temperatures, voltages, currents, and more.
368
+- [Hard drive temperature](/collectors/python.d.plugin/hddtemp/README.md): Monitor the temperature of storage
369
+ devices.
370
+- [HP Smart Storage Arrays](/collectors/python.d.plugin/hpssa/README.md): Monitor controller, cache module, logical
371
+ and physical drive state, and temperature using the `ssacli` tool.
372
+- [MegaRAID controllers](/collectors/python.d.plugin/megacli/README.md): Collect adapter, physical drives, and
373
+ battery stats using the `megacli` tool.
374
+- [NVIDIA GPU](/collectors/python.d.plugin/nvidia_smi/README.md): Monitor performance metrics (memory usage, fan
375
+ speed, pcie bandwidth utilization, temperature, and more) using the `nvidia-smi` tool.
376
+- [Sensors](/collectors/python.d.plugin/sensors/README.md): Reads system sensors information (temperature, voltage,
377
+ electric current, power, and more) from `/sys/devices/`.
378
+- [S.M.A.R.T](/collectors/python.d.plugin/smartd_log/README.md): Reads SMART Disk Monitoring daemon logs.
379
380
### Memory
381
407
-- [Available memory](/collectors/proc.plugin/README.md): Tracks changes in available RAM using the the `proc.plugin`
408
- collector.
409
-- [Committed memory](/collectors/proc.plugin/README.md): Monitor committed memory using the `proc.plugin` collector.
410
-- [Huge pages](/collectors/proc.plugin/README.md): Gather metrics about huge pages in Linux and FreeBSD with the
411
- `proc.plugin` collector.
412
-- [KSM](/collectors/proc.plugin/README.md): Measure the amount of merging, savings, and effectiveness using the
413
- `proc.plugin` collector.
414
-- [Numa](/collectors/proc.plugin/README.md): Gather metrics on the number of non-uniform memory access (NUMA) events
415
- every second using the `proc.plugin` collector.
416
-- [Page faults](/collectors/proc.plugin/README.md): Collect the number of memory page faults per second using the
417
- `proc.plugin` collector.
418
-- [RAM](/collectors/proc.plugin/README.md): Collect metrics on system RAM, available RAM, and more using the
419
- `proc.plugin` collector.
420
-- [SLAB](/collectors/slabinfo.plugin/README.md): Collect kernel SLAB details on Linux systems.
421
-- [swap](/collectors/proc.plugin/README.md): Monitor the amount of free and used swap at every second using the
422
- `proc.plugin` collector.
423
-- [Writeback memory](/collectors/proc.plugin/README.md): Collect how much memory is actively being written to disk at
424
- every second using the `proc.plugin` collector.
382
+- [Available memory](/collectors/proc.plugin/README.md): Tracks changes in available RAM using the the `proc.plugin`
383
+ collector.
384
+- [Committed memory](/collectors/proc.plugin/README.md): Monitor committed memory using the `proc.plugin` collector.
385
+- [Huge pages](/collectors/proc.plugin/README.md): Gather metrics about huge pages in Linux and FreeBSD with the
386
+ `proc.plugin` collector.
387
+- [KSM](/collectors/proc.plugin/README.md): Measure the amount of merging, savings, and effectiveness using the
388
+ `proc.plugin` collector.
389
+- [Numa](/collectors/proc.plugin/README.md): Gather metrics on the number of non-uniform memory access (NUMA) events
390
+ every second using the `proc.plugin` collector.
391
+- [Page faults](/collectors/proc.plugin/README.md): Collect the number of memory page faults per second using the
392
+ `proc.plugin` collector.
393
+- [RAM](/collectors/proc.plugin/README.md): Collect metrics on system RAM, available RAM, and more using the
394
+ `proc.plugin` collector.
395
+- [SLAB](/collectors/slabinfo.plugin/README.md): Collect kernel SLAB details on Linux systems.
396
+- [swap](/collectors/proc.plugin/README.md): Monitor the amount of free and used swap at every second using the
397
+ `proc.plugin` collector.
398
+- [Writeback memory](/collectors/proc.plugin/README.md): Collect how much memory is actively being written to disk at
399
+ every second using the `proc.plugin` collector.
400
401
### Networks
402
428
-- [Access points](/collectors/charts.d.plugin/ap/README.md): Visualizes data related to access points.
429
-- [fping.plugin](fping.plugin/README.md): Measure network latency, jitter and packet loss between the monitored node
430
- and any number of remote network end points.
431
-- [Netfilter](/collectors/nfacct.plugin/README.md): Collect netfilter firewall, connection tracker, and accounting
432
- metrics using `libmnl` and `libnetfilter_acct`.
433
-- [Network stack](/collectors/proc.plugin/README.md): Monitor the networking stack for errors, TCP connection aborts,
434
- bandwidth, and more.
435
-- [Network QoS](/collectors/tc.plugin/README.md): Collect traffic QoS metrics (`tc`) of Linux network interfaces.
436
-- [SYNPROXY](/collectors/proc.plugin/README.md): Monitor entries uses, SYN packets received, TCP cookies, and more.
403
+- [Access points](/collectors/charts.d.plugin/ap/README.md): Visualizes data related to access points.
404
+- [fping.plugin](fping.plugin/README.md): Measure network latency, jitter and packet loss between the monitored node
405
+ and any number of remote network end points.
406
+- [Netfilter](/collectors/nfacct.plugin/README.md): Collect netfilter firewall, connection tracker, and accounting
407
+ metrics using `libmnl` and `libnetfilter_acct`.
408
+- [Network stack](/collectors/proc.plugin/README.md): Monitor the networking stack for errors, TCP connection aborts,
409
+ bandwidth, and more.
410
+- [Network QoS](/collectors/tc.plugin/README.md): Collect traffic QoS metrics (`tc`) of Linux network interfaces.
411
+- [SYNPROXY](/collectors/proc.plugin/README.md): Monitor entries uses, SYN packets received, TCP cookies, and more.
412
413
### Operating systems
414
440
-- [freebsd.plugin](freebsd.plugin/README.md): Collect resource usage and performance data on FreeBSD systems.
441
-- [macOS](/collectors/macos.plugin/README.md): Collect resource usage and performance data on macOS systems.
415
+- [freebsd.plugin](freebsd.plugin/README.md): Collect resource usage and performance data on FreeBSD systems.
416
+- [macOS](/collectors/macos.plugin/README.md): Collect resource usage and performance data on macOS systems.
417
418
### Processes
419
445
-- [Applications](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, eBPF, and other metrics per
446
- application using the `apps.plugin` collector.
447
-- [systemd](/collectors/cgroups.plugin/README.md): Monitor the CPU and memory usage of systemd services using the
448
- `cgroups.plugin` collector.
449
-- [systemd unit states](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/systemdunits): See the
450
- state (active, inactive, activating, deactivating, failed) of various systemd unit types.
451
-- [System processes](/collectors/proc.plugin/README.md): Collect metrics on system load and total processes running
452
- using `/proc/loadavg` and the `proc.plugin` collector.
453
-- [Uptime](/collectors/proc.plugin/README.md): Monitor the uptime of a system using the `proc.plugin` collector.
420
+- [Applications](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, eBPF, and other metrics per
421
+ application using the `apps.plugin` collector.
422
+- [systemd](/collectors/cgroups.plugin/README.md): Monitor the CPU and memory usage of systemd services using the
423
+ `cgroups.plugin` collector.
424
+- [systemd unit states](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/systemdunits): See the
425
+ state (active, inactive, activating, deactivating, failed) of various systemd unit types.
426
+- [System processes](/collectors/proc.plugin/README.md): Collect metrics on system load and total processes running
427
+ using `/proc/loadavg` and the `proc.plugin` collector.
428
+- [Uptime](/collectors/proc.plugin/README.md): Monitor the uptime of a system using the `proc.plugin` collector.
429
430
### Resources
431
457
-- [CPU frequency](/collectors/proc.plugin/README.md): Monitor CPU frequency, as set by the `cpufreq` kernel module,
458
- using the `proc.plugin` collector.
459
-- [CPU idle](/collectors/proc.plugin/README.md): Measure CPU idle every second using the `proc.plugin` collector.
460
-- [CPU performance](/collectors/perf.plugin/README.md): Collect CPU performance metrics using performance monitoring
461
- units (PMU).
462
-- [CPU throttling](/collectors/proc.plugin/README.md): Gather metrics about thermal throttling using the `/proc/stat`
463
- module and the `proc.plugin` collector.
464
-- [CPU utilization](/collectors/proc.plugin/README.md): Capture CPU utilization, both system-wide and per-core, using
465
- the `/proc/stat` module and the `proc.plugin` collector.
466
-- [Entropy](/collectors/proc.plugin/README.md): Monitor the available entropy on a system using the `proc.plugin`
467
- collector.
468
-- [Interprocess Communication (IPC)](/collectors/proc.plugin/README.md): Monitor IPC semaphores and shared memory
469
- using the `proc.plugin` collector.
470
-- [Interrupts](/collectors/proc.plugin/README.md): Monitor interrupts per second using the `proc.plugin` collector.
471
-- [IdleJitter](/collectors/idlejitter.plugin/README.md): Measure CPU latency and jitter on all operating systems.
472
-- [SoftIRQs](/collectors/proc.plugin/README.md): Collect metrics on SoftIRQs, both system-wide and per-core, using the
473
- `proc.plugin` collector.
474
-- [SoftNet](/collectors/proc.plugin/README.md): Capture SoftNet events per second, both system-wide and per-core,
475
- using the `proc.plugin` collector.
432
+- [CPU frequency](/collectors/proc.plugin/README.md): Monitor CPU frequency, as set by the `cpufreq` kernel module,
433
+ using the `proc.plugin` collector.
434
+- [CPU idle](/collectors/proc.plugin/README.md): Measure CPU idle every second using the `proc.plugin` collector.
435
+- [CPU performance](/collectors/perf.plugin/README.md): Collect CPU performance metrics using performance monitoring
436
+ units (PMU).
437
+- [CPU throttling](/collectors/proc.plugin/README.md): Gather metrics about thermal throttling using the `/proc/stat`
438
+ module and the `proc.plugin` collector.
439
+- [CPU utilization](/collectors/proc.plugin/README.md): Capture CPU utilization, both system-wide and per-core, using
440
+ the `/proc/stat` module and the `proc.plugin` collector.
441
+- [Entropy](/collectors/proc.plugin/README.md): Monitor the available entropy on a system using the `proc.plugin`
442
+ collector.
443
+- [Interprocess Communication (IPC)](/collectors/proc.plugin/README.md): Monitor IPC semaphores and shared memory
444
+ using the `proc.plugin` collector.
445
+- [Interrupts](/collectors/proc.plugin/README.md): Monitor interrupts per second using the `proc.plugin` collector.
446
+- [IdleJitter](/collectors/idlejitter.plugin/README.md): Measure CPU latency and jitter on all operating systems.
447
+- [SoftIRQs](/collectors/proc.plugin/README.md): Collect metrics on SoftIRQs, both system-wide and per-core, using the
448
+ `proc.plugin` collector.
449
+- [SoftNet](/collectors/proc.plugin/README.md): Capture SoftNet events per second, both system-wide and per-core,
450
+ using the `proc.plugin` collector.
451
452
### Users
453
479
-- [systemd-logind](/collectors/python.d.plugin/logind/README.md): Monitor active sessions, users, and seats tracked
480
- by `systemd-logind` or `elogind`.
481
-- [User/group usage](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, and other metrics per user
482
- and user group using the `apps.plugin` collector.
454
+- [systemd-logind](/collectors/python.d.plugin/logind/README.md): Monitor active sessions, users, and seats tracked
455
+ by `systemd-logind` or `elogind`.
456
+- [User/group usage](/collectors/apps.plugin/README.md): Gather CPU, disk, memory, network, and other metrics per user
457
+ and user group using the `apps.plugin` collector.
458
459
## Netdata collectors
460
461
These collectors are recursive in nature, in that they monitor some function of the Netdata Agent itself. Some
462
collectors are described only in code and associated charts in Netdata dashboards.
463
489
-- [ACLK (code only)](https://github.com/netdata/netdata/blob/master/aclk/legacy/aclk_stats.c): View whether a Netdata
490
- Agent is connected to Netdata Cloud via the [ACLK](/aclk/README.md), the volume of queries, process times, and more.
491
-- [Alarms](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/alarms): This collector creates an
492
- **Alarms** menu with one line plot showing the alarm states of a Netdata Agent over time.
493
-- [Anomalies](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/anomalies): This collector uses the
494
- Python PyOD library to perform unsupervised anomaly detection on your Netdata charts and/or dimensions.
495
-- [Exporting (code only)](https://github.com/netdata/netdata/blob/master/exporting/send_internal_metrics.c): Gather
496
- metrics on CPU utilization for the [exporting engine](/exporting/README.md), and specific metrics for each enabled
497
- exporting connector.
498
-- [Global statistics (code only)](https://github.com/netdata/netdata/blob/master/daemon/global_statistics.c): See
499
- metrics on the CPU utilization, network traffic, volume of web clients, API responses, database engine usage, and
500
- more.
464
+- [ACLK (code only)](https://github.com/netdata/netdata/blob/master/aclk/legacy/aclk_stats.c): View whether a Netdata
465
+ Agent is connected to Netdata Cloud via the [ACLK](/aclk/README.md), the volume of queries, process times, and more.
466
+- [Alarms](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/alarms): This collector creates an
467
+ **Alarms** menu with one line plot showing the alarm states of a Netdata Agent over time.
468
+- [Anomalies](https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin/anomalies): This collector uses the
469
+ Python PyOD library to perform unsupervised anomaly detection on your Netdata charts and/or dimensions.
470
+- [Exporting (code only)](https://github.com/netdata/netdata/blob/master/exporting/send_internal_metrics.c): Gather
471
+ metrics on CPU utilization for the [exporting engine](/exporting/README.md), and specific metrics for each enabled
472
+ exporting connector.
473
+- [Global statistics (code only)](https://github.com/netdata/netdata/blob/master/daemon/global_statistics.c): See
474
+ metrics on the CPU utilization, network traffic, volume of web clients, API responses, database engine usage, and
475
+ more.
476
477
## Orchestrators
478
@@ -506,31 +481,29 @@ Plugin orchestrators organize and run many of the above collectors.
481
If you're interested in developing a new collector that you'd like to contribute to Netdata, we highly recommend using
482
the `go.d.plugin`.
483
509
-- [go.d.plugin](https://github.com/netdata/go.d.plugin): An orchestrator for data collection modules written in `go`.
510
-- [python.d.plugin](python.d.plugin/README.md): An orchestrator for data collection modules written in `python` v2/v3.
511
-- [charts.d.plugin](charts.d.plugin/README.md): An orchestrator for data collection modules written in `bash` v4+.
512
-- [node.d.plugin](node.d.plugin/README.md): An orchestrator for data collection modules written in `node.js`.
484
+- [go.d.plugin](https://github.com/netdata/go.d.plugin): An orchestrator for data collection modules written in `go`.
485
+- [python.d.plugin](python.d.plugin/README.md): An orchestrator for data collection modules written in `python` v2/v3.
486
+- [charts.d.plugin](charts.d.plugin/README.md): An orchestrator for data collection modules written in `bash` v4+.
487
+- [node.d.plugin](node.d.plugin/README.md): An orchestrator for data collection modules written in `node.js`.
488
489
## Third-party collectors
490
491
These collectors are developed and maintained by third parties and, unlike the other collectors, are not installed by
492
default. To use a third-party collector, visit their GitHub/documentation page and follow their installation procedures.
493
519
-- [CyberPower UPS](https://github.com/HawtDogFlvrWtr/netdata_cyberpwrups_plugin): Polls CyberPower UPS data using
520
- PowerPanel® Personal Linux.
521
-- [Logged-in users](https://github.com/veksh/netdata-numsessions): Collect the number of currently logged-on users.
522
-- [nextcloud](https://github.com/arnowelzel/netdata-nextcloud): Monitor Nextcloud servers.
523
-- [nim-netdata-plugin](https://github.com/FedericoCeratto/nim-netdata-plugin): A helper to create native Netdata
524
- plugins using Nim.
525
-- [Nvidia GPUs](https://github.com/coraxx/netdata_nv_plugin): Monitor Nvidia GPUs.
526
-- [Teamspeak 3](https://github.com/coraxx/netdata_ts3_plugin): Pulls active users and bandwidth from TeamSpeak 3
527
- servers.
528
-- [SSH](https://github.com/Yaser-Amiri/netdata-ssh-module): Monitor failed authentication requests of an SSH server.
494
+- [CyberPower UPS](https://github.com/HawtDogFlvrWtr/netdata_cyberpwrups_plugin): Polls CyberPower UPS data using
495
+ PowerPanel® Personal Linux.
496
+- [Logged-in users](https://github.com/veksh/netdata-numsessions): Collect the number of currently logged-on users.
497
+- [nextcloud](https://github.com/arnowelzel/netdata-nextcloud): Monitor Nextcloud servers.
498
+- [nim-netdata-plugin](https://github.com/FedericoCeratto/nim-netdata-plugin): A helper to create native Netdata
499
+ plugins using Nim.
500
+- [Nvidia GPUs](https://github.com/coraxx/netdata_nv_plugin): Monitor Nvidia GPUs.
501
+- [Teamspeak 3](https://github.com/coraxx/netdata_ts3_plugin): Pulls active users and bandwidth from TeamSpeak 3
502
+ servers.
503
+- [SSH](https://github.com/Yaser-Amiri/netdata-ssh-module): Monitor failed authentication requests of an SSH server.
504
505
## Etc
506
532
-- [checks.plugin](checks.plugin/README.md): A debugging collector, disabled by default.
533
-- [charts.d example](charts.d.plugin/example/README.md): An example `charts.d` collector.
534
-- [python.d example](python.d.plugin/example/README.md): An example `python.d` collector.
535
-
536
-
507
+- [checks.plugin](checks.plugin/README.md): A debugging collector, disabled by default.
508
+- [charts.d example](charts.d.plugin/example/README.md): An example `charts.d` collector.
509
+- [python.d example](python.d.plugin/example/README.md): An example `python.d` collector.
collectors/python.d.plugin/Makefile.am
-16
@@ -43,41 +43,30 @@ include adaptec_raid/Makefile.inc
43
include alarms/Makefile.inc
44
include am2320/Makefile.inc
45
include anomalies/Makefile.inc
46
-include apache/Makefile.inc
46
include beanstalk/Makefile.inc
47
include bind_rndc/Makefile.inc
48
include boinc/Makefile.inc
49
include ceph/Makefile.inc
50
include changefinder/Makefile.inc
51
include chrony/Makefile.inc
53
-include couchdb/Makefile.inc
54
-include dnsdist/Makefile.inc
55
-include dns_query_time/Makefile.inc
52
include dockerd/Makefile.inc
53
include dovecot/Makefile.inc
58
-include elasticsearch/Makefile.inc
59
-include energid/Makefile.inc
54
include example/Makefile.inc
55
include exim/Makefile.inc
56
include fail2ban/Makefile.inc
63
-include freeradius/Makefile.inc
57
include gearman/Makefile.inc
58
include go_expvar/Makefile.inc
59
include haproxy/Makefile.inc
60
include hddtemp/Makefile.inc
68
-include httpcheck/Makefile.inc
61
include hpssa/Makefile.inc
62
include icecast/Makefile.inc
63
include ipfs/Makefile.inc
72
-include isc_dhcpd/Makefile.inc
64
include litespeed/Makefile.inc
65
include logind/Makefile.inc
66
include megacli/Makefile.inc
67
include memcached/Makefile.inc
68
include mongodb/Makefile.inc
69
include monit/Makefile.inc
79
-include mysql/Makefile.inc
80
-include nginx/Makefile.inc
70
include nginx_plus/Makefile.inc
71
include nvidia_smi/Makefile.inc
72
include nsd/Makefile.inc
@@ -85,15 +74,11 @@ include ntpd/Makefile.inc
74
include ovpn_status_log/Makefile.inc
75
include openldap/Makefile.inc
76
include oracledb/Makefile.inc
88
-include phpfpm/Makefile.inc
89
-include portcheck/Makefile.inc
77
include postfix/Makefile.inc
78
include postgres/Makefile.inc
92
-include powerdns/Makefile.inc
79
include proxysql/Makefile.inc
80
include puppet/Makefile.inc
81
include rabbitmq/Makefile.inc
96
-include redis/Makefile.inc
82
include rethinkdbs/Makefile.inc
83
include retroshare/Makefile.inc
84
include riakkv/Makefile.inc
@@ -109,7 +94,6 @@ include traefik/Makefile.inc
94
include uwsgi/Makefile.inc
95
include varnish/Makefile.inc
96
include w1sensor/Makefile.inc
112
-include web_log/Makefile.inc
97
include zscores/Makefile.inc
98
99
pythonmodulesdir=$(pythondir)/python_modules
collectors/python.d.plugin/README.md
+1
-2
@@ -227,8 +227,7 @@ For additional security it uses python `subprocess.Popen` (without `shell=True`
227
228
_Examples: `apache`, `nginx`, `tomcat`_
229
230
-_Multiple Endpoints (urls) Examples: [`rabbitmq`](/collectors/python.d.plugin/rabbitmq/README.md) (simpler) ,
231
-[`elasticsearch`](/collectors/python.d.plugin/elasticsearch/README.md) (threaded)_
230
+_Multiple Endpoints (urls) Examples: [`rabbitmq`](/collectors/python.d.plugin/rabbitmq/README.md) (simpler).
231
232
233
_Variables from config file_: `url`, `user`, `pass`.
collectors/python.d.plugin/apache/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += apache/apache.chart.py
9
-dist_pythonconfig_DATA += apache/apache.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += apache/README.md apache/Makefile.inc
13
-
collectors/python.d.plugin/apache/README.md
deleted
-82
@@ -1,82 +0,0 @@
1
-<!--
2
-title: "Apache monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/apache/README.md
4
-sidebar_label: "Apache"
5
--->
6
-
7
-# Apache monitoring with Netdata
8
-
9
-Monitors one or more Apache servers depending on configuration.
10
-
11
-## Requirements
12
-
13
-- apache with enabled `mod_status`
14
-
15
-It produces the following charts:
16
-
17
-1. **Requests** in requests/s
18
-
19
- - requests
20
-
21
-2. **Connections**
22
-
23
- - connections
24
-
25
-3. **Async Connections**
26
-
27
- - keepalive
28
- - closing
29
- - writing
30
-
31
-4. **Bandwidth** in kilobytes/s
32
-
33
- - sent
34
-
35
-5. **Workers**
36
-
37
- - idle
38
- - busy
39
-
40
-6. **Lifetime Avg. Requests/s** in requests/s
41
-
42
- - requests_sec
43
-
44
-7. **Lifetime Avg. Bandwidth/s** in kilobytes/s
45
-
46
- - size_sec
47
-
48
-8. **Lifetime Avg. Response Size** in bytes/request
49
-
50
- - size_req
51
-
52
-## Configuration
53
-
54
-Edit the `python.d/apache.conf` configuration file using `edit-config` from the Netdata [config
55
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
56
-
57
-```bash
58
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
59
-sudo ./edit-config python.d/apache.conf
60
-```
61
-
62
-Needs only `url` to server's `server-status?auto`
63
-
64
-Example for two servers:
65
-
66
-```yaml
67
-update_every : 10
68
-priority : 90100
69
-
70
-local:
71
- url : 'http://localhost/server-status?auto'
72
-
73
-remote:
74
- url : 'http://www.apache.org/server-status?auto'
75
- update_every : 5
76
-```
77
-
78
-Without configuration, module attempts to connect to `http://localhost/server-status?auto`
79
-
80
----
81
-
82
-
collectors/python.d.plugin/apache/apache.chart.py
deleted
-159
@@ -1,159 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: apache netdata python.d module
3
-# Author: Pawel Krupa (paulfantom)
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-from bases.FrameworkServices.UrlService import UrlService
7
-
8
-ORDER = [
9
- 'requests',
10
- 'connections',
11
- 'conns_async',
12
- 'net',
13
- 'workers',
14
- 'reqpersec',
15
- 'bytespersec',
16
- 'bytesperreq',
17
-]
18
-
19
-CHARTS = {
20
- 'bytesperreq': {
21
- 'options': [None, 'Lifetime Avg. Request Size', 'KiB',
22
- 'statistics', 'apache.bytesperreq', 'area'],
23
- 'lines': [
24
- ['size_req', 'size', 'absolute', 1, 1024 * 100000]
25
- ]},
26
- 'workers': {
27
- 'options': [None, 'Workers', 'workers', 'workers', 'apache.workers', 'stacked'],
28
- 'lines': [
29
- ['idle'],
30
- ['busy'],
31
- ]},
32
- 'reqpersec': {
33
- 'options': [None, 'Lifetime Avg. Requests/s', 'requests/s', 'statistics',
34
- 'apache.reqpersec', 'area'],
35
- 'lines': [
36
- ['requests_sec', 'requests', 'absolute', 1, 100000]
37
- ]},
38
- 'bytespersec': {
39
- 'options': [None, 'Lifetime Avg. Bandwidth/s', 'kilobits/s', 'statistics',
40
- 'apache.bytespersec', 'area'],
41
- 'lines': [
42
- ['size_sec', None, 'absolute', 8, 1000 * 100000]
43
- ]},
44
- 'requests': {
45
- 'options': [None, 'Requests', 'requests/s', 'requests', 'apache.requests', 'line'],
46
- 'lines': [
47
- ['requests', None, 'incremental']
48
- ]},
49
- 'net': {
50
- 'options': [None, 'Bandwidth', 'kilobits/s', 'bandwidth', 'apache.net', 'area'],
51
- 'lines': [
52
- ['sent', None, 'incremental', 8, 1]
53
- ]},
54
- 'connections': {
55
- 'options': [None, 'Connections', 'connections', 'connections', 'apache.connections', 'line'],
56
- 'lines': [
57
- ['connections']
58
- ]},
59
- 'conns_async': {
60
- 'options': [None, 'Async Connections', 'connections', 'connections', 'apache.conns_async', 'stacked'],
61
- 'lines': [
62
- ['keepalive'],
63
- ['closing'],
64
- ['writing']
65
- ]}
66
-}
67
-
68
-ASSIGNMENT = {
69
- 'BytesPerReq': 'size_req',
70
- 'IdleWorkers': 'idle',
71
- 'IdleServers': 'idle_servers',
72
- 'BusyWorkers': 'busy',
73
- 'BusyServers': 'busy_servers',
74
- 'ReqPerSec': 'requests_sec',
75
- 'BytesPerSec': 'size_sec',
76
- 'Total Accesses': 'requests',
77
- 'Total kBytes': 'sent',
78
- 'ConnsTotal': 'connections',
79
- 'ConnsAsyncKeepAlive': 'keepalive',
80
- 'ConnsAsyncClosing': 'closing',
81
- 'ConnsAsyncWriting': 'writing'
82
-}
83
-
84
-FLOAT_VALUES = [
85
- 'BytesPerReq',
86
- 'ReqPerSec',
87
- 'BytesPerSec',
88
-]
89
-
90
-LIGHTTPD_MARKER = 'idle_servers'
91
-
92
-
93
-class Service(UrlService):
94
- def __init__(self, configuration=None, name=None):
95
- UrlService.__init__(self, configuration=configuration, name=name)
96
- self.order = ORDER
97
- self.definitions = CHARTS
98
- self.url = self.configuration.get('url', 'http://localhost/server-status?auto')
99
-
100
- def check(self):
101
- self._manager = self._build_manager()
102
-
103
- data = self._get_data()
104
-
105
- if not data:
106
- return None
107
-
108
- if LIGHTTPD_MARKER in data:
109
- self.turn_into_lighttpd()
110
-
111
- return True
112
-
113
- def _get_data(self):
114
- """
115
- Format data received from http request
116
- :return: dict
117
- """
118
- raw_data = self._get_raw_data()
119
-
120
- if not raw_data:
121
- return None
122
-
123
- data = dict()
124
-
125
- for line in raw_data.split('\n'):
126
- try:
127
- parse_line(line, data)
128
- except ValueError:
129
- continue
130
-
131
- return data or None
132
-
133
- def turn_into_lighttpd(self):
134
- self.module_name = 'lighttpd'
135
- for chart in self.definitions:
136
- if chart == 'workers':
137
- lines = self.definitions[chart]['lines']
138
- lines[0] = ['idle_servers', 'idle']
139
- lines[1] = ['busy_servers', 'busy']
140
- opts = self.definitions[chart]['options']
141
- opts[1] = opts[1].replace('apache', 'lighttpd')
142
- opts[4] = opts[4].replace('apache', 'lighttpd')
143
-
144
-
145
-def parse_line(line, data):
146
- parts = line.split(':')
147
-
148
- if len(parts) != 2:
149
- return
150
-
151
- key, value = parts[0], parts[1]
152
-
153
- if key not in ASSIGNMENT:
154
- return
155
-
156
- if key in FLOAT_VALUES:
157
- data[ASSIGNMENT[key]] = int((float(value) * 100000))
158
- else:
159
- data[ASSIGNMENT[key]] = int(value)
collectors/python.d.plugin/apache/apache.conf
deleted
-85
@@ -1,85 +0,0 @@
1
-# netdata python.d.plugin configuration for apache
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, apache also supports the following:
63
-#
64
-# url: 'URL' # the URL to fetch apache's mod_status stats
65
-#
66
-# if the URL is password protected, the following are supported:
67
-#
68
-# user: 'username'
69
-# pass: 'password'
70
-
71
-# ----------------------------------------------------------------------
72
-# AUTO-DETECTION JOBS
73
-# only one of them will run (they have the same name)
74
-
75
-localhost:
76
- name : 'local'
77
- url : 'http://localhost/server-status?auto'
78
-
79
-localipv4:
80
- name : 'local'
81
- url : 'http://127.0.0.1/server-status?auto'
82
-
83
-localipv6:
84
- name : 'local'
85
- url : 'http://[::1]/server-status?auto'
collectors/python.d.plugin/couchdb/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += couchdb/couchdb.chart.py
9
-dist_pythonconfig_DATA += couchdb/couchdb.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += couchdb/README.md couchdb/Makefile.inc
13
-
collectors/python.d.plugin/couchdb/README.md
deleted
-53
@@ -1,53 +0,0 @@
1
-<!--
2
-title: "Apache CouchDB monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/couchdb/README.md
4
-sidebar_label: "CouchDB"
5
--->
6
-
7
-# Apache CouchDB monitoring with Netdata
8
-
9
-Monitors vital statistics of a local Apache CouchDB 2.x server, including:
10
-
11
-- Overall server reads/writes
12
-- HTTP traffic breakdown
13
- - Request methods (`GET`, `PUT`, `POST`, etc.)
14
- - Response status codes (`200`, `201`, `4xx`, etc.)
15
-- Active server tasks
16
-- Replication status (CouchDB 2.1 and up only)
17
-- Erlang VM stats
18
-- Optional per-database statistics: sizes, # of docs, # of deleted docs
19
-
20
-## Configuration
21
-
22
-Edit the `python.d/couchdb.conf` configuration file using `edit-config` from the Netdata [config
23
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
24
-
25
-```bash
26
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
27
-sudo ./edit-config python.d/couchdb.conf
28
-```
29
-
30
-Sample for a local server running on port 5984:
31
-
32
-```yaml
33
-local:
34
- user: 'admin'
35
- pass: 'password'
36
- node: 'couchdb@127.0.0.1'
37
-```
38
-
39
-Be sure to specify a correct admin-level username and password.
40
-
41
-You may also need to change the `node` name; this should match the value of `-name NODENAME` in your CouchDB's `etc/vm.args` file. Typically this is of the form `couchdb@fully.qualified.domain.name` in a cluster, or `couchdb@127.0.0.1` / `couchdb@localhost` for a single-node server.
42
-
43
-If you want per-database statistics, these need to be added to the configuration, separated by spaces:
44
-
45
-```yaml
46
-local:
47
- ...
48
- databases: 'db1 db2 db3 ...'
49
-```
50
-
51
----
52
-
53
-
collectors/python.d.plugin/couchdb/couchdb.chart.py
deleted
-398
@@ -1,398 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: couchdb netdata python.d module
3
-# Author: wohali <wohali@apache.org>
4
-# Thanks to ilyam8 for good examples :)
5
-# SPDX-License-Identifier: GPL-3.0-or-later
6
-
7
-from collections import namedtuple, defaultdict
8
-from json import loads
9
-from socket import gethostbyname, gaierror
10
-from threading import Thread
11
-
12
-try:
13
- from queue import Queue
14
-except ImportError:
15
- from Queue import Queue
16
-
17
-from bases.FrameworkServices.UrlService import UrlService
18
-
19
-update_every = 1
20
-
21
-METHODS = namedtuple('METHODS', ['get_data', 'url', 'stats'])
22
-
23
-OVERVIEW_STATS = [
24
- 'couchdb.database_reads.value',
25
- 'couchdb.database_writes.value',
26
- 'couchdb.httpd.view_reads.value',
27
- 'couchdb.httpd_request_methods.COPY.value',
28
- 'couchdb.httpd_request_methods.DELETE.value',
29
- 'couchdb.httpd_request_methods.GET.value',
30
- 'couchdb.httpd_request_methods.HEAD.value',
31
- 'couchdb.httpd_request_methods.OPTIONS.value',
32
- 'couchdb.httpd_request_methods.POST.value',
33
- 'couchdb.httpd_request_methods.PUT.value',
34
- 'couchdb.httpd_status_codes.200.value',
35
- 'couchdb.httpd_status_codes.201.value',
36
- 'couchdb.httpd_status_codes.202.value',
37
- 'couchdb.httpd_status_codes.204.value',
38
- 'couchdb.httpd_status_codes.206.value',
39
- 'couchdb.httpd_status_codes.301.value',
40
- 'couchdb.httpd_status_codes.302.value',
41
- 'couchdb.httpd_status_codes.304.value',
42
- 'couchdb.httpd_status_codes.400.value',
43
- 'couchdb.httpd_status_codes.401.value',
44
- 'couchdb.httpd_status_codes.403.value',
45
- 'couchdb.httpd_status_codes.404.value',
46
- 'couchdb.httpd_status_codes.405.value',
47
- 'couchdb.httpd_status_codes.406.value',
48
- 'couchdb.httpd_status_codes.409.value',
49
- 'couchdb.httpd_status_codes.412.value',
50
- 'couchdb.httpd_status_codes.413.value',
51
- 'couchdb.httpd_status_codes.414.value',
52
- 'couchdb.httpd_status_codes.415.value',
53
- 'couchdb.httpd_status_codes.416.value',
54
- 'couchdb.httpd_status_codes.417.value',
55
- 'couchdb.httpd_status_codes.500.value',
56
- 'couchdb.httpd_status_codes.501.value',
57
- 'couchdb.open_os_files.value',
58
- 'couch_replicator.jobs.running.value',
59
- 'couch_replicator.jobs.pending.value',
60
- 'couch_replicator.jobs.crashed.value',
61
-]
62
-
63
-SYSTEM_STATS = [
64
- 'context_switches',
65
- 'run_queue',
66
- 'ets_table_count',
67
- 'reductions',
68
- 'memory.atom',
69
- 'memory.atom_used',
70
- 'memory.binary',
71
- 'memory.code',
72
- 'memory.ets',
73
- 'memory.other',
74
- 'memory.processes',
75
- 'io_input',
76
- 'io_output',
77
- 'os_proc_count',
78
- 'process_count',
79
- 'internal_replication_jobs'
80
-]
81
-
82
-DB_STATS = [
83
- 'doc_count',
84
- 'doc_del_count',
85
- 'sizes.file',
86
- 'sizes.external',
87
- 'sizes.active'
88
-]
89
-
90
-ORDER = [
91
- 'activity',
92
- 'request_methods',
93
- 'response_codes',
94
- 'active_tasks',
95
- 'replicator_jobs',
96
- 'open_files',
97
- 'db_sizes_file',
98
- 'db_sizes_external',
99
- 'db_sizes_active',
100
- 'db_doc_counts',
101
- 'db_doc_del_counts',
102
- 'erlang_memory',
103
- 'erlang_proc_counts',
104
- 'erlang_peak_msg_queue',
105
- 'erlang_reductions'
106
-]
107
-
108
-CHARTS = {
109
- 'activity': {
110
- 'options': [None, 'Overall Activity', 'requests/s',
111
- 'dbactivity', 'couchdb.activity', 'stacked'],
112
- 'lines': [
113
- ['couchdb_database_reads', 'DB reads', 'incremental'],
114
- ['couchdb_database_writes', 'DB writes', 'incremental'],
115
- ['couchdb_httpd_view_reads', 'View reads', 'incremental']
116
- ]
117
- },
118
- 'request_methods': {
119
- 'options': [None, 'HTTP request methods', 'requests/s',
120
- 'httptraffic', 'couchdb.request_methods',
121
- 'stacked'],
122
- 'lines': [
123
- ['couchdb_httpd_request_methods_COPY', 'COPY', 'incremental'],
124
- ['couchdb_httpd_request_methods_DELETE', 'DELETE', 'incremental'],
125
- ['couchdb_httpd_request_methods_GET', 'GET', 'incremental'],
126
- ['couchdb_httpd_request_methods_HEAD', 'HEAD', 'incremental'],
127
- ['couchdb_httpd_request_methods_OPTIONS', 'OPTIONS',
128
- 'incremental'],
129
- ['couchdb_httpd_request_methods_POST', 'POST', 'incremental'],
130
- ['couchdb_httpd_request_methods_PUT', 'PUT', 'incremental']
131
- ]
132
- },
133
- 'response_codes': {
134
- 'options': [None, 'HTTP response status codes', 'responses/s',
135
- 'httptraffic', 'couchdb.response_codes',
136
- 'stacked'],
137
- 'lines': [
138
- ['couchdb_httpd_status_codes_200', '200 OK', 'incremental'],
139
- ['couchdb_httpd_status_codes_201', '201 Created', 'incremental'],
140
- ['couchdb_httpd_status_codes_202', '202 Accepted', 'incremental'],
141
- ['couchdb_httpd_status_codes_2xx', 'Other 2xx Success',
142
- 'incremental'],
143
- ['couchdb_httpd_status_codes_3xx', '3xx Redirection',
144
- 'incremental'],
145
- ['couchdb_httpd_status_codes_4xx', '4xx Client error',
146
- 'incremental'],
147
- ['couchdb_httpd_status_codes_5xx', '5xx Server error',
148
- 'incremental']
149
- ]
150
- },
151
- 'open_files': {
152
- 'options': [None, 'Open files', 'files', 'ops', 'couchdb.open_files', 'line'],
153
- 'lines': [
154
- ['couchdb_open_os_files', '# files', 'absolute']
155
- ]
156
- },
157
- 'active_tasks': {
158
- 'options': [None, 'Active task breakdown', 'tasks', 'ops', 'couchdb.active_tasks', 'stacked'],
159
- 'lines': [
160
- ['activetasks_indexer', 'Indexer', 'absolute'],
161
- ['activetasks_database_compaction', 'DB Compaction', 'absolute'],
162
- ['activetasks_replication', 'Replication', 'absolute'],
163
- ['activetasks_view_compaction', 'View Compaction', 'absolute']
164
- ]
165
- },
166
- 'replicator_jobs': {
167
- 'options': [None, 'Replicator job breakdown', 'jobs', 'ops', 'couchdb.replicator_jobs', 'stacked'],
168
- 'lines': [
169
- ['couch_replicator_jobs_running', 'Running', 'absolute'],
170
- ['couch_replicator_jobs_pending', 'Pending', 'absolute'],
171
- ['couch_replicator_jobs_crashed', 'Crashed', 'absolute'],
172
- ['internal_replication_jobs', 'Internal replication jobs',
173
- 'absolute']
174
- ]
175
- },
176
- 'erlang_memory': {
177
- 'options': [None, 'Erlang VM memory usage', 'B', 'erlang', 'couchdb.erlang_vm_memory', 'stacked'],
178
- 'lines': [
179
- ['memory_atom', 'atom', 'absolute'],
180
- ['memory_binary', 'binaries', 'absolute'],
181
- ['memory_code', 'code', 'absolute'],
182
- ['memory_ets', 'ets', 'absolute'],
183
- ['memory_processes', 'procs', 'absolute'],
184
- ['memory_other', 'other', 'absolute']
185
- ]
186
- },
187
- 'erlang_reductions': {
188
- 'options': [None, 'Erlang reductions', 'count', 'erlang', 'couchdb.reductions', 'line'],
189
- 'lines': [
190
- ['reductions', 'reductions', 'incremental']
191
- ]
192
- },
193
- 'erlang_proc_counts': {
194
- 'options': [None, 'Process counts', 'count', 'erlang', 'couchdb.proccounts', 'line'],
195
- 'lines': [
196
- ['os_proc_count', 'OS procs', 'absolute'],
197
- ['process_count', 'erl procs', 'absolute']
198
- ]
199
- },
200
- 'erlang_peak_msg_queue': {
201
- 'options': [None, 'Peak message queue size', 'count', 'erlang', 'couchdb.peakmsgqueue',
202
- 'line'],
203
- 'lines': [
204
- ['peak_msg_queue', 'peak size', 'absolute']
205
- ]
206
- },
207
- # Lines for the following are added as part of check()
208
- 'db_sizes_file': {
209
- 'options': [None, 'Database sizes (file)', 'KiB', 'perdbstats', 'couchdb.db_sizes_file', 'line'],
210
- 'lines': []
211
- },
212
- 'db_sizes_external': {
213
- 'options': [None, 'Database sizes (external)', 'KiB', 'perdbstats', 'couchdb.db_sizes_external', 'line'],
214
- 'lines': []
215
- },
216
- 'db_sizes_active': {
217
- 'options': [None, 'Database sizes (active)', 'KiB', 'perdbstats', 'couchdb.db_sizes_active', 'line'],
218
- 'lines': []
219
- },
220
- 'db_doc_counts': {
221
- 'options': [None, 'Database # of docs', 'docs',
222
- 'perdbstats', 'couchdb_db_doc_count', 'line'],
223
- 'lines': []
224
- },
225
- 'db_doc_del_counts': {
226
- 'options': [None, 'Database # of deleted docs', 'docs', 'perdbstats', 'couchdb_db_doc_del_count', 'line'],
227
- 'lines': []
228
- }
229
-}
230
-
231
-
232
-class Service(UrlService):
233
- def __init__(self, configuration=None, name=None):
234
- UrlService.__init__(self, configuration=configuration, name=name)
235
- self.order = ORDER
236
- self.definitions = CHARTS
237
- self.host = self.configuration.get('host', '127.0.0.1')
238
- self.port = self.configuration.get('port', 5984)
239
- self.node = self.configuration.get('node', 'couchdb@127.0.0.1')
240
- self.scheme = self.configuration.get('scheme', 'http')
241
- self.user = self.configuration.get('user')
242
- self.password = self.configuration.get('pass')
243
- try:
244
- self.dbs = self.configuration.get('databases').split(' ')
245
- except (KeyError, AttributeError):
246
- self.dbs = list()
247
-
248
- def check(self):
249
- if not (self.host and self.port):
250
- self.error('Host is not defined in the module configuration file')
251
- return False
252
- try:
253
- self.host = gethostbyname(self.host)
254
- except gaierror as error:
255
- self.error(str(error))
256
- return False
257
- self.url = '{scheme}://{host}:{port}'.format(scheme=self.scheme,
258
- host=self.host,
259
- port=self.port)
260
- stats = self.url + '/_node/{node}/_stats'.format(node=self.node)
261
- active_tasks = self.url + '/_active_tasks'
262
- system = self.url + '/_node/{node}/_system'.format(node=self.node)
263
- self.methods = [METHODS(get_data=self._get_overview_stats,
264
- url=stats,
265
- stats=OVERVIEW_STATS),
266
- METHODS(get_data=self._get_active_tasks_stats,
267
- url=active_tasks,
268
- stats=None),
269
- METHODS(get_data=self._get_overview_stats,
270
- url=system,
271
- stats=SYSTEM_STATS),
272
- METHODS(get_data=self._get_dbs_stats,
273
- url=self.url,
274
- stats=DB_STATS)]
275
- # must initialise manager before using _get_raw_data
276
- self._manager = self._build_manager()
277
- self.dbs = [db for db in self.dbs
278
- if self._get_raw_data(self.url + '/' + db)]
279
- for db in self.dbs:
280
- self.definitions['db_sizes_file']['lines'].append(
281
- ['db_' + db + '_sizes_file', db, 'absolute', 1, 1000]
282
- )
283
- self.definitions['db_sizes_external']['lines'].append(
284
- ['db_' + db + '_sizes_external', db, 'absolute', 1, 1000]
285
- )
286
- self.definitions['db_sizes_active']['lines'].append(
287
- ['db_' + db + '_sizes_active', db, 'absolute', 1, 1000]
288
- )
289
- self.definitions['db_doc_counts']['lines'].append(
290
- ['db_' + db + '_doc_count', db, 'absolute']
291
- )
292
- self.definitions['db_doc_del_counts']['lines'].append(
293
- ['db_' + db + '_doc_del_count', db, 'absolute']
294
- )
295
- return UrlService.check(self)
296
-
297
- def _get_data(self):
298
- threads = list()
299
- queue = Queue()
300
- result = dict()
301
-
302
- for method in self.methods:
303
- th = Thread(target=method.get_data,
304
- args=(queue, method.url, method.stats))
305
- th.start()
306
- threads.append(th)
307
-
308
- for thread in threads:
309
- thread.join()
310
- result.update(queue.get())
311
-
312
- # self.info('couchdb result = ' + str(result))
313
- return result or None
314
-
315
- def _get_overview_stats(self, queue, url, stats):
316
- raw_data = self._get_raw_data(url)
317
- if not raw_data:
318
- return queue.put(dict())
319
- data = loads(raw_data)
320
- to_netdata = self._fetch_data(raw_data=data, metrics=stats)
321
- if 'message_queues' in data:
322
- to_netdata['peak_msg_queue'] = get_peak_msg_queue(data)
323
- return queue.put(to_netdata)
324
-
325
- def _get_active_tasks_stats(self, queue, url, _):
326
- taskdict = defaultdict(int)
327
- taskdict["activetasks_indexer"] = 0
328
- taskdict["activetasks_database_compaction"] = 0
329
- taskdict["activetasks_replication"] = 0
330
- taskdict["activetasks_view_compaction"] = 0
331
- raw_data = self._get_raw_data(url)
332
- if not raw_data:
333
- return queue.put(dict())
334
- data = loads(raw_data)
335
- for task in data:
336
- taskdict["activetasks_" + task["type"]] += 1
337
- return queue.put(dict(taskdict))
338
-
339
- def _get_dbs_stats(self, queue, url, stats):
340
- to_netdata = {}
341
- for db in self.dbs:
342
- raw_data = self._get_raw_data(url + '/' + db)
343
- if not raw_data:
344
- continue
345
- data = loads(raw_data)
346
- for metric in stats:
347
- value = data
348
- metrics_list = metric.split('.')
349
- try:
350
- for m in metrics_list:
351
- value = value[m]
352
- except (KeyError, TypeError) as e:
353
- self.debug('cannot process ' + metric + ' for ' + db
354
- + ": " + str(e))
355
- continue
356
- metric_name = 'db_{0}_{1}'.format(db, '_'.join(metrics_list))
357
- to_netdata[metric_name] = value
358
- return queue.put(to_netdata)
359
-
360
- def _fetch_data(self, raw_data, metrics):
361
- data = dict()
362
- for metric in metrics:
363
- value = raw_data
364
- metrics_list = metric.split('.')
365
- try:
366
- for m in metrics_list:
367
- value = value[m]
368
- except (KeyError, TypeError) as e:
369
- self.debug('cannot process ' + metric + ': ' + str(e))
370
- continue
371
- # strip off .value from end of stat
372
- if metrics_list[-1] == 'value':
373
- metrics_list = metrics_list[:-1]
374
- # sum up 3xx/4xx/5xx
375
- if metrics_list[0:2] == ['couchdb', 'httpd_status_codes'] and \
376
- int(metrics_list[2]) > 202:
377
- metrics_list[2] = '{0}xx'.format(int(metrics_list[2]) // 100)
378
- if '_'.join(metrics_list) in data:
379
- data['_'.join(metrics_list)] += value
380
- else:
381
- data['_'.join(metrics_list)] = value
382
- else:
383
- data['_'.join(metrics_list)] = value
384
- return data
385
-
386
-
387
-def get_peak_msg_queue(data):
388
- maxsize = 0
389
- queues = data['message_queues']
390
- for queue in iter(queues.values()):
391
- if isinstance(queue, dict) and 'count' in queue:
392
- value = queue['count']
393
- elif isinstance(queue, int):
394
- value = queue
395
- else:
396
- continue
397
- maxsize = max(maxsize, value)
398
- return maxsize
collectors/python.d.plugin/couchdb/couchdb.conf
deleted
-89
@@ -1,89 +0,0 @@
1
-# netdata python.d.plugin configuration for couchdb
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# By default, CouchDB only updates its stats every 10 seconds.
24
-update_every: 10
25
-
26
-# priority controls the order of charts at the netdata dashboard.
27
-# Lower numbers move the charts towards the top of the page.
28
-# If unset, the default for python.d.plugin is used.
29
-# priority: 60000
30
-
31
-# penalty indicates whether to apply penalty to update_every in case of failures.
32
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
33
-# penalty: yes
34
-
35
-# autodetection_retry sets the job re-check interval in seconds.
36
-# The job is not deleted if check fails.
37
-# Attempts to start the job are made once every autodetection_retry.
38
-# This feature is disabled by default.
39
-# autodetection_retry: 0
40
-
41
-# ----------------------------------------------------------------------
42
-# JOBS (data collection sources)
43
-#
44
-# The default JOBS share the same *name*. JOBS with the same name
45
-# are mutually exclusive. Only one of them will be allowed running at
46
-# any time. This allows autodetection to try several alternatives and
47
-# pick the one that works.
48
-#
49
-# Any number of jobs is supported.
50
-#
51
-# All python.d.plugin JOBS (for all its modules) support a set of
52
-# predefined parameters. These are:
53
-#
54
-# job_name:
55
-# name: myname # the JOB's name as it will appear at the
56
-# # dashboard (by default is the job_name)
57
-# # JOBs sharing a name are mutually exclusive
58
-# update_every: 1 # the JOB's data collection frequency
59
-# priority: 60000 # the JOB's order on the dashboard
60
-# penalty: yes # the JOB's penalty
61
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
62
-#
63
-# Additionally to the above, the couchdb plugin also supports the following:
64
-#
65
-# host: 'ipaddress' # Server ip address or hostname. Default: 127.0.0.1
66
-# port: 'port' # CouchDB port. Default: 15672
67
-# scheme: 'scheme' # http or https. Default: http
68
-# node: 'couchdb@127.0.0.1' # CouchDB node name. Same as -name vm.args argument.
69
-#
70
-# if the URL is password protected, the following are supported:
71
-#
72
-# user: 'username'
73
-# pass: 'password'
74
-#
75
-# if db-specific stats are desired, place their names in databases:
76
-# databases: 'npm-registry animaldb'
77
-#
78
-# ----------------------------------------------------------------------
79
-# AUTO-DETECTION JOBS
80
-# only one of them will run (they have the same name)
81
-#
82
-localhost:
83
- name: 'local'
84
- host: '127.0.0.1'
85
- port: '5984'
86
- node: 'couchdb@127.0.0.1'
87
- scheme: 'http'
88
-# user: 'admin'
89
-# pass: 'password'
collectors/python.d.plugin/dns_query_time/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += dns_query_time/dns_query_time.chart.py
9
-dist_pythonconfig_DATA += dns_query_time/dns_query_time.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += dns_query_time/README.md dns_query_time/Makefile.inc
13
-
collectors/python.d.plugin/dns_query_time/README.md
deleted
-29
@@ -1,29 +0,0 @@
1
-<!--
2
-title: "DNS query RTT monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/dns_query_time/README.md
4
-sidebar_label: "DNS query RTT"
5
--->
6
-
7
-# DNS query RTT monitoring with Netdata
8
-
9
-Measures DNS query round trip time.
10
-
11
-**Requirement:**
12
-
13
-- `python-dnspython` package
14
-
15
-It produces one aggregate chart or one chart per DNS server, showing the query time.
16
-
17
-## Configuration
18
-
19
-Edit the `python.d/dns_query_time.conf` configuration file using `edit-config` from the Netdata [config
20
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
21
-
22
-```bash
23
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
24
-sudo ./edit-config python.d/dns_query_time.conf
25
-```
26
-
27
----
28
-
29
-
collectors/python.d.plugin/dns_query_time/dns_query_time.chart.py
deleted
-149
@@ -1,149 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: dns_query_time netdata python.d module
3
-# Author: ilyam8
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-from random import choice
7
-from socket import getaddrinfo, gaierror
8
-from threading import Thread
9
-
10
-try:
11
- import dns.message
12
- import dns.query
13
- import dns.name
14
-
15
- DNS_PYTHON = True
16
-except ImportError:
17
- DNS_PYTHON = False
18
-
19
-try:
20
- from queue import Queue
21
-except ImportError:
22
- from Queue import Queue
23
-
24
-from bases.FrameworkServices.SimpleService import SimpleService
25
-
26
-update_every = 5
27
-
28
-
29
-class Service(SimpleService):
30
- def __init__(self, configuration=None, name=None):
31
- SimpleService.__init__(self, configuration=configuration, name=name)
32
- self.order = list()
33
- self.definitions = dict()
34
- self.timeout = self.configuration.get('response_timeout', 4)
35
- self.aggregate = self.configuration.get('aggregate', True)
36
- self.domains = self.configuration.get('domains')
37
- self.server_list = self.configuration.get('dns_servers')
38
-
39
- def check(self):
40
- if not DNS_PYTHON:
41
- self.error("'python-dnspython' package is needed to use dns_query_time.chart.py")
42
- return False
43
-
44
- self.timeout = self.timeout if isinstance(self.timeout, int) else 4
45
-
46
- if not all([self.domains, self.server_list,
47
- isinstance(self.server_list, str), isinstance(self.domains, str)]):
48
- self.error("server_list and domain_list can't be empty")
49
- return False
50
- else:
51
- self.domains, self.server_list = self.domains.split(), self.server_list.split()
52
-
53
- for ns in self.server_list:
54
- if not check_ns(ns):
55
- self.info('Bad NS: %s' % ns)
56
- self.server_list.remove(ns)
57
- if not self.server_list:
58
- return False
59
-
60
- data = self._get_data(timeout=1)
61
-
62
- down_servers = [s for s in data if data[s] == -100]
63
- for down in down_servers:
64
- down = down[3:].replace('_', '.')
65
- self.info('Removed due to non response %s' % down)
66
- self.server_list.remove(down)
67
- if not self.server_list:
68
- return False
69
-
70
- self.order, self.definitions = create_charts(aggregate=self.aggregate, server_list=self.server_list)
71
- return True
72
-
73
- def _get_data(self, timeout=None):
74
- return dns_request(self.server_list, timeout or self.timeout, self.domains)
75
-
76
-
77
-def dns_request(server_list, timeout, domains):
78
- threads = list()
79
- que = Queue()
80
- result = dict()
81
-
82
- def dns_req(ns, t, q):
83
- domain = dns.name.from_text(choice(domains))
84
- request = dns.message.make_query(domain, dns.rdatatype.A)
85
-
86
- try:
87
- resp = dns.query.udp(request, ns, timeout=t)
88
- if (resp.rcode() == dns.rcode.NOERROR and resp.answer):
89
- query_time = resp.time * 1000
90
- else:
91
- query_time = -100
92
- except dns.exception.Timeout:
93
- query_time = -100
94
- finally:
95
- q.put({'_'.join(['ns', ns.replace('.', '_')]): query_time})
96
-
97
- for server in server_list:
98
- th = Thread(target=dns_req, args=(server, timeout, que))
99
- th.start()
100
- threads.append(th)
101
-
102
- for th in threads:
103
- th.join()
104
- result.update(que.get())
105
-
106
- return result
107
-
108
-
109
-def check_ns(ns):
110
- try:
111
- return getaddrinfo(ns, 'domain')[0][4][0]
112
- except gaierror:
113
- return False
114
-
115
-
116
-def create_charts(aggregate, server_list):
117
- if aggregate:
118
- order = ['dns_group']
119
- definitions = {
120
- 'dns_group': {
121
- 'options': [None, 'DNS Response Time', 'ms', 'name servers', 'dns_query_time.response_time', 'line'],
122
- 'lines': []
123
- }
124
- }
125
- for ns in server_list:
126
- dim = [
127
- '_'.join(['ns', ns.replace('.', '_')]),
128
- ns,
129
- 'absolute',
130
- ]
131
- definitions['dns_group']['lines'].append(dim)
132
-
133
- return order, definitions
134
- else:
135
- order = [''.join(['dns_', ns.replace('.', '_')]) for ns in server_list]
136
- definitions = dict()
137
-
138
- for ns in server_list:
139
- definitions[''.join(['dns_', ns.replace('.', '_')])] = {
140
- 'options': [None, 'DNS Response Time', 'ms', ns, 'dns_query_time.response_time', 'area'],
141
- 'lines': [
142
- [
143
- '_'.join(['ns', ns.replace('.', '_')]),
144
- ns,
145
- 'absolute',
146
- ]
147
- ]
148
- }
149
- return order, definitions
collectors/python.d.plugin/dns_query_time/dns_query_time.conf
deleted
-69
@@ -1,69 +0,0 @@
1
-# netdata python.d.plugin configuration for dns_query_time
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, dns_query_time also supports the following:
63
-#
64
-# dns_servers: 'dns servers' # List of dns servers to query
65
-# domains: 'domains' # List of domains
66
-# aggregate: yes/no # Aggregate all servers in one chart or not
67
-# response_timeout: 4 # Dns query response timeout (query = -100 if response time > response_time)
68
-#
69
-# ----------------------------------------------------------------------
\ No newline at end of file
collectors/python.d.plugin/dnsdist/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += dnsdist/dnsdist.chart.py
9
-dist_pythonconfig_DATA += dnsdist/dnsdist.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += dnsdist/README.md dnsdist/Makefile.inc
13
-
collectors/python.d.plugin/dnsdist/README.md
deleted
-72
@@ -1,72 +0,0 @@
1
-<!--
2
-title: "PowerDNS dnsdist monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/dnsdist/README.md
4
-sidebar_label: "PowerDNS dnsdist"
5
--->
6
-
7
-# PowerDNS dnsdist monitoring with Netdata
8
-
9
-Collects load-balancer performance and health metrics, and draws the following charts:
10
-
11
-1. **Response latency**
12
-
13
- - latency-slow
14
- - latency100-1000
15
- - latency50-100
16
- - latency10-50
17
- - latency1-10
18
- - latency0-1
19
-
20
-2. **Cache performance**
21
-
22
- - cache-hits
23
- - cache-misses
24
-
25
-3. **ACL events**
26
-
27
- - acl-drops
28
- - rule-drop
29
- - rule-nxdomain
30
- - rule-refused
31
-
32
-4. **Noncompliant data**
33
-
34
- - empty-queries
35
- - no-policy
36
- - noncompliant-queries
37
- - noncompliant-responses
38
-
39
-5. **Queries**
40
-
41
- - queries
42
- - rdqueries
43
- - rdqueries
44
-
45
-6. **Health**
46
-
47
- - downstream-send-errors
48
- - downstream-timeouts
49
- - servfail-responses
50
- - trunc-failures
51
-
52
-## Configuration
53
-
54
-Edit the `python.d/dnsdist.conf` configuration file using `edit-config` from the Netdata [config
55
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
56
-
57
-```bash
58
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
59
-sudo ./edit-config python.d/dnsdist.conf
60
-```
61
-
62
-```yaml
63
-localhost:
64
- name : 'local'
65
- url : 'http://127.0.0.1:5053/jsonstat?command=stats'
66
- user : 'username'
67
- pass : 'password'
68
- header:
69
- X-API-Key: 'dnsdist-api-key'
70
-```
71
-
72
-
collectors/python.d.plugin/dnsdist/dnsdist.chart.py
deleted
-131
@@ -1,131 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# SPDX-License-Identifier: GPL-3.0-or-later
3
-
4
-from json import loads
5
-
6
-from bases.FrameworkServices.UrlService import UrlService
7
-
8
-ORDER = [
9
- 'queries',
10
- 'queries_dropped',
11
- 'packets_dropped',
12
- 'answers',
13
- 'backend_responses',
14
- 'backend_commerrors',
15
- 'backend_errors',
16
- 'cache',
17
- 'servercpu',
18
- 'servermem',
19
- 'query_latency',
20
- 'query_latency_avg'
21
-]
22
-
23
-CHARTS = {
24
- 'queries': {
25
- 'options': [None, 'Client queries received', 'queries/s', 'queries', 'dnsdist.queries', 'line'],
26
- 'lines': [
27
- ['queries', 'all', 'incremental'],
28
- ['rdqueries', 'recursive', 'incremental'],
29
- ['empty-queries', 'empty', 'incremental']
30
- ]
31
- },
32
- 'queries_dropped': {
33
- 'options': [None, 'Client queries dropped', 'queries/s', 'queries', 'dnsdist.queries_dropped', 'line'],
34
- 'lines': [
35
- ['rule-drop', 'rule drop', 'incremental'],
36
- ['dyn-blocked', 'dynamic block', 'incremental'],
37
- ['no-policy', 'no policy', 'incremental'],
38
- ['noncompliant-queries', 'non compliant', 'incremental']
39
- ]
40
- },
41
- 'packets_dropped': {
42
- 'options': [None, 'Packets dropped', 'packets/s', 'packets', 'dnsdist.packets_dropped', 'line'],
43
- 'lines': [
44
- ['acl-drops', 'acl', 'incremental']
45
- ]
46
- },
47
- 'answers': {
48
- 'options': [None, 'Answers statistics', 'answers/s', 'answers', 'dnsdist.answers', 'line'],
49
- 'lines': [
50
- ['self-answered', 'self answered', 'incremental'],
51
- ['rule-nxdomain', 'nxdomain', 'incremental', -1],
52
- ['rule-refused', 'refused', 'incremental', -1],
53
- ['trunc-failures', 'trunc failures', 'incremental', -1]
54
- ]
55
- },
56
- 'backend_responses': {
57
- 'options': [None, 'Backend responses', 'responses/s', 'backends', 'dnsdist.backend_responses', 'line'],
58
- 'lines': [
59
- ['responses', 'responses', 'incremental']
60
- ]
61
- },
62
- 'backend_commerrors': {
63
- 'options': [None, 'Backend Communication Errors', 'errors/s', 'backends', 'dnsdist.backend_commerrors', 'line'],
64
- 'lines': [
65
- ['downstream-send-errors', 'send errors', 'incremental']
66
- ]
67
- },
68
- 'backend_errors': {
69
- 'options': [None, 'Backend error responses', 'responses/s', 'backends', 'dnsdist.backend_errors', 'line'],
70
- 'lines': [
71
- ['downstream-timeouts', 'timeout', 'incremental'],
72
- ['servfail-responses', 'servfail', 'incremental'],
73
- ['noncompliant-responses', 'non compliant', 'incremental']
74
- ]
75
- },
76
- 'cache': {
77
- 'options': [None, 'Cache performance', 'answers/s', 'cache', 'dnsdist.cache', 'area'],
78
- 'lines': [
79
- ['cache-hits', 'hits', 'incremental'],
80
- ['cache-misses', 'misses', 'incremental', -1]
81
- ]
82
- },
83
- 'servercpu': {
84
- 'options': [None, 'DNSDIST server CPU utilization', 'ms/s', 'server', 'dnsdist.servercpu', 'stacked'],
85
- 'lines': [
86
- ['cpu-sys-msec', 'system state', 'incremental'],
87
- ['cpu-user-msec', 'user state', 'incremental']
88
- ]
89
- },
90
- 'servermem': {
91
- 'options': [None, 'DNSDIST server memory utilization', 'MiB', 'server', 'dnsdist.servermem', 'area'],
92
- 'lines': [
93
- ['real-memory-usage', 'memory usage', 'absolute', 1, 1 << 20]
94
- ]
95
- },
96
- 'query_latency': {
97
- 'options': [None, 'Query latency', 'queries/s', 'latency', 'dnsdist.query_latency', 'stacked'],
98
- 'lines': [
99
- ['latency0-1', '1ms', 'incremental'],
100
- ['latency1-10', '10ms', 'incremental'],
101
- ['latency10-50', '50ms', 'incremental'],
102
- ['latency50-100', '100ms', 'incremental'],
103
- ['latency100-1000', '1sec', 'incremental'],
104
- ['latency-slow', 'slow', 'incremental']
105
- ]
106
- },
107
- 'query_latency_avg': {
108
- 'options': [None, 'Average latency for the last N queries', 'microseconds', 'latency',
109
- 'dnsdist.query_latency_avg', 'line'],
110
- 'lines': [
111
- ['latency-avg100', '100', 'absolute'],
112
- ['latency-avg1000', '1k', 'absolute'],
113
- ['latency-avg10000', '10k', 'absolute'],
114
- ['latency-avg1000000', '1000k', 'absolute']
115
- ]
116
- }
117
-}
118
-
119
-
120
-class Service(UrlService):
121
- def __init__(self, configuration=None, name=None):
122
- UrlService.__init__(self, configuration=configuration, name=name)
123
- self.order = ORDER
124
- self.definitions = CHARTS
125
-
126
- def _get_data(self):
127
- data = self._get_raw_data()
128
- if not data:
129
- return None
130
-
131
- return loads(data)
collectors/python.d.plugin/dnsdist/dnsdist.conf
deleted
-83
@@ -1,83 +0,0 @@
1
-# netdata python.d.plugin configuration for dnsdist
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-#update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-#autodetection_retry: 1
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-#
63
-# Additionally to the above, dnsdist also supports the following:
64
-#
65
-# url: 'URL' # the URL to fetch dnsdist performance statistics
66
-# user: 'username' # username for basic auth
67
-# pass: 'password' # password for basic auth
68
-# header:
69
-# X-API-Key: 'Key' # API key
70
-#
71
-# ----------------------------------------------------------------------
72
-# AUTO-DETECTION JOBS
73
-# only one of them will run (they have the same name)
74
-
75
-# localhost:
76
-# name : 'local'
77
-# url : 'http://127.0.0.1:5053/jsonstat?command=stats'
78
-# user : 'username'
79
-# pass : 'password'
80
-# header:
81
-# X-API-Key: 'dnsdist-api-key'
82
-
83
-
collectors/python.d.plugin/elasticsearch/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += elasticsearch/elasticsearch.chart.py
9
-dist_pythonconfig_DATA += elasticsearch/elasticsearch.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += elasticsearch/README.md elasticsearch/Makefile.inc
13
-
collectors/python.d.plugin/elasticsearch/README.md
deleted
-94
@@ -1,94 +0,0 @@
1
-<!--
2
-title: "Elasticsearch monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/elasticsearch/README.md
4
-sidebar_label: "Elasticsearch"
5
--->
6
-
7
-# Elasticsearch monitoring with Netdata
8
-
9
-Monitors [Elasticsearch](https://www.elastic.co/products/elasticsearch) performance and health metrics.
10
-
11
-It produces:
12
-
13
-1. **Search performance** charts:
14
-
15
- - Number of queries, fetches
16
- - Time spent on queries, fetches
17
- - Query and fetch latency
18
-
19
-2. **Indexing performance** charts:
20
-
21
- - Number of documents indexed, index refreshes, flushes
22
- - Time spent on indexing, refreshing, flushing
23
- - Indexing and flushing latency
24
-
25
-3. **Memory usage and garbage collection** charts:
26
-
27
- - JVM heap currently in use, committed
28
- - Count of garbage collections
29
- - Time spent on garbage collections
30
-
31
-4. **Host metrics** charts:
32
-
33
- - Available file descriptors in percent
34
- - Opened HTTP connections
35
- - Cluster communication transport metrics
36
-
37
-5. **Queues and rejections** charts:
38
-
39
- - Number of queued/rejected threads in thread pool
40
-
41
-6. **Fielddata cache** charts:
42
-
43
- - Fielddata cache size
44
- - Fielddata evictions and circuit breaker tripped count
45
-
46
-7. **Cluster health API** charts:
47
-
48
- - Cluster status
49
- - Nodes and tasks statistics
50
- - Shards statistics
51
-
52
-8. **Cluster stats API** charts:
53
-
54
- - Nodes statistics
55
- - Query cache statistics
56
- - Docs statistics
57
- - Store statistics
58
- - Indices and shards statistics
59
-
60
-9. **Indices** charts (per index statistics, disabled by default):
61
-
62
- - Docs count
63
- - Store size
64
- - Num of replicas
65
- - Health status
66
-
67
-## Configuration
68
-
69
-Edit the `python.d/elasticsearch.conf` configuration file using `edit-config` from the Netdata [config
70
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
71
-
72
-```bash
73
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
74
-sudo ./edit-config python.d/elasticsearch.conf
75
-```
76
-
77
-Sample:
78
-
79
-```yaml
80
-local:
81
- host : 'ipaddress' # Elasticsearch server ip address or hostname.
82
- port : 'port' # Port on which elasticsearch listens.
83
- scheme : 'http' # URL scheme. Use 'https' if your elasticsearch uses TLS.
84
- node_status : yes/no # Get metrics from "/_nodes/_local/stats". Enabled by default.
85
- cluster_health : yes/no # Get metrics from "/_cluster/health". Enabled by default.
86
- cluster_stats : yes/no # Get metrics from "'/_cluster/stats". Enabled by default.
87
- indices_stats : yes/no # Get metrics from "/_cat/indices". Disabled by default.
88
-```
89
-
90
-If no configuration is given, module will try to connect to `http://127.0.0.1:9200`.
91
-
92
----
93
-
94
-
collectors/python.d.plugin/elasticsearch/elasticsearch.chart.py
deleted
-808
@@ -1,808 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: elastic search node stats netdata python.d module
3
-# Author: ilyam8
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import json
7
-import threading
8
-
9
-from collections import namedtuple
10
-from socket import gethostbyname, gaierror
11
-
12
-try:
13
- from queue import Queue
14
-except ImportError:
15
- from Queue import Queue
16
-
17
-from bases.FrameworkServices.UrlService import UrlService
18
-
19
-# default module values (can be overridden per job in `config`)
20
-update_every = 5
21
-
22
-METHODS = namedtuple('METHODS', ['get_data', 'url', 'run'])
23
-
24
-NODE_STATS = [
25
- 'indices.search.fetch_current',
26
- 'indices.search.fetch_total',
27
- 'indices.search.query_current',
28
- 'indices.search.query_total',
29
- 'indices.search.query_time_in_millis',
30
- 'indices.search.fetch_time_in_millis',
31
- 'indices.indexing.index_total',
32
- 'indices.indexing.index_current',
33
- 'indices.indexing.index_time_in_millis',
34
- 'indices.refresh.total',
35
- 'indices.refresh.total_time_in_millis',
36
- 'indices.flush.total',
37
- 'indices.flush.total_time_in_millis',
38
- 'indices.translog.operations',
39
- 'indices.translog.size_in_bytes',
40
- 'indices.translog.uncommitted_operations',
41
- 'indices.translog.uncommitted_size_in_bytes',
42
- 'indices.segments.count',
43
- 'indices.segments.terms_memory_in_bytes',
44
- 'indices.segments.stored_fields_memory_in_bytes',
45
- 'indices.segments.term_vectors_memory_in_bytes',
46
- 'indices.segments.norms_memory_in_bytes',
47
- 'indices.segments.points_memory_in_bytes',
48
- 'indices.segments.doc_values_memory_in_bytes',
49
- 'indices.segments.index_writer_memory_in_bytes',
50
- 'indices.segments.version_map_memory_in_bytes',
51
- 'indices.segments.fixed_bit_set_memory_in_bytes',
52
- 'jvm.gc.collectors.young.collection_count',
53
- 'jvm.gc.collectors.old.collection_count',
54
- 'jvm.gc.collectors.young.collection_time_in_millis',
55
- 'jvm.gc.collectors.old.collection_time_in_millis',
56
- 'jvm.mem.heap_used_percent',
57
- 'jvm.mem.heap_used_in_bytes',
58
- 'jvm.mem.heap_committed_in_bytes',
59
- 'jvm.buffer_pools.direct.count',
60
- 'jvm.buffer_pools.direct.used_in_bytes',
61
- 'jvm.buffer_pools.direct.total_capacity_in_bytes',
62
- 'jvm.buffer_pools.mapped.count',
63
- 'jvm.buffer_pools.mapped.used_in_bytes',
64
- 'jvm.buffer_pools.mapped.total_capacity_in_bytes',
65
- 'thread_pool.bulk.queue',
66
- 'thread_pool.bulk.rejected',
67
- 'thread_pool.write.queue',
68
- 'thread_pool.write.rejected',
69
- 'thread_pool.index.queue',
70
- 'thread_pool.index.rejected',
71
- 'thread_pool.search.queue',
72
- 'thread_pool.search.rejected',
73
- 'thread_pool.merge.queue',
74
- 'thread_pool.merge.rejected',
75
- 'indices.fielddata.memory_size_in_bytes',
76
- 'indices.fielddata.evictions',
77
- 'breakers.fielddata.tripped',
78
- 'http.current_open',
79
- 'transport.rx_size_in_bytes',
80
- 'transport.tx_size_in_bytes',
81
- 'process.max_file_descriptors',
82
- 'process.open_file_descriptors'
83
-]
84
-
85
-CLUSTER_STATS = [
86
- 'nodes.count.data',
87
- 'nodes.count.master',
88
- 'nodes.count.total',
89
- 'nodes.count.coordinating_only',
90
- 'nodes.count.ingest',
91
- 'indices.docs.count',
92
- 'indices.query_cache.hit_count',
93
- 'indices.query_cache.miss_count',
94
- 'indices.store.size_in_bytes',
95
- 'indices.count',
96
- 'indices.shards.total'
97
-]
98
-
99
-HEALTH_STATS = [
100
- 'number_of_nodes',
101
- 'number_of_data_nodes',
102
- 'number_of_pending_tasks',
103
- 'number_of_in_flight_fetch',
104
- 'active_shards',
105
- 'relocating_shards',
106
- 'unassigned_shards',
107
- 'delayed_unassigned_shards',
108
- 'initializing_shards',
109
- 'active_shards_percent_as_number'
110
-]
111
-
112
-LATENCY = {
113
- 'query_latency': {
114
- 'total': 'indices_search_query_total',
115
- 'spent_time': 'indices_search_query_time_in_millis'
116
- },
117
- 'fetch_latency': {
118
- 'total': 'indices_search_fetch_total',
119
- 'spent_time': 'indices_search_fetch_time_in_millis'
120
- },
121
- 'indexing_latency': {
122
- 'total': 'indices_indexing_index_total',
123
- 'spent_time': 'indices_indexing_index_time_in_millis'
124
- },
125
- 'flushing_latency': {
126
- 'total': 'indices_flush_total',
127
- 'spent_time': 'indices_flush_total_time_in_millis'
128
- }
129
-}
130
-
131
-# charts order (can be overridden if you want less charts, or different order)
132
-ORDER = [
133
- 'search_performance_total',
134
- 'search_performance_current',
135
- 'search_performance_time',
136
- 'search_latency',
137
- 'index_performance_total',
138
- 'index_performance_current',
139
- 'index_performance_time',
140
- 'index_latency',
141
- 'index_translog_operations',
142
- 'index_translog_size',
143
- 'index_segments_count',
144
- 'index_segments_memory_writer',
145
- 'index_segments_memory',
146
- 'jvm_mem_heap',
147
- 'jvm_mem_heap_bytes',
148
- 'jvm_buffer_pool_count',
149
- 'jvm_direct_buffers_memory',
150
- 'jvm_mapped_buffers_memory',
151
- 'jvm_gc_count',
152
- 'jvm_gc_time',
153
- 'host_metrics_file_descriptors',
154
- 'host_metrics_http',
155
- 'host_metrics_transport',
156
- 'thread_pool_queued',
157
- 'thread_pool_rejected',
158
- 'fielddata_cache',
159
- 'fielddata_evictions_tripped',
160
- 'cluster_health_status',
161
- 'cluster_health_nodes',
162
- 'cluster_health_pending_tasks',
163
- 'cluster_health_flight_fetch',
164
- 'cluster_health_shards',
165
- 'cluster_stats_nodes',
166
- 'cluster_stats_query_cache',
167
- 'cluster_stats_docs',
168
- 'cluster_stats_store',
169
- 'cluster_stats_indices',
170
- 'cluster_stats_shards_total',
171
- 'index_docs_count',
172
- 'index_store_size',
173
- 'index_replica',
174
- 'index_health',
175
-]
176
-
177
-CHARTS = {
178
- 'search_performance_total': {
179
- 'options': [None, 'Queries And Fetches', 'events/s', 'search performance',
180
- 'elastic.search_performance_total', 'stacked'],
181
- 'lines': [
182
- ['indices_search_query_total', 'queries', 'incremental'],
183
- ['indices_search_fetch_total', 'fetches', 'incremental']
184
- ]
185
- },
186
- 'search_performance_current': {
187
- 'options': [None, 'Queries and Fetches In Progress', 'events', 'search performance',
188
- 'elastic.search_performance_current', 'stacked'],
189
- 'lines': [
190
- ['indices_search_query_current', 'queries', 'absolute'],
191
- ['indices_search_fetch_current', 'fetches', 'absolute']
192
- ]
193
- },
194
- 'search_performance_time': {
195
- 'options': [None, 'Time Spent On Queries And Fetches', 'seconds', 'search performance',
196
- 'elastic.search_performance_time', 'stacked'],
197
- 'lines': [
198
- ['indices_search_query_time_in_millis', 'query', 'incremental', 1, 1000],
199
- ['indices_search_fetch_time_in_millis', 'fetch', 'incremental', 1, 1000]
200
- ]
201
- },
202
- 'search_latency': {
203
- 'options': [None, 'Query And Fetch Latency', 'milliseconds', 'search performance', 'elastic.search_latency',
204
- 'stacked'],
205
- 'lines': [
206
- ['query_latency', 'query', 'absolute', 1, 1000],
207
- ['fetch_latency', 'fetch', 'absolute', 1, 1000]
208
- ]
209
- },
210
- 'index_performance_total': {
211
- 'options': [None, 'Indexed Documents, Index Refreshes, Index Flushes To Disk', 'events/s',
212
- 'indexing performance', 'elastic.index_performance_total', 'stacked'],
213
- 'lines': [
214
- ['indices_indexing_index_total', 'indexed', 'incremental'],
215
- ['indices_refresh_total', 'refreshes', 'incremental'],
216
- ['indices_flush_total', 'flushes', 'incremental']
217
- ]
218
- },
219
- 'index_performance_current': {
220
- 'options': [None, 'Number Of Documents Currently Being Indexed', 'currently indexed',
221
- 'indexing performance', 'elastic.index_performance_current', 'stacked'],
222
- 'lines': [
223
- ['indices_indexing_index_current', 'documents', 'absolute']
224
- ]
225
- },
226
- 'index_performance_time': {
227
- 'options': [None, 'Time Spent On Indexing, Refreshing, Flushing', 'seconds', 'indexing performance',
228
- 'elastic.index_performance_time', 'stacked'],
229
- 'lines': [
230
- ['indices_indexing_index_time_in_millis', 'indexing', 'incremental', 1, 1000],
231
- ['indices_refresh_total_time_in_millis', 'refreshing', 'incremental', 1, 1000],
232
- ['indices_flush_total_time_in_millis', 'flushing', 'incremental', 1, 1000]
233
- ]
234
- },
235
- 'index_latency': {
236
- 'options': [None, 'Indexing And Flushing Latency', 'milliseconds', 'indexing performance',
237
- 'elastic.index_latency', 'stacked'],
238
- 'lines': [
239
- ['indexing_latency', 'indexing', 'absolute', 1, 1000],
240
- ['flushing_latency', 'flushing', 'absolute', 1, 1000]
241
- ]
242
- },
243
- 'index_translog_operations': {
244
- 'options': [None, 'Translog Operations', 'operations', 'translog',
245
- 'elastic.index_translog_operations', 'area'],
246
- 'lines': [
247
- ['indices_translog_operations', 'total', 'absolute'],
248
- ['indices_translog_uncommitted_operations', 'uncommitted', 'absolute']
249
- ]
250
- },
251
- 'index_translog_size': {
252
- 'options': [None, 'Translog Size', 'MiB', 'translog',
253
- 'elastic.index_translog_size', 'area'],
254
- 'lines': [
255
- ['indices_translog_size_in_bytes', 'total', 'absolute', 1, 1048567],
256
- ['indices_translog_uncommitted_size_in_bytes', 'uncommitted', 'absolute', 1, 1048567]
257
- ]
258
- },
259
- 'index_segments_count': {
260
- 'options': [None, 'Total Number Of Indices Segments', 'segments', 'indices segments',
261
- 'elastic.index_segments_count', 'line'],
262
- 'lines': [
263
- ['indices_segments_count', 'segments', 'absolute']
264
- ]
265
- },
266
- 'index_segments_memory_writer': {
267
- 'options': [None, 'Index Writer Memory Usage', 'MiB', 'indices segments',
268
- 'elastic.index_segments_memory_writer', 'area'],
269
- 'lines': [
270
- ['indices_segments_index_writer_memory_in_bytes', 'total', 'absolute', 1, 1048567]
271
- ]
272
- },
273
- 'index_segments_memory': {
274
- 'options': [None, 'Indices Segments Memory Usage', 'MiB', 'indices segments',
275
- 'elastic.index_segments_memory', 'stacked'],
276
- 'lines': [
277
- ['indices_segments_terms_memory_in_bytes', 'terms', 'absolute', 1, 1048567],
278
- ['indices_segments_stored_fields_memory_in_bytes', 'stored fields', 'absolute', 1, 1048567],
279
- ['indices_segments_term_vectors_memory_in_bytes', 'term vectors', 'absolute', 1, 1048567],
280
- ['indices_segments_norms_memory_in_bytes', 'norms', 'absolute', 1, 1048567],
281
- ['indices_segments_points_memory_in_bytes', 'points', 'absolute', 1, 1048567],
282
- ['indices_segments_doc_values_memory_in_bytes', 'doc values', 'absolute', 1, 1048567],
283
- ['indices_segments_version_map_memory_in_bytes', 'version map', 'absolute', 1, 1048567],
284
- ['indices_segments_fixed_bit_set_memory_in_bytes', 'fixed bit set', 'absolute', 1, 1048567]
285
- ]
286
- },
287
- 'jvm_mem_heap': {
288
- 'options': [None, 'JVM Heap Percentage Currently in Use', 'percentage', 'memory usage and gc',
289
- 'elastic.jvm_heap', 'area'],
290
- 'lines': [
291
- ['jvm_mem_heap_used_percent', 'inuse', 'absolute']
292
- ]
293
- },
294
- 'jvm_mem_heap_bytes': {
295
- 'options': [None, 'JVM Heap Commit And Usage', 'MiB', 'memory usage and gc',
296
- 'elastic.jvm_heap_bytes', 'area'],
297
- 'lines': [
298
- ['jvm_mem_heap_committed_in_bytes', 'committed', 'absolute', 1, 1048576],
299
- ['jvm_mem_heap_used_in_bytes', 'used', 'absolute', 1, 1048576]
300
- ]
301
- },
302
- 'jvm_buffer_pool_count': {
303
- 'options': [None, 'JVM Buffers', 'pools', 'memory usage and gc',
304
- 'elastic.jvm_buffer_pool_count', 'line'],
305
- 'lines': [
306
- ['jvm_buffer_pools_direct_count', 'direct', 'absolute'],
307
- ['jvm_buffer_pools_mapped_count', 'mapped', 'absolute']
308
- ]
309
- },
310
- 'jvm_direct_buffers_memory': {
311
- 'options': [None, 'JVM Direct Buffers Memory', 'MiB', 'memory usage and gc',
312
- 'elastic.jvm_direct_buffers_memory', 'area'],
313
- 'lines': [
314
- ['jvm_buffer_pools_direct_used_in_bytes', 'used', 'absolute', 1, 1048567],
315
- ['jvm_buffer_pools_direct_total_capacity_in_bytes', 'total capacity', 'absolute', 1, 1048567]
316
- ]
317
- },
318
- 'jvm_mapped_buffers_memory': {
319
- 'options': [None, 'JVM Mapped Buffers Memory', 'MiB', 'memory usage and gc',
320
- 'elastic.jvm_mapped_buffers_memory', 'area'],
321
- 'lines': [
322
- ['jvm_buffer_pools_mapped_used_in_bytes', 'used', 'absolute', 1, 1048567],
323
- ['jvm_buffer_pools_mapped_total_capacity_in_bytes', 'total capacity', 'absolute', 1, 1048567]
324
- ]
325
- },
326
- 'jvm_gc_count': {
327
- 'options': [None, 'Garbage Collections', 'events/s', 'memory usage and gc', 'elastic.gc_count', 'stacked'],
328
- 'lines': [
329
- ['jvm_gc_collectors_young_collection_count', 'young', 'incremental'],
330
- ['jvm_gc_collectors_old_collection_count', 'old', 'incremental']
331
- ]
332
- },
333
- 'jvm_gc_time': {
334
- 'options': [None, 'Time Spent On Garbage Collections', 'milliseconds', 'memory usage and gc',
335
- 'elastic.gc_time', 'stacked'],
336
- 'lines': [
337
- ['jvm_gc_collectors_young_collection_time_in_millis', 'young', 'incremental'],
338
- ['jvm_gc_collectors_old_collection_time_in_millis', 'old', 'incremental']
339
- ]
340
- },
341
- 'thread_pool_queued': {
342
- 'options': [None, 'Number Of Queued Threads In Thread Pool', 'queued threads', 'queues and rejections',
343
- 'elastic.thread_pool_queued', 'stacked'],
344
- 'lines': [
345
- ['thread_pool_bulk_queue', 'bulk', 'absolute'],
346
- ['thread_pool_write_queue', 'write', 'absolute'],
347
- ['thread_pool_index_queue', 'index', 'absolute'],
348
- ['thread_pool_search_queue', 'search', 'absolute'],
349
- ['thread_pool_merge_queue', 'merge', 'absolute']
350
- ]
351
- },
352
- 'thread_pool_rejected': {
353
- 'options': [None, 'Rejected Threads In Thread Pool', 'rejected threads', 'queues and rejections',
354
- 'elastic.thread_pool_rejected', 'stacked'],
355
- 'lines': [
356
- ['thread_pool_bulk_rejected', 'bulk', 'absolute'],
357
- ['thread_pool_write_rejected', 'write', 'absolute'],
358
- ['thread_pool_index_rejected', 'index', 'absolute'],
359
- ['thread_pool_search_rejected', 'search', 'absolute'],
360
- ['thread_pool_merge_rejected', 'merge', 'absolute']
361
- ]
362
- },
363
- 'fielddata_cache': {
364
- 'options': [None, 'Fielddata Cache', 'MiB', 'fielddata cache', 'elastic.fielddata_cache', 'line'],
365
- 'lines': [
366
- ['indices_fielddata_memory_size_in_bytes', 'cache', 'absolute', 1, 1048576]
367
- ]
368
- },
369
- 'fielddata_evictions_tripped': {
370
- 'options': [None, 'Fielddata Evictions And Circuit Breaker Tripped Count', 'events/s',
371
- 'fielddata cache', 'elastic.fielddata_evictions_tripped', 'line'],
372
- 'lines': [
373
- ['indices_fielddata_evictions', 'evictions', 'incremental'],
374
- ['indices_fielddata_tripped', 'tripped', 'incremental']
375
- ]
376
- },
377
- 'cluster_health_nodes': {
378
- 'options': [None, 'Nodes Statistics', 'nodes', 'cluster health API',
379
- 'elastic.cluster_health_nodes', 'area'],
380
- 'lines': [
381
- ['number_of_nodes', 'nodes', 'absolute'],
382
- ['number_of_data_nodes', 'data_nodes', 'absolute'],
383
- ]
384
- },
385
- 'cluster_health_pending_tasks': {
386
- 'options': [None, 'Tasks Statistics', 'tasks', 'cluster health API',
387
- 'elastic.cluster_health_pending_tasks', 'line'],
388
- 'lines': [
389
- ['number_of_pending_tasks', 'pending_tasks', 'absolute'],
390
- ]
391
- },
392
- 'cluster_health_flight_fetch': {
393
- 'options': [None, 'In Flight Fetches Statistics', 'fetches', 'cluster health API',
394
- 'elastic.cluster_health_flight_fetch', 'line'],
395
- 'lines': [
396
- ['number_of_in_flight_fetch', 'in_flight_fetch', 'absolute']
397
- ]
398
- },
399
- 'cluster_health_status': {
400
- 'options': [None, 'Cluster Status', 'status', 'cluster health API',
401
- 'elastic.cluster_health_status', 'area'],
402
- 'lines': [
403
- ['status_green', 'green', 'absolute'],
404
- ['status_red', 'red', 'absolute'],
405
- ['status_yellow', 'yellow', 'absolute']
406
- ]
407
- },
408
- 'cluster_health_shards': {
409
- 'options': [None, 'Shards Statistics', 'shards', 'cluster health API',
410
- 'elastic.cluster_health_shards', 'stacked'],
411
- 'lines': [
412
- ['active_shards', 'active_shards', 'absolute'],
413
- ['relocating_shards', 'relocating_shards', 'absolute'],
414
- ['unassigned_shards', 'unassigned', 'absolute'],
415
- ['delayed_unassigned_shards', 'delayed_unassigned', 'absolute'],
416
- ['initializing_shards', 'initializing', 'absolute'],
417
- ['active_shards_percent_as_number', 'active_percent', 'absolute']
418
- ]
419
- },
420
- 'cluster_stats_nodes': {
421
- 'options': [None, 'Nodes Statistics', 'nodes', 'cluster stats API',
422
- 'elastic.cluster_nodes', 'area'],
423
- 'lines': [
424
- ['nodes_count_data', 'data', 'absolute'],
425
- ['nodes_count_master', 'master', 'absolute'],
426
- ['nodes_count_total', 'total', 'absolute'],
427
- ['nodes_count_ingest', 'ingest', 'absolute'],
428
- ['nodes_count_coordinating_only', 'coordinating_only', 'absolute']
429
- ]
430
- },
431
- 'cluster_stats_query_cache': {
432
- 'options': [None, 'Query Cache Statistics', 'queries', 'cluster stats API',
433
- 'elastic.cluster_query_cache', 'stacked'],
434
- 'lines': [
435
- ['indices_query_cache_hit_count', 'hit', 'incremental'],
436
- ['indices_query_cache_miss_count', 'miss', 'incremental']
437
- ]
438
- },
439
- 'cluster_stats_docs': {
440
- 'options': [None, 'Docs Statistics', 'docs', 'cluster stats API',
441
- 'elastic.cluster_docs', 'line'],
442
- 'lines': [
443
- ['indices_docs_count', 'docs', 'absolute']
444
- ]
445
- },
446
- 'cluster_stats_store': {
447
- 'options': [None, 'Store Statistics', 'MiB', 'cluster stats API',
448
- 'elastic.cluster_store', 'line'],
449
- 'lines': [
450
- ['indices_store_size_in_bytes', 'size', 'absolute', 1, 1048567]
451
- ]
452
- },
453
- 'cluster_stats_indices': {
454
- 'options': [None, 'Indices Statistics', 'indices', 'cluster stats API',
455
- 'elastic.cluster_indices', 'line'],
456
- 'lines': [
457
- ['indices_count', 'indices', 'absolute'],
458
- ]
459
- },
460
- 'cluster_stats_shards_total': {
461
- 'options': [None, 'Total Shards Statistics', 'shards', 'cluster stats API',
462
- 'elastic.cluster_shards_total', 'line'],
463
- 'lines': [
464
- ['indices_shards_total', 'shards', 'absolute']
465
- ]
466
- },
467
- 'host_metrics_transport': {
468
- 'options': [None, 'Cluster Communication Transport Metrics', 'kilobit/s', 'host metrics',
469
- 'elastic.host_transport', 'area'],
470
- 'lines': [
471
- ['transport_rx_size_in_bytes', 'in', 'incremental', 8, 1000],
472
- ['transport_tx_size_in_bytes', 'out', 'incremental', -8, 1000]
473
- ]
474
- },
475
- 'host_metrics_file_descriptors': {
476
- 'options': [None, 'Available File Descriptors In Percent', 'percentage', 'host metrics',
477
- 'elastic.host_descriptors', 'area'],
478
- 'lines': [
479
- ['file_descriptors_used', 'used', 'absolute', 1, 10]
480
- ]
481
- },
482
- 'host_metrics_http': {
483
- 'options': [None, 'Opened HTTP Connections', 'connections', 'host metrics',
484
- 'elastic.host_http_connections', 'line'],
485
- 'lines': [
486
- ['http_current_open', 'opened', 'absolute', 1, 1]
487
- ]
488
- },
489
- 'index_docs_count': {
490
- 'options': [None, 'Docs Count', 'count', 'indices', 'elastic.index_docs', 'line'],
491
- 'lines': []
492
- },
493
- 'index_store_size': {
494
- 'options': [None, 'Store Size', 'bytes', 'indices', 'elastic.index_store_size', 'line'],
495
- 'lines': []
496
- },
497
- 'index_replica': {
498
- 'options': [None, 'Replica', 'count', 'indices', 'elastic.index_replica', 'line'],
499
- 'lines': []
500
- },
501
- 'index_health': {
502
- 'options': [None, 'Health', 'status', 'indices', 'elastic.index_health', 'line'],
503
- 'lines': []
504
- },
505
-}
506
-
507
-
508
-def convert_index_store_size_to_bytes(size):
509
- # can be b, kb, mb, gb or None
510
- if size is None:
511
- return -1
512
- if size.endswith('kb'):
513
- return round(float(size[:-2]) * 1024)
514
- elif size.endswith('mb'):
515
- return round(float(size[:-2]) * 1024 * 1024)
516
- elif size.endswith('gb'):
517
- return round(float(size[:-2]) * 1024 * 1024 * 1024)
518
- elif size.endswith('tb'):
519
- return round(float(size[:-2]) * 1024 * 1024 * 1024 * 1024)
520
- elif size.endswith('b'):
521
- return round(float(size[:-1]))
522
- return -1
523
-
524
-
525
-def convert_index_null_value(value):
526
- if value is None:
527
- return -1
528
- return value
529
-
530
-
531
-def convert_index_health(health):
532
- if health == 'green':
533
- return 0
534
- elif health == 'yellow':
535
- return 1
536
- elif health == 'read':
537
- return 2
538
- return -1
539
-
540
-
541
-def get_survive_any(method):
542
- def w(*args):
543
- try:
544
- method(*args)
545
- except Exception as error:
546
- self, queue, url = args[0], args[1], args[2]
547
- self.error("error during '{0}' : {1}".format(url, error))
548
- queue.put(dict())
549
-
550
- return w
551
-
552
-
553
-class Service(UrlService):
554
- def __init__(self, configuration=None, name=None):
555
- UrlService.__init__(self, configuration=configuration, name=name)
556
- self.order = ORDER
557
- self.definitions = CHARTS
558
- self.host = self.configuration.get('host')
559
- self.port = self.configuration.get('port', 9200)
560
- self.url = '{scheme}://{host}:{port}'.format(
561
- scheme=self.configuration.get('scheme', 'http'),
562
- host=self.host,
563
- port=self.port,
564
- )
565
- self.latency = dict()
566
- self.methods = list()
567
- self.collected_indices = set()
568
-
569
- def check(self):
570
- if not self.host:
571
- self.error('Host is not defined in the module configuration file')
572
- return False
573
-
574
- try:
575
- self.host = gethostbyname(self.host)
576
- except gaierror as error:
577
- self.error(repr(error))
578
- return False
579
-
580
- self.methods = [
581
- METHODS(
582
- get_data=self._get_node_stats,
583
- url=self.url + '/_nodes/_local/stats',
584
- run=self.configuration.get('node_stats', True),
585
- ),
586
- METHODS(
587
- get_data=self._get_cluster_health,
588
- url=self.url + '/_cluster/health',
589
- run=self.configuration.get('cluster_health', True)
590
- ),
591
- METHODS(
592
- get_data=self._get_cluster_stats,
593
- url=self.url + '/_cluster/stats',
594
- run=self.configuration.get('cluster_stats', True),
595
- ),
596
- METHODS(
597
- get_data=self._get_indices,
598
- url=self.url + '/_cat/indices?format=json',
599
- run=self.configuration.get('indices_stats', False),
600
- ),
601
- ]
602
- return UrlService.check(self)
603
-
604
- def _get_data(self):
605
- threads = list()
606
- queue = Queue()
607
- result = dict()
608
-
609
- for method in self.methods:
610
- if not method.run:
611
- continue
612
- th = threading.Thread(
613
- target=method.get_data,
614
- args=(queue, method.url),
615
- )
616
- th.daemon = True
617
- th.start()
618
- threads.append(th)
619
-
620
- for thread in threads:
621
- thread.join()
622
- result.update(queue.get())
623
-
624
- return result or None
625
-
626
- def add_index_to_charts(self, idx_name):
627
- for name in ('index_docs_count', 'index_store_size', 'index_replica', 'index_health'):
628
- chart = self.charts[name]
629
- dim = ['{0}_{1}'.format(idx_name, name), idx_name]
630
- chart.add_dimension(dim)
631
-
632
- @get_survive_any
633
- def _get_indices(self, queue, url):
634
- # [
635
- # {
636
- # "pri.store.size": "650b",
637
- # "health": "yellow",
638
- # "status": "open",
639
- # "index": "twitter",
640
- # "pri": "5",
641
- # "rep": "1",
642
- # "docs.count": "10",
643
- # "docs.deleted": "3",
644
- # "store.size": "650b"
645
- # },
646
- # {
647
- # "status":"open",
648
- # "index":".kibana_3",
649
- # "health":"red",
650
- # "uuid":"umAdNrq6QaOXrmZjAowTNw",
651
- # "store.size":null,
652
- # "pri.store.size":null,
653
- # "docs.count":null,
654
- # "rep":"0",
655
- # "pri":"1",
656
- # "docs.deleted":null
657
- # },
658
- # {
659
- # "health" : "green",
660
- # "status" : "close",
661
- # "index" : "siem-events-2021.09.12",
662
- # "uuid" : "mTQ-Yl5TS7S3lGoRORE-Pg",
663
- # "pri" : "4",
664
- # "rep" : "0",
665
- # "docs.count" : null,
666
- # "docs.deleted" : null,
667
- # "store.size" : null,
668
- # "pri.store.size" : null
669
- # }
670
- # ]
671
- raw_data = self._get_raw_data(url)
672
- if not raw_data:
673
- return queue.put(dict())
674
-
675
- indices = self.json_parse(raw_data)
676
- if not indices:
677
- return queue.put(dict())
678
-
679
- charts_initialized = len(self.charts) != 0
680
- data = dict()
681
- for idx in indices:
682
- try:
683
- name = idx['index']
684
- is_system_index = name.startswith('.')
685
- if is_system_index:
686
- continue
687
-
688
- v = {
689
- '{0}_index_replica'.format(name): idx['rep'],
690
- '{0}_index_health'.format(name): convert_index_health(idx['health']),
691
- }
692
- docs_count = convert_index_null_value(idx['docs.count'])
693
- if docs_count != -1:
694
- v['{0}_index_docs_count'.format(name)] = idx['docs.count']
695
- size = convert_index_store_size_to_bytes(idx['store.size'])
696
- if size != -1:
697
- v['{0}_index_store_size'.format(name)] = size
698
- except KeyError as error:
699
- self.debug("error on parsing index : {0}".format(repr(error)))
700
- continue
701
-
702
- data.update(v)
703
- if name not in self.collected_indices and charts_initialized:
704
- self.collected_indices.add(name)
705
- self.add_index_to_charts(name)
706
-
707
- return queue.put(data)
708
-
709
- @get_survive_any
710
- def _get_cluster_health(self, queue, url):
711
- raw = self._get_raw_data(url)
712
- if not raw:
713
- return queue.put(dict())
714
-
715
- parsed = self.json_parse(raw)
716
- if not parsed:
717
- return queue.put(dict())
718
-
719
- data = fetch_data(raw_data=parsed, metrics=HEALTH_STATS)
720
- dummy = {
721
- 'status_green': 0,
722
- 'status_red': 0,
723
- 'status_yellow': 0,
724
- }
725
- data.update(dummy)
726
- current_status = 'status_' + parsed['status']
727
- data[current_status] = 1
728
-
729
- return queue.put(data)
730
-
731
- @get_survive_any
732
- def _get_cluster_stats(self, queue, url):
733
- raw = self._get_raw_data(url)
734
- if not raw:
735
- return queue.put(dict())
736
-
737
- parsed = self.json_parse(raw)
738
- if not parsed:
739
- return queue.put(dict())
740
-
741
- data = fetch_data(raw_data=parsed, metrics=CLUSTER_STATS)
742
-
743
- return queue.put(data)
744
-
745
- @get_survive_any
746
- def _get_node_stats(self, queue, url):
747
- raw = self._get_raw_data(url)
748
- if not raw:
749
- return queue.put(dict())
750
-
751
- parsed = self.json_parse(raw)
752
- if not parsed:
753
- return queue.put(dict())
754
-
755
- node = list(parsed['nodes'].keys())[0]
756
- data = fetch_data(raw_data=parsed['nodes'][node], metrics=NODE_STATS)
757
-
758
- # Search, index, flush, fetch performance latency
759
- for key in LATENCY:
760
- try:
761
- data[key] = self.find_avg(
762
- total=data[LATENCY[key]['total']],
763
- spent_time=data[LATENCY[key]['spent_time']],
764
- key=key)
765
- except KeyError:
766
- continue
767
- if 'process_open_file_descriptors' in data and 'process_max_file_descriptors' in data:
768
- v = float(data['process_open_file_descriptors']) / data['process_max_file_descriptors'] * 1000
769
- data['file_descriptors_used'] = round(v)
770
-
771
- return queue.put(data)
772
-
773
- def json_parse(self, reply):
774
- try:
775
- return json.loads(reply)
776
- except ValueError as err:
777
- self.error(err)
778
- return None
779
-
780
- def find_avg(self, total, spent_time, key):
781
- if key not in self.latency:
782
- self.latency[key] = dict(total=total, spent_time=spent_time)
783
- return 0
784
-
785
- if self.latency[key]['total'] != total:
786
- spent_diff = spent_time - self.latency[key]['spent_time']
787
- total_diff = total - self.latency[key]['total']
788
- latency = float(spent_diff) / float(total_diff) * 1000
789
- self.latency[key]['total'] = total
790
- self.latency[key]['spent_time'] = spent_time
791
- return latency
792
-
793
- self.latency[key]['spent_time'] = spent_time
794
- return 0
795
-
796
-
797
-def fetch_data(raw_data, metrics):
798
- data = dict()
799
- for metric in metrics:
800
- value = raw_data
801
- metrics_list = metric.split('.')
802
- try:
803
- for m in metrics_list:
804
- value = value[m]
805
- except (KeyError, TypeError):
806
- continue
807
- data['_'.join(metrics_list)] = value
808
- return data
collectors/python.d.plugin/elasticsearch/elasticsearch.conf
deleted
-83
@@ -1,83 +0,0 @@
1
-# netdata python.d.plugin configuration for elasticsearch stats
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, elasticsearch plugin also supports the following:
63
-#
64
-# host : 'ipaddress' # Elasticsearch server ip address or hostname.
65
-# port : 'port' # Port on which elasticsearch listens.
66
-# node_status : yes/no # Get metrics from "/_nodes/_local/stats". Enabled by default.
67
-# cluster_health : yes/no # Get metrics from "/_cluster/health". Enabled by default.
68
-# cluster_stats : yes/no # Get metrics from "'/_cluster/stats". Enabled by default.
69
-# indices_stats : yes/no # Get metrics from "/_cat/indices". Disabled by default.
70
-#
71
-#
72
-# if the URL is password protected, the following are supported:
73
-#
74
-# user: 'username'
75
-# pass: 'password'
76
-#
77
-# ----------------------------------------------------------------------
78
-# AUTO-DETECTION JOBS
79
-# only one of them will run (they have the same name)
80
-#
81
-local:
82
- host: '127.0.0.1'
83
- port: '9200'
collectors/python.d.plugin/energid/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += energid/energid.chart.py
9
-dist_pythonconfig_DATA += energid/energid.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += energid/README.md energid/Makefile.inc
13
-
collectors/python.d.plugin/energid/README.md
deleted
-77
@@ -1,77 +0,0 @@
1
-<!--
2
-title: "Energi Core node monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/energid/README.md
4
-sidebar_label: "Energi Core"
5
--->
6
-
7
-# Energi Core node monitoring with Netdata
8
-
9
-Monitors blockchain, memory, network and unspent transactions statistics.
10
-
11
-
12
-As [Energi Core](https://github.com/energicryptocurrency/energi) Gen 1 & 2 are based on the original Bitcoin code and
13
-supports very similar JSON RPC, there is quite high chance the module works
14
-with many others forks including bitcoind itself.
15
-
16
-Introduces several new charts:
17
-
18
-1. **Blockchain Index**
19
- - blocks
20
- - headers
21
-
22
-2. **Blockchain Difficulty**
23
- - diff
24
-
25
-3. **MemPool** in MiB
26
- - Max
27
- - Usage
28
- - TX Size
29
-
30
-4. **Secure Memory** in KiB
31
- - Total
32
- - Locked
33
- - Used
34
-
35
-5. **Network**
36
- - Connections
37
-
38
-6. **UTXO** (Unspent Transaction Output)
39
- - UTXO
40
- - Xfers (related transactions)
41
-
42
-Configuration is needed in most cases of secure deployment to specify RPC
43
-credentials. However, Energi, Bitcoin and Dash daemons are checked on
44
-startup by default.
45
-
46
-It may be desired to increase retry count for production use due to possibly
47
-long daemon startup.
48
-
49
-## Configuration
50
-
51
-Edit the `python.d/energid.conf` configuration file using `edit-config` from the Netdata [config
52
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
53
-
54
-```bash
55
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
56
-sudo ./edit-config python.d/energid.conf
57
-```
58
-
59
-Sample:
60
-
61
-```yaml
62
-energi:
63
- host: '127.0.0.1'
64
- port: 9796
65
- user: energi
66
- pass: energi
67
-
68
-bitcoin:
69
- host: '127.0.0.1'
70
- port: 8332
71
- user: bitcoin
72
- pass: bitcoin
73
-```
74
-
75
----
76
-
77
-
collectors/python.d.plugin/energid/energid.chart.py
deleted
-163
@@ -1,163 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: Energi Core / Bitcoin netdata python.d module
3
-# Author: Andrey Galkin <andrey@futoin.org> (andvgal)
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-#
6
-# This module is designed for energid, but it should work with many other Bitcoin forks
7
-# which support more or less standard JSON-RPC.
8
-#
9
-
10
-import json
11
-
12
-from bases.FrameworkServices.UrlService import UrlService
13
-
14
-update_every = 5
15
-
16
-ORDER = [
17
- 'blockindex',
18
- 'difficulty',
19
- 'mempool',
20
- 'secmem',
21
- 'network',
22
- 'timeoffset',
23
- 'utxo',
24
- 'xfers',
25
-]
26
-
27
-CHARTS = {
28
- 'blockindex': {
29
- 'options': [None, 'Blockchain Index', 'count', 'blockchain', 'energi.blockindex', 'area'],
30
- 'lines': [
31
- ['blockchain_blocks', 'blocks', 'absolute'],
32
- ['blockchain_headers', 'headers', 'absolute'],
33
- ]
34
- },
35
- 'difficulty': {
36
- 'options': [None, 'Blockchain Difficulty', 'difficulty', 'blockchain', 'energi.difficulty', 'line'],
37
- 'lines': [
38
- ['blockchain_difficulty', 'Diff', 'absolute'],
39
- ],
40
- },
41
- 'mempool': {
42
- 'options': [None, 'MemPool', 'MiB', 'memory', 'energid.mempool', 'area'],
43
- 'lines': [
44
- ['mempool_max', 'Max', 'absolute', None, 1024 * 1024],
45
- ['mempool_current', 'Usage', 'absolute', None, 1024 * 1024],
46
- ['mempool_txsize', 'TX Size', 'absolute', None, 1024 * 1024],
47
- ],
48
- },
49
- 'secmem': {
50
- 'options': [None, 'Secure Memory', 'KiB', 'memory', 'energid.secmem', 'area'],
51
- 'lines': [
52
- ['secmem_total', 'Total', 'absolute', None, 1024],
53
- ['secmem_locked', 'Locked', 'absolute', None, 1024],
54
- ['secmem_used', 'Used', 'absolute', None, 1024],
55
- ],
56
- },
57
- 'network': {
58
- 'options': [None, 'Network', 'count', 'network', 'energid.network', 'line'],
59
- 'lines': [
60
- ['network_connections', 'Connections', 'absolute'],
61
- ],
62
- },
63
- 'timeoffset': {
64
- 'options': [None, 'Network', 'seconds', 'network', 'energid.timeoffset', 'line'],
65
- 'lines': [
66
- ['network_timeoffset', 'offseet', 'absolute'],
67
- ],
68
- },
69
- 'utxo': {
70
- 'options': [None, 'UTXO', 'count', 'UTXO', 'energid.utxo', 'line'],
71
- 'lines': [
72
- ['utxo_count', 'UTXO', 'absolute'],
73
- ],
74
- },
75
- 'xfers': {
76
- 'options': [None, 'UTXO', 'count', 'UTXO', 'energid.xfers', 'line'],
77
- 'lines': [
78
- ['utxo_xfers', 'Xfers', 'absolute'],
79
- ],
80
- },
81
-}
82
-
83
-METHODS = {
84
- 'getblockchaininfo': lambda r: {
85
- 'blockchain_blocks': r['blocks'],
86
- 'blockchain_headers': r['headers'],
87
- 'blockchain_difficulty': r['difficulty'],
88
- },
89
- 'getmempoolinfo': lambda r: {
90
- 'mempool_txcount': r['size'],
91
- 'mempool_txsize': r['bytes'],
92
- 'mempool_current': r['usage'],
93
- 'mempool_max': r['maxmempool'],
94
- },
95
- 'getmemoryinfo': lambda r: dict([
96
- ('secmem_' + k, v) for (k, v) in r['locked'].items()
97
- ]),
98
- 'getnetworkinfo': lambda r: {
99
- 'network_timeoffset': r['timeoffset'],
100
- 'network_connections': r['connections'],
101
- },
102
- 'gettxoutsetinfo': lambda r: {
103
- 'utxo_count': r['txouts'],
104
- 'utxo_xfers': r['transactions'],
105
- 'utxo_size': r['disk_size'],
106
- 'utxo_amount': r['total_amount'],
107
- },
108
-}
109
-
110
-JSON_RPC_VERSION = '1.1'
111
-
112
-
113
-class Service(UrlService):
114
- def __init__(self, configuration=None, name=None):
115
- UrlService.__init__(self, configuration=configuration, name=name)
116
- self.order = ORDER
117
- self.definitions = CHARTS
118
- self.host = self.configuration.get('host', '127.0.0.1')
119
- self.port = self.configuration.get('port', 9796)
120
- self.url = '{scheme}://{host}:{port}'.format(
121
- scheme=self.configuration.get('scheme', 'http'),
122
- host=self.host,
123
- port=self.port,
124
- )
125
- self.method = 'POST'
126
- self.header = {
127
- 'Content-Type': 'application/json',
128
- }
129
-
130
- def _get_data(self):
131
- #
132
- # Bitcoin family speak JSON-RPC version 1.0 for maximum compatibility,
133
- # but uses JSON-RPC 1.1/2.0 standards for parts of the 1.0 standard that were
134
- # unspecified (HTTP errors and contents of 'error').
135
- #
136
- # 1.0 spec: https://www.jsonrpc.org/specification_v1
137
- # 2.0 spec: https://www.jsonrpc.org/specification
138
- #
139
- # The call documentation: https://github.com/energicryptocurrency/core-api-documentation
140
- #
141
- batch = []
142
-
143
- for i, method in enumerate(METHODS):
144
- batch.append({
145
- 'version': JSON_RPC_VERSION,
146
- 'id': i,
147
- 'method': method,
148
- 'params': [],
149
- })
150
-
151
- result = self._get_raw_data(body=json.dumps(batch))
152
-
153
- if not result:
154
- return None
155
-
156
- result = json.loads(result.decode('utf-8'))
157
- data = dict()
158
-
159
- for i, (_, handler) in enumerate(METHODS.items()):
160
- r = result[i]
161
- data.update(handler(r['result']))
162
-
163
- return data
collectors/python.d.plugin/energid/energid.conf
deleted
-90
@@ -1,90 +0,0 @@
1
-# netdata python.d.plugin configuration for energid
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, energid also supports the following:
63
-#
64
-# host: 'IP or HOSTNAME' # type <str> the RPC host to connect to
65
-# port: PORT # type <int> the RPC port to connect to
66
-# user: 'RPC username' # type <str> the RPC username to use
67
-# pass: 'RPC password' # type <str> the RPC password to use
68
-#
69
-# ----------------------------------------------------------------------
70
-# AUTO-DETECTION JOBS
71
-# only one of them will run (they have the same name)
72
-#
73
-
74
-# Defaults:
75
-# host: '127.0.0.1'
76
-# user:
77
-# pass:
78
-#
79
-
80
-# Energi mainnet
81
-energi:
82
- port: 9796
83
-
84
-# Most likely supported Bitcoin mainnet
85
-bitcoin:
86
- port: 8332
87
-
88
-# Most likely supported Dash mainnet
89
-dash:
90
- port: 9998
collectors/python.d.plugin/freeradius/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += freeradius/freeradius.chart.py
9
-dist_pythonconfig_DATA += freeradius/freeradius.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += freeradius/README.md freeradius/Makefile.inc
13
-
collectors/python.d.plugin/freeradius/README.md
deleted
-90
@@ -1,90 +0,0 @@
1
-<!--
2
-title: "FreeRADIUS monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/freeradius/README.md
4
-sidebar_label: "FreeRADIUS"
5
--->
6
-
7
-# FreeRADIUS monitoring with Netdata
8
-
9
-Uses the `radclient` command to provide freeradius statistics. It is not recommended to run it every second.
10
-
11
-It produces:
12
-
13
-1. **Authentication counters:**
14
-
15
- - access-accepts
16
- - access-rejects
17
- - auth-dropped-requests
18
- - auth-duplicate-requests
19
- - auth-invalid-requests
20
- - auth-malformed-requests
21
- - auth-unknown-types
22
-
23
-2. **Accounting counters:** [optional]
24
-
25
- - accounting-requests
26
- - accounting-responses
27
- - acct-dropped-requests
28
- - acct-duplicate-requests
29
- - acct-invalid-requests
30
- - acct-malformed-requests
31
- - acct-unknown-types
32
-
33
-3. **Proxy authentication counters:** [optional]
34
-
35
- - proxy-access-accepts
36
- - proxy-access-rejects
37
- - proxy-auth-dropped-requests
38
- - proxy-auth-duplicate-requests
39
- - proxy-auth-invalid-requests
40
- - proxy-auth-malformed-requests
41
- - proxy-auth-unknown-types
42
-
43
-4. **Proxy accounting counters:** [optional]
44
-
45
- - proxy-accounting-requests
46
- - proxy-accounting-responses
47
- - proxy-acct-dropped-requests
48
- - proxy-acct-duplicate-requests
49
- - proxy-acct-invalid-requests
50
- - proxy-acct-malformed-requests
51
- - proxy-acct-unknown-typesa
52
-
53
-## Configuration
54
-
55
-Edit the `python.d/freeradius.conf` configuration file using `edit-config` from the Netdata [config
56
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
57
-
58
-```bash
59
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
60
-sudo ./edit-config python.d/freeradius.conf
61
-```
62
-
63
-Sample:
64
-
65
-```yaml
66
-local:
67
- host : 'localhost'
68
- port : '18121'
69
- secret : 'adminsecret'
70
- acct : False # Freeradius accounting statistics.
71
- proxy_auth : False # Freeradius proxy authentication statistics.
72
- proxy_acct : False # Freeradius proxy accounting statistics.
73
-```
74
-
75
-**Freeradius server configuration:**
76
-
77
-The configuration for the status server is automatically created in the sites-available directory.
78
-By default, server is enabled and can be queried from every client.
79
-FreeRADIUS will only respond to status-server messages, if the status-server virtual server has been enabled.
80
-
81
-To do this, create a link from the sites-enabled directory to the status file in the sites-available directory:
82
-
83
-- cd sites-enabled
84
-- ln -s ../sites-available/status status
85
-
86
-and restart/reload your FREERADIUS server.
87
-
88
----
89
-
90
-
collectors/python.d.plugin/freeradius/freeradius.chart.py
deleted
-177
@@ -1,177 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: freeradius netdata python.d module
3
-# Author: ilyam8
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import re
7
-from subprocess import Popen, PIPE
8
-
9
-from bases.FrameworkServices.SimpleService import SimpleService
10
-from bases.collection import find_binary
11
-
12
-update_every = 15
13
-
14
-PARSER = re.compile(r'((?<=-)[AP][a-zA-Z-]+) = (\d+)')
15
-
16
-RADIUS_MSG = 'Message-Authenticator = 0x00, FreeRADIUS-Statistics-Type = 15, Response-Packet-Type = Access-Accept'
17
-
18
-RADCLIENT_RETRIES = 1
19
-RADCLIENT_TIMEOUT = 1
20
-
21
-DEFAULT_HOST = 'localhost'
22
-DEFAULT_PORT = 18121
23
-DEFAULT_DO_ACCT = False
24
-DEFAULT_DO_PROXY_AUTH = False
25
-DEFAULT_DO_PROXY_ACCT = False
26
-
27
-ORDER = [
28
- 'authentication',
29
- 'accounting',
30
- 'proxy-auth',
31
- 'proxy-acct',
32
-]
33
-
34
-CHARTS = {
35
- 'authentication': {
36
- 'options': [None, 'Authentication', 'packets/s', 'authentication', 'freerad.auth', 'line'],
37
- 'lines': [
38
- ['access-accepts', None, 'incremental'],
39
- ['access-rejects', None, 'incremental'],
40
- ['auth-dropped-requests', 'dropped-requests', 'incremental'],
41
- ['auth-duplicate-requests', 'duplicate-requests', 'incremental'],
42
- ['auth-invalid-requests', 'invalid-requests', 'incremental'],
43
- ['auth-malformed-requests', 'malformed-requests', 'incremental'],
44
- ['auth-unknown-types', 'unknown-types', 'incremental']
45
- ]
46
- },
47
- 'accounting': {
48
- 'options': [None, 'Accounting', 'packets/s', 'accounting', 'freerad.acct', 'line'],
49
- 'lines': [
50
- ['accounting-requests', 'requests', 'incremental'],
51
- ['accounting-responses', 'responses', 'incremental'],
52
- ['acct-dropped-requests', 'dropped-requests', 'incremental'],
53
- ['acct-duplicate-requests', 'duplicate-requests', 'incremental'],
54
- ['acct-invalid-requests', 'invalid-requests', 'incremental'],
55
- ['acct-malformed-requests', 'malformed-requests', 'incremental'],
56
- ['acct-unknown-types', 'unknown-types', 'incremental']
57
- ]
58
- },
59
- 'proxy-auth': {
60
- 'options': [None, 'Proxy Authentication', 'packets/s', 'authentication', 'freerad.proxy.auth', 'line'],
61
- 'lines': [
62
- ['proxy-access-accepts', 'access-accepts', 'incremental'],
63
- ['proxy-access-rejects', 'access-rejects', 'incremental'],
64
- ['proxy-auth-dropped-requests', 'dropped-requests', 'incremental'],
65
- ['proxy-auth-duplicate-requests', 'duplicate-requests', 'incremental'],
66
- ['proxy-auth-invalid-requests', 'invalid-requests', 'incremental'],
67
- ['proxy-auth-malformed-requests', 'malformed-requests', 'incremental'],
68
- ['proxy-auth-unknown-types', 'unknown-types', 'incremental']
69
- ]
70
- },
71
- 'proxy-acct': {
72
- 'options': [None, 'Proxy Accounting', 'packets/s', 'accounting', 'freerad.proxy.acct', 'line'],
73
- 'lines': [
74
- ['proxy-accounting-requests', 'requests', 'incremental'],
75
- ['proxy-accounting-responses', 'responses', 'incremental'],
76
- ['proxy-acct-dropped-requests', 'dropped-requests', 'incremental'],
77
- ['proxy-acct-duplicate-requests', 'duplicate-requests', 'incremental'],
78
- ['proxy-acct-invalid-requests', 'invalid-requests', 'incremental'],
79
- ['proxy-acct-malformed-requests', 'malformed-requests', 'incremental'],
80
- ['proxy-acct-unknown-types', 'unknown-types', 'incremental']
81
- ]
82
- }
83
-}
84
-
85
-
86
-def radclient_status(radclient, retries, timeout, host, port, secret):
87
- # radclient -r 1 -t 1 -x 127.0.0.1:18121 status secret
88
-
89
- return '{radclient} -r {num_retries} -t {timeout} -x {host}:{port} status {secret}'.format(
90
- radclient=radclient,
91
- num_retries=retries,
92
- timeout=timeout,
93
- host=host,
94
- port=port,
95
- secret=secret,
96
- ).split()
97
-
98
-
99
-class Service(SimpleService):
100
- def __init__(self, configuration=None, name=None):
101
- SimpleService.__init__(self, configuration=configuration, name=name)
102
- self.order = ORDER
103
- self.definitions = CHARTS
104
- self.host = self.configuration.get('host', DEFAULT_HOST)
105
- self.port = self.configuration.get('port', DEFAULT_PORT)
106
- self.secret = self.configuration.get('secret')
107
- self.do_acct = self.configuration.get('acct', DEFAULT_DO_ACCT)
108
- self.do_proxy_auth = self.configuration.get('proxy_auth', DEFAULT_DO_PROXY_AUTH)
109
- self.do_proxy_acct = self.configuration.get('proxy_acct', DEFAULT_DO_PROXY_ACCT)
110
- self.echo = find_binary('echo')
111
- self.radclient = find_binary('radclient')
112
- self.sub_echo = [self.echo, RADIUS_MSG]
113
- self.sub_radclient = radclient_status(
114
- self.radclient, RADCLIENT_RETRIES, RADCLIENT_TIMEOUT, self.host, self.port, self.secret,
115
- )
116
-
117
- def check(self):
118
- if not self.radclient:
119
- self.error("Can't locate 'radclient' binary or binary is not executable by netdata user")
120
- return False
121
-
122
- if not self.echo:
123
- self.error("Can't locate 'echo' binary or binary is not executable by netdata user")
124
- return None
125
-
126
- if not self.secret:
127
- self.error("'secret' isn't set")
128
- return None
129
-
130
- if not self.get_raw_data():
131
- self.error('Request returned no data. Is server alive?')
132
- return False
133
-
134
- if not self.do_acct:
135
- self.order.remove('accounting')
136
-
137
- if not self.do_proxy_auth:
138
- self.order.remove('proxy-auth')
139
-
140
- if not self.do_proxy_acct:
141
- self.order.remove('proxy-acct')
142
-
143
- return True
144
-
145
- def get_data(self):
146
- """
147
- Format data received from shell command
148
- :return: dict
149
- """
150
- result = self.get_raw_data()
151
-
152
- if not result:
153
- return None
154
-
155
- return dict(
156
- (key.lower(), value) for key, value in PARSER.findall(result)
157
- )
158
-
159
- def get_raw_data(self):
160
- """
161
- The following code is equivalent to
162
- 'echo "Message-Authenticator = 0x00, FreeRADIUS-Statistics-Type = 15, Response-Packet-Type = Access-Accept"
163
- | radclient -t 1 -r 1 host:port status secret'
164
- :return: str
165
- """
166
- try:
167
- process_echo = Popen(self.sub_echo, stdout=PIPE, stderr=PIPE, shell=False)
168
- process_rad = Popen(self.sub_radclient, stdin=process_echo.stdout, stdout=PIPE, stderr=PIPE, shell=False)
169
- process_echo.stdout.close()
170
- raw_result = process_rad.communicate()[0]
171
- except OSError:
172
- return None
173
-
174
- if process_rad.returncode is 0:
175
- return raw_result.decode()
176
-
177
- return None
collectors/python.d.plugin/freeradius/freeradius.conf
deleted
-80
@@ -1,80 +0,0 @@
1
-# netdata python.d.plugin configuration for freeradius
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, freeradius also supports the following:
63
-#
64
-# host: 'host' # Default: 'localhost'. Server ip address or hostname.
65
-# port: 'port' # Default: '18121'. Port on which freeradius server listen (type = status).
66
-# secret: 'secret' # Default: 'adminsecret'.
67
-# acct: yes/no # Default: no. Freeradius accounting statistics.
68
-# proxy_auth: yes/no # Default: no. Freeradius proxy authentication statistics.
69
-# proxy_acct: yes/no # Default: no. Freeradius proxy accounting statistics.
70
-#
71
-# ------------------------------------------------------------------------------------------------------------------
72
-# Freeradius server configuration:
73
-# The configuration for the status server is automatically created in the sites-available directory.
74
-# By default, server is enabled and can be queried from every client.
75
-# FreeRADIUS will only respond to status-server messages, if the status-server virtual server has been enabled.
76
-# To do this, create a link from the sites-enabled directory to the status file in the sites-available directory:
77
-# cd sites-enabled
78
-# ln -s ../sites-available/status status
79
-# and restart/reload your FREERADIUS server.
80
-# ------------------------------------------------------------------------------------------------------------------
collectors/python.d.plugin/httpcheck/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += httpcheck/httpcheck.chart.py
9
-dist_pythonconfig_DATA += httpcheck/httpcheck.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += httpcheck/README.md httpcheck/Makefile.inc
13
-
collectors/python.d.plugin/httpcheck/README.md
deleted
-59
@@ -1,59 +0,0 @@
1
-<!--
2
-title: "HTTP endpoint monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/httpcheck/README.md
4
-sidebar_label: "HTTP endpoints"
5
--->
6
-
7
-# HTTP endpoint monitoring with Netdata
8
-
9
-Monitors remote http server for availability and response time.
10
-
11
-Following charts are drawn per job:
12
-
13
-1. **Response time** ms
14
-
15
- - Time in 0.1 ms resolution in which the server responds.
16
- If the connection failed, the value is missing.
17
-
18
-2. **Status** boolean
19
-
20
- - Connection successful
21
- - Unexpected content: No Regex match found in the response
22
- - Unexpected status code: Do we get 500 errors?
23
- - Connection failed: port not listening or blocked
24
- - Connection timed out: host or port unreachable
25
-
26
-## Configuration
27
-
28
-Edit the [`python.d/httpcheck.conf`](https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/httpcheck/httpcheck.conf) configuration file using `edit-config` from the Netdata [config
29
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
30
-
31
-```bash
32
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
33
-sudo ./edit-config python.d/httpcheck.conf
34
-```
35
-
36
-Sample configuration and their default values.
37
-
38
-```yaml
39
-server:
40
- url: 'http://host:port/path' # required
41
- status_accepted: # optional
42
- - 200
43
- timeout: 1 # optional, supports decimals (e.g. 0.2)
44
- update_every: 3 # optional
45
- regex: 'REGULAR_EXPRESSION' # optional, see https://docs.python.org/3/howto/regex.html
46
- redirect: yes # optional
47
-```
48
-
49
-### Notes
50
-
51
-- The status chart is primarily intended for alarms, badges or for access via API.
52
-- A system/service/firewall might block Netdata's access if a portscan or
53
- similar is detected.
54
-- This plugin is meant for simple use cases. Currently, the accuracy of the
55
- response time is low and should be used as reference only.
56
-
57
----
58
-
59
-
collectors/python.d.plugin/httpcheck/httpcheck.chart.py
deleted
-125
@@ -1,125 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: http check netdata python.d module
3
-# Original Author: ccremer (github.com/ccremer)
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import re
7
-
8
-import urllib3
9
-
10
-try:
11
- from time import monotonic as time
12
-except ImportError:
13
- from time import time
14
-
15
-from bases.FrameworkServices.UrlService import UrlService
16
-
17
-# default module values (can be overridden per job in `config`)
18
-update_every = 3
19
-priority = 60000
20
-
21
-# Response
22
-HTTP_RESPONSE_TIME = 'time'
23
-HTTP_RESPONSE_LENGTH = 'length'
24
-
25
-# Status dimensions
26
-HTTP_SUCCESS = 'success'
27
-HTTP_BAD_CONTENT = 'bad_content'
28
-HTTP_BAD_STATUS = 'bad_status'
29
-HTTP_TIMEOUT = 'timeout'
30
-HTTP_NO_CONNECTION = 'no_connection'
31
-
32
-ORDER = [
33
- 'response_time',
34
- 'response_length',
35
- 'status',
36
-]
37
-
38
-CHARTS = {
39
- 'response_time': {
40
- 'options': [None, 'HTTP response time', 'milliseconds', 'response', 'httpcheck.responsetime', 'line'],
41
- 'lines': [
42
- [HTTP_RESPONSE_TIME, 'time', 'absolute', 100, 1000]
43
- ]
44
- },
45
- 'response_length': {
46
- 'options': [None, 'HTTP response body length', 'characters', 'response', 'httpcheck.responselength', 'line'],
47
- 'lines': [
48
- [HTTP_RESPONSE_LENGTH, 'length', 'absolute']
49
- ]
50
- },
51
- 'status': {
52
- 'options': [None, 'HTTP status', 'boolean', 'status', 'httpcheck.status', 'line'],
53
- 'lines': [
54
- [HTTP_SUCCESS, 'success', 'absolute'],
55
- [HTTP_BAD_CONTENT, 'bad content', 'absolute'],
56
- [HTTP_BAD_STATUS, 'bad status', 'absolute'],
57
- [HTTP_TIMEOUT, 'timeout', 'absolute'],
58
- [HTTP_NO_CONNECTION, 'no connection', 'absolute']
59
- ]
60
- }
61
-}
62
-
63
-
64
-class Service(UrlService):
65
- def __init__(self, configuration=None, name=None):
66
- UrlService.__init__(self, configuration=configuration, name=name)
67
- self.order = ORDER
68
- self.definitions = CHARTS
69
- pattern = self.configuration.get('regex')
70
- self.regex = re.compile(pattern) if pattern else None
71
- self.status_codes_accepted = self.configuration.get('status_accepted', [200])
72
- self.follow_redirect = self.configuration.get('redirect', True)
73
-
74
- def _get_data(self):
75
- """
76
- Format data received from http request
77
- :return: dict
78
- """
79
- data = dict()
80
- data[HTTP_SUCCESS] = 0
81
- data[HTTP_BAD_CONTENT] = 0
82
- data[HTTP_BAD_STATUS] = 0
83
- data[HTTP_TIMEOUT] = 0
84
- data[HTTP_NO_CONNECTION] = 0
85
- url = self.url
86
- try:
87
- start = time()
88
- status, content = self._get_raw_data_with_status(retries=1 if self.follow_redirect else False,
89
- redirect=self.follow_redirect)
90
- diff = time() - start
91
- data[HTTP_RESPONSE_TIME] = max(round(diff * 10000), 0)
92
- self.debug('Url: {url}. Host responded with status code {code} in {diff} s'.format(
93
- url=url, code=status, diff=diff
94
- ))
95
- self.process_response(content, data, status)
96
-
97
- except urllib3.exceptions.NewConnectionError as error:
98
- self.debug('Connection failed: {url}. Error: {error}'.format(url=url, error=error))
99
- data[HTTP_NO_CONNECTION] = 1
100
-
101
- except (urllib3.exceptions.TimeoutError, urllib3.exceptions.PoolError) as error:
102
- self.debug('Connection timed out: {url}. Error: {error}'.format(url=url, error=error))
103
- data[HTTP_TIMEOUT] = 1
104
-
105
- except urllib3.exceptions.HTTPError as error:
106
- self.debug('Connection failed: {url}. Error: {error}'.format(url=url, error=error))
107
- data[HTTP_NO_CONNECTION] = 1
108
-
109
- except (TypeError, AttributeError) as error:
110
- self.error('Url: {url}. Error: {error}'.format(url=url, error=error))
111
- return None
112
-
113
- return data
114
-
115
- def process_response(self, content, data, status):
116
- data[HTTP_RESPONSE_LENGTH] = len(content)
117
- self.debug('Content: \n\n{content}\n'.format(content=content))
118
- if status in self.status_codes_accepted:
119
- if self.regex and self.regex.search(content) is None:
120
- self.debug('No match for regex "{regex}" found'.format(regex=self.regex.pattern))
121
- data[HTTP_BAD_CONTENT] = 1
122
- else:
123
- data[HTTP_SUCCESS] = 1
124
- else:
125
- data[HTTP_BAD_STATUS] = 1
collectors/python.d.plugin/httpcheck/httpcheck.conf
deleted
-107
@@ -1,107 +0,0 @@
1
-# netdata python.d.plugin configuration for httpcheck
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the httpcheck default is used, which is at 3 seconds.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# chart_cleanup sets the default chart cleanup interval in iterations.
35
-# A chart is marked as obsolete if it has not been updated
36
-# 'chart_cleanup' iterations in a row.
37
-# They will be hidden immediately (not offered to dashboard viewer,
38
-# streamed upstream and archived to external databases) and deleted one hour
39
-# later (configurable from netdata.conf).
40
-# -- For this plugin, cleanup MUST be disabled, otherwise we lose response
41
-# time charts
42
-chart_cleanup: 0
43
-
44
-# Autodetection and retries do not work for this plugin
45
-
46
-# ----------------------------------------------------------------------
47
-# JOBS (data collection sources)
48
-#
49
-# The default JOBS share the same *name*. JOBS with the same name
50
-# are mutually exclusive. Only one of them will be allowed running at
51
-# any time. This allows autodetection to try several alternatives and
52
-# pick the one that works.
53
-#
54
-# Any number of jobs is supported.
55
-#
56
-# -------------------------------
57
-# ATTENTION: Any valid configuration will be accepted, even if initial connection fails!
58
-# -------------------------------
59
-#
60
-# There is intentionally no default config, e.g. for 'localhost'
61
-
62
-# job_name:
63
-# name: myname # [optional] the JOB's name as it will appear at the
64
-# # dashboard (by default is the job_name)
65
-# # JOBs sharing a name are mutually exclusive
66
-# update_every: 3 # [optional] the JOB's data collection frequency
67
-# priority: 60000 # [optional] the JOB's order on the dashboard
68
-# penalty: yes # the JOB's penalty
69
-# timeout: 1 # [optional] the timeout when connecting, supports decimals (e.g. 0.5s)
70
-# url: 'http[s]://host-ip-or-dns[:port][path]'
71
-# # [required] the remote host url to connect to. If [:port] is missing, it defaults to 80
72
-# # for HTTP and 443 for HTTPS. [path] is optional too, defaults to /
73
-# header: {'Content-Type': 'application/json'}
74
-# # [optional] the HTTP header sent with the request.
75
-# method: GET # [optional] the HTTP request method (POST, PUT, DELETE, HEAD etc.)
76
-# redirect: yes # [optional] If the remote host returns 3xx status codes, the redirection url will be
77
-# # followed (default).
78
-# body: {'key': 'value'} # [optional] the body sent with the request (e.g. POST, PUT, PATCH).
79
-# status_accepted: # [optional] By default, 200 is accepted. Anything else will result in 'bad status' in the
80
-# # status chart, however: The response time will still be > 0, since the
81
-# # host responded with something.
82
-# # If redirect is enabled, the accepted status will be checked against the redirected page.
83
-# - 200 # Multiple status codes are possible. If you specify 'status_accepted', you would still
84
-# # need to add '200'. E.g. 'status_accepted: [301]' will trigger an error in 'bad status'
85
-# # if code is 200. Do specify numerical entries such as 200, not 'OK'.
86
-# regex: None # [optional] If the status code is accepted, the content of the response will be searched for this
87
-# # regex (if defined). Be aware that you may need to escape the regex string. If redirect is enabled,
88
-# # the regex will be matched to the redirected page, not the initial 3xx response.
89
-
90
-# Simple example:
91
-#
92
-# jira:
93
-# url: 'https://jira.localdomain/'
94
-
95
-
96
-# Complex example:
97
-#
98
-# cool_website:
99
-# url: 'http://cool.website:8080/home'
100
-# status_accepted:
101
-# - 200
102
-# - 204
103
-# regex: <title>My cool website!<\/title>
104
-# timeout: 2
105
-
106
-# This plugin is intended for simple cases. Currently, the accuracy of the response time is low and should be used as reference only.
107
-
collectors/python.d.plugin/isc_dhcpd/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += isc_dhcpd/isc_dhcpd.chart.py
9
-dist_pythonconfig_DATA += isc_dhcpd/isc_dhcpd.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += isc_dhcpd/README.md isc_dhcpd/Makefile.inc
13
-
collectors/python.d.plugin/isc_dhcpd/README.md
deleted
-57
@@ -1,57 +0,0 @@
1
-<!--
2
-title: "ISC DHCP monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/isc_dhcpd/README.md
4
-sidebar_label: "ISC DHCP"
5
--->
6
-
7
-# ISC DHCP monitoring with Netdata
8
-
9
-Monitors the leases database to show all active leases for given pools.
10
-
11
-## Requirements
12
-
13
-- dhcpd leases file MUST BE readable by Netdata
14
-- pools MUST BE in CIDR format
15
-- `python-ipaddress` package is needed in Python2
16
-
17
-It produces:
18
-
19
-1. **Pools utilization** Aggregate chart for all pools.
20
-
21
- - utilization in percent
22
-
23
-2. **Total leases**
24
-
25
- - leases (overall number of leases for all pools)
26
-
27
-3. **Active leases** for every pools
28
-
29
- - leases (number of active leases in pool)
30
-
31
-## Configuration
32
-
33
-Edit the `python.d/isc_dhcpd.conf` configuration file using `edit-config` from the Netdata [config
34
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
35
-
36
-```bash
37
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
38
-sudo ./edit-config python.d/isc_dhcpd.conf
39
-```
40
-
41
-Sample:
42
-
43
-```yaml
44
-local:
45
- leases_path: '/var/lib/dhcp/dhcpd.leases'
46
- pools:
47
- office: '192.168.2.0/24' # name(dimension): pool in CIDR format
48
- wifi: '192.168.3.10-192.168.3.20' # name(dimension): pool in IP Range format
49
- 192.168.4.0/24: '192.168.4.0/24' # name(dimension): pool in CIDR format
50
- wifi-guest: '192.168.5.0/24 192.168.6.10-192.168.6.20' # name(dimension): pool in CIDR + IP Range format
51
-```
52
-
53
-The module will not work If no configuration is given.
54
-
55
----
56
-
57
-
collectors/python.d.plugin/isc_dhcpd/isc_dhcpd.chart.py
deleted
-269
@@ -1,269 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: isc dhcpd lease netdata python.d module
3
-# Author: ilyam8
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import os
7
-import re
8
-import time
9
-
10
-try:
11
- import ipaddress
12
-
13
- HAVE_IP_ADDRESS = True
14
-except ImportError:
15
- HAVE_IP_ADDRESS = False
16
-
17
-from collections import defaultdict
18
-from copy import deepcopy
19
-
20
-from bases.FrameworkServices.SimpleService import SimpleService
21
-
22
-ORDER = [
23
- 'pools_utilization',
24
- 'pools_active_leases',
25
- 'leases_total',
26
-]
27
-
28
-CHARTS = {
29
- 'pools_utilization': {
30
- 'options': [None, 'Pools Utilization', 'percentage', 'utilization', 'isc_dhcpd.utilization', 'line'],
31
- 'lines': []
32
- },
33
- 'pools_active_leases': {
34
- 'options': [None, 'Active Leases Per Pool', 'leases', 'active leases', 'isc_dhcpd.active_leases', 'line'],
35
- 'lines': []
36
- },
37
- 'leases_total': {
38
- 'options': [None, 'All Active Leases', 'leases', 'active leases', 'isc_dhcpd.leases_total', 'line'],
39
- 'lines': [
40
- ['leases_total', 'leases', 'absolute']
41
- ],
42
- 'variables': [
43
- ['leases_size']
44
- ]
45
- }
46
-}
47
-
48
-POOL_CIDR = "CIDR"
49
-POOL_IP_RANGE = "IP_RANGE"
50
-POOL_UNKNOWN = "UNKNOWN"
51
-
52
-def detect_ip_type(ip):
53
- ip_type = ip.split("-")
54
- if len(ip_type) == 1:
55
- return POOL_CIDR
56
- elif len(ip_type) == 2:
57
- return POOL_IP_RANGE
58
- else:
59
- return POOL_UNKNOWN
60
-
61
-
62
-class DhcpdLeasesFile:
63
- def __init__(self, path):
64
- self.path = path
65
- self.mod_time = 0
66
- self.size = 0
67
-
68
- def is_valid(self):
69
- return os.path.isfile(self.path) and os.access(self.path, os.R_OK)
70
-
71
- def is_changed(self):
72
- mod_time = os.path.getmtime(self.path)
73
- if mod_time != self.mod_time:
74
- self.mod_time = mod_time
75
- self.size = int(os.path.getsize(self.path) / 1024)
76
- return True
77
- return False
78
-
79
- def get_data(self):
80
- try:
81
- with open(self.path) as leases:
82
- result = defaultdict(dict)
83
- for row in leases:
84
- row = row.strip()
85
- if row.startswith('lease'):
86
- address = row[6:-2]
87
- elif row.startswith('iaaddr'):
88
- address = row[7:-2]
89
- elif row.startswith('ends'):
90
- result[address]['ends'] = row[5:-1]
91
- elif row.startswith('binding state'):
92
- result[address]['state'] = row[14:-1]
93
- return dict((k, v) for k, v in result.items() if len(v) == 2)
94
- except (OSError, IOError):
95
- return None
96
-
97
-
98
-class Pool:
99
- def __init__(self, name, network):
100
- self.id = re.sub(r'[:/.-]+', '_', name)
101
- self.name = name
102
-
103
- self.networks = list()
104
- for network in network.split(" "):
105
- if not network:
106
- continue
107
-
108
- ip_type = detect_ip_type(ip=network)
109
- if ip_type == POOL_CIDR:
110
- self.networks.append(PoolCIDR(network=network))
111
- elif ip_type == POOL_IP_RANGE:
112
- self.networks.append(PoolIPRange(ip_range=network))
113
- else:
114
- raise ValueError('Network ({0}) incorrect syntax, expect CIDR or IPRange format.'.format(network))
115
-
116
- def num_hosts(self):
117
- return sum([network.num_hosts() for network in self.networks])
118
-
119
- def __contains__(self, item):
120
- for network in self.networks:
121
- if item in network:
122
- return True
123
- return False
124
-
125
-
126
-class PoolCIDR:
127
- def __init__(self, network):
128
- self.network = ipaddress.ip_network(address=u'%s' % network)
129
-
130
- def num_hosts(self):
131
- return self.network.num_addresses - 2
132
-
133
- def __contains__(self, item):
134
- return item.address in self.network
135
-
136
-
137
-class PoolIPRange:
138
- def __init__(self, ip_range):
139
- ip_range = ip_range.split("-")
140
- self.networks = list(self._summarize_address_range(ip_range[0], ip_range[1]))
141
-
142
- @staticmethod
143
- def ip_address(ip):
144
- return ipaddress.ip_address(u'%s' % ip)
145
-
146
- def _summarize_address_range(self, first, last):
147
- address_first = self.ip_address(first)
148
- address_last = self.ip_address(last)
149
- return ipaddress.summarize_address_range(address_first, address_last)
150
-
151
- def num_hosts(self):
152
- return sum([network.num_addresses for network in self.networks])
153
-
154
- def __contains__(self, item):
155
- for network in self.networks:
156
- if item.address in network:
157
- return True
158
- return False
159
-
160
-
161
-class Lease:
162
- def __init__(self, address, ends, state):
163
- self.address = ipaddress.ip_address(address=u'%s' % address)
164
- self.ends = ends
165
- self.state = state
166
-
167
- def is_active(self, current_time):
168
- # lease_end_time might be epoch
169
- if self.ends.startswith('epoch'):
170
- epoch = int(self.ends.split()[1].replace(';', ''))
171
- return epoch - current_time > 0
172
- # max. int for lease-time causes lease to expire in year 2038.
173
- # dhcpd puts 'never' in the ends section of active lease
174
- elif self.ends == 'never':
175
- return True
176
- return time.mktime(time.strptime(self.ends, '%w %Y/%m/%d %H:%M:%S')) - current_time > 0
177
-
178
- def is_valid(self):
179
- return self.state == 'active'
180
-
181
-
182
-class Service(SimpleService):
183
- def __init__(self, configuration=None, name=None):
184
- SimpleService.__init__(self, configuration=configuration, name=name)
185
- self.order = ORDER
186
- self.definitions = deepcopy(CHARTS)
187
- lease_path = self.configuration.get('leases_path', '/var/lib/dhcp/dhcpd.leases')
188
- self.dhcpd_leases = DhcpdLeasesFile(path=lease_path)
189
- self.pools = list()
190
- self.data = dict()
191
-
192
- # Will work only with 'default' db-time-format (weekday year/month/day hour:minute:second)
193
- # TODO: update algorithm to parse correctly 'local' db-time-format
194
-
195
- def check(self):
196
- if not HAVE_IP_ADDRESS:
197
- self.error("'python-ipaddress' package is needed")
198
- return False
199
-
200
- if not self.dhcpd_leases.is_valid():
201
- self.error("Make sure '{path}' is exist and readable by netdata".format(path=self.dhcpd_leases.path))
202
- return False
203
-
204
- pools = self.configuration.get('pools')
205
- if not pools:
206
- self.error('Pools are not defined')
207
- return False
208
-
209
- for pool in pools:
210
- try:
211
- new_pool = Pool(name=pool, network=pools[pool])
212
- except ValueError as error:
213
- self.error("'{pool}' was removed, error: {error}".format(pool=pools[pool], error=error))
214
- else:
215
- self.pools.append(new_pool)
216
-
217
- self.create_charts()
218
- return bool(self.pools)
219
-
220
- def get_data(self):
221
- """
222
- :return: dict
223
- """
224
- if not self.dhcpd_leases.is_changed():
225
- return self.data
226
-
227
- raw_leases = self.dhcpd_leases.get_data()
228
- if not raw_leases:
229
- self.data = dict()
230
- return None
231
-
232
- active_leases = list()
233
- current_time = time.mktime(time.gmtime())
234
-
235
- for address in raw_leases:
236
- try:
237
- new_lease = Lease(address, **raw_leases[address])
238
- except ValueError:
239
- continue
240
- else:
241
- if new_lease.is_active(current_time) and new_lease.is_valid():
242
- active_leases.append(new_lease)
243
-
244
- for pool in self.pools:
245
- count = len([ip for ip in active_leases if ip in pool])
246
- self.data[pool.id + '_active_leases'] = count
247
- self.data[pool.id + '_utilization'] = float(count) / pool.num_hosts() * 10000
248
-
249
- self.data['leases_size'] = self.dhcpd_leases.size
250
- self.data['leases_total'] = len(active_leases)
251
-
252
- return self.data
253
-
254
- def create_charts(self):
255
- for pool in self.pools:
256
- dim = [
257
- pool.id + '_utilization',
258
- pool.name,
259
- 'absolute',
260
- 1,
261
- 100,
262
- ]
263
- self.definitions['pools_utilization']['lines'].append(dim)
264
-
265
- dim = [
266
- pool.id + '_active_leases',
267
- pool.name,
268
- ]
269
- self.definitions['pools_active_leases']['lines'].append(dim)
collectors/python.d.plugin/isc_dhcpd/isc_dhcpd.conf
deleted
-80
@@ -1,80 +0,0 @@
1
-# netdata python.d.plugin configuration for isc dhcpd leases
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, isc_dhcpd supports the following:
63
-#
64
-# leases_path: 'PATH' # the path to dhcpd.leases file
65
-# pools:
66
-# office: '192.168.2.0/24' # name(dimension): pool in CIDR format
67
-# wifi: '192.168.3.10-192.168.3.20' # name(dimension): pool in IP Range format
68
-# 192.168.4.0/24: '192.168.4.0/24' # name(dimension): pool in CIDR format
69
-# wifi-guest: '192.168.5.0/24 192.168.6.10-192.168.6.20' # name(dimension): pool in CIDR + IP Range format
70
-#
71
-#-----------------------------------------------------------------------
72
-# IMPORTANT notes
73
-#
74
-# 1. Make sure leases file is readable by netdata.
75
-# 2. Current implementation works only with 'default' db-time-format
76
-# (weekday year/month/day hour:minute:second).
77
-# This is the default, so it will work in most cases.
78
-# 3. Pools MUST BE in CIDR format.
79
-#
80
-# ----------------------------------------------------------------------
collectors/python.d.plugin/mysql/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += mysql/mysql.chart.py
9
-dist_pythonconfig_DATA += mysql/mysql.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += mysql/README.md mysql/Makefile.inc
13
-
collectors/python.d.plugin/mysql/README.md
deleted
-396
@@ -1,396 +0,0 @@
1
-<!--
2
-title: "MySQL monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/mysql/README.md
4
-sidebar_label: "MySQL"
5
--->
6
-
7
-# MySQL monitoring with Netdata
8
-
9
-Monitors one or more MySQL servers.
10
-
11
-## Requirements
12
-
13
-- python library [MySQLdb](https://github.com/PyMySQL/mysqlclient-python) (faster) or [PyMySQL](https://github.com/PyMySQL/PyMySQL) (slower)
14
-- `netdata` local user to connect to the MySQL server.
15
-
16
-To create the `netdata` user, execute the following in the MySQL shell:
17
-
18
-```sh
19
-create user 'netdata'@'localhost';
20
-grant usage, replication client on *.* to 'netdata'@'localhost';
21
-flush privileges;
22
-```
23
-The `netdata` user will have the ability to connect to the MySQL server on `localhost` without a password.
24
-It will only be able to gather MySQL statistics without being able to alter or affect MySQL operations in any way.
25
-
26
-This module will produce following charts (if data is available):
27
-
28
-1. **Bandwidth** in kilobits/s
29
-
30
- - in
31
- - out
32
-
33
-2. **Queries** in queries/sec
34
-
35
- - queries
36
- - questions
37
- - slow queries
38
-
39
-3. **Queries By Type** in queries/s
40
-
41
- - select
42
- - delete
43
- - update
44
- - insert
45
- - cache hits
46
- - replace
47
-
48
-4. **Handlers** in handlers/s
49
-
50
- - commit
51
- - delete
52
- - prepare
53
- - read first
54
- - read key
55
- - read next
56
- - read prev
57
- - read rnd
58
- - read rnd next
59
- - rollback
60
- - savepoint
61
- - savepoint rollback
62
- - update
63
- - write
64
-
65
-5. **Table Locks** in locks/s
66
-
67
- - immediate
68
- - waited
69
-
70
-6. **Table Select Join Issues** in joins/s
71
-
72
- - full join
73
- - full range join
74
- - range
75
- - range check
76
- - scan
77
-
78
-7. **Table Sort Issues** in joins/s
79
-
80
- - merge passes
81
- - range
82
- - scan
83
-
84
-8. **Tmp Operations** in created/s
85
-
86
- - disk tables
87
- - files
88
- - tables
89
-
90
-9. **Connections** in connections/s
91
-
92
- - all
93
- - aborted
94
-
95
-10. **Connections Active** in connections/s
96
-
97
- - active
98
- - limit
99
- - max active
100
-
101
-11. **Binlog Cache** in threads
102
-
103
- - disk
104
- - all
105
-
106
-12. **Threads** in transactions/s
107
-
108
- - connected
109
- - cached
110
- - running
111
-
112
-13. **Threads Creation Rate** in threads/s
113
-
114
- - created
115
-
116
-14. **Threads Cache Misses** in misses
117
-
118
- - misses
119
-
120
-15. **InnoDB I/O Bandwidth** in KiB/s
121
-
122
- - read
123
- - write
124
-
125
-16. **InnoDB I/O Operations** in operations/s
126
-
127
- - reads
128
- - writes
129
- - fsyncs
130
-
131
-17. **InnoDB Pending I/O Operations** in operations/s
132
-
133
- - reads
134
- - writes
135
- - fsyncs
136
-
137
-18. **InnoDB Log Operations** in operations/s
138
-
139
- - waits
140
- - write requests
141
- - writes
142
-
143
-19. **InnoDB OS Log Pending Operations** in operations
144
-
145
- - fsyncs
146
- - writes
147
-
148
-20. **InnoDB OS Log Operations** in operations/s
149
-
150
- - fsyncs
151
-
152
-21. **InnoDB OS Log Bandwidth** in KiB/s
153
-
154
- - write
155
-
156
-22. **InnoDB Current Row Locks** in operations
157
-
158
- - current waits
159
-
160
-23. **InnoDB Row Operations** in operations/s
161
-
162
- - inserted
163
- - read
164
- - updated
165
- - deleted
166
-
167
-24. **InnoDB Buffer Pool Pages** in pages
168
-
169
- - data
170
- - dirty
171
- - free
172
- - misc
173
- - total
174
-
175
-25. **InnoDB Buffer Pool Flush Pages Requests** in requests/s
176
-
177
- - flush pages
178
-
179
-26. **InnoDB Buffer Pool Bytes** in MiB
180
-
181
- - data
182
- - dirty
183
-
184
-27. **InnoDB Buffer Pool Operations** in operations/s
185
-
186
- - disk reads
187
- - wait free
188
-
189
-28. **QCache Operations** in queries/s
190
-
191
- - hits
192
- - lowmem prunes
193
- - inserts
194
- - no caches
195
-
196
-29. **QCache Queries in Cache** in queries
197
-
198
- - queries
199
-
200
-30. **QCache Free Memory** in MiB
201
-
202
- - free
203
-
204
-31. **QCache Memory Blocks** in blocks
205
-
206
- - free
207
- - total
208
-
209
-32. **MyISAM Key Cache Blocks** in blocks
210
-
211
- - unused
212
- - used
213
- - not flushed
214
-
215
-33. **MyISAM Key Cache Requests** in requests/s
216
-
217
- - reads
218
- - writes
219
-
220
-34. **MyISAM Key Cache Requests** in requests/s
221
-
222
- - reads
223
- - writes
224
-
225
-35. **MyISAM Key Cache Disk Operations** in operations/s
226
-
227
- - reads
228
- - writes
229
-
230
-36. **Open Files** in files
231
-
232
- - files
233
-
234
-37. **Opened Files Rate** in files/s
235
-
236
- - files
237
-
238
-38. **Binlog Statement Cache** in statements/s
239
-
240
- - disk
241
- - all
242
-
243
-39. **Connection Errors** in errors/s
244
-
245
- - accept
246
- - internal
247
- - max
248
- - peer addr
249
- - select
250
- - tcpwrap
251
-
252
-40. **Slave Behind Seconds** in seconds
253
-
254
- - time
255
-
256
-41. **I/O / SQL Thread Running State** in bool
257
-
258
- - sql
259
- - io
260
-
261
-42. **Galera Replicated Writesets** in writesets/s
262
-
263
- - rx
264
- - tx
265
-
266
-43. **Galera Replicated Bytes** in KiB/s
267
-
268
- - rx
269
- - tx
270
-
271
-44. **Galera Queue** in writesets
272
-
273
- - rx
274
- - tx
275
-
276
-45. **Galera Replication Conflicts** in transactions
277
-
278
- - bf aborts
279
- - cert fails
280
-
281
-46. **Galera Flow Control** in ms
282
-
283
- - paused
284
-
285
-47. **Galera Cluster Status** in status
286
-
287
- - status
288
-
289
-48. **Galera Cluster State** in state
290
-
291
- - state
292
-
293
-49. **Galera Number of Nodes in the Cluster** in num
294
-
295
- - nodes
296
-
297
-50. **Galera Total Weight of the Current Members in the Cluster** in weight
298
-
299
- - weight
300
-
301
-51. **Galera Whether the Node is Connected to the Cluster** in boolean
302
-
303
- - connected
304
-
305
-52. **Galera Whether the Node is Ready to Accept Queries** in boolean
306
-
307
- - ready
308
-
309
-53. **Galera Open Transactions** in num
310
-
311
- - open transactions
312
-
313
-54. **Galera Total Number of WSRep (applier/rollbacker) Threads** in num
314
-
315
- - threads
316
-
317
-55. **Users CPU time** in percentage
318
-
319
- - users
320
-
321
-**Per user statistics:**
322
-
323
-1. **Rows Operations** in operations/s
324
-
325
- - read
326
- - send
327
- - updated
328
- - inserted
329
- - deleted
330
-
331
-2. **Commands** in commands/s
332
-
333
- - select
334
- - update
335
- - other
336
-
337
-## Configuration
338
-
339
-Edit the `python.d/mysql.conf` configuration file using `edit-config` from the Netdata [config
340
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
341
-
342
-```bash
343
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
344
-sudo ./edit-config python.d/mysql.conf
345
-```
346
-
347
-You can provide, per server, the following:
348
-
349
-1. username which have access to database (defaults to 'root')
350
-2. password (defaults to none)
351
-3. mysql my.cnf configuration file
352
-4. mysql socket (optional)
353
-5. mysql host (ip or hostname)
354
-6. mysql port (defaults to 3306)
355
-7. ssl connection parameters
356
-
357
- - key: the path name of the client private key file.
358
- - cert: the path name of the client public key certificate file.
359
- - ca: the path name of the Certificate Authority (CA) certificate file. This option, if used, must specify the
360
- same certificate used by the server.
361
- - capath: the path name of the directory that contains trusted SSL CA certificate files.
362
- - cipher: the list of permitted ciphers for SSL encryption.
363
-
364
-Here is an example for 3 servers:
365
-
366
-```yaml
367
-update_every : 10
368
-priority : 90100
369
-
370
-local:
371
- 'my.cnf' : '/etc/mysql/my.cnf'
372
- priority : 90000
373
-
374
-local_2:
375
- user : 'root'
376
- pass : 'blablablabla'
377
- socket : '/var/run/mysqld/mysqld.sock'
378
- update_every : 1
379
-
380
-remote:
381
- user : 'admin'
382
- pass : 'bla'
383
- host : 'example.org'
384
- port : 9000
385
-```
386
-
387
-If no configuration is given, the module will attempt to connect to MySQL server via a unix socket at
388
-`/var/run/mysqld/mysqld.sock` without password and with username `root` or `netdata` (you granted permissions for `netdata` user in the Requirements section of this document).
389
-
390
-`userstats` graph works only if you enable the plugin in MariaDB server and set proper MySQL privileges (SUPER or
391
-PROCESS). For more details, please check the [MariaDB User Statistics
392
-page](https://mariadb.com/kb/en/library/user-statistics/)
393
-
394
----
395
-
396
-
collectors/python.d.plugin/mysql/mysql.chart.py
deleted
-976
@@ -1,976 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: MySQL netdata python.d module
3
-# Author: Pawel Krupa (paulfantom)
4
-# Author: Ilya Mashchenko (ilyam8)
5
-# SPDX-License-Identifier: GPL-3.0-or-later
6
-
7
-from bases.FrameworkServices.MySQLService import MySQLService
8
-
9
-# query executed on MySQL server
10
-QUERY_GLOBAL = 'SHOW GLOBAL STATUS;'
11
-QUERY_SLAVE = 'SHOW SLAVE STATUS;'
12
-QUERY_VARIABLES = 'SHOW GLOBAL VARIABLES LIKE \'max_connections\';'
13
-QUERY_USER_STATISTICS = 'SHOW USER_STATISTICS;'
14
-
15
-GLOBAL_STATS = [
16
- 'Bytes_received',
17
- 'Bytes_sent',
18
- 'Queries',
19
- 'Questions',
20
- 'Slow_queries',
21
- 'Handler_commit',
22
- 'Handler_delete',
23
- 'Handler_prepare',
24
- 'Handler_read_first',
25
- 'Handler_read_key',
26
- 'Handler_read_next',
27
- 'Handler_read_prev',
28
- 'Handler_read_rnd',
29
- 'Handler_read_rnd_next',
30
- 'Handler_rollback',
31
- 'Handler_savepoint',
32
- 'Handler_savepoint_rollback',
33
- 'Handler_update',
34
- 'Handler_write',
35
- 'Table_locks_immediate',
36
- 'Table_locks_waited',
37
- 'Select_full_join',
38
- 'Select_full_range_join',
39
- 'Select_range',
40
- 'Select_range_check',
41
- 'Select_scan',
42
- 'Sort_merge_passes',
43
- 'Sort_range',
44
- 'Sort_scan',
45
- 'Created_tmp_disk_tables',
46
- 'Created_tmp_files',
47
- 'Created_tmp_tables',
48
- 'Connections',
49
- 'Aborted_connects',
50
- 'Max_used_connections',
51
- 'Binlog_cache_disk_use',
52
- 'Binlog_cache_use',
53
- 'Threads_connected',
54
- 'Threads_created',
55
- 'Threads_cached',
56
- 'Threads_running',
57
- 'Thread_cache_misses',
58
- 'Innodb_data_read',
59
- 'Innodb_data_written',
60
- 'Innodb_data_reads',
61
- 'Innodb_data_writes',
62
- 'Innodb_data_fsyncs',
63
- 'Innodb_data_pending_reads',
64
- 'Innodb_data_pending_writes',
65
- 'Innodb_data_pending_fsyncs',
66
- 'Innodb_log_waits',
67
- 'Innodb_log_write_requests',
68
- 'Innodb_log_writes',
69
- 'Innodb_os_log_fsyncs',
70
- 'Innodb_os_log_pending_fsyncs',
71
- 'Innodb_os_log_pending_writes',
72
- 'Innodb_os_log_written',
73
- 'Innodb_row_lock_current_waits',
74
- 'Innodb_rows_inserted',
75
- 'Innodb_rows_read',
76
- 'Innodb_rows_updated',
77
- 'Innodb_rows_deleted',
78
- 'Innodb_buffer_pool_pages_data',
79
- 'Innodb_buffer_pool_pages_dirty',
80
- 'Innodb_buffer_pool_pages_free',
81
- 'Innodb_buffer_pool_pages_flushed',
82
- 'Innodb_buffer_pool_pages_misc',
83
- 'Innodb_buffer_pool_pages_total',
84
- 'Innodb_buffer_pool_bytes_data',
85
- 'Innodb_buffer_pool_bytes_dirty',
86
- 'Innodb_buffer_pool_read_ahead',
87
- 'Innodb_buffer_pool_read_ahead_evicted',
88
- 'Innodb_buffer_pool_read_ahead_rnd',
89
- 'Innodb_buffer_pool_read_requests',
90
- 'Innodb_buffer_pool_write_requests',
91
- 'Innodb_buffer_pool_reads',
92
- 'Innodb_buffer_pool_wait_free',
93
- 'Innodb_deadlocks',
94
- 'Qcache_hits',
95
- 'Qcache_lowmem_prunes',
96
- 'Qcache_inserts',
97
- 'Qcache_not_cached',
98
- 'Qcache_queries_in_cache',
99
- 'Qcache_free_memory',
100
- 'Qcache_free_blocks',
101
- 'Qcache_total_blocks',
102
- 'Key_blocks_unused',
103
- 'Key_blocks_used',
104
- 'Key_blocks_not_flushed',
105
- 'Key_read_requests',
106
- 'Key_write_requests',
107
- 'Key_reads',
108
- 'Key_writes',
109
- 'Open_files',
110
- 'Opened_files',
111
- 'Binlog_stmt_cache_disk_use',
112
- 'Binlog_stmt_cache_use',
113
- 'Connection_errors_accept',
114
- 'Connection_errors_internal',
115
- 'Connection_errors_max_connections',
116
- 'Connection_errors_peer_address',
117
- 'Connection_errors_select',
118
- 'Connection_errors_tcpwrap',
119
- 'Com_delete',
120
- 'Com_insert',
121
- 'Com_select',
122
- 'Com_update',
123
- 'Com_replace'
124
-]
125
-
126
-GALERA_STATS = [
127
- 'wsrep_local_recv_queue',
128
- 'wsrep_local_send_queue',
129
- 'wsrep_received',
130
- 'wsrep_replicated',
131
- 'wsrep_received_bytes',
132
- 'wsrep_replicated_bytes',
133
- 'wsrep_local_bf_aborts',
134
- 'wsrep_local_cert_failures',
135
- 'wsrep_flow_control_paused_ns',
136
- 'wsrep_cluster_weight',
137
- 'wsrep_cluster_size',
138
- 'wsrep_cluster_status',
139
- 'wsrep_local_state',
140
- 'wsrep_open_transactions',
141
- 'wsrep_connected',
142
- 'wsrep_ready',
143
- 'wsrep_thread_count'
144
-]
145
-
146
-
147
-def slave_seconds(value):
148
- try:
149
- return int(value)
150
- except (TypeError, ValueError):
151
- return -1
152
-
153
-
154
-def slave_running(value):
155
- return 1 if value == 'Yes' else -1
156
-
157
-
158
-SLAVE_STATS = [
159
- ('Seconds_Behind_Master', slave_seconds),
160
- ('Slave_SQL_Running', slave_running),
161
- ('Slave_IO_Running', slave_running)
162
-]
163
-
164
-USER_STATISTICS = [
165
- 'Select_commands',
166
- 'Update_commands',
167
- 'Other_commands',
168
- 'Cpu_time',
169
- 'Rows_read',
170
- 'Rows_sent',
171
- 'Rows_deleted',
172
- 'Rows_inserted',
173
- 'Rows_updated'
174
-]
175
-
176
-VARIABLES = [
177
- 'max_connections'
178
-]
179
-
180
-ORDER = [
181
- 'net',
182
- 'queries',
183
- 'queries_type',
184
- 'handlers',
185
- 'table_locks',
186
- 'join_issues',
187
- 'sort_issues',
188
- 'tmp',
189
- 'connections',
190
- 'connections_active',
191
- 'connection_errors',
192
- 'binlog_cache',
193
- 'binlog_stmt_cache',
194
- 'threads',
195
- 'threads_creation_rate',
196
- 'thread_cache_misses',
197
- 'innodb_io',
198
- 'innodb_io_ops',
199
- 'innodb_io_pending_ops',
200
- 'innodb_log',
201
- 'innodb_os_log',
202
- 'innodb_os_log_fsync_writes',
203
- 'innodb_os_log_io',
204
- 'innodb_cur_row_lock',
205
- 'innodb_deadlocks',
206
- 'innodb_rows',
207
- 'innodb_buffer_pool_pages',
208
- 'innodb_buffer_pool_flush_pages_requests',
209
- 'innodb_buffer_pool_bytes',
210
- 'innodb_buffer_pool_read_ahead',
211
- 'innodb_buffer_pool_reqs',
212
- 'innodb_buffer_pool_ops',
213
- 'qcache_ops',
214
- 'qcache',
215
- 'qcache_freemem',
216
- 'qcache_memblocks',
217
- 'key_blocks',
218
- 'key_requests',
219
- 'key_disk_ops',
220
- 'files',
221
- 'files_rate',
222
- 'slave_behind',
223
- 'slave_status',
224
- 'galera_writesets',
225
- 'galera_bytes',
226
- 'galera_queue',
227
- 'galera_conflicts',
228
- 'galera_flow_control',
229
- 'galera_cluster_status',
230
- 'galera_cluster_state',
231
- 'galera_cluster_size',
232
- 'galera_cluster_weight',
233
- 'galera_connected',
234
- 'galera_ready',
235
- 'galera_open_transactions',
236
- 'galera_thread_count',
237
- 'userstats_cpu',
238
-]
239
-
240
-CHARTS = {
241
- 'net': {
242
- 'options': [None, 'Bandwidth', 'kilobits/s', 'bandwidth', 'mysql.net', 'area'],
243
- 'lines': [
244
- ['Bytes_received', 'in', 'incremental', 8, 1000],
245
- ['Bytes_sent', 'out', 'incremental', -8, 1000]
246
- ]
247
- },
248
- 'queries': {
249
- 'options': [None, 'Queries', 'queries/s', 'queries', 'mysql.queries', 'line'],
250
- 'lines': [
251
- ['Queries', 'queries', 'incremental'],
252
- ['Questions', 'questions', 'incremental'],
253
- ['Slow_queries', 'slow_queries', 'incremental']
254
- ]
255
- },
256
- 'queries_type': {
257
- 'options': [None, 'Query Type', 'queries/s', 'query_types', 'mysql.queries_type', 'stacked'],
258
- 'lines': [
259
- ['Com_select', 'select', 'incremental'],
260
- ['Com_delete', 'delete', 'incremental'],
261
- ['Com_update', 'update', 'incremental'],
262
- ['Com_insert', 'insert', 'incremental'],
263
- ['Qcache_hits', 'cache_hits', 'incremental'],
264
- ['Com_replace', 'replace', 'incremental']
265
- ]
266
- },
267
- 'handlers': {
268
- 'options': [None, 'Handlers', 'handlers/s', 'handlers', 'mysql.handlers', 'line'],
269
- 'lines': [
270
- ['Handler_commit', 'commit', 'incremental'],
271
- ['Handler_delete', 'delete', 'incremental'],
272
- ['Handler_prepare', 'prepare', 'incremental'],
273
- ['Handler_read_first', 'read_first', 'incremental'],
274
- ['Handler_read_key', 'read_key', 'incremental'],
275
- ['Handler_read_next', 'read_next', 'incremental'],
276
- ['Handler_read_prev', 'read_prev', 'incremental'],
277
- ['Handler_read_rnd', 'read_rnd', 'incremental'],
278
- ['Handler_read_rnd_next', 'read_rnd_next', 'incremental'],
279
- ['Handler_rollback', 'rollback', 'incremental'],
280
- ['Handler_savepoint', 'savepoint', 'incremental'],
281
- ['Handler_savepoint_rollback', 'savepoint_rollback', 'incremental'],
282
- ['Handler_update', 'update', 'incremental'],
283
- ['Handler_write', 'write', 'incremental']
284
- ]
285
- },
286
- 'table_locks': {
287
- 'options': [None, 'Tables Locks', 'locks/s', 'locks', 'mysql.table_locks', 'line'],
288
- 'lines': [
289
- ['Table_locks_immediate', 'immediate', 'incremental'],
290
- ['Table_locks_waited', 'waited', 'incremental', -1, 1]
291
- ]
292
- },
293
- 'join_issues': {
294
- 'options': [None, 'Select Join Issues', 'joins/s', 'issues', 'mysql.join_issues', 'line'],
295
- 'lines': [
296
- ['Select_full_join', 'full_join', 'incremental'],
297
- ['Select_full_range_join', 'full_range_join', 'incremental'],
298
- ['Select_range', 'range', 'incremental'],
299
- ['Select_range_check', 'range_check', 'incremental'],
300
- ['Select_scan', 'scan', 'incremental']
301
- ]
302
- },
303
- 'sort_issues': {
304
- 'options': [None, 'Sort Issues', 'issues/s', 'issues', 'mysql.sort_issues', 'line'],
305
- 'lines': [
306
- ['Sort_merge_passes', 'merge_passes', 'incremental'],
307
- ['Sort_range', 'range', 'incremental'],
308
- ['Sort_scan', 'scan', 'incremental']
309
- ]
310
- },
311
- 'tmp': {
312
- 'options': [None, 'Tmp Operations', 'counter', 'temporaries', 'mysql.tmp', 'line'],
313
- 'lines': [
314
- ['Created_tmp_disk_tables', 'disk_tables', 'incremental'],
315
- ['Created_tmp_files', 'files', 'incremental'],
316
- ['Created_tmp_tables', 'tables', 'incremental']
317
- ]
318
- },
319
- 'connections': {
320
- 'options': [None, 'Connections', 'connections/s', 'connections', 'mysql.connections', 'line'],
321
- 'lines': [
322
- ['Connections', 'all', 'incremental'],
323
- ['Aborted_connects', 'aborted', 'incremental']
324
- ]
325
- },
326
- 'connections_active': {
327
- 'options': [None, 'Connections Active', 'connections', 'connections', 'mysql.connections_active', 'line'],
328
- 'lines': [
329
- ['Threads_connected', 'active', 'absolute'],
330
- ['max_connections', 'limit', 'absolute'],
331
- ['Max_used_connections', 'max_active', 'absolute']
332
- ]
333
- },
334
- 'binlog_cache': {
335
- 'options': [None, 'Binlog Cache', 'transactions/s', 'binlog', 'mysql.binlog_cache', 'line'],
336
- 'lines': [
337
- ['Binlog_cache_disk_use', 'disk', 'incremental'],
338
- ['Binlog_cache_use', 'all', 'incremental']
339
- ]
340
- },
341
- 'threads': {
342
- 'options': [None, 'Threads', 'threads', 'threads', 'mysql.threads', 'line'],
343
- 'lines': [
344
- ['Threads_connected', 'connected', 'absolute'],
345
- ['Threads_cached', 'cached', 'absolute', -1, 1],
346
- ['Threads_running', 'running', 'absolute'],
347
- ]
348
- },
349
- 'threads_creation_rate': {
350
- 'options': [None, 'Threads Creation Rate', 'threads/s', 'threads', 'mysql.threads_creation_rate', 'line'],
351
- 'lines': [
352
- ['Threads_created', 'created', 'incremental'],
353
- ]
354
- },
355
- 'thread_cache_misses': {
356
- 'options': [None, 'mysql Threads Cache Misses', 'misses', 'threads', 'mysql.thread_cache_misses', 'area'],
357
- 'lines': [
358
- ['Thread_cache_misses', 'misses', 'absolute', 1, 100]
359
- ]
360
- },
361
- 'innodb_io': {
362
- 'options': [None, 'InnoDB I/O Bandwidth', 'KiB/s', 'innodb', 'mysql.innodb_io', 'area'],
363
- 'lines': [
364
- ['Innodb_data_read', 'read', 'incremental', 1, 1024],
365
- ['Innodb_data_written', 'write', 'incremental', -1, 1024]
366
- ]
367
- },
368
- 'innodb_io_ops': {
369
- 'options': [None, 'InnoDB I/O Operations', 'operations/s', 'innodb', 'mysql.innodb_io_ops', 'line'],
370
- 'lines': [
371
- ['Innodb_data_reads', 'reads', 'incremental'],
372
- ['Innodb_data_writes', 'writes', 'incremental', -1, 1],
373
- ['Innodb_data_fsyncs', 'fsyncs', 'incremental']
374
- ]
375
- },
376
- 'innodb_io_pending_ops': {
377
- 'options': [None, 'InnoDB Pending I/O Operations', 'operations', 'innodb',
378
- 'mysql.innodb_io_pending_ops', 'line'],
379
- 'lines': [
380
- ['Innodb_data_pending_reads', 'reads', 'absolute'],
381
- ['Innodb_data_pending_writes', 'writes', 'absolute', -1, 1],
382
- ['Innodb_data_pending_fsyncs', 'fsyncs', 'absolute']
383
- ]
384
- },
385
- 'innodb_log': {
386
- 'options': [None, 'InnoDB Log Operations', 'operations/s', 'innodb', 'mysql.innodb_log', 'line'],
387
- 'lines': [
388
- ['Innodb_log_waits', 'waits', 'incremental'],
389
- ['Innodb_log_write_requests', 'write_requests', 'incremental', -1, 1],
390
- ['Innodb_log_writes', 'writes', 'incremental', -1, 1],
391
- ]
392
- },
393
- 'innodb_os_log': {
394
- 'options': [None, 'InnoDB OS Log Pending Operations', 'operations', 'innodb', 'mysql.innodb_os_log', 'line'],
395
- 'lines': [
396
- ['Innodb_os_log_pending_fsyncs', 'fsyncs', 'absolute'],
397
- ['Innodb_os_log_pending_writes', 'writes', 'absolute', -1, 1],
398
- ]
399
- },
400
- 'innodb_os_log_fsync_writes': {
401
- 'options': [None, 'InnoDB OS Log Operations', 'operations/s', 'innodb', 'mysql.innodb_os_log_fsyncs', 'line'],
402
- 'lines': [
403
- ['Innodb_os_log_fsyncs', 'fsyncs', 'incremental'],
404
- ]
405
- },
406
- 'innodb_os_log_io': {
407
- 'options': [None, 'InnoDB OS Log Bandwidth', 'KiB/s', 'innodb', 'mysql.innodb_os_log_io', 'area'],
408
- 'lines': [
409
- ['Innodb_os_log_written', 'write', 'incremental', -1, 1024],
410
- ]
411
- },
412
- 'innodb_cur_row_lock': {
413
- 'options': [None, 'InnoDB Current Row Locks', 'operations', 'innodb',
414
- 'mysql.innodb_cur_row_lock', 'area'],
415
- 'lines': [
416
- ['Innodb_row_lock_current_waits', 'current_waits', 'absolute']
417
- ]
418
- },
419
- 'innodb_deadlocks': {
420
- 'options': [None, 'InnoDB Deadlocks', 'operations/s', 'innodb',
421
- 'mysql.innodb_deadlocks', 'area'],
422
- 'lines': [
423
- ['Innodb_deadlocks', 'deadlocks', 'incremental']
424
- ]
425
- },
426
- 'innodb_rows': {
427
- 'options': [None, 'InnoDB Row Operations', 'operations/s', 'innodb', 'mysql.innodb_rows', 'area'],
428
- 'lines': [
429
- ['Innodb_rows_inserted', 'inserted', 'incremental'],
430
- ['Innodb_rows_read', 'read', 'incremental', 1, 1],
431
- ['Innodb_rows_updated', 'updated', 'incremental', 1, 1],
432
- ['Innodb_rows_deleted', 'deleted', 'incremental', -1, 1],
433
- ]
434
- },
435
- 'innodb_buffer_pool_pages': {
436
- 'options': [None, 'InnoDB Buffer Pool Pages', 'pages', 'innodb',
437
- 'mysql.innodb_buffer_pool_pages', 'line'],
438
- 'lines': [
439
- ['Innodb_buffer_pool_pages_data', 'data', 'absolute'],
440
- ['Innodb_buffer_pool_pages_dirty', 'dirty', 'absolute', -1, 1],
441
- ['Innodb_buffer_pool_pages_free', 'free', 'absolute'],
442
- ['Innodb_buffer_pool_pages_misc', 'misc', 'absolute', -1, 1],
443
- ['Innodb_buffer_pool_pages_total', 'total', 'absolute']
444
- ]
445
- },
446
- 'innodb_buffer_pool_flush_pages_requests': {
447
- 'options': [None, 'InnoDB Buffer Pool Flush Pages Requests', 'requests/s', 'innodb',
448
- 'mysql.innodb_buffer_pool_pages_flushed', 'line'],
449
- 'lines': [
450
- ['Innodb_buffer_pool_pages_flushed', 'flush pages', 'incremental'],
451
- ]
452
- },
453
- 'innodb_buffer_pool_bytes': {
454
- 'options': [None, 'InnoDB Buffer Pool Bytes', 'MiB', 'innodb', 'mysql.innodb_buffer_pool_bytes', 'area'],
455
- 'lines': [
456
- ['Innodb_buffer_pool_bytes_data', 'data', 'absolute', 1, 1024 * 1024],
457
- ['Innodb_buffer_pool_bytes_dirty', 'dirty', 'absolute', -1, 1024 * 1024]
458
- ]
459
- },
460
- 'innodb_buffer_pool_read_ahead': {
461
- 'options': [None, 'mysql InnoDB Buffer Pool Read Ahead', 'operations/s', 'innodb',
462
- 'mysql.innodb_buffer_pool_read_ahead', 'area'],
463
- 'lines': [
464
- ['Innodb_buffer_pool_read_ahead', 'all', 'incremental'],
465
- ['Innodb_buffer_pool_read_ahead_evicted', 'evicted', 'incremental', -1, 1],
466
- ['Innodb_buffer_pool_read_ahead_rnd', 'random', 'incremental']
467
- ]
468
- },
469
- 'innodb_buffer_pool_reqs': {
470
- 'options': [None, 'InnoDB Buffer Pool Requests', 'requests/s', 'innodb',
471
- 'mysql.innodb_buffer_pool_reqs', 'area'],
472
- 'lines': [
473
- ['Innodb_buffer_pool_read_requests', 'reads', 'incremental'],
474
- ['Innodb_buffer_pool_write_requests', 'writes', 'incremental', -1, 1]
475
- ]
476
- },
477
- 'innodb_buffer_pool_ops': {
478
- 'options': [None, 'InnoDB Buffer Pool Operations', 'operations/s', 'innodb',
479
- 'mysql.innodb_buffer_pool_ops', 'area'],
480
- 'lines': [
481
- ['Innodb_buffer_pool_reads', 'disk reads', 'incremental'],
482
- ['Innodb_buffer_pool_wait_free', 'wait free', 'incremental', -1, 1]
483
- ]
484
- },
485
- 'qcache_ops': {
486
- 'options': [None, 'QCache Operations', 'queries/s', 'qcache', 'mysql.qcache_ops', 'line'],
487
- 'lines': [
488
- ['Qcache_hits', 'hits', 'incremental'],
489
- ['Qcache_lowmem_prunes', 'lowmem prunes', 'incremental', -1, 1],
490
- ['Qcache_inserts', 'inserts', 'incremental'],
491
- ['Qcache_not_cached', 'not cached', 'incremental', -1, 1]
492
- ]
493
- },
494
- 'qcache': {
495
- 'options': [None, 'QCache Queries in Cache', 'queries', 'qcache', 'mysql.qcache', 'line'],
496
- 'lines': [
497
- ['Qcache_queries_in_cache', 'queries', 'absolute']
498
- ]
499
- },
500
- 'qcache_freemem': {
501
- 'options': [None, 'QCache Free Memory', 'MiB', 'qcache', 'mysql.qcache_freemem', 'area'],
502
- 'lines': [
503
- ['Qcache_free_memory', 'free', 'absolute', 1, 1024 * 1024]
504
- ]
505
- },
506
- 'qcache_memblocks': {
507
- 'options': [None, 'QCache Memory Blocks', 'blocks', 'qcache', 'mysql.qcache_memblocks', 'line'],
508
- 'lines': [
509
- ['Qcache_free_blocks', 'free', 'absolute'],
510
- ['Qcache_total_blocks', 'total', 'absolute']
511
- ]
512
- },
513
- 'key_blocks': {
514
- 'options': [None, 'MyISAM Key Cache Blocks', 'blocks', 'myisam', 'mysql.key_blocks', 'line'],
515
- 'lines': [
516
- ['Key_blocks_unused', 'unused', 'absolute'],
517
- ['Key_blocks_used', 'used', 'absolute', -1, 1],
518
- ['Key_blocks_not_flushed', 'not flushed', 'absolute']
519
- ]
520
- },
521
- 'key_requests': {
522
- 'options': [None, 'MyISAM Key Cache Requests', 'requests/s', 'myisam', 'mysql.key_requests', 'area'],
523
- 'lines': [
524
- ['Key_read_requests', 'reads', 'incremental'],
525
- ['Key_write_requests', 'writes', 'incremental', -1, 1]
526
- ]
527
- },
528
- 'key_disk_ops': {
529
- 'options': [None, 'MyISAM Key Cache Disk Operations', 'operations/s',
530
- 'myisam', 'mysql.key_disk_ops', 'area'],
531
- 'lines': [
532
- ['Key_reads', 'reads', 'incremental'],
533
- ['Key_writes', 'writes', 'incremental', -1, 1]
534
- ]
535
- },
536
- 'files': {
537
- 'options': [None, 'Open Files', 'files', 'files', 'mysql.files', 'line'],
538
- 'lines': [
539
- ['Open_files', 'files', 'absolute']
540
- ]
541
- },
542
- 'files_rate': {
543
- 'options': [None, 'Opened Files Rate', 'files/s', 'files', 'mysql.files_rate', 'line'],
544
- 'lines': [
545
- ['Opened_files', 'files', 'incremental']
546
- ]
547
- },
548
- 'binlog_stmt_cache': {
549
- 'options': [None, 'Binlog Statement Cache', 'statements/s', 'binlog',
550
- 'mysql.binlog_stmt_cache', 'line'],
551
- 'lines': [
552
- ['Binlog_stmt_cache_disk_use', 'disk', 'incremental'],
553
- ['Binlog_stmt_cache_use', 'all', 'incremental']
554
- ]
555
- },
556
- 'connection_errors': {
557
- 'options': [None, 'Connection Errors', 'connections/s', 'connections',
558
- 'mysql.connection_errors', 'line'],
559
- 'lines': [
560
- ['Connection_errors_accept', 'accept', 'incremental'],
561
- ['Connection_errors_internal', 'internal', 'incremental'],
562
- ['Connection_errors_max_connections', 'max', 'incremental'],
563
- ['Connection_errors_peer_address', 'peer_addr', 'incremental'],
564
- ['Connection_errors_select', 'select', 'incremental'],
565
- ['Connection_errors_tcpwrap', 'tcpwrap', 'incremental']
566
- ]
567
- },
568
- 'slave_behind': {
569
- 'options': [None, 'Slave Behind Seconds', 'seconds', 'slave', 'mysql.slave_behind', 'line'],
570
- 'lines': [
571
- ['Seconds_Behind_Master', 'seconds', 'absolute']
572
- ]
573
- },
574
- 'slave_status': {
575
- 'options': [None, 'Slave Status', 'status', 'slave', 'mysql.slave_status', 'line'],
576
- 'lines': [
577
- ['Slave_SQL_Running', 'sql_running', 'absolute'],
578
- ['Slave_IO_Running', 'io_running', 'absolute']
579
- ]
580
- },
581
- 'galera_writesets': {
582
- 'options': [None, 'Replicated Writesets', 'writesets/s', 'galera', 'mysql.galera_writesets', 'line'],
583
- 'lines': [
584
- ['wsrep_received', 'rx', 'incremental'],
585
- ['wsrep_replicated', 'tx', 'incremental', -1, 1],
586
- ]
587
- },
588
- 'galera_bytes': {
589
- 'options': [None, 'Replicated Bytes', 'KiB/s', 'galera', 'mysql.galera_bytes', 'area'],
590
- 'lines': [
591
- ['wsrep_received_bytes', 'rx', 'incremental', 1, 1024],
592
- ['wsrep_replicated_bytes', 'tx', 'incremental', -1, 1024],
593
- ]
594
- },
595
- 'galera_queue': {
596
- 'options': [None, 'Galera Queue', 'writesets', 'galera', 'mysql.galera_queue', 'line'],
597
- 'lines': [
598
- ['wsrep_local_recv_queue', 'rx', 'absolute'],
599
- ['wsrep_local_send_queue', 'tx', 'absolute', -1, 1],
600
- ]
601
- },
602
- 'galera_conflicts': {
603
- 'options': [None, 'Replication Conflicts', 'transactions', 'galera', 'mysql.galera_conflicts', 'area'],
604
- 'lines': [
605
- ['wsrep_local_bf_aborts', 'bf_aborts', 'incremental'],
606
- ['wsrep_local_cert_failures', 'cert_fails', 'incremental', -1, 1],
607
- ]
608
- },
609
- 'galera_flow_control': {
610
- 'options': [None, 'Flow Control', 'millisec', 'galera', 'mysql.galera_flow_control', 'area'],
611
- 'lines': [
612
- ['wsrep_flow_control_paused_ns', 'paused', 'incremental', 1, 1000000],
613
- ]
614
- },
615
- 'galera_cluster_status': {
616
- 'options': [None, 'Cluster Component Status', 'status', 'galera', 'mysql.galera_cluster_status', 'line'],
617
- 'lines': [
618
- ['wsrep_cluster_status', 'status', 'absolute'],
619
- ]
620
- },
621
- 'galera_cluster_state': {
622
- 'options': [None, 'Cluster Component State', 'state', 'galera', 'mysql.galera_cluster_state', 'line'],
623
- 'lines': [
624
- ['wsrep_local_state', 'state', 'absolute'],
625
- ]
626
- },
627
- 'galera_cluster_size': {
628
- 'options': [None, 'Number of Nodes in the Cluster', 'num', 'galera', 'mysql.galera_cluster_size', 'line'],
629
- 'lines': [
630
- ['wsrep_cluster_size', 'nodes', 'absolute'],
631
- ]
632
- },
633
- 'galera_cluster_weight': {
634
- 'options': [None, 'The Total Weight of the Current Members in the Cluster', 'weight', 'galera',
635
- 'mysql.galera_cluster_weight', 'line'],
636
- 'lines': [
637
- ['wsrep_cluster_weight', 'weight', 'absolute'],
638
- ]
639
- },
640
- 'galera_connected': {
641
- 'options': [None, 'Whether the Node is Connected to the Cluster', 'boolean', 'galera',
642
- 'mysql.galera_connected', 'line'],
643
- 'lines': [
644
- ['wsrep_connected', 'connected', 'absolute'],
645
- ]
646
- },
647
- 'galera_ready': {
648
- 'options': [None, 'Whether the Node is Ready to Accept Queries', 'boolean', 'galera',
649
- 'mysql.galera_ready', 'line'],
650
- 'lines': [
651
- ['wsrep_ready', 'ready', 'absolute'],
652
- ]
653
- },
654
- 'galera_open_transactions': {
655
- 'options': [None, 'Open Transactions', 'num', 'galera', 'mysql.galera_open_transactions', 'line'],
656
- 'lines': [
657
- ['wsrep_open_transactions', 'open transactions', 'absolute'],
658
- ]
659
- },
660
- 'galera_thread_count': {
661
- 'options': [None, 'Total Number of WSRep (applier/rollbacker) Threads', 'num', 'galera',
662
- 'mysql.galera_thread_count', 'line'],
663
- 'lines': [
664
- ['wsrep_thread_count', 'threads', 'absolute'],
665
- ]
666
- },
667
- 'userstats_cpu': {
668
- 'options': [None, 'Users CPU time', 'percentage', 'userstats', 'mysql.userstats_cpu', 'stacked'],
669
- 'lines': []
670
- }
671
-}
672
-
673
-
674
-def slave_status_chart_template(channel_name):
675
- order = [
676
- 'slave_behind_{0}'.format(channel_name),
677
- 'slave_status_{0}'.format(channel_name)
678
- ]
679
-
680
- charts = {
681
- order[0]: {
682
- 'options': [None, 'Slave Behind Seconds Channel {0}'.format(channel_name),
683
- 'seconds', 'slave', 'mysql.slave_behind', 'line'],
684
- 'lines': [
685
- ['Seconds_Behind_Master_{0}'.format(channel_name), 'seconds', 'absolute']
686
- ]
687
- },
688
- order[1]: {
689
- 'options': [None, 'Slave Status Channel {0}'.format(channel_name),
690
- 'status', 'slave', 'mysql.slave_status', 'line'],
691
- 'lines': [
692
- ['Slave_SQL_Running_{0}'.format(channel_name), 'sql_running', 'absolute'],
693
- ['Slave_IO_Running_{0}'.format(channel_name), 'io_running', 'absolute']
694
- ]
695
- },
696
- }
697
-
698
- return order, charts
699
-
700
-
701
-def userstats_chart_template(name):
702
- order = [
703
- 'userstats_rows_{0}'.format(name),
704
- 'userstats_commands_{0}'.format(name)
705
- ]
706
- family = 'userstats {0}'.format(name)
707
-
708
- charts = {
709
- order[0]: {
710
- 'options': [None, 'Rows Operations', 'operations/s', family, 'mysql.userstats_rows', 'stacked'],
711
- 'lines': [
712
- ['userstats_{0}_Rows_read'.format(name), 'read', 'incremental'],
713
- ['userstats_{0}_Rows_send'.format(name), 'send', 'incremental'],
714
- ['userstats_{0}_Rows_updated'.format(name), 'updated', 'incremental'],
715
- ['userstats_{0}_Rows_inserted'.format(name), 'inserted', 'incremental'],
716
- ['userstats_{0}_Rows_deleted'.format(name), 'deleted', 'incremental']
717
- ]
718
- },
719
- order[1]: {
720
- 'options': [None, 'Commands', 'commands/s', family, 'mysql.userstats_commands', 'stacked'],
721
- 'lines': [
722
- ['userstats_{0}_Select_commands'.format(name), 'select', 'incremental'],
723
- ['userstats_{0}_Update_commands'.format(name), 'update', 'incremental'],
724
- ['userstats_{0}_Other_commands'.format(name), 'other', 'incremental']
725
- ]
726
- }
727
- }
728
-
729
- return order, charts
730
-
731
-
732
-# https://dev.mysql.com/doc/refman/8.0/en/replication-channels.html
733
-DEFAULT_REPL_CHANNEL = ''
734
-
735
-
736
-# Write Set REPlication
737
-# https://galeracluster.com/library/documentation/galera-status-variables.html
738
-# https://www.percona.com/doc/percona-xtradb-cluster/LATEST/wsrep-status-index.html
739
-class WSRepDataConverter:
740
- unknown_value = -1
741
-
742
- def convert(self, key, value):
743
- if key == 'wsrep_connected':
744
- return self.convert_connected(value)
745
- elif key == 'wsrep_ready':
746
- return self.convert_ready(value)
747
- elif key == 'wsrep_cluster_status':
748
- return self.convert_cluster_status(value)
749
- return value
750
-
751
- def convert_connected(self, value):
752
- # https://www.percona.com/doc/percona-xtradb-cluster/LATEST/wsrep-status-index.html#wsrep_connected
753
- if value == 'OFF':
754
- return 0
755
- if value == 'ON':
756
- return 1
757
- return self.unknown_value
758
-
759
- def convert_ready(self, value):
760
- # https://www.percona.com/doc/percona-xtradb-cluster/LATEST/wsrep-status-index.html#wsrep_ready
761
- if value == 'OFF':
762
- return 0
763
- if value == 'ON':
764
- return 1
765
- return self.unknown_value
766
-
767
- def convert_cluster_status(self, value):
768
- # https://www.percona.com/doc/percona-xtradb-cluster/LATEST/wsrep-status-index.html#wsrep_cluster_status
769
- # https://github.com/codership/wsrep-API/blob/eab2d5d5a31672c0b7d116ef1629ff18392fd7d0/wsrep_api.h
770
- # typedef enum wsrep_view_status {
771
- # WSREP_VIEW_PRIMARY, //!< primary group configuration (quorum present)
772
- # WSREP_VIEW_NON_PRIMARY, //!< non-primary group configuration (quorum lost)
773
- # WSREP_VIEW_DISCONNECTED, //!< not connected to group, retrying.
774
- # WSREP_VIEW_MAX
775
- # } wsrep_view_status_t;
776
- value = value.lower()
777
- if value == 'primary':
778
- return 0
779
- elif value == 'non-primary':
780
- return 1
781
- elif value == 'disconnected':
782
- return 2
783
- return self.unknown_value
784
-
785
-
786
-wsrep_converter = WSRepDataConverter()
787
-
788
-
789
-class Service(MySQLService):
790
- def __init__(self, configuration=None, name=None):
791
- MySQLService.__init__(self, configuration=configuration, name=name)
792
- self.order = ORDER
793
- self.definitions = CHARTS
794
- self.queries = dict(
795
- global_status=QUERY_GLOBAL,
796
- slave_status=QUERY_SLAVE,
797
- variables=QUERY_VARIABLES,
798
- user_statistics=QUERY_USER_STATISTICS,
799
- )
800
- self.repl_channels = [DEFAULT_REPL_CHANNEL]
801
-
802
- def _get_data(self):
803
-
804
- raw_data = self._get_raw_data(description=True)
805
-
806
- if not raw_data:
807
- return None
808
-
809
- data = dict()
810
-
811
- if 'global_status' in raw_data:
812
- global_status = self.get_global_status(raw_data['global_status'])
813
- if global_status:
814
- data.update(global_status)
815
-
816
- if 'slave_status' in raw_data:
817
- status = self.get_slave_status(raw_data['slave_status'])
818
- if status:
819
- data.update(status)
820
-
821
- if 'user_statistics' in raw_data:
822
- if raw_data['user_statistics'][0]:
823
- data.update(self.get_userstats(raw_data))
824
- else:
825
- self.queries.pop('user_statistics')
826
-
827
- if 'variables' in raw_data:
828
- variables = dict(raw_data['variables'][0])
829
- for key in VARIABLES:
830
- if key in variables:
831
- data[key] = variables[key]
832
-
833
- return data or None
834
-
835
- @staticmethod
836
- def convert_wsrep(key, value):
837
- return wsrep_converter.convert(key, value)
838
-
839
- def get_global_status(self, raw_global_status):
840
- # (
841
- # (
842
- # ('Aborted_clients', '18'),
843
- # ('Aborted_connects', '33'),
844
- # ('Access_denied_errors', '80'),
845
- # ('Acl_column_grants', '0'),
846
- # ('Acl_database_grants', '0'),
847
- # ('Acl_function_grants', '0'),
848
- # ('wsrep_ready', 'OFF'),
849
- # ('wsrep_rollbacker_thread_count', '0'),
850
- # ('wsrep_thread_count', '0')
851
- # ),
852
- # (
853
- # ('Variable_name', 253, 60, 64, 64, 0, 0),
854
- # ('Value', 253, 48, 2048, 2048, 0, 0),
855
- # )
856
- # )
857
- rows = raw_global_status[0]
858
- if not rows:
859
- return
860
-
861
- global_status = dict(rows)
862
- data = dict()
863
-
864
- for key in GLOBAL_STATS:
865
- if key not in global_status:
866
- continue
867
- value = global_status[key]
868
- data[key] = value
869
-
870
- for key in GALERA_STATS:
871
- if key not in global_status:
872
- continue
873
- value = global_status[key]
874
- value = self.convert_wsrep(key, value)
875
- data[key] = value
876
-
877
- if 'Threads_created' in data and 'Connections' in data:
878
- data['Thread_cache_misses'] = round(int(data['Threads_created']) / float(data['Connections']) * 10000)
879
- return data
880
-
881
- def get_slave_status(self, slave_status_data):
882
- rows, description = slave_status_data[0], slave_status_data[1]
883
- description_keys = [v[0] for v in description]
884
- if not rows:
885
- return
886
-
887
- data = dict()
888
- for row in rows:
889
- slave_data = dict(zip(description_keys, row))
890
- channel_name = slave_data.get('Channel_Name', DEFAULT_REPL_CHANNEL)
891
-
892
- if channel_name not in self.repl_channels and len(self.charts) > 0:
893
- self.add_repl_channel_charts(channel_name)
894
- self.repl_channels.append(channel_name)
895
-
896
- for key, func in SLAVE_STATS:
897
- if key not in slave_data:
898
- continue
899
-
900
- value = slave_data[key]
901
- if channel_name:
902
- key = '{0}_{1}'.format(key, channel_name)
903
- data[key] = func(value)
904
-
905
- return data
906
-
907
- def add_repl_channel_charts(self, name):
908
- self.add_new_charts(slave_status_chart_template, name)
909
-
910
- def get_userstats(self, raw_data):
911
- # (
912
- # (
913
- # ('netdata', 1L, 0L, 60L, 0.15842499999999984, 0.15767439999999996, 5206L, 963957L, 0L, 0L,
914
- # 61L, 0L, 0L, 0L, 0L, 0L, 62L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L),
915
- # ),
916
- # (
917
- # ('User', 253, 7, 128, 128, 0, 0),
918
- # ('Total_connections', 3, 2, 11, 11, 0, 0),
919
- # ('Concurrent_connections', 3, 1, 11, 11, 0, 0),
920
- # ('Connected_time', 3, 2, 11, 11, 0, 0),
921
- # ('Busy_time', 5, 20, 21, 21, 31, 0),
922
- # ('Cpu_time', 5, 20, 21, 21, 31, 0),
923
- # ('Bytes_received', 8, 4, 21, 21, 0, 0),
924
- # ('Bytes_sent', 8, 6, 21, 21, 0, 0),
925
- # ('Binlog_bytes_written', 8, 1, 21, 21, 0, 0),
926
- # ('Rows_read', 8, 1, 21, 21, 0, 0),
927
- # ('Rows_sent', 8, 2, 21, 21, 0, 0),
928
- # ('Rows_deleted', 8, 1, 21, 21, 0, 0),
929
- # ('Rows_inserted', 8, 1, 21, 21, 0, 0),
930
- # ('Rows_updated', 8, 1, 21, 21, 0, 0),
931
- # ('Select_commands', 8, 2, 21, 21, 0, 0),
932
- # ('Update_commands', 8, 1, 21, 21, 0, 0),
933
- # ('Other_commands', 8, 2, 21, 21, 0, 0),
934
- # ('Commit_transactions', 8, 1, 21, 21, 0, 0),
935
- # ('Rollback_transactions', 8, 1, 21, 21, 0, 0),
936
- # ('Denied_connections', 8, 1, 21, 21, 0, 0),
937
- # ('Lost_connections', 8, 1, 21, 21, 0, 0),
938
- # ('Access_denied', 8, 1, 21, 21, 0, 0),
939
- # ('Empty_queries', 8, 2, 21, 21, 0, 0),
940
- # ('Total_ssl_connections', 8, 1, 21, 21, 0, 0),
941
- # ('Max_statement_time_exceeded', 8, 1, 21, 21, 0, 0)
942
- # )
943
- # )
944
- data = dict()
945
- userstats_vars = [e[0] for e in raw_data['user_statistics'][1]]
946
- for i, _ in enumerate(raw_data['user_statistics'][0]):
947
- user_name = raw_data['user_statistics'][0][i][0]
948
- userstats = dict(zip(userstats_vars, raw_data['user_statistics'][0][i]))
949
-
950
- if len(self.charts) > 0:
951
- if ('userstats_{0}_Cpu_time'.format(user_name)) not in self.charts['userstats_cpu']:
952
- self.add_userstats_dimensions(user_name)
953
- self.create_new_userstats_charts(user_name)
954
-
955
- for key in USER_STATISTICS:
956
- if key in userstats:
957
- data['userstats_{0}_{1}'.format(user_name, key)] = userstats[key]
958
-
959
- return data
960
-
961
- def add_userstats_dimensions(self, name):
962
- self.charts['userstats_cpu'].add_dimension(['userstats_{0}_Cpu_time'.format(name), name, 'incremental', 100, 1])
963
-
964
- def create_new_userstats_charts(self, tube):
965
- self.add_new_charts(userstats_chart_template, tube)
966
-
967
- def add_new_charts(self, template, *params):
968
- order, charts = template(*params)
969
-
970
- for chart_name in order:
971
- params = [chart_name] + charts[chart_name]['options']
972
- dimensions = charts[chart_name]['lines']
973
-
974
- new_chart = self.charts.add_chart(params)
975
- for dimension in dimensions:
976
- new_chart.add_dimension(dimension)
collectors/python.d.plugin/mysql/mysql.conf
deleted
-293
@@ -1,293 +0,0 @@
1
-# netdata python.d.plugin configuration for mysql
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-
35
-# autodetection_retry sets the job re-check interval in seconds.
36
-# The job is not deleted if check fails.
37
-# Attempts to start the job are made once every autodetection_retry.
38
-# This feature is disabled by default.
39
-# autodetection_retry: 0
40
-
41
-# ----------------------------------------------------------------------
42
-# JOBS (data collection sources)
43
-#
44
-# The default JOBS share the same *name*. JOBS with the same name
45
-# are mutually exclusive. Only one of them will be allowed running at
46
-# any time. This allows autodetection to try several alternatives and
47
-# pick the one that works.
48
-#
49
-# Any number of jobs is supported.
50
-#
51
-# All python.d.plugin JOBS (for all its modules) support a set of
52
-# predefined parameters. These are:
53
-#
54
-# job_name:
55
-# name: myname # the JOB's name as it will appear at the
56
-# # dashboard (by default is the job_name)
57
-# # JOBs sharing a name are mutually exclusive
58
-# update_every: 1 # the JOB's data collection frequency
59
-# priority: 60000 # the JOB's order on the dashboard
60
-# penalty: yes # the JOB's penalty
61
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
62
-#
63
-# Additionally to the above, mysql also supports the following:
64
-#
65
-# socket: 'path/to/mysql.sock'
66
-#
67
-# or
68
-# host: 'IP or HOSTNAME' # the host to connect to
69
-# port: PORT # the port to connect to
70
-#
71
-# in all cases, the following can also be set:
72
-#
73
-# user: 'username' # the mysql username to use
74
-# pass: 'password' # the mysql password to use
75
-#
76
-# ssl connection parameters
77
-#
78
-# ssl:
79
-# key: 'key' # the path name of the client private key file.
80
-# cert: 'cert' # the path name of the client public key certificate file.
81
-# ca: 'ca' # the path name of the Certificate Authority (CA) certificate file. This option, if used, must specify the same certificate used by the server.
82
-# capath: 'capath' # the path name of the directory that contains trusted SSL CA certificate files.
83
-# cipher: [ciphers] # the list of permitted ciphers for SSL encryption.
84
-
85
-# ----------------------------------------------------------------------
86
-# mySQL CONFIGURATION
87
-#
88
-# netdata does not need any privilege - only the ability to connect
89
-# to the mysql server (netdata will not be able to see any data).
90
-#
91
-# Execute these commands to give the local user 'netdata' the ability
92
-# to connect to the mysql server on localhost, without a password:
93
-#
94
-# > create user 'netdata'@'localhost';
95
-# > grant usage on *.* to 'netdata'@'localhost';
96
-# > flush privileges;
97
-#
98
-# with the above statements, netdata will be able to gather mysql
99
-# statistics, without the ability to see or alter any data or affect
100
-# mysql operation in any way. No change is required below.
101
-#
102
-# If you need to monitor mysql replication too, use this instead:
103
-#
104
-# > create user 'netdata'@'localhost';
105
-# > grant replication client on *.* to 'netdata'@'localhost';
106
-# > flush privileges;
107
-#
108
-
109
-# ----------------------------------------------------------------------
110
-# AUTO-DETECTION JOBS
111
-# only one of them will run (they have the same name)
112
-
113
-mycnf1:
114
- name : 'local'
115
- 'my.cnf' : '/etc/my.cnf'
116
-
117
-mycnf2:
118
- name : 'local'
119
- 'my.cnf' : '/etc/mysql/my.cnf'
120
-
121
-debiancnf:
122
- name : 'local'
123
- 'my.cnf' : '/etc/mysql/debian.cnf'
124
-
125
-socket1:
126
- name : 'local'
127
- # user : ''
128
- # pass : ''
129
- socket : '/var/run/mysqld/mysqld.sock'
130
-
131
-socket2:
132
- name : 'local'
133
- # user : ''
134
- # pass : ''
135
- socket : '/var/run/mysqld/mysql.sock'
136
-
137
-socket3:
138
- name : 'local'
139
- # user : ''
140
- # pass : ''
141
- socket : '/var/lib/mysql/mysql.sock'
142
-
143
-socket4:
144
- name : 'local'
145
- # user : ''
146
- # pass : ''
147
- socket : '/tmp/mysql.sock'
148
-
149
-tcp:
150
- name : 'local'
151
- # user : ''
152
- # pass : ''
153
- host : 'localhost'
154
- port : '3306'
155
- # keep in mind port might be ignored by mysql, if host = 'localhost'
156
- # http://serverfault.com/questions/337818/how-to-force-mysql-to-connect-by-tcp-instead-of-a-unix-socket/337844#337844
157
-
158
-tcpipv4:
159
- name : 'local'
160
- # user : ''
161
- # pass : ''
162
- host : '127.0.0.1'
163
- port : '3306'
164
-
165
-tcpipv6:
166
- name : 'local'
167
- # user : ''
168
- # pass : ''
169
- host : '::1'
170
- port : '3306'
171
-
172
-
173
-# Now we try the same as above with user: root
174
-# A few systems configure mysql to accept passwordless
175
-# root access.
176
-
177
-mycnf1_root:
178
- name : 'local'
179
- user : 'root'
180
- 'my.cnf' : '/etc/my.cnf'
181
-
182
-mycnf2_root:
183
- name : 'local'
184
- user : 'root'
185
- 'my.cnf' : '/etc/mysql/my.cnf'
186
-
187
-socket1_root:
188
- name : 'local'
189
- user : 'root'
190
- # pass : ''
191
- socket : '/var/run/mysqld/mysqld.sock'
192
-
193
-socket2_root:
194
- name : 'local'
195
- user : 'root'
196
- # pass : ''
197
- socket : '/var/run/mysqld/mysql.sock'
198
-
199
-socket3_root:
200
- name : 'local'
201
- user : 'root'
202
- # pass : ''
203
- socket : '/var/lib/mysql/mysql.sock'
204
-
205
-socket4_root:
206
- name : 'local'
207
- user : 'root'
208
- # pass : ''
209
- socket : '/tmp/mysql.sock'
210
-
211
-tcp_root:
212
- name : 'local'
213
- user : 'root'
214
- # pass : ''
215
- host : 'localhost'
216
- port : '3306'
217
- # keep in mind port might be ignored by mysql, if host = 'localhost'
218
- # http://serverfault.com/questions/337818/how-to-force-mysql-to-connect-by-tcp-instead-of-a-unix-socket/337844#337844
219
-
220
-tcpipv4_root:
221
- name : 'local'
222
- user : 'root'
223
- # pass : ''
224
- host : '127.0.0.1'
225
- port : '3306'
226
-
227
-tcpipv6_root:
228
- name : 'local'
229
- user : 'root'
230
- # pass : ''
231
- host : '::1'
232
- port : '3306'
233
-
234
-
235
-# Now we try the same as above with user: netdata
236
-
237
-mycnf1_netdata:
238
- name : 'local'
239
- user : 'netdata'
240
- 'my.cnf' : '/etc/my.cnf'
241
-
242
-mycnf2_netdata:
243
- name : 'local'
244
- user : 'netdata'
245
- 'my.cnf' : '/etc/mysql/my.cnf'
246
-
247
-socket1_netdata:
248
- name : 'local'
249
- user : 'netdata'
250
- # pass : ''
251
- socket : '/var/run/mysqld/mysqld.sock'
252
-
253
-socket2_netdata:
254
- name : 'local'
255
- user : 'netdata'
256
- # pass : ''
257
- socket : '/var/run/mysqld/mysql.sock'
258
-
259
-socket3_netdata:
260
- name : 'local'
261
- user : 'netdata'
262
- # pass : ''
263
- socket : '/var/lib/mysql/mysql.sock'
264
-
265
-socket4_netdata:
266
- name : 'local'
267
- user : 'netdata'
268
- # pass : ''
269
- socket : '/tmp/mysql.sock'
270
-
271
-tcp_netdata:
272
- name : 'local'
273
- user : 'netdata'
274
- # pass : ''
275
- host : 'localhost'
276
- port : '3306'
277
- # keep in mind port might be ignored by mysql, if host = 'localhost'
278
- # http://serverfault.com/questions/337818/how-to-force-mysql-to-connect-by-tcp-instead-of-a-unix-socket/337844#337844
279
-
280
-tcpipv4_netdata:
281
- name : 'local'
282
- user : 'netdata'
283
- # pass : ''
284
- host : '127.0.0.1'
285
- port : '3306'
286
-
287
-tcpipv6_netdata:
288
- name : 'local'
289
- user : 'netdata'
290
- # pass : ''
291
- host : '::1'
292
- port : '3306'
293
-
collectors/python.d.plugin/nginx/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += nginx/nginx.chart.py
9
-dist_pythonconfig_DATA += nginx/nginx.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += nginx/README.md nginx/Makefile.inc
13
-
collectors/python.d.plugin/nginx/README.md
deleted
-65
@@ -1,65 +0,0 @@
1
-<!--
2
-title: "NGINX monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/nginx/README.md
4
-sidebar_label: "NGINX"
5
--->
6
-
7
-# NGINX monitoring with Netdata
8
-
9
-Monitors one or more NGINX servers depending on configuration. Servers can be either local or remote.
10
-
11
-## Requirements
12
-
13
-- nginx with configured 'ngx_http_stub_status_module'
14
-- 'location /stub_status'
15
-
16
-Example nginx configuration can be found in 'python.d/nginx.conf'
17
-
18
-It produces following charts:
19
-
20
-1. **Active Connections**
21
-
22
- - active
23
-
24
-2. **Requests** in requests/s
25
-
26
- - requests
27
-
28
-3. **Active Connections by Status**
29
-
30
- - reading
31
- - writing
32
- - waiting
33
-
34
-4. **Connections Rate** in connections/s
35
-
36
- - accepts
37
- - handled
38
-
39
-## Configuration
40
-
41
-Edit the `python.d/nginx.conf` configuration file using `edit-config` from the Netdata [config
42
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
43
-
44
-```bash
45
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
46
-sudo ./edit-config python.d/nginx.conf
47
-```
48
-
49
-Needs only `url` to server's `stub_status`.
50
-
51
-Here is an example for local server:
52
-
53
-```yaml
54
-update_every : 10
55
-priority : 90100
56
-
57
-local:
58
- url : 'http://localhost/stub_status'
59
-```
60
-
61
-Without configuration, module attempts to connect to `http://localhost/stub_status`
62
-
63
----
64
-
65
-
collectors/python.d.plugin/nginx/nginx.chart.py
deleted
-71
@@ -1,71 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: nginx netdata python.d module
3
-# Author: Pawel Krupa (paulfantom)
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-from bases.FrameworkServices.UrlService import UrlService
7
-
8
-ORDER = [
9
- 'connections',
10
- 'requests',
11
- 'connection_status',
12
- 'connect_rate',
13
-]
14
-
15
-CHARTS = {
16
- 'connections': {
17
- 'options': [None, 'Active Connections', 'connections', 'active connections',
18
- 'nginx.connections', 'line'],
19
- 'lines': [
20
- ['active']
21
- ]
22
- },
23
- 'requests': {
24
- 'options': [None, 'Requests', 'requests/s', 'requests', 'nginx.requests', 'line'],
25
- 'lines': [
26
- ['requests', None, 'incremental']
27
- ]
28
- },
29
- 'connection_status': {
30
- 'options': [None, 'Active Connections by Status', 'connections', 'status',
31
- 'nginx.connection_status', 'line'],
32
- 'lines': [
33
- ['reading'],
34
- ['writing'],
35
- ['waiting', 'idle']
36
- ]
37
- },
38
- 'connect_rate': {
39
- 'options': [None, 'Connections Rate', 'connections/s', 'connections rate',
40
- 'nginx.connect_rate', 'line'],
41
- 'lines': [
42
- ['accepts', 'accepted', 'incremental'],
43
- ['handled', None, 'incremental']
44
- ]
45
- }
46
-}
47
-
48
-
49
-class Service(UrlService):
50
- def __init__(self, configuration=None, name=None):
51
- UrlService.__init__(self, configuration=configuration, name=name)
52
- self.order = ORDER
53
- self.definitions = CHARTS
54
- self.url = self.configuration.get('url', 'http://localhost/stub_status')
55
-
56
- def _get_data(self):
57
- """
58
- Format data received from http request
59
- :return: dict
60
- """
61
- try:
62
- raw = self._get_raw_data().split(" ")
63
- return {'active': int(raw[2]),
64
- 'requests': int(raw[9]),
65
- 'reading': int(raw[11]),
66
- 'writing': int(raw[13]),
67
- 'waiting': int(raw[15]),
68
- 'accepts': int(raw[7]),
69
- 'handled': int(raw[8])}
70
- except (ValueError, AttributeError):
71
- return None
collectors/python.d.plugin/nginx/nginx.conf
deleted
-107
@@ -1,107 +0,0 @@
1
-# netdata python.d.plugin configuration for nginx
2
-#
3
-# You must have ngx_http_stub_status_module configured on your nginx server for this
4
-# plugin to work. The following is an example config.
5
-# It must be located inside a server { } block.
6
-#
7
-# location /stub_status {
8
-# stub_status;
9
-# # Security: Only allow access from the IP below.
10
-# allow 192.168.1.200;
11
-# # Deny anyone else
12
-# deny all;
13
-# }
14
-#
15
-# This file is in YaML format. Generally the format is:
16
-#
17
-# name: value
18
-#
19
-# There are 2 sections:
20
-# - global variables
21
-# - one or more JOBS
22
-#
23
-# JOBS allow you to collect values from multiple sources.
24
-# Each source will have its own set of charts.
25
-#
26
-# JOB parameters have to be indented (using spaces only, example below).
27
-
28
-# ----------------------------------------------------------------------
29
-# Global Variables
30
-# These variables set the defaults for all JOBs, however each JOB
31
-# may define its own, overriding the defaults.
32
-
33
-# update_every sets the default data collection frequency.
34
-# If unset, the python.d.plugin default is used.
35
-# update_every: 1
36
-
37
-# priority controls the order of charts at the netdata dashboard.
38
-# Lower numbers move the charts towards the top of the page.
39
-# If unset, the default for python.d.plugin is used.
40
-# priority: 60000
41
-
42
-# penalty indicates whether to apply penalty to update_every in case of failures.
43
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
44
-# penalty: yes
45
-
46
-# autodetection_retry sets the job re-check interval in seconds.
47
-# The job is not deleted if check fails.
48
-# Attempts to start the job are made once every autodetection_retry.
49
-# This feature is disabled by default.
50
-# autodetection_retry: 0
51
-
52
-# ----------------------------------------------------------------------
53
-# JOBS (data collection sources)
54
-#
55
-# The default JOBS share the same *name*. JOBS with the same name
56
-# are mutually exclusive. Only one of them will be allowed running at
57
-# any time. This allows autodetection to try several alternatives and
58
-# pick the one that works.
59
-#
60
-# Any number of jobs is supported.
61
-#
62
-# All python.d.plugin JOBS (for all its modules) support a set of
63
-# predefined parameters. These are:
64
-#
65
-# job_name:
66
-# name: myname # the JOB's name as it will appear at the
67
-# # dashboard (by default is the job_name)
68
-# # JOBs sharing a name are mutually exclusive
69
-# update_every: 1 # the JOB's data collection frequency
70
-# priority: 60000 # the JOB's order on the dashboard
71
-# penalty: yes # the JOB's penalty
72
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
73
-#
74
-# Additionally to the above, this plugin also supports the following:
75
-#
76
-# url: 'URL' # the URL to fetch nginx's status stats
77
-#
78
-# if the URL is password protected, the following are supported:
79
-#
80
-# user: 'username'
81
-# pass: 'password'
82
-#
83
-# Example
84
-#
85
-# RemoteNginx:
86
-# name : 'Reverse_Proxy'
87
-# url : 'http://yourdomain.com/stub_status'
88
-#
89
-# "RemoteNginx" will show up in Netdata logs. "Reverse Proxy" will show up in the menu
90
-# in the nginx section.
91
-
92
-# ----------------------------------------------------------------------
93
-# AUTO-DETECTION JOBS
94
-# only one of them will run (they have the same name)
95
-
96
-localhost:
97
- name : 'local'
98
- url : 'http://localhost/stub_status'
99
-
100
-localipv4:
101
- name : 'local'
102
- url : 'http://127.0.0.1/stub_status'
103
-
104
-localipv6:
105
- name : 'local'
106
- url : 'http://[::1]/stub_status'
107
-
collectors/python.d.plugin/phpfpm/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += phpfpm/phpfpm.chart.py
9
-dist_pythonconfig_DATA += phpfpm/phpfpm.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += phpfpm/README.md phpfpm/Makefile.inc
13
-
collectors/python.d.plugin/phpfpm/README.md
deleted
-51
@@ -1,51 +0,0 @@
1
-<!--
2
-title: "PHP-FPM monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/phpfpm/README.md
4
-sidebar_label: "PHP-FPM"
5
--->
6
-
7
-# PHP-FPM monitoring with Netdata
8
-
9
-Monitors one or more PHP-FPM instances depending on configuration.
10
-
11
-## Requirements
12
-
13
-- `PHP-FPM` with [enabled `status` page](https://easyengine.io/tutorials/php/fpm-status-page/)
14
-- access to `status` page via web server
15
-
16
-## Charts
17
-
18
-It produces following charts:
19
-
20
-- Active Connections in `connections`
21
-- Requests in `requests/s`
22
-- Performance in `status`
23
-- Requests Duration Among All Idle Processes in `milliseconds`
24
-- Last Request CPU Usage Among All Idle Processes in `percentage`
25
-- Last Request Memory Usage Among All Idle Processes in `KB`
26
-
27
-## Configuration
28
-
29
-Edit the `python.d/phpfpm.conf` configuration file using `edit-config` from the Netdata [config
30
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
31
-
32
-```bash
33
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
34
-sudo ./edit-config python.d/phpfpm.conf
35
-```
36
-
37
-Needs only `url` to server's `status`. Here is an example for local and remote instances:
38
-
39
-```yaml
40
-local:
41
- url : 'http://localhost/status?full&json'
42
-
43
-remote:
44
- url : 'http://203.0.113.10/status?full&json'
45
-```
46
-
47
-Without configuration, module attempts to connect to `http://localhost/status`
48
-
49
----
50
-
51
-
collectors/python.d.plugin/phpfpm/phpfpm.chart.py
deleted
-174
@@ -1,174 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: PHP-FPM netdata python.d module
3
-# Author: Pawel Krupa (paulfantom)
4
-# Author: Ilya Mashchenko (ilyam8)
5
-# SPDX-License-Identifier: GPL-3.0-or-later
6
-
7
-import json
8
-import re
9
-
10
-from bases.FrameworkServices.UrlService import UrlService
11
-
12
-REGEX = re.compile(r'([a-z][a-z ]+): ([\d.]+)')
13
-
14
-POOL_INFO = [
15
- ('active processes', 'active'),
16
- ('max active processes', 'maxActive'),
17
- ('idle processes', 'idle'),
18
- ('accepted conn', 'requests'),
19
- ('max children reached', 'reached'),
20
- ('slow requests', 'slow')
21
-]
22
-
23
-PER_PROCESS_INFO = [
24
- ('request duration', 'ReqDur'),
25
- ('last request cpu', 'ReqCpu'),
26
- ('last request memory', 'ReqMem')
27
-]
28
-
29
-
30
-def average(collection):
31
- return sum(collection, 0.0) / max(len(collection), 1)
32
-
33
-
34
-CALC = [
35
- ('min', min),
36
- ('max', max),
37
- ('avg', average)
38
-]
39
-
40
-ORDER = [
41
- 'connections',
42
- 'requests',
43
- 'performance',
44
- 'request_duration',
45
- 'request_cpu',
46
- 'request_mem',
47
-]
48
-
49
-CHARTS = {
50
- 'connections': {
51
- 'options': [None, 'PHP-FPM Active Connections', 'connections', 'active connections', 'phpfpm.connections',
52
- 'line'],
53
- 'lines': [
54
- ['active'],
55
- ['maxActive', 'max active'],
56
- ['idle']
57
- ]
58
- },
59
- 'requests': {
60
- 'options': [None, 'PHP-FPM Requests', 'requests/s', 'requests', 'phpfpm.requests', 'line'],
61
- 'lines': [
62
- ['requests', None, 'incremental']
63
- ]
64
- },
65
- 'performance': {
66
- 'options': [None, 'PHP-FPM Performance', 'status', 'performance', 'phpfpm.performance', 'line'],
67
- 'lines': [
68
- ['reached', 'max children reached'],
69
- ['slow', 'slow requests']
70
- ]
71
- },
72
- 'request_duration': {
73
- 'options': [None, 'PHP-FPM Requests Duration Among All Idle Processes', 'milliseconds', 'request duration',
74
- 'phpfpm.request_duration',
75
- 'line'],
76
- 'lines': [
77
- ['minReqDur', 'min', 'absolute', 1, 1000],
78
- ['maxReqDur', 'max', 'absolute', 1, 1000],
79
- ['avgReqDur', 'avg', 'absolute', 1, 1000]
80
- ]
81
- },
82
- 'request_cpu': {
83
- 'options': [None, 'PHP-FPM Last Request CPU Usage Among All Idle Processes', 'percentage', 'request CPU',
84
- 'phpfpm.request_cpu', 'line'],
85
- 'lines': [
86
- ['minReqCpu', 'min'],
87
- ['maxReqCpu', 'max'],
88
- ['avgReqCpu', 'avg']
89
- ]
90
- },
91
- 'request_mem': {
92
- 'options': [None, 'PHP-FPM Last Request Memory Usage Among All Idle Processes', 'KB', 'request memory',
93
- 'phpfpm.request_mem', 'line'],
94
- 'lines': [
95
- ['minReqMem', 'min', 'absolute', 1, 1024],
96
- ['maxReqMem', 'max', 'absolute', 1, 1024],
97
- ['avgReqMem', 'avg', 'absolute', 1, 1024]
98
- ]
99
- }
100
-}
101
-
102
-
103
-class Service(UrlService):
104
- def __init__(self, configuration=None, name=None):
105
- UrlService.__init__(self, configuration=configuration, name=name)
106
- self.order = ORDER
107
- self.definitions = CHARTS
108
- self.url = self.configuration.get('url', 'http://localhost/status?full&json')
109
- self.json = '&json' in self.url or '?json' in self.url
110
- self.json_full = self.url.endswith(('?full&json', '?json&full'))
111
- self.if_all_processes_running = dict(
112
- [(c_name + p_name, 0) for c_name, func in CALC for metric, p_name in PER_PROCESS_INFO]
113
- )
114
-
115
- def _get_data(self):
116
- """
117
- Format data received from http request
118
- :return: dict
119
- """
120
- raw = self._get_raw_data()
121
- if not raw:
122
- return None
123
-
124
- raw_json = parse_raw_data_(is_json=self.json, raw_data=raw)
125
-
126
- # Per Pool info: active connections, requests and performance charts
127
- to_netdata = fetch_data_(raw_data=raw_json, metrics_list=POOL_INFO)
128
-
129
- # Per Process Info: duration, cpu and memory charts (min, max, avg)
130
- if self.json_full:
131
- p_info = dict()
132
- to_netdata.update(self.if_all_processes_running) # If all processes are in running state
133
- # Metrics are always 0 if the process is not in Idle state because calculation is done
134
- # when the request processing has terminated
135
- for process in [p for p in raw_json['processes'] if p['state'] == 'Idle']:
136
- p_info.update(fetch_data_(raw_data=process, metrics_list=PER_PROCESS_INFO, pid=str(process['pid'])))
137
-
138
- if p_info:
139
- for new_name in PER_PROCESS_INFO:
140
- for name, func in CALC:
141
- to_netdata[name + new_name[1]] = func([p_info[k] for k in p_info if new_name[1] in k])
142
-
143
- return to_netdata or None
144
-
145
-
146
-def fetch_data_(raw_data, metrics_list, pid=''):
147
- """
148
- :param raw_data: dict
149
- :param metrics_list: list
150
- :param pid: str
151
- :return: dict
152
- """
153
- result = dict()
154
- for metric, new_name in metrics_list:
155
- if metric in raw_data:
156
- result[new_name + pid] = float(raw_data[metric])
157
- return result
158
-
159
-
160
-def parse_raw_data_(is_json, raw_data):
161
- """
162
- :param is_json: bool
163
- :param regex: compiled regular expr
164
- :param raw_data: dict
165
- :return: dict
166
- """
167
- if is_json:
168
- try:
169
- return json.loads(raw_data)
170
- except ValueError:
171
- return dict()
172
- else:
173
- raw_data = ' '.join(raw_data.split())
174
- return dict(REGEX.findall(raw_data))
collectors/python.d.plugin/phpfpm/phpfpm.conf
deleted
-88
@@ -1,88 +0,0 @@
1
-# netdata python.d.plugin configuration for PHP-FPM
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, PHP-FPM also supports the following:
63
-#
64
-# url: 'URL' # the URL to fetch nginx's status stats
65
-# # Be sure and include ?full&status at the end of the url
66
-#
67
-# if the URL is password protected, the following are supported:
68
-#
69
-# user: 'username'
70
-# pass: 'password'
71
-#
72
-
73
-# ----------------------------------------------------------------------
74
-# AUTO-DETECTION JOBS
75
-# only one of them will run (they have the same name)
76
-
77
-localhost:
78
- name : 'local'
79
- url : "http://localhost/status?full&json"
80
-
81
-localipv4:
82
- name : 'local'
83
- url : "http://127.0.0.1/status?full&json"
84
-
85
-localipv6:
86
- name : 'local'
87
- url : "http://[::1]/status?full&json"
88
-
collectors/python.d.plugin/portcheck/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += portcheck/portcheck.chart.py
9
-dist_pythonconfig_DATA += portcheck/portcheck.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += portcheck/README.md portcheck/Makefile.inc
13
-
collectors/python.d.plugin/portcheck/README.md
deleted
-52
@@ -1,52 +0,0 @@
1
-<!--
2
-title: "TCP endpoint monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/portcheck/README.md
4
-sidebar_label: "TCP endpoints"
5
--->
6
-
7
-# TCP endpoint monitoring with Netdata
8
-
9
-Monitors TCP endpoint availability and response time.
10
-
11
-Following charts are drawn per host:
12
-
13
-1. **Latency** ms
14
-
15
- - Time required to connect to a TCP port.
16
- Displays latency in 0.1 ms resolution. If the connection failed, the value is missing.
17
-
18
-2. **Status** boolean
19
-
20
- - Connection successful
21
- - Could not create socket: possible DNS problems
22
- - Connection refused: port not listening or blocked
23
- - Connection timed out: host or port unreachable
24
-
25
-## Configuration
26
-
27
-Edit the `python.d/portcheck.conf` configuration file using `edit-config` from the Netdata [config
28
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
29
-
30
-```bash
31
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
32
-sudo ./edit-config python.d/portcheck.conf
33
-```
34
-
35
-```yaml
36
-server:
37
- host: 'dns or ip' # required
38
- port: 22 # required
39
- timeout: 1 # optional
40
- update_every: 1 # optional
41
-```
42
-
43
-### notes
44
-
45
-- The error chart is intended for alarms, badges or for access via API.
46
-- A system/service/firewall might block Netdata's access if a portscan or
47
- similar is detected.
48
-- Currently, the accuracy of the latency is low and should be used as reference only.
49
-
50
----
51
-
52
-
collectors/python.d.plugin/portcheck/portcheck.chart.py
deleted
-157
@@ -1,157 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: simple port check netdata python.d module
3
-# Original Author: ccremer (github.com/ccremer)
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import socket
7
-
8
-try:
9
- from time import monotonic as time
10
-except ImportError:
11
- from time import time
12
-
13
-from bases.FrameworkServices.SimpleService import SimpleService
14
-
15
-PORT_LATENCY = 'connect'
16
-
17
-PORT_SUCCESS = 'success'
18
-PORT_TIMEOUT = 'timeout'
19
-PORT_FAILED = 'no_connection'
20
-
21
-ORDER = ['latency', 'status']
22
-
23
-CHARTS = {
24
- 'latency': {
25
- 'options': [None, 'TCP connect latency', 'milliseconds', 'latency', 'portcheck.latency', 'line'],
26
- 'lines': [
27
- [PORT_LATENCY, 'connect', 'absolute', 100, 1000]
28
- ]
29
- },
30
- 'status': {
31
- 'options': [None, 'Portcheck status', 'boolean', 'status', 'portcheck.status', 'line'],
32
- 'lines': [
33
- [PORT_SUCCESS, 'success', 'absolute'],
34
- [PORT_TIMEOUT, 'timeout', 'absolute'],
35
- [PORT_FAILED, 'no connection', 'absolute']
36
- ]
37
- }
38
-}
39
-
40
-
41
-# Not deriving from SocketService, too much is different
42
-class Service(SimpleService):
43
- def __init__(self, configuration=None, name=None):
44
- SimpleService.__init__(self, configuration=configuration, name=name)
45
- self.order = ORDER
46
- self.definitions = CHARTS
47
- self.host = self.configuration.get('host')
48
- self.port = self.configuration.get('port')
49
- self.timeout = self.configuration.get('timeout', 1)
50
-
51
- def check(self):
52
- """
53
- Parse configuration, check if configuration is available, and dynamically create chart lines data
54
- :return: boolean
55
- """
56
- if self.host is None or self.port is None:
57
- self.error('Host or port missing')
58
- return False
59
- if not isinstance(self.port, int):
60
- self.error('"port" is not an integer. Specify a numerical value, not service name.')
61
- return False
62
-
63
- self.debug('Enabled portcheck: {host}:{port}, update every {update}s, timeout: {timeout}s'.format(
64
- host=self.host, port=self.port, update=self.update_every, timeout=self.timeout
65
- ))
66
- # We will accept any (valid-ish) configuration, even if initial connection fails (a service might be down from
67
- # the beginning)
68
- return True
69
-
70
- def _get_data(self):
71
- """
72
- Get data from socket
73
- :return: dict
74
- """
75
- data = dict()
76
- data[PORT_SUCCESS] = 0
77
- data[PORT_TIMEOUT] = 0
78
- data[PORT_FAILED] = 0
79
-
80
- success = False
81
- try:
82
- for socket_config in socket.getaddrinfo(self.host, self.port, socket.AF_UNSPEC, socket.SOCK_STREAM):
83
- # use first working socket
84
- sock = self._create_socket(socket_config)
85
- if sock is not None:
86
- self._connect2socket(data, socket_config, sock)
87
- self._disconnect(sock)
88
- success = True
89
- break
90
- except socket.gaierror as error:
91
- self.debug('Failed to connect to "{host}:{port}", error: {error}'.format(
92
- host=self.host, port=self.port, error=error
93
- ))
94
-
95
- # We could not connect
96
- if not success:
97
- data[PORT_FAILED] = 1
98
-
99
- return data
100
-
101
- def _create_socket(self, socket_config):
102
- af, sock_type, proto, _, sa = socket_config
103
- try:
104
- self.debug('Creating socket to "{address}", port {port}'.format(address=sa[0], port=sa[1]))
105
- sock = socket.socket(af, sock_type, proto)
106
- sock.settimeout(self.timeout)
107
- return sock
108
- except socket.error as error:
109
- self.debug('Failed to create socket "{address}", port {port}, error: {error}'.format(
110
- address=sa[0], port=sa[1], error=error
111
- ))
112
- return None
113
-
114
- def _connect2socket(self, data, socket_config, sock):
115
- """
116
- Connect to a socket, passing the result of getaddrinfo()
117
- :return: dict
118
- """
119
-
120
- _, _, _, _, sa = socket_config
121
- port = str(sa[1])
122
- try:
123
- self.debug('Connecting socket to "{address}", port {port}'.format(address=sa[0], port=port))
124
- start = time()
125
- sock.connect(sa)
126
- diff = time() - start
127
- self.debug('Connected to "{address}", port {port}, latency {latency}'.format(
128
- address=sa[0], port=port, latency=diff
129
- ))
130
- # we will set it at least 0.1 ms. 0.0 would mean failed connection (handy for 3rd-party-APIs)
131
- data[PORT_LATENCY] = max(round(diff * 10000), 0)
132
- data[PORT_SUCCESS] = 1
133
-
134
- except socket.timeout as error:
135
- self.debug('Socket timed out on "{address}", port {port}, error: {error}'.format(
136
- address=sa[0], port=port, error=error
137
- ))
138
- data[PORT_TIMEOUT] = 1
139
-
140
- except socket.error as error:
141
- self.debug('Failed to connect to "{address}", port {port}, error: {error}'.format(
142
- address=sa[0], port=port, error=error
143
- ))
144
- data[PORT_FAILED] = 1
145
-
146
- def _disconnect(self, sock):
147
- """
148
- Close socket connection
149
- :return:
150
- """
151
- if sock is not None:
152
- try:
153
- self.debug('Closing socket')
154
- sock.shutdown(2) # 0 - read, 1 - write, 2 - all
155
- sock.close()
156
- except socket.error:
157
- pass
collectors/python.d.plugin/portcheck/portcheck.conf
deleted
-74
@@ -1,74 +0,0 @@
1
-# netdata python.d.plugin configuration for portcheck
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# chart_cleanup sets the default chart cleanup interval in iterations.
35
-# A chart is marked as obsolete if it has not been updated
36
-# 'chart_cleanup' iterations in a row.
37
-# They will be hidden immediately (not offered to dashboard viewer,
38
-# streamed upstream and archived to external databases) and deleted one hour
39
-# later (configurable from netdata.conf).
40
-# -- For this plugin, cleanup MUST be disabled, otherwise we lose latency chart
41
-chart_cleanup: 0
42
-
43
-# Autodetection and retries do not work for this plugin
44
-
45
-# ----------------------------------------------------------------------
46
-# JOBS (data collection sources)
47
-#
48
-# The default JOBS share the same *name*. JOBS with the same name
49
-# are mutually exclusive. Only one of them will be allowed running at
50
-# any time. This allows autodetection to try several alternatives and
51
-# pick the one that works.
52
-#
53
-# Any number of jobs is supported.
54
-#
55
-# -------------------------------
56
-# ATTENTION: Any valid configuration will be accepted, even if initial connection fails!
57
-# -------------------------------
58
-#
59
-# There is intentionally no default config for 'localhost'
60
-
61
-# job_name:
62
-# name: myname # [optional] the JOB's name as it will appear at the
63
-# # dashboard (by default is the job_name)
64
-# # JOBs sharing a name are mutually exclusive
65
-# update_every: 1 # [optional] the JOB's data collection frequency
66
-# priority: 60000 # [optional] the JOB's order on the dashboard
67
-# penalty: yes # the JOB's penalty
68
-# timeout: 1 # [optional] the socket timeout when connecting
69
-# host: 'dns or ip' # [required] the remote host address in either IPv4, IPv6 or as DNS name.
70
-# port: 22 # [required] the port number to check. Specify an integer, not service name.
71
-
72
-# You just have been warned about possible portscan blocking. The portcheck plugin is meant for simple use cases.
73
-# Currently, the accuracy of the latency is low and should be used as reference only.
74
-
collectors/python.d.plugin/powerdns/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += powerdns/powerdns.chart.py
9
-dist_pythonconfig_DATA += powerdns/powerdns.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += powerdns/README.md powerdns/Makefile.inc
13
-
collectors/python.d.plugin/powerdns/README.md
deleted
-104
@@ -1,104 +0,0 @@
1
-<!--
2
-title: "PowerDNS monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/powerdns/README.md
4
-sidebar_label: "PowerDNS"
5
--->
6
-
7
-# PowerDNS monitoring with Netdata
8
-
9
-Monitors authoritative server and recursor statistics.
10
-
11
-Powerdns charts:
12
-
13
-1. **Queries and Answers**
14
-
15
- - udp-queries
16
- - udp-answers
17
- - tcp-queries
18
- - tcp-answers
19
-
20
-2. **Cache Usage**
21
-
22
- - query-cache-hit
23
- - query-cache-miss
24
- - packetcache-hit
25
- - packetcache-miss
26
-
27
-3. **Cache Size**
28
-
29
- - query-cache-size
30
- - packetcache-size
31
- - key-cache-size
32
- - meta-cache-size
33
-
34
-4. **Latency**
35
-
36
- - latency
37
-
38
- Powerdns Recursor charts:
39
-
40
-1. **Questions In**
41
-
42
- - questions
43
- - ipv6-questions
44
- - tcp-queries
45
-
46
-2. **Questions Out**
47
-
48
- - all-outqueries
49
- - ipv6-outqueries
50
- - tcp-outqueries
51
- - throttled-outqueries
52
-
53
-3. **Answer Times**
54
-
55
- - answers-slow
56
- - answers0-1
57
- - answers1-10
58
- - answers10-100
59
- - answers100-1000
60
-
61
-4. **Timeouts**
62
-
63
- - outgoing-timeouts
64
- - outgoing4-timeouts
65
- - outgoing6-timeouts
66
-
67
-5. **Drops**
68
-
69
- - over-capacity-drops
70
-
71
-6. **Cache Usage**
72
-
73
- - cache-hits
74
- - cache-misses
75
- - packetcache-hits
76
- - packetcache-misses
77
-
78
-7. **Cache Size**
79
-
80
- - cache-entries
81
- - packetcache-entries
82
- - negcache-entries
83
-
84
-## Configuration
85
-
86
-Edit the `python.d/powerdns.conf` configuration file using `edit-config` from the Netdata [config
87
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
88
-
89
-```bash
90
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
91
-sudo ./edit-config python.d/powerdns.conf
92
-```
93
-
94
-```yaml
95
-local:
96
- name : 'local'
97
- url : 'http://127.0.0.1:8081/api/v1/servers/localhost/statistics'
98
- header :
99
- X-API-Key: 'change_me'
100
-```
101
-
102
----
103
-
104
-
collectors/python.d.plugin/powerdns/powerdns.chart.py
deleted
-153
@@ -1,153 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: powerdns netdata python.d module
3
-# Author: Ilya Mashchenko (ilyam8)
4
-# Author: Luke Whitworth
5
-# SPDX-License-Identifier: GPL-3.0-or-later
6
-
7
-from json import loads
8
-
9
-from bases.FrameworkServices.UrlService import UrlService
10
-
11
-ORDER = [
12
- 'questions',
13
- 'cache_usage',
14
- 'cache_size',
15
- 'latency',
16
-]
17
-
18
-CHARTS = {
19
- 'questions': {
20
- 'options': [None, 'PowerDNS Queries and Answers', 'count', 'questions', 'powerdns.questions', 'line'],
21
- 'lines': [
22
- ['udp-queries', None, 'incremental'],
23
- ['udp-answers', None, 'incremental'],
24
- ['tcp-queries', None, 'incremental'],
25
- ['tcp-answers', None, 'incremental']
26
- ]
27
- },
28
- 'cache_usage': {
29
- 'options': [None, 'PowerDNS Cache Usage', 'count', 'cache', 'powerdns.cache_usage', 'line'],
30
- 'lines': [
31
- ['query-cache-hit', None, 'incremental'],
32
- ['query-cache-miss', None, 'incremental'],
33
- ['packetcache-hit', 'packet-cache-hit', 'incremental'],
34
- ['packetcache-miss', 'packet-cache-miss', 'incremental']
35
- ]
36
- },
37
- 'cache_size': {
38
- 'options': [None, 'PowerDNS Cache Size', 'count', 'cache', 'powerdns.cache_size', 'line'],
39
- 'lines': [
40
- ['query-cache-size', None, 'absolute'],
41
- ['packetcache-size', 'packet-cache-size', 'absolute'],
42
- ['key-cache-size', None, 'absolute'],
43
- ['meta-cache-size', None, 'absolute']
44
- ]
45
- },
46
- 'latency': {
47
- 'options': [None, 'PowerDNS Latency', 'microseconds', 'latency', 'powerdns.latency', 'line'],
48
- 'lines': [
49
- ['latency', None, 'absolute']
50
- ]
51
- }
52
-}
53
-
54
-RECURSOR_ORDER = ['questions-in', 'questions-out', 'answer-times', 'timeouts', 'drops', 'cache_usage', 'cache_size']
55
-
56
-RECURSOR_CHARTS = {
57
- 'questions-in': {
58
- 'options': [None, 'PowerDNS Recursor Questions In', 'count', 'questions', 'powerdns_recursor.questions-in',
59
- 'line'],
60
- 'lines': [
61
- ['questions', None, 'incremental'],
62
- ['ipv6-questions', None, 'incremental'],
63
- ['tcp-questions', None, 'incremental']
64
- ]
65
- },
66
- 'questions-out': {
67
- 'options': [None, 'PowerDNS Recursor Questions Out', 'count', 'questions', 'powerdns_recursor.questions-out',
68
- 'line'],
69
- 'lines': [
70
- ['all-outqueries', None, 'incremental'],
71
- ['ipv6-outqueries', None, 'incremental'],
72
- ['tcp-outqueries', None, 'incremental'],
73
- ['throttled-outqueries', None, 'incremental']
74
- ]
75
- },
76
- 'answer-times': {
77
- 'options': [None, 'PowerDNS Recursor Answer Times', 'count', 'performance', 'powerdns_recursor.answer-times',
78
- 'line'],
79
- 'lines': [
80
- ['answers-slow', None, 'incremental'],
81
- ['answers0-1', None, 'incremental'],
82
- ['answers1-10', None, 'incremental'],
83
- ['answers10-100', None, 'incremental'],
84
- ['answers100-1000', None, 'incremental']
85
- ]
86
- },
87
- 'timeouts': {
88
- 'options': [None, 'PowerDNS Recursor Questions Time', 'count', 'performance', 'powerdns_recursor.timeouts',
89
- 'line'],
90
- 'lines': [
91
- ['outgoing-timeouts', None, 'incremental'],
92
- ['outgoing4-timeouts', None, 'incremental'],
93
- ['outgoing6-timeouts', None, 'incremental']
94
- ]
95
- },
96
- 'drops': {
97
- 'options': [None, 'PowerDNS Recursor Drops', 'count', 'performance', 'powerdns_recursor.drops', 'line'],
98
- 'lines': [
99
- ['over-capacity-drops', None, 'incremental']
100
- ]
101
- },
102
- 'cache_usage': {
103
- 'options': [None, 'PowerDNS Recursor Cache Usage', 'count', 'cache', 'powerdns_recursor.cache_usage', 'line'],
104
- 'lines': [
105
- ['cache-hits', None, 'incremental'],
106
- ['cache-misses', None, 'incremental'],
107
- ['packetcache-hits', 'packet-cache-hit', 'incremental'],
108
- ['packetcache-misses', 'packet-cache-miss', 'incremental']
109
- ]
110
- },
111
- 'cache_size': {
112
- 'options': [None, 'PowerDNS Recursor Cache Size', 'count', 'cache', 'powerdns_recursor.cache_size', 'line'],
113
- 'lines': [
114
- ['cache-entries', None, 'absolute'],
115
- ['packetcache-entries', None, 'absolute'],
116
- ['negcache-entries', None, 'absolute']
117
- ]
118
- }
119
-}
120
-
121
-
122
-class Service(UrlService):
123
- def __init__(self, configuration=None, name=None):
124
- UrlService.__init__(self, configuration=configuration, name=name)
125
- self.order = ORDER
126
- self.definitions = CHARTS
127
- self.url = configuration.get('url', 'http://127.0.0.1:8081/api/v1/servers/localhost/statistics')
128
-
129
- def check(self):
130
- self._manager = self._build_manager()
131
- if not self._manager:
132
- return None
133
-
134
- d = self._get_data()
135
- if not d:
136
- return False
137
-
138
- if is_recursor(d):
139
- self.order = RECURSOR_ORDER
140
- self.definitions = RECURSOR_CHARTS
141
- self.module_name = 'powerdns_recursor'
142
-
143
- return True
144
-
145
- def _get_data(self):
146
- data = self._get_raw_data()
147
- if not data:
148
- return None
149
- return dict((d['name'], d['value']) for d in loads(data))
150
-
151
-
152
-def is_recursor(d):
153
- return 'over-capacity-drops' in d and 'tcp-questions' in d
collectors/python.d.plugin/powerdns/powerdns.conf
deleted
-76
@@ -1,76 +0,0 @@
1
-# netdata python.d.plugin configuration for powerdns
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, apache also supports the following:
63
-#
64
-# url: 'URL' # the URL to fetch powerdns performance statistics
65
-# header:
66
-# X-API-Key: 'Key' # API key
67
-#
68
-# ----------------------------------------------------------------------
69
-# AUTO-DETECTION JOBS
70
-# only one of them will run (they have the same name)
71
-
72
-# localhost:
73
-# name : 'local'
74
-# url : 'http://127.0.0.1:8081/api/v1/servers/localhost/statistics'
75
-# header:
76
-# X-API-Key: 'change_me'
collectors/python.d.plugin/redis/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += redis/redis.chart.py
9
-dist_pythonconfig_DATA += redis/redis.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += redis/README.md redis/Makefile.inc
13
-
collectors/python.d.plugin/redis/README.md
deleted
-64
@@ -1,64 +0,0 @@
1
-<!--
2
-title: "Redis monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/redis/README.md
4
-sidebar_label: "Redis"
5
--->
6
-
7
-# Redis monitoring with Netdata
8
-
9
-Monitors database status. It reads server response to `INFO` command.
10
-
11
-Following charts are drawn:
12
-
13
-1. **Operations** per second
14
-
15
- - operations
16
-
17
-2. **Hit rate** in percent
18
-
19
- - rate
20
-
21
-3. **Memory utilization** in kilobytes
22
-
23
- - total
24
- - lua
25
-
26
-4. **Database keys**
27
-
28
- - lines are creates dynamically based on how many databases are there
29
-
30
-5. **Clients**
31
-
32
- - connected
33
- - blocked
34
-
35
-6. **Slaves**
36
-
37
- - connected
38
-
39
-## Configuration
40
-
41
-Edit the `python.d/redis.conf` configuration file using `edit-config` from the Netdata [config
42
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
43
-
44
-```bash
45
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
46
-sudo ./edit-config python.d/redis.conf
47
-```
48
-
49
-```yaml
50
-socket:
51
- name : 'local'
52
- socket : '/var/lib/redis/redis.sock'
53
-
54
-localhost:
55
- name : 'local'
56
- host : 'localhost'
57
- port : 6379
58
-```
59
-
60
-When no configuration file is found, module tries to connect to TCP/IP socket: `localhost:6379`.
61
-
62
----
63
-
64
-
collectors/python.d.plugin/redis/redis.chart.py
deleted
-268
@@ -1,268 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: redis netdata python.d module
3
-# Author: Pawel Krupa (paulfantom)
4
-# Author: Ilya Mashchenko (ilyam8)
5
-# SPDX-License-Identifier: GPL-3.0-or-later
6
-
7
-import re
8
-from copy import deepcopy
9
-
10
-from bases.FrameworkServices.SocketService import SocketService
11
-
12
-REDIS_ORDER = [
13
- 'operations',
14
- 'hit_rate',
15
- 'memory',
16
- 'keys_redis',
17
- 'eviction',
18
- 'net',
19
- 'connections',
20
- 'clients',
21
- 'slaves',
22
- 'persistence',
23
- 'bgsave_now',
24
- 'bgsave_health',
25
- 'uptime',
26
-]
27
-
28
-PIKA_ORDER = [
29
- 'operations',
30
- 'hit_rate',
31
- 'memory',
32
- 'keys_pika',
33
- 'connections',
34
- 'clients',
35
- 'slaves',
36
- 'uptime',
37
-]
38
-
39
-CHARTS = {
40
- 'operations': {
41
- 'options': [None, 'Operations', 'operations/s', 'operations', 'redis.operations', 'line'],
42
- 'lines': [
43
- ['total_commands_processed', 'commands', 'incremental'],
44
- ['instantaneous_ops_per_sec', 'operations', 'absolute']
45
- ]
46
- },
47
- 'hit_rate': {
48
- 'options': [None, 'Hit rate', 'percentage', 'hits', 'redis.hit_rate', 'line'],
49
- 'lines': [
50
- ['hit_rate', 'rate', 'absolute']
51
- ]
52
- },
53
- 'memory': {
54
- 'options': [None, 'Memory utilization', 'KiB', 'memory', 'redis.memory', 'area'],
55
- 'lines': [
56
- ['maxmemory', 'max', 'absolute', 1, 1024],
57
- ['used_memory', 'total', 'absolute', 1, 1024],
58
- ['used_memory_lua', 'lua', 'absolute', 1, 1024]
59
- ]
60
- },
61
- 'net': {
62
- 'options': [None, 'Bandwidth', 'kilobits/s', 'network', 'redis.net', 'area'],
63
- 'lines': [
64
- ['total_net_input_bytes', 'in', 'incremental', 8, 1000],
65
- ['total_net_output_bytes', 'out', 'incremental', -8, 1000]
66
- ]
67
- },
68
- 'keys_redis': {
69
- 'options': [None, 'Keys per Database', 'keys', 'keys', 'redis.keys', 'line'],
70
- 'lines': []
71
- },
72
- 'keys_pika': {
73
- 'options': [None, 'Keys', 'keys', 'keys', 'redis.keys', 'line'],
74
- 'lines': [
75
- ['kv_keys', 'kv', 'absolute'],
76
- ['hash_keys', 'hash', 'absolute'],
77
- ['list_keys', 'list', 'absolute'],
78
- ['zset_keys', 'zset', 'absolute'],
79
- ['set_keys', 'set', 'absolute']
80
- ]
81
- },
82
- 'eviction': {
83
- 'options': [None, 'Evicted Keys', 'keys', 'keys', 'redis.eviction', 'line'],
84
- 'lines': [
85
- ['evicted_keys', 'evicted', 'absolute']
86
- ]
87
- },
88
- 'connections': {
89
- 'options': [None, 'Connections', 'connections/s', 'connections', 'redis.connections', 'line'],
90
- 'lines': [
91
- ['total_connections_received', 'received', 'incremental', 1],
92
- ['rejected_connections', 'rejected', 'incremental', -1]
93
- ]
94
- },
95
- 'clients': {
96
- 'options': [None, 'Clients', 'clients', 'connections', 'redis.clients', 'line'],
97
- 'lines': [
98
- ['connected_clients', 'connected', 'absolute', 1],
99
- ['blocked_clients', 'blocked', 'absolute', -1]
100
- ]
101
- },
102
- 'slaves': {
103
- 'options': [None, 'Slaves', 'slaves', 'replication', 'redis.slaves', 'line'],
104
- 'lines': [
105
- ['connected_slaves', 'connected', 'absolute']
106
- ]
107
- },
108
- 'persistence': {
109
- 'options': [None, 'Persistence Changes Since Last Save', 'changes', 'persistence',
110
- 'redis.rdb_changes', 'line'],
111
- 'lines': [
112
- ['rdb_changes_since_last_save', 'changes', 'absolute']
113
- ]
114
- },
115
- 'bgsave_now': {
116
- 'options': [None, 'Duration of the RDB Save Operation', 'seconds', 'persistence',
117
- 'redis.bgsave_now', 'absolute'],
118
- 'lines': [
119
- ['rdb_bgsave_in_progress', 'rdb save', 'absolute']
120
- ]
121
- },
122
- 'bgsave_health': {
123
- 'options': [None, 'Status of the Last RDB Save Operation', 'status', 'persistence',
124
- 'redis.bgsave_health', 'line'],
125
- 'lines': [
126
- ['rdb_last_bgsave_status', 'rdb save', 'absolute']
127
- ]
128
- },
129
- 'uptime': {
130
- 'options': [None, 'Uptime', 'seconds', 'uptime', 'redis.uptime', 'line'],
131
- 'lines': [
132
- ['uptime_in_seconds', 'uptime', 'absolute']
133
- ]
134
- }
135
-}
136
-
137
-
138
-def copy_chart(name):
139
- return {name: deepcopy(CHARTS[name])}
140
-
141
-
142
-RE = re.compile(r'\n([a-z_0-9 ]+):(?:keys=)?([^,\r]+)')
143
-
144
-
145
-class Service(SocketService):
146
- def __init__(self, configuration=None, name=None):
147
- SocketService.__init__(self, configuration=configuration, name=name)
148
- self.order = list()
149
- self.definitions = dict()
150
- self._keep_alive = True
151
- self.host = self.configuration.get('host', 'localhost')
152
- self.port = self.configuration.get('port', 6379)
153
- self.unix_socket = self.configuration.get('socket')
154
- p = self.configuration.get('pass')
155
- self.auth_request = 'AUTH {0} \r\n'.format(p).encode() if p else None
156
- self.request = 'INFO\r\n'.encode()
157
- self.bgsave_time = 0
158
- self.keyspace_dbs = set()
159
-
160
- def do_auth(self):
161
- resp = self._get_raw_data(request=self.auth_request)
162
- if not resp:
163
- return False
164
- if resp.strip() != '+OK':
165
- self.error('invalid password')
166
- return False
167
- return True
168
-
169
- def get_raw_and_parse(self):
170
- if self.auth_request and not self.do_auth():
171
- return None
172
-
173
- resp = self._get_raw_data()
174
-
175
- if not resp:
176
- return None
177
-
178
- parsed = RE.findall(resp)
179
-
180
- if not parsed:
181
- self.error('response is invalid/empty')
182
- return None
183
-
184
- return dict((k.replace(' ', '_'), v) for k, v in parsed)
185
-
186
- def get_data(self):
187
- """
188
- Get data from socket
189
- :return: dict
190
- """
191
- data = self.get_raw_and_parse()
192
- if not data:
193
- return None
194
-
195
- self.calc_hit_rate(data)
196
- self.calc_redis_keys(data)
197
- self.calc_redis_rdb_save_operations(data)
198
- return data
199
-
200
- @staticmethod
201
- def calc_hit_rate(data):
202
- try:
203
- hits = int(data['keyspace_hits'])
204
- misses = int(data['keyspace_misses'])
205
- data['hit_rate'] = hits * 100 / (hits + misses)
206
- except (KeyError, ZeroDivisionError):
207
- data['hit_rate'] = 0
208
-
209
- def calc_redis_keys(self, data):
210
- if not data.get('redis_version'):
211
- return
212
- # db0:keys=2,expires=0,avg_ttl=0
213
- new_keyspace_dbs = [k for k in data if k.startswith('db') and k not in self.keyspace_dbs]
214
- for db in new_keyspace_dbs:
215
- self.keyspace_dbs.add(db)
216
- self.charts['keys_redis'].add_dimension([db, None, 'absolute'])
217
- for db in self.keyspace_dbs:
218
- if db not in data:
219
- data[db] = 0
220
-
221
- def calc_redis_rdb_save_operations(self, data):
222
- if not (data.get('redis_version') and data.get('rdb_bgsave_in_progress')):
223
- return
224
- if data['rdb_bgsave_in_progress'] != '0':
225
- self.bgsave_time += self.update_every
226
- else:
227
- self.bgsave_time = 0
228
-
229
- data['rdb_last_bgsave_status'] = 0 if data['rdb_last_bgsave_status'] == 'ok' else 1
230
- data['rdb_bgsave_in_progress'] = self.bgsave_time
231
-
232
- def check(self):
233
- """
234
- Parse configuration, check if redis is available, and dynamically create chart lines data
235
- :return: boolean
236
- """
237
- data = self.get_raw_and_parse()
238
-
239
- if not data:
240
- return False
241
-
242
- self.order = PIKA_ORDER if data.get('pika_version') else REDIS_ORDER
243
-
244
- for n in self.order:
245
- self.definitions.update(copy_chart(n))
246
-
247
- return True
248
-
249
- def _check_raw_data(self, data):
250
- """
251
- Check if all data has been gathered from socket.
252
- Parse first line containing message length and check against received message
253
- :param data: str
254
- :return: boolean
255
- """
256
- length = len(data)
257
- supposed = data.split('\n')[0][1:-1]
258
- offset = len(supposed) + 4 # 1 dollar sing, 1 new line character + 1 ending sequence '\r\n'
259
- if not supposed.isdigit():
260
- return True
261
- supposed = int(supposed)
262
-
263
- if length - offset >= supposed:
264
- self.debug('received full response from redis')
265
- return True
266
-
267
- self.debug('waiting more data from redis')
268
- return False
collectors/python.d.plugin/redis/redis.conf
deleted
-110
@@ -1,110 +0,0 @@
1
-# netdata python.d.plugin configuration for redis
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-#
50
-# All python.d.plugin JOBS (for all its modules) support a set of
51
-# predefined parameters. These are:
52
-#
53
-# job_name:
54
-# name: myname # the JOB's name as it will appear at the
55
-# # dashboard (by default is the job_name)
56
-# # JOBs sharing a name are mutually exclusive
57
-# update_every: 1 # the JOB's data collection frequency
58
-# priority: 60000 # the JOB's order on the dashboard
59
-# penalty: yes # the JOB's penalty
60
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
61
-#
62
-# Additionally to the above, redis also supports the following:
63
-#
64
-# socket: 'path/to/redis.sock'
65
-#
66
-# or
67
-# host: 'IP or HOSTNAME' # the host to connect to
68
-# port: PORT # the port to connect to
69
-#
70
-# and
71
-# pass: 'password' # the redis password to use for AUTH command
72
-#
73
-
74
-# ----------------------------------------------------------------------
75
-# AUTO-DETECTION JOBS
76
-# only one of them will run (they have the same name)
77
-
78
-socket1:
79
- name : 'local'
80
- socket : '/tmp/redis.sock'
81
- # pass : ''
82
-
83
-socket2:
84
- name : 'local'
85
- socket : '/var/run/redis/redis.sock'
86
- # pass : ''
87
-
88
-socket3:
89
- name : 'local'
90
- socket : '/var/lib/redis/redis.sock'
91
- # pass : ''
92
-
93
-localhost:
94
- name : 'local'
95
- host : 'localhost'
96
- port : 6379
97
- # pass : ''
98
-
99
-localipv4:
100
- name : 'local'
101
- host : '127.0.0.1'
102
- port : 6379
103
- # pass : ''
104
-
105
-localipv6:
106
- name : 'local'
107
- host : '::1'
108
- port : 6379
109
- # pass : ''
110
-
collectors/python.d.plugin/web_log/Makefile.inc
deleted
-13
@@ -1,13 +0,0 @@
1
-# SPDX-License-Identifier: GPL-3.0-or-later
2
-
3
-# THIS IS NOT A COMPLETE Makefile
4
-# IT IS INCLUDED BY ITS PARENT'S Makefile.am
5
-# IT IS REQUIRED TO REFERENCE ALL FILES RELATIVE TO THE PARENT
6
-
7
-# install these files
8
-dist_python_DATA += web_log/web_log.chart.py
9
-dist_pythonconfig_DATA += web_log/web_log.conf
10
-
11
-# do not install these files, but include them in the distribution
12
-dist_noinst_DATA += web_log/README.md web_log/Makefile.inc
13
-
collectors/python.d.plugin/web_log/README.md
deleted
-219
@@ -1,219 +0,0 @@
1
-<!--
2
-title: "Web server log (Apache, NGINX, Squid) monitoring with Netdata"
3
-custom_edit_url: https://github.com/netdata/netdata/edit/master/collectors/python.d.plugin/web_log/README.md
4
-sidebar_label: "Web server logs (Apache, NGINX, Squid)"
5
--->
6
-
7
-# Web server log (Apache, NGINX, Squid) monitoring with Netdata
8
-
9
-Tails access log file and Collects web server/caching proxy metrics.
10
-
11
-## Motivation
12
-
13
-Web server log files exist for more than 20 years. All web servers of all kinds, from all vendors, [since the time NCSA httpd was powering the web](https://en.wikipedia.org/wiki/NCSA_HTTPd), produce log files, saving in real-time all accesses to web sites and APIs.
14
-
15
-Yet, after the appearance of google analytics and similar services, and the recent rise of APM (Application Performance Monitoring) with sophisticated time-series databases that collect and analyze metrics at the application level, all these web server log files are mostly just filling our disks, rotated every night without any use whatsoever.
16
-
17
-Netdata turns this "useless" log file, into a powerful performance and health monitoring tool, capable of detecting, **in real-time**, most common web server problems, such as:
18
-
19
-- too many redirects (i.e. **oops!** *this should not redirect clients to itself*)
20
-- too many bad requests (i.e. **oops!** *a few files were not uploaded*)
21
-- too many internal server errors (i.e. **oops!** *this release crashes too much*)
22
-- unreasonably too many requests (i.e. **oops!** *we are under attack*)
23
-- unreasonably few requests (i.e. **oops!** *call the network guys*)
24
-- unreasonably slow responses (i.e. **oops!** *the database is slow again*)
25
-- too few successful responses (i.e. **oops!** *help us God!*)
26
-
27
-## Usage
28
-
29
-If Netdata is installed on a system running a web server, it will detect it and it will automatically present a series of charts, with information obtained from the web server API, like these (*these do not come from the web server log file*):
30
-
31
-
32
-*[**netdata**](https://my-netdata.io/) charts based on metrics collected by querying the `nginx` API (i.e. `/stub_status`).*
33
-
34
-> [**netdata**](https://my-netdata.io/) supports `apache`, `nginx`, `lighttpd` and `tomcat`. To obtain real-time information from a web server API, the web server needs to expose it. For directions on configuring your web server, check the config files for each web server. There is a directory with a config file for each web server under `/etc/netdata/python.d/`.
35
-
36
-## Configuration
37
-
38
-Edit the `python.d/web_log.conf` configuration file using `edit-config` from the Netdata [config
39
-directory](/docs/configure/nodes.md), which is typically at `/etc/netdata`.
40
-
41
-```bash
42
-cd /etc/netdata # Replace this path with your Netdata config directory, if different
43
-sudo ./edit-config python.d/web_log.conf
44
-```
45
-
46
-[**netdata**](https://my-netdata.io/) has a powerful `web_log` plugin, capable of incrementally parsing any number of web server log files. This plugin is automatically started with [**netdata**](https://my-netdata.io/) and comes, pre-configured, for finding web server log files on popular distributions. Its configuration is at `/etc/netdata/python.d/web_log.conf`, like this:
47
-
48
-```yaml
49
-nginx_log:
50
- name : 'nginx_log'
51
- path : '/var/log/nginx/access.log'
52
-
53
-apache_log:
54
- name : 'apache_log'
55
- path : '/var/log/apache/other_vhosts_access.log'
56
- categories:
57
- cacti : 'cacti.*'
58
- observium : 'observium'
59
-```
60
-
61
-The module has preconfigured jobs for nginx, apache and gunicorn on various distros.
62
-You can add one such section for each of your web server log files.
63
-
64
-> **Important**<br/>Keep in mind [**netdata**](https://my-netdata.io/) runs as user `netdata`. So, make sure user `netdata` has access to the logs directory and can read the log file.
65
-
66
-## Charts
67
-
68
-Once you have all log files configured and [**netdata**](https://my-netdata.io/) restarted, **for each log file** you will get a section at the [**netdata**](https://my-netdata.io/) dashboard, with the following charts.
69
-
70
-### Responses by status
71
-
72
-In this chart we tried to provide a meaningful status for all responses. So:
73
-
74
-- `success` counts all the valid responses (i.e. `1xx` informational, `2xx` successful and `304` not modified).
75
-- `error` are `5xx` internal server errors. These are very bad, they mean your web site or API is facing difficulties.
76
-- `redirect` are `3xx` responses, except `304`. All `3xx` are redirects, but `304` means "not modified" - it tells the browsers the content they already have is still valid and can be used as-is. So, we decided to account it as a successful response.
77
-- `bad` are bad requests that cannot be served.
78
-- `other` as all the other, non-standard, types of responses.
79
-
80
-
81
-
82
-### Responses by type
83
-
84
-Then, we group all responses by code family, without interpreting their meaning.
85
-**Response by type** requests/s
86
-
87
-- success (1xx, 2xx, 304)
88
-- error (5xx)
89
-- redirect (3xx except 304)
90
-- bad (4xx)
91
-- other (all other responses)
92
-
93
-
94
-
95
-### Responses by code family
96
-
97
-Here we show all the response codes in detail.
98
-
99
-**Response by code family** requests/s
100
-
101
-- 1xx (informational)
102
-- 2xx (successful)
103
-- 3xx (redirect)
104
-- 4xx (bad)
105
-- 5xx (internal server errors)
106
-- other (non-standart responses)
107
-- unmatched (the lines in the log file that are not matched)
108
-
109
-
110
-
111
-> **Important**<br/>If your application is using hundreds of non-standard response codes, your browser may become slow while viewing this chart, so we have added a configuration [option to disable this chart](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L63).
112
-
113
-### Detailed Response Codes
114
-
115
-Number of responses for each response code family individually (requests/s)
116
-
117
-### Bandwidth
118
-
119
-This is a nice view of the traffic the web server is receiving and is sending.
120
-
121
-What is important to know for this chart, is that the bandwidth used for each request and response is accounted at the time the log is written. Since [**netdata**](https://my-netdata.io/) refreshes this chart every single second, you may have unrealistic spikes is the size of the requests or responses is too big. The reason is simple: a response may have needed 1 minute to be completed, but all the bandwidth used during that minute for the specific response will be accounted at the second the log line is written.
122
-
123
-As the legend on the chart suggests, you can use FireQOS to setup QoS on the web server ports and IPs to accurately measure the bandwidth the web server is using. Actually, [there may be a few more reasons to install QoS on your servers](/collectors/tc.plugin/README.md#tcplugin)...
124
-
125
-**Bandwidth** KB/s
126
-
127
-- received (bandwidth of requests)
128
-- send (bandwidth of responses)
129
-
130
-
131
-
132
-> **Important**<br/>Most web servers do not log the request size by default.<br/>So, [unless you have configured your web server to log the size of requests](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L76-L89), the `received` dimension will be always zero.
133
-
134
-### Timings
135
-
136
-[**netdata**](https://my-netdata.io/) will also render the `minimum`, `average` and `maximum` time the web server needed to respond to requests.
137
-
138
-Keep in mind most web servers timings start at the reception of the full request, until the dispatch of the last byte of the response. So, they include network latencies of responses, but they do not include network latencies of requests.
139
-
140
-**Timings** ms (request processing time)
141
-
142
-- min (bandwidth of requests)
143
-- max (bandwidth of responses)
144
-- average (bandwidth of responses)
145
-
146
-
147
-
148
-> **Important**<br/>Most web servers do not log timing information by default.<br/>So, [unless you have configured your web server to also log timings](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L76-L89), this chart will not exist.
149
-
150
-### URL patterns
151
-
152
-This is a very interesting chart. It is configured entirely by you.
153
-
154
-[**netdata**](https://my-netdata.io/) can map the URLs found in the log file into categories. You can define these categories, by providing names and regular expressions in `web_log.conf`.
155
-
156
-So, this configuration:
157
-
158
-```yaml
159
-nginx_netdata: # name the charts
160
- path: '/var/log/nginx/access.log' # web server log file
161
- categories:
162
- badges : '^/api/v1/badge\.svg'
163
- charts : '^/api/v1/(data|chart|charts)'
164
- registry : '^/api/v1/registry'
165
- alarms : '^/api/v1/alarm'
166
- allmetrics : '^/api/v1/allmetrics'
167
- api_other : '^/api/'
168
- netdata_conf: '^/netdata.conf'
169
- api_old : '^/(data|datasource|graph|list|all\.json)'
170
-```
171
-
172
-Produces the following chart. The `categories` section is matched in the order given. So, pay attention to the order you give your patterns.
173
-
174
-
175
-
176
-### HTTP methods
177
-
178
-This chart breaks down requests by HTTP method used.
179
-
180
-
181
-
182
-### IP versions
183
-
184
-This one provides requests per IP version used by the clients (`IPv4`, `IPv6`).
185
-
186
-
187
-
188
-### Unique clients
189
-
190
-The last charts are about the unique IPs accessing your web server.
191
-
192
-**Current Poll Unique Client IPs** unique ips/s. This one counts the unique IPs for each data collection iteration (i.e. **unique clients per second**).
193
-
194
-
195
-
196
-**All Time Unique Client IPs** unique ips/s. Counts the unique IPs, since the last [**netdata**](https://my-netdata.io/) restart.
197
-
198
-
199
-
200
-> **Important**<br/>To provide this information `web_log` plugin keeps in memory all the IPs seen by the web server. Although this does not require so much memory, if you have a web server with several million unique client IPs, we suggest to [disable this chart](https://github.com/netdata/netdata/blob/419cd0a237275e5eeef3f92dcded84e735ee6c58/conf.d/python.d/web_log.conf#L64).
201
-
202
-## Alarms
203
-
204
-The magic of [**netdata**](https://my-netdata.io/) is that all metrics are collected per second, and all metrics can be used or correlated to provide real-time alarms. Out of the box, [**netdata**](https://my-netdata.io/) automatically attaches the following alarms] to all `web_log` charts (i.e. to all log files configured, individually):
205
-
206
-| alarm|description|minimum<br/>requests|warning|critical|
207
-|:----|-----------|:------------------:|:-----:|:------:|
208
-| `1m_redirects`|The ratio of HTTP redirects (3xx except 304) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is suffering from too many or circular redirects.*<br/> <br/>(i.e. **oops!** *this should not redirect clients to itself*)|120/min|> 20%|> 30%|
209
-| `1m_bad_requests`|The ratio of HTTP bad requests (4xx) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is receiving too many bad requests, including `404`, not found.*<br/> <br/>(i.e. **oops!** *a few files were not uploaded*)|120/min|> 30%|> 50%|
210
-| `1m_internal_errors`|The ratio of HTTP internal server errors (5xx), over all the requests, during the last minute.<br/> <br/>*Detects if the site is facing difficulties to serve requests.*<br/> <br/>(i.e. **oops!** *this release crashes too much*)|120/min|> 2%|> 5%|
211
-| `5m_requests_ratio`|The percentage of successful web requests of the last 5 minutes, compared with the previous 5 minutes.<br/> <br/>*Detects if the site or the web API is suddenly getting too many or too few requests.*<br/> <br/>(i.e. too many = **oops!** *we are under attack*)<br/>(i.e. too few = **oops!** *call the network guys*)|120/5min|> double or \< half|> 4x or \< 1/4x|
212
-| `web_slow`|The average time to respond to requests, over the last 1 minute, compared to the average of last 10 minutes.<br/> <br/>*Detects if the site or the web API is suddenly a lot slower.*<br/> <br/>(i.e. **oops!** *the database is slow again*)|120/min|> 2x|> 4x|
213
-| `1m_successful`|The ratio of successful HTTP responses (1xx, 2xx, 304) over all the requests, during the last minute.<br/> <br/>*Detects if the site or the web API is performing within limits.*<br/> <br/>(i.e. **oops!** *help us God!*)|120/min|\< 85%|\< 75%|
214
-
215
-The column `minimum requests` state the minimum number of requests required for the alarm to be evaluated. We found that when the site is receiving requests above this rate, these alarms are pretty accurate (i.e. no false-positives).
216
-
217
-Netdata alarms are user-configurable. Sample config files can be found under directory `health/health.d` of the [Netdata GitHub repository](https://github.com/netdata/netdata/).
218
-
219
-
collectors/python.d.plugin/web_log/web_log.chart.py
deleted
-1194
@@ -1,1194 +0,0 @@
1
-# -*- coding: utf-8 -*-
2
-# Description: web log netdata python.d module
3
-# Author: ilyam8
4
-# SPDX-License-Identifier: GPL-3.0-or-later
5
-
6
-import bisect
7
-import os
8
-import re
9
-from collections import namedtuple, defaultdict
10
-from copy import deepcopy
11
-
12
-try:
13
- from itertools import filterfalse
14
-except ImportError:
15
- from itertools import ifilter as filter
16
- from itertools import ifilterfalse as filterfalse
17
-
18
-try:
19
- from sys import maxint
20
-except ImportError:
21
- from sys import maxsize as maxint
22
-
23
-from bases.collection import read_last_line
24
-from bases.FrameworkServices.LogService import LogService
25
-
26
-ORDER_APACHE_CACHE = [
27
- 'apache_cache',
28
-]
29
-
30
-ORDER_WEB = [
31
- 'response_statuses',
32
- 'response_codes',
33
- 'bandwidth',
34
- 'response_time',
35
- 'response_time_hist',
36
- 'response_time_upstream',
37
- 'response_time_upstream_hist',
38
- 'requests_per_url',
39
- 'requests_per_user_defined',
40
- 'http_method',
41
- 'vhost',
42
- 'port',
43
- 'http_version',
44
- 'requests_per_ipproto',
45
- 'clients',
46
- 'clients_all'
47
-]
48
-
49
-ORDER_SQUID = [
50
- 'squid_response_statuses',
51
- 'squid_response_codes',
52
- 'squid_detailed_response_codes',
53
- 'squid_method',
54
- 'squid_mime_type',
55
- 'squid_hier_code',
56
- 'squid_transport_methods',
57
- 'squid_transport_errors',
58
- 'squid_code',
59
- 'squid_handling_opts',
60
- 'squid_object_types',
61
- 'squid_cache_events',
62
- 'squid_bytes',
63
- 'squid_duration',
64
- 'squid_clients',
65
- 'squid_clients_all'
66
-]
67
-
68
-CHARTS_WEB = {
69
- 'response_codes': {
70
- 'options': [None, 'Response Codes', 'requests/s', 'responses', 'web_log.response_codes', 'stacked'],
71
- 'lines': [
72
- ['2xx', None, 'incremental'],
73
- ['5xx', None, 'incremental'],
74
- ['3xx', None, 'incremental'],
75
- ['4xx', None, 'incremental'],
76
- ['1xx', None, 'incremental'],
77
- ['0xx', 'other', 'incremental'],
78
- ['unmatched', None, 'incremental']
79
- ]
80
- },
81
- 'bandwidth': {
82
- 'options': [None, 'Bandwidth', 'kilobits/s', 'bandwidth', 'web_log.bandwidth', 'area'],
83
- 'lines': [
84
- ['resp_length', 'received', 'incremental', 8, 1000],
85
- ['bytes_sent', 'sent', 'incremental', -8, 1000]
86
- ]
87
- },
88
- 'response_time': {
89
- 'options': [None, 'Processing Time', 'milliseconds', 'timings', 'web_log.response_time', 'area'],
90
- 'lines': [
91
- ['resp_time_min', 'min', 'incremental', 1, 1000],
92
- ['resp_time_max', 'max', 'incremental', 1, 1000],
93
- ['resp_time_avg', 'avg', 'incremental', 1, 1000]
94
- ]
95
- },
96
- 'response_time_hist': {
97
- 'options': [None, 'Processing Time Histogram', 'requests/s', 'timings', 'web_log.response_time_hist', 'line'],
98
- 'lines': []
99
- },
100
- 'response_time_upstream': {
101
- 'options': [None, 'Processing Time Upstream', 'milliseconds', 'timings',
102
- 'web_log.response_time_upstream', 'area'],
103
- 'lines': [
104
- ['resp_time_upstream_min', 'min', 'incremental', 1, 1000],
105
- ['resp_time_upstream_max', 'max', 'incremental', 1, 1000],
106
- ['resp_time_upstream_avg', 'avg', 'incremental', 1, 1000]
107
- ]
108
- },
109
- 'response_time_upstream_hist': {
110
- 'options': [None, 'Processing Time Histogram', 'requests/s', 'timings',
111
- 'web_log.response_time_upstream_hist', 'line'],
112
- 'lines': []
113
- },
114
- 'clients': {
115
- 'options': [None, 'Current Poll Unique Client IPs', 'unique ips', 'clients', 'web_log.clients', 'stacked'],
116
- 'lines': [
117
- ['unique_cur_ipv4', 'ipv4', 'incremental', 1, 1],
118
- ['unique_cur_ipv6', 'ipv6', 'incremental', 1, 1]
119
- ]
120
- },
121
- 'clients_all': {
122
- 'options': [None, 'All Time Unique Client IPs', 'unique ips', 'clients', 'web_log.clients_all', 'stacked'],
123
- 'lines': [
124
- ['unique_tot_ipv4', 'ipv4', 'absolute', 1, 1],
125
- ['unique_tot_ipv6', 'ipv6', 'absolute', 1, 1]
126
- ]
127
- },
128
- 'http_method': {
129
- 'options': [None, 'Requests Per HTTP Method', 'requests/s', 'http methods', 'web_log.http_method', 'stacked'],
130
- 'lines': [
131
- ['GET', 'GET', 'incremental', 1, 1]
132
- ]
133
- },
134
- 'http_version': {
135
- 'options': [None, 'Requests Per HTTP Version', 'requests/s', 'http versions',
136
- 'web_log.http_version', 'stacked'],
137
- 'lines': []
138
- },
139
- 'requests_per_ipproto': {
140
- 'options': [None, 'Requests Per IP Protocol', 'requests/s', 'ip protocols', 'web_log.requests_per_ipproto',
141
- 'stacked'],
142
- 'lines': [
143
- ['req_ipv4', 'ipv4', 'incremental', 1, 1],
144
- ['req_ipv6', 'ipv6', 'incremental', 1, 1]
145
- ]
146
- },
147
- 'response_statuses': {
148
- 'options': [None, 'Response Statuses', 'requests/s', 'responses', 'web_log.response_statuses', 'stacked'],
149
- 'lines': [
150
- ['successful_requests', 'success', 'incremental', 1, 1],
151
- ['server_errors', 'error', 'incremental', 1, 1],
152
- ['redirects', 'redirect', 'incremental', 1, 1],
153
- ['bad_requests', 'bad', 'incremental', 1, 1],
154
- ['other_requests', 'other', 'incremental', 1, 1]
155
- ]
156
- },
157
- 'requests_per_url': {
158
- 'options': [None, 'Requests Per Url', 'requests/s', 'urls', 'web_log.requests_per_url', 'stacked'],
159
- 'lines': [
160
- ['url_pattern_other', 'other', 'incremental', 1, 1]
161
- ]
162
- },
163
- 'requests_per_user_defined': {
164
- 'options': [None, 'Requests Per User Defined Pattern', 'requests/s', 'user defined',
165
- 'web_log.requests_per_user_defined', 'stacked'],
166
- 'lines': [
167
- ['user_pattern_other', 'other', 'incremental', 1, 1]
168
- ]
169
- },
170
- 'port': {
171
- 'options': [None, 'Requests Per Port', 'requests/s', 'port', 'web_log.port', 'stacked'],
172
- 'lines': [
173
- ['port_80', 'http', 'incremental', 1, 1],
174
- ['port_443', 'https', 'incremental', 1, 1]
175
- ]
176
- },
177
- 'vhost': {
178
- 'options': [None, 'Requests Per Vhost', 'requests/s', 'vhost', 'web_log.vhost', 'stacked'],
179
- 'lines': []
180
- }
181
-}
182
-
183
-CHARTS_APACHE_CACHE = {
184
- 'apache_cache': {
185
- 'options': [None, 'Apache Cached Responses', 'percentage', 'cached', 'web_log.apache_cache_cache',
186
- 'stacked'],
187
- 'lines': [
188
- ['hit', 'cache', 'percentage-of-absolute-row'],
189
- ['miss', None, 'percentage-of-absolute-row'],
190
- ['other', None, 'percentage-of-absolute-row']
191
- ]
192
- }
193
-}
194
-
195
-CHARTS_SQUID = {
196
- 'squid_duration': {
197
- 'options': [None, 'Elapsed Time The Transaction Busied The Cache',
198
- 'milliseconds', 'squid_timings', 'web_log.squid_duration', 'area'],
199
- 'lines': [
200
- ['duration_min', 'min', 'incremental', 1, 1000],
201
- ['duration_max', 'max', 'incremental', 1, 1000],
202
- ['duration_avg', 'avg', 'incremental', 1, 1000]
203
- ]
204
- },
205
- 'squid_bytes': {
206
- 'options': [None, 'Amount Of Data Delivered To The Clients',
207
- 'kilobits/s', 'squid_bandwidth', 'web_log.squid_bytes', 'area'],
208
- 'lines': [
209
- ['bytes', 'sent', 'incremental', 8, 1000]
210
- ]
211
- },
212
- 'squid_response_statuses': {
213
- 'options': [None, 'Response Statuses', 'responses/s', 'squid_responses', 'web_log.squid_response_statuses',
214
- 'stacked'],
215
- 'lines': [
216
- ['successful_requests', 'success', 'incremental', 1, 1],
217
- ['server_errors', 'error', 'incremental', 1, 1],
218
- ['redirects', 'redirect', 'incremental', 1, 1],
219
- ['bad_requests', 'bad', 'incremental', 1, 1],
220
- ['other_requests', 'other', 'incremental', 1, 1]
221
- ]
222
- },
223
- 'squid_response_codes': {
224
- 'options': [None, 'Response Codes', 'responses/s', 'squid_responses',
225
- 'web_log.squid_response_codes', 'stacked'],
226
- 'lines': [
227
- ['2xx', None, 'incremental'],
228
- ['5xx', None, 'incremental'],
229
- ['3xx', None, 'incremental'],
230
- ['4xx', None, 'incremental'],
231
- ['1xx', None, 'incremental'],
232
- ['0xx', None, 'incremental'],
233
- ['other', None, 'incremental'],
234
- ['unmatched', None, 'incremental']
235
- ]
236
- },
237
- 'squid_code': {
238
- 'options': [None, 'Responses Per Cache Result Of The Request',
239
- 'requests/s', 'squid_squid_cache', 'web_log.squid_code', 'stacked'],
240
- 'lines': []
241
- },
242
- 'squid_detailed_response_codes': {
243
- 'options': [None, 'Detailed Response Codes',
244
- 'responses/s', 'squid_responses', 'web_log.squid_detailed_response_codes', 'stacked'],
245
- 'lines': []
246
- },
247
- 'squid_hier_code': {
248
- 'options': [None, 'Responses Per Hierarchy Code',
249
- 'requests/s', 'squid_hierarchy', 'web_log.squid_hier_code', 'stacked'],
250
- 'lines': []
251
- },
252
- 'squid_method': {
253
- 'options': [None, 'Requests Per Method',
254
- 'requests/s', 'squid_requests', 'web_log.squid_method', 'stacked'],
255
- 'lines': []
256
- },
257
- 'squid_mime_type': {
258
- 'options': [None, 'Requests Per MIME Type',
259
- 'requests/s', 'squid_requests', 'web_log.squid_mime_type', 'stacked'],
260
- 'lines': []
261
- },
262
- 'squid_clients': {
263
- 'options': [None, 'Current Poll Unique Client IPs', 'unique ips', 'squid_clients',
264
- 'web_log.squid_clients', 'stacked'],
265
- 'lines': [
266
- ['unique_ipv4', 'ipv4', 'incremental'],
267
- ['unique_ipv6', 'ipv6', 'incremental']
268
- ]
269
- },
270
- 'squid_clients_all': {
271
- 'options': [None, 'All Time Unique Client IPs', 'unique ips', 'squid_clients',
272
- 'web_log.squid_clients_all', 'stacked'],
273
- 'lines': [
274
- ['unique_tot_ipv4', 'ipv4', 'absolute'],
275
- ['unique_tot_ipv6', 'ipv6', 'absolute']
276
- ]
277
- },
278
- 'squid_transport_methods': {
279
- 'options': [None, 'Transport Methods', 'requests/s', 'squid_squid_transport',
280
- 'web_log.squid_transport_methods', 'stacked'],
281
- 'lines': []
282
- },
283
- 'squid_transport_errors': {
284
- 'options': [None, 'Transport Errors', 'requests/s', 'squid_squid_transport',
285
- 'web_log.squid_transport_errors', 'stacked'],
286
- 'lines': []
287
- },
288
- 'squid_handling_opts': {
289
- 'options': [None, 'Handling Opts', 'requests/s', 'squid_squid_cache',
290
- 'web_log.squid_handling_opts', 'stacked'],
291
- 'lines': []
292
- },
293
- 'squid_object_types': {
294
- 'options': [None, 'Object Types', 'objects/s', 'squid_squid_cache',
295
- 'web_log.squid_object_types', 'stacked'],
296
- 'lines': []
297
- },
298
- 'squid_cache_events': {
299
- 'options': [None, 'Cache Events', 'events/s', 'squid_squid_cache',
300
- 'web_log.squid_cache_events', 'stacked'],
301
- 'lines': []
302
- }
303
-}
304
-
305
-NAMED_PATTERN = namedtuple('PATTERN', ['description', 'func'])
306
-
307
-DET_RESP_AGGR = ['', '_1xx', '_2xx', '_3xx', '_4xx', '_5xx', '_Other']
308
-
309
-SQUID_CODES = {
310
- 'TCP': 'squid_transport_methods',
311
- 'UDP': 'squid_transport_methods',
312
- 'NONE': 'squid_transport_methods',
313
- 'CLIENT': 'squid_handling_opts',
314
- 'IMS': 'squid_handling_opts',
315
- 'ASYNC': 'squid_handling_opts',
316
- 'SWAPFAIL': 'squid_handling_opts',
317
- 'REFRESH': 'squid_handling_opts',
318
- 'SHARED': 'squid_handling_opts',
319
- 'REPLY': 'squid_handling_opts',
320
- 'NEGATIVE': 'squid_object_types',
321
- 'STALE': 'squid_object_types',
322
- 'OFFLINE': 'squid_object_types',
323
- 'INVALID': 'squid_object_types',
324
- 'FAIL': 'squid_object_types',
325
- 'MODIFIED': 'squid_object_types',
326
- 'UNMODIFIED': 'squid_object_types',
327
- 'REDIRECT': 'squid_object_types',
328
- 'HIT': 'squid_cache_events',
329
- 'MEM': 'squid_cache_events',
330
- 'MISS': 'squid_cache_events',
331
- 'DENIED': 'squid_cache_events',
332
- 'NOFETCH': 'squid_cache_events',
333
- 'TUNNEL': 'squid_cache_events',
334
- 'ABORTED': 'squid_transport_errors',
335
- 'TIMEOUT': 'squid_transport_errors'
336
-}
337
-
338
-REQUEST_REGEX = re.compile(r'(?P<method>[A-Z]+) (?P<url>[^ ]+) [A-Z]+/(?P<http_version>\d(?:.\d)?)')
339
-
340
-MIME_TYPES = ['application', 'audio', 'example', 'font', 'image', 'message', 'model', 'multipart', 'text', 'video']
341
-
342
-
343
-class Service(LogService):
344
- def __init__(self, configuration=None, name=None):
345
- """
346
- :param configuration:
347
- :param name:
348
- """
349
- LogService.__init__(self, configuration=configuration, name=name)
350
- self.configuration = configuration
351
- self.log_path = self.configuration.get('path')
352
- self.job = None
353
-
354
- def check(self):
355
- """
356
- :return: bool
357
-
358
- 1. "log_path" is specified in the module configuration file
359
- 2. "log_path" must be readable by netdata user and must exist
360
- 3. "log_path' must not be empty. We need at least 1 line to find appropriate pattern to parse
361
- 4. other checks depends on log "type"
362
- """
363
-
364
- log_type = self.configuration.get('type', 'web')
365
- log_types = dict(web=Web, apache_cache=ApacheCache, squid=Squid)
366
-
367
- if log_type not in log_types:
368
- self.error('bad log type {log_type}. Supported types: {types}'.format(log_type=log_type,
369
- types=log_types.keys()))
370
- return False
371
-
372
- if not self.log_path:
373
- self.error('log path is not specified')
374
- return False
375
-
376
- if not (self._find_recent_log_file() and os.access(self.log_path, os.R_OK)):
377
- self.error('{log_file} not readable or not exist'.format(log_file=self.log_path))
378
- return False
379
-
380
- if not os.path.getsize(self.log_path):
381
- self.error('{log_file} is empty'.format(log_file=self.log_path))
382
- return False
383
-
384
- self.job = log_types[log_type](self)
385
- if self.job.check():
386
- self.order = self.job.order
387
- self.definitions = self.job.definitions
388
- return True
389
- return False
390
-
391
- def _get_data(self):
392
- return self.job.get_data(self._get_raw_data())
393
-
394
-
395
-class Web:
396
- def __init__(self, service):
397
- self.service = service
398
- self.order = ORDER_WEB[:]
399
- self.definitions = deepcopy(CHARTS_WEB)
400
- self.pre_filter = check_patterns('filter', self.configuration.get('filter'))
401
- self.storage = dict()
402
- self.data = {
403
- 'bytes_sent': 0,
404
- 'resp_length': 0,
405
- 'resp_time_min': 0,
406
- 'resp_time_max': 0,
407
- 'resp_time_avg': 0,
408
- 'resp_time_upstream_min': 0,
409
- 'resp_time_upstream_max': 0,
410
- 'resp_time_upstream_avg': 0,
411
- 'unique_cur_ipv4': 0,
412
- 'unique_cur_ipv6': 0,
413
- '2xx': 0,
414
- '5xx': 0,
415
- '3xx': 0,
416
- '4xx': 0,
417
- '1xx': 0,
418
- '0xx': 0,
419
- 'unmatched': 0,
420
- 'req_ipv4': 0,
421
- 'req_ipv6': 0,
422
- 'unique_tot_ipv4': 0,
423
- 'unique_tot_ipv6': 0,
424
- 'successful_requests': 0,
425
- 'redirects': 0,
426
- 'bad_requests': 0,
427
- 'server_errors': 0,
428
- 'other_requests': 0,
429
- 'GET': 0
430
- }
431
-
432
- def __getattr__(self, item):
433
- return getattr(self.service, item)
434
-
435
- def check(self):
436
- last_line = read_last_line(self.log_path)
437
- if not last_line:
438
- return False
439
- # Custom_log_format or predefined log format.
440
- if self.configuration.get('custom_log_format'):
441
- match_dict, error = self.find_regex_custom(last_line)
442
- else:
443
- match_dict, error = self.find_regex(last_line)
444
-
445
- # "match_dict" is None if there are any problems
446
- if match_dict is None:
447
- self.error(error)
448
- return False
449
-
450
- self.storage['unique_all_time'] = list()
451
- self.storage['url_pattern'] = check_patterns('url_pattern', self.configuration.get('categories'))
452
- self.storage['user_pattern'] = check_patterns('user_pattern', self.configuration.get('user_defined'))
453
-
454
- self.create_web_charts(match_dict) # Create charts
455
- self.info('Collected data: %s' % list(match_dict.keys()))
456
- return True
457
-
458
- def create_web_charts(self, match_dict):
459
- """
460
- :param match_dict: dict: regex.search.groupdict(). Ex. {'address': '127.0.0.1', 'code': '200', 'method': 'GET'}
461
- :return:
462
- Create/remove additional charts depending on the 'match_dict' keys and configuration file options
463
- """
464
- if 'resp_time' not in match_dict:
465
- self.order.remove('response_time')
466
- self.order.remove('response_time_hist')
467
- if 'resp_time_upstream' not in match_dict:
468
- self.order.remove('response_time_upstream')
469
- self.order.remove('response_time_upstream_hist')
470
-
471
- # Add 'response_time_hist' and 'response_time_upstream_hist' charts if is specified in the configuration
472
- histogram = self.configuration.get('histogram', None)
473
- if isinstance(histogram, list):
474
- self.storage['bucket_index'] = histogram[:]
475
- self.storage['bucket_index'].append(maxint)
476
- self.storage['buckets'] = [0] * (len(histogram) + 1)
477
- self.storage['upstream_buckets'] = [0] * (len(histogram) + 1)
478
- hist_lines = self.definitions['response_time_hist']['lines']
479
- upstream_hist_lines = self.definitions['response_time_upstream_hist']['lines']
480
- for i, le in enumerate(histogram):
481
- hist_key = 'response_time_hist_%d' % i
482
- upstream_hist_key = 'response_time_upstream_hist_%d' % i
483
- hist_lines.append([hist_key, str(le), 'incremental', 1, 1])
484
- upstream_hist_lines.append([upstream_hist_key, str(le), 'incremental', 1, 1])
485
-
486
- hist_lines.append(['response_time_hist_%d' % len(histogram), '+Inf', 'incremental', 1, 1])
487
- upstream_hist_lines.append(['response_time_upstream_hist_%d' % len(histogram), '+Inf', 'incremental', 1, 1])
488
- elif histogram is not None:
489
- self.error('expect histogram list, but was {0}'.format(type(histogram)))
490
-
491
- if not self.configuration.get('all_time', True):
492
- self.order.remove('clients_all')
493
-
494
- # Add 'detailed_response_codes' chart if specified in the configuration
495
- if self.configuration.get('detailed_response_codes', True):
496
- if self.configuration.get('detailed_response_aggregate', True):
497
- codes = DET_RESP_AGGR[:1]
498
- else:
499
- codes = DET_RESP_AGGR[1:]
500
-
501
- for code in codes:
502
- self.order.append('detailed_response_codes%s' % code)
503
- self.definitions['detailed_response_codes%s' % code] = {
504
- 'options': [None, 'Detailed Response Codes %s' % code[1:], 'requests/s', 'responses',
505
- 'web_log.detailed_response_codes%s' % code, 'stacked'],
506
- 'lines': []
507
- }
508
-
509
- # Add 'requests_per_url' chart if specified in the configuration
510
- if self.storage['url_pattern']:
511
- for elem in self.storage['url_pattern']:
512
- dim = [elem.description, elem.description[12:], 'incremental']
513
- self.definitions['requests_per_url']['lines'].append(dim)
514
- self.data[elem.description] = 0
515
- self.data['url_pattern_other'] = 0
516
- else:
517
- self.order.remove('requests_per_url')
518
-
519
- # Add 'requests_per_user_defined' chart if specified in the configuration
520
- if self.storage['user_pattern'] and 'user_defined' in match_dict:
521
- for elem in self.storage['user_pattern']:
522
- dim = [elem.description, elem.description[13:], 'incremental']
523
- self.definitions['requests_per_user_defined']['lines'].append(dim)
524
- self.data[elem.description] = 0
525
- self.data['user_pattern_other'] = 0
526
- else:
527
- self.order.remove('requests_per_user_defined')
528
-
529
- def get_data(self, raw_data=None):
530
- """
531
- Parses new log lines
532
- :return: dict OR None
533
- None if _get_raw_data method fails.
534
- In all other cases - dict.
535
- """
536
- if not raw_data:
537
- return None if raw_data is None else self.data
538
-
539
- filtered_data = filter_data(raw_data=raw_data, pre_filter=self.pre_filter)
540
-
541
- unique_current = set()
542
- timings = defaultdict(lambda: dict(minimum=None, maximum=0, summary=0, count=0))
543
-
544
- for line in filtered_data:
545
- match = self.storage['regex'].search(line)
546
- if match:
547
- match_dict = match.groupdict()
548
- try:
549
- code = match_dict['code'][0] + 'xx'
550
- self.data[code] += 1
551
- except KeyError:
552
- self.data['0xx'] += 1
553
- # detailed response code
554
- if self.configuration.get('detailed_response_codes', True):
555
- self.get_data_per_response_codes_detailed(code=match_dict['code'])
556
- # response statuses
557
- self.get_data_per_statuses(code=match_dict['code'])
558
- # requests per user defined pattern
559
- if self.storage['user_pattern'] and 'user_defined' in match_dict:
560
- self.get_data_per_pattern(row=match_dict['user_defined'],
561
- other='user_pattern_other',
562
- pattern=self.storage['user_pattern'])
563
- # method, url, http version
564
- self.get_data_from_request_field(match_dict=match_dict)
565
- # bandwidth sent
566
- bytes_sent = match_dict['bytes_sent'] if '-' not in match_dict['bytes_sent'] else 0
567
- self.data['bytes_sent'] += int(bytes_sent)
568
- # request processing time and bandwidth received
569
- if 'resp_length' in match_dict:
570
- resp_length = match_dict['resp_length'] if '-' not in match_dict['resp_length'] else 0
571
- self.data['resp_length'] += int(resp_length)
572
- if 'resp_time' in match_dict:
573
- resp_time = self.storage['func_resp_time'](float(match_dict['resp_time']))
574
- get_timings(timings=timings['resp_time'], time=resp_time)
575
- if 'bucket_index' in self.storage:
576
- get_hist(self.storage['bucket_index'], self.storage['buckets'], resp_time / 1000)
577
- if 'resp_time_upstream' in match_dict and match_dict['resp_time_upstream'] != '-':
578
- resp_time_upstream = self.storage['func_resp_time'](float(match_dict['resp_time_upstream']))
579
- get_timings(timings=timings['resp_time_upstream'], time=resp_time_upstream)
580
- if 'bucket_index' in self.storage:
581
- get_hist(self.storage['bucket_index'], self.storage['upstream_buckets'], resp_time / 1000)
582
- # requests per ip proto
583
- proto = 'ipv6' if ':' in match_dict['address'] else 'ipv4'
584
- self.data['req_' + proto] += 1
585
- # unique clients ips
586
- if self.configuration.get('all_time', True):
587
- if address_not_in_pool(pool=self.storage['unique_all_time'],
588
- address=match_dict['address'],
589
- pool_size=self.data['unique_tot_ipv4'] + self.data['unique_tot_ipv6']):
590
- self.data['unique_tot_' + proto] += 1
591
- if match_dict['address'] not in unique_current:
592
- self.data['unique_cur_' + proto] += 1
593
- unique_current.add(match_dict['address'])
594
- else:
595
- self.data['unmatched'] += 1
596
-
597
- # timings
598
- for elem in timings:
599
- self.data[elem + '_min'] += timings[elem]['minimum']
600
- self.data[elem + '_avg'] += timings[elem]['summary'] / timings[elem]['count']
601
- self.data[elem + '_max'] += timings[elem]['maximum']
602
-
603
- # histogram
604
- if 'bucket_index' in self.storage:
605
- buckets = self.storage['buckets']
606
- upstream_buckets = self.storage['upstream_buckets']
607
- for i in range(0, len(self.storage['bucket_index'])):
608
- hist_key = 'response_time_hist_%d' % i
609
- upstream_hist_key = 'response_time_upstream_hist_%d' % i
610
- self.data[hist_key] = buckets[i]
611
- self.data[upstream_hist_key] = upstream_buckets[i]
612
-
613
- return self.data
614
-
615
- def find_regex(self, last_line):
616
- """
617
- :param last_line: str: literally last line from log file
618
- :return: tuple where:
619
- [0]: dict or None: match_dict or None
620
- [1]: str: error description
621
- We need to find appropriate pattern for current log file
622
- All logic is do a regex search through the string for all predefined patterns
623
- until we find something or fail.
624
- """
625
- # REGEX: 1.IPv4 address 2.HTTP method 3. URL 4. Response code
626
- # 5. Bytes sent 6. Response length 7. Response process time
627
- default = re.compile(r'(?P<address>[\da-f.:]+|localhost)'
628
- r' -.*?"(?P<request>[^"]*)"'
629
- r' (?P<code>[1-9]\d{2})'
630
- r' (?P<bytes_sent>\d+|-)')
631
-
632
- apache_ext_insert = re.compile(r'(?P<address>[\da-f.:]+|localhost)'
633
- r' -.*?"(?P<request>[^"]*)"'
634
- r' (?P<code>[1-9]\d{2})'
635
- r' (?P<bytes_sent>\d+|-)'
636
- r' (?P<resp_length>\d+|-)'
637
- r' (?P<resp_time>\d+) ')
638
-
639
- apache_ext_append = re.compile(r'(?P<address>[\da-f.:]+|localhost)'
640
- r' -.*?"(?P<request>[^"]*)"'
641
- r' (?P<code>[1-9]\d{2})'
642
- r' (?P<bytes_sent>\d+|-)'
643
- r' .*?'
644
- r' (?P<resp_length>\d+|-)'
645
- r' (?P<resp_time>\d+)'
646
- r'(?: |$)')
647
-
648
- nginx_ext_insert = re.compile(r'(?P<address>[\da-f.:]+)'
649
- r' -.*?"(?P<request>[^"]*)"'
650
- r' (?P<code>[1-9]\d{2})'
651
- r' (?P<bytes_sent>\d+)'
652
- r' (?P<resp_length>\d+)'
653
- r' (?P<resp_time>\d+\.\d+) ')
654
-
655
- nginx_ext2_insert = re.compile(r'(?P<address>[\da-f.:]+)'
656
- r' -.*?"(?P<request>[^"]*)"'
657
- r' (?P<code>[1-9]\d{2})'
658
- r' (?P<bytes_sent>\d+)'
659
- r' (?P<resp_length>\d+)'
660
- r' (?P<resp_time>\d+\.\d+)'
661
- r' (?P<resp_time_upstream>[\d.-]+)')
662
-
663
- nginx_ext_append = re.compile(r'(?P<address>[\da-f.:]+)'
664
- r' -.*?"(?P<request>[^"]*)"'
665
- r' (?P<code>[1-9]\d{2})'
666
- r' (?P<bytes_sent>\d+)'
667
- r' .*?'
668
- r' (?P<resp_length>\d+)'
669
- r' (?P<resp_time>\d+\.\d+)')
670
-
671
- def func_usec(time):
672
- return time
673
-
674
- def func_sec(time):
675
- return time * 1000000
676
-
677
- r_regex = [apache_ext_insert, apache_ext_append,
678
- nginx_ext2_insert, nginx_ext_insert, nginx_ext_append,
679
- default]
680
- r_function = [func_usec, func_usec, func_sec, func_sec, func_sec, func_usec]
681
- regex_function = zip(r_regex, r_function)
682
-
683
- match_dict = dict()
684
- for regex, func in regex_function:
685
- match = regex.search(last_line)
686
- if match:
687
- self.storage['regex'] = regex
688
- self.storage['func_resp_time'] = func
689
- match_dict = match.groupdict()
690
- break
691
-
692
- return find_regex_return(match_dict=match_dict or None,
693
- msg='Unknown log format. You need to use "custom_log_format" feature.')
694
-
695
- def find_regex_custom(self, last_line):
696
- """
697
- :param last_line: str: literally last line from log file
698
- :return: tuple where:
699
- [0]: dict or None: match_dict or None
700
- [1]: str: error description
701
-
702
- We are here only if "custom_log_format" is in logs. We need to make sure:
703
- 1. "custom_log_format" is a dict
704
- 2. "pattern" in "custom_log_format" and pattern is <str> instance
705
- 3. if "time_multiplier" is in "custom_log_format" it must be <int> or <float> instance
706
-
707
- If all parameters is ok we need to make sure:
708
- 1. Pattern search is success
709
- 2. Pattern search contains named subgroups (?P<subgroup_name>) (= "match_dict")
710
-
711
- If pattern search is success we need to make sure:
712
- 1. All mandatory keys ['address', 'code', 'bytes_sent', 'method', 'url'] are in "match_dict"
713
-
714
- If this is True we need to make sure:
715
- 1. All mandatory key values from "match_dict" have the correct format
716
- ("code" is integer, "method" is uppercase word, etc)
717
-
718
- If non mandatory keys in "match_dict" we need to make sure:
719
- 1. All non mandatory key values from match_dict ['resp_length', 'resp_time'] have the correct format
720
- ("resp_length" is integer or "-", "resp_time" is integer or float)
721
-
722
- """
723
- if not hasattr(self.configuration.get('custom_log_format'), 'keys'):
724
- return find_regex_return(msg='Custom log: "custom_log_format" is not a <dict>')
725
-
726
- pattern = self.configuration.get('custom_log_format', dict()).get('pattern')
727
- if not (pattern and isinstance(pattern, str)):
728
- return find_regex_return(msg='Custom log: "pattern" option is not specified or type is not <str>')
729
-
730
- resp_time_func = self.configuration.get('custom_log_format', dict()).get('time_multiplier') or 0
731
-
732
- if not isinstance(resp_time_func, (int, float)):
733
- return find_regex_return(msg='Custom log: "time_multiplier" is not an integer or a float')
734
-
735
- try:
736
- regex = re.compile(pattern)
737
- except re.error as error:
738
- return find_regex_return(msg='Pattern compile error: %s' % str(error))
739
-
740
- match = regex.search(last_line)
741
- if not match:
742
- return find_regex_return(msg='Custom log: pattern search FAILED')
743
-
744
- match_dict = match.groupdict() or None
745
- if match_dict is None:
746
- return find_regex_return(msg='Custom log: search OK but contains no named subgroups'
747
- ' (you need to use ?P<subgroup_name>)')
748
- mandatory_dict = {'address': r'[\w.:-]+',
749
- 'code': r'[1-9]\d{2}',
750
- 'bytes_sent': r'\d+|-'}
751
- optional_dict = {'resp_length': r'\d+|-',
752
- 'resp_time': r'[\d.]+',
753
- 'resp_time_upstream': r'[\d.-]+',
754
- 'method': r'[A-Z]+',
755
- 'http_version': r'\d(?:.\d)?'}
756
-
757
- mandatory_values = set(mandatory_dict) - set(match_dict)
758
- if mandatory_values:
759
- return find_regex_return(msg='Custom log: search OK but some mandatory keys (%s) are missing'
760
- % list(mandatory_values))
761
- for key in mandatory_dict:
762
- if not re.search(mandatory_dict[key], match_dict[key]):
763
- return find_regex_return(msg='Custom log: can\'t parse "%s": %s'
764
- % (key, match_dict[key]))
765
-
766
- optional_values = set(optional_dict) & set(match_dict)
767
- for key in optional_values:
768
- if not re.search(optional_dict[key], match_dict[key]):
769
- return find_regex_return(msg='Custom log: can\'t parse "%s": %s'
770
- % (key, match_dict[key]))
771
-
772
- dot_in_time = '.' in match_dict.get('resp_time', '')
773
- if dot_in_time:
774
- self.storage['func_resp_time'] = lambda time: time * (resp_time_func or 1000000)
775
- else:
776
- self.storage['func_resp_time'] = lambda time: time * (resp_time_func or 1)
777
-
778
- self.storage['regex'] = regex
779
- return find_regex_return(match_dict=match_dict)
780
-
781
- def get_data_from_request_field(self, match_dict):
782
- if match_dict.get('request'):
783
- match_dict = REQUEST_REGEX.search(match_dict['request'])
784
- if match_dict:
785
- match_dict = match_dict.groupdict()
786
- else:
787
- return
788
- # requests per url
789
- if match_dict.get('url') and self.storage['url_pattern']:
790
- self.get_data_per_pattern(row=match_dict['url'],
791
- other='url_pattern_other',
792
- pattern=self.storage['url_pattern'])
793
- # requests per http method
794
- if match_dict.get('method'):
795
- if match_dict['method'] not in self.data:
796
- self.charts['http_method'].add_dimension([match_dict['method'],
797
- match_dict['method'],
798
- 'incremental'])
799
- self.data[match_dict['method']] = 0
800
- self.data[match_dict['method']] += 1
801
- # requests per http version
802
- if match_dict.get('http_version'):
803
- dim_id = match_dict['http_version'].replace('.', '_')
804
- if dim_id not in self.data:
805
- self.charts['http_version'].add_dimension([dim_id,
806
- match_dict['http_version'],
807
- 'incremental'])
808
- self.data[dim_id] = 0
809
- self.data[dim_id] += 1
810
- # requests per port number
811
- if match_dict.get('port'):
812
- if match_dict['port'] not in self.data:
813
- self.charts['port'].add_dimension([match_dict['port'],
814
- match_dict['port'],
815
- 'incremental'])
816
- self.data[match_dict['port']] = 0
817
- self.data[match_dict['port']] += 1
818
- # requests per vhost
819
- if match_dict.get('vhost'):
820
- dim_id = match_dict['vhost'].replace('.', '_')
821
- if dim_id not in self.data:
822
- self.charts['vhost'].add_dimension([dim_id,
823
- match_dict['vhost'],
824
- 'incremental'])
825
- self.data[dim_id] = 0
826
- self.data[dim_id] += 1
827
-
828
- def get_data_per_response_codes_detailed(self, code):
829
- """
830
- :param code: str: CODE from parsed line. Ex.: '202, '499'
831
- :return:
832
- Calls add_new_dimension method If the value is found for the first time
833
- """
834
- if code not in self.data:
835
- if self.configuration.get('detailed_response_aggregate', True):
836
- self.charts['detailed_response_codes'].add_dimension([code, code, 'incremental'])
837
- self.data[code] = 0
838
- else:
839
- code_index = int(code[0]) if int(code[0]) < 6 else 6
840
- chart_key = 'detailed_response_codes' + DET_RESP_AGGR[code_index]
841
- self.charts[chart_key].add_dimension([code, code, 'incremental'])
842
- self.data[code] = 0
843
- self.data[code] += 1
844
-
845
- def get_data_per_pattern(self, row, other, pattern):
846
- """
847
- :param row: str:
848
- :param other: str:
849
- :param pattern: named tuple: (['pattern_description', 'regular expression'])
850
- :return:
851
- Scan through string looking for the first location where patterns produce a match for all user
852
- defined patterns
853
- """
854
- match = None
855
- for elem in pattern:
856
- if elem.func(row):
857
- self.data[elem.description] += 1
858
- match = True
859
- break
860
- if not match:
861
- self.data[other] += 1
862
-
863
- def get_data_per_statuses(self, code):
864
- """
865
- :param code: str: response status code. Ex.: '202', '499'
866
- :return:
867
- """
868
- code_class = code[0]
869
- if code_class == '2' or code == '304' or code_class == '1' or code == '401':
870
- self.data['successful_requests'] += 1
871
- elif code_class == '3':
872
- self.data['redirects'] += 1
873
- elif code_class == '4':
874
- self.data['bad_requests'] += 1
875
- elif code_class == '5':
876
- self.data['server_errors'] += 1
877
- else:
878
- self.data['other_requests'] += 1
879
-
880
-
881
-class ApacheCache:
882
- def __init__(self, service):
883
- self.service = service
884
- self.order = ORDER_APACHE_CACHE
885
- self.definitions = CHARTS_APACHE_CACHE
886
-
887
- @staticmethod
888
- def check():
889
- return True
890
-
891
- @staticmethod
892
- def get_data(raw_data=None):
893
- data = dict(hit=0, miss=0, other=0)
894
- if not raw_data:
895
- return None if raw_data is None else data
896
-
897
- for line in raw_data:
898
- if 'cache hit' in line:
899
- data['hit'] += 1
900
- elif 'cache miss' in line:
901
- data['miss'] += 1
902
- else:
903
- data['other'] += 1
904
- return data
905
-
906
-
907
-class Squid:
908
- def __init__(self, service):
909
- self.service = service
910
- self.order = ORDER_SQUID
911
- self.definitions = CHARTS_SQUID
912
- self.pre_filter = check_patterns('filter', self.configuration.get('filter'))
913
- self.storage = dict()
914
- self.data = {
915
- 'duration_max': 0,
916
- 'duration_avg': 0,
917
- 'duration_min': 0,
918
- 'bytes': 0,
919
- '0xx': 0,
920
- '1xx': 0,
921
- '2xx': 0,
922
- '3xx': 0,
923
- '4xx': 0,
924
- '5xx': 0,
925
- 'other': 0,
926
- 'unmatched': 0,
927
- 'unique_ipv4': 0,
928
- 'unique_ipv6': 0,
929
- 'unique_tot_ipv4': 0,
930
- 'unique_tot_ipv6': 0,
931
- 'successful_requests': 0,
932
- 'redirects': 0,
933
- 'bad_requests': 0,
934
- 'server_errors': 0,
935
- 'other_requests': 0
936
- }
937
-
938
- def __getattr__(self, item):
939
- return getattr(self.service, item)
940
-
941
- def check(self):
942
- last_line = read_last_line(self.log_path)
943
- if not last_line:
944
- return False
945
- self.storage['unique_all_time'] = list()
946
- self.storage['regex'] = re.compile(r'[0-9.]+\s+(?P<duration>[0-9]+)'
947
- r' (?P<client_address>[\da-f.:]+)'
948
- r' (?P<squid_code>[A-Z_]+)/'
949
- r'(?P<http_code>[0-9]+)'
950
- r' (?P<bytes>[0-9]+)'
951
- r' (?P<method>[A-Z_]+)'
952
- r' (?P<url>[^ ]+)'
953
- r' (?P<user>[^ ]+)'
954
- r' (?P<hier_code>[A-Z_]+)/[\da-z.:-]+'
955
- r' (?P<mime_type>[A-Za-z-]*)')
956
-
957
- match = self.storage['regex'].search(last_line)
958
- if not match:
959
- self.error('Regex not matches (%s)' % self.storage['regex'].pattern)
960
- return False
961
- self.storage['dynamic'] = {
962
- 'http_code': {
963
- 'chart': 'squid_detailed_response_codes',
964
- 'func_dim_id': None,
965
- 'func_dim': None
966
- },
967
- 'hier_code': {
968
- 'chart': 'squid_hier_code',
969
- 'func_dim_id': None,
970
- 'func_dim': lambda v: v.replace('HIER_', '')
971
- },
972
- 'method': {
973
- 'chart': 'squid_method',
974
- 'func_dim_id': None,
975
- 'func_dim': None
976
- },
977
- 'mime_type': {
978
- 'chart': 'squid_mime_type',
979
- 'func_dim_id': lambda v: str.lower(v) if str.lower(v) in MIME_TYPES else 'unknown',
980
- 'func_dim': None
981
- }
982
- }
983
- if not self.configuration.get('all_time', True):
984
- self.order.remove('squid_clients_all')
985
- return True
986
-
987
- def get_data(self, raw_data=None):
988
- if not raw_data:
989
- return None if raw_data is None else self.data
990
-
991
- filtered_data = filter_data(raw_data=raw_data, pre_filter=self.pre_filter)
992
-
993
- unique_ip = set()
994
- timings = defaultdict(lambda: dict(minimum=None, maximum=0, summary=0, count=0))
995
-
996
- for row in filtered_data:
997
- match = self.storage['regex'].search(row)
998
- if match:
999
- match = match.groupdict()
1000
- if match['duration'] != '0':
1001
- get_timings(timings=timings['duration'], time=float(match['duration']) * 1000)
1002
- try:
1003
- self.data[match['http_code'][0] + 'xx'] += 1
1004
- except KeyError:
1005
- self.data['other'] += 1
1006
-
1007
- self.get_data_per_statuses(match['http_code'])
1008
-
1009
- self.get_data_per_squid_code(match['squid_code'])
1010
-
1011
- self.data['bytes'] += int(match['bytes'])
1012
-
1013
- proto = 'ipv4' if '.' in match['client_address'] else 'ipv6'
1014
- # unique clients ips
1015
- if self.configuration.get('all_time', True):
1016
- if address_not_in_pool(pool=self.storage['unique_all_time'],
1017
- address=match['client_address'],
1018
- pool_size=self.data['unique_tot_ipv4'] + self.data['unique_tot_ipv6']):
1019
- self.data['unique_tot_' + proto] += 1
1020
-
1021
- if match['client_address'] not in unique_ip:
1022
- self.data['unique_' + proto] += 1
1023
- unique_ip.add(match['client_address'])
1024
-
1025
- for key, values in self.storage['dynamic'].items():
1026
- if match[key] == '-':
1027
- continue
1028
- dimension_id = values['func_dim_id'](match[key]) if values['func_dim_id'] else match[key]
1029
- if dimension_id not in self.data:
1030
- dimension = values['func_dim'](match[key]) if values['func_dim'] else dimension_id
1031
- self.charts[values['chart']].add_dimension([dimension_id,
1032
- dimension,
1033
- 'incremental'])
1034
- self.data[dimension_id] = 0
1035
- self.data[dimension_id] += 1
1036
- else:
1037
- self.data['unmatched'] += 1
1038
-
1039
- for elem in timings:
1040
- self.data[elem + '_min'] += timings[elem]['minimum']
1041
- self.data[elem + '_avg'] += timings[elem]['summary'] / timings[elem]['count']
1042
- self.data[elem + '_max'] += timings[elem]['maximum']
1043
- return self.data
1044
-
1045
- def get_data_per_statuses(self, code):
1046
- """
1047
- :param code: str: response status code. Ex.: '202', '499'
1048
- :return:
1049
- """
1050
- code_class = code[0]
1051
- if code_class == '2' or code == '304' or code_class == '1' or code == '000':
1052
- self.data['successful_requests'] += 1
1053
- elif code_class == '3':
1054
- self.data['redirects'] += 1
1055
- elif code_class == '4':
1056
- self.data['bad_requests'] += 1
1057
- elif code_class == '5' or code_class == '6':
1058
- self.data['server_errors'] += 1
1059
- else:
1060
- self.data['other_requests'] += 1
1061
-
1062
- def get_data_per_squid_code(self, code):
1063
- """
1064
- :param code: str: squid response code. Ex.: 'TCP_MISS', 'TCP_MISS_ABORTED'
1065
- :return:
1066
- """
1067
- if code not in self.data:
1068
- self.charts['squid_code'].add_dimension([code, code, 'incremental'])
1069
- self.data[code] = 0
1070
- self.data[code] += 1
1071
-
1072
- for tag in code.split('_'):
1073
- try:
1074
- chart_key = SQUID_CODES[tag]
1075
- except KeyError:
1076
- continue
1077
- dimension_id = '_'.join(['code_detailed', tag])
1078
- if dimension_id not in self.data:
1079
- self.charts[chart_key].add_dimension([dimension_id, tag, 'incremental'])
1080
- self.data[dimension_id] = 0
1081
- self.data[dimension_id] += 1
1082
-
1083
-
1084
-def get_timings(timings, time):
1085
- """
1086
- :param timings:
1087
- :param time:
1088
- :return:
1089
- """
1090
- if timings['minimum'] is None:
1091
- timings['minimum'] = time
1092
- if time > timings['maximum']:
1093
- timings['maximum'] = time
1094
- elif time < timings['minimum']:
1095
- timings['minimum'] = time
1096
- timings['summary'] += time
1097
- timings['count'] += 1
1098
-
1099
-
1100
-def get_hist(index, buckets, time):
1101
- """
1102
- :param index: histogram index (Ex. [10, 50, 100, 150, ...])
1103
- :param buckets: histogram buckets
1104
- :param time: time
1105
- :return: None
1106
- """
1107
- for i in range(len(index) - 1, -1, -1):
1108
- if time <= index[i]:
1109
- buckets[i] += 1
1110
- else:
1111
- break
1112
-
1113
-
1114
-def address_not_in_pool(pool, address, pool_size):
1115
- """
1116
- :param pool: list of ip addresses
1117
- :param address: ip address
1118
- :param pool_size: current pool size
1119
- :return: True if address not in pool. False otherwise.
1120
- """
1121
- index = bisect.bisect_left(pool, address)
1122
- if index < pool_size:
1123
- if pool[index] == address:
1124
- return False
1125
- bisect.insort_left(pool, address)
1126
- return True
1127
- bisect.insort_left(pool, address)
1128
- return True
1129
-
1130
-
1131
-def find_regex_return(match_dict=None, msg='Generic error message'):
1132
- """
1133
- :param match_dict: dict: re.search.groupdict() or None
1134
- :param msg: str: error description
1135
- :return: tuple:
1136
- """
1137
- return match_dict, msg
1138
-
1139
-
1140
-def check_patterns(string, dimension_regex_dict):
1141
- """
1142
- :param string: str:
1143
- :param dimension_regex_dict: dict: ex. {'dim1': '<pattern1>', 'dim2': '<pattern2>'}
1144
- :return: list of named tuples or None:
1145
- We need to make sure all patterns are valid regular expressions
1146
- """
1147
- if not hasattr(dimension_regex_dict, 'keys'):
1148
- return None
1149
-
1150
- result = list()
1151
-
1152
- def valid_pattern(pattern):
1153
- """
1154
- :param pattern: str
1155
- :return: re.compile(pattern) or None
1156
- """
1157
- if not isinstance(pattern, str):
1158
- return False
1159
- try:
1160
- return re.compile(pattern)
1161
- except re.error:
1162
- return False
1163
-
1164
- def func_search(pattern):
1165
- def closure(v):
1166
- return pattern.search(v)
1167
-
1168
- return closure
1169
-
1170
- for dimension, regex in dimension_regex_dict.items():
1171
- valid = valid_pattern(regex)
1172
- if isinstance(dimension, str) and valid_pattern:
1173
- func = func_search(valid)
1174
- result.append(NAMED_PATTERN(description='_'.join([string, dimension]),
1175
- func=func))
1176
- return result or None
1177
-
1178
-
1179
-def filter_data(raw_data, pre_filter):
1180
- """
1181
- :param raw_data:
1182
- :param pre_filter:
1183
- :return:
1184
- """
1185
-
1186
- if not pre_filter:
1187
- return raw_data
1188
- filtered = raw_data
1189
- for elem in pre_filter:
1190
- if elem.description == 'filter_include':
1191
- filtered = filter(elem.func, filtered)
1192
- elif elem.description == 'filter_exclude':
1193
- filtered = filterfalse(elem.func, filtered)
1194
- return filtered
collectors/python.d.plugin/web_log/web_log.conf
deleted
-219
@@ -1,219 +0,0 @@
1
-# netdata python.d.plugin configuration for web log
2
-#
3
-# This file is in YaML format. Generally the format is:
4
-#
5
-# name: value
6
-#
7
-# There are 2 sections:
8
-# - global variables
9
-# - one or more JOBS
10
-#
11
-# JOBS allow you to collect values from multiple sources.
12
-# Each source will have its own set of charts.
13
-#
14
-# JOB parameters have to be indented (using spaces only, example below).
15
-
16
-# ----------------------------------------------------------------------
17
-# Global Variables
18
-# These variables set the defaults for all JOBs, however each JOB
19
-# may define its own, overriding the defaults.
20
-
21
-# update_every sets the default data collection frequency.
22
-# If unset, the python.d.plugin default is used.
23
-# update_every: 1
24
-
25
-# priority controls the order of charts at the netdata dashboard.
26
-# Lower numbers move the charts towards the top of the page.
27
-# If unset, the default for python.d.plugin is used.
28
-# priority: 60000
29
-
30
-# penalty indicates whether to apply penalty to update_every in case of failures.
31
-# Penalty will increase every 5 failed updates in a row. Maximum penalty is 10 minutes.
32
-# penalty: yes
33
-
34
-# autodetection_retry sets the job re-check interval in seconds.
35
-# The job is not deleted if check fails.
36
-# Attempts to start the job are made once every autodetection_retry.
37
-# This feature is disabled by default.
38
-# autodetection_retry: 0
39
-
40
-# ----------------------------------------------------------------------
41
-# JOBS (data collection sources)
42
-#
43
-# The default JOBS share the same *name*. JOBS with the same name
44
-# are mutually exclusive. Only one of them will be allowed running at
45
-# any time. This allows autodetection to try several alternatives and
46
-# pick the one that works.
47
-#
48
-# Any number of jobs is supported.
49
-
50
-# ----------------------------------------------------------------------
51
-# PLUGIN CONFIGURATION
52
-#
53
-# All python.d.plugin JOBS (for all its modules) support a set of
54
-# predefined parameters. These are:
55
-#
56
-# job_name:
57
-# name: myname # the JOB's name as it will appear at the
58
-# # dashboard (by default is the job_name)
59
-# # JOBs sharing a name are mutually exclusive
60
-# update_every: 1 # the JOB's data collection frequency
61
-# priority: 60000 # the JOB's order on the dashboard
62
-# penalty: yes # the JOB's penalty
63
-# autodetection_retry: 0 # the JOB's re-check interval in seconds
64
-#
65
-# Additionally to the above, web_log also supports the following:
66
-#
67
-# path: 'PATH' # the path to web server log file
68
-# path: 'PATH[0-9]*[0-9]' # log files with date suffix are also supported
69
-# detailed_response_codes: yes/no # default: yes. Additional chart where response codes are not grouped
70
-# detailed_response_aggregate: yes/no # default: yes. Not aggregated detailed response codes charts
71
-# all_time : yes/no # default: yes. All time unique client IPs chart (50000 addresses ~ 400KB)
72
-# filter: # filter with regex
73
-# include: 'REGEX' # only those rows that matches the regex
74
-# exclude: 'REGEX' # all rows except those that matches the regex
75
-# categories: # requests per url chart configuration
76
-# cacti: 'cacti.*' # name(dimension): REGEX to match
77
-# observium: 'observium.*' # name(dimension): REGEX to match
78
-# stub_status: 'stub_status' # name(dimension): REGEX to match
79
-# user_defined: # requests per pattern in <user_defined> field (custom_log_format)
80
-# cacti: 'cacti.*' # name(dimension): REGEX to match
81
-# observium: 'observium.*' # name(dimension): REGEX to match
82
-# stub_status: 'stub_status' # name(dimension): REGEX to match
83
-# custom_log_format: # define a custom log format
84
-# pattern: '(?P<address>[\da-f.:]+) -.*?"(?P<method>[A-Z]+) (?P<url>.*?)" (?P<code>[1-9]\d{2}) (?P<bytes_sent>\d+) (?P<resp_length>\d+) (?P<resp_time>\d+\.\d+) '
85
-# time_multiplier: 1000000 # type <int>/<float> - convert time to microseconds
86
-# histogram: [1,3,10,30,100, ...] # type list of int - Cumulative histogram of response time in milli seconds
87
-
88
-# ----------------------------------------------------------------------
89
-# WEB SERVER CONFIGURATION
90
-#
91
-# Make sure the web server log directory and the web server log files
92
-# can be read by user 'netdata'.
93
-#
94
-# To enable the timings chart and the requests size dimension, the
95
-# web server needs to log them. This is how to add them:
96
-#
97
-# nginx:
98
-# log_format netdata '$remote_addr - $remote_user [$time_local] '
99
-# '"$request" $status $body_bytes_sent '
100
-# '$request_length $request_time $upstream_response_time '
101
-# '"$http_referer" "$http_user_agent"';
102
-# access_log /var/log/nginx/access.log netdata;
103
-#
104
-# apache (you need mod_logio enabled):
105
-# LogFormat "%h %l %u %t \"%r\" %>s %O %I %D \"%{Referer}i\" \"%{User-Agent}i\"" vhost_netdata
106
-# LogFormat "%h %l %u %t \"%r\" %>s %O %I %D \"%{Referer}i\" \"%{User-Agent}i\"" netdata
107
-# CustomLog "/var/log/apache2/access.log" netdata
108
-
109
-# ----------------------------------------------------------------------
110
-# VHOST AND PORT
111
-# if your want to graph the request/sec per virtual host and per port (to check the number of requests in http vs https)
112
-
113
-# in apache : (%v gives the hostname, %p the port number)
114
-# LogFormat "%v %p %h %t \"%r\" %>s %O %I %D \"%{Referer}i\" \"%{User-Agent}i\"" vhost_netdata
115
-#
116
-# and in this file in apache_vhosts_log section, add :
117
-# custom_log_format:
118
-# pattern: '(?P<vhost>[a-zA-Z\d.-_]+) (?P<port>\d+) (?P<address>[\da-f.:]+) \[.*\] "(?P<method>[A-Z]+)[^"]*" (?P<code>[1-9]\d{2}) (?P<bytes_sent>\d+) (?P<resp_length>\d+) (?P<resp_time>\d+)'
119
-
120
-# in nginx: ($host or $http_host give the hostname, $server_port the port number)
121
-# log_format netdatavhost '$host $server_port $remote_addr - $remote_user [$time_local] '
122
-# '"$request" $status $body_bytes_sent '
123
-# '$request_length $request_time $upstream_response_time '
124
-# '"$http_referer" "$http_user_agent"';
125
-#
126
-# access_log /var/log/nginx/access.log netdatavhost;
127
-#
128
-# be aware that the access_log directive in a server{} block overwrites the one in http{}, if your vhosts have individual log
129
-# files, you have to specify the general netdata log in each vhost as a second access_log statement.
130
-#
131
-# and in this file in nginx_log section, add :
132
-# custom_log_format:
133
-# pattern: '(?P<vhost>[a-zA-Z\d.-_\[\]]+) (?P<port>\d+) (?P<address>[\da-f.:]+) .* "(?P<method>[A-Z]+)[^"]*" (?P<code>[1-9]\d{2}) (?P<bytes_sent>\d+) (?P<resp_length>\d+) (?P<resp_time>\d+)'
134
-
135
-# ----------------------------------------------------------------------
136
-# AUTO-DETECTION JOBS
137
-# only one of them per web server will run (when they have the same name)
138
-
139
-
140
-# -------------------------------------------
141
-# nginx log on various distros
142
-
143
-# debian, arch
144
-nginx_log:
145
- name: 'nginx'
146
- path: '/var/log/nginx/access.log'
147
-
148
-# gentoo
149
-nginx_log2:
150
- name: 'nginx'
151
- path: '/var/log/nginx/localhost.access_log'
152
-
153
-
154
-# -------------------------------------------
155
-# apache log on various distros
156
-
157
-# debian
158
-apache_log:
159
- name: 'apache'
160
- path: '/var/log/apache2/access.log'
161
-
162
-# gentoo
163
-apache_log2:
164
- name: 'apache'
165
- path: '/var/log/apache2/access_log'
166
-
167
-# arch
168
-apache_log3:
169
- name: 'apache'
170
- path: '/var/log/httpd/access_log'
171
-
172
-# debian
173
-apache_vhosts_log:
174
- name: 'apache_vhosts'
175
- path: '/var/log/apache2/other_vhosts_access.log'
176
-
177
-
178
-# -------------------------------------------
179
-# gunicorn log on various distros
180
-
181
-gunicorn_log:
182
- name: 'gunicorn'
183
- path: '/var/log/gunicorn/access.log'
184
-
185
-gunicorn_log2:
186
- name: 'gunicorn'
187
- path: '/var/log/gunicorn/gunicorn-access.log'
188
-
189
-# -------------------------------------------
190
-# Apache Cache
191
-apache_cache:
192
- name: 'apache_cache'
193
- type: 'apache_cache'
194
- path: '/var/log/apache/cache.log'
195
-
196
-apache2_cache:
197
- name: 'apache_cache'
198
- type: 'apache_cache'
199
- path: '/var/log/apache2/cache.log'
200
-
201
-httpd_cache:
202
- name: 'apache_cache'
203
- type: 'apache_cache'
204
- path: '/var/log/httpd/cache.log'
205
-
206
-# -------------------------------------------
207
-# Squid
208
-
209
-# debian/ubuntu
210
-squid_log1:
211
- name: 'squid'
212
- type: 'squid'
213
- path: '/var/log/squid3/access.log'
214
-
215
-#gentoo
216
-squid_log2:
217
- name: 'squid'
218
- type: 'squid'
219
- path: '/var/log/squid/access.log'
docs/collect/application-metrics.md
+1
-1
@@ -36,7 +36,7 @@ Our most popular application collectors:
36
- [Nginx](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/nginx/): Monitor web server
37
status information by gathering metrics via `ngx_http_stub_status_module`.
38
- [Postgres](/collectors/python.d.plugin/postgres/README.md): Collect database health and performance metrics.
39
-- [ElasticSearch](/collectors/python.d.plugin/elasticsearch/README.md): Collect search engine performance and health
39
+- [ElasticSearch](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/elasticsearch): Collect search engine performance and health
40
statistics. Optionally collects per-index metrics.
41
- [PHP-FPM](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/phpfpm/): Collect application
42
summary and processes health metrics by scraping the status page (`/status?full`).
docs/guides/collect-apache-nginx-web-logs.md
+4
-39
@@ -12,15 +12,11 @@ By parsing web server log files with Netdata, and seeing the volume of redirects
12
you can better understand what's happening on your infrastructure. Too many bad requests? Maybe a recent deploy missed a
13
few small SVG icons. Too many requests? Time to batten down the hatches—it's a DDoS.
14
15
-Netdata has been capable of monitoring web log files for quite some time, thanks for the [weblog python.d
16
-module](/collectors/python.d.plugin/web_log/README.md), but we recently refactored this module in Go, and that effort
17
-comes with a ton of improvements.
18
-
19
-You can now use the [LTSV log format](http://ltsv.org/), track TLS and cipher usage, and the whole parser is faster than
15
+You can use the [LTSV log format](http://ltsv.org/), track TLS and cipher usage, and the whole parser is faster than
16
ever. In one test on a system with SSD storage, the collector consistently parsed the logs for 200,000 requests in
21
-200ms, using ~30% of a single core. To learn more about these improvements, see our [v1.19 release post](https://blog.netdata.cloud/posts/release-1.19/).
17
+200ms, using ~30% of a single core.
18
23
-The [go.d plugin](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/) is currently compatible
19
+The [web_log](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/) collector is currently compatible
20
with [Nginx](https://nginx.org/en/) and [Apache](https://httpd.apache.org/).
21
22
This guide will walk you through using the new Go-based web log collector to turn the logs these web servers
@@ -34,33 +30,6 @@ installation procedures.
30
Almost all web server installations will need _no_ configuration to start collecting metrics. As long as your web server
31
has readable access log file, you can configure the web log plugin to access and parse it.
32
37
-## Configure the web log collector
38
-
39
-To use the Go version of this plugin, you need to explicitly enable it, and disable the deprecated Python version.
40
-First, open `python.d.conf`:
41
-
42
-```bash
43
-cd /etc/netdata/ # Replace with your Netdata configuration directory, if not /etc/netdata/
44
-./edit-config python.d.conf
45
-```
46
-
47
-Find the `web_log` line, uncomment it, and set it to `web_log: no`. Next, open the `go.d.conf` file for editing.
48
-
49
-```bash
50
-./edit-config go.d.conf
51
-```
52
-
53
-Find the `web_log` line again, uncomment it, and set it to `web_log: yes`.
54
-
55
-Finally, restart Netdata with `sudo systemctl restart netdata`, or the [appropriate
56
-method](/docs/configure/start-stop-restart.md) for your system. You should see metrics in your Netdata dashboard!
57
-
58
-
60
-
61
-If you don't see web log charts, or **web log nginx**/**web log apache** menus on the right-hand side of your dashboard,
62
-continue reading for other configuration options.
63
-
33
## Custom configuration of the web log collector
34
35
The web log collector's default configuration comes with a few example jobs that should cover most Linux distributions
@@ -152,11 +121,7 @@ documentation](/health/README.md).
121
122
## What's next?
123
155
-Now that you have web log collection up and running, we recommend you take a look at the documentation for our
156
-[python.d](/collectors/python.d.plugin/web_log/README.md) for some ideas of how you can turn these rather "boring" logs
157
-into powerful real-time tools for keeping your servers happy.
124
+Now that you have web log collection up and running, we recommend you take a look at the collector's [documentation](https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/weblog/) for some ideas of how you can turn these rather "boring" logs into powerful real-time tools for keeping your servers happy.
125
126
Don't forget to give GitHub user [Wing924](https://github.com/Wing924) a big 👍 for his hard work in starting up the Go
127
refactoring effort.
161
-
162
-
web/gui/dashboard_info.js
+4
-4
@@ -427,7 +427,7 @@ netdataDashboard.menu = {
427
'web_log': {
428
title: undefined,
429
icon: '<i class="fas fa-file-alt"></i>',
430
- info: 'Information extracted from a server log file. <code>web_log</code> plugin incrementally parses the server log file to provide, in real-time, a break down of key server performance metrics. For web servers, an extended log file format may optionally be used (for <code>nginx</code> and <code>apache</code>) offering timing information and bandwidth for both requests and responses. <code>web_log</code> plugin may also be configured to provide a break down of requests per URL pattern (check <a href="https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/web_log/web_log.conf" target="_blank"><code>/etc/netdata/python.d/web_log.conf</code></a>).'
430
+ info: 'Information extracted from a server log file. <code>web_log</code> plugin incrementally parses the server log file to provide, in real-time, a break down of key server performance metrics. For web servers, an extended log file format may optionally be used (for <code>nginx</code> and <code>apache</code>) offering timing information and bandwidth for both requests and responses. <code>web_log</code> plugin may also be configured to provide a break down of requests per URL pattern (check <a href="https://github.com/netdata/go.d.plugin/blob/master/config/go.d/web_log.conf" target="_blank"><code>/etc/netdata/go.d/web_log.conf</code></a>).'
431
},
432
433
'squid': {
@@ -739,7 +739,7 @@ netdataDashboard.submenu = {
739
},
740
741
'web_log.urls': {
742
- info: 'Number of requests for each <code>URL pattern</code> defined in <a href="https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/web_log/web_log.conf" target="_blank"><code>/etc/netdata/python.d/web_log.conf</code></a>. This chart counts all requests matching the URL patterns defined, independently of the web server response codes (i.e. both successful and unsuccessful).'
742
+ info: 'Number of requests for each <code>URL pattern</code> defined in <a href="https://github.com/netdata/go.d.plugin/blob/master/config/go.d/web_log.conf" target="_blank"><code>/etc/netdata/go.d/web_log.conf</code></a>. This chart counts all requests matching the URL patterns defined, independently of the web server response codes (i.e. both successful and unsuccessful).'
743
},
744
745
'web_log.clients': {
@@ -4938,7 +4938,7 @@ netdataDashboard.context = {
4938
},
4939
4940
'web_log.clients_all': {
4941
- info: 'Unique client IPs accessing the web server since the last restart of netdata. This plugin keeps in memory all the unique IPs that have accessed the web server. On very busy web servers (several millions of unique IPs) you may want to disable this chart (check <a href="https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/web_log/web_log.conf" target="_blank"><code>/etc/netdata/python.d/web_log.conf</code></a>).'
4941
+ info: 'Unique client IPs accessing the web server since the last restart of netdata. This plugin keeps in memory all the unique IPs that have accessed the web server. On very busy web servers (several millions of unique IPs) you may want to disable this chart (check <a href="https://github.com/netdata/go.d.plugin/blob/master/config/go.d/web_log.conf" target="_blank"><code>/etc/netdata/go.d/web_log.conf</code></a>).'
4942
},
4943
4944
// ------------------------------------------------------------------------
@@ -5069,7 +5069,7 @@ netdataDashboard.context = {
5069
},
5070
5071
'web_log.squid_clients_all': {
5072
- info: 'Unique client IPs accessing squid since the last restart of netdata. This plugin keeps in memory all the unique IPs that have accessed the server. On very busy squid servers (several millions of unique IPs) you may want to disable this chart (check <a href="https://github.com/netdata/netdata/blob/master/collectors/python.d.plugin/web_log/web_log.conf" target="_blank"><code>/etc/netdata/python.d/web_log.conf</code></a>).'
5072
+ info: 'Unique client IPs accessing squid since the last restart of netdata. This plugin keeps in memory all the unique IPs that have accessed the server. On very busy squid servers (several millions of unique IPs) you may want to disable this chart (check <a href="https://github.com/netdata/go.d.plugin/blob/master/config/go.d/web_log.conf" target="_blank"><code>/etc/netdata/go.d/web_log.conf</code></a>).'
5073
},
5074
5075
'web_log.squid_transport_methods': {