Another Readme Update (#4612)
* updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme * updated readme
Costa Tsaousis committed
Nov 12, 2018 at 04:56 UTC
10a919ccfa4a1c471ddc38d93f8ac057942db9ea
1 file changed
+50
-100
README.md
+50
-100
@@ -4,14 +4,19 @@
4
5
---
6
7
-**Netdata** is a system for **distributed real-time performance and health monitoring**.
8
-
9
-It provides **unparalleled insights**, **in real-time**, of everything happening on the systems it runs (including containers and applications such as web and database servers), using **modern interactive web dashboards**.
7
+**Netdata** is **distributed, real-time, performance and health monitoring for systems and applications**. It is based on a powerful monitoring agent you install on all your systems and containers.
8
+
9
+Netdata provides **unparalleled insights**, **in real-time**, of everything happening on the systems it runs (including web servers, databases, applications), using **highly interactive web dashboards**. It can run autonomously, without any third party components, or it can be integrated to existing monitoring toolchains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
10
11
_Netdata is **fast** and **efficient**, designed to permanently run on all systems (**physical** & **virtual** servers, **containers**, **IoT** devices), without disrupting their core function._
12
-
12
+
13
Netdata is **free, open-source software** and it currently runs on **Linux**, **FreeBSD**, and **MacOS**.
14
15
+
16
+
17
+Netdata is in the [Cloud Native Computing Foundation (CNCF) landscape](https://landscape.cncf.io/grouping=no&sort=stars).
18
+Check the [CNCF TOC Netdata presentation](https://docs.google.com/presentation/d/18C8bCTbtgKDWqPa57GXIjB2PbjjpjsUNkLtZEz6YK8s/edit?usp=sharing).
19
+
20
---
21
22
People get **addicted to netdata**.<br/>
@@ -21,6 +26,7 @@ Once you use it on your systems, **there is no going back**! *You have been warn
26
27
[](https://twitter.com/intent/tweet?text=Netdata,%20real-time%20performance%20and%20health%20monitoring,%20done%20right!&url=https://my-netdata.io/&via=linuxnetdata&hashtags=netdata,monitoring)
28
29
+
30
## Contents
31
32
1. [How it looks](#how-it-looks) - have a quick look at it
@@ -28,13 +34,14 @@ Once you use it on your systems, **there is no going back**! *You have been warn
34
3. [Quick Start](#quick-start) - try it now on your systems
35
4. [Why Netdata](#why-netdata) - why people love netdata, how it compares with other solutions
36
5. [News](#news) - latest news about netdata
31
-6. [infographic](#infographic) - everything about netdata, in a page
32
-7. [Features](#features) - what features does it have
33
-8. [Visualization](#visualization) - unique visualization features
34
-9. [What does it monitor](#what-does-it-monitor) - which metrics it collects
35
-10. [Documentation](#documentation) - read the docs
36
-11. [Community](#community) - disucss with others and get support
37
-12. [License](#license) - check the license of netdata
37
+6. [How it works](#how-it-works) - high level diagram of how netdata works
38
+7. [infographic](#infographic) - everything about netdata, in a page
39
+8. [Features](#features) - what features does it have
40
+9. [Visualization](#visualization) - unique visualization features
41
+10. [What does it monitor](#what-does-it-monitor) - which metrics it collects
42
+11. [Documentation](#documentation) - read the docs
43
+12. [Community](#community) - disucss with others and get support
44
+13. [License](#license) - check the license of netdata
45
46
47
## How it looks
@@ -45,27 +52,26 @@ The following animated image, shows the top part of a typical netdata dashboard.
52
53
*A typical netdata dashboard, in 1:1 timing. Charts can be panned by dragging them, zoomed in/out with `SHIFT` + `mouse wheel`, an area can be selected for zoom-in with `SHIFT` + `mouse selection`. Netdata is highly interactive and **real-time**, optimized to get the work done!*
54
48
-> *We have a few online demos to check: [http://my-netdata.io](http://my-netdata.io)*
55
+> *We have a few online demos to see it in action: [http://my-netdata.io](http://my-netdata.io)*
56
57
## User base
58
52
-
53
-
54
-Netdata is in the [Cloud Native Computing Foundation (CNCF) landscape](https://landscape.cncf.io/grouping=no&sort=stars).
55
-Check the [CNCF TOC Netdata presentation](https://docs.google.com/presentation/d/18C8bCTbtgKDWqPa57GXIjB2PbjjpjsUNkLtZEz6YK8s/edit?usp=sharing).
56
-
57
-Netdata is a **robust** application. It has hundreds of thousands of users, all over the world.
59
+Netdata is used by hundreds of thousands of users all over the world.
60
Check our [GitHub watchers list](https://github.com/netdata/netdata/watchers).
59
-You will find users working for: **Amazon**, **Atos**, **Baidu**, **Cisco Systems**, **Citrix**, **Deutsche Telekom**, **DigitalOcean**,
61
+You will find people working for **Amazon**, **Atos**, **Baidu**, **Cisco Systems**, **Citrix**, **Deutsche Telekom**, **DigitalOcean**,
62
**Elastic**, **EPAM Systems**, **Ericsson**, **Google**, **Groupon**, **Hortonworks**, **HP**, **Huawei**,
63
**IBM**, **Microsoft**, **NewRelic**, **Nvidia**, **Red Hat**, **SAP**, **Selectel**, **TicketMaster**,
64
**Vimeo**, and many more!
65
64
-#### Docker pulls
65
-Docker pulls as reported by docker hub:<br/>[](https://hub.docker.com/r/netdata/netdata/) [](https://hub.docker.com/r/firehol/netdata/) [](https://hub.docker.com/r/titpetric/netdata/)
66
+### Docker pulls
67
+We provide docker images for the most common architectures. These are statistics reported by docker hub:
68
67
-#### Anonymous global public netdata registry
68
-*Since May 16th 2016 (the date the [global public netdata registry](https://github.com/netdata/netdata/wiki/mynetdata-menu-item) was released):*<br/>[](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
69
+[](https://hub.docker.com/r/netdata/netdata/) [](https://hub.docker.com/r/firehol/netdata/) [](https://hub.docker.com/r/titpetric/netdata/)
70
+
71
+### Registry
72
+When you install multiple netdata, they are integrated into **one distributed application**, via a [netdata registry](https://github.com/netdata/netdata/wiki/mynetdata-menu-item). This is a web browser feature and it allows us to count the number of unique users and unique netdata servers installed. The following information comes from the global public netdata registry we run:
73
+
74
+[](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
75
76
*in the last 24 hours:*<br/> [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
77
@@ -124,80 +130,6 @@ Netdata is **open-source**, **free**, super **fast**, very **easy**, completely
130
It has been designed by **SysAdmins**, **DevOps** and **Developers** for troubleshooting performance problems,
131
not just visualize metrics.
132
127
-### Simplicity
128
-
129
-> Most monitoring solutions require endless configuration of whatever imaginable.
130
-
131
-Well... this is a linux box. Why do we need to configure every single metric we need to monitor.
132
-Of course it has a CPU and RAM and a few disks, and ethernet ports, it may run a firewall, a web server, or a database server and so on.
133
-Why do we need to configure all these metrics?
134
-
135
-Netdata metrics collection is designed to support **configuration-less** operation. So, you just install and run netdata.
136
-You will need to configure something only if it cannot be auto-detected.
137
-
138
-Of course you can enable, tweak or disable things.
139
-But by default, if netdata can connect to a web server you run on your systems, it will automatically
140
-collect all performance metrics. This happens for all data collection plugins when technically possible.
141
-It will also automatically collect all available system values for CPU, memory, disks, network interfaces,
142
-QoS (with labels if you also use [FireQOS](http://firehol.org/)), etc.
143
-Even for processes that do not offer performance metrics, it will automatically group the whole process
144
-tree and provide metrics like CPU usage, memory allocated, opened files, sockets, disk activity, swap
145
-activity, etc per application group.
146
-
147
-### Performance monitoring
148
-
149
-According to reports, performance issues are 10x more common compared to outages.
150
-
151
-> Take any performance monitoring solution and try to troubleshoot a performance problem.
152
-> At the end of the day you will have to `ssh` to the server(s) to understand what exactly is happening.
153
-> You will have to use `iostat`, `iotop`, `vmstat`, `top`, `ethtool` and probably a few dozen more console tools to figure
154
-> out the problem.
155
-
156
-With netdata, this need is eliminated significantly. Of course you will ssh. Just not for monitoring performance.
157
-
158
-One key parameter to effectively troubleshoot performance issues, is that the root cause is most probably unknown.
159
-If you were aware of the element that affected performance, most probably you would have fixed it already.
160
-
161
-The approach of most monitoring solutions (including commercial SaaS providers) that instruct their users and customers
162
-to collect only the metrics they understand, is contradictory to the nature of performance monitoring. If we knew the metrics
163
-before hand, most probably we would have a lot less performance issues.
164
-
165
-So, Netdata collects everything. The more metrics collected, the more insights we will have when we need them.
166
-
167
-Netdata is better than most console tools. Netdata visualizes the data, while the console tools just show their values.
168
-The detail is the same. Actually, netdata is more precise than most console tools,
169
-it will interpolate all collected values to second boundary, so that even if something took a few microseconds more to be
170
-collected, netdata will correctly estimate the per second rate.
171
-
172
-### Realtime monitoring
173
-
174
-> Any performance monitoring solution that does not go down to per second collection and visualization of the data,
175
-> is useless. It will make you happy to have it, but it will not help you more than that.
176
-
177
-Visualizing the present in **real-time and in great detail**, is the most important value a performance monitoring
178
-solution should provide. The next most important is the last hour, again per second. The next is the last 8 hours
179
-and so on, up to a week. In my 20+ years in IT, I needed just once or twice to look a year back. And this was mainly
180
-out of curiosity.
181
-
182
-Of course, real-time monitoring requires resources. So netdata is extremely optimized to be very efficient:
183
-
184
-- collecting performance data is a repeating process - you do the same thing again and again.
185
- Netdata has been designed to learn from each iteration, so that the next one will be faster.
186
- It learns the sizes of files (it even keeps them open when it can), the number of lines and
187
- words per line they contain, the sizes of the buffers it needs to process them, etc.
188
- It adapts, so that everything will be as ready as possible for the next iteration.
189
-
190
-- internally, it uses hashes and indexes (b-trees), to speed up lookups of metrics, charts, dimensions, settings.
191
-
192
-- it has an in-memory round robin database based on a custom floating point number that allows it to pack values
193
- and flags together, in 32 bits, to lower its memory footprint.
194
-
195
-- its internal web server is capable of generating JSON responses from live performance data with speeds comparable
196
- to static content delivery (it does not use printf, it is actually 11 times faster than in generating JSON compared
197
- to printf).
198
-
199
-Netdata will use some CPU and memory, but it will not produce any disk I/O at all, apart its logs (which you can disable if you like).
200
-
133
134
## News
135
@@ -223,7 +155,25 @@ Netdata used to be a [firehol.org](https://firehol.org) project, accessible as `
155
156
Netdata now has its own github organization `netdata`, so all github URLs are now `netdata/netdata`. The old github URLs, repo clones, forks, etc redirect automatically to the new repo.
157
226
-
158
+## How it works
159
+
160
+Netdata is a highly efficient, highly modular, metrics management engine. Its lockless design makes it ideal for concurrent operations on the metrics.
161
+
162
+
163
+
164
+This is how it works:
165
+
166
+Function|Description|Documentation
167
+:---:|:---|:---:
168
+**Collect**|Multiple independent data collection workers are collecting metrics from their sources using the optimal protocol for each application and push the metrics to the database. Each data collection worker has lockless write access to the metrics it collects.|[Collectors](https://github.com/netdata/netdata/tree/master/collectors#data-collection-plugins)
169
+**Store**|Metrics are stored in RAM in a round robin database (ring buffer), using a custom made floating point number for minimal footprint.|[Database](https://github.com/netdata/netdata/tree/master/database#netdata-database)
170
+**Check**|A lockless independent watchdog is evaluating **health checks** on the collected metrics, triggers alarms, maintains a health transaction log and dispatches alarm notifications.|[Health](https://github.com/netdata/netdata/tree/master/health#health-monitoring)
171
+**Stream**|An lockless independent worker is streaming metrics, in full detail and in real-time, to remote netdata servers, as soon as they are collected.|[Streaming](https://github.com/netdata/netdata/tree/master/streaming#metrics-streaming)
172
+**Archieve**|A lockless independent worker is down-sampling the metrics and pushes them to **backend** time-series databases.|[Backends](https://github.com/netdata/netdata/tree/master/backends)
173
+**Query**|Multiple independent workers are attached to the [internal web server](https://github.com/netdata/netdata/tree/master/web/server#netdata-web-server), servicing API requests, including [data queries](https://github.com/netdata/netdata/tree/master/web/api/queries#database-queries).|[API](https://github.com/netdata/netdata/tree/master/web/api#api)
174
+
175
+The result is a highly efficient, low latency system, supporting multiple readers and one writer on each metric.
176
+
177
## Infographic
178
179
This is a high level overview of netdata feature set and architecture.
@@ -274,7 +224,7 @@ To improve clarity on charts, netdata dashboards present **positive** values for
224
225
*Netdata charts showing the bandwidth and packets of a network interface. `received` is positive and `sent` is negative.*
226
277
-### Non zero-based y-axis
227
+### Autoscaled y-axis
228
229
Netdata charts automatically zoom vertically, to visualize the variation of each metric within the visible time-frame.
230
@@ -372,7 +322,6 @@ Its [Plugin API](collectors/plugins.d) supports all programing languages (anythi
322
- **AP** - collects Linux access point performance data (`hostapd`).
323
- **SNMP** - SNMP devices can be monitored too (although you will need to configure these).
324
- **port_check** - checks TCP ports for availability and response time.
375
-- **IPVS** - collects metrics from the Linux IPVS load balancer.
325
- **LibreSwan** - collects metrics per IPSEC tunnel.
326
327
#### Processes
@@ -404,6 +353,7 @@ Its [Plugin API](collectors/plugins.d) supports all programing languages (anythi
353
- **Squid** - multiple servers, each showing: clients bandwidth and requests, servers bandwidth and requests.
354
- **Traefik** - connects to multiple traefik instances (local or remote) to collect API metrics (response status code, response time, average response time and server uptime).
355
- **Varnish** - threads, sessions, hits, objects, backends, etc.
356
+- **IPVS** - collects metrics from the Linux IPVS load balancer.
357
358
#### Database Servers
359
- **CouchDB** - reads/writes, request methods, status codes, tasks, replication, per-db, etc.