@cryptotaxi247 / netdata-1 / commits / 44ae8f620

Remove distributed-data-architecture.md and omit mentions to it (#17097)

Fotis Voutsas committed Mar 4, 2024 at 13:36 UTC 44ae8f6204f6d41d8047b82b760399f3daa5e6bc
4 files changed +2 -81
docs/glossary.md
-2
@@ -53,8 +53,6 @@ Use the alphabatized list below to find the answer to your single-term questions
53
54 - [**Dimension**](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/interact-new-charts.md#dimensions): A dimension is a value that gets shown on a chart.
55
56 -- [**Distributed Architecture**](https://github.com/netdata/netdata/blob/master/docs/store/distributed-data-architecture.md): The data architecture mindset with which Netdata was built, where all data are collected and stored on the edge, whenever it's possible, creating countless benefits.
57 -
56 ## E
57
58 - [**External Plugins**](https://github.com/netdata/netdata/blob/master/src/collectors/plugins.d/README.md): These gather metrics from external processes, such as a webserver or database, and run as independent processes that communicate with the Netdata daemon via pipes.
docs/metrics-storage-management/enable-streaming.md
+1 -3
@@ -10,9 +10,7 @@ When one node streams metrics to another, the node receiving metrics can visuali
10 [export](https://github.com/netdata/netdata/blob/master/docs/export/external-databases.md) all metrics to an external TSDB. When Netdata streams metrics to another
11 Netdata, the receiving one is able to perform everything a Netdata instance is capable of.
12
13 -Streaming lets you decide exactly how you want to store and maintain metrics data. While we believe Netdata's
14 -[distributed architecture](https://github.com/netdata/netdata/blob/master/docs/store/distributed-data-architecture.md) is
15 -ideal for speed and scale, streaming provides centralization options and high data availability.
13 +Streaming lets you decide exactly how you want to store and maintain metrics data. While we believe Netdata's distributed architecture is ideal for speed and scale, streaming provides centralization options and high data availability.
14
15 This document will get you started quickly with streaming. More advanced concepts and suggested production deployments
16 can be found in the [streaming and replication reference](https://github.com/netdata/netdata/blob/master/src/streaming/README.md).
docs/quickstart/infrastructure.md
+1 -1
@@ -10,7 +10,7 @@ nodes running the Netdata Agent. A node is any system in your infrastructure tha
10 physical or virtual machine (VM), container, cloud deployment, or edge/IoT device.
11
12 The Netdata Agent uses zero-configuration collectors to gather metrics from every application and container instantly,
13 -and uses Netdata's [distributed data architecture](https://github.com/netdata/netdata/blob/master/docs/store/distributed-data-architecture.md) to store metrics
13 +and uses Netdata's distributed data architecture to store metrics
14 locally. Without a slow and troublesome centralized data lake for your infrastructure's metrics, you reduce the
15 resources you need to invest in, and the complexity of, monitoring your infrastructure.
16
docs/store/distributed-data-architecture.md deleted
-75
@@ -1,75 +0,0 @@
1 -# Distributed data architecture
2 -
3 -Learn how Netdata's distributed data architecture enables us to store metrics on the edge nodes for security, high performance and scalability.
4 -
5 -This way, it helps you collect and store per-second metrics from any number of nodes.
6 -Every node in your infrastructure, whether it's one or a thousand, stores the metrics it collects.
7 -
8 -Netdata Cloud bridges the gap between many distributed databases by _centralizing the interface_ you use to query and
9 -visualize your nodes' metrics. When you [look at charts in Netdata Cloud](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/interact-new-charts.md)
10 -, the metrics values are queried directly from that node's database and securely streamed to Netdata Cloud, which
11 -proxies them to your browser.
12 -
13 -Netdata's distributed data architecture has a number of benefits:
14 -
15 -- **Performance**: Every query to a node's database takes only a few milliseconds to complete for responsiveness when
16 - viewing dashboards or using features
17 - like [Metric Correlations](https://github.com/netdata/netdata/blob/master/docs/cloud/insights/metric-correlations.md).
18 -- **Scalability**: As your infrastructure scales, install the Netdata Agent on every new node to immediately add it to
19 - your monitoring solution without adding cost or complexity.
20 -- **1-second granularity**: Without an expensive centralized data lake, you can store all of your nodes' per-second
21 - metrics, for any period of time, while keeping costs down.
22 -- **No filtering or selecting of metrics**: Because Netdata's distributed data architecture allows you to store all
23 - metrics, you don't have to configure which metrics you retain. Keep everything for full visibility during
24 - troubleshooting and root cause analysis.
25 -- **Easy maintenance**: There is no centralized data lake to purchase, allocate, monitor, and update, removing
26 - complexity from your monitoring infrastructure.
27 -
28 -## Ephemerality of metrics
29 -
30 -The ephemerality of metrics plays an important role in retention. In environments where metrics collection is dynamic and
31 -new metrics are constantly being generated, we are interested about 2 parameters:
32 -
33 -1. The **expected concurrent number of metrics** as an average for the lifetime of the database. This affects mainly the
34 - storage requirements.
35 -
36 -2. The **expected total number of unique metrics** for the lifetime of the database. This affects mainly the memory
37 - requirements for having all these metrics indexed and available to be queried.
38 -
39 -## Granularity of metrics
40 -
41 -The granularity of metrics (the frequency they are collected and stored, i.e. their resolution) is significantly
42 -affecting retention.
43 -
44 -Lowering the granularity from per second to every two seconds, will double their retention and half the CPU requirements
45 -of the Netdata Agent, without affecting disk space or memory requirements.
46 -
47 -## Long-term metrics storage with Netdata
48 -
49 -Any node running the Netdata Agent can store long-term metrics for any retention period, given you allocate the
50 -appropriate amount of RAM and disk space.
51 -
52 -Read our document on changing [how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md) on your nodes for
53 -details.
54 -
55 -You can also stream between nodes using [streaming](https://github.com/netdata/netdata/blob/master/src/streaming/README.md), allowing to replicate databases and create
56 -your own centralized data lake of metrics, if you choose to do so.
57 -
58 -While a distributed data architecture is the default when monitoring infrastructure with Netdata, you can also configure
59 -its behavior based on your needs or the type of infrastructure you manage.
60 -
61 -To archive metrics to an external time-series database, such as InfluxDB, Graphite, OpenTSDB, Elasticsearch,
62 -TimescaleDB, and many others, see details on [integrating Netdata via exporting](https://github.com/netdata/netdata/blob/master/docs/export/external-databases.md).
63 -
64 -When you use the database engine to store your metrics, you can always perform a quick backup of a node's
65 -`/var/cache/netdata/dbengine/` folder using the tool of your choice.
66 -
67 -## Does Netdata Cloud store my metrics?
68 -
69 -Netdata Cloud does not store metric values.
70 -
71 -To enable certain features, such as [viewing active alerts](https://github.com/netdata/netdata/blob/master/docs/monitor/view-active-alerts.md)
72 -or [filtering by hostname](https://github.com/netdata/netdata/blob/master/docs/cloud/visualize/node-filter.md), Netdata Cloud does
73 -store configured alerts, their status, and a list of active collectors.
74 -
75 -Netdata does not and never will sell your personal data or data about your deployment.