@cryptotaxi247 / netdata / commits / f340a0e3f

docs: consolidate dbengine retention sizing guidance (#22517)

* docs: update docs/netdata-dbengine-retention-size-overage-and-storage-limits * docs: clarify child impact on parent retention * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * docs: clarify where child streamed data is stored on parent * docs: clarify independent child and parent storage in streaming * docs: improve retention docs clarity and flow * docs: clarify dbengine retention overhead wording * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * PR review --------- Co-authored-by: nedi-app[bot] <nedi-app[bot]@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Fotis Voutsas <fotis@netdata.cloud> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

nedi-app[bot] committed Jun 5, 2026 at 11:36 UTC f340a0e3ffb77f2d6ea3a02ffa6528ce7fd93dcf

3 files changed +25 -5

docs/netdata-agent/sizing-netdata-agents/disk-requirements-and-retention.md

+14 -1

@@ -43,12 +43,25 @@ Netdata Agent metrics storage is limited to 3 GiB by default (configurable), usi

46 -Data is deleted when retention enforcement detects that **either** the size limit or the time limit has been reached, whichever comes first. Retention is enforced asynchronously — dbengine evaluates quotas and schedules rotation both on a background timer and after normal activity (such as extent writes), deleting whole datafiles until the retention check no longer reports the tier over its limit. Actual disk usage may temporarily exceed the configured size limit. The number of metrics collected determines how far back in time retention extends within the size limit.

46 +Data is deleted when retention enforcement detects that **either** the size limit or the time limit has been reached, whichever comes first. Actual disk usage may temporarily exceed the configured size limit because retention size is a soft target, not a hard cap. For the detailed enforcement behavior, see [Retention Size Enforcement](/src/database/README.md#retention-size-enforcement).

48 In practice, with default settings and an ingestion rate of about 4,000 metrics per second, Netdata provides about 14 days of high resolution (per-second) data, 3 months of medium resolution (per-minute) data, and more than 1 year of low resolution (per-hour) data.

50 These limits are fully configurable. See [Changing how long Netdata stores metrics](/src/database/CONFIGURATION.md#tiers).

52 +### Parent Retention Sizing

53 +

54 +On Netdata Parents, retention size is enforced per tier for all metrics stored by that Parent, not per Child. All streaming Children share the Parent's tier quota, so there is no per-Child disk space limit.

55 +

56 +When sizing a Parent, account for the total metric count across all Children and leave room for dbengine datafiles plus journal/index overhead (`.ndf`, `.njf`, and `.njfv2` files). Parent nodes with many Children can have higher aggregate metric cardinality, which can increase journal/index overhead compared to a standalone Agent.

57 +

58 +For details about how dbengine enforces retention size limits, see [Retention Size Enforcement](/src/database/README.md#retention-size-enforcement).

59 +

60 +Child and Parent storage are independent:

61 +

62 +- **On the Child (local):** Controlled by the Child's `[db].mode`.

63 +- **On the Parent (received stream):** Controlled by the Parent's settings. Metrics streamed from Children can be persisted on the Parent and count against the Parent's per-tier retention limits.

64 +

65 **Configuring dbengine mode and retention**:

67 - Enable dbengine mode: The dbengine mode is already the default, so no configuration change is necessary. For reference, the dbengine mode can be configured by setting `[db].mode` to `dbengine` in `netdata.conf`.

docs/observability-centralization-points/best-practices.md

+2 -2

@@ -73,7 +73,7 @@ Netdata supports three retention strategies. Choose the one that best fits your

74 2. **Space-based retention** (recommended for predictable disk usage):

76 - Targets keeping storage usage within defined limits, at the cost of variable retention duration. Retention size is enforced asynchronously, not in real-time at write time — dbengine evaluates quotas and schedules rotation both on a background timer and after normal activity (such as extent writes), deleting whole datafiles until the retention check no longer reports the tier over its limit. Actual disk usage may temporarily exceed the configured limit — especially on tier 0 with high ingestion rates from many streaming children. The amount of overshoot is workload-dependent (ingestion rate, compression variance, rotation throughput); provision additional disk headroom beyond the configured limit to reduce the risk of disk-full conditions.

76 + Targets keeping storage usage within defined limits, at the cost of variable retention duration.

78 ```ini

79 [db]

@@ -101,7 +101,7 @@ Netdata supports three retention strategies. Choose the one that best fits your

101

102 :::warning

103

104 -Retention size limits are soft targets, not hard caps. Enforcement is periodic and asynchronous — quota checks and rotation scheduling occur both on a background timer and after normal dbengine activity (such as extent writes). Actual disk usage can exceed the configured limit between enforcement cycles. This is most noticeable on tier 0 with high metric volumes (e.g., parent nodes receiving streams from many children). Always provision more disk space than your configured limit to avoid unexpected disk-full conditions.

104 +Retention size limits are soft targets, not hard caps. Actual disk usage can exceed the configured limit, especially on tier 0 with high metric volumes from streaming Children. Always provision more disk space than your configured limit to avoid unexpected disk-full conditions. For the detailed enforcement behavior, see [Retention Size Enforcement](/src/database/README.md#retention-size-enforcement).

105

106 :::

107

src/database/CONFIGURATION.md

+9 -2

@@ -18,7 +18,14 @@ Use [`edit-config`](/docs/netdata-agent/configuration/README.md#edit-configurati

19 :::note

21 -In a Parent-Child setup, these settings manage the entire storage space used by the Parent for storing metrics collected both by itself and its Children.

21 +In a Parent-Child setup, these settings control the Parent's total storage for metrics collected locally and metrics received from Children.

22 +

23 +Child and Parent storage are independent:

24 +

25 +- A Child can keep local history based on its own `[db].mode`.

26 +- Streamed metrics can also be persisted on the Parent, in the Parent's own dbengine files.

27 +

28 +Retention size is enforced **per-tier**, not per Child, so all streaming Children share the Parent's tier quota. For Parent sizing guidance, see [Parent Retention Sizing](/docs/netdata-agent/sizing-netdata-agents/disk-requirements-and-retention.md#parent-retention-sizing).

30 :::

@@ -32,7 +39,7 @@ You can fine-tune retention for each tier by setting a time limit or size limit.

40 :::note

35 -Retention size limits are soft targets enforced periodically and asynchronously — not hard caps enforced at write time. Quota checks and rotation scheduling occur both on a background timer and after normal dbengine activity (such as extent writes). Actual disk usage can temporarily exceed the configured limit. This is most noticeable on tier 0 with high metric volumes, such as parent nodes receiving streams from many children. Provision more disk space than your configured limit to accommodate temporary overshoot.

42 +Retention size limits are soft targets, not hard caps enforced at write time. Actual disk usage can temporarily exceed the configured limit. For the detailed enforcement behavior, see [Retention Size Enforcement](/src/database/README.md#retention-size-enforcement).

44 :::