@cryptotaxi247 / netdata-1 / commits / a399128db

config parsers (#17238)

* added parser for durations * preliminary work for timeframes * Update CMakeLists.txt * updated parsing and generation for durations * renames * report parser errors; added compatibility to existing config_parse_duration() * duration parsing is used on most netdata.conf and stream.conf entries * more uses of duration parsing; simplification of stream.conf * code cleanup * more duration changes * added html playground * improved js code * duration parsing applied to dbengine retention * fixed doc * simplified logic; added size parser * added parsing for sizes * renames and documentation updates * hide appconfig internals from the rest of netdata * fix crash on cleanup of streaming receivers * fix buffer overflow in gorilla compression * config return values are const * ksm set to auto * support reformatting migrated values * removed obsolete metrics correlations settings * split appconfig to multiple files * durations documentation * sizes documentation * added backward compatibility in retention configuration * provide description on migrations and reformattings * config options are now a double linked list * config sections are now a double linked list; config uses spinlocks; code cleanup and renames * added data type to all config options * update data types * split appconfig api to multiple files * code cleanup and renames * removed size units above PiB * Revert "fix buffer overflow in gorilla compression" This reverts commit 3d5c48e84ba87935bdc2d285494c40d422b4577f. * appconfig internal api changes

Costa Tsaousis committed Sep 4, 2024 at 14:42 UTC a399128dbf685ba709a601944df41ae1ada04234

155 files changed +3479 -1934

CMakeLists.txt

+27

@@ -716,6 +716,8 @@ set(LIBNETDATA_FILES

716 src/libnetdata/dictionary/dictionary-callbacks.h

717 src/libnetdata/linked-lists.h

718 src/libnetdata/storage-point.h

719 + src/libnetdata/parsers/parsers.h

720 + src/libnetdata/parsers/duration.c

721 src/libnetdata/bitmap64.h

722 src/libnetdata/os/gettid.c

723 src/libnetdata/os/gettid.h

@@ -762,6 +764,31 @@ set(LIBNETDATA_FILES

764 src/libnetdata/paths/paths.c

765 src/libnetdata/paths/paths.h

766 src/libnetdata/json/json-c-parser-inline.c

767 + src/libnetdata/parsers/duration.h

768 + src/libnetdata/parsers/timeframe.c

769 + src/libnetdata/parsers/timeframe.h

770 + src/libnetdata/parsers/size.c

771 + src/libnetdata/parsers/size.h

772 + src/libnetdata/libjudy/judy-malloc.c

773 + src/libnetdata/libjudy/judy-malloc.h

774 + src/libnetdata/config/appconfig_internals.h

775 + src/libnetdata/config/appconfig_exporters.c

776 + src/libnetdata/config/appconfig_conf_file.c

777 + src/libnetdata/config/appconfig_cleanup.c

778 + src/libnetdata/config/appconfig_sections.c

779 + src/libnetdata/config/appconfig_options.c

780 + src/libnetdata/config/appconfig_migrate.c

781 + src/libnetdata/config/appconfig_traversal.c

782 + src/libnetdata/config/appconfig_api_sizes.c

783 + src/libnetdata/config/appconfig_api_sizes.h

784 + src/libnetdata/config/appconfig_api_durations.c

785 + src/libnetdata/config/appconfig_api_durations.h

786 + src/libnetdata/config/appconfig_api_numbers.c

787 + src/libnetdata/config/appconfig_api_numbers.h

788 + src/libnetdata/config/appconfig_api_text.c

789 + src/libnetdata/config/appconfig_api_text.h

790 + src/libnetdata/config/appconfig_api_boolean.c

791 + src/libnetdata/config/appconfig_api_boolean.h

792 )

793

794 if(ENABLE_PLUGIN_EBPF)

README.md

+1 -1

@@ -398,7 +398,7 @@ This is what you should expect:

398

399 - For production systems, each Netdata Agent with default settings (everything enabled, ML, Health, DB) should consume about 5% CPU utilization of one core and about 150 MiB or RAM.

400

401 - By using a Netdata parent and streaming all metrics to that parent, you can disable ML & health and use an ephemeral DB mode (like `alloc`) on the children, leading to utilization of about 1% CPU of a single core and 100 MiB of RAM. Of course, these depend on how many metrics are collected.

401 + By using a Netdata parent and streaming all metrics to that parent, you can disable ML & health and use an ephemeral DB (like `alloc`) on the children, leading to utilization of about 1% CPU of a single core and 100 MiB of RAM. Of course, these depend on how many metrics are collected.

402

403 - For Netdata Parents, for about 1 to 2 million metrics, all collected every second, we suggest a server with 16 cores and 32GB RAM. Less than half of it will be used for data collection and ML. The rest will be available for queries.

404

docs/deployment-guides/deployment-strategies.md

+5 -10

@@ -95,23 +95,18 @@ On the Parent, edit `netdata.conf` by using the [edit-config](/docs/netdata-agen

95 ```yaml

96 [db]

97 mode = dbengine

98 + dbengine tier backfill = new

99 storage tiers = 3

99 - # To allow memory pressure to offload index from ram

100 - dbengine page descriptors in file mapped memory = yes

100 + dbengine page cache size = 1.4GiB

101 # storage tier 0

102 update every = 1

103 - dbengine multihost disk space MB = 12000

104 - dbengine page cache size MB = 1400

103 + dbengine tier 0 retention space = 12GiB

104 # storage tier 1

106 - dbengine tier 1 page cache size MB = 512

107 - dbengine tier 1 multihost disk space MB = 4096

105 dbengine tier 1 update every iterations = 60

109 - dbengine tier 1 backfill = new

106 + dbengine tier 1 retention space = 4GiB

107 # storage tier 2

111 - dbengine tier 2 page cache size MB = 128

112 - dbengine tier 2 multihost disk space MB = 2048

108 dbengine tier 2 update every iterations = 60

114 - dbengine tier 2 backfill = new

109 + dbengine tier 2 retention space = 2GiB

110 [ml]

111 # Enabled by default

112 # enabled = yes

docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md

+29 -13

@@ -7,9 +7,9 @@ space**. This provides greater control and helps you optimize storage usage for

9 |:----:|:-------------------:|:----------:|:-----------------------:|

10 -| 0 | high (per second) | 14 days | 1 GiB |

11 -| 1 | middle (per minute) | 3 months | 1 GiB |

12 -| 2 | low (per hour) | 2 years | 1 GiB |

10 +| 0 | high (per second) | 14d | 1 GiB |

11 +| 1 | middle (per minute) | 3mo | 1 GiB |

12 +| 2 | low (per hour) | 2y | 1 GiB |

14 > **Note**: If a user sets a disk space size less than 256 MB for a tier, Netdata will automatically adjust it to 256 MB.

@@ -17,7 +17,7 @@ With these defaults, Netdata requires approximately 4 GiB of storage space (incl

18 ## Retention Settings

20 -> **In a parent-child setup**, these settings manage the shared storage space utilized by the Netdata parent agent for

20 +> **In a parent-child setup**, these settings manage the shared storage space used by the Netdata parent agent for

21 > storing metrics collected by both the parent and its child nodes.

23 You can fine-tune retention for each tier by setting a time limit or size limit. Setting a limit to 0 disables it,

@@ -38,16 +38,16 @@ You can change these limits in `netdata.conf`:

38 storage tiers = 3

40 # Tier 0, per second data. Set to 0 for no limit.

41 - dbengine tier 0 disk space MB = 1024

42 - dbengine tier 0 retention days = 14

41 + dbengine tier 0 retention size = 1GiB

42 + dbengine tier 0 retention time = 14d

44 # Tier 1, per minute data. Set to 0 for no limit.

45 - dbengine tier 1 disk space MB = 1024

46 - dbengine tier 1 retention days = 90

45 + dbengine tier 1 retention size = 1GiB

46 + dbengine tier 1 retention time = 3mo

48 # Tier 2, per hour data. Set to 0 for no limit.

49 - dbengine tier 2 disk space MB = 1024

50 - dbengine tier 2 retention days = 730

49 + dbengine tier 2 retention size = 1GiB

50 + dbengine tier 2 retention time = 2y

51 ```

53 ## Monitoring Retention Utilization

@@ -58,6 +58,24 @@ your storage space (disk space limits) and time (time limits) are used for metri

59 ## Legacy configuration

61 +### v1.99.0 and prior

62 +

63 +Netdata prior to v2 supports the following configuration options in `netdata.conf`.

64 +They have the same defaults as the latest v2, but the unit of each value is given in the option name, not at the value.

65 +

66 +```

67 +storage tiers = 3

68 +# Tier 0, per second data. Set to 0 for no limit.

69 +dbengine tier 0 disk space MB = 1024

70 +dbengine tier 0 retention days = 14

71 +# Tier 1, per minute data. Set to 0 for no limit.

72 +dbengine tier 1 disk space MB = 1024

73 +dbengine tier 1 retention days = 90

74 +# Tier 2, per hour data. Set to 0 for no limit.

75 +dbengine tier 2 disk space MB = 1024

76 +dbengine tier 2 retention days = 730

77 +```

78 +

79 ### v1.45.6 and prior

81 Netdata versions prior to v1.46.0 relied on a disk space-based retention.

@@ -76,13 +94,10 @@ You can change these limits in `netdata.conf`:

94 [db]

95 mode = dbengine

96 storage tiers = 3

79 -

97 # Tier 0, per second data

98 dbengine multihost disk space MB = 256

82 -

99 # Tier 1, per minute data

100 dbengine tier 1 multihost disk space MB = 1024

85 -

101 # Tier 2, per hour data

102 dbengine tier 2 multihost disk space MB = 1024

103 ```

@@ -113,6 +128,7 @@ If `dbengine disk space MB`(**deprecated**) is set to the default `256`, each in

128 which means the total disk space required to store all instances is,

129 roughly, `256 MiB * 1 parent * 4 child nodes = 1280 MiB`.

130

131 +

132 #### Backward compatibility

133

134 All existing metrics belonging to child nodes are automatically converted to legacy dbengine instances and the localhost

docs/netdata-agent/sizing-netdata-agents/ram-requirements.md

+2 -2

@@ -34,8 +34,8 @@ about 16 GiB

35 There are 2 cache sizes that can be configured in `netdata.conf`:

37 -1. `[db].dbengine page cache size MB`: this is the main cache that keeps metrics data into memory. When data are not found in it, the extent cache is consulted, and if not found in that either, they are loaded from disk.

38 -2. `[db].dbengine extent cache size MB`: this is the compressed extent cache. It keeps in memory compressed data blocks, as they appear on disk, to avoid reading them again. Data found in the extend cache but not in the main cache have to be uncompressed to be queried.

37 +1. `[db].dbengine page cache size`: this is the main cache that keeps metrics data into memory. When data are not found in it, the extent cache is consulted, and if not found in that either, they are loaded from disk.

38 +2. `[db].dbengine extent cache size`: this is the compressed extent cache. It keeps in memory compressed data blocks, as they appear on disk, to avoid reading them again. Data found in the extend cache but not in the main cache have to be uncompressed to be queried.

40 Both of them are dynamically adjusted to use some of the total memory computed above. The configuration in `netdata.conf` allows providing additional memory to them, increasing their caching efficiency.

docs/observability-centralization-points/metrics-centralization-points/clustering-and-high-availability-of-netdata-parents.md

+3 -3

@@ -45,6 +45,6 @@ The easiest way is to `rsync` the directory `/var/cache/netdata` from the existi

46 To configure retention at the new Netdata Parent, set in `netdata.conf` the following to at least the values the old Netdata Parent has:

48 -- `[db].dbengine multihost disk space MB`, this is the max disk size for `tier0`. The default is 256MiB.

49 -- `[db].dbengine tier 1 multihost disk space MB`, this is the max disk space for `tier1`. The default is 50% of `tier0`.

50 -- `[db].dbengine tier 2 multihost disk space MB`, this is the max disk space for `tier2`. The default is 50% of `tier1`.

48 +- `[db].dbengine tier 0 retention size`, this is the max disk size for `tier0`. The default is 1GiB.

49 +- `[db].dbengine tier 1 retention size`, this is the max disk space for `tier1`. The default is 1GiB.

50 +- `[db].dbengine tier 2 retention size`, this is the max disk space for `tier2`. The default is 1GiB.

docs/observability-centralization-points/metrics-centralization-points/replication-of-past-samples.md

+2 -2

@@ -45,13 +45,13 @@ The following `netdata.conf` configuration parameters affect replication.

46 On the receiving side (Netdata Parent):

48 -- `[db].seconds to replicate` limits the maximum time to be replicated. The default is 1 day (86400 seconds). Keep in mind that replication is also limited by the `tier0` retention the sending side has.

48 +- `[db].replication period` limits the maximum time to be replicated. The default is 1 day. Keep in mind that replication is also limited by the `tier0` retention the sending side has.

50 On the sending side (Netdata Children, or Netdata Parent when parents are clustered):

52 - `[db].replication threads` controls how many concurrent threads will be replicating metrics. The default is 1. Usually the performance is about 2 million samples per second per thread, so increasing this number may allow replication to progress faster between Netdata Parents.

54 -- `[db].cleanup obsolete charts after secs` controls for how much time after metrics stop being collected will not be available for replication. The default is 1 hour (3600 seconds). If you plan to have scheduled maintenance on Netdata Parents of more than 1 hour, we recommend increasing this setting. Keep in mind however, that increasing this duration in highly ephemeral environments can have an impact on RAM utilization, since metrics will be considered as collected for longer durations.

54 +- `[db].cleanup obsolete charts after` controls for how much time after metrics stop being collected will not be available for replication. The default is 1 hour (3600 seconds). If you plan to have scheduled maintenance on Netdata Parents of more than 1 hour, we recommend increasing this setting. Keep in mind however, that increasing this duration in highly ephemeral environments can have an impact on RAM utilization, since metrics will be considered as collected for longer durations.

56 ## Monitoring Replication Progress

integrations/integrations.js

+2 -2

@@ -1308,7 +1308,7 @@ export const integrations = [

1308 "most_popular": false

1309 },

1310 "overview": "# Disk space\n\nPlugin: diskspace.plugin\nModule: diskspace.plugin\n\n## Overview\n\nMonitor Disk space metrics for proficient storage management. Keep track of usage, free space, and error rates to prevent disk space issues.\n\n\n\nThis collector is supported on all platforms.\n\nThis collector supports collecting metrics from multiple instances of this integration, including remote instances.\n\n\n### Default Behavior\n\n#### Auto-Detection\n\nThe plugin reads data from `/proc/self/mountinfo` and `/proc/diskstats file`.\n\n#### Limits\n\nThe default configuration for this integration does not impose any limits on data collection.\n\n#### Performance Impact\n\nThe default configuration for this integration is not expected to impose a significant performance impact on the system.\n",

1311 - "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\nConfiguration for this specific integration is located in the `[plugin:proc:diskspace]` section within that file.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nYou can also specify per mount point `[plugin:proc:diskspace:mountpoint]`\n\n{% details open=true summary=\"Config options\" %}\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| update every | Data collection frequency. | 1 | no |\n| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |\n| check for new mount points every | Parse proc files frequency. | 15 | no |\n| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |\n| exclude space metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | *gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs | no |\n| exclude inode metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | msdosfs msdos vfat overlayfs aufs* *unionfs | no |\n| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n\n{% /details %}\n#### Examples\nThere are no configuration examples.\n\n",

1311 + "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\nConfiguration for this specific integration is located in the `[plugin:proc:diskspace]` section within that file.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nYou can also specify per mount point `[plugin:proc:diskspace:mountpoint]`\n\n{% details open=true summary=\"Config options\" %}\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| update every | Data collection frequency. | 1 | no |\n| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |\n| check for new mount points every | Parse proc files frequency. | 15s | no |\n| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |\n| exclude space metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | *gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs | no |\n| exclude inode metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | msdosfs msdos vfat overlayfs aufs* *unionfs | no |\n| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n\n{% /details %}\n#### Examples\nThere are no configuration examples.\n\n",

1312 "troubleshooting": "",

1313 "alerts": "## Alerts\n\n\nThe following alerts are available:\n\n| Alert name | On metric | Description |\n|:------------|:----------|:------------|\n| [ disk_space_usage ](https://github.com/netdata/netdata/blob/master/src/health/health.d/disks.conf) | disk.space | disk ${label:mount_point} space utilization |\n| [ disk_inode_usage ](https://github.com/netdata/netdata/blob/master/src/health/health.d/disks.conf) | disk.inodes | disk ${label:mount_point} inode utilization |\n",

1314 "metrics": "## Metrics\n\nMetrics grouped by *scope*.\n\nThe scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.\n\n\n\n### Per mount point\n\n\n\nLabels:\n\n| Label | Description |\n|:-----------|:----------------|\n| mount_point | Path used to mount a filesystem |\n| filesystem | The filesystem used to format a partition. |\n| mount_root | Root directory where mount points are present. |\n\nMetrics:\n\n| Metric | Dimensions | Unit |\n|:------|:----------|:----|\n| disk.space | avail, used, reserved_for_root | GiB |\n| disk.inodes | avail, used, reserved_for_root | inodes |\n\n",

@@ -17655,7 +17655,7 @@ export const integrations = [

17655 "most_popular": false

17656 },

17657 "overview": "# Idle OS Jitter\n\nPlugin: idlejitter.plugin\nModule: idlejitter.plugin\n\n## Overview\n\nMonitor delays in timing for user processes caused by scheduling limitations to optimize the system to run latency sensitive applications with minimal jitter, improving consistency and quality of service.\n\n\nA thread is spawned that requests to sleep for fixed amount of time. When the system wakes it up, it measures how many microseconds have passed. The difference between the requested and the actual duration of the sleep, is the idle jitter. This is done dozens of times per second to ensure we have a representative sample.\n\n\nThis collector is supported on all platforms.\n\nThis collector only supports collecting metrics from a single instance of this integration.\n\n\n### Default Behavior\n\n#### Auto-Detection\n\nThis integration will run by default on all supported systems.\n\n#### Limits\n\nThe default configuration for this integration does not impose any limits on data collection.\n\n#### Performance Impact\n\nThe default configuration for this integration is not expected to impose a significant performance impact on the system.\n",

17658 - "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nThis integration only supports a single configuration option, and most users will not need to change it.\n\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| loop time in ms | Specifies the target time for the data collection thread to sleep, measured in miliseconds. | 20 | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

17658 + "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nThis integration only supports a single configuration option, and most users will not need to change it.\n\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| loop time | Specifies the target time for the data collection thread to sleep. | 20ms | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

17659 "troubleshooting": "",

17660 "alerts": "## Alerts\n\nThere are no alerts configured by default for this integration.\n",

17661 "metrics": "## Metrics\n\nMetrics grouped by *scope*.\n\nThe scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.\n\n\n\n### Per Idle OS Jitter instance\n\n\n\nThis scope has no labels.\n\nMetrics:\n\n| Metric | Dimensions | Unit |\n|:------|:----------|:----|\n| system.idlejitter | min, max, average | microseconds lost/s |\n\n",

integrations/integrations.json

+2 -2

@@ -1306,7 +1306,7 @@

1306 "most_popular": false

1307 },

1308 "overview": "# Disk space\n\nPlugin: diskspace.plugin\nModule: diskspace.plugin\n\n## Overview\n\nMonitor Disk space metrics for proficient storage management. Keep track of usage, free space, and error rates to prevent disk space issues.\n\n\n\nThis collector is supported on all platforms.\n\nThis collector supports collecting metrics from multiple instances of this integration, including remote instances.\n\n\n### Default Behavior\n\n#### Auto-Detection\n\nThe plugin reads data from `/proc/self/mountinfo` and `/proc/diskstats file`.\n\n#### Limits\n\nThe default configuration for this integration does not impose any limits on data collection.\n\n#### Performance Impact\n\nThe default configuration for this integration is not expected to impose a significant performance impact on the system.\n",

1309 - "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\nConfiguration for this specific integration is located in the `[plugin:proc:diskspace]` section within that file.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nYou can also specify per mount point `[plugin:proc:diskspace:mountpoint]`\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| update every | Data collection frequency. | 1 | no |\n| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |\n| check for new mount points every | Parse proc files frequency. | 15 | no |\n| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |\n| exclude space metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | *gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs | no |\n| exclude inode metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | msdosfs msdos vfat overlayfs aufs* *unionfs | no |\n| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

1309 + "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\nConfiguration for this specific integration is located in the `[plugin:proc:diskspace]` section within that file.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nYou can also specify per mount point `[plugin:proc:diskspace:mountpoint]`\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| update every | Data collection frequency. | 1 | no |\n| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |\n| check for new mount points every | Parse proc files frequency. | 15s | no |\n| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |\n| exclude space metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | *gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs | no |\n| exclude inode metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | msdosfs msdos vfat overlayfs aufs* *unionfs | no |\n| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

1310 "troubleshooting": "",

1311 "alerts": "## Alerts\n\n\nThe following alerts are available:\n\n| Alert name | On metric | Description |\n|:------------|:----------|:------------|\n| [ disk_space_usage ](https://github.com/netdata/netdata/blob/master/src/health/health.d/disks.conf) | disk.space | disk ${label:mount_point} space utilization |\n| [ disk_inode_usage ](https://github.com/netdata/netdata/blob/master/src/health/health.d/disks.conf) | disk.inodes | disk ${label:mount_point} inode utilization |\n",

1312 "metrics": "## Metrics\n\nMetrics grouped by *scope*.\n\nThe scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.\n\n\n\n### Per mount point\n\n\n\nLabels:\n\n| Label | Description |\n|:-----------|:----------------|\n| mount_point | Path used to mount a filesystem |\n| filesystem | The filesystem used to format a partition. |\n| mount_root | Root directory where mount points are present. |\n\nMetrics:\n\n| Metric | Dimensions | Unit |\n|:------|:----------|:----|\n| disk.space | avail, used, reserved_for_root | GiB |\n| disk.inodes | avail, used, reserved_for_root | inodes |\n\n",

@@ -17653,7 +17653,7 @@

17653 "most_popular": false

17654 },

17655 "overview": "# Idle OS Jitter\n\nPlugin: idlejitter.plugin\nModule: idlejitter.plugin\n\n## Overview\n\nMonitor delays in timing for user processes caused by scheduling limitations to optimize the system to run latency sensitive applications with minimal jitter, improving consistency and quality of service.\n\n\nA thread is spawned that requests to sleep for fixed amount of time. When the system wakes it up, it measures how many microseconds have passed. The difference between the requested and the actual duration of the sleep, is the idle jitter. This is done dozens of times per second to ensure we have a representative sample.\n\n\nThis collector is supported on all platforms.\n\nThis collector only supports collecting metrics from a single instance of this integration.\n\n\n### Default Behavior\n\n#### Auto-Detection\n\nThis integration will run by default on all supported systems.\n\n#### Limits\n\nThe default configuration for this integration does not impose any limits on data collection.\n\n#### Performance Impact\n\nThe default configuration for this integration is not expected to impose a significant performance impact on the system.\n",

17656 - "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nThis integration only supports a single configuration option, and most users will not need to change it.\n\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| loop time in ms | Specifies the target time for the data collection thread to sleep, measured in miliseconds. | 20 | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

17656 + "setup": "## Setup\n\n### Prerequisites\n\nNo action required.\n\n### Configuration\n\n#### File\n\nThe configuration file name for this integration is `netdata.conf`.\n\nThe file format is a modified INI syntax. The general structure is:\n\n```ini\n[section1]\n option1 = some value\n option2 = some other value\n\n[section2]\n option3 = some third value\n```\nYou can edit the configuration file using the `edit-config` script from the\nNetdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).\n\n```bash\ncd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata\nsudo ./edit-config netdata.conf\n```\n#### Options\n\nThis integration only supports a single configuration option, and most users will not need to change it.\n\n\n| Name | Description | Default | Required |\n|:----|:-----------|:-------|:--------:|\n| loop time | Specifies the target time for the data collection thread to sleep. | 20ms | no |\n\n#### Examples\nThere are no configuration examples.\n\n",

17657 "troubleshooting": "",

17658 "alerts": "## Alerts\n\nThere are no alerts configured by default for this integration.\n",

17659 "metrics": "## Metrics\n\nMetrics grouped by *scope*.\n\nThe scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.\n\n\n\n### Per Idle OS Jitter instance\n\n\n\nThis scope has no labels.\n\nMetrics:\n\n| Metric | Dimensions | Unit |\n|:------|:----------|:----|\n| system.idlejitter | min, max, average | microseconds lost/s |\n\n",

packaging/dag/files/child_stream.conf

+2 -2

@@ -2,9 +2,9 @@

2 enabled = {{ enabled }}

3 destination = {{ destination }}

4 api key = {{ api_key }}

5 - timeout seconds = {{ timeout_seconds }}

5 + timeout = {{ timeout_seconds }}

6 default port = {{ default_port }}

7 send charts matching = {{ send_charts_matching }}

8 buffer size bytes = {{ buffer_size_bytes }}

9 - reconnect delay seconds = {{ reconnect_delay_seconds }}

9 + reconnect delay = {{ reconnect_delay_seconds }}

10 initial clock resync iterations = {{ initial_clock_resync_iterations }}

packaging/dag/files/parent_stream.conf

+2 -2

@@ -1,7 +1,7 @@

1 [{{ api_key }}]

2 enabled = {{ enabled }}

3 allow from = {{ allow_from }}

4 - default history = {{ default_history }}

4 + retention = {{ default_history }}

5 health enabled by default = {{ health_enabled_by_default }}

6 - default postpone alarms on connect seconds = {{ default_postpone_alarms_on_connect_seconds }}

6 + postpone alerts on connect = {{ default_postpone_alarms_on_connect_seconds }}

7 multiple connections = {{ multiple_connections }}

packaging/installer/methods/kubernetes.md

+4 -4

@@ -87,9 +87,9 @@ On an existing installation, in order to connect it to Netdata Cloud you will ne

87 configs:

88 netdata:

89 data: |

90 - [global]

91 - memory mode = ram

92 - history = 3600

90 + [db]

91 + db = ram

92 + retention = 3600

93 [health]

94 enabled = no

95 ```

@@ -103,7 +103,7 @@ On an existing installation, in order to connect it to Netdata Cloud you will ne

103

104 > :bookmark_tabs: Info

105 >

106 - > These override settings, along with the Helm chart's defaults, will retain an hour's worth of metrics (`history = 3600`, or `3600 seconds`) on each child node. Based on your metrics retention needs, and the resources available on your cluster, you may want to increase the `history` setting.

106 + > These override settings, along with the Helm chart's defaults, will retain an hour's worth of metrics (`retention = 3600`, or `3600 seconds`) on each child node. Based on your metrics retention needs, and the resources available on your cluster, you may want to increase the `history` setting.

107

108 3. To apply these new settings, run:

109

src/aclk/aclk_capas.c

+3 -5

@@ -28,8 +28,8 @@ const struct capability *aclk_get_agent_capas()

28 agent_capabilities[2].version = ml_capable() ? 1 : 0;

29 agent_capabilities[2].enabled = ml_enabled(localhost);

31 - agent_capabilities[3].version = enable_metric_correlations ? metric_correlations_version : 0;

32 - agent_capabilities[3].enabled = enable_metric_correlations;

31 + agent_capabilities[3].version = metric_correlations_version;

32 + agent_capabilities[3].enabled = 1;

34 agent_capabilities[7].enabled = localhost->health.health_enabled;

@@ -44,9 +44,7 @@ struct capability *aclk_get_node_instance_capas(RRDHOST *host)

44 struct capability ni_caps[] = {

45 { .name = "proto", .version = 1, .enabled = 1 },

46 { .name = "ml", .version = ml_capable(), .enabled = ml_enabled(host) },

47 - { .name = "mc",

48 - .version = enable_metric_correlations ? metric_correlations_version : 0,

49 - .enabled = enable_metric_correlations },

47 + { .name = "mc", .version = metric_correlations_version, .enabled = 1 },

48 { .name = "ctx", .version = 1, .enabled = 1 },

49 { .name = "funcs", .version = functions ? 1 : 0, .enabled = functions ? 1 : 0 },

50 { .name = "http_api_v2", .version = HTTP_API_V2_VERSION, .enabled = 1 },

src/claim/claim-with-api.c

+1 -12

@@ -401,18 +401,7 @@ bool claim_agent_from_environment(void) {

401 }

402

403 bool claim_agent_from_claim_conf(void) {

404 - static struct config claim_config = {

405 - .first_section = NULL,

406 - .last_section = NULL,

407 - .mutex = NETDATA_MUTEX_INITIALIZER,

408 - .index = {

409 - .avl_tree = {

410 - .root = NULL,

411 - .compar = appconfig_section_compare

412 - },

413 - .rwlock = AVL_LOCK_INITIALIZER

414 - }

415 - };

404 + static struct config claim_config = APPCONFIG_INITIALIZER;

405 static SPINLOCK spinlock = NETDATA_SPINLOCK_INITIALIZER;

406 bool ret = false;

407

src/claim/cloud-conf.c

+1 -12

@@ -2,18 +2,7 @@

3 #include "claim.h"

5 -struct config cloud_config = {

6 - .first_section = NULL,

7 - .last_section = NULL,

8 - .mutex = NETDATA_MUTEX_INITIALIZER,

9 - .index = {

10 - .avl_tree = {

11 - .root = NULL,

12 - .compar = appconfig_section_compare

13 - },

14 - .rwlock = AVL_LOCK_INITIALIZER

15 - }

16 -};

5 +struct config cloud_config = APPCONFIG_INITIALIZER;

7 const char *cloud_config_url_get(void) {

8 return appconfig_get(&cloud_config, CONFIG_SECTION_GLOBAL, "url", DEFAULT_CLOUD_BASE_URL);

src/collectors/cgroups.plugin/cgroup-discovery.c

+1 -1

@@ -23,7 +23,7 @@ struct cgroup *discovered_cgroup_root = NULL;

24 char cgroup_chart_id_prefix[] = "cgroup_";

25 char services_chart_id_prefix[] = "systemd_";

26 -char *cgroups_rename_script = NULL;

26 +const char *cgroups_rename_script = NULL;

28 // Shared memory with information from detected cgroups

29 netdata_ebpf_cgroup_shm_t shm_cgroup_ebpf = {NULL, NULL};

src/collectors/cgroups.plugin/cgroup-internals.h

+2 -2

@@ -273,7 +273,7 @@ struct discovery_thread {

273

274 extern struct discovery_thread discovery_thread;

275

276 -extern char *cgroups_rename_script;

276 +extern const char *cgroups_rename_script;

277 extern char cgroup_chart_id_prefix[];

278 extern char services_chart_id_prefix[];

279 extern uv_mutex_t cgroup_root_mutex;

@@ -313,7 +313,7 @@ extern SIMPLE_PATTERN *enabled_cgroup_renames;

313 extern SIMPLE_PATTERN *systemd_services_cgroups;

314 extern SIMPLE_PATTERN *entrypoint_parent_process_comm;

315

316 -extern char *cgroups_network_interface_script;

316 +extern const char *cgroups_network_interface_script;

317

318 extern int cgroups_check;

319

src/collectors/cgroups.plugin/sys_fs_cgroup.c

+9 -5

@@ -39,7 +39,7 @@ SIMPLE_PATTERN *search_cgroup_paths = NULL;

39 SIMPLE_PATTERN *enabled_cgroup_renames = NULL;

40 SIMPLE_PATTERN *systemd_services_cgroups = NULL;

41 SIMPLE_PATTERN *entrypoint_parent_process_comm = NULL;

42 -char *cgroups_network_interface_script = NULL;

42 +const char *cgroups_network_interface_script = NULL;

43 int cgroups_check = 0;

44 uint32_t Read_hash = 0;

45 uint32_t Write_hash = 0;

@@ -229,13 +229,17 @@ void read_cgroup_plugin_configuration() {

229 throttled_time_hash = simple_hash("throttled_time");

230 throttled_usec_hash = simple_hash("throttled_usec");

231

232 - cgroup_update_every = (int)config_get_number("plugin:cgroups", "update every", localhost->rrd_update_every);

233 - if(cgroup_update_every < localhost->rrd_update_every)

232 + cgroup_update_every = (int)config_get_duration_seconds("plugin:cgroups", "update every", localhost->rrd_update_every);

233 + if(cgroup_update_every < localhost->rrd_update_every) {

234 cgroup_update_every = localhost->rrd_update_every;

235 + config_set_duration_seconds("plugin:cgroups", "update every", localhost->rrd_update_every);

236 + }

237

236 - cgroup_check_for_new_every = (int)config_get_number("plugin:cgroups", "check for new cgroups every", cgroup_check_for_new_every);

237 - if(cgroup_check_for_new_every < cgroup_update_every)

238 + cgroup_check_for_new_every = (int)config_get_duration_seconds("plugin:cgroups", "check for new cgroups every", cgroup_check_for_new_every);

239 + if(cgroup_check_for_new_every < cgroup_update_every) {

240 cgroup_check_for_new_every = cgroup_update_every;

241 + config_set_duration_seconds("plugin:cgroups", "check for new cgroups every", cgroup_check_for_new_every);

242 + }

243

244 cgroup_use_unified_cgroups = config_get_boolean_ondemand("plugin:cgroups", "use unified cgroups", CONFIG_BOOLEAN_AUTO);

245 if (cgroup_use_unified_cgroups == CONFIG_BOOLEAN_AUTO)

src/collectors/diskspace.plugin/integrations/disk_space.md

+9 -9

@@ -121,16 +121,16 @@ You can also specify per mount point `[plugin:proc:diskspace:mountpoint]`

121

122 <details open><summary>Config options</summary>

123

125 -|:----|:-----------|:-------|:--------:|

126 -| update every | Data collection frequency. | 1 | no |

127 -| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |

128 -| check for new mount points every | Parse proc files frequency. | 15 | no |

129 -| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |

125 +|:----|:-----------|:---------------------------------------------------------------------------------|:--------:|

126 +| update every | Data collection frequency. | 1 | no |

127 +| remove charts of unmounted disks | Remove chart when a device is unmounted on host. | yes | no |

128 +| check for new mount points every | Parse proc files frequency. | 15s | no |

129 +| exclude space metrics on paths | Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern. | /proc/* /sys/* /var/run/user/* /run/user/* /snap/* /var/lib/docker/* | no |

130 | exclude space metrics on filesystems | Do not show metrics (charts) for listed filesystems. This option accepts netdata simple pattern. | *gvfs *gluster* *s3fs *ipfs *davfs2 *httpfs *sshfs *gdfs *moosefs fusectl autofs | no |

132 -| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |

133 -| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |

132 +| space usage for all disks | Define if plugin will show metrics for space usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |

133 +| inodes usage for all disks | Define if plugin will show metrics for inode usage. When value is set to `auto` plugin will try to access information to display if filesystem or path was not discarded with previous option. | auto | no |

134

135 </details>

136

src/collectors/diskspace.plugin/metadata.yaml

+1 -1

@@ -63,7 +63,7 @@ modules:

63 required: false

64 - name: check for new mount points every

65 description: Parse proc files frequency.

66 - default_value: 15

66 + default_value: 15s

67 required: false

68 - name: exclude space metrics on paths

69 description: Do not show metrics (charts) for listed paths. This option accepts netdata simple pattern.

src/collectors/diskspace.plugin/plugin_diskspace.c

+5 -3

@@ -855,11 +855,13 @@ void *diskspace_main(void *ptr) {

855

856 cleanup_mount_points = config_get_boolean(CONFIG_SECTION_DISKSPACE, "remove charts of unmounted disks" , cleanup_mount_points);

857

858 - int update_every = (int)config_get_number(CONFIG_SECTION_DISKSPACE, "update every", localhost->rrd_update_every);

859 - if(update_every < localhost->rrd_update_every)

858 + int update_every = (int)config_get_duration_seconds(CONFIG_SECTION_DISKSPACE, "update every", localhost->rrd_update_every);

859 + if(update_every < localhost->rrd_update_every) {

860 update_every = localhost->rrd_update_every;

861 + config_set_duration_seconds(CONFIG_SECTION_DISKSPACE, "update every", update_every);

862 + }

863

862 - check_for_new_mountpoints_every = (int)config_get_number(CONFIG_SECTION_DISKSPACE, "check for new mount points every", check_for_new_mountpoints_every);

864 + check_for_new_mountpoints_every = (int)config_get_duration_seconds(CONFIG_SECTION_DISKSPACE, "check for new mount points every", check_for_new_mountpoints_every);

865 if(check_for_new_mountpoints_every < update_every)

866 check_for_new_mountpoints_every = update_every;

867

src/collectors/ebpf.plugin/ebpf.c

+13 -15

@@ -19,11 +19,7 @@ char *ebpf_plugin_dir = PLUGINS_DIR;

19 static char *ebpf_configured_log_dir = LOG_DIR;

21 char *ebpf_algorithms[] = { EBPF_CHART_ALGORITHM_ABSOLUTE, EBPF_CHART_ALGORITHM_INCREMENTAL};

22 -struct config collector_config = { .first_section = NULL,

23 - .last_section = NULL,

24 - .mutex = NETDATA_MUTEX_INITIALIZER,

25 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

26 - .rwlock = AVL_LOCK_INITIALIZER } };

22 +struct config collector_config = APPCONFIG_INITIALIZER;

24 int running_on_kernel = 0;

25 int ebpf_nprocs;

@@ -661,7 +657,7 @@ struct vfs_bpf *vfs_bpf_obj = NULL;

657 #else

658 void *default_btf = NULL;

659 #endif

664 -char *btf_path = NULL;

660 +const char *btf_path = NULL;

661

662 /*****************************************************************

663 *

@@ -1836,7 +1832,7 @@ static void ebpf_parse_ip_list_unsafe(void **out, char *ip)

1832 *

1833 * @param ptr is a pointer with the text to parse.

1834 */

1839 -void ebpf_parse_ips_unsafe(char *ptr)

1835 +void ebpf_parse_ips_unsafe(const char *ptr)

1836 {

1837 // No value

1838 if (unlikely(!ptr))

@@ -1927,7 +1923,7 @@ static inline void fill_port_list(ebpf_network_viewer_port_list_t **out, ebpf_ne

1923 * @param out a pointer to store the link list

1924 * @param service the service used to create the structure that will be linked.

1925 */

1930 -static void ebpf_parse_service_list(void **out, char *service)

1926 +static void ebpf_parse_service_list(void **out, const char *service)

1927 {

1928 ebpf_network_viewer_port_list_t **list = (ebpf_network_viewer_port_list_t **)out;

1929 struct servent *serv = getservbyname((const char *)service, "tcp");

@@ -1956,8 +1952,10 @@ static void ebpf_parse_service_list(void **out, char *service)

1952 * @param out a pointer to store the link list

1953 * @param range the informed range for the user.

1954 */

1959 -static void ebpf_parse_port_list(void **out, char *range)

1960 -{

1955 +static void ebpf_parse_port_list(void **out, const char *range_param) {

1956 + char range[strlen(range_param) + 1];

1957 + strncpyz(range, range_param, strlen(range_param));

1958 +

1959 int first, last;

1960 ebpf_network_viewer_port_list_t **list = (ebpf_network_viewer_port_list_t **)out;

1961

@@ -2029,7 +2027,7 @@ static void ebpf_parse_port_list(void **out, char *range)

2027 *

2028 * @param ptr is a pointer with the text to parse.

2029 */

2032 -void ebpf_parse_ports(char *ptr)

2030 +void ebpf_parse_ports(const char *ptr)

2031 {

2032 // No value

2033 if (unlikely(!ptr))

@@ -2480,7 +2478,7 @@ static void ebpf_link_hostname(ebpf_network_viewer_hostname_list_t **out, ebpf_n

2478 * @param out is the output link list

2479 * @param parse is a pointer with the text to parser.

2480 */

2483 -static void ebpf_link_hostnames(char *parse)

2481 +static void ebpf_link_hostnames(const char *parse)

2482 {

2483 // No value

2484 if (unlikely(!parse))

@@ -2536,7 +2534,7 @@ void parse_network_viewer_section(struct config *cfg)

2534 EBPF_CONFIG_RESOLVE_SERVICE,

2535 CONFIG_BOOLEAN_YES);

2536

2539 - char *value = appconfig_get(cfg, EBPF_NETWORK_VIEWER_SECTION, EBPF_CONFIG_PORTS, NULL);

2537 + const char *value = appconfig_get(cfg, EBPF_NETWORK_VIEWER_SECTION, EBPF_CONFIG_PORTS, NULL);

2538 ebpf_parse_ports(value);

2539

2540 if (network_viewer_opt.hostname_resolution_enabled) {

@@ -2684,7 +2682,7 @@ static void ebpf_allocate_common_vectors()

2682 *

2683 * @param ptr the option given by users

2684 */

2687 -static inline void ebpf_how_to_load(char *ptr)

2685 +static inline void ebpf_how_to_load(const char *ptr)

2686 {

2687 if (!strcasecmp(ptr, EBPF_CFG_LOAD_MODE_RETURN))

2688 ebpf_set_thread_mode(MODE_RETURN);

@@ -2808,7 +2806,7 @@ static void read_collector_values(int *disable_cgroups,

2806 int update_every, netdata_ebpf_load_mode_t origin)

2807 {

2808 // Read global section

2811 - char *value;

2809 + const char *value;

2810 if (appconfig_exists(&collector_config, EBPF_GLOBAL_SECTION, "load")) // Backward compatibility

2811 value = appconfig_get(&collector_config, EBPF_GLOBAL_SECTION, "load",

2812 EBPF_CFG_LOAD_MODE_DEFAULT);

src/collectors/ebpf.plugin/ebpf_cachestat.c

+1 -5

@@ -43,11 +43,7 @@ ebpf_local_maps_t cachestat_maps[] = {{.name = "cstat_global", .internal_input =

43 #endif

44 }};

46 -struct config cachestat_config = { .first_section = NULL,

47 - .last_section = NULL,

48 - .mutex = NETDATA_MUTEX_INITIALIZER,

49 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

50 - .rwlock = AVL_LOCK_INITIALIZER } };

46 +struct config cachestat_config = APPCONFIG_INITIALIZER;

48 netdata_ebpf_targets_t cachestat_targets[] = { {.name = "add_to_page_cache_lru", .mode = EBPF_LOAD_TRAMPOLINE},

49 {.name = "mark_page_accessed", .mode = EBPF_LOAD_TRAMPOLINE},

src/collectors/ebpf.plugin/ebpf_dcstat.c

+1 -5

@@ -12,11 +12,7 @@ netdata_dcstat_pid_t *dcstat_vector = NULL;

12 static netdata_idx_t dcstat_hash_values[NETDATA_DCSTAT_IDX_END];

13 static netdata_idx_t *dcstat_values = NULL;

15 -struct config dcstat_config = { .first_section = NULL,

16 - .last_section = NULL,

17 - .mutex = NETDATA_MUTEX_INITIALIZER,

18 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

19 - .rwlock = AVL_LOCK_INITIALIZER } };

15 +struct config dcstat_config = APPCONFIG_INITIALIZER;

17 ebpf_local_maps_t dcstat_maps[] = {{.name = "dcstat_global", .internal_input = NETDATA_DIRECTORY_CACHE_END,

18 .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC,

src/collectors/ebpf.plugin/ebpf_disk.c

+1 -5

@@ -6,11 +6,7 @@

6 #include "ebpf.h"

7 #include "ebpf_disk.h"

9 -struct config disk_config = { .first_section = NULL,

10 - .last_section = NULL,

11 - .mutex = NETDATA_MUTEX_INITIALIZER,

12 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

13 - .rwlock = AVL_LOCK_INITIALIZER } };

9 +struct config disk_config = APPCONFIG_INITIALIZER;

11 static ebpf_local_maps_t disk_maps[] = {{.name = "tbl_disk_iocall", .internal_input = NETDATA_DISK_HISTOGRAM_LENGTH,

12 .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC,

src/collectors/ebpf.plugin/ebpf_fd.c

+1 -3

@@ -46,9 +46,7 @@ static ebpf_local_maps_t fd_maps[] = {{.name = "tbl_fd_pid", .internal_input = N

46 }};

49 -struct config fd_config = { .first_section = NULL, .last_section = NULL, .mutex = NETDATA_MUTEX_INITIALIZER,

50 - .index = {.avl_tree = { .root = NULL, .compar = appconfig_section_compare },

51 - .rwlock = AVL_LOCK_INITIALIZER } };

49 +struct config fd_config = APPCONFIG_INITIALIZER;

51 static netdata_idx_t fd_hash_values[NETDATA_FD_COUNTER];

52 static netdata_idx_t *fd_values = NULL;

src/collectors/ebpf.plugin/ebpf_filesystem.c

+1 -5

@@ -2,11 +2,7 @@

3 #include "ebpf_filesystem.h"

5 -struct config fs_config = { .first_section = NULL,

6 - .last_section = NULL,

7 - .mutex = NETDATA_MUTEX_INITIALIZER,

8 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

9 - .rwlock = AVL_LOCK_INITIALIZER } };

5 +struct config fs_config = APPCONFIG_INITIALIZER;

7 ebpf_local_maps_t ext4_maps[] = {{.name = "tbl_ext4", .internal_input = NETDATA_KEY_CALLS_SYNC,

8 .user_input = 0, .type = NETDATA_EBPF_MAP_STATIC,

src/collectors/ebpf.plugin/ebpf_hardirq.c

+1 -5

@@ -3,11 +3,7 @@

3 #include "ebpf.h"

4 #include "ebpf_hardirq.h"

6 -struct config hardirq_config = { .first_section = NULL,

7 - .last_section = NULL,

8 - .mutex = NETDATA_MUTEX_INITIALIZER,

9 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

10 - .rwlock = AVL_LOCK_INITIALIZER } };

6 +struct config hardirq_config = APPCONFIG_INITIALIZER;

8 static ebpf_local_maps_t hardirq_maps[] = {

9 {

src/collectors/ebpf.plugin/ebpf_mdflush.c

+1 -5

@@ -3,11 +3,7 @@

3 #include "ebpf.h"

4 #include "ebpf_mdflush.h"

6 -struct config mdflush_config = { .first_section = NULL,

7 - .last_section = NULL,

8 - .mutex = NETDATA_MUTEX_INITIALIZER,

9 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

10 - .rwlock = AVL_LOCK_INITIALIZER } };

6 +struct config mdflush_config = APPCONFIG_INITIALIZER;

8 #define MDFLUSH_MAP_COUNT 0

9 static ebpf_local_maps_t mdflush_maps[] = {

src/collectors/ebpf.plugin/ebpf_mount.c

+1 -3

@@ -22,9 +22,7 @@ static char *mount_dimension_name[NETDATA_EBPF_MOUNT_SYSCALL] = { "mount", "umou

22 static netdata_syscall_stat_t mount_aggregated_data[NETDATA_EBPF_MOUNT_SYSCALL];

23 static netdata_publish_syscall_t mount_publish_aggregated[NETDATA_EBPF_MOUNT_SYSCALL];

25 -struct config mount_config = { .first_section = NULL, .last_section = NULL, .mutex = NETDATA_MUTEX_INITIALIZER,

26 - .index = {.avl_tree = { .root = NULL, .compar = appconfig_section_compare },

27 - .rwlock = AVL_LOCK_INITIALIZER } };

25 +struct config mount_config = APPCONFIG_INITIALIZER;

27 static netdata_idx_t mount_hash_values[NETDATA_MOUNT_END];

src/collectors/ebpf.plugin/ebpf_oomkill.c

+1 -5

@@ -3,11 +3,7 @@

3 #include "ebpf.h"

4 #include "ebpf_oomkill.h"

6 -struct config oomkill_config = { .first_section = NULL,

7 - .last_section = NULL,

8 - .mutex = NETDATA_MUTEX_INITIALIZER,

9 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

10 - .rwlock = AVL_LOCK_INITIALIZER } };

6 +struct config oomkill_config = APPCONFIG_INITIALIZER;

8 #define OOMKILL_MAP_KILLCNT 0

9 static ebpf_local_maps_t oomkill_maps[] = {

src/collectors/ebpf.plugin/ebpf_process.c

+1 -5

@@ -57,11 +57,7 @@ ebpf_process_stat_t *process_stat_vector = NULL;

57 static netdata_syscall_stat_t process_aggregated_data[NETDATA_KEY_PUBLISH_PROCESS_END];

58 static netdata_publish_syscall_t process_publish_aggregated[NETDATA_KEY_PUBLISH_PROCESS_END];

60 -struct config process_config = { .first_section = NULL,

61 - .last_section = NULL,

62 - .mutex = NETDATA_MUTEX_INITIALIZER,

63 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

64 - .rwlock = AVL_LOCK_INITIALIZER } };

60 +struct config process_config = APPCONFIG_INITIALIZER;

62 /*****************************************************************

63 *

src/collectors/ebpf.plugin/ebpf_shm.c

+1 -5

@@ -12,11 +12,7 @@ netdata_ebpf_shm_t *shm_vector = NULL;

12 static netdata_idx_t shm_hash_values[NETDATA_SHM_END];

13 static netdata_idx_t *shm_values = NULL;

15 -struct config shm_config = { .first_section = NULL,

16 - .last_section = NULL,

17 - .mutex = NETDATA_MUTEX_INITIALIZER,

18 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

19 - .rwlock = AVL_LOCK_INITIALIZER } };

15 +struct config shm_config = APPCONFIG_INITIALIZER;

17 static ebpf_local_maps_t shm_maps[] = {{.name = "tbl_pid_shm", .internal_input = ND_EBPF_DEFAULT_PID_SIZE,

18 .user_input = 0,

src/collectors/ebpf.plugin/ebpf_socket.c

+9 -13

@@ -77,11 +77,7 @@ netdata_socket_t *socket_values;

77 ebpf_network_viewer_port_list_t *listen_ports = NULL;

78 ebpf_addresses_t tcp_v6_connect_address = {.function = "tcp_v6_connect", .hash = 0, .addr = 0, .type = 0};

80 -struct config socket_config = { .first_section = NULL,

81 - .last_section = NULL,

82 - .mutex = NETDATA_MUTEX_INITIALIZER,

83 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

84 - .rwlock = AVL_LOCK_INITIALIZER } };

80 +struct config socket_config = APPCONFIG_INITIALIZER;

82 netdata_ebpf_targets_t socket_targets[] = { {.name = "inet_csk_accept", .mode = EBPF_LOAD_PROBE},

83 {.name = "tcp_retransmit_skb", .mode = EBPF_LOAD_PROBE},

@@ -2708,7 +2704,7 @@ static void ebpf_socket_initialize_global_vectors()

2704 * @param hash the calculated hash for the dimension name.

2705 * @param name the dimension name.

2706 */

2711 -static void ebpf_link_dimension_name(char *port, uint32_t hash, char *value)

2707 +static void ebpf_link_dimension_name(const char *port, uint32_t hash, const char *value)

2708 {

2709 int test = str2i(port);

2710 if (test < NETDATA_MINIMUM_PORT_VALUE || test > NETDATA_MAXIMUM_PORT_VALUE){

@@ -2753,15 +2749,15 @@ static void ebpf_link_dimension_name(char *port, uint32_t hash, char *value)

2749 *

2750 * @param cfg the configuration structure

2751 */

2752 +

2753 +static bool config_service_value_cb(void *data __maybe_unused, const char *name, const char *value) {

2754 + ebpf_link_dimension_name(name, simple_hash(name), value);

2755 + return true;

2756 +}

2757 +

2758 void ebpf_parse_service_name_section(struct config *cfg)

2759 {

2758 - struct section *co = appconfig_get_section(cfg, EBPF_SERVICE_NAME_SECTION);

2759 - if (co) {

2760 - struct config_option *cv;

2761 - for (cv = co->values; cv ; cv = cv->next) {

2762 - ebpf_link_dimension_name(cv->name, cv->hash, cv->value);

2763 - }

2764 - }

2760 + appconfig_foreach_value_in_section(cfg, EBPF_SERVICE_NAME_SECTION, config_service_value_cb, NULL);

2761

2762 // Always associated the default port to Netdata

2763 ebpf_network_viewer_dim_name_t *names = network_viewer_opt.names;

src/collectors/ebpf.plugin/ebpf_socket.h

+2 -2

@@ -339,8 +339,8 @@ extern ebpf_network_viewer_port_list_t *listen_ports;

339 void update_listen_table(uint16_t value, uint16_t proto, netdata_passive_connection_t *values);

340 void ebpf_fill_ip_list_unsafe(ebpf_network_viewer_ip_list_t **out, ebpf_network_viewer_ip_list_t *in, char *table);

341 void ebpf_parse_service_name_section(struct config *cfg);

342 -void ebpf_parse_ips_unsafe(char *ptr);

343 -void ebpf_parse_ports(char *ptr);

342 +void ebpf_parse_ips_unsafe(const char *ptr);

343 +void ebpf_parse_ports(const char *ptr);

344 void ebpf_socket_read_open_connections(BUFFER *buf, struct ebpf_module *em);

345 void ebpf_socket_fill_publish_apps(ebpf_socket_publish_apps_t *curr, netdata_socket_t *ns);

346

src/collectors/ebpf.plugin/ebpf_softirq.c

+1 -5

@@ -3,11 +3,7 @@

3 #include "ebpf.h"

4 #include "ebpf_softirq.h"

6 -struct config softirq_config = { .first_section = NULL,

7 - .last_section = NULL,

8 - .mutex = NETDATA_MUTEX_INITIALIZER,

9 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

10 - .rwlock = AVL_LOCK_INITIALIZER } };

6 +struct config softirq_config = APPCONFIG_INITIALIZER;

8 #define SOFTIRQ_MAP_LATENCY 0

9 static ebpf_local_maps_t softirq_maps[] = {

src/collectors/ebpf.plugin/ebpf_swap.c

+1 -5

@@ -12,11 +12,7 @@ static netdata_idx_t *swap_values = NULL;

13 netdata_ebpf_swap_t *swap_vector = NULL;

15 -struct config swap_config = { .first_section = NULL,

16 - .last_section = NULL,

17 - .mutex = NETDATA_MUTEX_INITIALIZER,

18 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

19 - .rwlock = AVL_LOCK_INITIALIZER } };

15 +struct config swap_config = APPCONFIG_INITIALIZER;

17 static ebpf_local_maps_t swap_maps[] = {{.name = "tbl_pid_swap", .internal_input = ND_EBPF_DEFAULT_PID_SIZE,

18 .user_input = 0,

src/collectors/ebpf.plugin/ebpf_sync.c

+1 -5

@@ -100,11 +100,7 @@ ebpf_local_maps_t sync_file_range_maps[] = {{.name = "tbl_syncfr", .internal_inp

100 #endif

101 }};

102

103 -struct config sync_config = { .first_section = NULL,

104 - .last_section = NULL,

105 - .mutex = NETDATA_MUTEX_INITIALIZER,

106 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

107 - .rwlock = AVL_LOCK_INITIALIZER } };

103 +struct config sync_config = APPCONFIG_INITIALIZER;

104

105 netdata_ebpf_targets_t sync_targets[] = { {.name = NETDATA_SYSCALLS_SYNC, .mode = EBPF_LOAD_TRAMPOLINE},

106 {.name = NETDATA_SYSCALLS_SYNCFS, .mode = EBPF_LOAD_TRAMPOLINE},

src/collectors/ebpf.plugin/ebpf_vfs.c

+1 -5

@@ -52,11 +52,7 @@ struct netdata_static_thread ebpf_read_vfs = {

52 .start_routine = NULL

53 };

55 -struct config vfs_config = { .first_section = NULL,

56 - .last_section = NULL,

57 - .mutex = NETDATA_MUTEX_INITIALIZER,

58 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

59 - .rwlock = AVL_LOCK_INITIALIZER } };

55 +struct config vfs_config = APPCONFIG_INITIALIZER;

57 netdata_ebpf_targets_t vfs_targets[] = { {.name = "vfs_write", .mode = EBPF_LOAD_TRAMPOLINE},

58 {.name = "vfs_writev", .mode = EBPF_LOAD_TRAMPOLINE},

src/collectors/idlejitter.plugin/integrations/idle_os_jitter.md

+2 -2

@@ -109,8 +109,8 @@ This integration only supports a single configuration option, and most users wil

109

110

112 -|:----|:-----------|:-------|:--------:|

113 -| loop time in ms | Specifies the target time for the data collection thread to sleep, measured in miliseconds. | 20 | no |

112 +|:----|:-----------|:--------|:--------:|

113 +| loop time | Specifies the target time for the data collection thread to sleep. | 20ms | no |

114

115 #### Examples

116 There are no configuration examples.

src/collectors/idlejitter.plugin/metadata.yaml

+2 -2

@@ -55,10 +55,10 @@ modules:

55 title: ''

56 enabled: false

57 list:

58 - - name: loop time in ms

58 + - name: loop time

59 description: >

60 Specifies the target time for the data collection thread to sleep, measured in miliseconds.

61 - default_value: 20

61 + default_value: 20ms

62 required: false

63 examples:

64 folding:

src/collectors/idlejitter.plugin/plugin_idlejitter.c

+2 -2

@@ -22,9 +22,9 @@ void *cpuidlejitter_main(void *ptr) {

22 worker_register("IDLEJITTER");

23 worker_register_job_name(0, "measurements");

25 - usec_t sleep_ut = config_get_number("plugin:idlejitter", "loop time in ms", CPU_IDLEJITTER_SLEEP_TIME_MS) * USEC_PER_MS;

25 + usec_t sleep_ut = config_get_duration_ms("plugin:idlejitter", "loop time", CPU_IDLEJITTER_SLEEP_TIME_MS) * USEC_PER_MS;

26 if(sleep_ut <= 0) {

27 - config_set_number("plugin:idlejitter", "loop time in ms", CPU_IDLEJITTER_SLEEP_TIME_MS);

27 + config_set_duration_ms("plugin:idlejitter", "loop time", CPU_IDLEJITTER_SLEEP_TIME_MS);

28 sleep_ut = CPU_IDLEJITTER_SLEEP_TIME_MS * USEC_PER_MS;

29 }

src/collectors/plugins.d/plugins_d.c

+1 -1

@@ -321,7 +321,7 @@ void *pluginsd_main(void *ptr) {

321 cd->unsafe.enabled = enabled;

322 cd->unsafe.running = false;

323

324 - cd->update_every = (int)config_get_number(cd->id, "update every", localhost->rrd_update_every);

324 + cd->update_every = (int)config_get_duration_seconds(cd->id, "update every", localhost->rrd_update_every);

325 cd->started_t = now_realtime_sec();

326

327 char *def = "";

src/collectors/proc.plugin/README.md

+2 -2

@@ -133,7 +133,7 @@ Then edit `netdata.conf` and find the following section. This is the basic plugi

133 # extended operations for all disks = auto

134 # backlog for all disks = auto

135 # bcache for all disks = auto

136 - # bcache priority stats update every = 0

136 + # bcache priority stats update every = off

137 # remove charts of removed disks = yes

138 # path to get block device = /sys/block/%s

139 # path to get block device bcache = /sys/block/%s/bcache

@@ -578,7 +578,7 @@ Default configuration will monitor only enabled infiniband ports, and refresh ne

578 # hardware errors counters = auto

579 # monitor only ports being active = auto

580 # disable by default interfaces matching =

581 - # refresh ports state every seconds = 30

581 + # refresh ports state every = 30s

582 ```

583

584 ## AMD GPUs

src/collectors/proc.plugin/ipc.c

+4 -4

@@ -182,7 +182,7 @@ static inline int ipc_sem_get_status(struct ipc_status *st) {

182 return 0;

183 }

184

185 -int ipc_msq_get_info(char *msg_filename, struct message_queue **message_queue_root) {

185 +static int ipc_msq_get_info(const char *msg_filename, struct message_queue **message_queue_root) {

186 static procfile *ff;

187 struct message_queue *msq;

188

@@ -238,7 +238,7 @@ int ipc_msq_get_info(char *msg_filename, struct message_queue **message_queue_ro

238 return 0;

239 }

240

241 -int ipc_shm_get_info(char *shm_filename, struct shm_stats *shm) {

241 +static int ipc_shm_get_info(const char *shm_filename, struct shm_stats *shm) {

242 static procfile *ff;

243

244 if(unlikely(!ff)) {

@@ -287,10 +287,10 @@ int do_ipc(int update_every, usec_t dt) {

287 static const RRDVAR_ACQUIRED *arrays_max = NULL, *semaphores_max = NULL;

288 static RRDSET *st_arrays = NULL;

289 static RRDDIM *rd_arrays = NULL;

290 - static char *msg_filename = NULL;

290 + static const char *msg_filename = NULL;

291 static struct message_queue *message_queue_root = NULL;

292 static long long dimensions_limit;

293 - static char *shm_filename = NULL;

293 + static const char *shm_filename = NULL;

294

295 if(unlikely(do_sem == -1)) {

296 do_msg = config_get_boolean("plugin:proc:ipc", "message queues", CONFIG_BOOLEAN_YES);

src/collectors/proc.plugin/plugin_proc.c

+1 -1

@@ -279,7 +279,7 @@ int get_numa_node_count(void)

279

280 char name[FILENAME_MAX + 1];

281 snprintfz(name, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/sys/devices/system/node");

282 - char *dirname = config_get("plugin:proc:/sys/devices/system/node", "directory to monitor", name);

282 + const char *dirname = config_get("plugin:proc:/sys/devices/system/node", "directory to monitor", name);

283

284 DIR *dir = opendir(dirname);

285 if (dir) {

src/collectors/proc.plugin/proc_diskstats.c

+11 -11

@@ -180,16 +180,16 @@ static struct disk {

180

181 #define rrdset_obsolete_and_pointer_null(st) do { if(st) { rrdset_is_obsolete___safe_from_collector_thread(st); (st) = NULL; } } while(st)

182

183 -static char *path_to_sys_dev_block_major_minor_string = NULL;

184 -static char *path_to_sys_block_device = NULL;

185 -static char *path_to_sys_block_device_bcache = NULL;

186 -static char *path_to_sys_devices_virtual_block_device = NULL;

187 -static char *path_to_device_mapper = NULL;

188 -static char *path_to_dev_disk = NULL;

189 -static char *path_to_sys_block = NULL;

190 -static char *path_to_device_label = NULL;

191 -static char *path_to_device_id = NULL;

192 -static char *path_to_veritas_volume_groups = NULL;

183 +static const char *path_to_sys_dev_block_major_minor_string = NULL;

184 +static const char *path_to_sys_block_device = NULL;

185 +static const char *path_to_sys_block_device_bcache = NULL;

186 +static const char *path_to_sys_devices_virtual_block_device = NULL;

187 +static const char *path_to_device_mapper = NULL;

188 +static const char *path_to_dev_disk = NULL;

189 +static const char *path_to_sys_block = NULL;

190 +static const char *path_to_device_label = NULL;

191 +static const char *path_to_device_id = NULL;

192 +static const char *path_to_veritas_volume_groups = NULL;

193 static int name_disks_by_id = CONFIG_BOOLEAN_NO;

194 static int global_bcache_priority_stats_update_every = 0; // disabled by default

195

@@ -1374,7 +1374,7 @@ int do_proc_diskstats(int update_every, usec_t dt) {

1374 global_do_ext = config_get_boolean_ondemand(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "extended operations for all disks", global_do_ext);

1375 global_do_backlog = config_get_boolean_ondemand(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "backlog for all disks", global_do_backlog);

1376 global_do_bcache = config_get_boolean_ondemand(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "bcache for all disks", global_do_bcache);

1377 - global_bcache_priority_stats_update_every = (int)config_get_number(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "bcache priority stats update every", global_bcache_priority_stats_update_every);

1377 + global_bcache_priority_stats_update_every = (int)config_get_duration_seconds(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "bcache priority stats update every", global_bcache_priority_stats_update_every);

1378

1379 global_cleanup_removed_disks = config_get_boolean(CONFIG_SECTION_PLUGIN_PROC_DISKSTATS, "remove charts of removed disks" , global_cleanup_removed_disks);

1380

src/collectors/proc.plugin/proc_mdstat.c

+1 -1

@@ -89,7 +89,7 @@ int do_proc_mdstat(int update_every, usec_t dt)

89 static int do_health = -1, do_nonredundant = -1, do_disks = -1, do_operations = -1, do_mismatch = -1,

90 do_mismatch_config = -1;

91 static int make_charts_obsolete = -1;

92 - static char *mdstat_filename = NULL, *mismatch_cnt_filename = NULL;

92 + static const char *mdstat_filename = NULL, *mismatch_cnt_filename = NULL;

93 static struct raid *raids = NULL;

94 static size_t raids_allocated = 0;

95 size_t raids_num = 0, raid_idx = 0, redundant_num = 0;

src/collectors/proc.plugin/proc_net_sockstat.c

+1 -1

@@ -128,7 +128,7 @@ int do_proc_net_sockstat(int update_every, usec_t dt) {

128 do_frag_sockets = config_get_boolean_ondemand("plugin:proc:/proc/net/sockstat", "ipv4 FRAG sockets", CONFIG_BOOLEAN_AUTO);

129 do_frag_mem = config_get_boolean_ondemand("plugin:proc:/proc/net/sockstat", "ipv4 FRAG memory", CONFIG_BOOLEAN_AUTO);

130

131 - update_constants_every = config_get_number("plugin:proc:/proc/net/sockstat", "update constants every", update_constants_every);

131 + update_constants_every = config_get_duration_seconds("plugin:proc:/proc/net/sockstat", "update constants every", update_constants_every);

132 update_constants_count = update_constants_every;

133

134 arl_sockets = arl_create("sockstat/sockets", arl_callback_str2kernel_uint_t, 60);

src/collectors/proc.plugin/proc_net_stat_conntrack.c

+1 -1

@@ -11,7 +11,7 @@ int do_proc_net_stat_conntrack(int update_every, usec_t dt) {

11 static int do_sockets = -1, do_new = -1, do_changes = -1, do_expect = -1, do_search = -1, do_errors = -1;

12 static usec_t get_max_every = 10 * USEC_PER_SEC, usec_since_last_max = 0;

13 static int read_full = 1;

14 - static char *nf_conntrack_filename, *nf_conntrack_count_filename, *nf_conntrack_max_filename;

14 + static const char *nf_conntrack_filename, *nf_conntrack_count_filename, *nf_conntrack_max_filename;

15 static const RRDVAR_ACQUIRED *rrdvar_max = NULL;

17 unsigned long long aentries = 0, asearched = 0, afound = 0, anew = 0, ainvalid = 0, aignore = 0, adelete = 0, adelete_list = 0,

src/collectors/proc.plugin/proc_net_wireless.c

+1 -1

@@ -208,7 +208,7 @@ int do_proc_net_wireless(int update_every, usec_t dt)

208 UNUSED(dt);

209 static procfile *ff = NULL;

210 static int do_status, do_quality = -1, do_discarded_packets, do_beacon;

211 - static char *proc_net_wireless_filename = NULL;

211 + static const char *proc_net_wireless_filename = NULL;

212

213 if (unlikely(do_quality == -1)) {

214 char filename[FILENAME_MAX + 1];

src/collectors/proc.plugin/proc_pressure.c

+2 -3

@@ -158,7 +158,7 @@ int do_proc_pressure(int update_every, usec_t dt) {

158 int i;

159

160 static usec_t next_pressure_dt = 0;

161 - static char *base_path = NULL;

161 + static const char *base_path = NULL;

162

163 update_every = (update_every < MIN_PRESSURE_UPDATE_EVERY) ? MIN_PRESSURE_UPDATE_EVERY : update_every;

164 pressure_update_every = update_every;

@@ -170,9 +170,8 @@ int do_proc_pressure(int update_every, usec_t dt) {

170 return 0;

171 }

172

173 - if (unlikely(!base_path)) {

173 + if (unlikely(!base_path))

174 base_path = config_get(CONFIG_SECTION_PLUGIN_PROC_PRESSURE, "base path of pressure metrics", "/proc/pressure");

175 - }

175

176 for (i = 0; i < PRESSURE_NUM_RESOURCES; i++) {

177 procfile *ff = resource_info[i].pf;

src/collectors/proc.plugin/proc_spl_kstat_zfs.c

+1 -1

@@ -18,7 +18,7 @@ int do_proc_spl_kstat_zfs_arcstats(int update_every, usec_t dt) {

19 static int do_zfs_stats = 0;

20 static procfile *ff = NULL;

21 - static char *dirname = NULL;

21 + static const char *dirname = NULL;

22 static ARL_BASE *arl_base = NULL;

24 arcstats.l2exist = -1;

src/collectors/proc.plugin/proc_stat.c

+3 -3

@@ -293,7 +293,7 @@ static void* wake_cpu_thread(void* core) {

293 return 0;

294 }

295

296 -static int read_schedstat(char *schedstat_filename, struct per_core_cpuidle_chart **cpuidle_charts_address, size_t *schedstat_cores_found) {

296 +static int read_schedstat(const char *schedstat_filename, struct per_core_cpuidle_chart **cpuidle_charts_address, size_t *schedstat_cores_found) {

297 static size_t cpuidle_charts_len = 0;

298 static procfile *ff = NULL;

299 struct per_core_cpuidle_chart *cpuidle_charts = *cpuidle_charts_address;

@@ -373,7 +373,7 @@ static int read_one_state(char *buf, const char *filename, int *fd) {

373 return 1;

374 }

375

376 -static int read_cpuidle_states(char *cpuidle_name_filename , char *cpuidle_time_filename, struct per_core_cpuidle_chart *cpuidle_charts, size_t core) {

376 +static int read_cpuidle_states(const char *cpuidle_name_filename, const char *cpuidle_time_filename, struct per_core_cpuidle_chart *cpuidle_charts, size_t core) {

377 char filename[FILENAME_MAX + 1];

378 static char next_state_filename[FILENAME_MAX + 1];

379 struct stat stbuf;

@@ -484,7 +484,7 @@ int do_proc_stat(int update_every, usec_t dt) {

484 static int do_cpu = -1, do_cpu_cores = -1, do_interrupts = -1, do_context = -1, do_forks = -1, do_processes = -1,

485 do_core_throttle_count = -1, do_package_throttle_count = -1, do_cpu_freq = -1, do_cpuidle = -1;

486 static uint32_t hash_intr, hash_ctxt, hash_processes, hash_procs_running, hash_procs_blocked;

487 - static char *core_throttle_count_filename = NULL, *package_throttle_count_filename = NULL, *scaling_cur_freq_filename = NULL,

487 + static const char *core_throttle_count_filename = NULL, *package_throttle_count_filename = NULL, *scaling_cur_freq_filename = NULL,

488 *time_in_state_filename = NULL, *schedstat_filename = NULL, *cpuidle_name_filename = NULL, *cpuidle_time_filename = NULL;

489 static const RRDVAR_ACQUIRED *cpus_var = NULL;

490 static int accurate_freq_avail = 0, accurate_freq_is_used = 0;

src/collectors/proc.plugin/proc_uptime.c

+1 -1

@@ -5,7 +5,7 @@

5 int do_proc_uptime(int update_every, usec_t dt) {

6 (void)dt;

8 - static char *uptime_filename = NULL;

8 + static const char *uptime_filename = NULL;

9 if(!uptime_filename) {

10 char filename[FILENAME_MAX + 1];

11 snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/proc/uptime");

src/collectors/proc.plugin/sys_class_drm.c

+1 -1

@@ -837,7 +837,7 @@ int do_sys_class_drm(int update_every, usec_t dt) {

837 if(unlikely(!drm_dir)) {

838 char filename[FILENAME_MAX + 1];

839 snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/sys/class/drm");

840 - char *drm_dir_name = config_get(CONFIG_SECTION_PLUGIN_PROC_DRM, "directory to monitor", filename);

840 + const char *drm_dir_name = config_get(CONFIG_SECTION_PLUGIN_PROC_DRM, "directory to monitor", filename);

841 if(unlikely(NULL == (drm_dir = opendir(drm_dir_name)))){

842 collector_error("Cannot read directory '%s'", drm_dir_name);

843 return 1;

src/collectors/proc.plugin/sys_class_infiniband.c

+2 -2

@@ -302,7 +302,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt)

302 static int initialized = 0;

303 static int enable_new_ports = -1, enable_only_active = CONFIG_BOOLEAN_YES;

304 static int do_bytes = -1, do_packets = -1, do_errors = -1, do_hwpackets = -1, do_hwerrors = -1;

305 - static char *sys_class_infiniband_dirname = NULL;

305 + static const char *sys_class_infiniband_dirname = NULL;

306

307 static long long int dt_to_refresh_ports = 0, last_refresh_ports_usec = 0;

308

@@ -332,7 +332,7 @@ int do_sys_class_infiniband(int update_every, usec_t dt)

332 SIMPLE_PATTERN_EXACT, true);

333

334 dt_to_refresh_ports =

335 - config_get_number(CONFIG_SECTION_PLUGIN_SYS_CLASS_INFINIBAND, "refresh ports state every seconds", 30) *

335 + config_get_duration_seconds(CONFIG_SECTION_PLUGIN_SYS_CLASS_INFINIBAND, "refresh ports state every", 30) *

336 USEC_PER_SEC;

337 if (dt_to_refresh_ports < 0)

338 dt_to_refresh_ports = 0;

src/collectors/proc.plugin/sys_class_power_supply.c

+1 -1

@@ -199,7 +199,7 @@ int do_sys_class_power_supply(int update_every, usec_t dt) {

199 (void)dt;

200 static int do_capacity = -1, do_power = -1, do_property[3] = {-1};

201 static int keep_fds_open = CONFIG_BOOLEAN_NO, keep_fds_open_config = -1;

202 - static char *dirname = NULL;

202 + static const char *dirname = NULL;

203

204 if(unlikely(do_capacity == -1)) {

205 do_capacity = config_get_boolean("plugin:proc:/sys/class/power_supply", "battery capacity", CONFIG_BOOLEAN_YES);

src/collectors/proc.plugin/sys_devices_pci_aer.c

+1 -1

@@ -2,7 +2,7 @@

3 #include "plugin_proc.h"

5 -static char *pci_aer_dirname = NULL;

5 +static const char *pci_aer_dirname = NULL;

7 typedef enum __attribute__((packed)) {

8 AER_DEV_NONFATAL = (1 << 0),

src/collectors/proc.plugin/sys_devices_system_edac_mc.c

+1 -1

@@ -37,7 +37,7 @@ struct mc {

37 };

39 static struct mc *mc_root = NULL;

40 -static char *mc_dirname = NULL;

40 +static const char *mc_dirname = NULL;

42 static void find_all_mc() {

43 char name[FILENAME_MAX + 1];

src/collectors/proc.plugin/sys_devices_system_node.c

+1 -1

@@ -15,7 +15,7 @@ static int find_all_nodes() {

15 int numa_node_count = 0;

16 char name[FILENAME_MAX + 1];

17 snprintfz(name, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/sys/devices/system/node");

18 - char *dirname = config_get("plugin:proc:/sys/devices/system/node", "directory to monitor", name);

18 + const char *dirname = config_get("plugin:proc:/sys/devices/system/node", "directory to monitor", name);

20 DIR *dir = opendir(dirname);

21 if(!dir) {

src/collectors/proc.plugin/sys_fs_btrfs.c

+2 -2

@@ -678,7 +678,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) {

678 , do_error_stats = CONFIG_BOOLEAN_AUTO;

679

680 static usec_t refresh_delta = 0, refresh_every = 60 * USEC_PER_SEC;

681 - static char *btrfs_path = NULL;

681 + static const char *btrfs_path = NULL;

682

683 (void)dt;

684

@@ -689,7 +689,7 @@ int do_sys_fs_btrfs(int update_every, usec_t dt) {

689 snprintfz(filename, FILENAME_MAX, "%s%s", netdata_configured_host_prefix, "/sys/fs/btrfs");

690 btrfs_path = config_get("plugin:proc:/sys/fs/btrfs", "path to monitor", filename);

691

692 - refresh_every = config_get_number("plugin:proc:/sys/fs/btrfs", "check for btrfs changes every", refresh_every / USEC_PER_SEC) * USEC_PER_SEC;

692 + refresh_every = config_get_duration_seconds("plugin:proc:/sys/fs/btrfs", "check for btrfs changes every", refresh_every / USEC_PER_SEC) * USEC_PER_SEC;

693 refresh_delta = refresh_every;

694

695 do_allocation_disks = config_get_boolean_ondemand("plugin:proc:/sys/fs/btrfs", "physical disks allocation", do_allocation_disks);

src/collectors/profile.plugin/plugin_profile.cc

+4 -2

@@ -194,9 +194,11 @@ static void profile_main_cleanup(void *pptr) {

194 extern "C" void *profile_main(void *ptr) {

195 CLEANUP_FUNCTION_REGISTER(profile_main_cleanup) cleanup_ptr = ptr;

196

197 - int UpdateEvery = (int) config_get_number(CONFIG_SECTION_PROFILE, "update every", 1);

198 - if (UpdateEvery < localhost->rrd_update_every)

197 + int UpdateEvery = (int) config_get_duration_seconds(CONFIG_SECTION_PROFILE, "update every", 1);

198 + if (UpdateEvery < localhost->rrd_update_every) {

199 UpdateEvery = localhost->rrd_update_every;

200 + config_set_duration_seconds(CONFIG_SECTION_PROFILE, "update every", UpdateEvery);

201 + }

202

203 // pick low-default values, in case this plugin is ever enabled accidentaly.

204 size_t NumThreads = config_get_number(CONFIG_SECTION_PROFILE, "number of threads", 2);

src/collectors/statsd.plugin/README.md

+4 -4

@@ -170,11 +170,11 @@ You can find the configuration at `/etc/netdata/netdata.conf`:

170 [statsd]

171 # enabled = yes

172 # decimal detail = 1000

173 - # update every (flushInterval) = 1

173 + # update every (flushInterval) = 1s

174 # udp messages to process at once = 10

175 # create private charts for metrics matching = *

176 # max private charts hard limit = 1000

177 - # cleanup obsolete charts after secs = 0

177 + # cleanup obsolete charts after = 0

178 # private charts memory mode = save

179 # private charts history = 3996

180 # histograms and timers percentile (percentThreshold) = 95.00000

@@ -204,7 +204,7 @@ You can find the configuration at `/etc/netdata/netdata.conf`:

204

205 is a space separated list of IPs and ports to listen to. The format is `PROTOCOL:IP:PORT` - if `PORT` is omitted, the `default port` will be used. If `IP` is IPv6, it needs to be enclosed in `[]`. `IP` can also be `*` (to listen on all IPs) or even a hostname.

206

207 -- `update every (flushInterval) = 1` seconds, controls the frequency StatsD will push the collected metrics to Netdata charts.

207 +- `update every (flushInterval) = 1s` controls the frequency StatsD will push the collected metrics to Netdata charts.

208

209 - `decimal detail = 1000` controls the number of fractional digits in gauges and histograms. Netdata collects metrics using signed 64-bit integers and their fractional detail is controlled using multipliers and divisors. This setting is used to multiply all collected values to convert them to integers and is also set as the divisors, so that the final data will be a floating point number with this fractional detail (1000 = X.0 - X.999, 10000 = X.0 - X.9999, etc).

210

@@ -238,7 +238,7 @@ The default behavior is to use the same settings as the rest of the Netdata Agen

238

239 For optimization reasons, Netdata imposes a hard limit on private metric charts. The limit is set via the `max private charts hard limit` setting (which defaults to 1000 charts). Metrics above this hard limit are still collected, but they can only be used in synthetic charts (once a metric is added to chart, it will be sent to backend servers too).

240

241 -If you have many ephemeral metrics collected (i.e. that you collect values for a certain amount of time), you can set the configuration option `set charts as obsolete after secs`. Setting a value in seconds here, means that Netdata will mark those metrics (and their private charts) as obsolete after the specified time has passed since the last sent metric value. Those charts will later be deleted according to the setting in `cleanup obsolete charts after secs`. Setting `set charts as obsolete after secs` to 0 (which is also the default value) will disable this functionality.

241 +If you have many ephemeral metrics collected (i.e. that you collect values for a certain amount of time), you can set the configuration option `set charts as obsolete after`. Setting a value in seconds here, means that Netdata will mark those metrics (and their private charts) as obsolete after the specified time has passed since the last sent metric value. Those charts will later be deleted according to the setting in `cleanup obsolete charts after`. Setting `set charts as obsolete after` to 0 (which is also the default value) will disable this functionality.

242

243 Example private charts (automatically generated without any configuration):

244

src/collectors/statsd.plugin/statsd.c

+23 -8

@@ -2491,10 +2491,11 @@ void *statsd_main(void *ptr) {

2491 statsd.enabled = config_get_boolean(CONFIG_SECTION_PLUGINS, "statsd", statsd.enabled);

2492

2493 statsd.update_every = default_rrd_update_every;

2494 - statsd.update_every = (int)config_get_number(CONFIG_SECTION_STATSD, "update every (flushInterval)", statsd.update_every);

2494 + statsd.update_every = (int)config_get_duration_seconds(CONFIG_SECTION_STATSD, "update every (flushInterval)", statsd.update_every);

2495 if(statsd.update_every < default_rrd_update_every) {

2496 collector_error("STATSD: minimum flush interval %d given, but the minimum is the update every of netdata. Using %d", statsd.update_every, default_rrd_update_every);

2497 statsd.update_every = default_rrd_update_every;

2498 + config_set_duration_seconds(CONFIG_SECTION_STATSD, "update every (flushInterval)", statsd.update_every);

2499 }

2500

2501 #ifdef HAVE_RECVMMSG

@@ -2504,13 +2505,26 @@ void *statsd_main(void *ptr) {

2505 statsd.charts_for = simple_pattern_create(

2506 config_get(CONFIG_SECTION_STATSD, "create private charts for metrics matching", "*"), NULL,

2507 SIMPLE_PATTERN_EXACT, true);

2507 - statsd.max_private_charts_hard = (size_t)config_get_number(CONFIG_SECTION_STATSD, "max private charts hard limit", (long long)statsd.max_private_charts_hard);

2508 - statsd.set_obsolete_after = (size_t)config_get_number(CONFIG_SECTION_STATSD, "set charts as obsolete after secs", (long long)statsd.set_obsolete_after);

2509 - statsd.decimal_detail = (collected_number)config_get_number(CONFIG_SECTION_STATSD, "decimal detail", (long long int)statsd.decimal_detail);

2510 - statsd.tcp_idle_timeout = (size_t) config_get_number(CONFIG_SECTION_STATSD, "disconnect idle tcp clients after seconds", (long long int)statsd.tcp_idle_timeout);

2511 - statsd.private_charts_hidden = (unsigned int)config_get_boolean(CONFIG_SECTION_STATSD, "private charts hidden", statsd.private_charts_hidden);

2508

2513 - statsd.histogram_percentile = (double)config_get_float(CONFIG_SECTION_STATSD, "histograms and timers percentile (percentThreshold)", statsd.histogram_percentile);

2509 + statsd.max_private_charts_hard =

2510 + (size_t)config_get_number(CONFIG_SECTION_STATSD, "max private charts hard limit", (long long)statsd.max_private_charts_hard);

2511 +

2512 + statsd.set_obsolete_after =

2513 + (size_t)config_get_duration_seconds(CONFIG_SECTION_STATSD, "set charts as obsolete after", (long long)statsd.set_obsolete_after);

2514 +

2515 + statsd.decimal_detail =

2516 + (collected_number)config_get_number(CONFIG_SECTION_STATSD, "decimal detail", (long long int)statsd.decimal_detail);

2517 +

2518 + statsd.tcp_idle_timeout =

2519 + (size_t) config_get_duration_seconds(CONFIG_SECTION_STATSD, "disconnect idle tcp clients after", (long long int)statsd.tcp_idle_timeout);

2520 +

2521 + statsd.private_charts_hidden =

2522 + (unsigned int)config_get_boolean(CONFIG_SECTION_STATSD, "private charts hidden", statsd.private_charts_hidden);

2523 +

2524 + statsd.histogram_percentile =

2525 + (double)config_get_double(

2526 + CONFIG_SECTION_STATSD, "histograms and timers percentile (percentThreshold)", statsd.histogram_percentile);

2527 +

2528 if(isless(statsd.histogram_percentile, 0) || isgreater(statsd.histogram_percentile, 100)) {

2529 collector_error("STATSD: invalid histograms and timers percentile %0.5f given", statsd.histogram_percentile);

2530 statsd.histogram_percentile = 95.0;

@@ -2521,7 +2535,8 @@ void *statsd_main(void *ptr) {

2535 statsd.histogram_percentile_str = strdupz(buffer);

2536 }

2537

2524 - statsd.dictionary_max_unique = config_get_number(CONFIG_SECTION_STATSD, "dictionaries max unique dimensions", statsd.dictionary_max_unique);

2538 + statsd.dictionary_max_unique =

2539 + config_get_number(CONFIG_SECTION_STATSD, "dictionaries max unique dimensions", statsd.dictionary_max_unique);

2540

2541 if(config_get_boolean(CONFIG_SECTION_STATSD, "add dimension for number of events received", 0)) {

2542 statsd.gauges.default_options |= STATSD_METRIC_OPTION_CHART_DIMENSION_COUNT;

src/collectors/tc.plugin/plugin_tc.c

+1 -1

@@ -912,7 +912,7 @@ void *tc_main(void *ptr) {

912 uint32_t first_hash;

913

914 snprintfz(command, TC_LINE_MAX, "%s/tc-qos-helper.sh", netdata_configured_primary_plugins_dir);

915 - char *tc_script = config_get("plugin:tc", "script to run to get tc values", command);

915 + const char *tc_script = config_get("plugin:tc", "script to run to get tc values", command);

916

917 while(service_running(SERVICE_COLLECTORS)) {

918 struct tc_device *device = NULL;

src/collectors/timex.plugin/plugin_timex.c

+4 -2

@@ -50,9 +50,11 @@ void *timex_main(void *ptr)

50 worker_register("TIMEX");

51 worker_register_job_name(0, "clock check");

53 - int update_every = (int)config_get_number(CONFIG_SECTION_TIMEX, "update every", 10);

54 - if (update_every < localhost->rrd_update_every)

53 + int update_every = (int)config_get_duration_seconds(CONFIG_SECTION_TIMEX, "update every", 10);

54 + if (update_every < localhost->rrd_update_every) {

55 update_every = localhost->rrd_update_every;

56 + config_set_duration_seconds(CONFIG_SECTION_TIMEX, "update every", update_every);

57 + }

59 int do_sync = config_get_boolean(CONFIG_SECTION_TIMEX, "clock synchronization state", CONFIG_BOOLEAN_YES);

60 int do_offset = config_get_boolean(CONFIG_SECTION_TIMEX, "time offset", CONFIG_BOOLEAN_YES);

src/daemon/commands.c

+3 -6

@@ -261,9 +261,8 @@ static cmd_status_t cmd_read_config_execute(char *args, char **message)

261 const char *conf_file = temp; /* "cloud" is cloud.conf, otherwise netdata.conf */

262 struct config *tmp_config = strcmp(conf_file, "cloud") ? &netdata_config : &cloud_config;

263

264 - char *value = appconfig_get(tmp_config, temp + offset + 1, temp + offset2 + 1, NULL);

265 - if (value == NULL)

266 - {

264 + const char *value = appconfig_get(tmp_config, temp + offset + 1, temp + offset2 + 1, NULL);

265 + if (value == NULL) {

266 netdata_log_error("Cannot execute read-config conf_file=%s section=%s / key=%s because no value set",

267 conf_file,

268 temp + offset + 1,

@@ -271,13 +270,11 @@ static cmd_status_t cmd_read_config_execute(char *args, char **message)

270 freez(temp);

271 return CMD_STATUS_FAILURE;

272 }

274 - else

275 - {

273 + else {

274 (*message) = strdupz(value);

275 freez(temp);

276 return CMD_STATUS_SUCCESS;

277 }

280 -

278 }

279

280 static cmd_status_t cmd_write_config_execute(char *args, char **message)

src/daemon/common.c

+15 -15

@@ -2,21 +2,21 @@

3 #include "common.h"

5 -char *netdata_configured_hostname = NULL;

6 -char *netdata_configured_user_config_dir = CONFIG_DIR;

7 -char *netdata_configured_stock_config_dir = LIBCONFIG_DIR;

8 -char *netdata_configured_log_dir = LOG_DIR;

9 -char *netdata_configured_primary_plugins_dir = PLUGINS_DIR;

10 -char *netdata_configured_web_dir = WEB_DIR;

11 -char *netdata_configured_cache_dir = CACHE_DIR;

12 -char *netdata_configured_varlib_dir = VARLIB_DIR;

13 -char *netdata_configured_lock_dir = VARLIB_DIR "/lock";

14 -char *netdata_configured_cloud_dir = VARLIB_DIR "/cloud.d";

15 -char *netdata_configured_home_dir = VARLIB_DIR;

16 -char *netdata_configured_host_prefix = NULL;

17 -char *netdata_configured_timezone = NULL;

18 -char *netdata_configured_abbrev_timezone = NULL;

19 -int32_t netdata_configured_utc_offset = 0;

5 +const char *netdata_configured_hostname = NULL;

6 +const char *netdata_configured_user_config_dir = CONFIG_DIR;

7 +const char *netdata_configured_stock_config_dir = LIBCONFIG_DIR;

8 +const char *netdata_configured_log_dir = LOG_DIR;

9 +const char *netdata_configured_primary_plugins_dir = PLUGINS_DIR;

10 +const char *netdata_configured_web_dir = WEB_DIR;

11 +const char *netdata_configured_cache_dir = CACHE_DIR;

12 +const char *netdata_configured_varlib_dir = VARLIB_DIR;

13 +const char *netdata_configured_lock_dir = VARLIB_DIR "/lock";

14 +const char *netdata_configured_cloud_dir = VARLIB_DIR "/cloud.d";

15 +const char *netdata_configured_home_dir = VARLIB_DIR;

16 +const char *netdata_configured_host_prefix = NULL;

17 +const char *netdata_configured_timezone = NULL;

18 +const char *netdata_configured_abbrev_timezone = NULL;

19 +int32_t netdata_configured_utc_offset = 0;

21 bool netdata_ready = false;

src/daemon/common.h

+14 -39

@@ -6,31 +6,6 @@

6 #include "libnetdata/libnetdata.h"

7 #include "libuv_workers.h"

9 -// ----------------------------------------------------------------------------

10 -// shortcuts for the default netdata configuration

11 -

12 -#define config_load(filename, overwrite_used, section) appconfig_load(&netdata_config, filename, overwrite_used, section)

13 -#define config_get(section, name, default_value) appconfig_get(&netdata_config, section, name, default_value)

14 -#define config_get_number(section, name, value) appconfig_get_number(&netdata_config, section, name, value)

15 -#define config_get_float(section, name, value) appconfig_get_float(&netdata_config, section, name, value)

16 -#define config_get_boolean(section, name, value) appconfig_get_boolean(&netdata_config, section, name, value)

17 -#define config_get_boolean_ondemand(section, name, value) appconfig_get_boolean_ondemand(&netdata_config, section, name, value)

18 -#define config_get_duration(section, name, value) appconfig_get_duration(&netdata_config, section, name, value)

19 -

20 -#define config_set(section, name, default_value) appconfig_set(&netdata_config, section, name, default_value)

21 -#define config_set_default(section, name, value) appconfig_set_default(&netdata_config, section, name, value)

22 -#define config_set_number(section, name, value) appconfig_set_number(&netdata_config, section, name, value)

23 -#define config_set_float(section, name, value) appconfig_set_float(&netdata_config, section, name, value)

24 -#define config_set_boolean(section, name, value) appconfig_set_boolean(&netdata_config, section, name, value)

25 -

26 -#define config_exists(section, name) appconfig_exists(&netdata_config, section, name)

27 -#define config_move(section_old, name_old, section_new, name_new) appconfig_move(&netdata_config, section_old, name_old, section_new, name_new)

28 -

29 -#define netdata_conf_generate(buffer, only_changed) appconfig_generate(&netdata_config, buffer, only_changed, true)

30 -

31 -#define config_section_destroy(section) appconfig_section_destroy_non_loaded(&netdata_config, section)

32 -#define config_section_option_destroy(section, name) appconfig_section_option_destroy_non_loaded(&netdata_config, section, name)

33 -

9 // ----------------------------------------------------------------------------

10 // netdata include files

@@ -96,20 +71,20 @@

71 #include "analytics.h"

73 // global netdata daemon variables

99 -extern char *netdata_configured_hostname;

100 -extern char *netdata_configured_user_config_dir;

101 -extern char *netdata_configured_stock_config_dir;

102 -extern char *netdata_configured_log_dir;

103 -extern char *netdata_configured_primary_plugins_dir;

104 -extern char *netdata_configured_web_dir;

105 -extern char *netdata_configured_cache_dir;

106 -extern char *netdata_configured_varlib_dir;

107 -extern char *netdata_configured_lock_dir;

108 -extern char *netdata_configured_cloud_dir;

109 -extern char *netdata_configured_home_dir;

110 -extern char *netdata_configured_host_prefix;

111 -extern char *netdata_configured_timezone;

112 -extern char *netdata_configured_abbrev_timezone;

74 +extern const char *netdata_configured_hostname;

75 +extern const char *netdata_configured_user_config_dir;

76 +extern const char *netdata_configured_stock_config_dir;

77 +extern const char *netdata_configured_log_dir;

78 +extern const char *netdata_configured_primary_plugins_dir;

79 +extern const char *netdata_configured_web_dir;

80 +extern const char *netdata_configured_cache_dir;

81 +extern const char *netdata_configured_varlib_dir;

82 +extern const char *netdata_configured_lock_dir;

83 +extern const char *netdata_configured_cloud_dir;

84 +extern const char *netdata_configured_home_dir;

85 +extern const char *netdata_configured_host_prefix;

86 +extern const char *netdata_configured_timezone;

87 +extern const char *netdata_configured_abbrev_timezone;

88 extern int32_t netdata_configured_utc_offset;

89 extern int netdata_anonymous_statistics_enabled;

src/daemon/config/README.md

+26 -28

@@ -86,24 +86,22 @@ Please note that your data history will be lost if you have modified `history` p

87 ### [db] section options

89 -| setting | default | info |

90 -|:---------------------------------------------:|:----------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

91 -| mode | `dbengine` | `dbengine`: The default for long-term metrics storage with efficient RAM and disk usage. Can be extended with `dbengine page cache size MB` and `dbengine disk space MB`. <br />`ram`: The round-robin database will be temporary and it will be lost when Netdata exits. <br />`alloc`: Similar to `ram`, but can significantly reduce memory usage, when combined with a low retention and does not support KSM. <br />`none`: Disables the database at this host, and disables health monitoring entirely, as that requires a database of metrics. Not to be used together with streaming. |

92 -| retention | `3600` | Used with `mode = ram/alloc`, not the default `mode = dbengine`. This number reflects the number of entries the `netdata` daemon will by default keep in memory for each chart dimension. Check [Memory Requirements](/src/database/README.md) for more information. |

93 -| storage tiers | `3` | The number of storage tiers you want to have in your dbengine. Check the tiering mechanism in the [dbengine's reference](/src/database/engine/README.md#tiering). You can have up to 5 tiers of data (including the _Tier 0_). This number ranges between 1 and 5. |

94 -| dbengine page cache size MB | `32` | Determines the amount of RAM in MiB that is dedicated to caching for _Tier 0_ Netdata metric values. |

95 -| dbengine tier **`N`** page cache size MB | `32` | Determines the amount of RAM in MiB that is dedicated for caching Netdata metric values of the **`N`** tier. <br /> `N belongs to [1..4]` |

96 -| dbengine disk space MB | `256` | Determines the amount of disk space in MiB that is dedicated to storing _Tier 0_ Netdata metric values and all related metadata describing them. This option is available **only for legacy configuration** (`Agent v1.23.2 and prior`). |

97 -| dbengine multihost disk space MB | `256` | Same functionality as `dbengine disk space MB`, but includes support for storing metrics streamed to a parent node by its children. Can be used in single-node environments as well. This setting is only for _Tier 0_ metrics. |

98 -| dbengine tier **`N`** multihost disk space MB | `256` | Same functionality as `dbengine multihost disk space MB`, but stores metrics of the **`N`** tier (both parent node and its children). Can be used in single-node environments as well. <br /> `N belongs to [1..4]` |

99 -| update every | `1` | The frequency in seconds, for data collection. For more information see the [performance guide](/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md). These metrics stored as _Tier 0_ data. Explore the tiering mechanism in the [dbengine's reference](/src/database/engine/README.md#tiering). |

100 -| dbengine tier **`N`** update every iterations | `60` | The down sampling value of each tier from the previous one. For each Tier, the greater by one Tier has N (equal to 60 by default) less data points of any metric it collects. This setting can take values from `2` up to `255`. <br /> `N belongs to [1..4]` |

101 -| dbengine tier **`N`** back fill | `New` | Specifies the strategy of recreating missing data on each Tier from the exact lower Tier. <br /> `New`: Sees the latest point on each Tier and save new points to it only if the exact lower Tier has available points for it's observation window (`dbengine tier N update every iterations` window). <br /> `none`: No back filling is applied. <br /> `N belongs to [1..4]` |

102 -| memory deduplication (ksm) | `yes` | When set to `yes`, Netdata will offer its in-memory round robin database and the dbengine page cache to kernel same page merging (KSM) for deduplication. For more information check [Memory Deduplication - Kernel Same Page Merging - KSM](/src/database/README.md#ksm) |

103 -| cleanup obsolete charts after secs | `3600` | See [monitoring ephemeral containers](/src/collectors/cgroups.plugin/README.md#monitoring-ephemeral-containers), also sets the timeout for cleaning up obsolete dimensions |

104 -| gap when lost iterations above | `1` | |

105 -| cleanup orphan hosts after secs | `3600` | How long to wait until automatically removing from the DB a remote Netdata host (child) that is no longer sending data. |

106 -| enable zero metrics | `no` | Set to `yes` to show charts when all their metrics are zero. |

89 +| setting | default | info |

90 +|:---------------------------------------------:|:-------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

91 +| mode | `dbengine` | `dbengine`: The default for long-term metrics storage with efficient RAM and disk usage. Can be extended with `dbengine page cache size` and `dbengine tier X retention size`. <br />`ram`: The round-robin database will be temporary and it will be lost when Netdata exits. <br />`alloc`: Similar to `ram`, but can significantly reduce memory usage, when combined with a low retention and does not support KSM. <br />`none`: Disables the database at this host, and disables health monitoring entirely, as that requires a database of metrics. Not to be used together with streaming. |

92 +| retention | `3600` | Used with `mode = ram/alloc`, not the default `mode = dbengine`. This number reflects the number of entries the `netdata` daemon will by default keep in memory for each chart dimension. Check [Memory Requirements](/src/database/README.md) for more information. |

93 +| storage tiers | `3` | The number of storage tiers you want to have in your dbengine. Check the tiering mechanism in the [dbengine's reference](/src/database/engine/README.md#tiering). You can have up to 5 tiers of data (including the _Tier 0_). This number ranges between 1 and 5. |

94 +| dbengine page cache size | `32MiB` | Determines the amount of RAM in MiB that is dedicated to caching for _Tier 0_ Netdata metric values. |

95 +| dbengine tier **`N`** retention size | `1GiB` | The disk space dedicated to metrics storage, per tier. Can be used in single-node environments as well. <br /> `N belongs to [1..4]` |

96 +| dbengine tier **`N`** retention time | `14d`, `3mo`, `1y`, `1y`, `1y` | The database retention, expressed in time. Can be used in single-node environments as well. <br /> `N belongs to [1..4]` |

97 +| update every | `1` | The frequency in seconds, for data collection. For more information see the [performance guide](/docs/netdata-agent/configuration/optimize-the-netdata-agents-performance.md). These metrics stored as _Tier 0_ data. Explore the tiering mechanism in the [dbengine's reference](/src/database/engine/README.md#tiering). |

98 +| dbengine tier **`N`** update every iterations | `60` | The down sampling value of each tier from the previous one. For each Tier, the greater by one Tier has N (equal to 60 by default) less data points of any metric it collects. This setting can take values from `2` up to `255`. <br /> `N belongs to [1..4]` |

99 +| dbengine tier back fill | `new` | Specifies the strategy of recreating missing data on higher database Tiers.<br /> `new`: Sees the latest point on each Tier and save new points to it only if the exact lower Tier has available points for it's observation window (`dbengine tier N update every iterations` window). <br /> `none`: No back filling is applied. <br /> `N belongs to [1..4]` |

100 +| memory deduplication (ksm) | `yes` | When set to `yes`, Netdata will offer its in-memory round robin database and the dbengine page cache to kernel same page merging (KSM) for deduplication. For more information check [Memory Deduplication - Kernel Same Page Merging - KSM](/src/database/README.md#ksm) |

101 +| cleanup obsolete charts after | `1h` | See [monitoring ephemeral containers](/src/collectors/cgroups.plugin/README.md#monitoring-ephemeral-containers), also sets the timeout for cleaning up obsolete dimensions |

102 +| gap when lost iterations above | `1` | |

103 +| cleanup orphan hosts after | `1h` | How long to wait until automatically removing from the DB a remote Netdata host (child) that is no longer sending data. |

104 +| enable zero metrics | `no` | Set to `yes` to show charts when all their metrics are zero. |

105

106 > ### Info

107 >

@@ -140,7 +138,7 @@ There are additional configuration options for the logs. For more info, see [Net

138 | health | `journal` | The filename to save the log of Netdata health collectors. You can also set it to `syslog` to send the access log to syslog, or `off` to disable this log. Defaults to `Journal` if using systemd. |

139 | daemon | `journal` | The filename to save the log of Netdata daemon. You can also set it to `syslog` to send the access log to syslog, or `off` to disable this log. Defaults to `Journal` if using systemd. |

140 | facility | `daemon` | A facility keyword is used to specify the type of system that is logging the message. |

143 -| logs flood protection period | `60` | Length of period (in sec) during which the number of errors should not exceed the `errors to trigger flood protection`. |

141 +| logs flood protection period | `1m` | Length of period during which the number of errors should not exceed the `errors to trigger flood protection`. |

142 | logs to trigger flood protection | `1000` | Number of errors written to the log in `errors flood protection period` sec before flood protection is activated. |

143 | level | `info` | Controls which log messages are logged, with error being the most important. Supported values: `info` and `error`. |

144

@@ -172,15 +170,15 @@ monitoring](/src/health/README.md).

170

171 [Alert notifications](/src/health/notifications/README.md) are configured in `health_alarm_notify.conf`.

172

175 -| setting | default | info |

176 -|:----------------------------------------------:|:------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

177 -| enabled | `yes` | Set to `no` to disable all alerts and notifications |

178 -| in memory max health log entries | 1000 | Size of the alert history held in RAM |

179 -| script to execute on alarm | `/usr/libexec/netdata/plugins.d/alarm-notify.sh` | The script that sends alert notifications. Note that in versions before 1.16, the plugins.d directory may be installed in a different location in certain OSs (e.g. under `/usr/lib/netdata`). |

180 -| run at least every seconds | `10` | Controls how often all alert conditions should be evaluated. |

181 -| postpone alarms during hibernation for seconds | `60` | Prevents false alerts. May need to be increased if you get alerts during hibernation. |

182 -| health log history | `432000` | Specifies the history of alert events (in seconds) kept in the agent's sqlite database. |

183 -| enabled alarms | * | Defines which alerts to load from both user and stock directories. This is a [simple pattern](/src/libnetdata/simple_pattern/README.md) list of alert or template names. Can be used to disable specific alerts. For example, `enabled alarms = !oom_kill *` will load all alerts except `oom_kill`. |

173 +| setting | default | info |

174 +|:--------------------------------------:|:------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

175 +| enabled | `yes` | Set to `no` to disable all alerts and notifications |

176 +| in memory max health log entries | 1000 | Size of the alert history held in RAM |

177 +| script to execute on alarm | `/usr/libexec/netdata/plugins.d/alarm-notify.sh` | The script that sends alert notifications. Note that in versions before 1.16, the plugins.d directory may be installed in a different location in certain OSs (e.g. under `/usr/lib/netdata`). |

178 +| run at least every | `10s` | Controls how often all alert conditions should be evaluated. |

179 +| postpone alarms during hibernation for | `1m` | Prevents false alerts. May need to be increased if you get alerts during hibernation. |

180 +| health log retention | `5d` | Specifies the history of alert events (in seconds) kept in the agent's sqlite database. |

181 +| enabled alarms | * | Defines which alerts to load from both user and stock directories. This is a [simple pattern](/src/libnetdata/simple_pattern/README.md) list of alert or template names. Can be used to disable specific alerts. For example, `enabled alarms = !oom_kill *` will load all alerts except `oom_kill`. |

182

183 ### [web] section options

184

src/daemon/daemon.c

+2 -2

@@ -58,7 +58,7 @@ static void change_dir_ownership(const char *dir, uid_t uid, gid_t gid, bool rec

58 fix_directory_file_permissions(dir, uid, gid, recursive);

59 }

61 -static void clean_directory(char *dirname)

61 +static void clean_directory(const char *dirname)

62 {

63 DIR *dir = opendir(dirname);

64 if(!dir) return;

@@ -188,7 +188,7 @@ static void oom_score_adj(void) {

188 }

189

190 // check the environment

191 - char *s = getenv("OOMScoreAdjust");

191 + const char *s = getenv("OOMScoreAdjust");

192 if(!s || !*s) {

193 snprintfz(buf, sizeof(buf) - 1, "%d", (int)wanted_score);

194 s = buf;

src/daemon/environment.c

+2 -2

@@ -69,7 +69,7 @@ void set_environment_for_plugins_and_scripts(void) {

69 buffer_free(user_plugins_dirs);

70 }

72 - char *default_port = appconfig_get(&netdata_config, CONFIG_SECTION_WEB, "default port", NULL);

72 + const char *default_port = appconfig_get(&netdata_config, CONFIG_SECTION_WEB, "default port", NULL);

73 int clean = 0;

74 if (!default_port) {

75 default_port = strdupz("19999");

@@ -78,7 +78,7 @@ void set_environment_for_plugins_and_scripts(void) {

79 nd_setenv("NETDATA_LISTEN_PORT", default_port, 1);

80 if (clean)

81 - freez(default_port);

81 + freez((char *)default_port);

83 // set the path we need

84 char path[4096], *p = getenv("PATH");

src/daemon/global_statistics.c

+8 -4

@@ -4221,9 +4221,11 @@ void *global_statistics_main(void *ptr)

4221 global_statistics_register_workers();

4222

4223 int update_every =

4224 - (int)config_get_number(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", localhost->rrd_update_every);

4225 - if (update_every < localhost->rrd_update_every)

4224 + (int)config_get_duration_seconds(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", localhost->rrd_update_every);

4225 + if (update_every < localhost->rrd_update_every) {

4226 update_every = localhost->rrd_update_every;

4227 + config_set_duration_seconds(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", update_every);

4228 + }

4229

4230 usec_t step = update_every * USEC_PER_SEC;

4231 heartbeat_t hb;

@@ -4277,9 +4279,11 @@ void *global_statistics_extended_main(void *ptr)

4279 global_statistics_register_workers();

4280

4281 int update_every =

4280 - (int)config_get_number(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", localhost->rrd_update_every);

4281 - if (update_every < localhost->rrd_update_every)

4282 + (int)config_get_duration_seconds(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", localhost->rrd_update_every);

4283 + if (update_every < localhost->rrd_update_every) {

4284 update_every = localhost->rrd_update_every;

4285 + config_set_duration_seconds(CONFIG_SECTION_GLOBAL_STATISTICS, "update every", update_every);

4286 + }

4287

4288 usec_t step = update_every * USEC_PER_SEC;

4289 heartbeat_t hb;

src/daemon/main.c

+133 -79

@@ -28,18 +28,7 @@ bool ieee754_doubles = false;

28 time_t netdata_start_time = 0;

29 struct netdata_static_thread *static_threads;

31 -struct config netdata_config = {

32 - .first_section = NULL,

33 - .last_section = NULL,

34 - .mutex = NETDATA_MUTEX_INITIALIZER,

35 - .index = {

36 - .avl_tree = {

37 - .root = NULL,

38 - .compar = appconfig_section_compare

39 - },

40 - .rwlock = AVL_LOCK_INITIALIZER

41 - }

42 -};

31 +struct config netdata_config = APPCONFIG_INITIALIZER;

33 typedef struct service_thread {

34 pid_t tid;

@@ -527,12 +516,12 @@ void web_server_threading_selection(void) {

516

517 int make_dns_decision(const char *section_name, const char *config_name, const char *default_value, SIMPLE_PATTERN *p)

518 {

530 - char *value = config_get(section_name,config_name,default_value);

519 + const char *value = config_get(section_name,config_name,default_value);

520 if(!strcmp("yes",value))

521 return 1;

522 if(!strcmp("no",value))

523 return 0;

535 - if(strcmp("heuristic",value))

524 + if(strcmp("heuristic",value) != 0)

525 netdata_log_error("Invalid configuration option '%s' for '%s'/'%s'. Valid options are 'yes', 'no' and 'heuristic'. Proceeding with 'heuristic'",

526 value, section_name, config_name);

527

@@ -542,11 +531,13 @@ int make_dns_decision(const char *section_name, const char *config_name, const c

531 void web_server_config_options(void)

532 {

533 web_client_timeout =

545 - (int)config_get_number(CONFIG_SECTION_WEB, "disconnect idle clients after seconds", web_client_timeout);

534 + (int)config_get_duration_seconds(CONFIG_SECTION_WEB, "disconnect idle clients after", web_client_timeout);

535 +

536 web_client_first_request_timeout =

547 - (int)config_get_number(CONFIG_SECTION_WEB, "timeout for first request", web_client_first_request_timeout);

537 + (int)config_get_duration_seconds(CONFIG_SECTION_WEB, "timeout for first request", web_client_first_request_timeout);

538 +

539 web_client_streaming_rate_t =

549 - config_get_number(CONFIG_SECTION_WEB, "accept a streaming request every seconds", web_client_streaming_rate_t);

540 + config_get_duration_seconds(CONFIG_SECTION_WEB, "accept a streaming request every", web_client_streaming_rate_t);

541

542 respect_web_browser_do_not_track_policy =

543 config_get_boolean(CONFIG_SECTION_WEB, "respect do not track policy", respect_web_browser_do_not_track_policy);

@@ -595,7 +586,7 @@ void web_server_config_options(void)

586

587 web_enable_gzip = config_get_boolean(CONFIG_SECTION_WEB, "enable gzip compression", web_enable_gzip);

588

598 - char *s = config_get(CONFIG_SECTION_WEB, "gzip compression strategy", "default");

589 + const char *s = config_get(CONFIG_SECTION_WEB, "gzip compression strategy", "default");

590 if(!strcmp(s, "default"))

591 web_gzip_strategy = Z_DEFAULT_STRATEGY;

592 else if(!strcmp(s, "filtered"))

@@ -842,7 +833,7 @@ static void log_init(void) {

833

834 time_t period = ND_LOG_DEFAULT_THROTTLE_PERIOD;

835 size_t logs = ND_LOG_DEFAULT_THROTTLE_LOGS;

845 - period = config_get_number(CONFIG_SECTION_LOGS, "logs flood protection period", period);

836 + period = config_get_duration_seconds(CONFIG_SECTION_LOGS, "logs flood protection period", period);

837 logs = (unsigned long)config_get_number(CONFIG_SECTION_LOGS, "logs to trigger flood protection", (long long int)logs);

838 nd_log_set_flood_protection(logs, period);

839

@@ -886,7 +877,7 @@ static void log_init(void) {

877 aclk_config_get_query_scope();

878 }

879

889 -static char *get_varlib_subdir_from_config(const char *prefix, const char *dir) {

880 +static const char *get_varlib_subdir_from_config(const char *prefix, const char *dir) {

881 char filename[FILENAME_MAX + 1];

882 snprintfz(filename, FILENAME_MAX, "%s/%s", prefix, dir);

883 return config_get(CONFIG_SECTION_DIRECTORIES, dir, filename);

@@ -894,6 +885,7 @@ static char *get_varlib_subdir_from_config(const char *prefix, const char *dir)

885

886 static void backwards_compatible_config() {

887 // move [global] options to the [web] section

888 +

889 config_move(CONFIG_SECTION_GLOBAL, "http port listen backlog",

890 CONFIG_SECTION_WEB, "listen backlog");

891

@@ -997,7 +989,10 @@ static void backwards_compatible_config() {

989 CONFIG_SECTION_PLUGINS, "statsd");

990

991 config_move(CONFIG_SECTION_GLOBAL, "memory mode",

1000 - CONFIG_SECTION_DB, "mode");

992 + CONFIG_SECTION_DB, "db");

993 +

994 + config_move(CONFIG_SECTION_DB, "mode",

995 + CONFIG_SECTION_DB, "db");

996

997 config_move(CONFIG_SECTION_GLOBAL, "history",

998 CONFIG_SECTION_DB, "retention");

@@ -1006,7 +1001,13 @@ static void backwards_compatible_config() {

1001 CONFIG_SECTION_DB, "update every");

1002

1003 config_move(CONFIG_SECTION_GLOBAL, "page cache size",

1009 - CONFIG_SECTION_DB, "dbengine page cache size MB");

1004 + CONFIG_SECTION_DB, "dbengine page cache size");

1005 +

1006 + config_move(CONFIG_SECTION_DB, "dbengine page cache size MB",

1007 + CONFIG_SECTION_DB, "dbengine page cache size");

1008 +

1009 + config_move(CONFIG_SECTION_DB, "dbengine extent cache size MB",

1010 + CONFIG_SECTION_DB, "dbengine extent cache size");

1011

1012 config_move(CONFIG_SECTION_DB, "page cache size",

1013 CONFIG_SECTION_DB, "dbengine page cache size MB");

@@ -1017,30 +1018,6 @@ static void backwards_compatible_config() {

1018 config_move(CONFIG_SECTION_DB, "page cache with malloc",

1019 CONFIG_SECTION_DB, "dbengine page cache with malloc");

1020

1020 - config_move(CONFIG_SECTION_GLOBAL, "dbengine disk space",

1021 - CONFIG_SECTION_DB, "dbengine disk space MB");

1022 -

1023 - config_move(CONFIG_SECTION_GLOBAL, "dbengine multihost disk space",

1024 - CONFIG_SECTION_DB, "dbengine multihost disk space MB");

1025 -

1026 - config_move(CONFIG_SECTION_DB, "dbengine disk space MB",

1027 - CONFIG_SECTION_DB, "dbengine multihost disk space MB");

1028 -

1029 - config_move(CONFIG_SECTION_DB, "dbengine multihost disk space MB",

1030 - CONFIG_SECTION_DB, "dbengine tier 0 disk space MB");

1031 -

1032 - config_move(CONFIG_SECTION_DB, "dbengine tier 1 multihost disk space MB",

1033 - CONFIG_SECTION_DB, "dbengine tier 1 disk space MB");

1034 -

1035 - config_move(CONFIG_SECTION_DB, "dbengine tier 2 multihost disk space MB",

1036 - CONFIG_SECTION_DB, "dbengine tier 2 disk space MB");

1037 -

1038 - config_move(CONFIG_SECTION_DB, "dbengine tier 3 multihost disk space MB",

1039 - CONFIG_SECTION_DB, "dbengine tier 3 disk space MB");

1040 -

1041 - config_move(CONFIG_SECTION_DB, "dbengine tier 4 multihost disk space MB",

1042 - CONFIG_SECTION_DB, "dbengine tier 4 disk space MB");

1043 -

1021 config_move(CONFIG_SECTION_GLOBAL, "memory deduplication (ksm)",

1022 CONFIG_SECTION_DB, "memory deduplication (ksm)");

1023

@@ -1054,17 +1031,67 @@ static void backwards_compatible_config() {

1031 CONFIG_SECTION_DB, "dbengine pages per extent");

1032

1033 config_move(CONFIG_SECTION_GLOBAL, "cleanup obsolete charts after seconds",

1057 - CONFIG_SECTION_DB, "cleanup obsolete charts after secs");

1034 + CONFIG_SECTION_DB, "cleanup obsolete charts after");

1035 +

1036 + config_move(CONFIG_SECTION_DB, "cleanup obsolete charts after secs",

1037 + CONFIG_SECTION_DB, "cleanup obsolete charts after");

1038

1039 config_move(CONFIG_SECTION_GLOBAL, "gap when lost iterations above",

1040 CONFIG_SECTION_DB, "gap when lost iterations above");

1041

1042 config_move(CONFIG_SECTION_GLOBAL, "cleanup orphan hosts after seconds",

1063 - CONFIG_SECTION_DB, "cleanup orphan hosts after secs");

1043 + CONFIG_SECTION_DB, "cleanup orphan hosts after");

1044 +

1045 + config_move(CONFIG_SECTION_DB, "cleanup orphan hosts after secs",

1046 + CONFIG_SECTION_DB, "cleanup orphan hosts after");

1047 +

1048 + config_move(CONFIG_SECTION_DB, "cleanup ephemeral hosts after secs",

1049 + CONFIG_SECTION_DB, "cleanup ephemeral hosts after");

1050 +

1051 + config_move(CONFIG_SECTION_DB, "seconds to replicate",

1052 + CONFIG_SECTION_DB, "replication period");

1053 +

1054 + config_move(CONFIG_SECTION_DB, "seconds per replication step",

1055 + CONFIG_SECTION_DB, "replication step");

1056

1057 config_move(CONFIG_SECTION_GLOBAL, "enable zero metrics",

1058 CONFIG_SECTION_DB, "enable zero metrics");

1059

1060 + // ----------------------------------------------------------------------------------------------------------------

1061 +

1062 + config_move(CONFIG_SECTION_GLOBAL, "dbengine disk space",

1063 + CONFIG_SECTION_DB, "dbengine tier 0 retention size");

1064 +

1065 + config_move(CONFIG_SECTION_GLOBAL, "dbengine multihost disk space",

1066 + CONFIG_SECTION_DB, "dbengine tier 0 retention size");

1067 +

1068 + config_move(CONFIG_SECTION_DB, "dbengine disk space MB",

1069 + CONFIG_SECTION_DB, "dbengine tier 0 retention size");

1070 +

1071 + for(size_t tier = 0; tier < RRD_STORAGE_TIERS ;tier++) {

1072 + char old_config[128], new_config[128];

1073 +

1074 + snprintfz(old_config, sizeof(old_config), "dbengine tier %zu retention days", tier);

1075 + snprintfz(new_config, sizeof(new_config), "dbengine tier %zu retention time", tier);

1076 + config_move(CONFIG_SECTION_DB, old_config,

1077 + CONFIG_SECTION_DB, new_config);

1078 +

1079 + if(tier == 0)

1080 + snprintfz(old_config, sizeof(old_config), "dbengine multihost disk space MB");

1081 + else

1082 + snprintfz(old_config, sizeof(old_config), "dbengine tier %zu multihost disk space MB", tier);

1083 + snprintfz(new_config, sizeof(new_config), "dbengine tier %zu retention size", tier);

1084 + config_move(CONFIG_SECTION_DB, old_config,

1085 + CONFIG_SECTION_DB, new_config);

1086 +

1087 + snprintfz(old_config, sizeof(old_config), "dbengine tier %zu disk space MB", tier);

1088 + snprintfz(new_config, sizeof(new_config), "dbengine tier %zu retention size", tier);

1089 + config_move(CONFIG_SECTION_DB, old_config,

1090 + CONFIG_SECTION_DB, new_config);

1091 + }

1092 +

1093 + // ----------------------------------------------------------------------------------------------------------------

1094 +

1095 config_move(CONFIG_SECTION_LOGS, "error",

1096 CONFIG_SECTION_LOGS, "daemon");

1097

@@ -1076,11 +1103,42 @@ static void backwards_compatible_config() {

1103

1104 config_move(CONFIG_SECTION_LOGS, "errors flood protection period",

1105 CONFIG_SECTION_LOGS, "logs flood protection period");

1106 +

1107 config_move(CONFIG_SECTION_HEALTH, "is ephemeral",

1108 CONFIG_SECTION_GLOBAL, "is ephemeral node");

1109

1110 config_move(CONFIG_SECTION_HEALTH, "has unstable connection",

1111 CONFIG_SECTION_GLOBAL, "has unstable connection");

1112 +

1113 + config_move(CONFIG_SECTION_HEALTH, "run at least every seconds",

1114 + CONFIG_SECTION_HEALTH, "run at least every");

1115 +

1116 + config_move(CONFIG_SECTION_HEALTH, "postpone alarms during hibernation for seconds",

1117 + CONFIG_SECTION_HEALTH, "postpone alarms during hibernation for");

1118 +

1119 + config_move(CONFIG_SECTION_HEALTH, "health log history",

1120 + CONFIG_SECTION_HEALTH, "health log retention");

1121 +

1122 + config_move(CONFIG_SECTION_REGISTRY, "registry expire idle persons days",

1123 + CONFIG_SECTION_REGISTRY, "registry expire idle persons");

1124 +

1125 + config_move(CONFIG_SECTION_WEB, "disconnect idle clients after seconds",

1126 + CONFIG_SECTION_WEB, "disconnect idle clients after");

1127 +

1128 + config_move(CONFIG_SECTION_WEB, "accept a streaming request every seconds",

1129 + CONFIG_SECTION_WEB, "accept a streaming request every");

1130 +

1131 + config_move(CONFIG_SECTION_STATSD, "set charts as obsolete after secs",

1132 + CONFIG_SECTION_STATSD, "set charts as obsolete after");

1133 +

1134 + config_move(CONFIG_SECTION_STATSD, "disconnect idle tcp clients after seconds",

1135 + CONFIG_SECTION_STATSD, "disconnect idle tcp clients after");

1136 +

1137 + config_move("plugin:idlejitter", "loop time in ms",

1138 + "plugin:idlejitter", "loop time");

1139 +

1140 + config_move("plugin:proc:/sys/class/infiniband", "refresh ports state every seconds",

1141 + "plugin:proc:/sys/class/infiniband", "refresh ports state every");

1142 }

1143

1144 static int get_hostname(char *buf, size_t buf_size) {

@@ -1125,22 +1183,22 @@ static void get_netdata_configured_variables()

1183 // ------------------------------------------------------------------------

1184 // get default database update frequency

1185

1128 - default_rrd_update_every = (int) config_get_number(CONFIG_SECTION_DB, "update every", UPDATE_EVERY);

1186 + default_rrd_update_every = (int) config_get_duration_seconds(CONFIG_SECTION_DB, "update every", UPDATE_EVERY);

1187 if(default_rrd_update_every < 1 || default_rrd_update_every > 600) {

1188 netdata_log_error("Invalid data collection frequency (update every) %d given. Defaulting to %d.", default_rrd_update_every, UPDATE_EVERY);

1189 default_rrd_update_every = UPDATE_EVERY;

1132 - config_set_number(CONFIG_SECTION_DB, "update every", default_rrd_update_every);

1190 + config_set_duration_seconds(CONFIG_SECTION_DB, "update every", default_rrd_update_every);

1191 }

1192

1193 // ------------------------------------------------------------------------

1136 - // get default memory mode for the database

1194 + // get the database selection

1195

1196 {

1139 - const char *mode = config_get(CONFIG_SECTION_DB, "mode", rrd_memory_mode_name(default_rrd_memory_mode));

1197 + const char *mode = config_get(CONFIG_SECTION_DB, "db", rrd_memory_mode_name(default_rrd_memory_mode));

1198 default_rrd_memory_mode = rrd_memory_mode_id(mode);

1199 if(strcmp(mode, rrd_memory_mode_name(default_rrd_memory_mode)) != 0) {

1200 netdata_log_error("Invalid memory mode '%s' given. Using '%s'", mode, rrd_memory_mode_name(default_rrd_memory_mode));

1143 - config_set(CONFIG_SECTION_DB, "mode", rrd_memory_mode_name(default_rrd_memory_mode));

1201 + config_set(CONFIG_SECTION_DB, "db", rrd_memory_mode_name(default_rrd_memory_mode));

1202 }

1203 }

1204

@@ -1194,17 +1252,19 @@ static void get_netdata_configured_variables()

1252 // ------------------------------------------------------------------------

1253 // get default Database Engine page cache size in MiB

1254

1197 - default_rrdeng_page_cache_mb = (int) config_get_number(CONFIG_SECTION_DB, "dbengine page cache size MB", default_rrdeng_page_cache_mb);

1198 - default_rrdeng_extent_cache_mb = (int) config_get_number(CONFIG_SECTION_DB, "dbengine extent cache size MB", default_rrdeng_extent_cache_mb);

1255 + default_rrdeng_page_cache_mb = (int) config_get_size_mb(CONFIG_SECTION_DB, "dbengine page cache size", default_rrdeng_page_cache_mb);

1256 + default_rrdeng_extent_cache_mb = (int) config_get_size_mb(CONFIG_SECTION_DB, "dbengine extent cache size", default_rrdeng_extent_cache_mb);

1257 db_engine_journal_check = config_get_boolean(CONFIG_SECTION_DB, "dbengine enable journal integrity check", CONFIG_BOOLEAN_NO);

1258

1201 - if(default_rrdeng_extent_cache_mb < 0)

1259 + if(default_rrdeng_extent_cache_mb < 0) {

1260 default_rrdeng_extent_cache_mb = 0;

1261 + config_set_size_mb(CONFIG_SECTION_DB, "dbengine extent cache size", default_rrdeng_extent_cache_mb);

1262 + }

1263

1264 if(default_rrdeng_page_cache_mb < RRDENG_MIN_PAGE_CACHE_SIZE_MB) {

1265 netdata_log_error("Invalid page cache size %d given. Defaulting to %d.", default_rrdeng_page_cache_mb, RRDENG_MIN_PAGE_CACHE_SIZE_MB);

1266 default_rrdeng_page_cache_mb = RRDENG_MIN_PAGE_CACHE_SIZE_MB;

1207 - config_set_number(CONFIG_SECTION_DB, "dbengine page cache size MB", default_rrdeng_page_cache_mb);

1267 + config_set_size_mb(CONFIG_SECTION_DB, "dbengine page cache size", default_rrdeng_page_cache_mb);

1268 }

1269

1270 // ------------------------------------------------------------------------

@@ -1237,28 +1297,24 @@ static void get_netdata_configured_variables()

1297 // get KSM settings

1298

1299 #ifdef MADV_MERGEABLE

1240 - enable_ksm = config_get_boolean(CONFIG_SECTION_DB, "memory deduplication (ksm)", enable_ksm);

1300 + enable_ksm = config_get_boolean_ondemand(CONFIG_SECTION_DB, "memory deduplication (ksm)", enable_ksm);

1301 #endif

1302

1303 // --------------------------------------------------------------------

1244 - // metric correlations

1304

1246 - enable_metric_correlations = config_get_boolean(CONFIG_SECTION_GLOBAL, "enable metric correlations", enable_metric_correlations);

1247 - default_metric_correlations_method = weights_string_to_method(config_get(

1248 - CONFIG_SECTION_GLOBAL, "metric correlations method",

1249 - weights_method_to_string(default_metric_correlations_method)));

1305 + rrdhost_free_ephemeral_time_s =

1306 + config_get_duration_seconds(CONFIG_SECTION_DB, "cleanup ephemeral hosts after", rrdhost_free_ephemeral_time_s);

1307

1251 - // --------------------------------------------------------------------

1308 + rrdset_free_obsolete_time_s =

1309 + config_get_duration_seconds(CONFIG_SECTION_DB, "cleanup obsolete charts after", rrdset_free_obsolete_time_s);

1310

1253 - rrdset_free_obsolete_time_s = config_get_number(CONFIG_SECTION_DB, "cleanup obsolete charts after secs", rrdset_free_obsolete_time_s);

1254 - rrdhost_free_ephemeral_time_s = config_get_number(CONFIG_SECTION_DB, "cleanup ephemeral hosts after secs", rrdhost_free_ephemeral_time_s);

1311 // Current chart locking and invalidation scheme doesn't prevent Netdata from segmentation faults if a short

1312 // cleanup delay is set. Extensive stress tests showed that 10 seconds is quite a safe delay. Look at

1313 // https://github.com/netdata/netdata/pull/11222#issuecomment-868367920 for more information.

1314 if (rrdset_free_obsolete_time_s < 10) {

1315 rrdset_free_obsolete_time_s = 10;

1260 - netdata_log_info("The \"cleanup obsolete charts after seconds\" option was set to 10 seconds.");

1261 - config_set_number(CONFIG_SECTION_DB, "cleanup obsolete charts after secs", rrdset_free_obsolete_time_s);

1316 + netdata_log_info("The \"cleanup obsolete charts after\" option was set to 10 seconds.");

1317 + config_set_duration_seconds(CONFIG_SECTION_DB, "cleanup obsolete charts after", rrdset_free_obsolete_time_s);

1318 }

1319

1320 gap_when_lost_iterations_above = (int)config_get_number(CONFIG_SECTION_DB, "gap when lost iterations above", gap_when_lost_iterations_above);

@@ -1278,7 +1334,7 @@ static void get_netdata_configured_variables()

1334

1335 }

1336

1281 -static void post_conf_load(char **user)

1337 +static void post_conf_load(const char **user)

1338 {

1339 // --------------------------------------------------------------------

1340 // get the user we should run

@@ -1293,7 +1349,7 @@ static void post_conf_load(char **user)

1349 }

1350 }

1351

1296 -static bool load_netdata_conf(char *filename, char overwrite_used, char **user) {

1352 +static bool load_netdata_conf(char *filename, char overwrite_used, const char **user) {

1353 errno_clear();

1354

1355 int ret = 0;

@@ -1406,7 +1462,7 @@ bool netdata_random_session_id_generate(void);

1462 int windows_perflib_dump(const char *key);

1463 #endif

1464

1409 -int unittest_prepare_rrd(char **user) {

1465 +int unittest_prepare_rrd(const char **user) {

1466 post_conf_load(user);

1467 get_netdata_configured_variables();

1468 default_rrd_update_every = 1;

@@ -1437,7 +1493,7 @@ int netdata_main(int argc, char **argv) {

1493 int config_loaded = 0;

1494 bool close_open_fds = true;

1495 size_t default_stacksize;

1440 - char *user = NULL;

1496 + const char *user = NULL;

1497

1498 #ifdef OS_WINDOWS

1499 int dont_fork = 1;

@@ -1787,7 +1843,7 @@ int netdata_main(int argc, char **argv) {

1843 // so the caller can use -c netdata.conf before or

1844 // after this parameter to prevent or allow overwriting

1845 // variables at netdata.conf

1790 - config_set_default(section, key, value);

1846 + config_set_default_raw_value(section, key, value);

1847

1848 // fprintf(stderr, "SET section '%s', key '%s', value '%s'\n", section, key, value);

1849 }

@@ -1820,7 +1876,7 @@ int netdata_main(int argc, char **argv) {

1876 // so the caller can use -c netdata.conf before or

1877 // after this parameter to prevent or allow overwriting

1878 // variables at netdata.conf

1823 - appconfig_set_default(tmp_config, section, key, value);

1879 + appconfig_set_default_raw_value(tmp_config, section, key, value);

1880

1881 // fprintf(stderr, "SET section '%s', key '%s', value '%s'\n", section, key, value);

1882 }

@@ -1922,7 +1978,7 @@ int netdata_main(int argc, char **argv) {

1978 // ------------------------------------------------------------------------

1979 // initialize netdata

1980 {

1925 - char *pmax = config_get(CONFIG_SECTION_GLOBAL, "glibc malloc arena max for plugins", "1");

1981 + const char *pmax = config_get(CONFIG_SECTION_GLOBAL, "glibc malloc arena max for plugins", "1");

1982 if(pmax && *pmax)

1983 setenv("MALLOC_ARENA_MAX", pmax, 1);

1984

@@ -1979,7 +2035,7 @@ int netdata_main(int argc, char **argv) {

2035 // --------------------------------------------------------------------

2036 // get the debugging flags from the configuration file

2037

1982 - char *flags = config_get(CONFIG_SECTION_LOGS, "debug flags", "0x0000000000000000");

2038 + const char *flags = config_get(CONFIG_SECTION_LOGS, "debug flags", "0x0000000000000000");

2039 nd_setenv("NETDATA_DEBUG_FLAGS", flags, 1);

2040

2041 debug_flags = strtoull(flags, NULL, 0);

@@ -2008,8 +2064,6 @@ int netdata_main(int argc, char **argv) {

2064

2065 check_local_streaming_capabilities();

2066

2011 - aral_judy_init();

2012 -

2067 get_system_timezone();

2068

2069 replication_initialize();

@@ -2159,7 +2213,7 @@ int netdata_main(int argc, char **argv) {

2213

2214 delta_startup_time("initialize threads after fork");

2215

2162 - netdata_threads_init_after_fork((size_t)config_get_number(CONFIG_SECTION_GLOBAL, "pthread stack size", (long)default_stacksize));

2216 + netdata_threads_init_after_fork((size_t)config_get_size_bytes(CONFIG_SECTION_GLOBAL, "pthread stack size", default_stacksize));

2217

2218 // initialize internal registry

2219 delta_startup_time("initialize registry");

src/database/contexts/api_v2_contexts_agents.c

+11 -29

@@ -5,30 +5,6 @@

6 void build_info_to_json_object(BUFFER *b);

8 -static void convert_seconds_to_dhms(time_t seconds, char *result, int result_size) {

9 - int days, hours, minutes;

10 -

11 - days = (int) (seconds / (24 * 3600));

12 - seconds = (int) (seconds % (24 * 3600));

13 - hours = (int) (seconds / 3600);

14 - seconds %= 3600;

15 - minutes = (int) (seconds / 60);

16 - seconds %= 60;

17 -

18 - // Format the result into the provided string buffer

19 - BUFFER *buf = buffer_create(128, NULL);

20 - if (days)

21 - buffer_sprintf(buf,"%d day%s%s", days, days==1 ? "" : "s", hours || minutes ? ", " : "");

22 - if (hours)

23 - buffer_sprintf(buf,"%d hour%s%s", hours, hours==1 ? "" : "s", minutes ? ", " : "");

24 - if (minutes)

25 - buffer_sprintf(buf,"%d minute%s%s", minutes, minutes==1 ? "" : "s", seconds ? ", " : "");

26 - if (seconds)

27 - buffer_sprintf(buf,"%d second%s", (int) seconds, seconds==1 ? "" : "s");

28 - strncpyz(result, buffer_tostring(buf), result_size);

29 - buffer_free(buf);

30 -}

31 -

8 void buffer_json_agents_v2(BUFFER *wb, struct query_timings *timings, time_t now_s, bool info, bool array) {

9 if(!now_s)

10 now_s = now_realtime_sec();

@@ -117,8 +93,8 @@ void buffer_json_agents_v2(BUFFER *wb, struct query_timings *timings, time_t now

93 buffer_json_add_array_item_object(wb);

94 buffer_json_member_add_uint64(wb, "tier", tier);

95 char human_retention[128];

120 - convert_seconds_to_dhms((time_t) group_seconds, human_retention, sizeof(human_retention) - 1);

121 - buffer_json_member_add_string(wb, "point_every", human_retention);

96 + duration_snprintf_time_t(human_retention, sizeof(human_retention), (stime_t)group_seconds);

97 + buffer_json_member_add_string(wb, "granularity", human_retention);

99 buffer_json_member_add_uint64(wb, "metrics", storage_engine_metrics(eng->seb, localhost->db[tier].si));

100 buffer_json_member_add_uint64(wb, "samples", storage_engine_samples(eng->seb, localhost->db[tier].si));

@@ -136,7 +112,9 @@ void buffer_json_agents_v2(BUFFER *wb, struct query_timings *timings, time_t now

112 buffer_json_member_add_time_t(wb, "to", now_s);

113 buffer_json_member_add_time_t(wb, "retention", retention);

114

139 - convert_seconds_to_dhms(retention, human_retention, sizeof(human_retention) - 1);

115 + duration_snprintf_hours(human_retention, sizeof(human_retention),

116 + (int)duration_round_to_resolution(retention, 3600));

117 +

118 buffer_json_member_add_string(wb, "retention_human", human_retention);

119

120 if(used || max) { // we have disk space information

@@ -148,12 +126,16 @@ void buffer_json_agents_v2(BUFFER *wb, struct query_timings *timings, time_t now

126 time_t actual_retention = MIN(space_retention, time_retention ? time_retention : space_retention);

127

128 if (time_retention) {

151 - convert_seconds_to_dhms(time_retention, human_retention, sizeof(human_retention) - 1);

129 + duration_snprintf_hours(human_retention, sizeof(human_retention),

130 + (int)duration_round_to_resolution(time_retention, 3600));

131 +

132 buffer_json_member_add_time_t(wb, "requested_retention", time_retention);

133 buffer_json_member_add_string(wb, "requested_retention_human", human_retention);

134 }

135

156 - convert_seconds_to_dhms(actual_retention, human_retention, sizeof(human_retention) - 1);

136 + duration_snprintf_hours(human_retention, sizeof(human_retention),

137 + (int)duration_round_to_resolution(actual_retention, 3600));

138 +

139 buffer_json_member_add_time_t(wb, "expected_retention", actual_retention);

140 buffer_json_member_add_string(wb, "expected_retention_human", human_retention);

141 }

src/database/rrd.h

+5 -5

@@ -1096,7 +1096,7 @@ typedef struct alarm_log {

1096 uint32_t next_alarm_id;

1097 unsigned int count;

1098 unsigned int max;

1099 - uint32_t health_log_history; // the health log history in seconds to be kept in db

1099 + uint32_t health_log_retention_s; // the health log retention in seconds to be kept in db

1100 ALARM_ENTRY *alarms;

1101 RW_SPINLOCK spinlock;

1102 } ALARM_LOG;

@@ -1384,7 +1384,7 @@ void rrddim_index_destroy(RRDSET *st);

1384 extern time_t rrdhost_free_orphan_time_s;

1385 extern time_t rrdhost_free_ephemeral_time_s;

1386

1387 -int rrd_init(char *hostname, struct rrdhost_system_info *system_info, bool unittest);

1387 +int rrd_init(const char *hostname, struct rrdhost_system_info *system_info, bool unittest);

1388

1389 RRDHOST *rrdhost_find_by_hostname(const char *hostname);

1390 RRDHOST *rrdhost_find_by_guid(const char *guid);

@@ -1405,9 +1405,9 @@ RRDHOST *rrdhost_find_or_create(

1405 RRD_MEMORY_MODE mode,

1406 unsigned int health_enabled,

1407 unsigned int rrdpush_enabled,

1408 - char *rrdpush_destination,

1409 - char *rrdpush_api_key,

1410 - char *rrdpush_send_charts_matching,

1408 + const char *rrdpush_destination,

1409 + const char *rrdpush_api_key,

1410 + const char *rrdpush_send_charts_matching,

1411 bool rrdpush_enable_replication,

1412 time_t rrdpush_seconds_to_replicate,

1413 time_t rrdpush_replication_step,

src/database/rrdhost.c

+32 -36

@@ -232,10 +232,10 @@ void set_host_properties(RRDHOST *host, int update_every, RRD_MEMORY_MODE memory

232 // RRDHOST - add a host

233

234 static void rrdhost_initialize_rrdpush_sender(RRDHOST *host,

235 - unsigned int rrdpush_enabled,

236 - char *rrdpush_destination,

237 - char *rrdpush_api_key,

238 - char *rrdpush_send_charts_matching

235 + unsigned int rrdpush_enabled,

236 + const char *rrdpush_destination,

237 + const char *rrdpush_api_key,

238 + const char *rrdpush_send_charts_matching

239 ) {

240 if(rrdhost_flag_check(host, RRDHOST_FLAG_RRDPUSH_SENDER_INITIALIZED)) return;

241

@@ -341,9 +341,9 @@ static RRDHOST *rrdhost_create(

341 RRD_MEMORY_MODE memory_mode,

342 unsigned int health_enabled,

343 unsigned int rrdpush_enabled,

344 - char *rrdpush_destination,

345 - char *rrdpush_api_key,

346 - char *rrdpush_send_charts_matching,

344 + const char *rrdpush_destination,

345 + const char *rrdpush_api_key,

346 + const char *rrdpush_send_charts_matching,

347 bool rrdpush_enable_replication,

348 time_t rrdpush_seconds_to_replicate,

349 time_t rrdpush_replication_step,

@@ -474,7 +474,7 @@ static RRDHOST *rrdhost_create(

474 if (is_localhost && host->system_info) {

475 host->system_info->ml_capable = ml_capable();

476 host->system_info->ml_enabled = ml_enabled(host);

477 - host->system_info->mc_version = enable_metric_correlations ? metric_correlations_version : 0;

477 + host->system_info->mc_version = metric_correlations_version;

478 }

479

480 // ------------------------------------------------------------------------

@@ -566,9 +566,9 @@ static void rrdhost_update(RRDHOST *host

566 , RRD_MEMORY_MODE mode

567 , unsigned int health_enabled

568 , unsigned int rrdpush_enabled

569 - , char *rrdpush_destination

570 - , char *rrdpush_api_key

571 - , char *rrdpush_send_charts_matching

569 + , const char *rrdpush_destination

570 + , const char *rrdpush_api_key

571 + , const char *rrdpush_send_charts_matching

572 , bool rrdpush_enable_replication

573 , time_t rrdpush_seconds_to_replicate

574 , time_t rrdpush_replication_step

@@ -706,9 +706,9 @@ RRDHOST *rrdhost_find_or_create(

706 , RRD_MEMORY_MODE mode

707 , unsigned int health_enabled

708 , unsigned int rrdpush_enabled

709 - , char *rrdpush_destination

710 - , char *rrdpush_api_key

711 - , char *rrdpush_send_charts_matching

709 + , const char *rrdpush_destination

710 + , const char *rrdpush_api_key

711 + , const char *rrdpush_send_charts_matching

712 , bool rrdpush_enable_replication

713 , time_t rrdpush_seconds_to_replicate

714 , time_t rrdpush_replication_step

@@ -862,7 +862,7 @@ RRD_BACKFILL get_dbengine_backfill(RRD_BACKFILL backfill)

862

863 #endif

864

865 -void dbengine_init(char *hostname) {

865 +static void dbengine_init(const char *hostname) {

866 #ifdef ENABLE_DBENGINE

867

868 use_direct_io = config_get_boolean(CONFIG_SECTION_DB, "dbengine use direct io", use_direct_io);

@@ -900,10 +900,10 @@ void dbengine_init(char *hostname) {

900 !config_exists(CONFIG_SECTION_DB, "dbengine tier 2 update every iterations") &&

901 !config_exists(CONFIG_SECTION_DB, "dbengine tier 3 update every iterations") &&

902 !config_exists(CONFIG_SECTION_DB, "dbengine tier 4 update every iterations") &&

903 - !config_exists(CONFIG_SECTION_DB, "dbengine tier 1 disk space MB") &&

904 - !config_exists(CONFIG_SECTION_DB, "dbengine tier 2 disk space MB") &&

905 - !config_exists(CONFIG_SECTION_DB, "dbengine tier 3 disk space MB") &&

906 - !config_exists(CONFIG_SECTION_DB, "dbengine tier 4 disk space MB"));

903 + !config_exists(CONFIG_SECTION_DB, "dbengine tier 1 retention size") &&

904 + !config_exists(CONFIG_SECTION_DB, "dbengine tier 2 retention size") &&

905 + !config_exists(CONFIG_SECTION_DB, "dbengine tier 3 retention size") &&

906 + !config_exists(CONFIG_SECTION_DB, "dbengine tier 4 retention size"));

907

908 default_backfill = get_dbengine_backfill(RRD_BACKFILL_NEW);

909 char dbengineconfig[200 + 1];

@@ -925,11 +925,11 @@ void dbengine_init(char *hostname) {

925 storage_tiers_grouping_iterations[tier] = grouping_iterations;

926 }

927

928 - default_multidb_disk_quota_mb = (int) config_get_number(CONFIG_SECTION_DB, "dbengine tier 0 disk space MB", RRDENG_DEFAULT_TIER_DISK_SPACE_MB);

928 + default_multidb_disk_quota_mb = (int) config_get_size_mb(CONFIG_SECTION_DB, "dbengine tier 0 retention size", RRDENG_DEFAULT_TIER_DISK_SPACE_MB);

929 if(default_multidb_disk_quota_mb && default_multidb_disk_quota_mb < RRDENG_MIN_DISK_SPACE_MB) {

930 netdata_log_error("Invalid disk space %d for tier 0 given. Defaulting to %d.", default_multidb_disk_quota_mb, RRDENG_MIN_DISK_SPACE_MB);

931 default_multidb_disk_quota_mb = RRDENG_MIN_DISK_SPACE_MB;

932 - config_set_number(CONFIG_SECTION_DB, "dbengine tier 0 disk space MB", default_multidb_disk_quota_mb);

932 + config_set_size_mb(CONFIG_SECTION_DB, "dbengine tier 0 retention size", default_multidb_disk_quota_mb);

933 }

934

935 #ifdef OS_WINDOWS

@@ -959,11 +959,11 @@ void dbengine_init(char *hostname) {

959 }

960

961 int disk_space_mb = tier ? RRDENG_DEFAULT_TIER_DISK_SPACE_MB : default_multidb_disk_quota_mb;

962 - snprintfz(dbengineconfig, sizeof(dbengineconfig) - 1, "dbengine tier %zu disk space MB", tier);

963 - disk_space_mb = config_get_number(CONFIG_SECTION_DB, dbengineconfig, disk_space_mb);

962 + snprintfz(dbengineconfig, sizeof(dbengineconfig) - 1, "dbengine tier %zu retention size", tier);

963 + disk_space_mb = config_get_size_mb(CONFIG_SECTION_DB, dbengineconfig, disk_space_mb);

964

965 - snprintfz(dbengineconfig, sizeof(dbengineconfig) - 1, "dbengine tier %zu retention days", tier);

966 - storage_tiers_retention_days[tier] = config_get_number(

965 + snprintfz(dbengineconfig, sizeof(dbengineconfig) - 1, "dbengine tier %zu retention time", tier);

966 + storage_tiers_retention_days[tier] = config_get_duration_days(

967 CONFIG_SECTION_DB, dbengineconfig, new_dbengine_defaults ? storage_tiers_retention_days[tier] : 0);

968

969 tiers_init[tier].disk_space_mb = (int) disk_space_mb;

@@ -1026,7 +1026,7 @@ void dbengine_init(char *hostname) {

1026

1027 void api_v1_management_init(void);

1028

1029 -int rrd_init(char *hostname, struct rrdhost_system_info *system_info, bool unittest) {

1029 +int rrd_init(const char *hostname, struct rrdhost_system_info *system_info, bool unittest) {

1030 rrdhost_init();

1031

1032 if (unlikely(sql_init_meta_database(DB_CHECK_NONE, system_info ? 0 : 1))) {

@@ -1445,6 +1445,11 @@ void rrdhost_set_is_parent_label(void) {

1445 }

1446 }

1447

1448 +static bool config_label_cb(void *data __maybe_unused, const char *name, const char *value) {

1449 + rrdlabels_add(localhost->rrdlabels, name, value, RRDLABEL_SRC_CONFIG);

1450 + return true;

1451 +}

1452 +

1453 static void rrdhost_load_config_labels(void) {

1454 int status = config_load(NULL, 1, CONFIG_SECTION_HOST_LABEL);

1455 if(!status) {

@@ -1454,16 +1459,7 @@ static void rrdhost_load_config_labels(void) {

1459 filename);

1460 }

1461

1457 - struct section *co = appconfig_get_section(&netdata_config, CONFIG_SECTION_HOST_LABEL);

1458 - if(co) {

1459 - config_section_wrlock(co);

1460 - struct config_option *cv;

1461 - for(cv = co->values; cv ; cv = cv->next) {

1462 - rrdlabels_add(localhost->rrdlabels, cv->name, cv->value, RRDLABEL_SRC_CONFIG);

1463 - cv->flags |= CONFIG_VALUE_USED;

1464 - }

1465 - config_section_unlock(co);

1466 - }

1462 + appconfig_foreach_value_in_section(&netdata_config, CONFIG_SECTION_HOST_LABEL, config_label_cb, NULL);

1463 }

1464

1465 static void rrdhost_load_kubernetes_labels(void) {

src/database/sqlite/sqlite_health.c

+1 -1

@@ -311,7 +311,7 @@ void sql_health_alarm_log_cleanup(RRDHOST *host)

311

312 int param = 0;

313 SQLITE_BIND_FAIL(done, sqlite3_bind_blob(res, ++param, &host->host_id.uuid, sizeof(host->host_id.uuid), SQLITE_STATIC));

314 - SQLITE_BIND_FAIL(done, sqlite3_bind_int64(res, ++param, (sqlite3_int64)host->health_log.health_log_history));

314 + SQLITE_BIND_FAIL(done, sqlite3_bind_int64(res, ++param, (sqlite3_int64)host->health_log.health_log_retention_s));

315

316 param = 0;

317 rc = sqlite3_step_monitored(res);

src/exporting/read_config.c

+4 -8

@@ -5,11 +5,7 @@

5 EXPORTING_OPTIONS global_exporting_options = EXPORTING_SOURCE_DATA_AVERAGE | EXPORTING_OPTION_SEND_NAMES;

6 const char *global_exporting_prefix = "netdata";

8 -struct config exporting_config = { .first_section = NULL,

9 - .last_section = NULL,

10 - .mutex = NETDATA_MUTEX_INITIALIZER,

11 - .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

12 - .rwlock = AVL_LOCK_INITIALIZER } };

8 +struct config exporting_config = APPCONFIG_INITIALIZER;

10 struct instance *prometheus_exporter_instance = NULL;

@@ -32,7 +28,7 @@ static _CONNECTOR_INSTANCE *find_instance(const char *section)

28 return local_ci;

29 }

35 -char *expconfig_get(struct config *root, const char *section, const char *name, const char *default_value)

31 +static const char *expconfig_get(struct config *root, const char *section, const char *name, const char *default_value)

32 {

33 _CONNECTOR_INSTANCE *local_ci;

@@ -243,7 +239,7 @@ struct engine *read_exporting_config()

239

240 prometheus_exporter_instance->config.options |= global_exporting_options & EXPORTING_OPTIONS_SOURCE_BITS;

241

246 - char *data_source = prometheus_config_get("data source", "average");

242 + const char *data_source = prometheus_config_get("data source", "average");

243 prometheus_exporter_instance->config.options =

244 exporting_parse_data_source(data_source, prometheus_exporter_instance->config.options);

245

@@ -378,7 +374,7 @@ struct engine *read_exporting_config()

374 tmp_instance->config.hosts_pattern = simple_pattern_create(

375 exporter_get(instance_name, "send hosts matching", "localhost *"), NULL, SIMPLE_PATTERN_EXACT, true);

376

381 - char *data_source = exporter_get(instance_name, "data source", "average");

377 + const char *data_source = exporter_get(instance_name, "data source", "average");

378

379 tmp_instance->config.options = exporting_parse_data_source(data_source, tmp_instance->config.options);

380 if (EXPORTING_OPTIONS_DATA_SOURCE(tmp_instance->config.options) != EXPORTING_SOURCE_DATA_AS_COLLECTED &&

src/health/health.c

+18 -19

@@ -14,7 +14,7 @@ struct health_plugin_globals health_globals = {

14 .use_summary_for_notifications = true,

16 .health_log_entries_max = HEALTH_LOG_ENTRIES_DEFAULT,

17 - .health_log_history = HEALTH_LOG_HISTORY_DEFAULT,

17 + .health_log_retention_s = HEALTH_LOG_RETENTION_DEFAULT,

19 .default_warn_repeat_every = 0,

20 .default_crit_repeat_every = 0,

@@ -55,17 +55,17 @@ static void health_load_config_defaults(void) {

55 health_globals.config.use_summary_for_notifications);

57 health_globals.config.default_warn_repeat_every =

58 - config_get_duration(CONFIG_SECTION_HEALTH, "default repeat warning", "never");

58 + config_get_duration_seconds(CONFIG_SECTION_HEALTH, "default repeat warning", 0);

60 health_globals.config.default_crit_repeat_every =

61 - config_get_duration(CONFIG_SECTION_HEALTH, "default repeat critical", "never");

61 + config_get_duration_seconds(CONFIG_SECTION_HEALTH, "default repeat critical", 0);

63 health_globals.config.health_log_entries_max =

64 config_get_number(CONFIG_SECTION_HEALTH, "in memory max health log entries",

65 health_globals.config.health_log_entries_max);

67 - health_globals.config.health_log_history =

68 - config_get_number(CONFIG_SECTION_HEALTH, "health log history", HEALTH_LOG_DEFAULT_HISTORY);

67 + health_globals.config.health_log_retention_s =

68 + config_get_duration_seconds(CONFIG_SECTION_HEALTH, "health log retention", HEALTH_LOG_RETENTION_DEFAULT);

70 snprintfz(filename, FILENAME_MAX, "%s/alarm-notify.sh", netdata_configured_primary_plugins_dir);

71 health_globals.config.default_exec =

@@ -76,14 +76,13 @@ static void health_load_config_defaults(void) {

76 NULL, SIMPLE_PATTERN_EXACT, true);

78 health_globals.config.run_at_least_every_seconds =

79 - (int)config_get_number(CONFIG_SECTION_HEALTH,

80 - "run at least every seconds",

81 - health_globals.config.run_at_least_every_seconds);

79 + (int)config_get_duration_seconds(CONFIG_SECTION_HEALTH, "run at least every",

80 + health_globals.config.run_at_least_every_seconds);

82 health_globals.config.postpone_alarms_during_hibernation_for_seconds =

84 - config_get_number(CONFIG_SECTION_HEALTH,

85 - "postpone alarms during hibernation for seconds",

86 - health_globals.config.postpone_alarms_during_hibernation_for_seconds);

83 + config_get_duration_seconds(CONFIG_SECTION_HEALTH,

84 + "postpone alarms during hibernation for",

85 + health_globals.config.postpone_alarms_during_hibernation_for_seconds);

87 health_globals.config.default_recipient =

88 string_strdupz("root");

@@ -115,27 +114,27 @@ static void health_load_config_defaults(void) {

114 (long)health_globals.config.health_log_entries_max);

115 }

116

118 - if (health_globals.config.health_log_history < HEALTH_LOG_MINIMUM_HISTORY) {

117 + if (health_globals.config.health_log_retention_s < HEALTH_LOG_MINIMUM_HISTORY) {

118 nd_log(NDLS_DAEMON, NDLP_WARNING,

120 - "Health configuration has invalid health log history %u. Using minimum %d",

121 - health_globals.config.health_log_history, HEALTH_LOG_MINIMUM_HISTORY);

119 + "Health configuration has invalid health log retention %u. Using minimum %d",

120 + health_globals.config.health_log_retention_s, HEALTH_LOG_MINIMUM_HISTORY);

121

123 - health_globals.config.health_log_history = HEALTH_LOG_MINIMUM_HISTORY;

124 - config_set_number(CONFIG_SECTION_HEALTH, "health log history", health_globals.config.health_log_history);

122 + health_globals.config.health_log_retention_s = HEALTH_LOG_MINIMUM_HISTORY;

123 + config_set_duration_seconds(CONFIG_SECTION_HEALTH, "health log retention", health_globals.config.health_log_retention_s);

124 }

125

126 nd_log(NDLS_DAEMON, NDLP_DEBUG,

127 "Health log history is set to %u seconds (%u days)",

129 - health_globals.config.health_log_history, health_globals.config.health_log_history / 86400);

128 + health_globals.config.health_log_retention_s, health_globals.config.health_log_retention_s / 86400);

129 }

130

132 -inline char *health_user_config_dir(void) {

131 +inline const char *health_user_config_dir(void) {

132 char buffer[FILENAME_MAX + 1];

133 snprintfz(buffer, FILENAME_MAX, "%s/health.d", netdata_configured_user_config_dir);

134 return config_get(CONFIG_SECTION_DIRECTORIES, "health config", buffer);

135 }

136

138 -inline char *health_stock_config_dir(void) {

137 +inline const char *health_stock_config_dir(void) {

138 char buffer[FILENAME_MAX + 1];

139 snprintfz(buffer, FILENAME_MAX, "%s/health.d", netdata_configured_stock_config_dir);

140 return config_get(CONFIG_SECTION_DIRECTORIES, "stock health config", buffer);

src/health/health.h

+4 -4

@@ -34,8 +34,8 @@ void health_entry_flags_to_json_array(BUFFER *wb, const char *key, HEALTH_ENTRY_

34 #define HEALTH_LISTEN_BACKLOG 4096

35 #endif

37 -#ifndef HEALTH_LOG_DEFAULT_HISTORY

38 -#define HEALTH_LOG_DEFAULT_HISTORY 432000

37 +#ifndef HEALTH_LOG_RETENTION_DEFAULT

38 +#define HEALTH_LOG_RETENTION_DEFAULT (5 * 86400)

39 #endif

41 #ifndef HEALTH_LOG_MINIMUM_HISTORY

@@ -75,8 +75,8 @@ ALARM_ENTRY* health_create_alarm_entry(

76 void health_alarm_log_add_entry(RRDHOST *host, ALARM_ENTRY *ae);

78 -char *health_user_config_dir(void);

79 -char *health_stock_config_dir(void);

78 +const char *health_user_config_dir(void);

79 +const char *health_stock_config_dir(void);

80 void health_alarm_log_free(RRDHOST *host);

82 void health_alarm_log_free_one_nochecks_nounlink(ALARM_ENTRY *ae);

src/health/health_config.c

+9 -9

@@ -29,14 +29,14 @@ static inline int health_parse_delay(

29 while(*s && isspace((uint8_t)*s)) *s++ = '\0';

31 if(!strcasecmp(key, "up")) {

32 - if (!config_parse_duration(value, delay_up_duration)) {

32 + if (!duration_parse_seconds(value, delay_up_duration)) {

33 netdata_log_error("Health configuration at line %zu of file '%s': invalid value '%s' for '%s' keyword",

34 line, filename, value, key);

35 }

36 else given_up = 1;

37 }

38 else if(!strcasecmp(key, "down")) {

39 - if (!config_parse_duration(value, delay_down_duration)) {

39 + if (!duration_parse_seconds(value, delay_down_duration)) {

40 netdata_log_error("Health configuration at line %zu of file '%s': invalid value '%s' for '%s' keyword",

41 line, filename, value, key);

42 }

@@ -51,7 +51,7 @@ static inline int health_parse_delay(

51 else given_multiplier = 1;

52 }

53 else if(!strcasecmp(key, "max")) {

54 - if (!config_parse_duration(value, delay_max_duration)) {

54 + if (!duration_parse_seconds(value, delay_max_duration)) {

55 netdata_log_error("Health configuration at line %zu of file '%s': invalid value '%s' for '%s' keyword",

56 line, filename, value, key);

57 }

@@ -139,13 +139,13 @@ static inline int health_parse_repeat(

139 return 1;

140 }

141 if(!strcasecmp(key, "warning")) {

142 - if (!config_parse_duration(value, (int*)warn_repeat_every)) {

142 + if (!duration_parse_seconds(value, (int *)warn_repeat_every)) {

143 netdata_log_error("Health configuration at line %zu of file '%s': invalid value '%s' for '%s' keyword",

144 line, file, value, key);

145 }

146 }

147 else if(!strcasecmp(key, "critical")) {

148 - if (!config_parse_duration(value, (int*)crit_repeat_every)) {

148 + if (!duration_parse_seconds(value, (int *)crit_repeat_every)) {

149 netdata_log_error("Health configuration at line %zu of file '%s': invalid value '%s' for '%s' keyword",

150 line, file, value, key);

151 }

@@ -273,7 +273,7 @@ static inline int health_parse_db_lookup(size_t line, const char *filename, char

273 while(*s && !isspace((uint8_t)*s)) s++;

274 while(*s && isspace((uint8_t)*s)) *s++ = '\0';

275

276 - if(!config_parse_duration(key, &ac->after)) {

276 + if(!duration_parse_seconds(key, &ac->after)) {

277 netdata_log_error("Health configuration at line %zu of file '%s': invalid duration '%s' after group method",

278 line, filename, key);

279 return 0;

@@ -294,7 +294,7 @@ static inline int health_parse_db_lookup(size_t line, const char *filename, char

294 while(*s && !isspace((uint8_t)*s)) s++;

295 while(*s && isspace((uint8_t)*s)) *s++ = '\0';

296

297 - if (!config_parse_duration(value, &ac->before)) {

297 + if (!duration_parse_seconds(value, &ac->before)) {

298 netdata_log_error("Health configuration at line %zu of file '%s': invalid duration '%s' for '%s' keyword",

299 line, filename, value, key);

300 }

@@ -304,7 +304,7 @@ static inline int health_parse_db_lookup(size_t line, const char *filename, char

304 while(*s && !isspace((uint8_t)*s)) s++;

305 while(*s && isspace((uint8_t)*s)) *s++ = '\0';

306

307 - if (!config_parse_duration(value, &ac->update_every)) {

307 + if (!duration_parse_seconds(value, &ac->update_every)) {

308 netdata_log_error("Health configuration at line %zu of file '%s': invalid duration '%s' for '%s' keyword",

309 line, filename, value, key);

310 }

@@ -725,7 +725,7 @@ int health_readfile(const char *filename, void *data __maybe_unused, bool stock_

725 health_parse_db_lookup(line, filename, value, ac);

726 }

727 else if(hash == hash_every && !strcasecmp(key, HEALTH_EVERY_KEY)) {

728 - if(!config_parse_duration(value, &ac->update_every))

728 + if(!duration_parse_seconds(value, &ac->update_every))

729 netdata_log_error(

730 "Health configuration at line %zu of file '%s' for alarm '%s' at key '%s' "

731 "cannot parse duration: '%s'.",

src/health/health_event_loop.c

+1 -1

@@ -130,7 +130,7 @@ static void health_initialize_rrdhost(RRDHOST *host) {

130 rrdhost_flag_set(host, RRDHOST_FLAG_INITIALIZED_HEALTH);

131

132 host->health_log.max = health_globals.config.health_log_entries_max;

133 - host->health_log.health_log_history = health_globals.config.health_log_history;

133 + host->health_log.health_log_retention_s = health_globals.config.health_log_retention_s;

134 host->health.health_default_exec = string_dup(health_globals.config.default_exec);

135 host->health.health_default_recipient = string_dup(health_globals.config.default_recipient);

136 host->health.use_summary_for_notifications = health_globals.config.use_summary_for_notifications;

src/health/health_internals.h

+2 -2

@@ -9,7 +9,7 @@

9 #define HEALTH_LOG_ENTRIES_MAX 100000U

10 #define HEALTH_LOG_ENTRIES_MIN 10U

12 -#define HEALTH_LOG_HISTORY_DEFAULT (5 * 86400)

12 +#define HEALTH_LOG_RETENTION_DEFAULT (5 * 86400)

14 #define HEALTH_CONF_MAX_LINE 4096

@@ -76,7 +76,7 @@ struct health_plugin_globals {

76 bool use_summary_for_notifications;

78 unsigned int health_log_entries_max;

79 - uint32_t health_log_history; // the health log history in seconds to be kept in db

79 + uint32_t health_log_retention_s; // the health log retention in seconds to be kept in db

81 STRING *silencers_filename;

82 STRING *default_exec;

src/libnetdata/aral/aral.c

+3 -3

@@ -72,7 +72,7 @@ struct aral {

72 struct {

73 bool enabled;

74 const char *filename;

75 - char **cache_dir;

75 + const char **cache_dir;

76 } mmap;

77 } config;

@@ -709,7 +709,7 @@ size_t aral_element_size(ARAL *ar) {

709 }

710

711 ARAL *aral_create(const char *name, size_t element_size, size_t initial_page_elements, size_t max_page_size,

712 - struct aral_statistics *stats, const char *filename, char **cache_dir, bool mmap, bool lockless) {

712 + struct aral_statistics *stats, const char *filename, const char **cache_dir, bool mmap, bool lockless) {

713 ARAL *ar = callocz(1, sizeof(ARAL));

714 ar->config.options = (lockless) ? ARAL_LOCKLESS : 0;

715 ar->config.requested_element_size = element_size;

@@ -1078,7 +1078,7 @@ int aral_stress_test(size_t threads, size_t elements, size_t seconds) {

1078 }

1079

1080 int aral_unittest(size_t elements) {

1081 - char *cache_dir = "/tmp/";

1081 + const char *cache_dir = "/tmp/";

1082

1083 struct aral_unittest_config auc = {

1084 .single_threaded = true,

src/libnetdata/aral/aral.h

+1 -1

@@ -28,7 +28,7 @@ struct aral_statistics {

28 };

30 ARAL *aral_create(const char *name, size_t element_size, size_t initial_page_elements, size_t max_page_size,

31 - struct aral_statistics *stats, const char *filename, char **cache_dir, bool mmap, bool lockless);

31 + struct aral_statistics *stats, const char *filename, const char **cache_dir, bool mmap, bool lockless);

32 size_t aral_element_size(ARAL *ar);

33 size_t aral_overhead(ARAL *ar);

34 size_t aral_structures(ARAL *ar);

src/libnetdata/clocks/clocks.c

+2 -2

@@ -452,7 +452,7 @@ static inline collected_number uptime_from_boottime(void) {

452 }

453

454 static procfile *read_proc_uptime_ff = NULL;

455 -static inline collected_number read_proc_uptime(char *filename) {

455 +static inline collected_number read_proc_uptime(const char *filename) {

456 if(unlikely(!read_proc_uptime_ff)) {

457 read_proc_uptime_ff = procfile_open(filename, " \t", PROCFILE_FLAG_DEFAULT);

458 if(unlikely(!read_proc_uptime_ff)) return 0;

@@ -473,7 +473,7 @@ static inline collected_number read_proc_uptime(char *filename) {

473 return (collected_number)(strtondd(procfile_lineword(read_proc_uptime_ff, 0, 0), NULL) * 1000.0);

474 }

475

476 -inline collected_number uptime_msec(char *filename){

476 +inline collected_number uptime_msec(const char *filename){

477 static int use_boottime = -1;

478

479 if(unlikely(use_boottime == -1)) {

src/libnetdata/clocks/clocks.h

+6 -1

@@ -18,7 +18,12 @@ struct timespec {

18 typedef uint64_t nsec_t;

19 typedef uint64_t msec_t;

20 typedef uint64_t usec_t;

21 +

22 +typedef int64_t snsec_t;

23 typedef int64_t susec_t;

24 +typedef int64_t smsec_t;

25 +

26 +typedef int64_t stime_t;

28 typedef struct heartbeat {

29 usec_t realtime;

@@ -151,7 +156,7 @@ time_t now_sec(clockid_t clk_id);

156 usec_t now_usec(clockid_t clk_id);

157 int now_timeval(clockid_t clk_id, struct timeval *tv);

158

154 -collected_number uptime_msec(char *filename);

159 +collected_number uptime_msec(const char *filename);

160

161 extern usec_t clock_monotonic_resolution;

162 extern usec_t clock_realtime_resolution;

src/libnetdata/config/appconfig.c

+47 -935

@@ -1,970 +1,82 @@

1 // SPDX-License-Identifier: GPL-3.0-or-later

3 -#include "../libnetdata.h"

3 +#include "appconfig_internals.h"

5 -/*

6 - * @Input:

7 - * Connector / instance to add to an internal structure

8 - * @Return

9 - * The current head of the linked list of connector_instance

10 - *

11 - */

12 -

13 -_CONNECTOR_INSTANCE *add_connector_instance(struct section *connector, struct section *instance)

14 -{

15 - static struct _connector_instance *global_connector_instance = NULL;

16 - struct _connector_instance *local_ci, *local_ci_tmp;

17 -

18 - if (unlikely(!connector)) {

19 - if (unlikely(!instance))

20 - return global_connector_instance;

21 -

22 - local_ci = global_connector_instance;

23 - while (local_ci) {

24 - local_ci_tmp = local_ci->next;

25 - freez(local_ci);

26 - local_ci = local_ci_tmp;

27 - }

28 - global_connector_instance = NULL;

29 - return NULL;

30 - }

31 -

32 - local_ci = callocz(1, sizeof(struct _connector_instance));

33 - local_ci->instance = instance;

34 - local_ci->connector = connector;

35 - strncpyz(local_ci->instance_name, instance->name, CONFIG_MAX_NAME);

36 - strncpyz(local_ci->connector_name, connector->name, CONFIG_MAX_NAME);

37 - local_ci->next = global_connector_instance;

38 - global_connector_instance = local_ci;

39 -

40 - return global_connector_instance;

41 -}

42 -

43 -int is_valid_connector(char *type, int check_reserved)

44 -{

45 - int rc = 1;

46 -

47 - if (unlikely(!type))

48 - return 0;

49 -

50 - if (!check_reserved) {

51 - if (unlikely(is_valid_connector(type,1))) {

52 - return 0;

53 - }

54 - //if (unlikely(*type == ':')

55 - // return 0;

56 - char *separator = strrchr(type, ':');

57 - if (likely(separator)) {

58 - *separator = '\0';

59 - rc = separator - type;

60 - } else

61 - return 0;

62 - }

63 -// else {

64 -// if (unlikely(is_valid_connector(type,1))) {

65 -// netdata_log_error("Section %s invalid -- reserved name", type);

66 -// return 0;

67 -// }

68 -// }

69 -

70 - if (!strcmp(type, "graphite") || !strcmp(type, "graphite:plaintext")) {

71 - return rc;

72 - } else if (!strcmp(type, "graphite:http") || !strcmp(type, "graphite:https")) {

73 - return rc;

74 - } else if (!strcmp(type, "json") || !strcmp(type, "json:plaintext")) {

75 - return rc;

76 - } else if (!strcmp(type, "json:http") || !strcmp(type, "json:https")) {

77 - return rc;

78 - } else if (!strcmp(type, "opentsdb") || !strcmp(type, "opentsdb:telnet")) {

79 - return rc;

80 - } else if (!strcmp(type, "opentsdb:http") || !strcmp(type, "opentsdb:https")) {

81 - return rc;

82 - } else if (!strcmp(type, "prometheus_remote_write")) {

83 - return rc;

84 - } else if (!strcmp(type, "prometheus_remote_write:http") || !strcmp(type, "prometheus_remote_write:https")) {

85 - return rc;

86 - } else if (!strcmp(type, "kinesis") || !strcmp(type, "kinesis:plaintext")) {

87 - return rc;

88 - } else if (!strcmp(type, "pubsub") || !strcmp(type, "pubsub:plaintext")) {

89 - return rc;

90 - } else if (!strcmp(type, "mongodb") || !strcmp(type, "mongodb:plaintext")) {

91 - return rc;

92 - }

93 -

94 - return 0;

95 -}

96 -

97 -// ----------------------------------------------------------------------------

98 -// locking

99 -

100 -inline void appconfig_wrlock(struct config *root) {

101 - netdata_mutex_lock(&root->mutex);

102 -}

103 -

104 -inline void appconfig_unlock(struct config *root) {

105 - netdata_mutex_unlock(&root->mutex);

106 -}

107 -

108 -inline void config_section_wrlock(struct section *co) {

109 - netdata_mutex_lock(&co->mutex);

110 -}

111 -

112 -inline void config_section_unlock(struct section *co) {

113 - netdata_mutex_unlock(&co->mutex);

114 -}

115 -

116 -

117 -// ----------------------------------------------------------------------------

118 -// config name-value index

119 -

120 -static int appconfig_option_compare(void *a, void *b) {

121 - if(((struct config_option *)a)->hash < ((struct config_option *)b)->hash) return -1;

122 - else if(((struct config_option *)a)->hash > ((struct config_option *)b)->hash) return 1;

123 - else return strcmp(((struct config_option *)a)->name, ((struct config_option *)b)->name);

124 -}

125 -

126 -#define appconfig_option_index_add(co, cv) (struct config_option *)avl_insert_lock(&((co)->values_index), (avl_t *)(cv))

127 -#define appconfig_option_index_del(co, cv) (struct config_option *)avl_remove_lock(&((co)->values_index), (avl_t *)(cv))

128 -

129 -static struct config_option *appconfig_option_index_find(struct section *co, const char *name, uint32_t hash) {

130 - struct config_option tmp;

131 - tmp.hash = (hash)?hash:simple_hash(name);

132 - tmp.name = (char *)name;

133 -

134 - return (struct config_option *)avl_search_lock(&(co->values_index), (avl_t *) &tmp);

135 -}

136 -

137 -

138 -// ----------------------------------------------------------------------------

139 -// config sections index

140 -

141 -int appconfig_section_compare(void *a, void *b) {

142 - if(((struct section *)a)->hash < ((struct section *)b)->hash) return -1;

143 - else if(((struct section *)a)->hash > ((struct section *)b)->hash) return 1;

144 - else return strcmp(((struct section *)a)->name, ((struct section *)b)->name);

145 -}

146 -

147 -#define appconfig_index_add(root, cfg) (struct section *)avl_insert_lock(&(root)->index, (avl_t *)(cfg))

148 -#define appconfig_index_del(root, cfg) (struct section *)avl_remove_lock(&(root)->index, (avl_t *)(cfg))

149 -

150 -static struct section *appconfig_index_find(struct config *root, const char *name, uint32_t hash) {

151 - struct section tmp;

152 - tmp.hash = (hash)?hash:simple_hash(name);

153 - tmp.name = (char *)name;

154 -

155 - return (struct section *)avl_search_lock(&root->index, (avl_t *) &tmp);

156 -}

157 -

158 -

159 -// ----------------------------------------------------------------------------

160 -// config section methods

161 -

162 -static inline struct section *appconfig_section_find(struct config *root, const char *section) {

163 - return appconfig_index_find(root, section, 0);

164 -}

165 -

166 -static inline struct section *appconfig_section_create(struct config *root, const char *section) {

167 - netdata_log_debug(D_CONFIG, "Creating section '%s'.", section);

168 -

169 - struct section *co = callocz(1, sizeof(struct section));

170 - co->name = strdupz(section);

171 - co->hash = simple_hash(co->name);

172 - netdata_mutex_init(&co->mutex);

173 -

174 - avl_init_lock(&co->values_index, appconfig_option_compare);

175 -

176 - if(unlikely(appconfig_index_add(root, co) != co))

177 - netdata_log_error("INTERNAL ERROR: indexing of section '%s', already exists.", co->name);

5 +int appconfig_exists(struct config *root, const char *section, const char *name) {

6 + struct config_section *sect = appconfig_section_find(root, section);

7 + if(!sect) return 0;

179 - appconfig_wrlock(root);

180 - struct section *co2 = root->last_section;

181 - if(co2) {

182 - co2->next = co;

183 - } else {

184 - root->first_section = co;

185 - }

186 - root->last_section = co;

187 - appconfig_unlock(root);

9 + struct config_option *opt = appconfig_option_find(sect, name);

10 + if(!opt) return 0;

189 - return co;

12 + return 1;

13 }

192 -void appconfig_section_destroy_non_loaded(struct config *root, const char *section)

193 -{

194 - struct section *co;

195 - struct config_option *cv, *cv_next;

196 -

197 - netdata_log_debug(D_CONFIG, "Destroying section '%s'.", section);

198 -

199 - co = appconfig_section_find(root, section);

200 - if(!co) {

201 - netdata_log_error("Could not destroy section '%s'. Not found.", section);

202 - return;

203 - }

204 -

205 - config_section_wrlock(co);

206 - for(cv = co->values; cv ; cv = cv->next) {

207 - if (cv->flags & CONFIG_VALUE_LOADED) {

208 - /* Do not destroy values that were loaded from the configuration files. */

209 - config_section_unlock(co);

210 - return;

211 - }

212 - }

213 - for(cv = co->values ; cv ; cv = cv_next) {

214 - cv_next = cv->next;

215 - if(unlikely(!appconfig_option_index_del(co, cv)))

216 - netdata_log_error("Cannot remove config option '%s' from section '%s'.", cv->name, co->name);

217 - freez(cv->value);

218 - freez(cv->name);

219 - freez(cv);

220 - }

221 - co->values = NULL;

222 - config_section_unlock(co);

223 -

224 - if (unlikely(!appconfig_index_del(root, co))) {

225 - netdata_log_error("Cannot remove section '%s' from config.", section);

15 +void appconfig_set_default_raw_value(struct config *root, const char *section, const char *name, const char *value) {

16 + struct config_section *sect = appconfig_section_find(root, section);

17 + if(!sect) {

18 + appconfig_set_raw_value(root, section, name, value, CONFIG_VALUE_TYPE_UNKNOWN);

19 return;

20 }

228 -

229 - appconfig_wrlock(root);

230 -

231 - if (root->first_section == co) {

232 - root->first_section = co->next;

233 -

234 - if (root->last_section == co)

235 - root->last_section = root->first_section;

236 - } else {

237 - struct section *co_cur = root->first_section, *co_prev = NULL;

238 -

239 - while(co_cur && co_cur != co) {

240 - co_prev = co_cur;

241 - co_cur = co_cur->next;

242 - }

243 -

244 - if (co_cur) {

245 - co_prev->next = co_cur->next;

246 -

247 - if (root->last_section == co_cur)

248 - root->last_section = co_prev;

249 - }

250 - }

251 -

252 - appconfig_unlock(root);

253 -

254 - avl_destroy_lock(&co->values_index);

255 - freez(co->name);

256 - pthread_mutex_destroy(&co->mutex);

257 - freez(co);

258 -}

259 -

260 -void appconfig_section_option_destroy_non_loaded(struct config *root, const char *section, const char *name)

261 -{

262 - netdata_log_debug(D_CONFIG, "Destroying section option '%s -> %s'.", section, name);

264 - struct section *co;

265 - co = appconfig_section_find(root, section);

266 - if (!co) {

267 - netdata_log_error("Could not destroy section option '%s -> %s'. The section not found.", section, name);

22 + struct config_option *opt = appconfig_option_find(sect, name);

23 + if(!opt) {

24 + appconfig_set_raw_value(root, section, name, value, CONFIG_VALUE_TYPE_UNKNOWN);

25 return;

26 }

271 - config_section_wrlock(co);

28 + opt->flags |= CONFIG_VALUE_USED;

273 - struct config_option *cv;

274 -

275 - cv = appconfig_option_index_find(co, name, simple_hash(name));

276 -

277 - if (cv && cv->flags & CONFIG_VALUE_LOADED) {

278 - config_section_unlock(co);

30 + if(opt->flags & CONFIG_VALUE_LOADED)

31 return;

280 - }

282 - if (unlikely(!(cv && appconfig_option_index_del(co, cv)))) {

283 - config_section_unlock(co);

284 - netdata_log_error("Could not destroy section option '%s -> %s'. The option not found.", section, name);

285 - return;

286 - }

33 + if(string_strcmp(opt->value, value) != 0) {

34 + opt->flags |= CONFIG_VALUE_CHANGED;

288 - if (co->values == cv) {

289 - co->values = co->values->next;

290 - } else {

291 - struct config_option *cv_cur = co->values, *cv_prev = NULL;

292 - while (cv_cur && cv_cur != cv) {

293 - cv_prev = cv_cur;

294 - cv_cur = cv_cur->next;

295 - }

296 - if (cv_cur) {

297 - cv_prev->next = cv_cur->next;

298 - }

36 + string_freez(opt->value);

37 + opt->value = string_strdupz(value);

38 }

300 -

301 - freez(cv->value);

302 - freez(cv->name);

303 - freez(cv);

304 -

305 - config_section_unlock(co);

306 - return;

307 -}

308 -

309 -// ----------------------------------------------------------------------------

310 -// config name-value methods

311 -

312 -static inline struct config_option *appconfig_value_create(struct section *co, const char *name, const char *value) {

313 - netdata_log_debug(D_CONFIG, "Creating config entry for name '%s', value '%s', in section '%s'.", name, value, co->name);

314 -

315 - struct config_option *cv = callocz(1, sizeof(struct config_option));

316 - cv->name = strdupz(name);

317 - cv->hash = simple_hash(cv->name);

318 - cv->value = strdupz(value);

319 -

320 - struct config_option *found = appconfig_option_index_add(co, cv);

321 - if(found != cv) {

322 - netdata_log_error("indexing of config '%s' in section '%s': already exists - using the existing one.", cv->name, co->name);

323 - freez(cv->value);

324 - freez(cv->name);

325 - freez(cv);

326 - return found;

327 - }

328 -

329 - config_section_wrlock(co);

330 - struct config_option *cv2 = co->values;

331 - if(cv2) {

332 - while (cv2->next) cv2 = cv2->next;

333 - cv2->next = cv;

334 - }

335 - else co->values = cv;

336 - config_section_unlock(co);

337 -

338 - return cv;

39 }

341 -int appconfig_exists(struct config *root, const char *section, const char *name) {

342 - struct config_option *cv;

343 -

344 - netdata_log_debug(D_CONFIG, "request to get config in section '%s', name '%s'", section, name);

345 -

346 - struct section *co = appconfig_section_find(root, section);

347 - if(!co) return 0;

348 -

349 - cv = appconfig_option_index_find(co, name, 0);

350 - if(!cv) return 0;

351 -

352 - return 1;

353 -}

354 -

355 -int appconfig_move(struct config *root, const char *section_old, const char *name_old, const char *section_new, const char *name_new) {

356 - struct config_option *cv_old, *cv_new;

357 - int ret = -1;

358 -

359 - netdata_log_debug(D_CONFIG, "request to rename config in section '%s', old name '%s', to section '%s', new name '%s'", section_old, name_old, section_new, name_new);

360 -

361 - struct section *co_old = appconfig_section_find(root, section_old);

362 - if(!co_old) return ret;

363 -

364 - struct section *co_new = appconfig_section_find(root, section_new);

365 - if(!co_new) co_new = appconfig_section_create(root, section_new);

366 -

367 - config_section_wrlock(co_old);

368 - if(co_old != co_new)

369 - config_section_wrlock(co_new);

370 -

371 - cv_old = appconfig_option_index_find(co_old, name_old, 0);

372 - if(!cv_old) goto cleanup;

373 -

374 - cv_new = appconfig_option_index_find(co_new, name_new, 0);

375 - if(cv_new) goto cleanup;

376 -

377 - if(unlikely(appconfig_option_index_del(co_old, cv_old) != cv_old))

378 - netdata_log_error("INTERNAL ERROR: deletion of config '%s' from section '%s', deleted the wrong config entry.", cv_old->name, co_old->name);

379 -

380 - if(co_old->values == cv_old) {

381 - co_old->values = cv_old->next;

382 - }

383 - else {

384 - struct config_option *t;

385 - for(t = co_old->values; t && t->next != cv_old ;t = t->next) ;

386 - if(!t || t->next != cv_old)

387 - netdata_log_error("INTERNAL ERROR: cannot find variable '%s' in section '%s' of the config - but it should be there.", cv_old->name, co_old->name);

388 - else

389 - t->next = cv_old->next;

390 - }

391 -

392 - freez(cv_old->name);

393 - cv_old->name = strdupz(name_new);

394 - cv_old->hash = simple_hash(cv_old->name);

395 -

396 - cv_new = cv_old;

397 - cv_new->next = co_new->values;

398 - co_new->values = cv_new;

399 -

400 - if(unlikely(appconfig_option_index_add(co_new, cv_old) != cv_old))

401 - netdata_log_error("INTERNAL ERROR: re-indexing of config '%s' in section '%s', already exists.", cv_old->name, co_new->name);

402 -

403 - ret = 0;

404 -

405 -cleanup:

406 - if(co_old != co_new)

407 - config_section_unlock(co_new);

408 - config_section_unlock(co_old);

409 - return ret;

410 -}

411 -

412 -char *appconfig_get_by_section(struct section *co, const char *name, const char *default_value)

413 -{

414 - struct config_option *cv;

415 -

416 - // Only calls internal to this file check for a NULL result and they do not supply a NULL arg.

417 - // External caller should treat NULL as an error case.

418 - cv = appconfig_option_index_find(co, name, 0);

419 - if (!cv) {

420 - if (!default_value) return NULL;

421 - cv = appconfig_value_create(co, name, default_value);

422 - if (!cv) return NULL;

423 - }

424 - cv->flags |= CONFIG_VALUE_USED;

425 -

426 - if((cv->flags & CONFIG_VALUE_LOADED) || (cv->flags & CONFIG_VALUE_CHANGED)) {

427 - // this is a loaded value from the config file

428 - // if it is different than the default, mark it

429 - if(!(cv->flags & CONFIG_VALUE_CHECKED)) {

430 - if(default_value && strcmp(cv->value, default_value) != 0) cv->flags |= CONFIG_VALUE_CHANGED;

431 - cv->flags |= CONFIG_VALUE_CHECKED;

432 - }

433 - }

434 -

435 - return(cv->value);

436 -}

437 -

438 -

439 -char *appconfig_get(struct config *root, const char *section, const char *name, const char *default_value)

440 -{

441 - if (default_value == NULL)

442 - netdata_log_debug(D_CONFIG, "request to get config in section '%s', name '%s' or fail", section, name);

443 - else

444 - netdata_log_debug(D_CONFIG, "request to get config in section '%s', name '%s', default_value '%s'", section, name, default_value);

445 -

446 - struct section *co = appconfig_section_find(root, section);

447 - if (!co && !default_value)

448 - return NULL;

449 - if(!co) co = appconfig_section_create(root, section);

450 -

451 - return appconfig_get_by_section(co, name, default_value);

452 -}

41 +bool stream_conf_needs_dbengine(struct config *root) {

42 + struct config_section *sect;

43 + bool ret = false;

454 -long long appconfig_get_number(struct config *root, const char *section, const char *name, long long value)

455 -{

456 - char buffer[100], *s;

457 - sprintf(buffer, "%lld", value);

45 + APPCONFIG_LOCK(root);

46 + for(sect = root->sections; sect; sect = sect->next) {

47 + if(string_strcmp(sect->name, "stream") == 0)

48 + continue; // the first section is not relevant

459 - s = appconfig_get(root, section, name, buffer);

460 - if(!s) return value;

461 -

462 - return strtoll(s, NULL, 0);

463 -}

464 -

465 -NETDATA_DOUBLE appconfig_get_float(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value)

466 -{

467 - char buffer[100], *s;

468 - sprintf(buffer, "%0.5" NETDATA_DOUBLE_MODIFIER, value);

469 -

470 - s = appconfig_get(root, section, name, buffer);

471 - if(!s) return value;

472 -

473 - return str2ndd(s, NULL);

474 -}

475 -

476 -inline int appconfig_test_boolean_value(char *s) {

477 - if(!strcasecmp(s, "yes") || !strcasecmp(s, "true") || !strcasecmp(s, "on")

478 - || !strcasecmp(s, "auto") || !strcasecmp(s, "on demand"))

479 - return 1;

480 -

481 - return 0;

482 -}

483 -

484 -int appconfig_get_boolean_by_section(struct section *co, const char *name, int value) {

485 - char *s;

486 -

487 - s = appconfig_get_by_section(co, name, (!value)?"no":"yes");

488 - if(!s) return value;

489 -

490 - return appconfig_test_boolean_value(s);

491 -}

492 -

493 -int appconfig_get_boolean(struct config *root, const char *section, const char *name, int value)

494 -{

495 - char *s;

496 - if(value) s = "yes";

497 - else s = "no";

498 -

499 - s = appconfig_get(root, section, name, s);

500 - if(!s) return value;

501 -

502 - return appconfig_test_boolean_value(s);

503 -}

504 -

505 -int appconfig_get_boolean_ondemand(struct config *root, const char *section, const char *name, int value)

506 -{

507 - char *s;

508 -

509 - if(value == CONFIG_BOOLEAN_AUTO)

510 - s = "auto";

511 -

512 - else if(value == CONFIG_BOOLEAN_NO)

513 - s = "no";

514 -

515 - else

516 - s = "yes";

517 -

518 - s = appconfig_get(root, section, name, s);

519 - if(!s) return value;

520 -

521 - if(!strcmp(s, "yes") || !strcmp(s, "true") || !strcmp(s, "on"))

522 - return CONFIG_BOOLEAN_YES;

523 - else if(!strcmp(s, "no") || !strcmp(s, "false") || !strcmp(s, "off"))

524 - return CONFIG_BOOLEAN_NO;

525 - else if(!strcmp(s, "auto") || !strcmp(s, "on demand"))

526 - return CONFIG_BOOLEAN_AUTO;

527 -

528 - return value;

529 -}

530 -

531 -const char *appconfig_set_default(struct config *root, const char *section, const char *name, const char *value)

532 -{

533 - struct config_option *cv;

534 -

535 - netdata_log_debug(D_CONFIG, "request to set default config in section '%s', name '%s', value '%s'", section, name, value);

536 -

537 - struct section *co = appconfig_section_find(root, section);

538 - if(!co) return appconfig_set(root, section, name, value);

539 -

540 - cv = appconfig_option_index_find(co, name, 0);

541 - if(!cv) return appconfig_set(root, section, name, value);

542 -

543 - cv->flags |= CONFIG_VALUE_USED;

544 -

545 - if(cv->flags & CONFIG_VALUE_LOADED)

546 - return cv->value;

547 -

548 - if(strcmp(cv->value, value) != 0) {

549 - cv->flags |= CONFIG_VALUE_CHANGED;

550 -

551 - freez(cv->value);

552 - cv->value = strdupz(value);

553 - }

554 -

555 - return cv->value;

556 -}

557 -

558 -const char *appconfig_set(struct config *root, const char *section, const char *name, const char *value)

559 -{

560 - struct config_option *cv;

561 -

562 - netdata_log_debug(D_CONFIG, "request to set config in section '%s', name '%s', value '%s'", section, name, value);

563 -

564 - struct section *co = appconfig_section_find(root, section);

565 - if(!co) co = appconfig_section_create(root, section);

566 -

567 - cv = appconfig_option_index_find(co, name, 0);

568 - if(!cv) cv = appconfig_value_create(co, name, value);

569 - cv->flags |= CONFIG_VALUE_USED;

570 -

571 - if(strcmp(cv->value, value) != 0) {

572 - cv->flags |= CONFIG_VALUE_CHANGED;

573 -

574 - freez(cv->value);

575 - cv->value = strdupz(value);

576 - }

577 -

578 - return value;

579 -}

580 -

581 -long long appconfig_set_number(struct config *root, const char *section, const char *name, long long value)

582 -{

583 - char buffer[100];

584 - sprintf(buffer, "%lld", value);

585 -

586 - appconfig_set(root, section, name, buffer);

587 -

588 - return value;

589 -}

590 -

591 -NETDATA_DOUBLE appconfig_set_float(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value)

592 -{

593 - char buffer[100];

594 - sprintf(buffer, "%0.5" NETDATA_DOUBLE_MODIFIER, value);

595 -

596 - appconfig_set(root, section, name, buffer);

597 -

598 - return value;

599 -}

600 -

601 -int appconfig_set_boolean(struct config *root, const char *section, const char *name, int value)

602 -{

603 - char *s;

604 - if(value) s = "yes";

605 - else s = "no";

606 -

607 - appconfig_set(root, section, name, s);

608 -

609 - return value;

610 -}

611 -

612 -int appconfig_get_duration(struct config *root, const char *section, const char *name, const char *value)

613 -{

614 - int result = 0;

615 - const char *s;

616 -

617 - s = appconfig_get(root, section, name, value);

618 - if(!s) goto fallback;

619 -

620 - if(!config_parse_duration(s, &result)) {

621 - netdata_log_error("config option '[%s].%s = %s' is configured with an valid duration", section, name, s);

622 - goto fallback;

623 - }

624 -

625 - return result;

626 -

627 - fallback:

628 - if(!config_parse_duration(value, &result))

629 - netdata_log_error("INTERNAL ERROR: default duration supplied for option '[%s].%s = %s' is not a valid duration", section, name, value);

630 -

631 - return result;

632 -}

633 -

634 -// ----------------------------------------------------------------------------

635 -// config load/save

636 -

637 -int appconfig_load(struct config *root, char *filename, int overwrite_used, const char *section_name)

638 -{

639 - int line = 0;

640 - struct section *co = NULL;

641 - int is_exporter_config = 0;

642 - int _connectors = 0; // number of exporting connector sections we have

643 - char working_instance[CONFIG_MAX_NAME + 1];

644 - char working_connector[CONFIG_MAX_NAME + 1];

645 - struct section *working_connector_section = NULL;

646 - int global_exporting_section = 0;

647 -

648 - char buffer[CONFIG_FILE_LINE_MAX + 1], *s;

649 -

650 - if(!filename) filename = CONFIG_DIR "/" CONFIG_FILENAME;

651 -

652 - netdata_log_debug(D_CONFIG, "CONFIG: opening config file '%s'", filename);

653 -

654 - FILE *fp = fopen(filename, "r");

655 - if(!fp) {

656 - if(errno != ENOENT)

657 - netdata_log_info("CONFIG: cannot open file '%s'. Using internal defaults.", filename);

658 -

659 - return 0;

660 - }

661 -

662 - uint32_t section_hash = 0;

663 - if(section_name) {

664 - section_hash = simple_hash(section_name);

665 - }

666 - is_exporter_config = (strstr(filename, EXPORTING_CONF) != NULL);

667 -

668 - while(fgets(buffer, CONFIG_FILE_LINE_MAX, fp) != NULL) {

669 - buffer[CONFIG_FILE_LINE_MAX] = '\0';

670 - line++;

671 -

672 - s = trim(buffer);

673 - if(!s || *s == '#') {

674 - netdata_log_debug(D_CONFIG, "CONFIG: ignoring line %d of file '%s', it is empty.", line, filename);

675 - continue;

676 - }

677 -

678 - int len = (int) strlen(s);

679 - if(*s == '[' && s[len - 1] == ']') {

680 - // new section

681 - s[len - 1] = '\0';

682 - s++;

683 -

684 - if (is_exporter_config) {

685 - global_exporting_section =

686 - !(strcmp(s, CONFIG_SECTION_EXPORTING)) || !(strcmp(s, CONFIG_SECTION_PROMETHEUS));

687 - if (unlikely(!global_exporting_section)) {

688 - int rc;

689 - rc = is_valid_connector(s, 0);

690 - if (likely(rc)) {

691 - strncpyz(working_connector, s, CONFIG_MAX_NAME);

692 - s = s + rc + 1;

693 - if (unlikely(!(*s))) {

694 - _connectors++;

695 - sprintf(buffer, "instance_%d", _connectors);

696 - s = buffer;

697 - }

698 - strncpyz(working_instance, s, CONFIG_MAX_NAME);

699 - working_connector_section = NULL;

700 - if (unlikely(appconfig_section_find(root, working_instance))) {

701 - netdata_log_error("Instance (%s) already exists", working_instance);

702 - co = NULL;

703 - continue;

704 - }

705 - } else {

706 - co = NULL;

707 - netdata_log_error("Section (%s) does not specify a valid connector", s);

708 - continue;

709 - }

710 - }

711 - }

712 -

713 - co = appconfig_section_find(root, s);

714 - if(!co) co = appconfig_section_create(root, s);

715 -

716 - if(co && section_name && overwrite_used && section_hash == co->hash && !strcmp(section_name, co->name)) {

717 - config_section_wrlock(co);

718 - struct config_option *cv2 = co->values;

719 - while (cv2) {

720 - struct config_option *save = cv2->next;

721 - struct config_option *found = appconfig_option_index_del(co, cv2);

722 - if(found != cv2)

723 - netdata_log_error("INTERNAL ERROR: Cannot remove '%s' from section '%s', it was not inserted before.",

724 - cv2->name, co->name);

725 -

726 - freez(cv2->name);

727 - freez(cv2->value);

728 - freez(cv2);

729 - cv2 = save;

730 - }

731 - co->values = NULL;

732 - config_section_unlock(co);

733 - }

734 -

735 - continue;

736 - }

737 -

738 - if(!co) {

739 - // line outside a section

740 - netdata_log_error("CONFIG: ignoring line %d ('%s') of file '%s', it is outside all sections.", line, s, filename);

741 - continue;

742 - }

743 -

744 - if(section_name && overwrite_used && section_hash != co->hash && strcmp(section_name, co->name)) {

745 - continue;

746 - }

747 -

748 - char *name = s;

749 - char *value = strchr(s, '=');

750 - if(!value) {

751 - netdata_log_error("CONFIG: ignoring line %d ('%s') of file '%s', there is no = in it.", line, s, filename);

752 - continue;

753 - }

754 - *value = '\0';

755 - value++;

756 -

757 - name = trim(name);

758 - value = trim(value);

759 -

760 - if(!name || *name == '#') {

761 - netdata_log_error("CONFIG: ignoring line %d of file '%s', name is empty.", line, filename);

50 + struct config_option *opt = appconfig_get_raw_value_of_option_in_section(sect, "enabled", NULL, CONFIG_VALUE_TYPE_UNKNOWN, NULL);

51 + if(!opt || !appconfig_test_boolean_value(string2str(opt->value)))

52 continue;

763 - }

764 -

765 - if(!value) value = "";

766 -

767 - struct config_option *cv = appconfig_option_index_find(co, name, 0);

768 -

769 - if (!cv) {

770 - cv = appconfig_value_create(co, name, value);

771 - if (likely(is_exporter_config) && unlikely(!global_exporting_section)) {

772 - if (unlikely(!working_connector_section)) {

773 - working_connector_section = appconfig_section_find(root, working_connector);

774 - if (!working_connector_section)

775 - working_connector_section = appconfig_section_create(root, working_connector);

776 - if (likely(working_connector_section)) {

777 - add_connector_instance(working_connector_section, co);

778 - }

779 - }

780 - }

781 - } else {

782 - if (((cv->flags & CONFIG_VALUE_USED) && overwrite_used) || !(cv->flags & CONFIG_VALUE_USED)) {

783 - netdata_log_debug(

784 - D_CONFIG, "CONFIG: line %d of file '%s', overwriting '%s/%s'.", line, filename, co->name, cv->name);

785 - freez(cv->value);

786 - cv->value = strdupz(value);

787 - } else

788 - netdata_log_debug(

789 - D_CONFIG,

790 - "CONFIG: ignoring line %d of file '%s', '%s/%s' is already present and used.",

791 - line,

792 - filename,

793 - co->name,

794 - cv->name);

795 - }

796 - cv->flags |= CONFIG_VALUE_LOADED;

797 - }

798 -

799 - fclose(fp);

801 - return 1;

802 -}

803 -

804 -void appconfig_generate(struct config *root, BUFFER *wb, int only_changed, bool netdata_conf)

805 -{

806 - int i, pri;

807 - struct section *co;

808 - struct config_option *cv;

809 -

810 - {

811 - int found_host_labels = 0;

812 - for (co = root->first_section; co; co = co->next)

813 - if(!strcmp(co->name, CONFIG_SECTION_HOST_LABEL))

814 - found_host_labels = 1;

815 -

816 - if(netdata_conf && !found_host_labels) {

817 - appconfig_section_create(root, CONFIG_SECTION_HOST_LABEL);

818 - appconfig_get(root, CONFIG_SECTION_HOST_LABEL, "name", "value");

54 + opt = appconfig_get_raw_value_of_option_in_section(sect, "db", NULL, CONFIG_VALUE_TYPE_UNKNOWN, NULL);

55 + if(opt && string_strcmp(opt->value, "dbengine") == 0) {

56 + ret = true;

57 + break;

58 }

59 }

60 + APPCONFIG_UNLOCK(root);

822 - if(netdata_conf) {

823 - buffer_strcat(wb,

824 - "# netdata configuration\n"

825 - "#\n"

826 - "# You can download the latest version of this file, using:\n"

827 - "#\n"

828 - "# wget -O /etc/netdata/netdata.conf http://localhost:19999/netdata.conf\n"

829 - "# or\n"

830 - "# curl -o /etc/netdata/netdata.conf http://localhost:19999/netdata.conf\n"

831 - "#\n"

832 - "# You can uncomment and change any of the options below.\n"

833 - "# The value shown in the commented settings, is the default value.\n"

834 - "#\n"

835 - "\n# global netdata configuration\n");

836 - }

837 -

838 - for(i = 0; i <= 17 ;i++) {

839 - appconfig_wrlock(root);

840 - for(co = root->first_section; co ; co = co->next) {

841 - if(!strcmp(co->name, CONFIG_SECTION_GLOBAL)) pri = 0;

842 - else if(!strcmp(co->name, CONFIG_SECTION_DB)) pri = 1;

843 - else if(!strcmp(co->name, CONFIG_SECTION_DIRECTORIES)) pri = 2;

844 - else if(!strcmp(co->name, CONFIG_SECTION_LOGS)) pri = 3;

845 - else if(!strcmp(co->name, CONFIG_SECTION_ENV_VARS)) pri = 4;

846 - else if(!strcmp(co->name, CONFIG_SECTION_HOST_LABEL)) pri = 5;

847 - else if(!strcmp(co->name, CONFIG_SECTION_SQLITE)) pri = 6;

848 - else if(!strcmp(co->name, CONFIG_SECTION_CLOUD)) pri = 7;

849 - else if(!strcmp(co->name, CONFIG_SECTION_ML)) pri = 8;

850 - else if(!strcmp(co->name, CONFIG_SECTION_HEALTH)) pri = 9;

851 - else if(!strcmp(co->name, CONFIG_SECTION_WEB)) pri = 10;

852 - else if(!strcmp(co->name, CONFIG_SECTION_WEBRTC)) pri = 11;

853 - // by default, new sections will get pri = 12 (set at the end, below)

854 - else if(!strcmp(co->name, CONFIG_SECTION_REGISTRY)) pri = 13;

855 - else if(!strcmp(co->name, CONFIG_SECTION_GLOBAL_STATISTICS)) pri = 14;

856 - else if(!strcmp(co->name, CONFIG_SECTION_PLUGINS)) pri = 15;

857 - else if(!strcmp(co->name, CONFIG_SECTION_STATSD)) pri = 16;

858 - else if(!strncmp(co->name, "plugin:", 7)) pri = 17; // << change the loop too if you change this

859 - else pri = 12; // this is used for any new (currently unknown) sections

860 -

861 - if(i == pri) {

862 - int loaded = 0;

863 - int used = 0;

864 - int changed = 0;

865 - int count = 0;

866 -

867 - config_section_wrlock(co);

868 - for(cv = co->values; cv ; cv = cv->next) {

869 - used += (cv->flags & CONFIG_VALUE_USED)?1:0;

870 - loaded += (cv->flags & CONFIG_VALUE_LOADED)?1:0;

871 - changed += (cv->flags & CONFIG_VALUE_CHANGED)?1:0;

872 - count++;

873 - }

874 - config_section_unlock(co);

875 -

876 - if(!count) continue;

877 - if(only_changed && !changed && !loaded) continue;

878 -

879 - if(!used) {

880 - buffer_sprintf(wb, "\n# section '%s' is not used.", co->name);

881 - }

882 -

883 - buffer_sprintf(wb, "\n[%s]\n", co->name);

884 -

885 - config_section_wrlock(co);

886 - for(cv = co->values; cv ; cv = cv->next) {

887 -

888 - if(used && !(cv->flags & CONFIG_VALUE_USED)) {

889 - buffer_sprintf(wb, "\n\t# option '%s' is not used.\n", cv->name);

890 - }

891 - buffer_sprintf(wb, "\t%s%s = %s\n",

892 - (

893 - !(cv->flags & CONFIG_VALUE_LOADED) &&

894 - !(cv->flags & CONFIG_VALUE_CHANGED) &&

895 - (cv->flags & CONFIG_VALUE_USED)

896 - )?"# ":"", cv->name, cv->value);

897 - }

898 - config_section_unlock(co);

899 - }

900 - }

901 - appconfig_unlock(root);

902 - }

62 + return ret;

63 }

905 -/**

906 - * Parse Duration

907 - *

908 - * Parse the string setting the result

909 - *

910 - * @param string the timestamp string

911 - * @param result the output variable

912 - *

913 - * @return It returns 1 on success and 0 otherwise

914 - */

915 -int config_parse_duration(const char* string, int* result) {

916 - while(*string && isspace((uint8_t)*string)) string++;

917 -

918 - if(unlikely(!*string)) goto fallback;

919 -

920 - if(*string == 'n' && !strcmp(string, "never")) {

921 - // this is a valid option

922 - *result = 0;

923 - return 1;

924 - }

65 +bool stream_conf_has_uuid_section(struct config *root) {

66 + struct config_section *sect = NULL;

67 + bool is_parent = false;

926 - // make sure it is a number

927 - if(!(isdigit((uint8_t)*string) || *string == '+' || *string == '-')) goto fallback;

69 + APPCONFIG_LOCK(root);

70 + for (sect = root->sections; sect; sect = sect->next) {

71 + nd_uuid_t uuid;

929 - char *e = NULL;

930 - NETDATA_DOUBLE n = str2ndd(string, &e);

931 - if(e && *e) {

932 - switch (*e) {

933 - case 'Y':

934 - *result = (int) (n * 31536000);

935 - break;

936 - case 'M':

937 - *result = (int) (n * 2592000);

938 - break;

939 - case 'w':

940 - *result = (int) (n * 604800);

941 - break;

942 - case 'd':

943 - *result = (int) (n * 86400);

944 - break;

945 - case 'h':

946 - *result = (int) (n * 3600);

947 - break;

948 - case 'm':

949 - *result = (int) (n * 60);

950 - break;

951 - case 's':

952 - default:

953 - *result = (int) (n);

954 - break;

73 + if (uuid_parse(string2str(sect->name), uuid) != -1 &&

74 + appconfig_get_boolean_by_section(sect, "enabled", 0)) {

75 + is_parent = true;

76 + break;

77 }

78 }

957 - else

958 - *result = (int)(n);

959 -

960 - return 1;

961 -

962 - fallback:

963 - *result = 0;

964 - return 0;

965 -}

79 + APPCONFIG_UNLOCK(root);

967 -struct section *appconfig_get_section(struct config *root, const char *name)

968 -{

969 - return appconfig_section_find(root, name);

81 + return is_parent;

82 }

src/libnetdata/config/appconfig.h

+49 -77

@@ -103,7 +103,6 @@

103 #define CONFIG_SECTION_GLOBAL_STATISTICS "global statistics"

104 #define CONFIG_SECTION_DB "db"

105

106 -

106 // these are used to limit the configuration names and values lengths

107 // they are not enforced by config.c functions (they will strdup() all strings, no matter of their length)

108 #define CONFIG_MAX_NAME 1024

@@ -113,94 +112,43 @@

112 // Config definitions

113 #define CONFIG_FILE_LINE_MAX ((CONFIG_MAX_NAME + CONFIG_MAX_VALUE + 1024) * 2)

114

116 -#define CONFIG_VALUE_LOADED 0x01 // has been loaded from the config

117 -#define CONFIG_VALUE_USED 0x02 // has been accessed from the program

118 -#define CONFIG_VALUE_CHANGED 0x04 // has been changed from the loaded value or the internal default value

119 -#define CONFIG_VALUE_CHECKED 0x08 // has been checked if the value is different from the default

120 -

121 -struct config_option {

122 - avl_t avl_node; // the index entry of this entry - this has to be first!

123 -

124 - uint8_t flags;

125 - uint32_t hash; // a simple hash to speed up searching

126 - // we first compare hashes, and only if the hashes are equal we do string comparisons

127 -

128 - char *name;

129 - char *value;

130 -

131 - struct config_option *next; // config->mutex protects just this

132 -};

133 -

134 -struct section {

135 - avl_t avl_node; // the index entry of this section - this has to be first!

136 -

137 - uint32_t hash; // a simple hash to speed up searching

138 - // we first compare hashes, and only if the hashes are equal we do string comparisons

139 -

140 - char *name;

141 -

142 - struct section *next; // global config_mutex protects just this

143 -

144 - struct config_option *values;

145 - avl_tree_lock values_index;

146 -

147 - netdata_mutex_t mutex; // this locks only the writers, to ensure atomic updates

148 - // readers are protected using the rwlock in avl_tree_lock

149 -};

115 +struct config_section;

116

117 struct config {

152 - struct section *first_section;

153 - struct section *last_section; // optimize inserting at the end

154 - netdata_mutex_t mutex;

118 + struct config_section *sections;

119 + SPINLOCK spinlock;

120 avl_tree_lock index;

121 };

122

158 -#define CONFIG_BOOLEAN_INVALID 100 // an invalid value to check for validity (used as default initialization when needed)

123 +#define APPCONFIG_INITIALIZER (struct config) { \

124 + .sections = NULL, \

125 + .spinlock = NETDATA_SPINLOCK_INITIALIZER, \

126 + .index = { \

127 + .avl_tree = { \

128 + .root = NULL, \

129 + .compar = appconfig_section_compare, \

130 + }, \

131 + .rwlock = AVL_LOCK_INITIALIZER, \

132 + }, \

133 + }

134

160 -#define CONFIG_BOOLEAN_NO 0 // disabled

161 -#define CONFIG_BOOLEAN_YES 1 // enabled

135 +int appconfig_load(struct config *root, char *filename, int overwrite_used, const char *section_name);

136

163 -#ifndef CONFIG_BOOLEAN_AUTO

164 -#define CONFIG_BOOLEAN_AUTO 2 // enabled if it has useful info when enabled

165 -#endif

137 +typedef bool (*appconfig_foreach_value_cb_t)(void *data, const char *name, const char *value);

138 +size_t appconfig_foreach_value_in_section(struct config *root, const char *section, appconfig_foreach_value_cb_t cb, void *data);

139

167 -int appconfig_load(struct config *root, char *filename, int overwrite_used, const char *section_name);

168 -void config_section_wrlock(struct section *co);

169 -void config_section_unlock(struct section *co);

170 -

171 -char *appconfig_get_by_section(struct section *co, const char *name, const char *default_value);

172 -char *appconfig_get(struct config *root, const char *section, const char *name, const char *default_value);

173 -long long appconfig_get_number(struct config *root, const char *section, const char *name, long long value);

174 -NETDATA_DOUBLE appconfig_get_float(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value);

175 -int appconfig_get_boolean_by_section(struct section *co, const char *name, int value);

176 -int appconfig_get_boolean(struct config *root, const char *section, const char *name, int value);

177 -int appconfig_get_boolean_ondemand(struct config *root, const char *section, const char *name, int value);

178 -int appconfig_get_duration(struct config *root, const char *section, const char *name, const char *value);

179 -

180 -const char *appconfig_set(struct config *root, const char *section, const char *name, const char *value);

181 -const char *appconfig_set_default(struct config *root, const char *section, const char *name, const char *value);

182 -long long appconfig_set_number(struct config *root, const char *section, const char *name, long long value);

183 -NETDATA_DOUBLE appconfig_set_float(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value);

184 -int appconfig_set_boolean(struct config *root, const char *section, const char *name, int value);

140 +// sets a raw value, only if it is not loaded from the config

141 +void appconfig_set_default_raw_value(struct config *root, const char *section, const char *name, const char *value);

142

143 int appconfig_exists(struct config *root, const char *section, const char *name);

144 int appconfig_move(struct config *root, const char *section_old, const char *name_old, const char *section_new, const char *name_new);

145 +int appconfig_move_everywhere(struct config *root, const char *name_old, const char *name_new);

146

147 void appconfig_generate(struct config *root, BUFFER *wb, int only_changed, bool netdata_conf);

148

149 int appconfig_section_compare(void *a, void *b);

150

193 -void appconfig_section_destroy_non_loaded(struct config *root, const char *section);

194 -void appconfig_section_option_destroy_non_loaded(struct config *root, const char *section, const char *name);

195 -

196 -int config_parse_duration(const char* string, int* result);

197 -

198 -struct section *appconfig_get_section(struct config *root, const char *name);

199 -

200 -void appconfig_wrlock(struct config *root);

201 -void appconfig_unlock(struct config *root);

202 -

203 -int appconfig_test_boolean_value(char *s);

151 +bool appconfig_test_boolean_value(const char *s);

152

153 struct connector_instance {

154 char instance_name[CONFIG_MAX_NAME + 1];

@@ -208,13 +156,37 @@ struct connector_instance {

156 };

157

158 typedef struct _connector_instance {

211 - struct section *connector; // actual connector

212 - struct section *instance; // This instance

159 + struct config_section *connector; // actual connector

160 + struct config_section *instance; // This instance

161 char instance_name[CONFIG_MAX_NAME + 1];

162 char connector_name[CONFIG_MAX_NAME + 1];

163 struct _connector_instance *next; // Next instance

164 } _CONNECTOR_INSTANCE;

165

218 -_CONNECTOR_INSTANCE *add_connector_instance(struct section *connector, struct section *instance);

166 +_CONNECTOR_INSTANCE *add_connector_instance(struct config_section *connector, struct config_section *instance);

167 +

168 +// ----------------------------------------------------------------------------

169 +// shortcuts for the default netdata configuration

170 +

171 +#define config_load(filename, overwrite_used, section) appconfig_load(&netdata_config, filename, overwrite_used, section)

172 +

173 +#define config_set_default_raw_value(section, name, value) appconfig_set_default_raw_value(&netdata_config, section, name, value)

174 +

175 +#define config_exists(section, name) appconfig_exists(&netdata_config, section, name)

176 +#define config_move(section_old, name_old, section_new, name_new) appconfig_move(&netdata_config, section_old, name_old, section_new, name_new)

177 +

178 +#define netdata_conf_generate(buffer, only_changed) appconfig_generate(&netdata_config, buffer, only_changed, true)

179 +

180 +#define config_section_destroy(section) appconfig_section_destroy_non_loaded(&netdata_config, section)

181 +#define config_section_option_destroy(section, name) appconfig_section_option_destroy_non_loaded(&netdata_config, section, name)

182 +

183 +bool stream_conf_needs_dbengine(struct config *root);

184 +bool stream_conf_has_uuid_section(struct config *root);

185 +

186 +#include "appconfig_api_text.h"

187 +#include "appconfig_api_numbers.h"

188 +#include "appconfig_api_boolean.h"

189 +#include "appconfig_api_sizes.h"

190 +#include "appconfig_api_durations.h"

191

220 -#endif /* NETDATA_CONFIG_H */

\ No newline at end of file

192 +#endif // NETDATA_CONFIG_H

src/libnetdata/config/appconfig_api_boolean.c new

+68

@@ -0,0 +1,68 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +#include "appconfig_api_boolean.h"

5 +

6 +bool appconfig_test_boolean_value(const char *s) {

7 + if(!strcasecmp(s, "yes") || !strcasecmp(s, "true") || !strcasecmp(s, "on")

8 + || !strcasecmp(s, "auto") || !strcasecmp(s, "on demand"))

9 + return true;

10 +

11 + return false;

12 +}

13 +

14 +int appconfig_get_boolean_by_section(struct config_section *sect, const char *name, int value) {

15 + struct config_option *opt = appconfig_get_raw_value_of_option_in_section(

16 + sect, name, (!value) ? "no" : "yes", CONFIG_VALUE_TYPE_BOOLEAN, NULL);

17 + if(!opt) return value;

18 +

19 + return appconfig_test_boolean_value(string2str(opt->value));

20 +}

21 +

22 +int appconfig_get_boolean(struct config *root, const char *section, const char *name, int value) {

23 + const char *s;

24 + if(value) s = "yes";

25 + else s = "no";

26 +

27 + struct config_option *opt = appconfig_get_raw_value(root, section, name, s, CONFIG_VALUE_TYPE_BOOLEAN, NULL);

28 + if(!opt) return value;

29 + s = string2str(opt->value);

30 +

31 + return appconfig_test_boolean_value(s);

32 +}

33 +

34 +int appconfig_get_boolean_ondemand(struct config *root, const char *section, const char *name, int value) {

35 + const char *s;

36 +

37 + if(value == CONFIG_BOOLEAN_AUTO)

38 + s = "auto";

39 +

40 + else if(value == CONFIG_BOOLEAN_NO)

41 + s = "no";

42 +

43 + else

44 + s = "yes";

45 +

46 + struct config_option *opt = appconfig_get_raw_value(root, section, name, s, CONFIG_VALUE_TYPE_BOOLEAN_ONDEMAND, NULL);

47 + if(!opt) return value;

48 +

49 + s = string2str(opt->value);

50 + if(!strcmp(s, "yes") || !strcmp(s, "true") || !strcmp(s, "on"))

51 + return CONFIG_BOOLEAN_YES;

52 + else if(!strcmp(s, "no") || !strcmp(s, "false") || !strcmp(s, "off"))

53 + return CONFIG_BOOLEAN_NO;

54 + else if(!strcmp(s, "auto") || !strcmp(s, "on demand"))

55 + return CONFIG_BOOLEAN_AUTO;

56 +

57 + return value;

58 +}

59 +

60 +int appconfig_set_boolean(struct config *root, const char *section, const char *name, int value) {

61 + const char *s;

62 + if(value) s = "yes";

63 + else s = "no";

64 +

65 + appconfig_set_raw_value(root, section, name, s, CONFIG_VALUE_TYPE_BOOLEAN);

66 +

67 + return value;

68 +}

src/libnetdata/config/appconfig_api_boolean.h new

+24

@@ -0,0 +1,24 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_API_BOOLEAN_H

4 +#define NETDATA_APPCONFIG_API_BOOLEAN_H

5 +

6 +#define CONFIG_BOOLEAN_INVALID 100 // an invalid value to check for validity (used as default initialization when needed)

7 +

8 +#define CONFIG_BOOLEAN_NO 0 // disabled

9 +#define CONFIG_BOOLEAN_YES 1 // enabled

10 +

11 +#ifndef CONFIG_BOOLEAN_AUTO

12 +#define CONFIG_BOOLEAN_AUTO 2 // enabled if it has useful info when enabled

13 +#endif

14 +

15 +int appconfig_get_boolean(struct config *root, const char *section, const char *name, int value);

16 +#define config_get_boolean(section, name, value) appconfig_get_boolean(&netdata_config, section, name, value)

17 +

18 +int appconfig_get_boolean_ondemand(struct config *root, const char *section, const char *name, int value);

19 +#define config_get_boolean_ondemand(section, name, value) appconfig_get_boolean_ondemand(&netdata_config, section, name, value)

20 +

21 +int appconfig_set_boolean(struct config *root, const char *section, const char *name, int value);

22 +#define config_set_boolean(section, name, value) appconfig_set_boolean(&netdata_config, section, name, value)

23 +

24 +#endif //NETDATA_APPCONFIG_API_BOOLEAN_H

src/libnetdata/config/appconfig_api_durations.c new

+134

@@ -0,0 +1,134 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +#include "appconfig_api_durations.h"

5 +

6 +

7 +static STRING *reformat_duration_seconds(STRING *value) {

8 + int result = 0;

9 + if(!duration_parse_seconds(string2str(value), &result))

10 + return value;

11 +

12 + char buf[128];

13 + if(duration_snprintf_time_t(buf, sizeof(buf), result) > 0 && string_strcmp(value, buf) != 0) {

14 + string_freez(value);

15 + return string_strdupz(buf);

16 + }

17 +

18 + return value;

19 +}

20 +

21 +time_t appconfig_get_duration_seconds(struct config *root, const char *section, const char *name, time_t default_value) {

22 + char default_str[128];

23 + duration_snprintf_time_t(default_str, sizeof(default_str), default_value);

24 +

25 + struct config_option *opt = appconfig_get_raw_value(

26 + root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_SECS, reformat_duration_seconds);

27 + if(!opt)

28 + return default_value;

29 +

30 + const char *s = string2str(opt->value);

31 +

32 + int result = 0;

33 + if(!duration_parse_seconds(s, &result)) {

34 + appconfig_set_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_SECS);

35 + netdata_log_error("config option '[%s].%s = %s' is configured with an invalid duration", section, name, s);

36 + return default_value;

37 + }

38 +

39 + return ABS(result);

40 +}

41 +

42 +time_t appconfig_set_duration_seconds(struct config *root, const char *section, const char *name, time_t value) {

43 + char str[128];

44 + duration_snprintf_time_t(str, sizeof(str), value);

45 +

46 + appconfig_set_raw_value(root, section, name, str, CONFIG_VALUE_TYPE_DURATION_IN_SECS);

47 + return value;

48 +}

49 +

50 +static STRING *reformat_duration_ms(STRING *value) {

51 + int64_t result = 0;

52 + if(!duration_parse_msec_t(string2str(value), &result))

53 + return value;

54 +

55 + char buf[128];

56 + if(duration_snprintf_msec_t(buf, sizeof(buf), result) > 0 && string_strcmp(value, buf) != 0) {

57 + string_freez(value);

58 + return string_strdupz(buf);

59 + }

60 +

61 + return value;

62 +}

63 +

64 +msec_t appconfig_get_duration_ms(struct config *root, const char *section, const char *name, msec_t default_value) {

65 + char default_str[128];

66 + duration_snprintf_msec_t(default_str, sizeof(default_str), default_value);

67 +

68 + struct config_option *opt = appconfig_get_raw_value(

69 + root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_MS, reformat_duration_ms);

70 + if(!opt)

71 + return default_value;

72 +

73 + const char *s = string2str(opt->value);

74 +

75 + smsec_t result = 0;

76 + if(!duration_parse_msec_t(s, &result)) {

77 + appconfig_set_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_MS);

78 + netdata_log_error("config option '[%s].%s = %s' is configured with an invalid duration", section, name, s);

79 + return default_value;

80 + }

81 +

82 + return ABS(result);

83 +}

84 +

85 +msec_t appconfig_set_duration_ms(struct config *root, const char *section, const char *name, msec_t value) {

86 + char str[128];

87 + duration_snprintf_msec_t(str, sizeof(str), (smsec_t)value);

88 +

89 + appconfig_set_raw_value(root, section, name, str, CONFIG_VALUE_TYPE_DURATION_IN_MS);

90 + return value;

91 +}

92 +

93 +static STRING *reformat_duration_days(STRING *value) {

94 + int64_t result = 0;

95 + if(!duration_parse_days(string2str(value), &result))

96 + return value;

97 +

98 + char buf[128];

99 + if(duration_snprintf_days(buf, sizeof(buf), result) > 0 && string_strcmp(value, buf) != 0) {

100 + string_freez(value);

101 + return string_strdupz(buf);

102 + }

103 +

104 + return value;

105 +}

106 +

107 +unsigned appconfig_get_duration_days(struct config *root, const char *section, const char *name, unsigned default_value) {

108 + char default_str[128];

109 + duration_snprintf_days(default_str, sizeof(default_str), (int)default_value);

110 +

111 + struct config_option *opt = appconfig_get_raw_value(

112 + root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_DAYS, reformat_duration_days);

113 + if(!opt)

114 + return default_value;

115 +

116 + const char *s = string2str(opt->value);

117 +

118 + int64_t result = 0;

119 + if(!duration_parse_days(s, &result)) {

120 + appconfig_set_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_DURATION_IN_DAYS);

121 + netdata_log_error("config option '[%s].%s = %s' is configured with an invalid duration", section, name, s);

122 + return default_value;

123 + }

124 +

125 + return (unsigned)ABS(result);

126 +}

127 +

128 +unsigned appconfig_set_duration_days(struct config *root, const char *section, const char *name, unsigned value) {

129 + char str[128];

130 + duration_snprintf_days(str, sizeof(str), value);

131 + appconfig_set_raw_value(root, section, name, str, CONFIG_VALUE_TYPE_DURATION_IN_DAYS);

132 + return value;

133 +}

134 +

src/libnetdata/config/appconfig_api_durations.h new

+21

@@ -0,0 +1,21 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_API_DURATIONS_H

4 +#define NETDATA_APPCONFIG_API_DURATIONS_H

5 +

6 +msec_t appconfig_get_duration_ms(struct config *root, const char *section, const char *name, msec_t default_value);

7 +msec_t appconfig_set_duration_ms(struct config *root, const char *section, const char *name, msec_t value);

8 +#define config_get_duration_ms(section, name, value) appconfig_get_duration_ms(&netdata_config, section, name, value)

9 +#define config_set_duration_ms(section, name, value) appconfig_set_duration_ms(&netdata_config, section, name, value)

10 +

11 +time_t appconfig_get_duration_seconds(struct config *root, const char *section, const char *name, time_t default_value);

12 +time_t appconfig_set_duration_seconds(struct config *root, const char *section, const char *name, time_t value);

13 +#define config_get_duration_seconds(section, name, value) appconfig_get_duration_seconds(&netdata_config, section, name, value)

14 +#define config_set_duration_seconds(section, name, value) appconfig_set_duration_seconds(&netdata_config, section, name, value)

15 +

16 +unsigned appconfig_get_duration_days(struct config *root, const char *section, const char *name, unsigned default_value);

17 +unsigned appconfig_set_duration_days(struct config *root, const char *section, const char *name, unsigned value);

18 +#define config_get_duration_days(section, name, value) appconfig_get_duration_days(&netdata_config, section, name, value)

19 +#define config_set_duration_days(section, name, value) appconfig_set_duration_days(&netdata_config, section, name, value)

20 +

21 +#endif //NETDATA_APPCONFIG_API_DURATIONS_H

src/libnetdata/config/appconfig_api_numbers.c new

+43

@@ -0,0 +1,43 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +#include "appconfig_api_numbers.h"

5 +

6 +long long appconfig_get_number(struct config *root, const char *section, const char *name, long long value) {

7 + char buffer[100];

8 + sprintf(buffer, "%lld", value);

9 +

10 + struct config_option *opt = appconfig_get_raw_value(root, section, name, buffer, CONFIG_VALUE_TYPE_INTEGER, NULL);

11 + if(!opt) return value;

12 +

13 + const char *s = string2str(opt->value);

14 + return strtoll(s, NULL, 0);

15 +}

16 +

17 +NETDATA_DOUBLE appconfig_get_double(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value) {

18 + char buffer[100];

19 + sprintf(buffer, "%0.5" NETDATA_DOUBLE_MODIFIER, value);

20 +

21 + struct config_option *opt = appconfig_get_raw_value(root, section, name, buffer, CONFIG_VALUE_TYPE_DOUBLE, NULL);

22 + if(!opt) return value;

23 +

24 + const char *s = string2str(opt->value);

25 + return str2ndd(s, NULL);

26 +}

27 +

28 +long long appconfig_set_number(struct config *root, const char *section, const char *name, long long value) {

29 + char buffer[100];

30 + sprintf(buffer, "%lld", value);

31 +

32 + appconfig_set_raw_value(root, section, name, buffer, CONFIG_VALUE_TYPE_INTEGER);

33 + return value;

34 +}

35 +

36 +NETDATA_DOUBLE appconfig_set_double(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value) {

37 + char buffer[100];

38 + sprintf(buffer, "%0.5" NETDATA_DOUBLE_MODIFIER, value);

39 +

40 + appconfig_set_raw_value(root, section, name, buffer, CONFIG_VALUE_TYPE_DOUBLE);

41 + return value;

42 +}

43 +

src/libnetdata/config/appconfig_api_numbers.h new

+16

@@ -0,0 +1,16 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_API_NUMBERS_H

4 +#define NETDATA_APPCONFIG_API_NUMBERS_H

5 +

6 +long long appconfig_get_number(struct config *root, const char *section, const char *name, long long value);

7 +long long appconfig_set_number(struct config *root, const char *section, const char *name, long long value);

8 +#define config_get_number(section, name, value) appconfig_get_number(&netdata_config, section, name, value)

9 +#define config_set_number(section, name, value) appconfig_set_number(&netdata_config, section, name, value)

10 +

11 +NETDATA_DOUBLE appconfig_get_double(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value);

12 +NETDATA_DOUBLE appconfig_set_double(struct config *root, const char *section, const char *name, NETDATA_DOUBLE value);

13 +#define config_get_double(section, name, value) appconfig_get_double(&netdata_config, section, name, value)

14 +#define config_set_double(section, name, value) appconfig_set_float(&netdata_config, section, name, value)

15 +

16 +#endif //NETDATA_APPCONFIG_API_NUMBERS_H

src/libnetdata/config/appconfig_api_sizes.c new

+86

@@ -0,0 +1,86 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +#include "appconfig_api_sizes.h"

5 +

6 +static STRING *reformat_size_bytes(STRING *value) {

7 + uint64_t result = 0;

8 + if(!size_parse_bytes(string2str(value), &result))

9 + return value;

10 +

11 + char buf[128];

12 + if(size_snprintf_bytes(buf, sizeof(buf), result) > 0 && string_strcmp(value, buf) != 0) {

13 + string_freez(value);

14 + return string_strdupz(buf);

15 + }

16 +

17 + return value;

18 +}

19 +

20 +uint64_t appconfig_get_size_bytes(struct config *root, const char *section, const char *name, uint64_t default_value) {

21 + char default_str[128];

22 + size_snprintf_bytes(default_str, sizeof(default_str), (int)default_value);

23 +

24 + struct config_option *opt =

25 + appconfig_get_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_SIZE_IN_BYTES, reformat_size_bytes);

26 + if(!opt)

27 + return default_value;

28 +

29 + const char *s = string2str(opt->value);

30 + uint64_t result = 0;

31 + if(!size_parse_bytes(s, &result)) {

32 + appconfig_set_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_SIZE_IN_BYTES);

33 + netdata_log_error("config option '[%s].%s = %s' is configured with an invalid size", section, name, s);

34 + return default_value;

35 + }

36 +

37 + return result;

38 +}

39 +

40 +uint64_t appconfig_set_size_bytes(struct config *root, const char *section, const char *name, uint64_t value) {

41 + char str[128];

42 + size_snprintf_bytes(str, sizeof(str), value);

43 + appconfig_set_raw_value(root, section, name, str, CONFIG_VALUE_TYPE_SIZE_IN_BYTES);

44 + return value;

45 +}

46 +

47 +static STRING *reformat_size_mb(STRING *value) {

48 + uint64_t result = 0;

49 + if(!size_parse_mb(string2str(value), &result))

50 + return value;

51 +

52 + char buf[128];

53 + if(size_snprintf_mb(buf, sizeof(buf), result) > 0 && string_strcmp(value, buf) != 0) {

54 + string_freez(value);

55 + return string_strdupz(buf);

56 + }

57 +

58 + return value;

59 +}

60 +

61 +uint64_t appconfig_get_size_mb(struct config *root, const char *section, const char *name, uint64_t default_value) {

62 + char default_str[128];

63 + size_snprintf_mb(default_str, sizeof(default_str), (int)default_value);

64 +

65 + struct config_option *opt =

66 + appconfig_get_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_SIZE_IN_MB, reformat_size_mb);

67 + if(!opt)

68 + return default_value;

69 +

70 + const char *s = string2str(opt->value);

71 + uint64_t result = 0;

72 + if(!size_parse_mb(s, &result)) {

73 + appconfig_set_raw_value(root, section, name, default_str, CONFIG_VALUE_TYPE_SIZE_IN_MB);

74 + netdata_log_error("config option '[%s].%s = %s' is configured with an invalid size", section, name, s);

75 + return default_value;

76 + }

77 +

78 + return (unsigned)result;

79 +}

80 +

81 +uint64_t appconfig_set_size_mb(struct config *root, const char *section, const char *name, uint64_t value) {

82 + char str[128];

83 + size_snprintf_mb(str, sizeof(str), value);

84 + appconfig_set_raw_value(root, section, name, str, CONFIG_VALUE_TYPE_SIZE_IN_MB);

85 + return value;

86 +}

src/libnetdata/config/appconfig_api_sizes.h new

+16

@@ -0,0 +1,16 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_API_SIZES_H

4 +#define NETDATA_APPCONFIG_API_SIZES_H

5 +

6 +uint64_t appconfig_get_size_bytes(struct config *root, const char *section, const char *name, uint64_t default_value);

7 +uint64_t appconfig_set_size_bytes(struct config *root, const char *section, const char *name, uint64_t value);

8 +#define config_get_size_bytes(section, name, value) appconfig_get_size_bytes(&netdata_config, section, name, value)

9 +#define config_set_size_bytes(section, name, value) appconfig_set_size_bytes(&netdata_config, section, name, value)

10 +

11 +uint64_t appconfig_get_size_mb(struct config *root, const char *section, const char *name, uint64_t default_value);

12 +uint64_t appconfig_set_size_mb(struct config *root, const char *section, const char *name, uint64_t value);

13 +#define config_get_size_mb(section, name, value) appconfig_get_size_mb(&netdata_config, section, name, value)

14 +#define config_set_size_mb(section, name, value) appconfig_set_size_mb(&netdata_config, section, name, value)

15 +

16 +#endif //NETDATA_APPCONFIG_API_SIZES_H

src/libnetdata/config/appconfig_api_text.c new

+17

@@ -0,0 +1,17 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +#include "appconfig_api_text.h"

5 +

6 +const char *appconfig_get(struct config *root, const char *section, const char *name, const char *default_value) {

7 + struct config_option *opt = appconfig_get_raw_value(root, section, name, default_value, CONFIG_VALUE_TYPE_TEXT, NULL);

8 + if(!opt)

9 + return default_value;

10 +

11 + return string2str(opt->value);

12 +}

13 +

14 +const char *appconfig_set(struct config *root, const char *section, const char *name, const char *value) {

15 + struct config_option *opt = appconfig_set_raw_value(root, section, name, value, CONFIG_VALUE_TYPE_TEXT);

16 + return string2str(opt->value);

17 +}

src/libnetdata/config/appconfig_api_text.h new

+12

@@ -0,0 +1,12 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_API_TEXT_H

4 +#define NETDATA_APPCONFIG_API_TEXT_H

5 +

6 +const char *appconfig_get(struct config *root, const char *section, const char *name, const char *default_value);

7 +const char *appconfig_set(struct config *root, const char *section, const char *name, const char *value);

8 +#define config_get(section, name, default_value) appconfig_get(&netdata_config, section, name, default_value)

9 +#define config_set(section, name, default_value) appconfig_set(&netdata_config, section, name, default_value)

10 +

11 +

12 +#endif //NETDATA_APPCONFIG_API_TEXT_H

src/libnetdata/config/appconfig_cleanup.c new

+60

@@ -0,0 +1,60 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +void appconfig_section_destroy_non_loaded(struct config *root, const char *section)

6 +{

7 + struct config_section *sect;

8 + struct config_option *opt;

9 +

10 + netdata_log_debug(D_CONFIG, "Destroying section '%s'.", section);

11 +

12 + sect = appconfig_section_find(root, section);

13 + if(!sect) {

14 + netdata_log_error("Could not destroy section '%s'. Not found.", section);

15 + return;

16 + }

17 +

18 + SECTION_LOCK(sect);

19 +

20 + // find if there is any loaded option

21 + for(opt = sect->values; opt; opt = opt->next) {

22 + if (opt->flags & CONFIG_VALUE_LOADED) {

23 + // do not destroy values that were loaded from the configuration files.

24 + SECTION_UNLOCK(sect);

25 + return;

26 + }

27 + }

28 +

29 + // no option is loaded, free them all

30 + appconfig_section_remove_and_delete(root, sect, false, true);

31 +}

32 +

33 +void appconfig_section_option_destroy_non_loaded(struct config *root, const char *section, const char *name) {

34 + struct config_section *sect;

35 + sect = appconfig_section_find(root, section);

36 + if (!sect) {

37 + netdata_log_error("Could not destroy section option '%s -> %s'. The section not found.", section, name);

38 + return;

39 + }

40 +

41 + SECTION_LOCK(sect);

42 +

43 + struct config_option *opt = appconfig_option_find(sect, name);

44 + if (opt && opt->flags & CONFIG_VALUE_LOADED) {

45 + SECTION_UNLOCK(sect);

46 + return;

47 + }

48 +

49 + if (unlikely(!(opt && appconfig_option_del(sect, opt)))) {

50 + SECTION_UNLOCK(sect);

51 + netdata_log_error("Could not destroy section option '%s -> %s'. The option not found.", section, name);

52 + return;

53 + }

54 +

55 + DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(sect->values, opt, prev, next);

56 +

57 + appconfig_option_free(opt);

58 + SECTION_UNLOCK(sect);

59 +}

60 +

src/libnetdata/config/appconfig_conf_file.c new

+320

@@ -0,0 +1,320 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +ENUM_STR_MAP_DEFINE(CONFIG_VALUE_TYPES) = {

6 + { .id = CONFIG_VALUE_TYPE_UNKNOWN, .name ="unknown", },

7 + { .id = CONFIG_VALUE_TYPE_TEXT, .name ="text", },

8 + { .id = CONFIG_VALUE_TYPE_HOSTNAME, .name ="hostname", },

9 + { .id = CONFIG_VALUE_TYPE_USERNAME, .name ="username", },

10 + { .id = CONFIG_VALUE_TYPE_FILENAME, .name ="filename", },

11 + { .id = CONFIG_VALUE_TYPE_PATH, .name ="path", },

12 + { .id = CONFIG_VALUE_TYPE_SIMPLE_PATTERN, .name ="simple pattern", },

13 + { .id = CONFIG_VALUE_TYPE_URL, .name ="URL", },

14 + { .id = CONFIG_VALUE_TYPE_ENUM, .name ="one of keywords", },

15 + { .id = CONFIG_VALUE_TYPE_BITMAP, .name ="any of keywords", },

16 + { .id = CONFIG_VALUE_TYPE_INTEGER, .name ="number (integer)", },

17 + { .id = CONFIG_VALUE_TYPE_DOUBLE, .name ="number (double)", },

18 + { .id = CONFIG_VALUE_TYPE_BOOLEAN, .name ="yes or no", },

19 + { .id = CONFIG_VALUE_TYPE_BOOLEAN_ONDEMAND, .name ="yes, no, or auto", },

20 + { .id = CONFIG_VALUE_TYPE_DURATION_IN_SECS, .name ="duration (seconds)", },

21 + { .id = CONFIG_VALUE_TYPE_DURATION_IN_MS, .name ="duration (ms)", },

22 + { .id = CONFIG_VALUE_TYPE_DURATION_IN_DAYS, .name ="duration (days)", },

23 + { .id = CONFIG_VALUE_TYPE_SIZE_IN_BYTES, .name ="size (bytes)", },

24 + { .id = CONFIG_VALUE_TYPE_SIZE_IN_MB, .name ="size (MiB)", },

25 +};

26 +

27 +ENUM_STR_DEFINE_FUNCTIONS(CONFIG_VALUE_TYPES, CONFIG_VALUE_TYPE_UNKNOWN, "unknown");

28 +

29 +

30 +// ----------------------------------------------------------------------------

31 +// config load/save

32 +

33 +int appconfig_load(struct config *root, char *filename, int overwrite_used, const char *section_name) {

34 + int line = 0;

35 + struct config_section *sect = NULL;

36 + int is_exporter_config = 0;

37 + int _connectors = 0; // number of exporting connector sections we have

38 + char working_instance[CONFIG_MAX_NAME + 1];

39 + char working_connector[CONFIG_MAX_NAME + 1];

40 + struct config_section *working_connector_section = NULL;

41 + int global_exporting_section = 0;

42 +

43 + char buffer[CONFIG_FILE_LINE_MAX + 1], *s;

44 +

45 + if(!filename) filename = CONFIG_DIR "/" CONFIG_FILENAME;

46 +

47 + netdata_log_debug(D_CONFIG, "CONFIG: opening config file '%s'", filename);

48 +

49 + FILE *fp = fopen(filename, "r");

50 + if(!fp) {

51 + if(errno != ENOENT)

52 + netdata_log_info("CONFIG: cannot open file '%s'. Using internal defaults.", filename);

53 +

54 + return 0;

55 + }

56 +

57 + CLEAN_STRING *section_string = string_strdupz(section_name);

58 + is_exporter_config = (strstr(filename, EXPORTING_CONF) != NULL);

59 +

60 + while(fgets(buffer, CONFIG_FILE_LINE_MAX, fp) != NULL) {

61 + buffer[CONFIG_FILE_LINE_MAX] = '\0';

62 + line++;

63 +

64 + s = trim(buffer);

65 + if(!s || *s == '#') {

66 + netdata_log_debug(D_CONFIG, "CONFIG: ignoring line %d of file '%s', it is empty.", line, filename);

67 + continue;

68 + }

69 +

70 + int len = (int) strlen(s);

71 + if(*s == '[' && s[len - 1] == ']') {

72 + // new section

73 + s[len - 1] = '\0';

74 + s++;

75 +

76 + if (is_exporter_config) {

77 + global_exporting_section = !(strcmp(s, CONFIG_SECTION_EXPORTING)) || !(strcmp(s, CONFIG_SECTION_PROMETHEUS));

78 +

79 + if (unlikely(!global_exporting_section)) {

80 + int rc;

81 + rc = is_valid_connector(s, 0);

82 + if (likely(rc)) {

83 + strncpyz(working_connector, s, CONFIG_MAX_NAME);

84 + s = s + rc + 1;

85 + if (unlikely(!(*s))) {

86 + _connectors++;

87 + sprintf(buffer, "instance_%d", _connectors);

88 + s = buffer;

89 + }

90 + strncpyz(working_instance, s, CONFIG_MAX_NAME);

91 + working_connector_section = NULL;

92 + if (unlikely(appconfig_section_find(root, working_instance))) {

93 + netdata_log_error("Instance (%s) already exists", working_instance);

94 + sect = NULL;

95 + continue;

96 + }

97 + }

98 + else {

99 + sect = NULL;

100 + netdata_log_error("Section (%s) does not specify a valid connector", s);

101 + continue;

102 + }

103 + }

104 + }

105 +

106 + sect = appconfig_section_find(root, s);

107 + if(!sect)

108 + sect = appconfig_section_create(root, s);

109 +

110 + if(sect && section_string && overwrite_used && section_string == sect->name) {

111 + SECTION_LOCK(sect);

112 +

113 + while(sect->values)

114 + appconfig_option_remove_and_delete(sect, sect->values, true);

115 +

116 + SECTION_UNLOCK(sect);

117 + }

118 +

119 + continue;

120 + }

121 +

122 + if(!sect) {

123 + // line outside a section

124 + netdata_log_error("CONFIG: ignoring line %d ('%s') of file '%s', it is outside all sections.", line, s, filename);

125 + continue;

126 + }

127 +

128 + if(section_string && overwrite_used && section_string != sect->name)

129 + continue;

130 +

131 + char *name = s;

132 + char *value = strchr(s, '=');

133 + if(!value) {

134 + netdata_log_error("CONFIG: ignoring line %d ('%s') of file '%s', there is no = in it.", line, s, filename);

135 + continue;

136 + }

137 + *value = '\0';

138 + value++;

139 +

140 + name = trim(name);

141 + value = trim(value);

142 +

143 + if(!name || *name == '#') {

144 + netdata_log_error("CONFIG: ignoring line %d of file '%s', name is empty.", line, filename);

145 + continue;

146 + }

147 +

148 + if(!value) value = "";

149 +

150 + struct config_option *opt = appconfig_option_find(sect, name);

151 +

152 + if (!opt) {

153 + opt = appconfig_option_create(sect, name, value);

154 + if (likely(is_exporter_config) && unlikely(!global_exporting_section)) {

155 + if (unlikely(!working_connector_section)) {

156 + working_connector_section = appconfig_section_find(root, working_connector);

157 + if (!working_connector_section)

158 + working_connector_section = appconfig_section_create(root, working_connector);

159 + if (likely(working_connector_section)) {

160 + add_connector_instance(working_connector_section, sect);

161 + }

162 + }

163 + }

164 + }

165 + else {

166 + if (((opt->flags & CONFIG_VALUE_USED) && overwrite_used) || !(opt->flags & CONFIG_VALUE_USED)) {

167 + string_freez(opt->value);

168 + opt->value = string_strdupz(value);

169 + }

170 + }

171 + opt->flags |= CONFIG_VALUE_LOADED;

172 + }

173 +

174 + fclose(fp);

175 +

176 + return 1;

177 +}

178 +

179 +void appconfig_generate(struct config *root, BUFFER *wb, int only_changed, bool netdata_conf)

180 +{

181 + int i, pri;

182 + struct config_section *sect;

183 + struct config_option *opt;

184 +

185 + {

186 + int found_host_labels = 0;

187 + for (sect = root->sections; sect; sect = sect->next)

188 + if(!string_strcmp(sect->name, CONFIG_SECTION_HOST_LABEL))

189 + found_host_labels = 1;

190 +

191 + if(netdata_conf && !found_host_labels) {

192 + appconfig_section_create(root, CONFIG_SECTION_HOST_LABEL);

193 + appconfig_get_raw_value(root, CONFIG_SECTION_HOST_LABEL, "name", "value", CONFIG_VALUE_TYPE_TEXT, NULL);

194 + }

195 + }

196 +

197 + if(netdata_conf) {

198 + buffer_strcat(wb,

199 + "# netdata configuration\n"

200 + "#\n"

201 + "# You can download the latest version of this file, using:\n"

202 + "#\n"

203 + "# wget -O /etc/netdata/netdata.conf http://localhost:19999/netdata.conf\n"

204 + "# or\n"

205 + "# curl -o /etc/netdata/netdata.conf http://localhost:19999/netdata.conf\n"

206 + "#\n"

207 + "# You can uncomment and change any of the options below.\n"

208 + "# The value shown in the commented settings, is the default value.\n"

209 + "#\n"

210 + "\n# global netdata configuration\n");

211 + }

212 +

213 + for(i = 0; i <= 17 ;i++) {

214 + APPCONFIG_LOCK(root);

215 + for(sect = root->sections; sect; sect = sect->next) {

216 + if(!string_strcmp(sect->name, CONFIG_SECTION_GLOBAL)) pri = 0;

217 + else if(!string_strcmp(sect->name, CONFIG_SECTION_DB)) pri = 1;

218 + else if(!string_strcmp(sect->name, CONFIG_SECTION_DIRECTORIES)) pri = 2;

219 + else if(!string_strcmp(sect->name, CONFIG_SECTION_LOGS)) pri = 3;

220 + else if(!string_strcmp(sect->name, CONFIG_SECTION_ENV_VARS)) pri = 4;

221 + else if(!string_strcmp(sect->name, CONFIG_SECTION_HOST_LABEL)) pri = 5;

222 + else if(!string_strcmp(sect->name, CONFIG_SECTION_SQLITE)) pri = 6;

223 + else if(!string_strcmp(sect->name, CONFIG_SECTION_CLOUD)) pri = 7;

224 + else if(!string_strcmp(sect->name, CONFIG_SECTION_ML)) pri = 8;

225 + else if(!string_strcmp(sect->name, CONFIG_SECTION_HEALTH)) pri = 9;

226 + else if(!string_strcmp(sect->name, CONFIG_SECTION_WEB)) pri = 10;

227 + else if(!string_strcmp(sect->name, CONFIG_SECTION_WEBRTC)) pri = 11;

228 + // by default, new sections will get pri = 12 (set at the end, below)

229 + else if(!string_strcmp(sect->name, CONFIG_SECTION_REGISTRY)) pri = 13;

230 + else if(!string_strcmp(sect->name, CONFIG_SECTION_GLOBAL_STATISTICS)) pri = 14;

231 + else if(!string_strcmp(sect->name, CONFIG_SECTION_PLUGINS)) pri = 15;

232 + else if(!string_strcmp(sect->name, CONFIG_SECTION_STATSD)) pri = 16;

233 + else if(!string_strncmp(sect->name, "plugin:", 7)) pri = 17; // << change the loop too if you change this

234 + else pri = 12; // this is used for any new (currently unknown) sections

235 +

236 + if(i == pri) {

237 + int loaded = 0;

238 + int used = 0;

239 + int changed = 0;

240 + int count = 0;

241 +

242 + SECTION_LOCK(sect);

243 + for(opt = sect->values; opt; opt = opt->next) {

244 + used += (opt->flags & CONFIG_VALUE_USED)?1:0;

245 + loaded += (opt->flags & CONFIG_VALUE_LOADED)?1:0;

246 + changed += (opt->flags & CONFIG_VALUE_CHANGED)?1:0;

247 + count++;

248 + }

249 + SECTION_UNLOCK(sect);

250 +

251 + if(!count) continue;

252 + if(only_changed && !changed && !loaded) continue;

253 +

254 + if(!used)

255 + buffer_sprintf(wb, "\n# section '%s' is not used.", string2str(sect->name));

256 +

257 + buffer_sprintf(wb, "\n[%s]\n", string2str(sect->name));

258 +

259 + size_t options_added = 0;

260 + bool last_had_comments = false;

261 + SECTION_LOCK(sect);

262 + for(opt = sect->values; opt; opt = opt->next) {

263 + bool unused = used && !(opt->flags & CONFIG_VALUE_USED);

264 + bool migrated = used && (opt->flags & CONFIG_VALUE_MIGRATED);

265 + bool reformatted = used && (opt->flags & CONFIG_VALUE_REFORMATTED);

266 + bool show_default = used && (opt->flags & (CONFIG_VALUE_LOADED|CONFIG_VALUE_CHANGED) && opt->value_default);

267 +

268 + if((unused || migrated || reformatted || show_default)) {

269 + if(options_added)

270 + buffer_strcat(wb, "\n");

271 +

272 + buffer_sprintf(wb, "\t#| >>> [%s].%s <<<\n",

273 + string2str(sect->name), string2str(opt->name));

274 +

275 + last_had_comments = true;

276 + }

277 + else if(last_had_comments) {

278 + buffer_strcat(wb, "\n");

279 + last_had_comments = false;

280 + }

281 +

282 + if(unused)

283 + buffer_sprintf(wb, "\t#| found in the config file, but is not used\n");

284 +

285 + if(migrated && reformatted)

286 + buffer_sprintf(wb, "\t#| migrated from: [%s].%s = %s\n",

287 + string2str(opt->migrated.section), string2str(opt->migrated.name),

288 + string2str(opt->value_original));

289 + else {

290 + if (migrated)

291 + buffer_sprintf(wb, "\t#| migrated from: [%s].%s\n",

292 + string2str(opt->migrated.section), string2str(opt->migrated.name));

293 +

294 + if (reformatted)

295 + buffer_sprintf(wb, "\t#| reformatted from: %s\n",

296 + string2str(opt->value_original));

297 + }

298 +

299 + if(show_default)

300 + buffer_sprintf(wb, "\t#| datatype: %s, default value: %s\n",

301 + CONFIG_VALUE_TYPES_2str(opt->type),

302 + string2str(opt->value_default));

303 +

304 + buffer_sprintf(wb, "\t%s%s = %s\n",

305 + (

306 + !(opt->flags & CONFIG_VALUE_LOADED) &&

307 + !(opt->flags & CONFIG_VALUE_CHANGED) &&

308 + (opt->flags & CONFIG_VALUE_USED)

309 + ) ? "# " : "",

310 + string2str(opt->name),

311 + string2str(opt->value));

312 +

313 + options_added++;

314 + }

315 + SECTION_UNLOCK(sect);

316 + }

317 + }

318 + APPCONFIG_UNLOCK(root);

319 + }

320 +}

src/libnetdata/config/appconfig_exporters.c new

+95

@@ -0,0 +1,95 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +/*

6 + * @Input:

7 + * Connector / instance to add to an internal structure

8 + * @Return

9 + * The current head of the linked list of connector_instance

10 + *

11 + */

12 +

13 +_CONNECTOR_INSTANCE *add_connector_instance(struct config_section *connector, struct config_section *instance)

14 +{

15 + static struct _connector_instance *global_connector_instance = NULL;

16 + struct _connector_instance *local_ci, *local_ci_tmp;

17 +

18 + if (unlikely(!connector)) {

19 + if (unlikely(!instance))

20 + return global_connector_instance;

21 +

22 + local_ci = global_connector_instance;

23 + while (local_ci) {

24 + local_ci_tmp = local_ci->next;

25 + freez(local_ci);

26 + local_ci = local_ci_tmp;

27 + }

28 + global_connector_instance = NULL;

29 + return NULL;

30 + }

31 +

32 + local_ci = callocz(1, sizeof(struct _connector_instance));

33 + local_ci->instance = instance;

34 + local_ci->connector = connector;

35 + strncpyz(local_ci->instance_name, string2str(instance->name), CONFIG_MAX_NAME);

36 + strncpyz(local_ci->connector_name, string2str(connector->name), CONFIG_MAX_NAME);

37 + local_ci->next = global_connector_instance;

38 + global_connector_instance = local_ci;

39 +

40 + return global_connector_instance;

41 +}

42 +

43 +int is_valid_connector(char *type, int check_reserved) {

44 + int rc = 1;

45 +

46 + if (unlikely(!type))

47 + return 0;

48 +

49 + if (!check_reserved) {

50 + if (unlikely(is_valid_connector(type,1))) {

51 + return 0;

52 + }

53 + //if (unlikely(*type == ':')

54 + // return 0;

55 + char *separator = strrchr(type, ':');

56 + if (likely(separator)) {

57 + *separator = '\0';

58 + rc = (int)(separator - type);

59 + } else

60 + return 0;

61 + }

62 + // else {

63 + // if (unlikely(is_valid_connector(type,1))) {

64 + // netdata_log_error("Section %s invalid -- reserved name", type);

65 + // return 0;

66 + // }

67 + // }

68 +

69 + if (!strcmp(type, "graphite") || !strcmp(type, "graphite:plaintext")) {

70 + return rc;

71 + } else if (!strcmp(type, "graphite:http") || !strcmp(type, "graphite:https")) {

72 + return rc;

73 + } else if (!strcmp(type, "json") || !strcmp(type, "json:plaintext")) {

74 + return rc;

75 + } else if (!strcmp(type, "json:http") || !strcmp(type, "json:https")) {

76 + return rc;

77 + } else if (!strcmp(type, "opentsdb") || !strcmp(type, "opentsdb:telnet")) {

78 + return rc;

79 + } else if (!strcmp(type, "opentsdb:http") || !strcmp(type, "opentsdb:https")) {

80 + return rc;

81 + } else if (!strcmp(type, "prometheus_remote_write")) {

82 + return rc;

83 + } else if (!strcmp(type, "prometheus_remote_write:http") || !strcmp(type, "prometheus_remote_write:https")) {

84 + return rc;

85 + } else if (!strcmp(type, "kinesis") || !strcmp(type, "kinesis:plaintext")) {

86 + return rc;

87 + } else if (!strcmp(type, "pubsub") || !strcmp(type, "pubsub:plaintext")) {

88 + return rc;

89 + } else if (!strcmp(type, "mongodb") || !strcmp(type, "mongodb:plaintext")) {

90 + return rc;

91 + }

92 +

93 + return 0;

94 +}

95 +

src/libnetdata/config/appconfig_internals.h new

+119

@@ -0,0 +1,119 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_APPCONFIG_INTERNALS_H

4 +#define NETDATA_APPCONFIG_INTERNALS_H

5 +

6 +#include "appconfig.h"

7 +

8 +typedef enum __attribute__((packed)) {

9 + CONFIG_VALUE_TYPE_UNKNOWN = 0,

10 + CONFIG_VALUE_TYPE_TEXT,

11 + CONFIG_VALUE_TYPE_HOSTNAME,

12 + CONFIG_VALUE_TYPE_USERNAME,

13 + CONFIG_VALUE_TYPE_FILENAME,

14 + CONFIG_VALUE_TYPE_PATH,

15 + CONFIG_VALUE_TYPE_SIMPLE_PATTERN,

16 + CONFIG_VALUE_TYPE_URL,

17 + CONFIG_VALUE_TYPE_ENUM,

18 + CONFIG_VALUE_TYPE_BITMAP,

19 + CONFIG_VALUE_TYPE_INTEGER,

20 + CONFIG_VALUE_TYPE_DOUBLE,

21 + CONFIG_VALUE_TYPE_BOOLEAN,

22 + CONFIG_VALUE_TYPE_BOOLEAN_ONDEMAND,

23 + CONFIG_VALUE_TYPE_DURATION_IN_SECS,

24 + CONFIG_VALUE_TYPE_DURATION_IN_MS,

25 + CONFIG_VALUE_TYPE_DURATION_IN_DAYS,

26 + CONFIG_VALUE_TYPE_SIZE_IN_BYTES,

27 + CONFIG_VALUE_TYPE_SIZE_IN_MB,

28 +} CONFIG_VALUE_TYPES;

29 +

30 +typedef enum __attribute__((packed)) {

31 + CONFIG_VALUE_LOADED = (1 << 0), // has been loaded from the config

32 + CONFIG_VALUE_USED = (1 << 1), // has been accessed from the program

33 + CONFIG_VALUE_CHANGED = (1 << 2), // has been changed from the loaded value or the internal default value

34 + CONFIG_VALUE_CHECKED = (1 << 3), // has been checked if the value is different from the default

35 + CONFIG_VALUE_MIGRATED = (1 << 4), // has been migrated from an old config

36 + CONFIG_VALUE_REFORMATTED = (1 << 5), // has been reformatted with the official formatting

37 +} CONFIG_VALUE_FLAGS;

38 +

39 +struct config_option {

40 + avl_t avl_node; // the index entry of this entry - this has to be first!

41 +

42 + CONFIG_VALUE_TYPES type;

43 + CONFIG_VALUE_FLAGS flags;

44 +

45 + STRING *name;

46 + STRING *value;

47 +

48 + STRING *value_original; // the original value of this option (the first value it got, independently on how it got it)

49 + STRING *value_default; // the internal default value of this option (the first value it got, from appconfig_get_XXX())

50 +

51 + // when we move options around, this is where we keep the original

52 + // section and name (of the first migration)

53 + struct {

54 + STRING *section;

55 + STRING *name;

56 + } migrated;

57 +

58 + struct config_option *prev, *next; // config->mutex protects just this

59 +};

60 +

61 +struct config_section {

62 + avl_t avl_node; // the index entry of this section - this has to be first!

63 +

64 + STRING *name;

65 +

66 + struct config_option *values;

67 + avl_tree_lock values_index;

68 +

69 + SPINLOCK spinlock;

70 + struct config_section *prev, *next; // global config_mutex protects just this

71 +};

72 +

73 +// ----------------------------------------------------------------------------

74 +// locking

75 +

76 +#define APPCONFIG_LOCK(root) spinlock_lock(&((root)->spinlock))

77 +#define APPCONFIG_UNLOCK(root) spinlock_unlock(&((root)->spinlock))

78 +#define SECTION_LOCK(sect) spinlock_lock(&((sect)->spinlock))

79 +#define SECTION_UNLOCK(sect) spinlock_unlock(&((sect)->spinlock));

80 +

81 +// config sections

82 +void appconfig_section_free(struct config_section *sect);

83 +void appconfig_section_remove_and_delete(struct config *root, struct config_section *sect, bool have_root_lock, bool have_sect_lock);

84 +#define appconfig_section_add(root, cfg) (struct config_section *)avl_insert_lock(&(root)->index, (avl_t *)(cfg))

85 +#define appconfig_section_del(root, cfg) (struct config_section *)avl_remove_lock(&(root)->index, (avl_t *)(cfg))

86 +struct config_section *appconfig_section_find(struct config *root, const char *name);

87 +struct config_section *appconfig_section_create(struct config *root, const char *section);

88 +

89 +// config options

90 +void appconfig_option_cleanup(struct config_option *opt);

91 +void appconfig_option_free(struct config_option *opt);

92 +void appconfig_option_remove_and_delete(struct config_section *sect, struct config_option *opt, bool have_sect_lock);

93 +void appconfig_option_remove_and_delete_all(struct config_section *sect, bool have_sect_lock);

94 +int appconfig_option_compare(void *a, void *b);

95 +#define appconfig_option_add(co, cv) (struct config_option *)avl_insert_lock(&((co)->values_index), (avl_t *)(cv))

96 +#define appconfig_option_del(co, cv) (struct config_option *)avl_remove_lock(&((co)->values_index), (avl_t *)(cv))

97 +struct config_option *appconfig_option_find(struct config_section *sect, const char *name);

98 +struct config_option *appconfig_option_create(struct config_section *sect, const char *name, const char *value);

99 +

100 +// lookup

101 +int appconfig_get_boolean_by_section(struct config_section *sect, const char *name, int value);

102 +

103 +typedef STRING *(*reformat_t)(STRING *value);

104 +struct config_option *appconfig_get_raw_value_of_option_in_section(struct config_section *sect, const char *option, const char *default_value, CONFIG_VALUE_TYPES type, reformat_t cb);

105 +struct config_option *appconfig_get_raw_value(struct config *root, const char *section, const char *option, const char *default_value, CONFIG_VALUE_TYPES type, reformat_t cb);

106 +

107 +void appconfig_set_raw_value_of_option(struct config_option *opt, const char *value, CONFIG_VALUE_TYPES type);

108 +struct config_option *appconfig_set_raw_value_of_option_in_section(struct config_section *sect, const char *option, const char *value, CONFIG_VALUE_TYPES type);

109 +struct config_option *appconfig_set_raw_value(struct config *root, const char *section, const char *option, const char *value, CONFIG_VALUE_TYPES type);

110 +

111 +// cleanup

112 +void appconfig_section_destroy_non_loaded(struct config *root, const char *section);

113 +void appconfig_section_option_destroy_non_loaded(struct config *root, const char *section, const char *name);

114 +

115 +// exporters

116 +_CONNECTOR_INSTANCE *add_connector_instance(struct config_section *connector, struct config_section *instance);

117 +int is_valid_connector(char *type, int check_reserved);

118 +

119 +#endif //NETDATA_APPCONFIG_INTERNALS_H

src/libnetdata/config/appconfig_migrate.c new

+95

@@ -0,0 +1,95 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +int appconfig_move(struct config *root, const char *section_old, const char *name_old, const char *section_new, const char *name_new) {

6 + struct config_option *opt_old, *opt_new;

7 + int ret = -1;

8 +

9 + netdata_log_debug(D_CONFIG, "request to rename config in section '%s', old name '%s', to section '%s', new name '%s'", section_old, name_old, section_new, name_new);

10 +

11 + struct config_section *sect_old = appconfig_section_find(root, section_old);

12 + if(!sect_old) return ret;

13 +

14 + struct config_section *sect_new = appconfig_section_find(root, section_new);

15 + if(!sect_new) sect_new = appconfig_section_create(root, section_new);

16 +

17 + SECTION_LOCK(sect_old);

18 + if(sect_old != sect_new)

19 + SECTION_LOCK(sect_new);

20 +

21 + opt_old = appconfig_option_find(sect_old, name_old);

22 + if(!opt_old) goto cleanup;

23 +

24 + opt_new = appconfig_option_find(sect_new, name_new);

25 + if(opt_new) goto cleanup;

26 +

27 + if(unlikely(appconfig_option_del(sect_old, opt_old) != opt_old))

28 + netdata_log_error("INTERNAL ERROR: deletion of config '%s' from section '%s', deleted the wrong config entry.",

29 + string2str(opt_old->name), string2str(sect_old->name));

30 +

31 + // remember the old position of the item

32 + struct config_option *opt_old_next = (sect_old == sect_new) ? opt_old->next : NULL;

33 +

34 + DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(sect_old->values, opt_old, prev, next);

35 +

36 + nd_log(NDLS_DAEMON, NDLP_WARNING,

37 + "CONFIG: option '[%s].%s' has been migrated to '[%s].%s'.",

38 + section_old, name_old,

39 + section_new, name_new);

40 +

41 + if(!opt_old->migrated.name) {

42 + string_freez(opt_old->migrated.section);

43 + opt_old->migrated.section = string_dup(sect_old->name);

44 + opt_old->migrated.name = opt_old->name;

45 + }

46 + else

47 + string_freez(opt_old->name);

48 +

49 + opt_old->name = string_strdupz(name_new);

50 + opt_old->flags |= CONFIG_VALUE_MIGRATED;

51 +

52 + opt_new = opt_old;

53 +

54 + // put in the list, but try to keep the order

55 + if(opt_old_next && sect_old == sect_new)

56 + DOUBLE_LINKED_LIST_INSERT_ITEM_BEFORE_UNSAFE(sect_new->values, opt_old_next, opt_new, prev, next);

57 + else {

58 + // we don't have the old next item (probably a different section?)

59 + // find the last MIGRATED one

60 + struct config_option *t = sect_new->values->prev;

61 + for (; t != sect_new->values ; t = t->prev) {

62 + if (t->flags & CONFIG_VALUE_MIGRATED)

63 + break;

64 + }

65 + if (t == sect_new->values)

66 + DOUBLE_LINKED_LIST_PREPEND_ITEM_UNSAFE(sect_new->values, opt_new, prev, next);

67 + else

68 + DOUBLE_LINKED_LIST_INSERT_ITEM_AFTER_UNSAFE(sect_new->values, t, opt_new, prev, next);

69 + }

70 +

71 + if(unlikely(appconfig_option_add(sect_new, opt_old) != opt_old))

72 + netdata_log_error("INTERNAL ERROR: re-indexing of config '%s' in section '%s', already exists.",

73 + string2str(opt_old->name), string2str(sect_new->name));

74 +

75 + ret = 0;

76 +

77 +cleanup:

78 + if(sect_old != sect_new)

79 + SECTION_UNLOCK(sect_new);

80 + SECTION_UNLOCK(sect_old);

81 + return ret;

82 +}

83 +

84 +int appconfig_move_everywhere(struct config *root, const char *name_old, const char *name_new) {

85 + int ret = -1;

86 + APPCONFIG_LOCK(root);

87 + struct config_section *sect;

88 + for(sect = root->sections; sect; sect = sect->next) {

89 + if(appconfig_move(root, string2str(sect->name), name_old, string2str(sect->name), name_new) == 0)

90 + ret = 0;

91 + }

92 + APPCONFIG_UNLOCK(root);

93 + return ret;

94 +}

95 +

src/libnetdata/config/appconfig_options.c new

+183

@@ -0,0 +1,183 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +// ----------------------------------------------------------------------------

6 +// config options index

7 +

8 +int appconfig_option_compare(void *a, void *b) {

9 + if(((struct config_option *)a)->name < ((struct config_option *)b)->name) return -1;

10 + else if(((struct config_option *)a)->name > ((struct config_option *)b)->name) return 1;

11 + else return string_cmp(((struct config_option *)a)->name, ((struct config_option *)b)->name);

12 +}

13 +

14 +struct config_option *appconfig_option_find(struct config_section *sect, const char *name) {

15 + struct config_option opt_tmp = {

16 + .name = string_strdupz(name),

17 + };

18 +

19 + struct config_option *rc = (struct config_option *)avl_search_lock(&(sect->values_index), (avl_t *) &opt_tmp);

20 +

21 + appconfig_option_cleanup(&opt_tmp);

22 + return rc;

23 +}

24 +

25 +// ----------------------------------------------------------------------------

26 +// config options methods

27 +

28 +void appconfig_option_cleanup(struct config_option *opt) {

29 + string_freez(opt->value);

30 + string_freez(opt->name);

31 + string_freez(opt->migrated.section);

32 + string_freez(opt->migrated.name);

33 + string_freez(opt->value_original);

34 + string_freez(opt->value_default);

35 +

36 + opt->value = NULL;

37 + opt->name = NULL;

38 + opt->migrated.section = NULL;

39 + opt->migrated.name = NULL;

40 + opt->value_original = NULL;

41 + opt->value_default = NULL;

42 +}

43 +

44 +void appconfig_option_free(struct config_option *opt) {

45 + appconfig_option_cleanup(opt);

46 + freez(opt);

47 +}

48 +

49 +struct config_option *appconfig_option_create(struct config_section *sect, const char *name, const char *value) {

50 + struct config_option *opt = callocz(1, sizeof(struct config_option));

51 + opt->name = string_strdupz(name);

52 + opt->value = string_strdupz(value);

53 + opt->value_original = string_dup(opt->value);

54 +

55 + struct config_option *opt_found = appconfig_option_add(sect, opt);

56 + if(opt_found != opt) {

57 + nd_log(NDLS_DAEMON, NDLP_INFO,

58 + "CONFIG: config '%s' in section '%s': already exists - using the existing one.",

59 + string2str(opt->name), string2str(sect->name));

60 + appconfig_option_free(opt);

61 + return opt_found;

62 + }

63 +

64 + SECTION_LOCK(sect);

65 + DOUBLE_LINKED_LIST_APPEND_ITEM_UNSAFE(sect->values, opt, prev, next);

66 + SECTION_UNLOCK(sect);

67 +

68 + return opt;

69 +}

70 +

71 +void appconfig_option_remove_and_delete(struct config_section *sect, struct config_option *opt, bool have_sect_lock) {

72 + struct config_option *opt_found = appconfig_option_del(sect, opt);

73 + if(opt_found != opt) {

74 + nd_log(NDLS_DAEMON, NDLP_ERR,

75 + "INTERNAL ERROR: Cannot remove '%s' from section '%s', it was not inserted before.",

76 + string2str(opt->name), string2str(sect->name));

77 + return;

78 + }

79 +

80 + if(!have_sect_lock)

81 + SECTION_LOCK(sect);

82 +

83 + DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(sect->values, opt, prev, next);

84 +

85 + if(!have_sect_lock)

86 + SECTION_UNLOCK(sect);

87 +

88 + appconfig_option_free(opt);

89 +}

90 +

91 +void appconfig_option_remove_and_delete_all(struct config_section *sect, bool have_sect_lock) {

92 + if(!have_sect_lock)

93 + SECTION_LOCK(sect);

94 +

95 + while(sect->values)

96 + appconfig_option_remove_and_delete(sect, sect->values, true);

97 +

98 + if(!have_sect_lock)

99 + SECTION_UNLOCK(sect);

100 +}

101 +

102 +void appconfig_get_raw_value_of_option(struct config_option *opt, const char *default_value, CONFIG_VALUE_TYPES type, reformat_t cb) {

103 + opt->flags |= CONFIG_VALUE_USED;

104 +

105 + if(type != CONFIG_VALUE_TYPE_UNKNOWN)

106 + opt->type = type;

107 +

108 + if((opt->flags & CONFIG_VALUE_LOADED) || (opt->flags & CONFIG_VALUE_CHANGED)) {

109 + // this is a loaded value from the config file

110 + // if it is different from the default, mark it

111 + if(!(opt->flags & CONFIG_VALUE_CHECKED)) {

112 + if(!(opt->flags & CONFIG_VALUE_REFORMATTED) && cb) {

113 + STRING *value_old = opt->value;

114 + opt->value = cb(opt->value);

115 + if(opt->value != value_old)

116 + opt->flags |= CONFIG_VALUE_REFORMATTED;

117 + }

118 +

119 + if(default_value && string_strcmp(opt->value, default_value) != 0)

120 + opt->flags |= CONFIG_VALUE_CHANGED;

121 +

122 + opt->flags |= CONFIG_VALUE_CHECKED;

123 + }

124 + }

125 +

126 + if(!opt->value_default)

127 + opt->value_default = string_strdupz(default_value);

128 +}

129 +

130 +struct config_option *appconfig_get_raw_value_of_option_in_section(struct config_section *sect, const char *option, const char *default_value, CONFIG_VALUE_TYPES type, reformat_t cb) {

131 + // Only calls internal to this file check for a NULL result, and they do not supply a NULL arg.

132 + // External caller should treat NULL as an error case.

133 + struct config_option *opt = appconfig_option_find(sect, option);

134 + if (!opt) {

135 + if (!default_value) return NULL;

136 + opt = appconfig_option_create(sect, option, default_value);

137 + if (!opt) return NULL;

138 + }

139 +

140 + appconfig_get_raw_value_of_option(opt, default_value, type, cb);

141 + return opt;

142 +}

143 +

144 +struct config_option *appconfig_get_raw_value(struct config *root, const char *section, const char *option, const char *default_value, CONFIG_VALUE_TYPES type, reformat_t cb) {

145 + struct config_section *sect = appconfig_section_find(root, section);

146 + if(!sect) {

147 + if(!default_value) return NULL;

148 + sect = appconfig_section_create(root, section);

149 + }

150 +

151 + return appconfig_get_raw_value_of_option_in_section(sect, option, default_value, type, cb);

152 +}

153 +

154 +void appconfig_set_raw_value_of_option(struct config_option *opt, const char *value, CONFIG_VALUE_TYPES type) {

155 + opt->flags |= CONFIG_VALUE_USED;

156 +

157 + if(opt->type == CONFIG_VALUE_TYPE_UNKNOWN)

158 + opt->type = type;

159 +

160 + if(string_strcmp(opt->value, value) != 0) {

161 + opt->flags |= CONFIG_VALUE_CHANGED;

162 +

163 + string_freez(opt->value);

164 + opt->value = string_strdupz(value);

165 + }

166 +}

167 +

168 +struct config_option *appconfig_set_raw_value_of_option_in_section(struct config_section *sect, const char *option, const char *value, CONFIG_VALUE_TYPES type) {

169 + struct config_option *opt = appconfig_option_find(sect, option);

170 + if(!opt)

171 + opt = appconfig_option_create(sect, option, value);

172 +

173 + appconfig_set_raw_value_of_option(opt, value, type);

174 + return opt;

175 +}

176 +

177 +struct config_option *appconfig_set_raw_value(struct config *root, const char *section, const char *option, const char *value, CONFIG_VALUE_TYPES type) {

178 + struct config_section *sect = appconfig_section_find(root, section);

179 + if(!sect)

180 + sect = appconfig_section_create(root, section);

181 +

182 + return appconfig_set_raw_value_of_option_in_section(sect, option, value, type);

183 +}

src/libnetdata/config/appconfig_sections.c new

+82

@@ -0,0 +1,82 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +// ----------------------------------------------------------------------------

6 +// config sections index

7 +

8 +int appconfig_section_compare(void *a, void *b) {

9 + if(((struct config_section *)a)->name < ((struct config_section *)b)->name) return -1;

10 + else if(((struct config_section *)a)->name > ((struct config_section *)b)->name) return 1;

11 + else return string_cmp(((struct config_section *)a)->name, ((struct config_section *)b)->name);

12 +}

13 +

14 +struct config_section *appconfig_section_find(struct config *root, const char *name) {

15 + struct config_section sect_tmp = {

16 + .name = string_strdupz(name),

17 + };

18 +

19 + struct config_section *rc = (struct config_section *)avl_search_lock(&root->index, (avl_t *) &sect_tmp);

20 + string_freez(sect_tmp.name);

21 + return rc;

22 +}

23 +

24 +// ----------------------------------------------------------------------------

25 +// config section methods

26 +

27 +void appconfig_section_free(struct config_section *sect) {

28 + avl_destroy_lock(&sect->values_index);

29 + string_freez(sect->name);

30 + freez(sect);

31 +}

32 +

33 +void appconfig_section_remove_and_delete(struct config *root, struct config_section *sect, bool have_root_lock, bool have_sect_lock) {

34 + struct config_section *sect_found = appconfig_section_del(root, sect);

35 + if(sect_found != sect) {

36 + nd_log(NDLS_DAEMON, NDLP_ERR,

37 + "INTERNAL ERROR: Cannot remove section '%s', it was not inserted before.",

38 + string2str(sect->name));

39 + return;

40 + }

41 +

42 + appconfig_option_remove_and_delete_all(sect, have_sect_lock);

43 +

44 + if(!have_root_lock)

45 + APPCONFIG_LOCK(root);

46 +

47 + DOUBLE_LINKED_LIST_REMOVE_ITEM_UNSAFE(root->sections, sect, prev, next);

48 +

49 + if(!have_root_lock)

50 + APPCONFIG_UNLOCK(root);

51 +

52 + // if the caller has the section lock, we will unlock it, to cleanup

53 + if(have_sect_lock)

54 + SECTION_UNLOCK(sect);

55 +

56 + appconfig_section_free(sect);

57 +}

58 +

59 +struct config_section *appconfig_section_create(struct config *root, const char *section) {

60 + struct config_section *sect = callocz(1, sizeof(struct config_section));

61 + sect->name = string_strdupz(section);

62 + spinlock_init(&sect->spinlock);

63 +

64 + avl_init_lock(&sect->values_index, appconfig_option_compare);

65 +

66 + struct config_section *sect_found = appconfig_section_add(root, sect);

67 + if(sect_found != sect) {

68 + nd_log(NDLS_DAEMON, NDLP_ERR,

69 + "CONFIG: section '%s', already exists, using existing.",

70 + string2str(sect->name));

71 + appconfig_section_free(sect);

72 + return sect_found;

73 + }

74 +

75 + APPCONFIG_LOCK(root);

76 + DOUBLE_LINKED_LIST_APPEND_ITEM_UNSAFE(root->sections, sect, prev, next);

77 + APPCONFIG_UNLOCK(root);

78 +

79 + return sect;

80 +}

81 +

82 +

src/libnetdata/config/appconfig_traversal.c new

+21

@@ -0,0 +1,21 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "appconfig_internals.h"

4 +

5 +size_t appconfig_foreach_value_in_section(struct config *root, const char *section, appconfig_foreach_value_cb_t cb, void *data) {

6 + size_t used = 0;

7 + struct config_section *co = appconfig_section_find(root, section);

8 + if(co) {

9 + SECTION_LOCK(co);

10 + struct config_option *cv;

11 + for(cv = co->values; cv ; cv = cv->next) {

12 + if(cb(data, string2str(cv->name), string2str(cv->value))) {

13 + cv->flags |= CONFIG_VALUE_USED;

14 + used++;

15 + }

16 + }

17 + SECTION_UNLOCK(co);

18 + }

19 +

20 + return used;

21 +}

src/libnetdata/ebpf/ebpf.c

+9 -9

@@ -1014,7 +1014,7 @@ int ebpf_load_config(struct config *config, char *filename)

1014 }

1015

1016

1017 -static netdata_run_mode_t ebpf_select_mode(char *mode)

1017 +static netdata_run_mode_t ebpf_select_mode(const char *mode)

1018 {

1019 if (!strcasecmp(mode,EBPF_CFG_LOAD_MODE_RETURN ))

1020 return MODE_RETURN;

@@ -1041,7 +1041,7 @@ static void ebpf_select_mode_string(char *output, size_t len, netdata_run_mode_t

1041 *

1042 * @return It returns the value to be used.

1043 */

1044 -netdata_ebpf_load_mode_t epbf_convert_string_to_load_mode(char *str)

1044 +netdata_ebpf_load_mode_t epbf_convert_string_to_load_mode(const char *str)

1045 {

1046 if (!strcasecmp(str, EBPF_CFG_CORE_PROGRAM))

1047 return EBPF_LOAD_CORE;

@@ -1094,7 +1094,7 @@ static char *ebpf_convert_collect_pid_to_string(netdata_apps_level_t level)

1094 *

1095 * @return it returns the level associated to the string or default when it is a wrong value

1096 */

1097 -netdata_apps_level_t ebpf_convert_string_to_apps_level(char *str)

1097 +netdata_apps_level_t ebpf_convert_string_to_apps_level(const char *str)

1098 {

1099 if (!strcasecmp(str, EBPF_CFG_PID_REAL_PARENT))

1100 return NETDATA_APPS_LEVEL_REAL_PARENT;

@@ -1114,7 +1114,7 @@ netdata_apps_level_t ebpf_convert_string_to_apps_level(char *str)

1114 * @param str value read from configuration file.

1115 * @param lmode load mode used by collector.

1116 */

1117 -netdata_ebpf_program_loaded_t ebpf_convert_core_type(char *str, netdata_run_mode_t lmode)

1117 +netdata_ebpf_program_loaded_t ebpf_convert_core_type(const char *str, netdata_run_mode_t lmode)

1118 {

1119 if (!strcasecmp(str, EBPF_CFG_ATTACH_TRACEPOINT))

1120 return EBPF_LOAD_TRACEPOINT;

@@ -1174,7 +1174,7 @@ struct btf *ebpf_parse_btf_file(const char *filename)

1174 * @param path is the fullpath

1175 * @param filename is the file inside BTF path.

1176 */

1177 -struct btf *ebpf_load_btf_file(char *path, char *filename)

1177 +struct btf *ebpf_load_btf_file(const char *path, const char *filename)

1178 {

1179 char fullpath[PATH_MAX + 1];

1180 snprintfz(fullpath, PATH_MAX, "%s/%s", path, filename);

@@ -1299,7 +1299,7 @@ void ebpf_update_module_using_config(ebpf_module_t *modules, netdata_ebpf_load_m

1299 {

1300 char default_value[EBPF_MAX_MODE_LENGTH + 1];

1301 ebpf_select_mode_string(default_value, EBPF_MAX_MODE_LENGTH, modules->mode);

1302 - char *load_mode = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_LOAD_MODE, default_value);

1302 + const char *load_mode = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_LOAD_MODE, default_value);

1303 modules->mode = ebpf_select_mode(load_mode);

1304

1305 modules->update_every = (int)appconfig_get_number(modules->cfg, EBPF_GLOBAL_SECTION,

@@ -1318,17 +1318,17 @@ void ebpf_update_module_using_config(ebpf_module_t *modules, netdata_ebpf_load_m

1318 EBPF_CFG_LIFETIME, EBPF_DEFAULT_LIFETIME);

1319

1320 char *value = ebpf_convert_load_mode_to_string(modules->load & NETDATA_EBPF_LOAD_METHODS);

1321 - char *type_format = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_TYPE_FORMAT, value);

1321 + const char *type_format = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_TYPE_FORMAT, value);

1322 netdata_ebpf_load_mode_t load = epbf_convert_string_to_load_mode(type_format);

1323 load = ebpf_select_load_mode(btf_file, load, kver, is_rh);

1324 modules->load = origin | load;

1325

1326 - char *core_attach = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_CORE_ATTACH, EBPF_CFG_ATTACH_TRAMPOLINE);

1326 + const char *core_attach = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_CORE_ATTACH, EBPF_CFG_ATTACH_TRAMPOLINE);

1327 netdata_ebpf_program_loaded_t fill_lm = ebpf_convert_core_type(core_attach, modules->mode);

1328 ebpf_update_target_with_conf(modules, fill_lm);

1329

1330 value = ebpf_convert_collect_pid_to_string(modules->apps_level);

1331 - char *collect_pid = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_COLLECT_PID, value);

1331 + const char *collect_pid = appconfig_get(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_COLLECT_PID, value);

1332 modules->apps_level = ebpf_convert_string_to_apps_level(collect_pid);

1333

1334 modules->maps_per_core = appconfig_get_boolean(modules->cfg, EBPF_GLOBAL_SECTION, EBPF_CFG_MAPS_PER_CORE,

src/libnetdata/ebpf/ebpf.h

+3 -3

@@ -470,13 +470,13 @@ int ebpf_disable_tracing_values(char *subsys, char *eventname);

470 // BTF helpers

471 #define NETDATA_EBPF_MAX_SYSCALL_LENGTH 255

472

473 -netdata_ebpf_load_mode_t epbf_convert_string_to_load_mode(char *str);

474 -netdata_ebpf_program_loaded_t ebpf_convert_core_type(char *str, netdata_run_mode_t lmode);

473 +netdata_ebpf_load_mode_t epbf_convert_string_to_load_mode(const char *str);

474 +netdata_ebpf_program_loaded_t ebpf_convert_core_type(const char *str, netdata_run_mode_t lmode);

475 void ebpf_select_host_prefix(char *output, size_t length, char *syscall, int kver);

476 #ifdef LIBBPF_MAJOR_VERSION

477 void ebpf_adjust_thread_load(ebpf_module_t *mod, struct btf *file);

478 struct btf *ebpf_parse_btf_file(const char *filename);

479 -struct btf *ebpf_load_btf_file(char *path, char *filename);

479 +struct btf *ebpf_load_btf_file(const char *path, const char *filename);

480 int ebpf_is_function_inside_btf(struct btf *file, char *function);

481 void ebpf_update_map_type(struct bpf_map *map, ebpf_local_maps_t *w);

482 void ebpf_define_map_type(ebpf_local_maps_t *maps, int maps_per_core, int kver);

src/libnetdata/libjudy/judy-malloc.c new

+78

@@ -0,0 +1,78 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "judy-malloc.h"

4 +

5 +#define MAX_JUDY_SIZE_TO_ARAL 24

6 +static bool judy_sizes_config[MAX_JUDY_SIZE_TO_ARAL + 1] = {

7 + [3] = true,

8 + [4] = true,

9 + [5] = true,

10 + [6] = true,

11 + [7] = true,

12 + [8] = true,

13 + [10] = true,

14 + [11] = true,

15 + [15] = true,

16 + [23] = true,

17 +};

18 +static ARAL *judy_sizes_aral[MAX_JUDY_SIZE_TO_ARAL + 1] = {};

19 +

20 +struct aral_statistics judy_sizes_aral_statistics = {};

21 +

22 +__attribute__((constructor)) void aral_judy_init(void) {

23 + for(size_t Words = 0; Words <= MAX_JUDY_SIZE_TO_ARAL; Words++)

24 + if(judy_sizes_config[Words]) {

25 + char buf[30+1];

26 + snprintfz(buf, sizeof(buf) - 1, "judy-%zu", Words * sizeof(Word_t));

27 + judy_sizes_aral[Words] = aral_create(

28 + buf,

29 + Words * sizeof(Word_t),

30 + 0,

31 + 65536,

32 + &judy_sizes_aral_statistics,

33 + NULL, NULL, false, false);

34 + }

35 +}

36 +

37 +size_t judy_aral_overhead(void) {

38 + return aral_overhead_from_stats(&judy_sizes_aral_statistics);

39 +}

40 +

41 +size_t judy_aral_structures(void) {

42 + return aral_structures_from_stats(&judy_sizes_aral_statistics);

43 +}

44 +

45 +static ARAL *judy_size_aral(Word_t Words) {

46 + if(Words <= MAX_JUDY_SIZE_TO_ARAL && judy_sizes_aral[Words])

47 + return judy_sizes_aral[Words];

48 +

49 + return NULL;

50 +}

51 +

52 +inline Word_t JudyMalloc(Word_t Words) {

53 + Word_t Addr;

54 +

55 + ARAL *ar = judy_size_aral(Words);

56 + if(ar)

57 + Addr = (Word_t) aral_mallocz(ar);

58 + else

59 + Addr = (Word_t) mallocz(Words * sizeof(Word_t));

60 +

61 + return(Addr);

62 +}

63 +

64 +inline void JudyFree(void * PWord, Word_t Words) {

65 + ARAL *ar = judy_size_aral(Words);

66 + if(ar)

67 + aral_freez(ar, PWord);

68 + else

69 + freez(PWord);

70 +}

71 +

72 +Word_t JudyMallocVirtual(Word_t Words) {

73 + return JudyMalloc(Words);

74 +}

75 +

76 +void JudyFreeVirtual(void * PWord, Word_t Words) {

77 + JudyFree(PWord, Words);

78 +}

src/libnetdata/libjudy/judy-malloc.h new

+11

@@ -0,0 +1,11 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_JUDY_MALLOC_H

4 +#define NETDATA_JUDY_MALLOC_H

5 +

6 +#include "../libnetdata.h"

7 +

8 +size_t judy_aral_overhead(void);

9 +size_t judy_aral_structures(void);

10 +

11 +#endif //NETDATA_JUDY_MALLOC_H

src/libnetdata/libnetdata.c

+1 -76

@@ -13,88 +13,13 @@

13 struct rlimit rlimit_nofile = { .rlim_cur = 1024, .rlim_max = 1024 };

15 #if defined(MADV_MERGEABLE)

16 -int enable_ksm = 1;

16 +int enable_ksm = CONFIG_BOOLEAN_AUTO;

17 #else

18 int enable_ksm = 0;

19 #endif

21 volatile sig_atomic_t netdata_exit = 0;

23 -#define MAX_JUDY_SIZE_TO_ARAL 24

24 -static bool judy_sizes_config[MAX_JUDY_SIZE_TO_ARAL + 1] = {

25 - [3] = true,

26 - [4] = true,

27 - [5] = true,

28 - [6] = true,

29 - [7] = true,

30 - [8] = true,

31 - [10] = true,

32 - [11] = true,

33 - [15] = true,

34 - [23] = true,

35 -};

36 -static ARAL *judy_sizes_aral[MAX_JUDY_SIZE_TO_ARAL + 1] = {};

37 -

38 -struct aral_statistics judy_sizes_aral_statistics = {};

39 -

40 -void aral_judy_init(void) {

41 - for(size_t Words = 0; Words <= MAX_JUDY_SIZE_TO_ARAL; Words++)

42 - if(judy_sizes_config[Words]) {

43 - char buf[30+1];

44 - snprintfz(buf, sizeof(buf) - 1, "judy-%zu", Words * sizeof(Word_t));

45 - judy_sizes_aral[Words] = aral_create(

46 - buf,

47 - Words * sizeof(Word_t),

48 - 0,

49 - 65536,

50 - &judy_sizes_aral_statistics,

51 - NULL, NULL, false, false);

52 - }

53 -}

54 -

55 -size_t judy_aral_overhead(void) {

56 - return aral_overhead_from_stats(&judy_sizes_aral_statistics);

57 -}

58 -

59 -size_t judy_aral_structures(void) {

60 - return aral_structures_from_stats(&judy_sizes_aral_statistics);

61 -}

62 -

63 -static ARAL *judy_size_aral(Word_t Words) {

64 - if(Words <= MAX_JUDY_SIZE_TO_ARAL && judy_sizes_aral[Words])

65 - return judy_sizes_aral[Words];

66 -

67 - return NULL;

68 -}

69 -

70 -inline Word_t JudyMalloc(Word_t Words) {

71 - Word_t Addr;

72 -

73 - ARAL *ar = judy_size_aral(Words);

74 - if(ar)

75 - Addr = (Word_t) aral_mallocz(ar);

76 - else

77 - Addr = (Word_t) mallocz(Words * sizeof(Word_t));

78 -

79 - return(Addr);

80 -}

81 -

82 -inline void JudyFree(void * PWord, Word_t Words) {

83 - ARAL *ar = judy_size_aral(Words);

84 - if(ar)

85 - aral_freez(ar, PWord);

86 - else

87 - freez(PWord);

88 -}

89 -

90 -Word_t JudyMallocVirtual(Word_t Words) {

91 - return JudyMalloc(Words);

92 -}

93 -

94 -void JudyFreeVirtual(void * PWord, Word_t Words) {

95 - JudyFree(PWord, Words);

96 -}

97 -

23 // ----------------------------------------------------------------------------

24 // memory allocation functions that handle failures

src/libnetdata/libnetdata.h

+3 -5

@@ -307,9 +307,7 @@ typedef uint32_t uid_t;

307 #define WARNUNUSED

308 #endif

309

310 -void aral_judy_init(void);

311 -size_t judy_aral_overhead(void);

312 -size_t judy_aral_structures(void);

310 +#include "libjudy/judy-malloc.h"

311

312 #define ABS(x) (((x) < 0)? (-(x)) : (x))

313 #define MIN(a,b) (((a)<(b))?(a):(b))

@@ -435,7 +433,7 @@ void netdata_cleanup_and_exit(int ret, const char *action, const char *action_re

433 void netdata_cleanup_and_exit(int ret, const char *action, const char *action_result, const char *action_data) NORETURN;

434 #endif

435

438 -extern char *netdata_configured_host_prefix;

436 +extern const char *netdata_configured_host_prefix;

437

438 #include "os/os.h"

439

@@ -459,6 +457,7 @@ extern char *netdata_configured_host_prefix;

457 #include "inlined.h"

458 #include "line_splitter/line_splitter.h"

459 #include "clocks/clocks.h"

460 +#include "parsers/parsers.h"

461 #include "datetime/iso8601.h"

462 #include "datetime/rfc3339.h"

463 #include "datetime/rfc7231.h"

@@ -639,7 +638,6 @@ void timing_action(TIMING_ACTION action, TIMING_STEP step);

638 int hash256_string(const unsigned char *string, size_t size, char *hash);

639

640 extern bool unittest_running;

642 -#define API_RELATIVE_TIME_MAX (3 * 365 * 86400)

641

642 bool rrdr_relative_window_to_absolute(time_t *after, time_t *before, time_t now);

643 bool rrdr_relative_window_to_absolute_query(time_t *after, time_t *before, time_t *now_ptr, bool unittest);

src/libnetdata/log/README.md

+1 -1

@@ -73,7 +73,7 @@ In `netdata.conf`, there are the following settings:

73 ```

74 [logs]

75 # logs to trigger flood protection = 1000

76 - # logs flood protection period = 60

76 + # logs flood protection period = 1m

77 # facility = daemon

78 # level = info

79 # daemon = journal

src/libnetdata/log/log.c

+11 -3

@@ -580,7 +580,14 @@ void nd_log_set_user_settings(ND_LOG_SOURCES source, const char *setting) {

580 *slash = '\0';

581 slash++;

582 ls->limits.logs_per_period = ls->limits.logs_per_period_backup = str2u(value);

583 - ls->limits.throttle_period = str2u(slash);

583 +

584 + int period;

585 + if(!duration_parse_seconds(slash, &period)) {

586 + nd_log(NDLS_DAEMON, NDLP_ERR, "Error while parsing period '%s'", slash);

587 + period = ND_LOG_DEFAULT_THROTTLE_PERIOD;

588 + }

589 +

590 + ls->limits.throttle_period = period;

591 }

592 else {

593 ls->limits.logs_per_period = ls->limits.logs_per_period_backup = str2u(value);

@@ -589,8 +596,9 @@ void nd_log_set_user_settings(ND_LOG_SOURCES source, const char *setting) {

596 }

597 }

598 else

592 - nd_log(NDLS_DAEMON, NDLP_ERR, "Error while parsing configuration of log source '%s'. "

593 - "In config '%s', '%s' is not understood.",

599 + nd_log(NDLS_DAEMON, NDLP_ERR,

600 + "Error while parsing configuration of log source '%s'. "

601 + "In config '%s', '%s' is not understood.",

602 nd_log_id2source(source), setting, name);

603 }

604 }

src/libnetdata/parsers/duration.c new

+251

@@ -0,0 +1,251 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "duration.h"

4 +

5 +#ifdef NSEC_PER_USEC

6 +#undef NSEC_PER_USEC

7 +#endif

8 +#define NSEC_PER_USEC (1000ULL)

9 +

10 +#ifdef USEC_PER_MS

11 +#undef USEC_PER_MS

12 +#endif

13 +#define USEC_PER_MS (1000ULL)

14 +

15 +#ifdef NSEC_PER_SEC

16 +#undef NSEC_PER_SEC

17 +#endif

18 +#define NSEC_PER_SEC (1000000000ULL)

19 +

20 +#define NSEC_PER_MS (USEC_PER_MS * NSEC_PER_USEC)

21 +#define NSEC_PER_MIN (NSEC_PER_SEC * 60ULL)

22 +#define NSEC_PER_HOUR (NSEC_PER_MIN * 60ULL)

23 +#define NSEC_PER_DAY (NSEC_PER_HOUR * 24ULL)

24 +#define NSEC_PER_WEEK (NSEC_PER_DAY * 7ULL)

25 +#define NSEC_PER_MONTH (NSEC_PER_DAY * 30ULL)

26 +#define NSEC_PER_QUARTER (NSEC_PER_MONTH * 3ULL)

27 +

28 +// more accurate, but not an integer multiple of days, weeks, months

29 +#define NSEC_PER_YEAR (NSEC_PER_DAY * 365ULL)

30 +

31 +// Define a structure to map time units to their multipliers

32 +static const struct duration_unit {

33 + const char *unit;

34 + const bool formatter; // true when this unit should be used when formatting to string

35 + const snsec_t multiplier;

36 +} units[] = {

37 +

38 + // IMPORTANT: the order of this array is crucial!

39 + // The array should be sorted from the smaller unit to the biggest unit.

40 +

41 + { .unit = "ns", .formatter = true, .multiplier = 1 }, // UCUM

42 + { .unit = "us", .formatter = true, .multiplier = NSEC_PER_USEC }, // UCUM

43 + { .unit = "ms", .formatter = true, .multiplier = NSEC_PER_MS }, // UCUM

44 + { .unit = "s", .formatter = true, .multiplier = NSEC_PER_SEC }, // UCUM

45 + { .unit = "m", .formatter = true, .multiplier = NSEC_PER_MIN }, // -

46 + { .unit = "min", .formatter = false, .multiplier = NSEC_PER_MIN }, // UCUM

47 + { .unit = "h", .formatter = true, .multiplier = NSEC_PER_HOUR }, // UCUM

48 + { .unit = "d", .formatter = true, .multiplier = NSEC_PER_DAY }, // UCUM

49 + { .unit = "w", .formatter = false, .multiplier = NSEC_PER_WEEK }, // -

50 + { .unit = "wk", .formatter = false, .multiplier = NSEC_PER_WEEK }, // UCUM

51 + { .unit = "mo", .formatter = true, .multiplier = NSEC_PER_MONTH }, // UCUM

52 + { .unit = "M", .formatter = false, .multiplier = NSEC_PER_MONTH }, // compatibility

53 + { .unit = "q", .formatter = false, .multiplier = NSEC_PER_QUARTER }, // -

54 + { .unit = "y", .formatter = true, .multiplier = NSEC_PER_YEAR }, // -

55 + { .unit = "Y", .formatter = false, .multiplier = NSEC_PER_YEAR }, // compatibility

56 + { .unit = "a", .formatter = false, .multiplier = NSEC_PER_YEAR }, // UCUM

57 +};

58 +

59 +static inline const struct duration_unit *duration_find_unit(const char *unit) {

60 + if(!unit || !*unit)

61 + unit = "ns";

62 +

63 + for (size_t i = 0; i < sizeof(units) / sizeof(units[0]); i++) {

64 + const struct duration_unit *du = &units[i];

65 + if ((uint8_t)unit[0] == (uint8_t)du->unit[0] && strcmp(unit, du->unit) == 0)

66 + return du;

67 + }

68 +

69 + return NULL;

70 +}

71 +

72 +inline int64_t duration_round_to_resolution(int64_t value, int64_t resolution) {

73 + if(value > 0)

74 + return (value + ((resolution - 1) / 2)) / resolution;

75 +

76 + if(value < 0)

77 + return (value - ((resolution - 1) / 2)) / resolution;

78 +

79 + return 0;

80 +}

81 +

82 +// -------------------------------------------------------------------------------------------------------------------

83 +// parse a duration string

84 +

85 +bool duration_parse(const char *duration, int64_t *result, const char *default_unit) {

86 + if (!duration || !*duration) {

87 + *result = 0;

88 + return false;

89 + }

90 +

91 + const struct duration_unit *du_def = duration_find_unit(default_unit);

92 + if(!du_def) {

93 + *result = 0;

94 + return false;

95 + }

96 +

97 + int64_t sign = 1;

98 + const char *s = duration;

99 + while (isspace((uint8_t)*s)) s++;

100 + if(*s == '-') {

101 + s++;

102 + sign = -1;

103 + }

104 +

105 + int64_t v = 0;

106 +

107 + while (*s) {

108 + // Skip leading spaces

109 + while (isspace((uint8_t)*s)) s++;

110 +

111 + // compatibility

112 + if(*s == 'n' && strcmp(s, "never") == 0) {

113 + *result = 0;

114 + return true;

115 + }

116 +

117 + if(*s == 'o' && strcmp(s, "off") == 0) {

118 + *result = 0;

119 + return true;

120 + }

121 +

122 + // Parse the number

123 + const char *number_start = s;

124 + NETDATA_DOUBLE value = str2ndd(s, (char **)&s);

125 +

126 + // If no valid number found, return default

127 + if (s == number_start) {

128 + *result = 0;

129 + return false;

130 + }

131 +

132 + // Skip spaces between number and unit

133 + while (isspace((uint8_t)*s)) s++;

134 +

135 + const char *unit_start = s;

136 + while (isalpha((uint8_t)*s)) s++;

137 +

138 + char unit[4];

139 + size_t unit_len = s - unit_start;

140 + const struct duration_unit *du;

141 + if (unit_len == 0)

142 + du = du_def;

143 + else {

144 + if (unit_len >= sizeof(unit)) unit_len = sizeof(unit) - 1;

145 + strncpyz(unit, unit_start, unit_len);

146 + du = duration_find_unit(unit);

147 + if(!du) {

148 + *result = 0;

149 + return false;

150 + }

151 + }

152 +

153 + v += (int64_t)round(value * (NETDATA_DOUBLE)du->multiplier);

154 + }

155 +

156 + v *= sign;

157 +

158 + if(du_def->multiplier == 1)

159 + *result = v;

160 + else

161 + *result = duration_round_to_resolution(v, du_def->multiplier);

162 +

163 + return true;

164 +}

165 +

166 +// --------------------------------------------------------------------------------------------------------------------

167 +// generate a string to represent a duration

168 +

169 +ssize_t duration_snprintf(char *dst, size_t dst_size, int64_t value, const char *unit) {

170 + if (!dst || dst_size == 0) return -1;

171 + if (dst_size == 1) {

172 + dst[0] = '\0';

173 + return -2;

174 + }

175 +

176 + if(value == 0)

177 + return snprintfz(dst, dst_size, "off");

178 +

179 + const char *sign = "";

180 + if(value < 0) {

181 + sign = "-";

182 + value = -value;

183 + }

184 +

185 + const struct duration_unit *du_min = duration_find_unit(unit);

186 + size_t offset = 0;

187 +

188 + int64_t nsec = value * du_min->multiplier;

189 +

190 + // Iterate through units from largest to smallest

191 + for (size_t i = sizeof(units) / sizeof(units[0]) - 1; i > 0 && nsec > 0; i--) {

192 + const struct duration_unit *du = &units[i];

193 + if(!units[i].formatter && du != du_min)

194 + continue;

195 +

196 + // IMPORTANT:

197 + // The week (7 days) is not aligned to the quarter (~91 days) or the year (365.25 days).

198 + // To make sure that the value returned can be parsed back without loss,

199 + // we have to round the value per unit (inside this loop), not globally.

200 + // Otherwise, we have to make sure that all larger units are integer multiples of the smaller ones.

201 +

202 + int64_t multiplier = units[i].multiplier;

203 + int64_t rounded = (du == du_min) ? (duration_round_to_resolution(nsec, multiplier) * multiplier) : nsec;

204 +

205 + int64_t unit_count = rounded / multiplier;

206 + if (unit_count > 0) {

207 + int written = snprintfz(dst + offset, dst_size - offset,

208 + "%s%" PRIi64 "%s", sign, unit_count, units[i].unit);

209 +

210 + if (written < 0)

211 + return -3;

212 +

213 + sign = "";

214 + offset += written;

215 +

216 + if (offset >= dst_size) {

217 + // buffer overflow

218 + return (ssize_t)offset;

219 + }

220 +

221 + if(unit_count * multiplier >= nsec)

222 + break;

223 + else

224 + nsec -= unit_count * multiplier;

225 + }

226 +

227 + if(du == du_min)

228 + // we should not go to smaller units

229 + break;

230 + }

231 +

232 + if (offset == 0)

233 + // nothing has been written

234 + offset = snprintfz(dst, dst_size, "off");

235 +

236 + return (ssize_t)offset;

237 +}

238 +

239 +// --------------------------------------------------------------------------------------------------------------------

240 +// compatibility for parsing seconds in int.

241 +

242 +bool duration_parse_seconds(const char *str, int *result) {

243 + int64_t v;

244 +

245 + if(duration_parse_time_t(str, &v)) {

246 + *result = (int)v;

247 + return true;

248 + }

249 +

250 + return false;

251 +}

src/libnetdata/parsers/duration.h new

+32

@@ -0,0 +1,32 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef LIBNETDATA_PARSERS_DURATION_H

4 +#define LIBNETDATA_PARSERS_DURATION_H

5 +

6 +#include "parsers.h"

7 +

8 +int64_t duration_round_to_resolution(int64_t value, int64_t resolution);

9 +

10 +// duration (string to number)

11 +bool duration_parse(const char *duration, int64_t *result, const char *default_unit);

12 +#define duration_parse_nsec_t(duration, ns_ptr) duration_parse(duration, ns_ptr, "ns")

13 +#define duration_parse_usec_t(duration, us_ptr) duration_parse(duration, us_ptr, "us")

14 +#define duration_parse_msec_t(duration, ms_ptr) duration_parse(duration, ms_ptr, "ms")

15 +#define duration_parse_time_t(duration, secs_ptr) duration_parse(duration, secs_ptr, "s")

16 +#define duration_parse_mins(duration, mins_ptr) duration_parse(duration, mins_ptr, "m")

17 +#define duration_parse_hours(duration, hours_ptr) duration_parse(duration, hours_ptr, "h")

18 +#define duration_parse_days(duration, days_ptr) duration_parse(duration, days_ptr, "d")

19 +

20 +// duration (number to string)

21 +ssize_t duration_snprintf(char *dst, size_t dst_size, int64_t value, const char *unit);

22 +#define duration_snprintf_nsec_t(dst, dst_size, ns) duration_snprintf(dst, dst_size, ns, "ns")

23 +#define duration_snprintf_usec_t(dst, dst_size, us) duration_snprintf(dst, dst_size, us, "us")

24 +#define duration_snprintf_msec_t(dst, dst_size, ms) duration_snprintf(dst, dst_size, ms, "ms")

25 +#define duration_snprintf_time_t(dst, dst_size, secs) duration_snprintf(dst, dst_size, secs, "s")

26 +#define duration_snprintf_mins(dst, dst_size, mins) duration_snprintf(dst, dst_size, mins, "m")

27 +#define duration_snprintf_hours(dst, dst_size, hours) duration_snprintf(dst, dst_size, hours, "h")

28 +#define duration_snprintf_days(dst, dst_size, days) duration_snprintf(dst, dst_size, days, "d")

29 +

30 +bool duration_parse_seconds(const char *str, int *result);

31 +

32 +#endif //LIBNETDATA_PARSERS_DURATION_H

src/libnetdata/parsers/duration.html new

+205

@@ -0,0 +1,205 @@

1 +<!DOCTYPE html>

2 +<html lang="en">

3 +<head>

4 + <meta charset="UTF-8">

5 + <meta http-equiv="X-UA-Compatible" content="IE=edge">

6 + <meta name="viewport" content="width=device-width, initial-scale=1.0">

7 + <title>Duration Converter</title>

8 + <style>

9 + table {

10 + width: 50%;

11 + border-collapse: collapse;

12 + margin-top: 20px;

13 + }

14 + table, th, td {

15 + border: 1px solid black;

16 + }

17 + th, td {

18 + padding: 10px;

19 + text-align: center;

20 + }

21 + .error {

22 + color: red;

23 + margin-top: 10px;

24 + }

25 + </style>

26 +</head>

27 +<body>

28 +<h1>Duration Converter</h1>

29 +<input type="text" id="durationInput" placeholder="Enter duration (e.g., 10d-12h)">

30 +<div id="errorMessage" class="error"></div>

31 +

32 +<table id="resultTable">

33 + <thead>

34 + <tr>

35 + <th>Unit</th>

36 + <th>Value</th>

37 + <th>Formatted</th>

38 + <th>Check</th>

39 + </tr>

40 + </thead>

41 + <tbody>

42 + </tbody>

43 +</table>

44 +

45 +<script>

46 + const NSEC_PER_USEC = 1000;

47 + const USEC_PER_MS = 1000;

48 + const NSEC_PER_SEC = 1000000000;

49 + const NSEC_PER_MS = USEC_PER_MS * NSEC_PER_USEC;

50 + const NSEC_PER_MIN = NSEC_PER_SEC * 60;

51 + const NSEC_PER_HOUR = NSEC_PER_MIN * 60;

52 + const NSEC_PER_DAY = NSEC_PER_HOUR * 24;

53 + const NSEC_PER_WEEK = NSEC_PER_DAY * 7;

54 + const NSEC_PER_YEAR = NSEC_PER_DAY * 365;

55 + const NSEC_PER_MONTH = NSEC_PER_DAY * 30;

56 + const NSEC_PER_QUARTER = NSEC_PER_MONTH * 3;

57 +

58 + const units = [

59 + { unit: "ns", formatter: true, multiplier: 1 },

60 + { unit: "us", formatter: true, multiplier: NSEC_PER_USEC },

61 + { unit: "ms", formatter: true, multiplier: NSEC_PER_MS },

62 + { unit: "s", formatter: true, multiplier: NSEC_PER_SEC },

63 + { unit: "m", formatter: true, multiplier: NSEC_PER_MIN },

64 + { unit: "min", formatter: false, multiplier: NSEC_PER_MIN },

65 + { unit: "h", formatter: true, multiplier: NSEC_PER_HOUR },

66 + { unit: "d", formatter: true, multiplier: NSEC_PER_DAY },

67 + { unit: "w", formatter: false, multiplier: NSEC_PER_WEEK },

68 + { unit: "wk", formatter: false, multiplier: NSEC_PER_WEEK },

69 + { unit: "mo", formatter: true, multiplier: NSEC_PER_MONTH },

70 + { unit: "M", formatter: false, multiplier: NSEC_PER_MONTH },

71 + { unit: "q", formatter: false, multiplier: NSEC_PER_QUARTER },

72 + { unit: "y", formatter: true, multiplier: NSEC_PER_YEAR },

73 + { unit: "Y", formatter: false, multiplier: NSEC_PER_YEAR },

74 + { unit: "a", formatter: false, multiplier: NSEC_PER_YEAR }

75 + ];

76 +

77 + function durationFindUnit(unit) {

78 + if (!unit) return units[0];

79 + return units.find(u => u.unit === unit) || null;

80 + }

81 +

82 + function roundToResolution(value, resolution) {

83 + if (value > 0) return Math.floor((value + (resolution - 1) / 2) / resolution);

84 + if (value < 0) return Math.ceil((value - (resolution - 1) / 2) / resolution);

85 + return 0;

86 + }

87 +

88 + function parseDouble(str) {

89 + str = str.trim();

90 + const match = str.match(/^[-+]?\d*\.?\d+/);

91 + if (match) {

92 + const number = parseFloat(match[0]);

93 + const remainingStr = str.slice(match[0].length).trim();

94 + return { number, remainingStr };

95 + }

96 + return { number: null, remainingStr: str };

97 + }

98 +

99 + function durationParse(duration, unit) {

100 + if (!duration || !unit) return false;

101 +

102 + let s = duration.trim();

103 + let nsec = 0;

104 + let isNegative = false;

105 +

106 + // Handle leading negative sign

107 + if (s.startsWith("-")) {

108 + isNegative = true;

109 + s = s.slice(1).trim();

110 + }

111 +

112 + while (s.length > 0) {

113 + s = s.trim();

114 +

115 + if (s.startsWith("never") || s.startsWith("off"))

116 + return 0;

117 +

118 + const { number, remainingStr } = parseDouble(s);

119 + if (number === null) return false;

120 +

121 + s = remainingStr;

122 +

123 + const match = s.match(/^([a-zA-Z]*)/);

124 + let currentUnit = unit;

125 + if (match && match[0].length > 0) {

126 + currentUnit = match[0];

127 + s = s.slice(match[0].length).trim();

128 + }

129 +

130 + const du = durationFindUnit(currentUnit);

131 + if (!du) return false;

132 +

133 + nsec += number * du.multiplier;

134 + }

135 +

136 + const unitMultiplier = durationFindUnit(unit).multiplier;

137 + nsec = roundToResolution(nsec, unitMultiplier);

138 +

139 + return isNegative ? -nsec : nsec;

140 + }

141 +

142 + function durationSnprintf(value, unit) {

143 + if (value === 0) return "off";

144 +

145 + const duMin = durationFindUnit(unit);

146 + let nsec = Math.abs(value) * duMin.multiplier;

147 +

148 + const isNegative = value < 0;

149 + let result = isNegative ? "-" : "";

150 +

151 + for (let i = units.length - 1; i >= 0 && nsec !== 0; i--) {

152 + const du = units[i];

153 + if (!du.formatter && du !== duMin) continue;

154 +

155 + const multiplier = du.multiplier;

156 + const rounded = (du === duMin) ? roundToResolution(nsec, multiplier) * multiplier : nsec;

157 + let unitCount = Math.floor(rounded / multiplier);

158 +

159 + if (unitCount !== 0) {

160 + result += `${unitCount}${du.unit}`;

161 + nsec -= unitCount * multiplier;

162 + }

163 +

164 + if (du === duMin) break;

165 + }

166 +

167 + return result || "off";

168 + }

169 +

170 + function updateTable() {

171 + const duration = document.getElementById("durationInput").value;

172 + const tableBody = document.getElementById("resultTable").querySelector("tbody");

173 + const errorMessage = document.getElementById("errorMessage");

174 + tableBody.innerHTML = "";

175 + errorMessage.textContent = "";

176 +

177 + units.forEach(unit => {

178 + let value = durationParse(duration, unit.unit);

179 + let formatted;

180 + let check;

181 + if(value === false) {

182 + value = "-";

183 + formatted = "";

184 + check = "parsing error";

185 + }

186 + else {

187 + formatted = durationSnprintf(value, unit.unit);

188 + const parsedValue = durationParse(formatted, unit.unit);

189 + check = (parsedValue === value) ? "ok" : `re-parsing error (${parsedValue})`;

190 + }

191 +

192 + const row = `<tr>

193 + <td>${unit.unit}</td>

194 + <td>${value}</td>

195 + <td>${formatted}</td>

196 + <td>${check}</td>

197 + </tr>`;

198 + tableBody.innerHTML += row;

199 + });

200 + }

201 +

202 + document.getElementById("durationInput").addEventListener("input", updateTable);

203 +</script>

204 +</body>

205 +</html>

src/libnetdata/parsers/durations.md new

+94

@@ -0,0 +1,94 @@

1 +## Durations in Netdata

2 +

3 +Netdata provides a flexible and powerful way to specify durations for various configurations and operations, such as alerts, database retention, and other configuration options. Durations can be expressed in a variety of units, ranging from nanoseconds to years, allowing users to define time intervals in a human-readable format.

4 +

5 +### Supported Duration Units

6 +

7 +Netdata supports a wide range of duration units. The system follows the Unified Code for Units of Measure (UCUM) standard where applicable. Below is a table of all the supported units, their corresponding representations, and their compatibility:

8 +

10 +|:------:|:------------:|:--------:|:-------------:|:---------:|

11 +| `ns` | Nanoseconds | `1ns` | UCUM | **Yes** |

12 +| `us` | Microseconds | `1000ns` | UCUM | **Yes** |

13 +| `ms` | Milliseconds | `1000us` | UCUM | **Yes** |

14 +| `s` | Seconds | `1000ms` | UCUM | **Yes** |

15 +| `m` | Minutes | `60s` | Natural | **Yes** |

16 +| `min` | Minutes | `60s` | UCUM | No |

17 +| `h` | Hours | `60m` | UCUM | **Yes** |

18 +| `d` | Days | `24h` | UCUM | **Yes** |

19 +| `w` | Weeks | `7d` | Natural | No |

20 +| `wk` | Weeks | `7d` | UCUM | No |

21 +| `mo` | Months | `30d` | UCUM | **Yes** |

22 +| `M` | Months | `30d` | Backwards | No |

23 +| `q` | Quarters | `3mo` | Natural | No |

24 +| `y` | Years | `365d` | Natural | **Yes** |

25 +| `Y` | Years | `365d` | Backwards | No |

26 +| `a` | Years | `365d` | UCUM | No |

27 +

28 +- **UCUM**: The unit is specified in the Unified Code for Units of Measure (UCUM) standard.

29 +- **Natural**: We feel that this is more natural for expressing durations with single letter units.

30 +- **Backwards**: This unit has been used in the past in Netdata, and we support it for backwards compatibility.

31 +

32 +### Duration Expression Format

33 +

34 +Netdata allows users to express durations in both simple and complex formats.

35 +

36 +- **Simple Formats**: A duration can be specified using a number followed by a unit, such as `5m` (5 minutes), `2h` (2 hours), or `1d` (1 day). Fractional numbers are also supported, such as `1.5d`, `3.5mo` or `1.2y`.

37 +

38 +- **Complex Formats**: A duration can also be composed of multiple units added together. For example:

39 + - `1y2mo3w4d` represents 1 year, 2 months, 3 weeks, and 4 days.

40 + - `15d-12h` represents 15 days minus 12 hours (which equals 14 days and 12 hours).

41 +

42 +Each number given in durations can be either positive or negative. For example `1h15m` is 1 hour and 15 minutes, but `1h-15m` results to `45m`.

43 +

44 +The same unit can be given multiple times, so that `1d0.5d` is `1d12h` and `1d-0.5d` is `12h`.

45 +

46 +The order of units in the expressions is irrelevant, so that `1m2h3d` is the same to `3d2h1m`.

47 +

48 +The system will parse durations with spaces in them, but we suggest to write them down in compact form, without spaces. This is required, especially in alerts configuration, since spaces in durations will affect how parent expressions are tokenized.

49 +

50 +### Duration Rounding

51 +

52 +Netdata provides various functions to parse and round durations according to specific needs:

53 +

54 +- **Default Rounding to Seconds**: Most duration uses in Netdata are rounded to the nearest second. For example, a duration of `1.4s` would round to `1s`, while `1.5s` would round to `2s`.

55 +

56 +- **Rounding to Larger Units**: In some cases, such as database retention, durations are rounded to larger units like days. Even when rounding to a larger unit, durations can still be expressed in smaller units (e.g., `24h86400s` for `2d`).

57 +

58 +### Maximum and Minimum Duration Limits

59 +

60 +Netdata's duration expressions can handle durations ranging from the minimum possible value of `-INT64_MAX` to the maximum of `INT64_MAX` in nanoseconds. This range translates approximately to durations between -292 years to +292 years.

61 +

62 +### Inconsistencies in Duration Units

63 +

64 +While Netdata provides a flexible system for specifying durations, some inconsistencies arise due to the way different units are defined:

65 +

66 +- **1 Year (`y`) = 365 Days (`d`)**: In Netdata, a year is defined as 365 days. This is an approximation, since the average year is about 365.25 days.

67 +

68 +- **1 Month (`mo`) = 30 Days (`d`)**: Similarly, a month in Netdata is defined as 30 days, which is also an approximation. In reality, months vary in length (28 to 31 days).

69 +

70 +- **1 Quarter (`q`) = 3 Months (`mo`) = 90 Days (`d`)**: A quarter is defined as 3 months, or 90 days, which aligns with the approximation of each month being 30 days.

71 +

72 +These definitions can lead to some unexpected results when performing arithmetic with durations:

73 +

74 +**Example of Inconsistency**:

75 +

76 +`1y-1d` in Netdata calculates to `364d` but also as `12mo4d` because `1y = 365d` and `1mo = 30d`. This is inconsistent because `1y` is defined as `12mo5d` or `4q5d` (given the approximations above).

77 +

78 +### Negative Durations

79 +

80 +When the first letter of a duration expression is the minus character, Netdata parses the entire expression as positive and then it negates the result. for example: `-1m15s` is `-75s`, not `-45s`. To get `-45s` the expression should be `-1m-15s`. So the initial `-` is treated like `-(expression)`.

81 +

82 +The same rule is applied when generating duration expressions.

83 +

84 +### Example Duration Expressions

85 +

86 +Here are some examples of valid duration expressions:

87 +

88 +1. **`30s`**: 30 seconds.

89 +2. **`5m`**: 5 minutes.

90 +3. **`2h30m`**: 2 hours and 30 minutes.

91 +4. **`1.5d`**: 1 day and 12 hours.

92 +5. **`1w3d4h`**: 1 week, 3 days, and 4 hours.

93 +6. **`1y2mo3d`**: 1 year, 2 months, and 3 days.

94 +7. **`15d-12h`**: 14 days and 12 hours.

src/libnetdata/parsers/parsers.h new

+11

@@ -0,0 +1,11 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_PARSERS_H

4 +#define NETDATA_PARSERS_H

5 +

6 +#include "../libnetdata.h"

7 +#include "size.h"

8 +#include "duration.h"

9 +#include "timeframe.h"

10 +

11 +#endif //NETDATA_PARSERS_H

src/libnetdata/parsers/size.c new

+205

@@ -0,0 +1,205 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "size.h"

4 +

5 +// Define multipliers for base 2 (binary) units

6 +#define SIZE_MULTIPLIER_BASE2 1024ULL

7 +#define SIZE_MULTIPLIER_KiB (SIZE_MULTIPLIER_BASE2)

8 +#define SIZE_MULTIPLIER_MiB (SIZE_MULTIPLIER_KiB * SIZE_MULTIPLIER_BASE2)

9 +#define SIZE_MULTIPLIER_GiB (SIZE_MULTIPLIER_MiB * SIZE_MULTIPLIER_BASE2)

10 +#define SIZE_MULTIPLIER_TiB (SIZE_MULTIPLIER_GiB * SIZE_MULTIPLIER_BASE2)

11 +#define SIZE_MULTIPLIER_PiB (SIZE_MULTIPLIER_TiB * SIZE_MULTIPLIER_BASE2)

12 +//#define SIZE_MULTIPLIER_EiB (SIZE_MULTIPLIER_PiB * SIZE_MULTIPLIER_BASE2)

13 +//#define SIZE_MULTIPLIER_ZiB (SIZE_MULTIPLIER_EiB * SIZE_MULTIPLIER_BASE2)

14 +//#define SIZE_MULTIPLIER_YiB (SIZE_MULTIPLIER_ZiB * SIZE_MULTIPLIER_BASE2)

15 +

16 +// Define multipliers for base 10 (decimal) units

17 +#define SIZE_MULTIPLIER_BASE10 1000ULL

18 +#define SIZE_MULTIPLIER_K (SIZE_MULTIPLIER_BASE10)

19 +#define SIZE_MULTIPLIER_M (SIZE_MULTIPLIER_K * SIZE_MULTIPLIER_BASE10)

20 +#define SIZE_MULTIPLIER_G (SIZE_MULTIPLIER_M * SIZE_MULTIPLIER_BASE10)

21 +#define SIZE_MULTIPLIER_T (SIZE_MULTIPLIER_G * SIZE_MULTIPLIER_BASE10)

22 +#define SIZE_MULTIPLIER_P (SIZE_MULTIPLIER_T * SIZE_MULTIPLIER_BASE10)

23 +//#define SIZE_MULTIPLIER_E (SIZE_MULTIPLIER_P * SIZE_MULTIPLIER_BASE10)

24 +//#define SIZE_MULTIPLIER_Z (SIZE_MULTIPLIER_E * SIZE_MULTIPLIER_BASE10)

25 +//#define SIZE_MULTIPLIER_Y (SIZE_MULTIPLIER_Z * SIZE_MULTIPLIER_BASE10)

26 +

27 +// Define a structure to map size units to their multipliers

28 +static const struct size_unit {

29 + const char *unit;

30 + const uint8_t base;

31 + const bool formatter; // true when this unit should be used when formatting to string

32 + const uint64_t multiplier;

33 +} size_units[] = {

34 + // the order of this table is important: smaller to bigger units!

35 +

36 + { .unit = "B", .base = 2, .formatter = true, .multiplier = 1ULL },

37 + { .unit = "k", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_K },

38 + { .unit = "K", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_K },

39 + { .unit = "KB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_K },

40 + { .unit = "KiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_KiB },

41 + { .unit = "M", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_M },

42 + { .unit = "MB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_M },

43 + { .unit = "MiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_MiB },

44 + { .unit = "G", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_G },

45 + { .unit = "GB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_G },

46 + { .unit = "GiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_GiB },

47 + { .unit = "T", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_T },

48 + { .unit = "TB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_T },

49 + { .unit = "TiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_TiB },

50 + { .unit = "P", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_P },

51 + { .unit = "PB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_P },

52 + { .unit = "PiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_PiB },

53 +// { .unit = "E", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_E },

54 +// { .unit = "EB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_E },

55 +// { .unit = "EiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_EiB },

56 +// { .unit = "Z", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_Z },

57 +// { .unit = "ZB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_Z },

58 +// { .unit = "ZiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_ZiB },

59 +// { .unit = "Y", .base = 10, .formatter = true, .multiplier = SIZE_MULTIPLIER_Y },

60 +// { .unit = "YB", .base = 10, .formatter = false, .multiplier = SIZE_MULTIPLIER_Y },

61 +// { .unit = "YiB", .base = 2, .formatter = true, .multiplier = SIZE_MULTIPLIER_YiB },

62 +};

63 +

64 +static inline const struct size_unit *size_find_unit(const char *unit) {

65 + if (!unit || !*unit) unit = "B";

66 +

67 + for (size_t i = 0; i < sizeof(size_units) / sizeof(size_units[0]); i++) {

68 + const struct size_unit *su = &size_units[i];

69 + if ((uint8_t)unit[0] == (uint8_t)su->unit[0] && strcmp(unit, su->unit) == 0)

70 + return su;

71 + }

72 +

73 + return NULL;

74 +}

75 +

76 +static inline double size_round_to_resolution_dbl2(uint64_t value, uint64_t resolution) {

77 + double converted = (double)value / (double)resolution;

78 + return round(converted * 100.0) / 100.0;

79 +}

80 +

81 +static inline uint64_t size_round_to_resolution_int(uint64_t value, uint64_t resolution) {

82 + return (value + (resolution / 2)) / resolution;

83 +}

84 +

85 +// -------------------------------------------------------------------------------------------------------------------

86 +// parse a size string

87 +

88 +bool size_parse(const char *size_str, uint64_t *result, const char *default_unit) {

89 + if (!size_str || !*size_str) {

90 + *result = 0;

91 + return false;

92 + }

93 +

94 + const struct size_unit *su_def = size_find_unit(default_unit);

95 + if(!su_def) {

96 + *result = 0;

97 + return false;

98 + }

99 +

100 + const char *s = size_str;

101 +

102 + // Skip leading spaces

103 + while (isspace((uint8_t)*s)) s++;

104 +

105 + if(strcmp(s, "off") == 0) {

106 + *result = 0;

107 + return true;

108 + }

109 +

110 + // Parse the number

111 + const char *number_start = s;

112 + NETDATA_DOUBLE value = strtondd(s, (char **)&s);

113 +

114 + // If no valid number found, return false

115 + if (s == number_start || value < 0) {

116 + *result = 0;

117 + return false;

118 + }

119 +

120 + // Skip spaces between number and unit

121 + while (isspace((uint8_t)*s)) s++;

122 +

123 + const char *unit_start = s;

124 + while (isalpha((uint8_t)*s)) s++;

125 +

126 + char unit[4];

127 + size_t unit_len = s - unit_start;

128 + const struct size_unit *su;

129 + if (unit_len == 0)

130 + su = su_def;

131 + else {

132 + if (unit_len >= sizeof(unit)) unit_len = sizeof(unit) - 1;

133 + strncpy(unit, unit_start, unit_len);

134 + unit[unit_len] = '\0';

135 + su = size_find_unit(unit);

136 + if (!su) {

137 + *result = 0;

138 + return false;

139 + }

140 + }

141 +

142 + uint64_t bytes = (uint64_t)round(value * (NETDATA_DOUBLE)su->multiplier);

143 + *result = size_round_to_resolution_int(bytes, su_def->multiplier);

144 +

145 + return true;

146 +}

147 +

148 +// --------------------------------------------------------------------------------------------------------------------

149 +// generate a string to represent a size

150 +

151 +ssize_t size_snprintf(char *dst, size_t dst_size, uint64_t value, const char *unit) {

152 + if (!dst || dst_size == 0) return -1;

153 + if (dst_size == 1) {

154 + dst[0] = '\0';

155 + return -2;

156 + }

157 +

158 + if (value == 0)

159 + return snprintfz(dst, dst_size, "off");

160 +

161 + const struct size_unit *su_def = size_find_unit(unit);

162 + if(!su_def) return -3;

163 +

164 + // use the units multiplier to find the units

165 + uint64_t bytes = value * su_def->multiplier;

166 +

167 + // Find the best unit to represent the size with up to 2 fractional digits

168 + const struct size_unit *su_best = su_def;

169 + for (size_t i = 0; i < sizeof(size_units) / sizeof(size_units[0]); i++) {

170 + const struct size_unit *su = &size_units[i];

171 + if (su->base != su_def->base || // not the right base

172 + su->multiplier < su_def->multiplier || // the multiplier is too small

173 + (!su->formatter && su != su_def) || // it is not to be used in formatting (except our unit)

174 + (bytes < su->multiplier && su != su_def) ) // the converted value will be <1.0

175 + continue;

176 +

177 + double converted = size_round_to_resolution_dbl2(bytes, su->multiplier);

178 +

179 + uint64_t reversed_bytes = (uint64_t)(converted * (double)su->multiplier);

180 +

181 + if (reversed_bytes == bytes)

182 + // no precision loss, this is good to use

183 + su_best = su;

184 + }

185 +

186 + double converted = size_round_to_resolution_dbl2(bytes, su_best->multiplier);

187 +

188 + // print it either with 0, 1 or 2 fractional digits

189 + int written;

190 + if(converted == (double)((uint64_t)converted))

191 + written = snprintfz(dst, dst_size, "%.0f%s", converted, su_best->unit);

192 + else if(converted * 10.0 == (double)((uint64_t)(converted * 10.0)))

193 + written = snprintfz(dst, dst_size, "%.1f%s", converted, su_best->unit);

194 + else

195 + written = snprintfz(dst, dst_size, "%.2f%s", converted, su_best->unit);

196 +

197 + if (written < 0)

198 + return -4;

199 +

200 + if ((size_t)written >= dst_size)

201 + return (ssize_t)(dst_size - 1);

202 +

203 + return written;

204 +}

205 +

src/libnetdata/parsers/size.h new

+20

@@ -0,0 +1,20 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef LIBNETDATA_PARSERS_SIZE_H

4 +#define LIBNETDATA_PARSERS_SIZE_H

5 +

6 +#include "parsers.h"

7 +

8 +bool size_parse(const char *size_str, uint64_t *result, const char *default_unit);

9 +#define size_parse_bytes(size_str, bytes) size_parse(size_str, bytes, "B")

10 +#define size_parse_kb(size_str, kb) size_parse(size_str, kb, "KiB")

11 +#define size_parse_mb(size_str, mb) size_parse(size_str, mb, "MiB")

12 +#define size_parse_gb(size_str, gb) size_parse(size_str, gb, "GiB")

13 +

14 +ssize_t size_snprintf(char *dst, size_t dst_size, uint64_t value, const char *unit);

15 +#define size_snprintf_bytes(dst, dst_size, value) size_snprintf(dst, dst_size, value, "B")

16 +#define size_snprintf_kb(dst, dst_size, value) size_snprintf(dst, dst_size, value, "KiB")

17 +#define size_snprintf_mb(dst, dst_size, value) size_snprintf(dst, dst_size, value, "MiB")

18 +#define size_snprintf_gb(dst, dst_size, value) size_snprintf(dst, dst_size, value, "GiB")

19 +

20 +#endif //LIBNETDATA_PARSERS_SIZE_H

src/libnetdata/parsers/sizes.md new

+52

@@ -0,0 +1,52 @@

1 +## Data Sizes in Netdata

2 +

3 +Netdata provides a flexible system for specifying and formatting data sizes, used in various configurations and operations such as disk space management, and memory usage. This system allows users to specify data sizes in a human-readable format using multiple units from bytes to terabytes, supporting both binary (base-2) and decimal (base-10) standards. All units are UCUM-based for consistency and clarity.

4 +

5 +### Supported Size Units

6 +

7 +The following table lists all supported units and their corresponding values:

8 +

10 +|:------:|:-----------:|:---------:|:-------:|:---------:|

11 +| `B` | Bytes | `1B` | - | **Yes** |

12 +| `k` | Kilobytes | `1000B` | Base-10 | No |

13 +| `K` | Kilobytes | `1000B` | Base-10 | No |

14 +| `KB` | Kilobytes | `1000B` | Base-10 | No |

15 +| `KiB` | Kibibytes | `1024B` | Base-2 | **Yes** |

16 +| `M` | Megabytes | `1000K` | Base-10 | No |

17 +| `MB` | Megabytes | `1000K` | Base-10 | No |

18 +| `MiB` | Mebibytes | `1024KiB` | Base-2 | **Yes** |

19 +| `G` | Gigabytes | `1000M` | Base-10 | No |

20 +| `GB` | Gigabytes | `1000M` | Base-10 | No |

21 +| `GiB` | Gibibytes | `1024MiB` | Base-2 | **Yes** |

22 +| `T` | Terabytes | `1000G` | Base-10 | No |

23 +| `TB` | Terabytes | `1000G` | Base-10 | No |

24 +| `TiB` | Tebibytes | `1024GiB` | Base-2 | **Yes** |

25 +| `P` | Petabytes | `1000T` | Base-10 | No |

26 +| `PB` | Petabytes | `1000T` | Base-10 | No |

27 +| `PiB` | Pebibytes | `1024TiB` | Base-2 | **Yes** |

28 +

29 +### Size Expression Format

30 +

31 +Netdata allows users to express sizes using a number followed by a unit, such as `500MiB` (500 Mebibytes), `1GB` (1 Gigabyte), or `256K` (256 Kilobytes).

32 +

33 +- **Case Sensitivity**: Note that the parsing of units is case-sensitive.

34 +

35 +### Size Formatting

36 +

37 +Netdata formats a numeric size value (in bytes) into a human-readable string with an appropriate unit. The formatter's goal is to select the largest unit that can represent the size exactly, using up to two fractional digits. If two fractional digits are not enough to precisely represent the byte count, the formatter will use a smaller unit until it can accurately express the size, eventually falling back to bytes (`B`) if necessary.

38 +

39 +When formatting, Netdata prefers Base-2 units (`KiB`, `MiB`, `GiB`, etc.).

40 +

41 +- **Examples of Size Formatting**:

42 + - **10,485,760 bytes** is formatted as `10MiB` (10 Mebibytes).

43 + - **1,024 bytes** is formatted as `1KiB` (1 Kibibyte).

44 + - **1,500 bytes** remains formatted as `1500B` because it cannot be precisely represented in `KiB` or any larger unit using up to two fractional digits.

45 +

46 +### Example Size Expressions

47 +

48 +Here are some examples of valid size expressions:

49 +

50 +1. `1024B`: 1024 bytes.

51 +2. `1KiB`: 1024 bytes.

52 +3. `5MiB`: 5 mebibytes (5 * 1024 * 1024 bytes).

src/libnetdata/parsers/timeframe.c new

+128

@@ -0,0 +1,128 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "timeframe.h"

4 +

5 +// --------------------------------------------------------------------------------------------------------------------

6 +// timeframe

7 +/*

8 +TIMEFRAME timeframe_parse(const char *txt) {

9 + if(!txt || !*txt)

10 + return TIMEFRAME_INVALID;

11 +

12 +char buf[strlen(txt) + 1];

13 +memcpy(buf, txt, strlen(txt) + 1);

14 +char *s = trim_all(buf);

15 +if(!s)

16 + return TIMEFRAME_INVALID;

17 +

18 +while(isspace(*s)) s++;

19 +

20 +if(strcasecmp(s, "this minute") == 0) {

21 + return (TIMEFRAME) {

22 + .after = API_RELATIVE_TIME_THIS_MINUTE,

23 + .before = 0,

24 + };

25 +}

26 +if(strcasecmp(s, "this hour") == 0) {

27 + return (TIMEFRAME) {

28 + .after = API_RELATIVE_TIME_THIS_HOUR,

29 + .before = 0,

30 + };

31 +}

32 +if(strcasecmp(s, "today") == 0) {

33 + return (TIMEFRAME) {

34 + .after = API_RELATIVE_TIME_TODAY,

35 + .before = 0,

36 + };

37 +}

38 +if(strcasecmp(s, "this week") == 0) {

39 + return (TIMEFRAME) {

40 + .after = API_RELATIVE_TIME_THIS_WEEK,

41 + .before = 0,

42 + };

43 +}

44 +if(strcasecmp(s, "this month") == 0) {

45 + return (TIMEFRAME) {

46 + .after = API_RELATIVE_TIME_THIS_MONTH,

47 + .before = 0,

48 + };

49 +}

50 +if(strcasecmp(s, "this year") == 0) {

51 + return (TIMEFRAME) {

52 + .after = API_RELATIVE_TIME_THIS_YEAR,

53 + .before = 0,

54 + };

55 +}

56 +

57 +if(strcasecmp(s, "last minute") == 0) {

58 + return (TIMEFRAME) {

59 + .after = -60,

60 + .before = API_RELATIVE_TIME_THIS_MINUTE,

61 + };

62 +}

63 +if(strcasecmp(s, "last hour") == 0) {

64 + return (TIMEFRAME) {

65 + .after = -3600,

66 + .before = API_RELATIVE_TIME_THIS_HOUR,

67 + };

68 +}

69 +if(strcasecmp(s, "yesterday") == 0) {

70 + return (TIMEFRAME) {

71 + .after = -86400,

72 + .before = API_RELATIVE_TIME_TODAY,

73 + };

74 +}

75 +if(strcasecmp(s, "this week") == 0) {

76 + return (TIMEFRAME) {

77 + .after = -86400 * 7,

78 + .before = API_RELATIVE_TIME_THIS_WEEK,

79 + };

80 +}

81 +if(strcasecmp(s, "this month") == 0) {

82 + return (TIMEFRAME) {

83 + .after = API_RELATIVE_TIME_LAST_MONTH,

84 + .before = API_RELATIVE_TIME_THIS_MONTH,

85 + };

86 +}

87 +if(strcasecmp(s, "this year") == 0) {

88 + return (TIMEFRAME) {

89 + .after = API_RELATIVE_TIME_LAST_YEAR,

90 + .before = API_RELATIVE_TIME_THIS_YEAR,

91 + };

92 +}

93 +

94 +const char *end;

95 +double after = strtondd(s, (char **)&end);

96 +

97 +if(end == s)

98 + return TIMEFRAME_INVALID;

99 +

100 +s = end;

101 +while(isspace(*s)) s++;

102 +

103 +time_t multiplier = 1;

104 +if(!isdigit(*s) && *s != '-') {

105 + // after has units

106 + bool found = false;

107 +

108 + for (size_t i = 0; i < sizeof(units) / sizeof(units[0]); i++) {

109 + size_t len = strlen(units[i].unit);

110 +

111 + if (units[i].multiplier >= 1 * NSEC_PER_USEC &&

112 + strncmp(s, units[i].unit, len) == 0 &&

113 + (isspace(s[len]) || s[len] == '-')) {

114 + multiplier = units[i].multiplier / NSEC_PER_SEC;

115 + found = true;

116 + s += len;

117 + }

118 + }

119 +

120 + if(!found)

121 + return TIMEFRAME_INVALID;

122 +}

123 +

124 +const char *dash = strchr(s, '-');

125 +if(!dash) return TIMEFRAME_INVALID;

126 +

127 +}

128 +*/

src/libnetdata/parsers/timeframe.h new

+28

@@ -0,0 +1,28 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#ifndef NETDATA_TIMEFRAME_H

4 +#define NETDATA_TIMEFRAME_H

5 +

6 +#include "parsers.h"

7 +

8 +typedef struct {

9 + time_t after;

10 + time_t before;

11 +} TIMEFRAME;

12 +

13 +#define API_RELATIVE_TIME_MAX (3 * 365 * 86400)

14 +

15 +#define API_RELATIVE_TIME_INVALID (-1000000000)

16 +

17 +#define API_RELATIVE_TIME_THIS_MINUTE (API_RELATIVE_TIME_INVALID - 1) // this minute at 00 seconds

18 +#define API_RELATIVE_TIME_THIS_HOUR (API_RELATIVE_TIME_INVALID - 2) // this hour at 00 minutes, 00 seconds

19 +#define API_RELATIVE_TIME_TODAY (API_RELATIVE_TIME_INVALID - 3) // today at 00:00:00

20 +#define API_RELATIVE_TIME_THIS_WEEK (API_RELATIVE_TIME_INVALID - 4) // this Monday, 00:00:00

21 +#define API_RELATIVE_TIME_THIS_MONTH (API_RELATIVE_TIME_INVALID - 5) // this month's 1st at 00:00:00

22 +#define API_RELATIVE_TIME_THIS_YEAR (API_RELATIVE_TIME_INVALID - 6) // this year's Jan 1st, at 00:00:00

23 +#define API_RELATIVE_TIME_LAST_MONTH (API_RELATIVE_TIME_INVALID - 7) // last month's 1st, at 00:00:00

24 +#define API_RELATIVE_TIME_LAST_YEAR (API_RELATIVE_TIME_INVALID - 8) // last year's Jan 1st, at 00:00:00

25 +

26 +#define TIMEFRAME_INVALID (TIMEFRAME){ .after = API_RELATIVE_TIME_INVALID, .before = API_RELATIVE_TIME_INVALID }

27 +

28 +#endif //NETDATA_TIMEFRAME_H

src/libnetdata/required_dummies.h

+1 -1

@@ -20,6 +20,6 @@ void service_exits(void){}

20 void rrd_collector_finished(void){}

22 // required by get_system_cpus()

23 -char *netdata_configured_host_prefix = "";

23 +const char *netdata_configured_host_prefix = "";

25 #endif // NETDATA_LIB_DUMMIES_H

src/libnetdata/socket/security.c

+1 -1

@@ -730,7 +730,7 @@ int security_test_certificate(SSL *ssl) {

730 *

731 * @return It returns 0 on success and -1 otherwise.

732 */

733 -int ssl_security_location_for_context(SSL_CTX *ctx, char *file, char *path) {

733 +int ssl_security_location_for_context(SSL_CTX *ctx, const char *file, const char *path) {

734 int load_custom = 1, load_default = 1;

735 if (file || path) {

736 if(!SSL_CTX_load_verify_locations(ctx, file, path)) {

src/libnetdata/socket/security.h

+1 -1

@@ -50,7 +50,7 @@ extern const char *tls_version;

50 extern const char *tls_ciphers;

51 extern bool netdata_ssl_validate_certificate;

52 extern bool netdata_ssl_validate_certificate_sender;

53 -int ssl_security_location_for_context(SSL_CTX *ctx,char *file,char *path);

53 +int ssl_security_location_for_context(SSL_CTX *ctx, const char *file, const char *path);

55 void netdata_ssl_initialize_openssl();

56 void netdata_ssl_cleanup();

src/libnetdata/socket/socket.c

+2 -2

@@ -748,9 +748,9 @@ int listen_sockets_setup(LISTEN_SOCKETS *sockets) {

748 }

749 else sockets->default_port = (uint16_t)new_port;

750

751 - char *s = appconfig_get(sockets->config, sockets->config_section, "bind to", sockets->default_bind_to);

751 + const char *s = appconfig_get(sockets->config, sockets->config_section, "bind to", sockets->default_bind_to);

752 while(*s) {

753 - char *e = s;

753 + const char *e = s;

754

755 // skip separators, moving both s(tart) and e(nd)

756 while(isspace((uint8_t)*e) || *e == ',') s = ++e;

src/libnetdata/string/string.h

@@ -30,6 +30,10 @@ static inline int string_strcmp(STRING *string, const char *s) {

30 return strcmp(string2str(string), s);

31 }

33 +static inline int string_strncmp(STRING *string, const char *s, size_t n) {

34 + return strncmp(string2str(string), s, n);

35 +}

36 +

37 void string_statistics(size_t *inserts, size_t *deletes, size_t *searches, size_t *entries, size_t *references, size_t *memory, size_t *duplications, size_t *releases);

39 int string_unittest(size_t entries);

src/ml/Config.cc

+11 -8

@@ -27,29 +27,32 @@ void ml_config_load(ml_config_t *cfg) {

28 unsigned max_train_samples = config_get_number(config_section_ml, "maximum num samples to train", 6 * 3600);

29 unsigned min_train_samples = config_get_number(config_section_ml, "minimum num samples to train", 1 * 900);

30 - unsigned train_every = config_get_number(config_section_ml, "train every", 3 * 3600);

30 + unsigned train_every = config_get_duration_seconds(config_section_ml, "train every", 3 * 3600);

32 unsigned num_models_to_use = config_get_number(config_section_ml, "number of models per dimension", 18);

33 - unsigned delete_models_older_than = config_get_number(config_section_ml, "delete models older than", 60 * 60 * 24 * 7);

33 + unsigned delete_models_older_than = config_get_duration_seconds(config_section_ml, "delete models older than", 60 * 60 * 24 * 7);

35 unsigned diff_n = config_get_number(config_section_ml, "num samples to diff", 1);

36 unsigned smooth_n = config_get_number(config_section_ml, "num samples to smooth", 3);

37 unsigned lag_n = config_get_number(config_section_ml, "num samples to lag", 5);

39 - double random_sampling_ratio = config_get_float(config_section_ml, "random sampling ratio", 1.0 / 5.0 /* default lag_n */);

39 + double random_sampling_ratio = config_get_double(config_section_ml, "random sampling ratio", 1.0 / 5.0 /* default lag_n */);

40 unsigned max_kmeans_iters = config_get_number(config_section_ml, "maximum number of k-means iterations", 1000);

42 - double dimension_anomaly_rate_threshold = config_get_float(config_section_ml, "dimension anomaly score threshold", 0.99);

42 + double dimension_anomaly_rate_threshold = config_get_double(config_section_ml, "dimension anomaly score threshold", 0.99);

44 - double host_anomaly_rate_threshold = config_get_float(config_section_ml, "host anomaly rate threshold", 1.0);

44 + double host_anomaly_rate_threshold = config_get_double(config_section_ml, "host anomaly rate threshold", 1.0);

45 std::string anomaly_detection_grouping_method = config_get(config_section_ml, "anomaly detection grouping method", "average");

46 - time_t anomaly_detection_query_duration = config_get_number(config_section_ml, "anomaly detection grouping duration", 5 * 60);

46 + time_t anomaly_detection_query_duration = config_get_duration_seconds(config_section_ml, "anomaly detection grouping duration", 5 * 60);

48 size_t num_training_threads = config_get_number(config_section_ml, "num training threads", 4);

49 size_t flush_models_batch_size = config_get_number(config_section_ml, "flush models batch size", 128);

51 - size_t suppression_window = config_get_number(config_section_ml, "dimension anomaly rate suppression window", 900);

52 - size_t suppression_threshold = config_get_number(config_section_ml, "dimension anomaly rate suppression threshold", suppression_window / 2);

51 + size_t suppression_window =

52 + config_get_duration_seconds(config_section_ml, "dimension anomaly rate suppression window", 900);

53 +

54 + size_t suppression_threshold =

55 + config_get_number(config_section_ml, "dimension anomaly rate suppression threshold", suppression_window / 2);

57 bool enable_statistics_charts = config_get_boolean(config_section_ml, "enable statistics charts", false);

src/ml/ml-configuration.md

+8 -8

@@ -15,7 +15,7 @@ Below is a list of all the available configuration params and their default valu

15 # enabled = yes

16 # maximum num samples to train = 21600

17 # minimum num samples to train = 900

18 - # train every = 10800

18 + # train every = 3h

19 # number of models per dimension = 18

20 # dbengine anomaly rate every = 30

21 # num samples to diff = 1

@@ -26,12 +26,12 @@ Below is a list of all the available configuration params and their default valu

26 # dimension anomaly score threshold = 0.99

27 # host anomaly rate threshold = 1.0

28 # anomaly detection grouping method = average

29 - # anomaly detection grouping duration = 300

29 + # anomaly detection grouping duration = 5m

30 # hosts to skip from training = !*

31 # charts to skip from training = netdata.*

32 - # dimension anomaly rate suppression window = 900

32 + # dimension anomaly rate suppression window = 15m

33 # dimension anomaly rate suppression threshold = 450

34 - # delete models older than = 604800

34 + # delete models older than = 7d

35 ```

37 ## Configuration Examples

@@ -88,8 +88,8 @@ flowchart BT

88 - `enabled`: `yes` to enable, `no` to disable.

89 - `maximum num samples to train`: (`3600`/`86400`) This is the maximum amount of time you would like to train each model on. For example, the default of `21600` trains on the preceding 6 hours of data, assuming an `update every` of 1 second.

90 - `minimum num samples to train`: (`900`/`21600`) This is the minimum amount of data required to be able to train a model. For example, the default of `900` implies that once at least 15 minutes of data is available for training, a model is trained, otherwise it is skipped and checked again at the next training run.

91 -- `train every`: (`1800`/`21600`) This is how often each model will be retrained. For example, the default of `10800` means that each model is retrained every 3 hours. Note: The training of all models is spread out across the `train every` period for efficiency, so in reality, it means that each model will be trained in a staggered manner within each `train every` period.

92 -- `number of models per dimension`: (`1`/`168`) This is the number of trained models that will be used for scoring. For example the default `number of models per dimension = 18` means that the most recently trained 18 models for the dimension will be used to determine the corresponding anomaly bit. This means that under default settings of `maximum num samples to train = 21600`, `train every = 10800` and `number of models per dimension = 18`, netdata will store and use the last 18 trained models for each dimension when determining the anomaly bit. This means that for the latest feature vector in this configuration to be considered anomalous it would need to look anomalous across _all_ the models trained for that dimension in the last 18*(10800/3600) ~= 54 hours. As such, increasing `number of models per dimension` may reduce some false positives since it will result in more models (covering a wider time frame of training) being used during scoring.

91 +- `train every`: (`3h`/`6h`) This is how often each model will be retrained. For example, the default of `3h` means that each model is retrained every 3 hours. Note: The training of all models is spread out across the `train every` period for efficiency, so in reality, it means that each model will be trained in a staggered manner within each `train every` period.

92 +- `number of models per dimension`: (`1`/`168`) This is the number of trained models that will be used for scoring. For example the default `number of models per dimension = 18` means that the most recently trained 18 models for the dimension will be used to determine the corresponding anomaly bit. This means that under default settings of `maximum num samples to train = 21600`, `train every = 3h` and `number of models per dimension = 18`, netdata will store and use the last 18 trained models for each dimension when determining the anomaly bit. This means that for the latest feature vector in this configuration to be considered anomalous it would need to look anomalous across _all_ the models trained for that dimension in the last 18*(10800/3600) ~= 54 hours. As such, increasing `number of models per dimension` may reduce some false positives since it will result in more models (covering a wider time frame of training) being used during scoring.

93 - `dbengine anomaly rate every`: (`30`/`900`) This is how often netdata will aggregate all the anomaly bits into a single chart (`anomaly_detection.anomaly_rates`). The aggregation into a single chart allows enabling anomaly rate ranking over _all_ metrics with one API call as opposed to a call per chart.

94 - `num samples to diff`: (`0`/`1`) This is a `0` or `1` to determine if you want the model to operate on differences of the raw data or just the raw data. For example, the default of `1` means that we take differences of the raw values. Using differences is more general and works on dimensions that might naturally tend to have some trends or cycles in them that is normal behavior to which we don't want to be too sensitive.

95 - `num samples to smooth`: (`0`/`5`) This is a small integer that controls the amount of smoothing applied as part of the feature processing used by the model. For example, the default of `3` means that the rolling average of the last 3 values is used. Smoothing like this helps the model be a little more robust to spiky types of dimensions that naturally "jump" up or down as part of their normal behavior.

@@ -99,7 +99,7 @@ flowchart BT

99 - `dimension anomaly score threshold`: (`0.01`/`5.00`) This is the threshold at which an individual dimension at a specific timestep is considered anomalous or not. For example, the default of `0.99` means that a dimension with an anomaly score of 99% or higher is flagged as anomalous. This is a normalized probability based on the training data, so the default of 99% means that anything that is as strange (based on distance measure) or more strange as the most strange 1% of data observed during training will be flagged as anomalous. If you wanted to make the anomaly detection on individual dimensions more sensitive you could try a value like `0.90` (90%) or to make it less sensitive you could try `1.5` (150%).

100 - `host anomaly rate threshold`: (`0.1`/`10.0`) This is the percentage of dimensions (based on all those enabled for anomaly detection) that need to be considered anomalous at specific timestep for the host itself to be considered anomalous. For example, the default value of `1.0` means that if more than 1% of dimensions are anomalous at the same time then the host itself is considered in an anomalous state.

101 - `anomaly detection grouping method`: The grouping method used when calculating node level anomaly rate.

102 -- `anomaly detection grouping duration`: (`60`/`900`) The duration across which to calculate the node level anomaly rate, the default of `900` means that the node level anomaly rate is calculated across a rolling 5 minute window.

102 +- `anomaly detection grouping duration`: (`1m`/`15m`) The duration across which to calculate the node level anomaly rate, the default of `900` means that the node level anomaly rate is calculated across a rolling 5 minute window.

103 - `hosts to skip from training`: This parameter allows you to turn off anomaly detection for any child hosts on a parent host by defining those you would like to skip from training here. For example, a value like `dev-*` skips all hosts on a parent that begin with the "dev-" prefix. The default value of `!*` means "don't skip any".

104 - `charts to skip from training`: This parameter allows you to exclude certain charts from anomaly detection. By default, only netdata related charts are excluded. This is to avoid the scenario where accessing the netdata dashboard could itself trigger some anomalies if you don't access them regularly. If you want to include charts that are excluded by default, add them in small groups and then measure any impact on performance before adding additional ones. Example: If you want to include system, apps, and user charts:`!system.* !apps.* !user.* *`.

105 -- `delete models older than`: (`86400`/`604800`) Delete old models from the database that are unused, by default models will be deleted after 7 days.

105 +- `delete models older than`: (`1d`/`7d`) Delete old models from the database that are unused, by default models will be deleted after 7 days.

src/registry/registry.h

+1 -1

@@ -76,7 +76,7 @@ void registry_statistics(void);

77 const char *registry_get_this_machine_guid(void);

78 char *registry_get_mgmt_api_key(void);

79 -char *registry_get_this_machine_hostname(void);

79 +const char *registry_get_this_machine_hostname(void);

81 int regenerate_guid(const char *guid, char *result);

src/registry/registry_init.c

+1 -1

@@ -93,7 +93,7 @@ int registry_init(void) {

94 // configuration options

95 registry.save_registry_every_entries = (unsigned long long)config_get_number(CONFIG_SECTION_REGISTRY, "registry save db every new entries", 1000000);

96 - registry.persons_expiration = config_get_number(CONFIG_SECTION_REGISTRY, "registry expire idle persons days", 365) * 86400;

96 + registry.persons_expiration = config_get_duration_days(CONFIG_SECTION_REGISTRY, "registry expire idle persons", 365) * 86400;

97 registry.registry_domain = config_get(CONFIG_SECTION_REGISTRY, "registry domain", "");

98 registry.registry_to_announce = config_get(CONFIG_SECTION_REGISTRY, "registry to announce", "https://registry.my-netdata.io");

99 registry.hostname = config_get(CONFIG_SECTION_REGISTRY, "registry hostname", netdata_configured_hostname);

src/registry/registry_internals.c

+1 -1

@@ -266,7 +266,7 @@ static inline int is_machine_guid_blacklisted(const char *guid) {

266 return 0;

267 }

268

269 -char *registry_get_this_machine_hostname(void) {

269 +const char *registry_get_this_machine_hostname(void) {

270 return registry.hostname;

271 }

272

src/registry/registry_internals.h

+7 -7

@@ -30,9 +30,9 @@ struct registry {

31 // configuration

32 unsigned long long save_registry_every_entries;

33 - char *registry_domain;

34 - char *hostname;

35 - char *registry_to_announce;

33 + const char *registry_domain;

34 + const char *hostname;

35 + const char *registry_to_announce;

36 const char *cloud_base_url;

37 time_t persons_expiration; // seconds to expire idle persons

38 int verify_cookies_redirects;

@@ -42,10 +42,10 @@ struct registry {

42 size_t max_name_length;

44 // file/path names

45 - char *pathname;

46 - char *db_filename;

47 - char *log_filename;

48 - char *machine_guid_filename;

45 + const char *pathname;

46 + const char *db_filename;

47 + const char *log_filename;

48 + const char *machine_guid_filename;

50 // open files

51 FILE *log_fp;

src/streaming/README.md

+73 -47

@@ -30,6 +30,8 @@ node**. This file is automatically generated by Netdata the first time it is sta

31 #### `[stream]` section

33 +This section is used by the sending Netdata.

34 +

35 | Setting | Default | Description |

36 |-------------------------------------------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

37 | `enabled` | `no` | Whether this node streams metrics to any parent. Change to `yes` to enable streaming. |

@@ -38,34 +40,62 @@ node**. This file is automatically generated by Netdata the first time it is sta

40 | `CApath` | `/etc/ssl/certs/` | The directory where known certificates are found. Defaults to OpenSSL's default path. |

41 | `CAfile` | `/etc/ssl/certs/cert.pem` | Add a parent node certificate to the list of known certificates in `CAPath`. |

42 | `api key` | | The `API_KEY` to use as the child node. |

41 -| `timeout seconds` | `60` | The timeout to connect and send metrics to a parent. |

43 +| `timeout` | `1m` | The timeout to connect and send metrics to a parent. |

44 | `default port` | `19999` | The port to use if `destination` does not specify one. |

45 | [`send charts matching`](#send-charts-matching) | `*` | A space-separated list of [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) to filter which charts are streamed. [Read more →](#send-charts-matching) |

46 | `buffer size bytes` | `10485760` | The size of the buffer to use when sending metrics. The default `10485760` equals a buffer of 10MB, which is good for 60 seconds of data. Increase this if you expect latencies higher than that. The buffer is flushed on reconnect. |

45 -| `reconnect delay seconds` | `5` | How long to wait until retrying to connect to the parent node. |

47 +| `reconnect delay` | `5s` | How long to wait until retrying to connect to the parent node. |

48 | `initial clock resync iterations` | `60` | Sync the clock of charts for how many seconds when starting. |

49 | `parent using h2o` | `no` | Set to yes if you are connecting to parent trough it's h2o webserver/port. Currently there is no reason to set this to `yes` unless you are testing the new h2o based netdata webserver. When production ready this will be set to `yes` as default. |

49 -### `[API_KEY]` and `[MACHINE_GUID]` sections

50 -

51 -| Setting | Default | Description |

52 -|-----------------------------------------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

53 -| `enabled` | `no` | Whether this API KEY enabled or disabled. |

54 -| [`allow from`](#allow-from) | `*` | A space-separated list of [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) matching the IPs of nodes that will stream metrics using this API key. [Read more →](#allow-from) |

55 -| `default history` | `3600` | The default amount of child metrics history to retain when using the `ram` memory mode. |

56 -| [`default memory mode`](#default-memory-mode) | `ram` | The [database](/src/database/README.md) to use for all nodes using this `API_KEY`. Valid settings are `dbengine`, `ram`, or `none`. [Read more →](#default-memory-mode) |

57 -| `health enabled by default` | `auto` | Whether alerts and notifications should be enabled for nodes using this `API_KEY`. `auto` enables alerts when the child is connected. `yes` enables alerts always, and `no` disables alerts. |

58 -| `default postpone alarms on connect seconds` | `60` | Postpone alerts and notifications for a period of time after the child connects. |

59 -| `default health log history` | `432000` | History of health log events (in seconds) kept in the database. |

60 -| `default proxy enabled` | | Route metrics through a proxy. |

61 -| `default proxy destination` | | Space-separated list of `IP:PORT` for proxies. |

62 -| `default proxy api key` | | The `API_KEY` of the proxy. |

63 -| `default send charts matching` | `*` | See [`send charts matching`](#send-charts-matching). |

64 -| `enable compression` | `yes` | Enable/disable stream compression. |

65 -| `enable replication` | `yes` | Enable/disable replication. |

66 -| `seconds to replicate` | `86400` | How many seconds of data to replicate from each child at a time |

67 -| `seconds per replication step` | `600` | The duration we want to replicate per each replication step. |

68 -| `is ephemeral node` | `no` | Indicate whether this child is an ephemeral node. An ephemeral node will become unavailable after the specified duration of "cleanup ephemeral hosts after secs" from the time of the node's last connection. |

51 +### `[API_KEY]` sections

52 +

53 +This section defines an API key for other agents to connect to this Netdata.

54 +

55 +| Setting | Default | Description |

56 +|------------------------------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

57 +| `enabled` | `no` | Whether this API KEY enabled or disabled. |

58 +| `type` | `api` | This section defines an API key. |

59 +| [`allow from`](#allow-from) | `*` | A space-separated list of [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) matching the IPs of nodes that will stream metrics using this API key. [Read more →](#allow-from) |

60 +| `retention` | `1h` | The default amount of child metrics history to retain when using the `ram` db. |

61 +| [`db`](#default-memory-mode) | `dbengine` | The [database](/src/database/README.md) to use for all nodes using this `API_KEY`. Valid settings are `dbengine`, `ram`, or `none`. [Read more →](#default-memory-mode) |

62 +| `health enabled by default` | `auto` | Whether alerts and notifications should be enabled for nodes using this `API_KEY`. `auto` enables alerts when the child is connected. `yes` enables alerts always, and `no` disables alerts. |

63 +| `postpone alerts on connect` | `1m` | Postpone alerts and notifications for a period of time after the child connects. |

64 +| `health log retention` | `5d` | History of health log events (in seconds) kept in the database. |

65 +| `proxy enabled` | | Route metrics through a proxy. |

66 +| `proxy destination` | | Space-separated list of `IP:PORT` for proxies. |

67 +| `proxy api key` | | The `API_KEY` of the proxy. |

68 +| `send charts matching` | `*` | See [`send charts matching`](#send-charts-matching). |

69 +| `enable compression` | `yes` | Enable/disable stream compression. |

70 +| `enable replication` | `yes` | Enable/disable replication. |

71 +| `replication period` | `1d` | Limits the maximum window that will be replicated from each child. |

72 +| `replication step` | `10m` | The duration we want to replicate per each replication step. |

73 +| `is ephemeral node` | `no` | Indicate whether this child is an ephemeral node. An ephemeral node will become unavailable after the specified duration of "cleanup ephemeral hosts after" from the time of the node's last connection. |

74 +

75 +

76 +### `[MACHINE_GUID]` sections

77 +

78 +This section is about customizing configuration for specific agents. It allows many agents to share the same API key, while providing customizability per remote agent.

79 +

80 +| Setting | Default | Description |

81 +|------------------------------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

82 +| `enabled` | `no` | Whether this MACHINE_GUID enabled or disabled. |

83 +| `type` | `machine` | This section defines the configuration for a specific agent. |

84 +| [`allow from`](#allow-from) | `*` | A space-separated list of [Netdata simple patterns](/src/libnetdata/simple_pattern/README.md) matching the IPs of nodes that will stream metrics using this API key. [Read more →](#allow-from) |

85 +| `retention` | `3600` | The default amount of child metrics history to retain when using the `ram` db. |

86 +| [`db`](#default-memory-mode) | `dbengine` | The [database](/src/database/README.md) to use for all nodes using this `API_KEY`. Valid settings are `dbengine`, `ram`, or `none`. [Read more →](#default-memory-mode) |

87 +| `health enabled` | `auto` | Whether alerts and notifications should be enabled for nodes using this `API_KEY`. `auto` enables alerts when the child is connected. `yes` enables alerts always, and `no` disables alerts. |

88 +| `postpone alerts on connect` | `1m` | Postpone alerts and notifications for a period of time after the child connects. |

89 +| `health log retention` | `5d` | History of health log events (in seconds) kept in the database. |

90 +| `proxy enabled` | | Route metrics through a proxy. |

91 +| `proxy destination` | | Space-separated list of `IP:PORT` for proxies. |

92 +| `proxy api key` | | The `API_KEY` of the proxy. |

93 +| `send charts matching` | `*` | See [`send charts matching`](#send-charts-matching). |

94 +| `enable compression` | `yes` | Enable/disable stream compression. |

95 +| `enable replication` | `yes` | Enable/disable replication. |

96 +| `replication period` | `1d` | Limits the maximum window that will be replicated from each child. |

97 +| `replication step` | `10m` | The duration we want to replicate per each replication step. |

98 +| `is ephemeral node` | `no` | Indicate whether this child is an ephemeral node. An ephemeral node will become unavailable after the specified duration of "cleanup ephemeral hosts after" from the time of the node's last connection. |

100 #### `destination`

101

@@ -131,7 +161,7 @@ To allow all IPs starting with `10.*`, except `10.1.2.3`:

161 > If you set specific IP addresses here, and also use the `allow connections` setting in the `[web]` section of

162 > `netdata.conf`, be sure to add the IP address there so that it can access the API port.

163

134 -#### `default memory mode`

164 +#### `db`

165

166 The [database](/src/database/README.md) to use for all nodes using this `API_KEY`.

167 Valid settings are `dbengine`, `ram`, , or `none`.

@@ -142,19 +172,15 @@ Valid settings are `dbengine`, `ram`, , or `none`.

172 streaming configurations that use ephemeral nodes.

173 - `none`: No database.

174

145 -When using `default memory mode = dbengine`, the parent node creates a separate instance of the TSDB to store metrics

146 -from child nodes. The [size of _each_ instance is configurable](/docs/netdata-agent/configuration/optimizing-metrics-database/change-metrics-storage.md) with the `page

147 -cache size` and `dbengine multihost disk space` settings in the `[global]` section in `netdata.conf`.

148 -

175 ### `netdata.conf`

176

151 -| Setting | Default | Description |

152 -|--------------------------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

153 -| `[global]` section | | |

154 -| `memory mode` | `dbengine` | Determines the [database type](/src/database/README.md) to be used on that node. Other options settings include `none`, and `ram`. `none` disables the database at this host. This also disables alerts and notifications, as those can't run without a database. |

155 -| `[web]` section | | |

156 -| `mode` | `static-threaded` | Determines the [web server](/src/web/server/README.md) type. The other option is `none`, which disables the dashboard, API, and registry. |

157 -| `accept a streaming request every seconds` | `0` | Set a limit on how often a parent node accepts streaming requests from child nodes. `0` equals no limit. If this is set, you may see `... too busy to accept new streaming request. Will be allowed in X secs` in Netdata's `error.log`. |

177 +| Setting | Default | Description |

178 +|------------------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

179 +| `[db]` section | | |

180 +| `mode` | `dbengine` | Determines the [database type](/src/database/README.md) to be used on that node. Other options settings include `none`, and `ram`. `none` disables the database at this host. This also disables alerts and notifications, as those can't run without a database. |

181 +| `[web]` section | | |

182 +| `mode` | `static-threaded` | Determines the [web server](/src/web/server/README.md) type. The other option is `none`, which disables the dashboard, API, and registry. |

183 +| `accept a streaming request every` | `off` | Set a limit on how often a parent node accepts streaming requests from child nodes. `0` equals no limit. If this is set, you may see `... too busy to accept new streaming request. Will be allowed in X secs` in Netdata's `error.log`. |

184

185 ### Basic use cases

186

@@ -178,13 +204,13 @@ default `dbengine` as specified by the `API_KEY`, and alerts are disabled.

204 ```conf

205 [API_KEY]

206 enabled = yes

181 - default memory mode = dbengine

182 - health enabled by default = auto

207 + db = dbengine

208 + health enabled = auto

209 allow from = *

210

211 [MACHINE_GUID]

212 enabled = yes

187 - memory mode = ram

213 + db = ram

214 health enabled = no

215 ```

216

@@ -413,7 +439,7 @@ In the following example, the proxy receives metrics from a child node using the

439

440 [66666666-7777-8888-9999-000000000000]

441 enabled = yes

416 - default memory mode = dbengine

442 + db = dbengine

443 ```

444

445 ### Ephemeral nodes

@@ -429,7 +455,7 @@ On the parent, set the following in `stream.conf`:

455 enabled = yes

456

457 # one hour of data for each of the child nodes

432 - default history = 3600

458 + history = 1h

459

460 # do not save child metrics on disk

461 default memory = ram

@@ -455,9 +481,9 @@ On the child nodes, set the following in `stream.conf`:

481 In addition, edit `netdata.conf` on each child node to disable the database and alerts.

482

483 ```bash

458 -[global]

484 +[db]

485 # disable the local database

460 - memory mode = none

486 + db = none

487

488 [health]

489 # disable health checks

@@ -476,11 +502,11 @@ Replication is enabled by default in Netdata, but you can customize the replicat

502 # Enable replication for all hosts using this api key. Default: yes.

503 enable replication = yes

504

479 - # How many seconds of data to replicate from each child at a time. Default: a day (86400 seconds).

480 - seconds to replicate = 86400

505 + # How many seconds of data to replicate from each child at a time. Default: a day.

506 + replication period = 1d

507

482 - # The duration we want to replicate per each replication step. Default: 600 seconds (10 minutes).

483 - seconds per replication step = 600

508 + # The duration we want to replicate per each replication step. Default: 10 minutes.

509 + replication step = 10m

510 ```

511

512 You can monitor the replication process in two ways:

@@ -597,9 +623,9 @@ ERROR : STREAM_SENDER[CHILD HOSTNAME] : STREAM child HOSTNAME [send to PARENT HO

623 ### Stream charts wrong

624

625 Chart data needs to be consistent between child and parent nodes. If there are differences between chart data on

600 -a parent and a child, such as gaps in metrics collection, it most often means your child's `memory mode`

626 +a parent and a child, such as gaps in metrics collection, it most often means your child's `[db].db` setting

627 does not match the parent's. To learn more about the different ways Netdata can store metrics, and thus keep chart

602 -data consistent, read our [memory mode documentation](/src/database/README.md).

628 +data consistent, read our [db documentation](/src/database/README.md).

629

630 ### Forbidding access

631

src/streaming/receiver.c

+33 -35

@@ -6,19 +6,6 @@

6 extern struct config stream_config;

8 void receiver_state_free(struct receiver_state *rpt) {

9 -

10 - freez(rpt->key);

11 - freez(rpt->hostname);

12 - freez(rpt->registry_hostname);

13 - freez(rpt->machine_guid);

14 - freez(rpt->os);

15 - freez(rpt->timezone);

16 - freez(rpt->abbrev_timezone);

17 - freez(rpt->client_ip);

18 - freez(rpt->client_port);

19 - freez(rpt->program_name);

20 - freez(rpt->program_version);

21 -

9 netdata_ssl_close(&rpt->ssl);

11 if(rpt->fd != -1) {

@@ -29,10 +16,21 @@ void receiver_state_free(struct receiver_state *rpt) {

16 rrdpush_decompressor_destroy(&rpt->decompressor);

18 if(rpt->system_info)

32 - rrdhost_system_info_free(rpt->system_info);

19 + rrdhost_system_info_free(rpt->system_info);

21 __atomic_sub_fetch(&netdata_buffers_statistics.rrdhost_receivers, sizeof(*rpt), __ATOMIC_RELAXED);

23 + freez(rpt->key);

24 + freez(rpt->hostname);

25 + freez(rpt->registry_hostname);

26 + freez(rpt->machine_guid);

27 + freez(rpt->os);

28 + freez(rpt->timezone);

29 + freez(rpt->abbrev_timezone);

30 + freez(rpt->client_ip);

31 + freez(rpt->client_port);

32 + freez(rpt->program_name);

33 + freez(rpt->program_version);

34 freez(rpt);

35 }

@@ -433,7 +431,7 @@ static bool rrdhost_set_receiver(RRDHOST *host, struct receiver_state *rpt) {

431 }

432 }

433

436 - host->health_log.health_log_history = rpt->config.alarms_history;

434 + host->health_log.health_log_retention_s = rpt->config.alarms_history;

435

436 // this is a test

437 // if(rpt->hops <= host->sender->hops)

@@ -589,7 +587,7 @@ static void rrdpush_receive(struct receiver_state *rpt)

587

588 rpt->config.health_enabled = health_plugin_enabled();

589 rpt->config.alarms_delay = 60;

592 - rpt->config.alarms_history = HEALTH_LOG_DEFAULT_HISTORY;

590 + rpt->config.alarms_history = HEALTH_LOG_RETENTION_DEFAULT;

591

592 rpt->config.rrdpush_enabled = (int)default_rrdpush_enabled;

593 rpt->config.rrdpush_destination = default_rrdpush_destination;

@@ -600,15 +598,15 @@ static void rrdpush_receive(struct receiver_state *rpt)

598 rpt->config.rrdpush_seconds_to_replicate = default_rrdpush_seconds_to_replicate;

599 rpt->config.rrdpush_replication_step = default_rrdpush_replication_step;

600

603 - rpt->config.update_every = (int)appconfig_get_number(&stream_config, rpt->machine_guid, "update every", rpt->config.update_every);

601 + rpt->config.update_every = (int)appconfig_get_duration_seconds(&stream_config, rpt->machine_guid, "update every", rpt->config.update_every);

602 if(rpt->config.update_every < 0) rpt->config.update_every = 1;

603

606 - rpt->config.history = (int)appconfig_get_number(&stream_config, rpt->key, "default history", rpt->config.history);

607 - rpt->config.history = (int)appconfig_get_number(&stream_config, rpt->machine_guid, "history", rpt->config.history);

604 + rpt->config.history = (int)appconfig_get_number(&stream_config, rpt->key, "retention", rpt->config.history);

605 + rpt->config.history = (int)appconfig_get_number(&stream_config, rpt->machine_guid, "retention", rpt->config.history);

606 if(rpt->config.history < 5) rpt->config.history = 5;

607

610 - rpt->config.mode = rrd_memory_mode_id(appconfig_get(&stream_config, rpt->key, "default memory mode", rrd_memory_mode_name(rpt->config.mode)));

611 - rpt->config.mode = rrd_memory_mode_id(appconfig_get(&stream_config, rpt->machine_guid, "memory mode", rrd_memory_mode_name(rpt->config.mode)));

608 + rpt->config.mode = rrd_memory_mode_id(appconfig_get(&stream_config, rpt->key, "db", rrd_memory_mode_name(rpt->config.mode)));

609 + rpt->config.mode = rrd_memory_mode_id(appconfig_get(&stream_config, rpt->machine_guid, "db", rrd_memory_mode_name(rpt->config.mode)));

610

611 if (unlikely(rpt->config.mode == RRD_MEMORY_MODE_DBENGINE && !dbengine_enabled)) {

612 netdata_log_error("STREAM '%s' [receive from %s:%s]: "

@@ -623,32 +621,32 @@ static void rrdpush_receive(struct receiver_state *rpt)

621 rpt->config.health_enabled = appconfig_get_boolean_ondemand(&stream_config, rpt->key, "health enabled by default", rpt->config.health_enabled);

622 rpt->config.health_enabled = appconfig_get_boolean_ondemand(&stream_config, rpt->machine_guid, "health enabled", rpt->config.health_enabled);

623

626 - rpt->config.alarms_delay = appconfig_get_number(&stream_config, rpt->key, "default postpone alarms on connect seconds", rpt->config.alarms_delay);

627 - rpt->config.alarms_delay = appconfig_get_number(&stream_config, rpt->machine_guid, "postpone alarms on connect seconds", rpt->config.alarms_delay);

624 + rpt->config.alarms_delay = appconfig_get_duration_seconds(&stream_config, rpt->key, "postpone alerts on connect", rpt->config.alarms_delay);

625 + rpt->config.alarms_delay = appconfig_get_duration_seconds(&stream_config, rpt->machine_guid, "postpone alerts on connect", rpt->config.alarms_delay);

626

629 - rpt->config.alarms_history = appconfig_get_number(&stream_config, rpt->key, "default health log history", rpt->config.alarms_history);

630 - rpt->config.alarms_history = appconfig_get_number(&stream_config, rpt->machine_guid, "health log history", rpt->config.alarms_history);

627 + rpt->config.alarms_history = appconfig_get_duration_seconds(&stream_config, rpt->key, "health log retention", rpt->config.alarms_history);

628 + rpt->config.alarms_history = appconfig_get_duration_seconds(&stream_config, rpt->machine_guid, "health log retention", rpt->config.alarms_history);

629

632 - rpt->config.rrdpush_enabled = appconfig_get_boolean(&stream_config, rpt->key, "default proxy enabled", rpt->config.rrdpush_enabled);

630 + rpt->config.rrdpush_enabled = appconfig_get_boolean(&stream_config, rpt->key, "proxy enabled", rpt->config.rrdpush_enabled);

631 rpt->config.rrdpush_enabled = appconfig_get_boolean(&stream_config, rpt->machine_guid, "proxy enabled", rpt->config.rrdpush_enabled);

632

635 - rpt->config.rrdpush_destination = appconfig_get(&stream_config, rpt->key, "default proxy destination", rpt->config.rrdpush_destination);

633 + rpt->config.rrdpush_destination = appconfig_get(&stream_config, rpt->key, "proxy destination", rpt->config.rrdpush_destination);

634 rpt->config.rrdpush_destination = appconfig_get(&stream_config, rpt->machine_guid, "proxy destination", rpt->config.rrdpush_destination);

635

638 - rpt->config.rrdpush_api_key = appconfig_get(&stream_config, rpt->key, "default proxy api key", rpt->config.rrdpush_api_key);

636 + rpt->config.rrdpush_api_key = appconfig_get(&stream_config, rpt->key, "proxy api key", rpt->config.rrdpush_api_key);

637 rpt->config.rrdpush_api_key = appconfig_get(&stream_config, rpt->machine_guid, "proxy api key", rpt->config.rrdpush_api_key);

638

641 - rpt->config.rrdpush_send_charts_matching = appconfig_get(&stream_config, rpt->key, "default proxy send charts matching", rpt->config.rrdpush_send_charts_matching);

639 + rpt->config.rrdpush_send_charts_matching = appconfig_get(&stream_config, rpt->key, "proxy send charts matching", rpt->config.rrdpush_send_charts_matching);

640 rpt->config.rrdpush_send_charts_matching = appconfig_get(&stream_config, rpt->machine_guid, "proxy send charts matching", rpt->config.rrdpush_send_charts_matching);

641

642 rpt->config.rrdpush_enable_replication = appconfig_get_boolean(&stream_config, rpt->key, "enable replication", rpt->config.rrdpush_enable_replication);

643 rpt->config.rrdpush_enable_replication = appconfig_get_boolean(&stream_config, rpt->machine_guid, "enable replication", rpt->config.rrdpush_enable_replication);

644

647 - rpt->config.rrdpush_seconds_to_replicate = appconfig_get_number(&stream_config, rpt->key, "seconds to replicate", rpt->config.rrdpush_seconds_to_replicate);

648 - rpt->config.rrdpush_seconds_to_replicate = appconfig_get_number(&stream_config, rpt->machine_guid, "seconds to replicate", rpt->config.rrdpush_seconds_to_replicate);

645 + rpt->config.rrdpush_seconds_to_replicate = appconfig_get_duration_seconds(&stream_config, rpt->key, "replication period", rpt->config.rrdpush_seconds_to_replicate);

646 + rpt->config.rrdpush_seconds_to_replicate = appconfig_get_duration_seconds(&stream_config, rpt->machine_guid, "replication period", rpt->config.rrdpush_seconds_to_replicate);

647

650 - rpt->config.rrdpush_replication_step = appconfig_get_number(&stream_config, rpt->key, "seconds per replication step", rpt->config.rrdpush_replication_step);

651 - rpt->config.rrdpush_replication_step = appconfig_get_number(&stream_config, rpt->machine_guid, "seconds per replication step", rpt->config.rrdpush_replication_step);

648 + rpt->config.rrdpush_replication_step = appconfig_get_number(&stream_config, rpt->key, "replication step", rpt->config.rrdpush_replication_step);

649 + rpt->config.rrdpush_replication_step = appconfig_get_number(&stream_config, rpt->machine_guid, "replication step", rpt->config.rrdpush_replication_step);

650

651 rpt->config.rrdpush_compression = default_rrdpush_compression_enabled;

652 rpt->config.rrdpush_compression = appconfig_get_boolean(&stream_config, rpt->key, "enable compression", rpt->config.rrdpush_compression);

@@ -659,7 +657,7 @@ static void rrdpush_receive(struct receiver_state *rpt)

657 is_ephemeral = appconfig_get_boolean(&stream_config, rpt->machine_guid, "is ephemeral node", is_ephemeral);

658

659 if(rpt->config.rrdpush_compression) {

662 - char *order = appconfig_get(&stream_config, rpt->key, "compression algorithms order", RRDPUSH_COMPRESSION_ALGORITHMS_ORDER);

660 + const char *order = appconfig_get(&stream_config, rpt->key, "compression algorithms order", RRDPUSH_COMPRESSION_ALGORITHMS_ORDER);

661 order = appconfig_get(&stream_config, rpt->machine_guid, "compression algorithms order", order);

662 rrdpush_parse_compression_order(rpt, order);

663 }

@@ -921,7 +919,7 @@ void *rrdpush_receiver_thread(void *ptr) {

919

920 worker_unregister();

921 rrdhost_clear_receiver(rpt);

924 - receiver_state_free(rpt);

922 rrdhost_set_is_parent_label();

923 + receiver_state_free(rpt);

924 return NULL;

925 }

src/streaming/rrdpush.c

+58 -66

@@ -25,30 +25,19 @@

25 *

26 */

28 -struct config stream_config = {

29 - .first_section = NULL,

30 - .last_section = NULL,

31 - .mutex = NETDATA_MUTEX_INITIALIZER,

32 - .index = {

33 - .avl_tree = {

34 - .root = NULL,

35 - .compar = appconfig_section_compare

36 - },

37 - .rwlock = AVL_LOCK_INITIALIZER

38 - }

39 -};

28 +struct config stream_config = APPCONFIG_INITIALIZER;

30 unsigned int default_rrdpush_enabled = 0;

32 unsigned int default_rrdpush_compression_enabled = 1;

44 -char *default_rrdpush_destination = NULL;

45 -char *default_rrdpush_api_key = NULL;

46 -char *default_rrdpush_send_charts_matching = "*";

33 +const char *default_rrdpush_destination = NULL;

34 +const char *default_rrdpush_api_key = NULL;

35 +const char *default_rrdpush_send_charts_matching = "*";

36 bool default_rrdpush_enable_replication = true;

37 time_t default_rrdpush_seconds_to_replicate = 86400;

38 time_t default_rrdpush_replication_step = 600;

50 -char *netdata_ssl_ca_path = NULL;

51 -char *netdata_ssl_ca_file = NULL;

39 +const char *netdata_ssl_ca_path = NULL;

40 +const char *netdata_ssl_ca_file = NULL;

42 static void load_stream_conf() {

43 errno_clear();

@@ -61,32 +50,36 @@ static void load_stream_conf() {

50 if(!appconfig_load(&stream_config, filename, 0, NULL))

51 nd_log_daemon(NDLP_NOTICE, "CONFIG: cannot load stock config '%s'. Running with internal defaults.", filename);

52 }

64 - freez(filename);

65 -}

66 -

67 -bool rrdpush_receiver_needs_dbengine() {

68 - struct section *co;

69 -

70 - for(co = stream_config.first_section; co; co = co->next) {

71 - if(strcmp(co->name, "stream") == 0)

72 - continue; // the first section is not relevant

74 - char *s;

75 -

76 - s = appconfig_get_by_section(co, "enabled", NULL);

77 - if(!s || !appconfig_test_boolean_value(s))

78 - continue;

79 -

80 - s = appconfig_get_by_section(co, "default memory mode", NULL);

81 - if(s && strcmp(s, "dbengine") == 0)

82 - return true;

54 + freez(filename);

84 - s = appconfig_get_by_section(co, "memory mode", NULL);

85 - if(s && strcmp(s, "dbengine") == 0)

86 - return true;

87 - }

56 + appconfig_move(&stream_config,

57 + CONFIG_SECTION_STREAM, "timeout seconds",

58 + CONFIG_SECTION_STREAM, "timeout");

59 +

60 + appconfig_move(&stream_config,

61 + CONFIG_SECTION_STREAM, "reconnect delay seconds",

62 + CONFIG_SECTION_STREAM, "reconnect delay");

63 +

64 + appconfig_move_everywhere(&stream_config, "default memory mode", "db");

65 + appconfig_move_everywhere(&stream_config, "memory mode", "db");

66 + appconfig_move_everywhere(&stream_config, "db mode", "db");

67 + appconfig_move_everywhere(&stream_config, "default history", "retention");

68 + appconfig_move_everywhere(&stream_config, "history", "retention");

69 + appconfig_move_everywhere(&stream_config, "default proxy enabled", "proxy enabled");

70 + appconfig_move_everywhere(&stream_config, "default proxy destination", "proxy destination");

71 + appconfig_move_everywhere(&stream_config, "default proxy api key", "proxy api key");

72 + appconfig_move_everywhere(&stream_config, "default proxy send charts matching", "proxy send charts matching");

73 + appconfig_move_everywhere(&stream_config, "default health log history", "health log retention");

74 + appconfig_move_everywhere(&stream_config, "health log history", "health log retention");

75 + appconfig_move_everywhere(&stream_config, "seconds to replicate", "replication period");

76 + appconfig_move_everywhere(&stream_config, "seconds per replication step", "replication step");

77 + appconfig_move_everywhere(&stream_config, "default postpone alarms on connect seconds", "postpone alerts on connect");

78 + appconfig_move_everywhere(&stream_config, "postpone alarms on connect seconds", "postpone alerts on connect");

79 +}

89 - return false;

81 +bool rrdpush_receiver_needs_dbengine(void) {

82 + return stream_conf_needs_dbengine(&stream_config);

83 }

85 int rrdpush_init() {

@@ -94,19 +87,33 @@ int rrdpush_init() {

87 // load stream.conf

88 load_stream_conf();

97 - default_rrdpush_enabled = (unsigned int)appconfig_get_boolean(&stream_config, CONFIG_SECTION_STREAM, "enabled", default_rrdpush_enabled);

98 - default_rrdpush_destination = appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "destination", "");

99 - default_rrdpush_api_key = appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "api key", "");

100 - default_rrdpush_send_charts_matching = appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "send charts matching", default_rrdpush_send_charts_matching);

90 + default_rrdpush_enabled =

91 + (unsigned int)appconfig_get_boolean(&stream_config, CONFIG_SECTION_STREAM, "enabled", default_rrdpush_enabled);

102 - default_rrdpush_enable_replication = config_get_boolean(CONFIG_SECTION_DB, "enable replication", default_rrdpush_enable_replication);

103 - default_rrdpush_seconds_to_replicate = config_get_number(CONFIG_SECTION_DB, "seconds to replicate", default_rrdpush_seconds_to_replicate);

104 - default_rrdpush_replication_step = config_get_number(CONFIG_SECTION_DB, "seconds per replication step", default_rrdpush_replication_step);

93 + default_rrdpush_destination =

94 + appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "destination", "");

106 - rrdhost_free_orphan_time_s = config_get_number(CONFIG_SECTION_DB, "cleanup orphan hosts after secs", rrdhost_free_orphan_time_s);

96 + default_rrdpush_api_key =

97 + appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "api key", "");

108 - default_rrdpush_compression_enabled = (unsigned int)appconfig_get_boolean(&stream_config, CONFIG_SECTION_STREAM,

109 - "enable compression", default_rrdpush_compression_enabled);

99 + default_rrdpush_send_charts_matching =

100 + appconfig_get(&stream_config, CONFIG_SECTION_STREAM, "send charts matching", default_rrdpush_send_charts_matching);

101 +

102 + default_rrdpush_enable_replication =

103 + config_get_boolean(CONFIG_SECTION_DB, "enable replication", default_rrdpush_enable_replication);

104 +

105 + default_rrdpush_seconds_to_replicate =

106 + config_get_duration_seconds(CONFIG_SECTION_DB, "replication period", default_rrdpush_seconds_to_replicate);

107 +

108 + default_rrdpush_replication_step =

109 + config_get_duration_seconds(CONFIG_SECTION_DB, "replication step", default_rrdpush_replication_step);

110 +

111 + rrdhost_free_orphan_time_s =

112 + config_get_duration_seconds(CONFIG_SECTION_DB, "cleanup orphan hosts after", rrdhost_free_orphan_time_s);

113 +

114 + default_rrdpush_compression_enabled =

115 + (unsigned int)appconfig_get_boolean(&stream_config, CONFIG_SECTION_STREAM,

116 + "enable compression", default_rrdpush_compression_enabled);

117

118 rrdpush_compression_levels[COMPRESSION_ALGORITHM_BROTLI] = (int)appconfig_get_number(

119 &stream_config, CONFIG_SECTION_STREAM, "brotli compression level",

@@ -201,22 +208,7 @@ static inline bool should_send_chart_matching(RRDSET *st, RRDSET_FLAGS flags) {

208 }

209

210 int configured_as_parent() {

204 - struct section *section = NULL;

205 - int is_parent = 0;

206 -

207 - appconfig_wrlock(&stream_config);

208 - for (section = stream_config.first_section; section; section = section->next) {

209 - nd_uuid_t uuid;

210 -

211 - if (uuid_parse(section->name, uuid) != -1 &&

212 - appconfig_get_boolean_by_section(section, "enabled", 0)) {

213 - is_parent = 1;

214 - break;

215 - }

216 - }

217 - appconfig_unlock(&stream_config);

218 -

219 - return is_parent;

211 + return stream_conf_has_uuid_section(&stream_config);

212 }

213

214 // chart labels

src/streaming/rrdpush.h

+6 -6

@@ -297,12 +297,12 @@ struct receiver_state {

297 time_t alarms_delay;

298 uint32_t alarms_history;

299 int rrdpush_enabled;

300 - char *rrdpush_api_key; // DONT FREE - it is allocated in appconfig

301 - char *rrdpush_send_charts_matching; // DONT FREE - it is allocated in appconfig

300 + const char *rrdpush_api_key; // DONT FREE - it is allocated in appconfig

301 + const char *rrdpush_send_charts_matching; // DONT FREE - it is allocated in appconfig

302 bool rrdpush_enable_replication;

303 time_t rrdpush_seconds_to_replicate;

304 time_t rrdpush_replication_step;

305 - char *rrdpush_destination; // DONT FREE - it is allocated in appconfig

305 + const char *rrdpush_destination; // DONT FREE - it is allocated in appconfig

306 unsigned int rrdpush_compression;

307 STREAM_CAPABILITIES compression_priorities[COMPRESSION_ALGORITHM_MAX];

308 } config;

@@ -349,9 +349,9 @@ struct rrdpush_destinations {

349

350 extern unsigned int default_rrdpush_enabled;

351 extern unsigned int default_rrdpush_compression_enabled;

352 -extern char *default_rrdpush_destination;

353 -extern char *default_rrdpush_api_key;

354 -extern char *default_rrdpush_send_charts_matching;

352 +extern const char *default_rrdpush_destination;

353 +extern const char *default_rrdpush_api_key;

354 +extern const char *default_rrdpush_send_charts_matching;

355 extern bool default_rrdpush_enable_replication;

356 extern time_t default_rrdpush_seconds_to_replicate;

357 extern time_t default_rrdpush_replication_step;

src/streaming/sender.c

+4 -4

@@ -465,8 +465,8 @@ void *rrdpush_sender_thread(void *ptr) {

465

466 netdata_log_info("STREAM %s [send]: thread created (task id %d)", rrdhost_hostname(s->host), gettid_cached());

467

468 - s->timeout = (int)appconfig_get_number(

469 - &stream_config, CONFIG_SECTION_STREAM, "timeout seconds", 600);

468 + s->timeout = (int)appconfig_get_duration_seconds(

469 + &stream_config, CONFIG_SECTION_STREAM, "timeout", 600);

470

471 s->default_port = (int)appconfig_get_number(

472 &stream_config, CONFIG_SECTION_STREAM, "default port", 19999);

@@ -474,8 +474,8 @@ void *rrdpush_sender_thread(void *ptr) {

474 s->buffer->max_size = (size_t)appconfig_get_number(

475 &stream_config, CONFIG_SECTION_STREAM, "buffer size bytes", 1024 * 1024 * 10);

476

477 - s->reconnect_delay = (unsigned int)appconfig_get_number(

478 - &stream_config, CONFIG_SECTION_STREAM, "reconnect delay seconds", 5);

477 + s->reconnect_delay = (unsigned int)appconfig_get_duration_seconds(

478 + &stream_config, CONFIG_SECTION_STREAM, "reconnect delay", 5);

479

480 remote_clock_resync_iterations = (unsigned int)appconfig_get_number(

481 &stream_config, CONFIG_SECTION_STREAM,

src/streaming/sender_internals.h

+2 -2

@@ -38,8 +38,8 @@

38 #endif

40 extern struct config stream_config;

41 -extern char *netdata_ssl_ca_path;

42 -extern char *netdata_ssl_ca_file;

41 +extern const char *netdata_ssl_ca_path;

42 +extern const char *netdata_ssl_ca_file;

44 bool attempt_to_connect(struct sender_state *state);

45 void rrdpush_sender_on_connect(RRDHOST *host);

src/streaming/stream.conf

+29 -29

@@ -62,7 +62,7 @@

62 #enable compression = yes

64 # The timeout to connect and send metrics

65 - #timeout seconds = 60

65 + #timeout = 1m

67 # If the destination line above does not specify a port, use this

68 #default port = 19999

@@ -84,7 +84,7 @@

85 # If the connection fails, or it disconnects,

86 # retry after that many seconds.

87 - #reconnect delay seconds = 5

87 + #reconnect delay = 5s

89 # Sync the clock of the charts for that many iterations, when starting.

90 # It is ignored when replication is enabled

@@ -127,19 +127,19 @@

127 # should also be matched at netdata.conf [web].allow connections from

128 #allow from = *

129

130 - # The default history in entries, for all hosts using this API key.

130 + # The history in entries (for db alloc or ram), for all hosts using this API key.

131 # You can also set it per host below.

132 - # For the default db mode (dbengine), this is ignored.

133 - #default history = 3600

132 + # For the default db (dbengine), this is ignored.

133 + #retention = 3600

134

135 - # The default memory mode to be used for all hosts using this API key.

135 + # The database to be used for all hosts using this API key.

136 # You can also set it per host below.

137 # If you don't set it here, the memory mode of netdata.conf will be used.

138 # Valid modes:

139 # ram keep it in RAM, don't touch the disk

140 # none no database at all (use this on headless proxies)

141 - # dbengine like a traditional database

142 - #default memory mode = dbengine

141 + # dbengine Netdata's high performance database

142 + #db = dbengine

143

144 # Shall we enable health monitoring for the hosts using this API key?

145 # 3 possible values:

@@ -151,18 +151,18 @@

151 # The default is taken from [health].enabled of netdata.conf

152 #health enabled by default = auto

153

154 - # postpone alarms for a short period after the sender is connected

155 - #default postpone alarms on connect seconds = 60

154 + # postpone alerts for a short period after the sender is connected

155 + #postpone alerts on connect = 1m

156

157 - # seconds of health log events to keep

158 - #default health log history = 432000

157 + # the duration to maintain health log events

158 + #health log retention = 5d

159

160 # need to route metrics differently? set these.

161 # the defaults are the ones at the [stream] section (above)

162 - #default proxy enabled = yes | no

163 - #default proxy destination = IP:PORT IP:PORT ...

164 - #default proxy api key = API_KEY

165 - #default proxy send charts matching = *

162 + #proxy enabled = yes | no

163 + #proxy destination = IP:PORT IP:PORT ...

164 + #proxy api key = API_KEY

165 + #proxy send charts matching = *

166

167 # Stream Compression

168 # By default it is enabled.

@@ -177,13 +177,13 @@

177 #enable replication = yes

178

179 # How many seconds to replicate from each child. Default: a day

180 - #seconds to replicate = 86400

180 + #replication period = 1d

181

182 # The duration we want to replicate per each step.

183 - #seconds per replication step = 600

183 + #replication step = 10m

184

185 # Indicate whether this child is an ephemeral node. An ephemeral node will become unavailable

186 - # after the specified duration of "cleanup ephemeral hosts after secs" (as defined in the db section of netdata.conf)

186 + # after the specified duration of "cleanup ephemeral hosts after" (as defined in the db section of netdata.conf)

187 # from the time of the node's last connection.

188 #is ephemeral node = no

189

@@ -221,20 +221,20 @@

221 #allow from = *

222

223 # The number of entries in the database.

224 - # This is ignored for db mode dbengine.

225 - #history = 3600

224 + # This is ignored for db dbengine.

225 + #retention = 3600

226

227 # The memory mode of the database: ram | none | dbengine

228 - #memory mode = dbengine

228 + #db = dbengine

229

230 # Health / alarms control: yes | no | auto

231 #health enabled = auto

232

233 - # postpone alarms when the sender connects

234 - #postpone alarms on connect seconds = 60

233 + # postpone alerts when the sender connects

234 + #postpone alerts on connect = 1m

235

236 - # seconds of health log events to keep

237 - #health log history = 432000

236 + # the duration to maintain health log events

237 + #health log retention = 5d

238

239 # need to route metrics differently?

240 # the defaults are the ones at the [API KEY] section

@@ -253,12 +253,12 @@

253 #enable replication = yes

254

255 # How many seconds to replicate from each child.

256 - #seconds to replicate = 86400

256 + #replication period = 1d

257

258 # The duration we want to replicate per each step.

259 - #seconds per replication step = 600

259 + #replication step = 10m

260

261 # Indicate whether this child is an ephemeral node. An ephemeral node will become unavailable

262 - # after the specified duration of "cleanup ephemeral hosts after secs" (as defined in the db section of netdata.conf)

262 + # after the specified duration of "cleanup ephemeral hosts after" (as defined in the db section of netdata.conf)

263 # from the time of the node's last connection.

264 #is ephemeral node = no

src/web/api/formatters/charts2json.c

+1 -1

@@ -37,7 +37,7 @@ const char* get_release_channel() {

37 }

39 void charts2json(RRDHOST *host, BUFFER *wb) {

40 - static char *custom_dashboard_info_js_filename = NULL;

40 + static const char *custom_dashboard_info_js_filename = NULL;

41 size_t c = 0, dimensions = 0, memory = 0, alarms = 0;

42 RRDSET *st;

src/web/api/health/README.md

+1 -1

@@ -33,7 +33,7 @@ The size of the alert log is configured in `netdata.conf`. There are 2 settings:

33 ```

34 [health]

35 in memory max health log entries = 1000

36 - health log history = 432000

36 + health log retention = 5d

37 ```

39 The API call retrieves all entries of the alert log:

src/web/api/queries/weights.c

+2 -4

@@ -4,9 +4,7 @@

4 #include "database/KolmogorovSmirnovDist.h"

6 #define MAX_POINTS 10000

7 -int enable_metric_correlations = CONFIG_BOOLEAN_YES;

7 int metric_correlations_version = 1;

9 -WEIGHTS_METHOD default_metric_correlations_method = WEIGHTS_METHOD_MC_KS2;

9 typedef struct weights_stats {

10 NETDATA_DOUBLE max_base_high_ratio;

@@ -36,7 +34,7 @@ WEIGHTS_METHOD weights_string_to_method(const char *method) {

34 if(strcmp(method, weights_methods[i].name) == 0)

35 return weights_methods[i].value;

39 - return default_metric_correlations_method;

37 + return WEIGHTS_METHOD_MC_KS2;

38 }

40 const char *weights_method_to_string(WEIGHTS_METHOD method) {

@@ -44,7 +42,7 @@ const char *weights_method_to_string(WEIGHTS_METHOD method) {

42 if(weights_methods[i].value == method)

43 return weights_methods[i].name;

47 - return "unknown";

45 + return "ks2";

46 }

48 // ----------------------------------------------------------------------------

src/web/api/queries/weights.h

-2

@@ -18,9 +18,7 @@ typedef enum {

18 WEIGHTS_FORMAT_MULTINODE = 3,

19 } WEIGHTS_FORMAT;

21 -extern int enable_metric_correlations;

21 extern int metric_correlations_version;

23 -extern WEIGHTS_METHOD default_metric_correlations_method;

23 typedef bool (*weights_interrupt_callback_t)(void *data);

src/web/api/v1/api_v1_manage.c

+1 -1

@@ -7,7 +7,7 @@ char *api_secret;

7 static char *get_mgmt_api_key(void) {

8 char filename[FILENAME_MAX + 1];

9 snprintfz(filename, FILENAME_MAX, "%s/netdata.api.key", netdata_configured_varlib_dir);

10 - char *api_key_filename=config_get(CONFIG_SECTION_REGISTRY, "netdata management api key file", filename);

10 + const char *api_key_filename = config_get(CONFIG_SECTION_REGISTRY, "netdata management api key file", filename);

11 static char guid[GUID_LEN + 1] = "";

13 if(likely(guid[0]))

src/web/api/v1/api_v1_weights.c

+1 -1

@@ -3,7 +3,7 @@

3 #include "api_v1_calls.h"

5 int api_v1_metric_correlations(RRDHOST *host, struct web_client *w, char *url) {

6 - return web_client_api_request_weights(host, w, url, default_metric_correlations_method, WEIGHTS_FORMAT_CHARTS, 1);

6 + return web_client_api_request_weights(host, w, url, WEIGHTS_METHOD_MC_KS2, WEIGHTS_FORMAT_CHARTS, 1);

7 }

9 int api_v1_weights(RRDHOST *host, struct web_client *w, char *url) {

src/web/rtc/webrtc.c

+6 -4

@@ -84,8 +84,8 @@ static struct {

84 bool enabled;

85 char *iceServers[WEBRTC_MAX_ICE_SERVERS];

86 int iceServersCount;

87 - char *proxyServer;

88 - char *bindAddress;

87 + const char *proxyServer;

88 + const char *bindAddress;

90 struct {

91 SPINLOCK spinlock;

@@ -142,10 +142,12 @@ static void webrtc_config_ice_servers(void) {

142 webrtc_base.iceServersCount = i;

143 internal_error(true, "WEBRTC: there are %d default ice servers: '%s'", webrtc_base.iceServersCount, buffer_tostring(wb));

144

145 - char *servers = config_get(CONFIG_SECTION_WEBRTC, "ice servers", buffer_tostring(wb));

145 + const char *servers = config_get(CONFIG_SECTION_WEBRTC, "ice servers", buffer_tostring(wb));

146

147 webrtc_base.iceServersCount = 0;

148 - char *s = servers, *e;

148 + char tmp[strlen(servers) + 1];

149 + strcpy(tmp, servers);

150 + char *s = tmp, *e;

151 while(*s) {

152 if(isspace(*s))

153 s++;

src/web/server/README.md

+36 -36

@@ -20,42 +20,42 @@ Scroll down to the `[web]` section to find the following settings.

21 ## Settings

23 -| Setting | Default | Description |

24 -|:-------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

25 -| `ssl key` | `/etc/netdata/ssl/key.pem` | Declare the location of an SSL key to [enable HTTPS](#enable-httpstls-support). |

26 -| `ssl certificate` | `/etc/netdata/ssl/cert.pem` | Declare the location of an SSL certificate to [enable HTTPS](#enable-httpstls-support). |

27 -| `tls version` | `1.3` | Choose which TLS version to use. While all versions are allowed (`1` or `1.0`, `1.1`, `1.2` and `1.3`), we recommend `1.3` for the most secure encryption. If left blank, Netdata uses the highest available protocol version on your system. |

28 -| `tls ciphers` | `none` | Choose which TLS cipher to use. Options include `TLS_AES_256_GCM_SHA384`, `TLS_CHACHA20_POLY1305_SHA256`, and `TLS_AES_128_GCM_SHA256`. If left blank, Netdata uses the default cipher list for that protocol provided by your TLS implementation. |

29 -| `ses max window` | `15` | See [single exponential smoothing](/src/web/api/queries/ses/README.md). |

30 -| `des max window` | `15` | See [double exponential smoothing](/src/web/api/queries/des/README.md). |

31 -| `mode` | `static-threaded` | Turns on (`static-threaded` or off (`none`) the static-threaded web server. See the [example](#disable-the-web-server) to turn off the web server and disable the dashboard. |

32 -| `listen backlog` | `4096` | The port backlog. Check `man 2 listen`. |

33 -| `default port` | `19999` | The listen port for the static web server. |

34 -| `web files owner` | `netdata` | The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, Netdata will only serve files owned by user given in `run as user`. |

35 -| `web files group` | `netdata` | If this is set, Netdata will check if the file is owned by this group and refuse to serve the file if it's not. |

36 -| `disconnect idle clients after seconds` | `60` | The time in seconds to disconnect web clients after being totally idle. |

37 -| `timeout for first request` | `60` | How long to wait for a client to send a request before closing the socket. Prevents slow request attacks. |

38 -| `accept a streaming request every seconds` | `0` | Can be used to set a limit on how often a parent node will accept streaming requests from child nodes in a [streaming and replication setup](/src/streaming/README.md). |

39 -| `respect do not track policy` | `no` | If set to `yes`, Netdata will respect the user's browser preferences for [Do Not Track](https://www.eff.org/issues/do-not-track) (DNT) and storing cookies. If DNT is _enabled_ in the browser, and this option is set to `yes`, nodes will not connect to any [registry](/src/registry/README.md). For certain browsers, users must disable DNT and change this option to `yes` for full functionality. |

40 -| `x-frame-options response header` | ` ` | Avoid [clickjacking attacks](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options), by ensuring that the content is not embedded into other sites. |

41 -| `allow connections from` | `localhost *` | Declare which IP addresses or full-qualified domain names (FQDNs) are allowed to connect to the web server, including the [dashboard](/docs/dashboards-and-charts/README.md) or [HTTP API](/src/web/api/README.md). This is a global setting with higher priority to any of the ones below. |

42 -| `allow connections by dns` | `heuristic` | See the [access list examples](#access-lists) for details on using `allow` settings. |

43 -| `allow dashboard from` | `localhost *` | |

44 -| `allow dashboard by dns` | `heuristic` | |

45 -| `allow badges from` | `*` | |

46 -| `allow badges by dns` | `heuristic` | |

47 -| `allow streaming from` | `*` | |

48 -| `allow streaming by dns` | `heuristic` | |

49 -| `allow netdata.conf` | `localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.* UNKNOWN` | |

50 -| `allow netdata.conf by dns` | `no` | |

51 -| `allow management from` | `localhost` | |

52 -| `allow management by dns` | `heuristic` | |

53 -| `enable gzip compression` | `yes` | When set to `yes`, Netdata web responses will be GZIP compressed, if the web client accepts such responses. |

54 -| `gzip compression strategy` | `default` | Valid settings are `default`, `filtered`, `huffman only`, `rle` and `fixed`. |

55 -| `gzip compression level` | `3` | Valid settings are 1 (fastest) to 9 (best ratio). |

56 -| `web server threads` | ` ` | How many processor threads the web server is allowed. The default is system-specific, the minimum of `6` or the number of CPU cores. |

57 -| `web server max sockets` | ` ` | Available sockets. The default is system-specific, automatically adjusted to 50% of the max number of open files Netdata is allowed to use (via `/etc/security/limits.conf` or systemd), to allow enough file descriptors to be available for data collection. |

58 -| `custom dashboard_info.js` | ` ` | Specifies the location of a custom `dashboard.js` file. See [customizing the standard dashboard](/docs/developer-and-contributor-corner/customize.md#customize-the-standard-dashboard) for details. |

23 +| Setting | Default | Description |

24 +|:-----------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

25 +| `ssl key` | `/etc/netdata/ssl/key.pem` | Declare the location of an SSL key to [enable HTTPS](#enable-httpstls-support). |

26 +| `ssl certificate` | `/etc/netdata/ssl/cert.pem` | Declare the location of an SSL certificate to [enable HTTPS](#enable-httpstls-support). |

27 +| `tls version` | `1.3` | Choose which TLS version to use. While all versions are allowed (`1` or `1.0`, `1.1`, `1.2` and `1.3`), we recommend `1.3` for the most secure encryption. If left blank, Netdata uses the highest available protocol version on your system. |

28 +| `tls ciphers` | `none` | Choose which TLS cipher to use. Options include `TLS_AES_256_GCM_SHA384`, `TLS_CHACHA20_POLY1305_SHA256`, and `TLS_AES_128_GCM_SHA256`. If left blank, Netdata uses the default cipher list for that protocol provided by your TLS implementation. |

29 +| `ses max window` | `15` | See [single exponential smoothing](/src/web/api/queries/ses/README.md). |

30 +| `des max window` | `15` | See [double exponential smoothing](/src/web/api/queries/des/README.md). |

31 +| `mode` | `static-threaded` | Turns on (`static-threaded` or off (`none`) the static-threaded web server. See the [example](#disable-the-web-server) to turn off the web server and disable the dashboard. |

32 +| `listen backlog` | `4096` | The port backlog. Check `man 2 listen`. |

33 +| `default port` | `19999` | The listen port for the static web server. |

34 +| `web files owner` | `netdata` | The user that owns the web static files. Netdata will refuse to serve a file that is not owned by this user, even if it has read access to that file. If the user given is not found, Netdata will only serve files owned by user given in `run as user`. |

35 +| `web files group` | `netdata` | If this is set, Netdata will check if the file is owned by this group and refuse to serve the file if it's not. |

36 +| `disconnect idle clients after` | `1m` | The time in seconds to disconnect web clients after being totally idle. |

37 +| `timeout for first request` | `1m` | How long to wait for a client to send a request before closing the socket. Prevents slow request attacks. |

38 +| `accept a streaming request every` | `off` | Can be used to set a limit on how often a parent node will accept streaming requests from child nodes in a [streaming and replication setup](/src/streaming/README.md). |

39 +| `respect do not track policy` | `no` | If set to `yes`, Netdata will respect the user's browser preferences for [Do Not Track](https://www.eff.org/issues/do-not-track) (DNT) and storing cookies. If DNT is _enabled_ in the browser, and this option is set to `yes`, nodes will not connect to any [registry](/src/registry/README.md). For certain browsers, users must disable DNT and change this option to `yes` for full functionality. |

40 +| `x-frame-options response header` | ` ` | Avoid [clickjacking attacks](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options), by ensuring that the content is not embedded into other sites. |

41 +| `allow connections from` | `localhost *` | Declare which IP addresses or full-qualified domain names (FQDNs) are allowed to connect to the web server, including the [dashboard](/docs/dashboards-and-charts/README.md) or [HTTP API](/src/web/api/README.md). This is a global setting with higher priority to any of the ones below. |

42 +| `allow connections by dns` | `heuristic` | See the [access list examples](#access-lists) for details on using `allow` settings. |

43 +| `allow dashboard from` | `localhost *` | |

44 +| `allow dashboard by dns` | `heuristic` | |

45 +| `allow badges from` | `*` | |

46 +| `allow badges by dns` | `heuristic` | |

47 +| `allow streaming from` | `*` | |

48 +| `allow streaming by dns` | `heuristic` | |

49 +| `allow netdata.conf` | `localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.* UNKNOWN` | |

50 +| `allow netdata.conf by dns` | `no` | |

51 +| `allow management from` | `localhost` | |

52 +| `allow management by dns` | `heuristic` | |

53 +| `enable gzip compression` | `yes` | When set to `yes`, Netdata web responses will be GZIP compressed, if the web client accepts such responses. |

54 +| `gzip compression strategy` | `default` | Valid settings are `default`, `filtered`, `huffman only`, `rle` and `fixed`. |

55 +| `gzip compression level` | `3` | Valid settings are 1 (fastest) to 9 (best ratio). |

56 +| `web server threads` | ` ` | How many processor threads the web server is allowed. The default is system-specific, the minimum of `6` or the number of CPU cores. |

57 +| `web server max sockets` | ` ` | Available sockets. The default is system-specific, automatically adjusted to 50% of the max number of open files Netdata is allowed to use (via `/etc/security/limits.conf` or systemd), to allow enough file descriptors to be available for data collection. |

58 +| `custom dashboard_info.js` | ` ` | Specifies the location of a custom `dashboard.js` file. See [customizing the standard dashboard](/docs/developer-and-contributor-corner/customize.md#customize-the-standard-dashboard) for details. |

60 ## Examples

src/web/server/web_client.c

+1 -1

@@ -6,7 +6,7 @@

6 // it is used by all netdata web servers

8 int respect_web_browser_do_not_track_policy = 0;

9 -char *web_x_frame_options = NULL;

9 +const char *web_x_frame_options = NULL;

11 int web_enable_gzip = 1, web_gzip_level = 3, web_gzip_strategy = Z_DEFAULT_STRATEGY;

src/web/server/web_client.h

+1 -1

@@ -12,7 +12,7 @@ extern int web_enable_gzip, web_gzip_level, web_gzip_strategy;

12 #define HTTP_REQ_MAX_HEADER_FETCH_TRIES 100

14 extern int respect_web_browser_do_not_track_policy;

15 -extern char *web_x_frame_options;

15 +extern const char *web_x_frame_options;

17 typedef enum __attribute__((packed)) {

18 HTTP_VALIDATION_OK,