| 1 | # Exporting Reference |
| 2 | |
| 3 | This reference guide provides comprehensive information about enabling, configuring, and monitoring Netdata's exporting engine for sending metrics to external time-series databases. |
| 4 | |
| 5 | For a quick introduction, read our [exporting metrics overview](/docs/exporting-metrics/README.md) or start with [enabling a connector](/docs/exporting-metrics/enable-an-exporting-connector.md). |
| 6 | |
| 7 | ## Core Capabilities |
| 8 | |
| 9 | The exporting engine features a modular structure that supports: |
| 10 | |
| 11 | - Multiple connector instances running simultaneously |
| 12 | - Different update intervals per connector |
| 13 | - Custom filters per connector instance |
| 14 | - Metric resampling to reduce database congestion |
| 15 | |
| 16 | :::info |
| 17 | |
| 18 | When you enable the exporting engine, Netdata exports metrics **starting from the restart time**, not the entire [historical database](/src/database/README.md). |
| 19 | |
| 20 | ::: |
| 21 | |
| 22 | ## Operation Modes |
| 23 | |
| 24 | Netdata provides three data export modes: |
| 25 | |
| 26 | | Mode | Description | Data Format | Use Case | |
| 27 | |:----------------:|:-----------------------------------------|:-----------------------------------------------|:--------------------------------------------------------| |
| 28 | | **as-collected** | Raw metrics in original units | Counters remain counters, gauges remain gauges | Time-series database experts who need raw data | |
| 29 | | **average** | Normalized metrics from Netdata database | All metrics sent as gauges in Netdata units | Simplified visualization with Netdata-centric workflows | |
| 30 | | **sum/volume** | Sum of interpolated values | Aggregated values over the export interval | Long-term trend analysis | |
| 31 | |
| 32 | :::tip |
| 33 | |
| 34 | **Choosing the Right Mode**: |
| 35 | |
| 36 | - Use `as-collected` if you're building monitoring around a time-series database and know how to convert units |
| 37 | - Use `average` for simpler long-term archiving that matches Netdata's visualization exactly |
| 38 | |
| 39 | ::: |
| 40 | |
| 41 | ## Supported Connectors |
| 42 | |
| 43 | | Connector | Protocol/Format | Metric Format | |
| 44 | |:----------------------------------------------------------------------------|:---------------------------|:-----------------------------------| |
| 45 | | [AWS Kinesis](/src/exporting/aws_kinesis/README.md) | JSON | Stream-based | |
| 46 | | [Google Pub/Sub](/src/exporting/pubsub/README.md) | JSON | Message-based | |
| 47 | | [Graphite](/src/exporting/graphite/README.md) | Plaintext | `prefix.hostname.chart.dimension` | |
| 48 | | [JSON Databases](/src/exporting/json/README.md) | JSON | Document-based | |
| 49 | | [OpenTSDB](/src/exporting/opentsdb/README.md) | Plaintext/HTTP | `prefix.chart.dimension` with tags | |
| 50 | | [MongoDB](/src/exporting/mongodb/README.md) | JSON | Document-based | |
| 51 | | [Prometheus](/src/exporting/prometheus/README.md) | HTTP scraping | Prometheus exposition format | |
| 52 | | [Prometheus Remote Write](/src/exporting/prometheus/remote_write/README.md) | Snappy-compressed protobuf | Binary over HTTP | |
| 53 | | [TimescaleDB](/src/exporting/TIMESCALE.md) | JSON streams | Time-series tables | |
| 54 | |
| 55 | ## Configuration Structure |
| 56 | |
| 57 | Your `exporting.conf` file contains these configuration blocks: |
| 58 | |
| 59 | ```text |
| 60 | [exporting:global] |
| 61 | enabled = yes |
| 62 | send configured labels = no |
| 63 | send automatic labels = no |
| 64 | update every = 10 |
| 65 | |
| 66 | [prometheus:exporter] |
| 67 | send names instead of ids = yes |
| 68 | send configured labels = yes |
| 69 | send automatic labels = no |
| 70 | send charts matching = * |
| 71 | send hosts matching = localhost * |
| 72 | prefix = netdata |
| 73 | |
| 74 | [graphite:my_graphite_instance] |
| 75 | enabled = yes |
| 76 | destination = localhost:2003 |
| 77 | data source = average |
| 78 | prefix = Netdata |
| 79 | hostname = my-name |
| 80 | update every = 10 |
| 81 | buffer on failures = 10 |
| 82 | timeout ms = 20000 |
| 83 | send charts matching = * |
| 84 | send hosts matching = localhost * |
| 85 | send names instead of ids = yes |
| 86 | send configured labels = yes |
| 87 | send automatic labels = yes |
| 88 | |
| 89 | [prometheus_remote_write:my_prometheus_remote_write_instance] |
| 90 | enabled = yes |
| 91 | destination = localhost |
| 92 | remote write URL path = /receive |
| 93 | |
| 94 | [kinesis:my_kinesis_instance] |
| 95 | enabled = yes |
| 96 | destination = us-east-1 |
| 97 | stream name = netdata |
| 98 | aws_access_key_id = my_access_key_id |
| 99 | aws_secret_access_key = my_aws_secret_access_key |
| 100 | |
| 101 | [pubsub:my_pubsub_instance] |
| 102 | enabled = yes |
| 103 | destination = pubsub.googleapis.com |
| 104 | credentials file = /etc/netdata/pubsub_credentials.json |
| 105 | project id = my_project |
| 106 | topic id = my_topic |
| 107 | |
| 108 | [mongodb:my_mongodb_instance] |
| 109 | enabled = yes |
| 110 | destination = localhost |
| 111 | database = my_database |
| 112 | collection = my_collection |
| 113 | |
| 114 | [json:my_json_instance] |
| 115 | enabled = yes |
| 116 | destination = localhost:5448 |
| 117 | |
| 118 | [opentsdb:my_opentsdb_plaintext_instance] |
| 119 | enabled = yes |
| 120 | destination = localhost:4242 |
| 121 | |
| 122 | [opentsdb:http:my_opentsdb_http_instance] |
| 123 | enabled = yes |
| 124 | destination = localhost:4242 |
| 125 | username = my_username |
| 126 | password = my_password |
| 127 | |
| 128 | [opentsdb:https:my_opentsdb_https_instance] |
| 129 | enabled = yes |
| 130 | destination = localhost:8082 |
| 131 | ``` |
| 132 | |
| 133 | ### Configuration Sections |
| 134 | |
| 135 | | Section | Purpose | |
| 136 | |:------------------------|:--------------------------------------------| |
| 137 | | `[exporting:global]` | Default settings for all connectors | |
| 138 | | `[prometheus:exporter]` | Prometheus API endpoint settings | |
| 139 | | `[<type>:<name>]` | Individual connector instance configuration | |
| 140 | |
| 141 | ### Connector Types |
| 142 | |
| 143 | Available connector types with optional modifiers: |
| 144 | |
| 145 | - `graphite` | `graphite:http` | `graphite:https` |
| 146 | - `opentsdb:telnet` | `opentsdb:http` | `opentsdb:https` |
| 147 | - `prometheus_remote_write` | `prometheus_remote_write:http` | `prometheus_remote_write:https` |
| 148 | - `json` | `json:http` | `json:https` |
| 149 | - `kinesis` | `pubsub` | `mongodb` |
| 150 | |
| 151 | ## Configuration Options |
| 152 | |
| 153 | ### Basic Settings |
| 154 | |
| 155 | | Option | Values | Description | |
| 156 | |:---------------|:-------------------------|:--------------------------------------------------------------| |
| 157 | | `enabled` | yes/no | Activates the connector instance | |
| 158 | | `data source` | as-collected/average/sum | Selects data export mode | |
| 159 | | `hostname` | string | Hostname for external database (default: `[global].hostname`) | |
| 160 | | `prefix` | string | Prefix added to all metrics | |
| 161 | | `update every` | seconds | Export interval with automatic randomization | |
| 162 | |
| 163 | ### Connection Settings |
| 164 | |
| 165 | | Option | Format | Description | |
| 166 | |:---------------------|:---------------------|:--------------------------------------------------------| |
| 167 | | `destination` | space-separated list | Target servers in `[PROTOCOL:]IP[:PORT]` format | |
| 168 | | `buffer on failures` | iterations | Buffer size when database unavailable | |
| 169 | | `timeout ms` | milliseconds | Processing timeout (default: `2 * update_every * 1000`) | |
| 170 | |
| 171 | #### Destination Examples |
| 172 | |
| 173 | IPv4 configuration: |
| 174 | |
| 175 | ```text |
| 176 | destination = 10.11.14.2:4242 10.11.14.3:4242 10.11.14.4:4242 |
| 177 | ``` |
| 178 | |
| 179 | IPv6 and IPv4 combined: |
| 180 | |
| 181 | ```text |
| 182 | destination = [ffff:...:0001]:2003 10.11.12.1:2003 |
| 183 | ``` |
| 184 | |
| 185 | Special destinations: |
| 186 | |
| 187 | - **Kinesis**: AWS region (e.g., `us-east-1`) |
| 188 | - **MongoDB**: [MongoDB URI](https://docs.mongodb.com/manual/reference/connection-string/) |
| 189 | - **Pub/Sub**: Service endpoint |
| 190 | |
| 191 | ### Filtering Options |
| 192 | |
| 193 | | Option | Pattern Format | Description | |
| 194 | |:-----------------------|:-------------------------|:--------------------------------------------------| |
| 195 | | `send hosts matching` | space-separated patterns | Filter hosts using `*` wildcard, `!` for negation | |
| 196 | | `send charts matching` | space-separated patterns | Filter charts by ID/name, `!` for negation | |
| 197 | |
| 198 | :::important |
| 199 | |
| 200 | Pattern matching follows first-match logic. Order matters when using negative patterns (`!`). |
| 201 | |
| 202 | Example: `!*child* *db*` matches all `*db*` hosts except those containing `*child*`. |
| 203 | |
| 204 | ::: |
| 205 | |
| 206 | ### Label Settings |
| 207 | |
| 208 | | Option | Values | Description | |
| 209 | |:----------------------------|:-------|:------------------------------------------------------------| |
| 210 | | `send names instead of ids` | yes/no | Use human-friendly names vs system IDs | |
| 211 | | `send configured labels` | yes/no | Include `[host labels]` from `netdata.conf` | |
| 212 | | `send automatic labels` | yes/no | Include auto-generated labels (`_os_name`, `_architecture`) | |
| 213 | |
| 214 | ## Chart Filtering |
| 215 | |
| 216 | Filter metrics through two methods: |
| 217 | |
| 218 | 1. **Configuration file**: |
| 219 | |
| 220 | ```text |
| 221 | [prometheus:exporter] |
| 222 | send charts matching = system.* |
| 223 | ``` |
| 224 | |
| 225 | 2. **URL parameter**: |
| 226 | |
| 227 | ```text |
| 228 | http://localhost:19999/api/v1/allmetrics?format=shell&filter=system.* |
| 229 | ``` |
| 230 | |
| 231 | ## HTTPS Support |
| 232 | |
| 233 | For databases without native TLS/SSL support, configure a reverse proxy: |
| 234 | |
| 235 | - [Nginx reverse proxy setup](/docs/netdata-agent/configuration/running-the-netdata-agent-behind-a-reverse-proxy/Running-behind-nginx.md) |
| 236 | |
| 237 | ## Performance Considerations |
| 238 | |
| 239 | The exporting engine operates independently to avoid slowing down Netdata. However: |
| 240 | |
| 241 | :::warning |
| 242 | |
| 243 | Multiple connector instances running batches simultaneously can consume significant CPU resources. Configure different update intervals to prevent synchronization. |
| 244 | |
| 245 | ::: |
| 246 | |
| 247 | ## Monitoring the Exporting Engine |
| 248 | |
| 249 | Netdata provides five monitoring charts under **Netdata Monitoring**: |
| 250 | |
| 251 | | Chart | Monitors | |
| 252 | |:-------------------------------|:-------------------------------------------| |
| 253 | | **Buffered metrics** | Number of metrics added to dispatch buffer | |
| 254 | | **Exporting data size** | Data volume (KB) added to buffer | |
| 255 | | **Exporting operations** | Operation count performed | |
| 256 | | **Exporting thread CPU usage** | CPU resources consumed by exporting thread | |
| 257 | |
| 258 |  |
| 259 | |
| 260 | ## Built-in Alerts |
| 261 | |
| 262 | The exporting engine includes three automatic alerts: |
| 263 | |
| 264 | | Alert | Monitors | |
| 265 | |:---------------------------|:----------------------------------------| |
| 266 | | `exporting_last_buffering` | Seconds since last successful buffering | |
| 267 | | `exporting_metrics_sent` | Percentage of successfully sent metrics | |
| 268 | | `exporting_metrics_lost` | Metrics lost due to repeated failures | |
| 269 | |
| 270 |  |
| 271 | |
| 272 | ## Fallback Script |
| 273 | |
| 274 | Netdata includes `nc-exporting.sh` for: |
| 275 | |
| 276 | - Saving metrics to disk during database outages |
| 277 | - Pushing cached metrics when database recovers |
| 278 | - Monitoring/tracing/debugging metric generation |