| 1 | # Writing metrics to TimescaleDB |
| 2 | |
| 3 | Thanks to Netdata's community of developers and system administrators, and Mahlon Smith |
| 4 | ([GitHub](https://github.com/mahlonsmith)/[Website](http://www.martini.nu/)) in particular, Netdata now supports |
| 5 | archiving metrics directly to TimescaleDB. |
| 6 | |
| 7 | What's TimescaleDB? Here's how their team defines the project on their [GitHub page](https://github.com/timescale/timescaledb): |
| 8 | |
| 9 | > TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from |
| 10 | > PostgreSQL, providing automatic partitioning across time and space (partitioning key), as well as full SQL support. |
| 11 | |
| 12 | ## Quickstart |
| 13 | |
| 14 | To get started archiving metrics to TimescaleDB right away, check out Mahlon's [`netdata-timescale-relay` |
| 15 | repository](https://github.com/mahlonsmith/netdata-timescale-relay) on GitHub. Please be aware that backends subsystem |
| 16 | was removed and Netdata configuration should be moved to the new `exporting.conf` configuration file. Use |
| 17 | |
| 18 | ```text |
| 19 | [json:my_instance] |
| 20 | ``` |
| 21 | |
| 22 | in `exporting.conf` instead of |
| 23 | |
| 24 | ```text |
| 25 | [backend] |
| 26 | type = json |
| 27 | ``` |
| 28 | |
| 29 | in `netdata.conf`. |
| 30 | |
| 31 | This small program takes JSON streams from a Netdata client and writes them to a PostgreSQL (aka TimescaleDB) table. |
| 32 | You'll run this program in parallel with Netdata, and after a short [configuration |
| 33 | process](https://github.com/mahlonsmith/netdata-timescale-relay#configuration), your metrics should start populating |
| 34 | TimescaleDB. |
| 35 | |
| 36 | Finally, another member of Netdata's community has built a project that quickly launches Netdata, TimescaleDB, and |
| 37 | Grafana in easy-to-manage Docker containers. Rune Juhl Jacobsen's |
| 38 | [project](https://github.com/runejuhl/grafana-timescaledb) uses a `Makefile` to create everything, which makes it |
| 39 | perfect for testing and experimentation. |
| 40 | |
| 41 | ## Netdata↔TimescaleDB in action |
| 42 | |
| 43 | Aside from creating incredible contributions to Netdata, Mahlon works at [LAIKA](https://www.laika.com/), an |
| 44 | Oregon-based animation studio that's helped create acclaimed films like _Coraline_ and _Kubo and the Two Strings_. |
| 45 | |
| 46 | As part of his work to maintain the company's infrastructure of render farms, workstations, and virtual machines, he's |
| 47 | using Netdata, `netdata-timescale-relay`, and TimescaleDB to store Netdata metrics alongside other data from other |
| 48 | sources. |
| 49 | |
| 50 | > LAIKA is a long-time PostgreSQL user and added TimescaleDB to their infrastructure in 2018 to help manage and store |
| 51 | > their IT metrics and time-series data. So far, the tool has been in production at LAIKA for over a year and helps them |
| 52 | > with their use case of time-based logging, where they record over 8 million metrics an hour for netdata content alone. |
| 53 | |
| 54 | By archiving Netdata metrics to a database like TimescaleDB, LAIKA can consolidate metrics data from distributed |
| 55 | machines efficiently. Mahlon can then correlate Netdata metrics with other sources directly in TimescaleDB. |
| 56 | |
| 57 | And, because LAIKA will soon be storing years worth of Netdata metrics data in TimescaleDB, they can analyze long-term |
| 58 | metrics as their films move from concept to final cut. |
| 59 | |
| 60 | Read the full blog post from LAIKA at the [TimescaleDB |
| 61 | blog](https://blog.timescale.com/blog/writing-it-metrics-from-netdata-to-timescaledb/amp/). |
| 62 | |
| 63 | Thank you to Mahlon, Rune, TimescaleDB, and the members of the Netdata community that requested and then built this |
| 64 | exporting connection between Netdata and TimescaleDB! |