TimescaleDB connection page (#7180)
* Added timescaledb page * Better title to fit tiny nav * GA link and some fixed grammar * Add reference in backends/README * Revert PostgreSQL change * Reverted on backends
Joel Hans committed
Nov 13, 2019 at 09:00 UTC
4207f1797ebaf295aa954858d27bfbbeacaf5081
3 files changed
+56
-1
README.md
+1
-1
@@ -394,7 +394,7 @@ This is what you should expect from Netdata:
394
395
### Integrations
396
397
-- **time-series dbs** - can archive its metrics to **Graphite**, **OpenTSDB**, **Prometheus**, **AWS Kinesis**, **MongoDB**, **JSON document DBs**, in the same or lower resolution (lower: to prevent it from congesting these servers due to the amount of data collected). Netdata also supports **Prometheus remote write API** which allows storing metrics to **Elasticsearch**, **Gnocchi**, **InfluxDB**, **Kafka**, **PostgreSQL/TimescaleDB**, **Splunk**, **VictoriaMetrics** and a lot of other [storage providers](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).
397
+- **time-series dbs** - can archive its metrics to **Graphite**, **OpenTSDB**, **Prometheus**, **AWS Kinesis**, **MongoDB**, **JSON document DBs**, in the same or lower resolution (lower: to prevent it from congesting these servers due to the amount of data collected). Netdata also supports **Prometheus remote write API** which allows storing metrics to **Elasticsearch**, **Gnocchi**, **InfluxDB**, **Kafka**, **PostgreSQL/TimescaleDB**, **Splunk**, **VictoriaMetrics**, and a lot of other [storage providers](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).
398
399
## Visualization
400
backends/README.md
+3
@@ -39,6 +39,9 @@ So, although Netdata collects metrics every second, it can send to the backend s
39
metrics are labeled in the format, which is used by Netdata for the [plaintext prometheus
40
protocol](prometheus/). Notes on using the remote write backend are [here](prometheus/remote_write/).
41
42
+ - ****TimescaleDB** via [community-built connector](TIMESCALE.md) that takes JSON streams from a Netdata client
43
+ and writes them to a TimescaleDB table.
44
+
45
- **AWS Kinesis Data Streams**
46
47
metrics are sent to the service in `JSON` format.
backends/TIMESCALE.md
new
+52
@@ -0,0 +1,52 @@
1
+# Writing metrics to TimescaleDB
2
+
3
+Thanks to Netdata's community of developers and system administrators, and Mahlon Smith
4
+([GitHub](https://github.com/mahlonsmith)/[Website](http://www.martini.nu/)) in particular, Netdata now supports
5
+archiving metrics directly to TimescaleDB.
6
+
7
+What's TimescaleDB? Here's how their team defines the project on their [GitHub page](https://github.com/timescale/timescaledb):
8
+
9
+> TimescaleDB is an open-source database designed to make SQL scalable for time-series data. It is engineered up from
10
+> PostgreSQL, providing automatic partitioning across time and space (partitioning key), as well as full SQL support.
11
+
12
+## Quickstart
13
+
14
+To get started archiving metrics to TimescaleDB right away, check out Mahlon's [`netdata-timescale-relay`
15
+repository](https://github.com/mahlonsmith/netdata-timescale-relay) on GitHub.
16
+
17
+This small program takes JSON streams from a Netdata client and writes them to a PostgreSQL (aka TimescaleDB) table.
18
+You'll run this program in parallel with Netdata, and after a short [configuration
19
+process](https://github.com/mahlonsmith/netdata-timescale-relay#configuration), your metrics should start populating
20
+TimescaleDB.
21
+
22
+Finally, another member of Netdata's community has built a project that quickly launches Netdata, TimescaleDB, and
23
+Grafana in easy-to-manage Docker containers. Rune Juhl Jacobsen's
24
+[project](https://github.com/runejuhl/grafana-timescaledb) uses a `Makefile` to create everything, which makes it
25
+perferct for testing and experimentation.
26
+
27
+## Netdata↔TimescaleDB in action
28
+
29
+Aside from creating incredible contributions to Netdata, Mahlon works at [LAIKA](https://www.laika.com/), an
30
+Oregon-based animation studio that's helped create acclaimed films like _Coraline_ and _Kubo and the Two Strings_.
31
+
32
+As part of his work to maintain the company's infrastructure of render farms, workstations, and virtual machines, he's
33
+using Netdata, `netdata-timescale-relay`, and TimescaleDB to store Netdata metrics alongside other data from other
34
+sources.
35
+
36
+> LAIKA is a long-time PostgreSQL user and added TimescaleDB to their infrastructure in 2018 to help manage and store
37
+> their IT metrics and time-series data. So far, the tool has been in production at LAIKA for over a year and helps them
38
+> with their use case of time-based logging, where they record over 8 million metrics an hour for netdata content alone.
39
+
40
+By archiving Netdata metrics to a backend like TimescaleDB, LAIKA can consolidate metrics data from distributed machines
41
+efficiently. Mahlon can then correlate Netdata metrics with other sources directly in TimescaleDB.
42
+
43
+And, because LAIKA will soon be storing years worth of Netdata metrics data in TimescaleDB, they can analyze long-term
44
+metrics as their films move from concept to final cut.
45
+
46
+Read the full blog post from LAIKA at the [TimescaleDB
47
+blog](https://blog.timescale.com/blog/writing-it-metrics-from-netdata-to-timescaledb/amp/).
48
+
49
+Thank you to Mahlon, Rune, TimescaleDB, and the members of the Netdata community that requested and then built this
50
+backend connection between Netdata and TimescaleDB!
51
+
52
+[](<>)