[Stream Compression] - Compressor buffer overflow causes stream corruption. (#12019)
odynik committed
Jan 20, 2022 at 22:51 UTC
0f0d881bf3484e060d58cc69fb3ea54d5861da11
2 files changed
+10
-4
streaming/README.md
+9
-3
@@ -362,13 +362,19 @@ To check if your Netdata Agent has stream compression enabled, run the following
362
```
363
Note: The `stream-compression` status can be `"enabled" | "disabled" | "N/A"`.
364
365
-Stream data compression is enabled by default on systems where LZ4 library v1.9.0+ is installed. A compressed data packet is determined and decompressed on the fly.
365
+A compressed data packet is determined and decompressed on the fly.
366
367
#### Limitations
368
-The current implementation of streaming data compression has the limitation that the size of single data block transmitted must not exceed 16384 bytes. If single data block size exceeds this limit, stream data compression should be disabled.
368
+ This limitation will be withdrawn asap and is work-in-progress.
369
+
370
+The current implementation of streaming data compression can support only a few number of dimensions in a chart with names that cannot exceed the size of 16384 bytes. In case you experience stream connection problems or gaps in the charts please disable stream compresssion in the `stream.conf` file. This limitation can be seen in the error.log file with the sequence of the following messages:
371
+```
372
+Compression error - data discarded
373
+Message size above limit:
374
+```
375
376
#### How to enable stream compression
371
-Netdata Agents are shipped with data compression enabled by default. You can also configure which streams will use compression.
377
+Netdata Agents are shipped with data compression disabled by default. You can also configure which streams will use compression.
378
379
With enabled stream compression, a Netdata Agent can negotiate streaming compression with other Netdata Agents. During the negotiation of streaming compression both Netdata Agents should support and enable compression in order to communicate over a compressed stream. The negotiation will result into an uncompressed stream, if one of the Netdata Agents doesn't support **or** has compression disabled.
380
streaming/rrdpush.c
+1
-1
@@ -41,7 +41,7 @@ struct config stream_config = {
41
42
unsigned int default_rrdpush_enabled = 0;
43
#ifdef ENABLE_COMPRESSION
44
-unsigned int default_compression_enabled = 1;
44
+unsigned int default_compression_enabled = 0;
45
#endif
46
char *default_rrdpush_destination = NULL;
47
char *default_rrdpush_api_key = NULL;