#Netdata YAML Parser/Generator Module

This module provides YAML parsing and generation capabilities for Netdata, with seamless conversion to/from json-c objects. It uses the libyaml library for parsing and generation, supporting the YAML subset that matches JSON 100%.

#Features

#Supported Operations

1. Parse YAML from various sources:

   • String buffer (yaml_parse_string)
   • File by filename (yaml_parse_filename)
   • File descriptor (yaml_parse_fd)

2. Generate YAML to various destinations:

   • BUFFER (yaml_generate_to_buffer)
   • File by filename (yaml_generate_to_filename)
   • File descriptor (yaml_generate_to_fd)

3. Data types supported:

   • Null values
   • Booleans
   • Numbers (integers and floating-point)
   • Strings
   • Arrays
   • Objects (maps)

#Supported YAML Features

#1. Basic Data Types

# Null values (all case-insensitive)
null_value1: null
null_value2: Null
null_value3: NULL
null_value4: ~

# Booleans (all case-insensitive)
bool_true: true    # also: True, TRUE, yes, Yes, YES, on, On, ON
bool_false: false  # also: False, FALSE, no, No, NO, off, Off, OFF

# Numbers
integer: 42
negative: -123
float: 3.14159
scientific: 1.23e-10

#2. Advanced Number Formats

# Hexadecimal (parsed as integers)
hex_lower: 0x1a
hex_upper: 0XFF
hex_large: 0xDEADBEEF

# Octal (YAML 1.2 style)
octal_new: 0o755
octal_caps: 0O644

# Binary
binary_lower: 0b1010
binary_upper: 0B11111111

# Numbers with underscores (YAML 1.2)
readable_int: 1_000_000
readable_float: 3.141_592_653
readable_hex: 0x1_A_B_C
readable_binary: 0b1010_1010

#3. Strings

# Plain strings
plain: Hello World

# Single quoted (literal strings)
single: 'This is a single quoted string'
single_escape: 'Can''t escape much in single quotes'

# Double quoted (with escape sequences)
double: "Hello\nWorld"
escaped: "Tab:\t Quote:\" Backslash:\\"

# Special strings that need quoting
quoted_null: "null"     # Without quotes would be null value
quoted_bool: "true"     # Without quotes would be boolean
quoted_number: "123"    # Without quotes would be number

#4. Collections

# Arrays
simple_array: [1, 2, 3]
block_array:
  - item1
  - item2
  - nested: value

# Objects/Maps
simple_object: {key1: value1, key2: value2}
block_object:
  name: John Doe
  age: 30
  address:
    street: 123 Main St
    city: Anytown

#5. Multiline Strings

# Literal block scalar (preserves newlines and spacing)
literal: |
  Line 1
  Line 2
    Indented line

# Folded block scalar (folds newlines to spaces)
folded: >
  This is a long
  paragraph that will
  be folded into a
  single line.

#Round-Trip Consistency

The module ensures round-trip consistency for most data types:

// Original JSON
{"number": 1.0, "text": "hello", "flag": true}

// Generated YAML
number: 1.0
text: hello
flag: true

// Parsed back to JSON
{"number": 1.0, "text": "hello", "flag": true}

Special handling for floating-point numbers ensures that 1.0 remains 1.0 and doesn't become 1.

#Limitations Due to libyaml

#1. Unsupported Escape Sequences

Octal escapes are not supported by libyaml: ```yaml

#This will fail to parse

invalid: "\101" # Octal escape for 'A'

#Workaround: use hex or unicode escapes

valid: "\x41" # Hex escape for 'A' valid: "\u0041" # Unicode escape for 'A' ```

#2. Null Bytes in Strings

libyaml has issues with embedded null bytes: ```yaml

#This may not work correctly

with_null: "before\x00after"

#The null byte may terminate the string early

#or cause unexpected behavior

### 3. Single-Quoted Literal Newlines

In single-quoted strings, literal newlines are converted to spaces by libyaml:
```yaml
# Input
single: 'line1
line2'

# libyaml parses this as
single: 'line1 line2'

# To preserve newlines, use double quotes or block scalars
double: "line1\nline2"
literal: |
  line1
  line2

#4. Block Scalar Indentation

Complex indentation in block scalars may not be preserved exactly: ```yaml

#Deep indentation might be normalized

literal: | deeply indented text

#May lose some leading spaces depending on context

### 5. Invalid Syntax Detection

Some invalid YAML syntax may be accepted by libyaml:
```yaml
# This should be invalid but might parse
- item
- - invalid nesting

# Proper nesting would be
- item
- 
  - valid nesting

#Usage Examples

#Parsing YAML

// Parse from string
BUFFER *error = buffer_create(0, NULL);
const char *yaml = "name: Netdata\nversion: 1.0\n";
struct json_object *json = yaml_parse_string(yaml, error, YAML2JSON_DEFAULT);

if (buffer_strlen(error) > 0) {
    fprintf(stderr, "Parse error: %s\n", buffer_tostring(error));
} else {
    // Use json object (note: NULL is valid for YAML null)
    if (json) {
        printf("Parsed: %s\n", json_object_to_json_string(json));
        json_object_put(json);
    } else {
        printf("Parsed YAML null value\n");
    }
}

buffer_free(error);

#Generating YAML

// Create JSON object
struct json_object *root = json_object_new_object();
json_object_object_add(root, "name", json_object_new_string("Netdata"));
json_object_object_add(root, "version", json_object_new_double(1.0));

// Generate YAML
BUFFER *output = buffer_create(0, NULL);
BUFFER *error = buffer_create(0, NULL);

if (yaml_generate_to_buffer(output, root, error)) {
    printf("Generated YAML:\n%s", buffer_tostring(output));
} else {
    fprintf(stderr, "Generation error: %s\n", buffer_tostring(error));
}

json_object_put(root);
buffer_free(output);
buffer_free(error);

#Testing

The module includes comprehensive unit tests covering:

1. Basic tests (yaml-unittest.c):

   • All data types
   • File operations
   • Error handling
   • Round-trip conversion

2. Comprehensive tests (yaml-comprehensive-unittest.c):

   • All YAML string styles and number formats
   • Edge cases, unicode and special characters
   • Large documents and deep nesting
   • Round-trip fidelity

#Implementation Notes

1. Null Handling: JSON null is represented as C NULL pointer in json-c, which is properly handled in both parsing and generation.

2. Number Parsing: The parser attempts to identify number types in this order:

   • Hexadecimal (0x/0X prefix)
   • Octal (0o/0O prefix)
   • Binary (0b/0B prefix)
   • Integer (decimal)
   • Floating-point

3. String Quoting: Strings are automatically quoted during generation if they:

   • Could be misinterpreted as null, boolean, or number
   • Contain special characters
   • Have leading/trailing spaces
   • Contain newlines

4. Memory Management: All allocated memory is properly freed. The module has been tested with AddressSanitizer and LeakSanitizer.

#Thread Safety

The module is thread-safe as long as: - Different threads use different parser/emitter instances - json-c objects are not shared between threads without proper synchronization - BUFFER objects are not shared between threads without proper synchronization