| 1 | # Log2journal Test Framework |
| 2 | |
| 3 | This directory contains the comprehensive test suite for log2journal YAML parsing and functionality. |
| 4 | |
| 5 | ## Test Framework Overview |
| 6 | |
| 7 | The test framework supports multiple test file formats for different testing scenarios: |
| 8 | |
| 9 | ### Standard Tests |
| 10 | 1. **`{testname}.yaml`** - YAML configuration file (optional if using internal config) |
| 11 | 2. **`{testname}.input`** - Input log lines to process (optional if testing with empty input) |
| 12 | 3. **`{testname}.output`** - Expected output after processing |
| 13 | |
| 14 | ### CLI Tests (Improved Format) |
| 15 | 1. **`{testname}.cmd`** - Command to execute with `${TESTED_LOG2JOURNAL_BIN}` variable |
| 16 | 2. **`{testname}.yaml`** - Base configuration (optional) |
| 17 | 3. **`{testname}.input`** - Input log lines |
| 18 | 4. **`{testname}.output`** - Expected output |
| 19 | |
| 20 | ### Failure Tests |
| 21 | 1. **`{testname}.yaml`** - Configuration that should fail |
| 22 | 2. **`{testname}.input`** - Input log lines (optional) |
| 23 | 3. **`{testname}.fail`** - Expected error message (or empty for any failure) |
| 24 | |
| 25 | ### Config Display Tests |
| 26 | 1. **`{testname}.yaml`** - Configuration file |
| 27 | 2. **`{testname}-final-config.yaml`** - Expected `--show-config` output |
| 28 | |
| 29 | ## Test Framework (Modern Format Only) |
| 30 | |
| 31 | The test framework now uses clean, explicit file formats: |
| 32 | |
| 33 | ### CLI Format |
| 34 | ```bash |
| 35 | # Clean command files: |
| 36 | # inject-append.cmd |
| 37 | ${TESTED_LOG2JOURNAL_BIN} -f inject-append.yaml --inject CLINEWKEY1=value1 --inject CLINEWKEY2=value2 |
| 38 | ``` |
| 39 | |
| 40 | ### Error Testing Format |
| 41 | ```bash |
| 42 | # Explicit failure tests: |
| 43 | # error-test.fail |
| 44 | YAML PARSER: syntax error at line 5 |
| 45 | ``` |
| 46 | |
| 47 | ## Running Tests |
| 48 | |
| 49 | ### Default (uses installed log2journal) |
| 50 | ```bash |
| 51 | ./tests.sh |
| 52 | ``` |
| 53 | |
| 54 | ### With custom binary |
| 55 | ```bash |
| 56 | # Set the binary to test (e.g., local build) |
| 57 | export TESTED_LOG2JOURNAL_BIN="../../../build/log2journal" |
| 58 | ./tests.sh |
| 59 | ``` |
| 60 | |
| 61 | ## Test Categories |
| 62 | |
| 63 | ### 1. Core Logic Tests (`logic-*.yaml` - 7 tests) |
| 64 | Tests fundamental log2journal functionality: |
| 65 | - **logic-rewrite-pipeline** - Rewrite pipeline with stop/continue behavior |
| 66 | - **logic-variable-substitution** - Variable replacement with ${VAR} syntax |
| 67 | - **logic-rename-chains** - Field renaming functionality |
| 68 | - **logic-filter-behavior** - Include/exclude filter patterns |
| 69 | - **logic-unmatched-handling** - Behavior for unmatched log lines |
| 70 | - **logic-pcre2-groups** - PCRE2 named capture groups |
| 71 | - **logic-key-validation** - Journal key naming validation |
| 72 | |
| 73 | ### 2. YAML Parsing Tests (`risk-*.yaml`, `yaml-*.yaml` - 13 tests) |
| 74 | Tests YAML parsing edge cases: |
| 75 | - **risk-yaml-constructs** - Complex YAML structures |
| 76 | - **risk-duplicate-behavior** - Duplicate key handling |
| 77 | - **risk-error-recovery** - Parser error recovery |
| 78 | - **risk-variable-edge-cases** - Variable substitution edge cases |
| 79 | - **yaml-multiline-complex** - Multiline strings, folded/literal scalars |
| 80 | - **yaml-edge-cases** - YAML type handling (bools, numbers, strings) |
| 81 | - **yaml-strings-comprehensive** - All YAML string quoting styles |
| 82 | |
| 83 | ### 3. Unicode and Encoding Tests (`unicode-*.yaml`, `encoding-*.yaml` - 8 tests) |
| 84 | Critical for log processing: |
| 85 | - **unicode-utf8** - UTF-8 multibyte characters, emojis |
| 86 | - **unicode-test** - Unicode in patterns and variable substitution |
| 87 | - **unicode-escape-sequences** - Unicode escape sequences (\uXXXX) |
| 88 | - **unicode-control-chars** - Control characters in logs |
| 89 | - **encoding-special-chars** - Control characters, special symbols |
| 90 | - **variable-substitution-unicode** - Unicode in ${VAR} substitutions |
| 91 | |
| 92 | ### 4. Error Handling Tests (`error-*.yaml` - 6 tests) |
| 93 | Tests that should fail gracefully: |
| 94 | - **error-invalid-syntax** - Invalid YAML syntax |
| 95 | - **error-missing-pattern** - Missing required fields |
| 96 | - **error-wrong-types** - Wrong data types |
| 97 | - **error-invalid-regex** - Invalid PCRE2 patterns |
| 98 | - **error-messages-validation** - Error message validation |
| 99 | - **error-recovery-comprehensive** - Comprehensive error scenarios |
| 100 | |
| 101 | ### 5. Advanced Feature Tests (`advanced-*.yaml` - 5 tests) |
| 102 | Complex functionality: |
| 103 | - **advanced-prefix** - Prefix application to all keys |
| 104 | - **advanced-filter** - Complex include/exclude patterns |
| 105 | - **advanced-unmatched** - Unmatched line handling with injection |
| 106 | - **advanced-pcre2-complex** - Complex PCRE2 patterns |
| 107 | - **advanced-edge-cases** - Combined edge cases |
| 108 | |
| 109 | ### 6. CLI Integration Tests (2 tests) |
| 110 | Test CLI/config interaction: |
| 111 | - **inject-append** - CLI appends to inject rules |
| 112 | - **filter-cli** - CLI filter behavior |
| 113 | |
| 114 | ### 7. Internal Config Tests |
| 115 | Using `-c` flag with built-in configs: |
| 116 | - **default** - Default configuration |
| 117 | - **nginx-combined** - Nginx combined log format |
| 118 | - **nginx-json** - Nginx JSON log format |
| 119 | - **logfmt** - Logfmt parsing |
| 120 | |
| 121 | ### 8. Boundary Tests (`boundary-*.yaml`, `edge-*.yaml` - 12 tests) |
| 122 | System limits and edge cases: |
| 123 | - **boundary-empty-config** - Minimal valid configuration |
| 124 | - **boundary-max-items** - Maximum array sizes (512 items) |
| 125 | - **boundary-key-length** - 64-character field limit testing |
| 126 | - **edge-empty-values** - Empty strings and values |
| 127 | - **edge-long-strings** - Very long input strings |
| 128 | - **edge-special-chars** - Special character handling |
| 129 | |
| 130 | ### 9. Real-World Tests (`real-world-*.yaml` - 5 tests) |
| 131 | Practical log format examples: |
| 132 | - **real-world-apache-logs** - Apache access log parsing |
| 133 | - **real-world-nginx-error** - Nginx error log format |
| 134 | - **real-world-docker-logs** - Docker container log format |
| 135 | - **real-world-syslog** - Traditional syslog parsing |
| 136 | - **real-world-multiline-stack** - Java stack trace handling |
| 137 | |
| 138 | ### 10. Full Integration Tests |
| 139 | - **full** - Complete configuration with all features |
| 140 | |
| 141 | ## Creating New Tests |
| 142 | |
| 143 | ### Standard Test |
| 144 | ```bash |
| 145 | # Create input file |
| 146 | echo "your test log line" > tests.d/mytest.input |
| 147 | |
| 148 | # Create YAML config |
| 149 | cat > tests.d/mytest.yaml << EOF |
| 150 | pattern: 'your pattern' |
| 151 | # other config... |
| 152 | EOF |
| 153 | |
| 154 | # Generate expected output |
| 155 | cat tests.d/mytest.input | $TESTED_LOG2JOURNAL_BIN -f tests.d/mytest.yaml > tests.d/mytest.output |
| 156 | ``` |
| 157 | |
| 158 | ### CLI Test |
| 159 | ```bash |
| 160 | # Create command file |
| 161 | echo '${TESTED_LOG2JOURNAL_BIN} -f mytest.yaml --inject KEY=value' > tests.d/mytest.cmd |
| 162 | |
| 163 | # Create config and input |
| 164 | echo 'pattern: "(?P<MESSAGE>.*)"' > tests.d/mytest.yaml |
| 165 | echo 'test input' > tests.d/mytest.input |
| 166 | |
| 167 | # Generate expected output (using your build) |
| 168 | cat tests.d/mytest.input | $TESTED_LOG2JOURNAL_BIN -f tests.d/mytest.yaml --inject KEY=value > tests.d/mytest.output |
| 169 | ``` |
| 170 | |
| 171 | ### Failure Test |
| 172 | ```bash |
| 173 | # Create config that should fail |
| 174 | echo 'invalid: yaml: syntax:' > tests.d/fail-test.yaml |
| 175 | |
| 176 | # Create expected error message |
| 177 | echo 'YAML PARSER: syntax error' > tests.d/fail-test.fail |
| 178 | |
| 179 | # Optionally create input |
| 180 | echo 'test input' > tests.d/fail-test.input |
| 181 | ``` |
| 182 | |
| 183 | ### Show-Config Test |
| 184 | ```bash |
| 185 | # Test with --show-config to verify CLI/YAML merging |
| 186 | echo "test" | $TESTED_LOG2JOURNAL_BIN -f tests.d/config.yaml --some-arg value --show-config | sed '1,/^$/d' > tests.d/testname-final-config.yaml |
| 187 | ``` |
| 188 | |
| 189 | ## Test Implementation Details |
| 190 | |
| 191 | ### Pattern Types |
| 192 | - **PCRE2 pattern**: Custom regex with named groups `(?P<name>...)` |
| 193 | - **`json`**: Parse JSON formatted logs |
| 194 | - **`logfmt`**: Parse logfmt formatted logs |
| 195 | |
| 196 | ### Variable Substitution |
| 197 | - `${VARIABLE}`: Replaced with variable value |
| 198 | - `${undefined}`: Replaced with empty string |
| 199 | - Variables can reference: |
| 200 | - Captured groups from pattern |
| 201 | - Other injected keys |
| 202 | - Renamed keys (after renaming) |
| 203 | |
| 204 | ### Processing Pipeline |
| 205 | 1. **EXTRACT** - Pattern matching extracts fields |
| 206 | 2. **PREFIX** - Apply prefix to all keys |
| 207 | 3. **RENAME** - Rename keys (currently broken) |
| 208 | 4. **INJECT** - Add constant fields |
| 209 | 5. **REWRITE** - Modify field values (currently broken) |
| 210 | 6. **FILTER** - Include/exclude fields |
| 211 | 7. **OUTPUT** - Generate Journal Export Format |
| 212 | |
| 213 | ### Key Behaviors |
| 214 | |
| 215 | #### Duplicate Keys |
| 216 | - In `inject`: All values are added (allows duplicates) |
| 217 | - In `rewrite`: All rules processed in order (pipeline) |
| 218 | - In `rename`: All renames attempted |
| 219 | |
| 220 | #### CLI Precedence |
| 221 | - **PREFIX**: CLI replaces config |
| 222 | - **INJECT**: CLI prepends to config |
| 223 | - **FILTER**: CLI replaces config |
| 224 | - **REWRITE/RENAME**: CLI arguments ignored (features broken) |
| 225 | |
| 226 | #### Journal Field Rules |
| 227 | - Field names: max 64 characters |
| 228 | - Only A-Z, 0-9, underscore allowed |
| 229 | - First character cannot be digit |
| 230 | - All keys converted to uppercase |
| 231 | - Non-alphanumeric → underscore |
| 232 | |
| 233 | ### Known Issues |
| 234 | 1. **Rewrite feature is broken** - Values never change |
| 235 | 2. **Rename feature is broken** - Simple rename works, chaining doesn't |
| 236 | 3. **Unmatched lines** - Currently accepts all input (pattern not enforced) |
| 237 | |
| 238 | ## Test Results |
| 239 | |
| 240 | Results stored in `/tmp/log2journal_test_results/`: |
| 241 | - `{testname}.out` - Actual output |
| 242 | - `{testname}.err` - Error output |
| 243 | - `{testname}.diff` - Difference from expected |
| 244 | - `{testname}-config.yaml` - Actual config from --show-config |
| 245 | |
| 246 | ## Current Test Coverage |
| 247 | |
| 248 | - **Total tests**: 81 |
| 249 | - **Categories covered**: All major features |
| 250 | - **Success rate**: 100% |
| 251 | |
| 252 | ### Coverage by Feature |
| 253 | - ✅ Pattern matching (PCRE2, JSON, logfmt) |
| 254 | - ✅ Variable substitution |
| 255 | - ✅ Prefix functionality |
| 256 | - ✅ Inject functionality |
| 257 | - ✅ Filter (include/exclude) |
| 258 | - ✅ Unicode/UTF-8 handling |
| 259 | - ✅ CLI argument precedence |
| 260 | - ✅ YAML parsing edge cases |
| 261 | - ⚠️ Rewrite (broken, tests document this) |
| 262 | - ⚠️ Rename (partially working) |
| 263 | - ❌ Filename tracking (pipe mode only) |
| 264 | |
| 265 | ## Troubleshooting Failed Tests |
| 266 | |
| 267 | The test framework includes built-in debugging capabilities: |
| 268 | |
| 269 | ### Verbose Mode |
| 270 | Show exact commands and full diff output: |
| 271 | ```bash |
| 272 | ./tests.sh --verbose --test {test-name} |
| 273 | ``` |
| 274 | |
| 275 | ### Run Specific Test |
| 276 | Test only one specific case: |
| 277 | ```bash |
| 278 | ./tests.sh --test {test-name} |
| 279 | ``` |
| 280 | |
| 281 | ### Debug Methodology |
| 282 | 1. **Identify the failing test** from test summary |
| 283 | 2. **Run with verbose output** to see exact command and diff |
| 284 | 3. **Check test files** to understand expected behavior: |
| 285 | - `.yaml` - Configuration file |
| 286 | - `.input` - Input log lines |
| 287 | - `.output` - Expected output |
| 288 | - `.cmd` - Custom command (overrides default) |
| 289 | - `.fail` - Expected error message (for failure tests) |
| 290 | 4. **Run command manually** to verify behavior: |
| 291 | ```bash |
| 292 | export TESTED_LOG2JOURNAL_BIN="/path/to/your/log2journal" |
| 293 | cat tests.d/{test}.input | $TESTED_LOG2JOURNAL_BIN -f tests.d/{test}.yaml |
| 294 | ``` |
| 295 | 5. **Update expected output** if behavior is correct: |
| 296 | ```bash |
| 297 | cat tests.d/{test}.input | $TESTED_LOG2JOURNAL_BIN -f tests.d/{test}.yaml > tests.d/{test}.output |
| 298 | ``` |
| 299 | |
| 300 | ### Common Issues |
| 301 | - **Path differences**: Error messages may include full binary paths |
| 302 | - **Missing binary**: Set `TESTED_LOG2JOURNAL_BIN` to correct path |
| 303 | - **Output changes**: Use verbose mode to see actual vs expected output |
| 304 | - **CLI argument issues**: Check `.cmd` file for correct syntax |
| 305 | - **Version differences**: Error test outputs automatically ignore version lines to prevent build-dependent failures |
| 306 | |
| 307 | ### Version-Agnostic Testing |
| 308 | For tests that include version information (like `error-*` tests): |
| 309 | - The framework automatically ignores version lines during comparison |
| 310 | - Expected output files can contain placeholder versions (e.g., `v0.0.0-0-g00000000`) |
| 311 | - Version format is still validated to ensure it matches the expected pattern |
| 312 | - This prevents tests from failing when the build version changes |
| 313 | |
| 314 | ## Framework Environment Variables |
| 315 | |
| 316 | - **`TESTED_LOG2JOURNAL_BIN`** - Path to log2journal binary for testing (default: `log2journal` from PATH) |
| 317 | - Set before running `tests.sh` to test different builds |
| 318 | |
| 319 | The test framework provides comprehensive coverage of log2journal functionality with clean, maintainable test definitions. |