| 1 | # Netdata Functions: Developer Guide |
| 2 | |
| 3 | > **Note**: This is the practical developer guide. For the complete technical specification, see [Functions v3 Protocol Reference](/src/plugins.d/FUNCTION_UI_REFERENCE.md). For topology Functions, use the dedicated [Topology Function Schema](/src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md). |
| 4 | |
| 5 | ## Overview |
| 6 | |
| 7 | This guide teaches you how to create Netdata functions that provide interactive data through the web UI. You'll learn to build both simple tables and advanced log explorers. |
| 8 | |
| 9 | **What You'll Learn:** |
| 10 | - How to create simple table functions for system monitoring |
| 11 | - How to build log explorer functions with faceted search |
| 12 | - All column types, options, and their UI effects |
| 13 | - Query patterns, filtering, and aggregation |
| 14 | - Real-world examples and best practices |
| 15 | |
| 16 | **Quick Navigation:** |
| 17 | - [Part 1: Simple Table Functions](#part-1-simple-table-functions) - Basic monitoring data |
| 18 | - [Part 2: Log Explorer Functions](#part-2-log-explorer-functions) - Historical data with search |
| 19 | - [Part 3: Complete Options Reference](#part-3-complete-options-reference) - Every option explained |
| 20 | - [Topology Functions](/src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md) - Graph payloads, evidence rows, aggregation policy, and telemetry overlays |
| 21 | |
| 22 | --- |
| 23 | |
| 24 | # Part 1: Simple Table Functions |
| 25 | |
| 26 | Simple table functions display current system state - processes, connections, services, etc. They're perfect for "top-like" views and system monitoring. |
| 27 | |
| 28 | **Implementation Architecture:** |
| 29 | - **Frontend handles**: Filtering, search, facet counting, sorting |
| 30 | - **Backend provides**: Raw data, column definitions, optional charts |
| 31 | - **Performance**: Limited by browser processing (good for \<10k rows) |
| 32 | - **Query parameter**: Processed in frontend only (substring search) |
| 33 | - **Histograms**: Not supported |
| 34 | |
| 35 | ## Your First Simple Table Function |
| 36 | |
| 37 | Start with this minimal working example: |
| 38 | |
| 39 | ```json |
| 40 | { |
| 41 | "status": 200, |
| 42 | "type": "table", |
| 43 | "has_history": false, |
| 44 | "help": "Shows system services", |
| 45 | "data": [ |
| 46 | ["nginx", 25, "Running"], |
| 47 | ["mysql", 67, "Stopped"], |
| 48 | ["redis", 91, "Running"] |
| 49 | ], |
| 50 | "columns": { |
| 51 | "name": { |
| 52 | "index": 0, |
| 53 | "name": "Service", |
| 54 | "type": "string", |
| 55 | "unique_key": true |
| 56 | }, |
| 57 | "cpu": { |
| 58 | "index": 1, |
| 59 | "name": "CPU %", |
| 60 | "type": "bar-with-integer", |
| 61 | "max": 100 |
| 62 | }, |
| 63 | "status": { |
| 64 | "index": 2, |
| 65 | "name": "Status", |
| 66 | "type": "string" |
| 67 | } |
| 68 | } |
| 69 | } |
| 70 | ``` |
| 71 | |
| 72 | This creates a basic 3-column table showing service names, CPU usage bars, and status. |
| 73 | |
| 74 | ## Essential Features for Simple Tables |
| 75 | |
| 76 | ### 1. Required Fields |
| 77 | |
| 78 | Every simple table function needs: |
| 79 | |
| 80 | ```json |
| 81 | { |
| 82 | "status": 200, // HTTP status (200 = success) |
| 83 | "type": "table", // Always "table" |
| 84 | "has_history": false, // Simple table (not log explorer) |
| 85 | "columns": {...}, // Column definitions |
| 86 | "data": [...] // Your actual data |
| 87 | } |
| 88 | ``` |
| 89 | |
| 90 | ### 2. Column Definitions |
| 91 | |
| 92 | Each column must have: |
| 93 | |
| 94 | ```json |
| 95 | "column_id": { |
| 96 | "index": 0, // Position in data arrays (0-based) |
| 97 | "name": "Display Name", // Column header shown to users |
| 98 | "type": "string" // How to render the data |
| 99 | } |
| 100 | ``` |
| 101 | |
| 102 | ### 3. The Data Array |
| 103 | |
| 104 | Data is an array of arrays - each inner array is one row: |
| 105 | |
| 106 | ```json |
| 107 | "data": [ |
| 108 | ["nginx", 25, "Running"], // Row 1: index 0=name, 1=cpu, 2=status |
| 109 | ["mysql", 67, "Stopped"], // Row 2 |
| 110 | ["redis", 91, "Running"] // Row 3 |
| 111 | ] |
| 112 | ``` |
| 113 | |
| 114 | **Important**: Values must be in the same order as column `index` fields. |
| 115 | |
| 116 | ## Adding Visual Polish |
| 117 | |
| 118 | Make your table more useful with these enhancements: |
| 119 | |
| 120 | ```json |
| 121 | { |
| 122 | "status": 200, |
| 123 | "type": "table", |
| 124 | "has_history": false, |
| 125 | "help": "System services with CPU usage and status", |
| 126 | "data": [ |
| 127 | ["nginx", 25, "Running", {"rowOptions": {"severity": "normal"}}], |
| 128 | ["mysql", 67, "Stopped", {"rowOptions": {"severity": "error"}}], |
| 129 | ["redis", 91, "Running", {"rowOptions": {"severity": "warning"}}] |
| 130 | ], |
| 131 | "columns": { |
| 132 | "name": { |
| 133 | "index": 0, |
| 134 | "name": "Service Name", |
| 135 | "type": "string", |
| 136 | "unique_key": true, |
| 137 | "sticky": true, |
| 138 | "filter": "multiselect" |
| 139 | }, |
| 140 | "cpu": { |
| 141 | "index": 1, |
| 142 | "name": "CPU Usage", |
| 143 | "type": "bar-with-integer", |
| 144 | "units": "%", |
| 145 | "max": 100, |
| 146 | "sort": "descending", |
| 147 | "filter": "range" |
| 148 | }, |
| 149 | "status": { |
| 150 | "index": 2, |
| 151 | "name": "Status", |
| 152 | "type": "string", |
| 153 | "visualization": "pill", |
| 154 | "filter": "multiselect" |
| 155 | } |
| 156 | }, |
| 157 | "default_sort_column": "cpu" |
| 158 | } |
| 159 | ``` |
| 160 | |
| 161 | **New Features Added:** |
| 162 | - **Row coloring**: `rowOptions` with severity levels |
| 163 | - **Sticky columns**: Pin important columns when scrolling |
| 164 | - **Filters**: Let users filter by categories or numeric ranges |
| 165 | - **Progress bars**: Visual CPU usage display |
| 166 | - **Pills**: Status badges with colors |
| 167 | - **Default sorting**: Start sorted by CPU usage |
| 168 | |
| 169 | ## Limitations of Simple Tables |
| 170 | |
| 171 | Simple tables have certain limitations due to their frontend-only processing architecture: |
| 172 | |
| 173 | - **No histograms**: Time-based visualization is not supported |
| 174 | - **Performance limits**: Frontend processing is limited by browser memory (recommend \<10k rows) |
| 175 | - **No backend search**: Query parameter is not sent to backend - search happens client-side only |
| 176 | - **Static facet counts**: Counts are computed by frontend from all received data |
| 177 | |
| 178 | For large datasets or advanced log analysis features, consider using log explorers (`has_history: true`). |
| 179 | |
| 180 | ## Advanced Simple Table Features |
| 181 | |
| 182 | ### Aggregated Views |
| 183 | |
| 184 | Some functions can show both detailed and aggregated data. When enabled, facet pills show both counts: |
| 185 | |
| 186 | ```json |
| 187 | { |
| 188 | "aggregated_view": { |
| 189 | "column": "Count", |
| 190 | "results_label": "unique combinations", |
| 191 | "aggregated_label": "connections" |
| 192 | } |
| 193 | } |
| 194 | ``` |
| 195 | |
| 196 | This enables smart facet pills like `"15 ⊃ 42"` meaning "15 connections aggregated into 42 rows". |
| 197 | |
| 198 | ### Charts Integration |
| 199 | |
| 200 | Simple tables support interactive charts computed from your table data. The backend defines available charts, and the frontend computes and renders them. |
| 201 | |
| 202 | **Chart Types Available:** |
| 203 | - `"bar"` - Basic bar chart |
| 204 | - `"stacked-bar"` - Multi-column stacked bars (most common) |
| 205 | - `"doughnut"` - Pie/doughnut chart |
| 206 | - `"value"` - Simple numeric display |
| 207 | |
| 208 | **Example Configuration:** |
| 209 | ```json |
| 210 | { |
| 211 | "charts": { |
| 212 | "cpu_usage": { |
| 213 | "name": "CPU Usage by Service", |
| 214 | "type": "stacked-bar", |
| 215 | "columns": ["user_cpu", "system_cpu", "guest_cpu"], |
| 216 | "groupBy": "column", |
| 217 | "aggregation": "sum" |
| 218 | }, |
| 219 | "memory_breakdown": { |
| 220 | "name": "Memory Types", |
| 221 | "type": "doughnut", |
| 222 | "columns": ["resident", "virtual", "shared"], |
| 223 | "groupBy": "all", |
| 224 | "aggregation": "sum" |
| 225 | } |
| 226 | }, |
| 227 | "default_charts": [ |
| 228 | ["cpu_usage", "status"], |
| 229 | ["memory_breakdown", "status"] |
| 230 | ] |
| 231 | } |
| 232 | ``` |
| 233 | |
| 234 | **Chart Options:** |
| 235 | - **`groupBy`**: |
| 236 | - `"column"` (default): Group by selected filter column |
| 237 | - `"all"`: Aggregate all data together |
| 238 | - **`aggregation`**: `sum`, `mean`, `max`, `min`, `count` |
| 239 | |
| 240 | **How It Works:** |
| 241 | 1. Frontend takes your table data |
| 242 | 2. Groups by selected column (e.g., "service type") |
| 243 | 3. Aggregates values using specified method (e.g., sum CPU values) |
| 244 | 4. Renders chart with one bar/slice per group |
| 245 | |
| 246 | ### Table Row Grouping |
| 247 | |
| 248 | Simple tables support grouping rows with customizable aggregation. When users group by a column, rows with the same value are combined using the aggregation method you specify. |
| 249 | |
| 250 | **Enable Grouping Options:** |
| 251 | ```json |
| 252 | { |
| 253 | "group_by": { |
| 254 | "aggregated": [{ |
| 255 | "id": "by_status", |
| 256 | "name": "By Status", |
| 257 | "column": "status" |
| 258 | }, { |
| 259 | "id": "by_user", |
| 260 | "name": "By User", |
| 261 | "column": "user" |
| 262 | }] |
| 263 | } |
| 264 | } |
| 265 | ``` |
| 266 | |
| 267 | **Define Column Aggregation:** |
| 268 | Each column needs a summary type to control how values are aggregated when grouping: |
| 269 | |
| 270 | ```c |
| 271 | // In your C backend code |
| 272 | buffer_rrdf_table_add_field( |
| 273 | wb, field_id++, "cpu_percent", "CPU %", |
| 274 | RRDF_FIELD_TYPE_BAR_WITH_INTEGER, |
| 275 | RRDF_FIELD_VISUAL_BAR, |
| 276 | RRDF_FIELD_TRANSFORM_NUMBER, |
| 277 | 2, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL, |
| 278 | RRDF_FIELD_SUMMARY_SUM, // Sum CPU values when grouping |
| 279 | RRDF_FIELD_FILTER_RANGE, |
| 280 | RRDF_FIELD_OPTS_VISIBLE, NULL |
| 281 | ); |
| 282 | |
| 283 | buffer_rrdf_table_add_field( |
| 284 | wb, field_id++, "process_count", "Processes", |
| 285 | RRDF_FIELD_TYPE_INTEGER, |
| 286 | RRDF_FIELD_VISUAL_VALUE, |
| 287 | RRDF_FIELD_TRANSFORM_NUMBER, |
| 288 | 0, "processes", NAN, RRDF_FIELD_SORT_DESCENDING, NULL, |
| 289 | RRDF_FIELD_SUMMARY_COUNT, // Count processes when grouping |
| 290 | RRDF_FIELD_FILTER_RANGE, |
| 291 | RRDF_FIELD_OPTS_VISIBLE, NULL |
| 292 | ); |
| 293 | ``` |
| 294 | |
| 295 | **Available Summary/Aggregation Types:** |
| 296 | - `RRDF_FIELD_SUMMARY_COUNT` - Count rows in group |
| 297 | - `RRDF_FIELD_SUMMARY_SUM` - Sum numeric values |
| 298 | - `RRDF_FIELD_SUMMARY_MEAN` - Average values |
| 299 | - `RRDF_FIELD_SUMMARY_MIN` - Minimum value |
| 300 | - `RRDF_FIELD_SUMMARY_MAX` - Maximum value |
| 301 | - `RRDF_FIELD_SUMMARY_UNIQUECOUNT` - Count unique values |
| 302 | - `RRDF_FIELD_SUMMARY_MEDIAN` - Median value |
| 303 | |
| 304 | **Example Result:** |
| 305 | When user groups by "service type", rows like: |
| 306 | ``` |
| 307 | nginx_worker1 25% Running |
| 308 | nginx_worker2 30% Running |
| 309 | mysql_main 45% Running |
| 310 | ``` |
| 311 | |
| 312 | Become: |
| 313 | ``` |
| 314 | nginx 55% 2 processes (25% + 30% CPU, 2 processes counted) |
| 315 | mysql 45% 1 process (45% CPU, 1 process counted) |
| 316 | ``` |
| 317 | |
| 318 | --- |
| 319 | |
| 320 | # Part 2: Log Explorer Functions |
| 321 | |
| 322 | Log explorer functions (`has_history: true`) provide advanced log analysis with full-text search, faceted filtering, time navigation, and histograms. Perfect for systemd journals, event logs, and audit trails. |
| 323 | |
| 324 | **Implementation Architecture:** |
| 325 | - **Backend handles**: Query processing, pattern matching, facet counting, filtering |
| 326 | - **Frontend provides**: UI for facets, histogram display, infinite scroll |
| 327 | - **Performance**: Scales to millions of records with sampling and server-side filtering |
| 328 | - **Query parameter**: Processed by backend using Netdata simple patterns |
| 329 | - **Histograms**: Optional, generated by backend in Netdata chart format |
| 330 | - **Uses facets library**: All processing leverages `libnetdata/facets/` |
| 331 | |
| 332 | ### Complete Log Explorer Example |
| 333 | |
| 334 | ```json |
| 335 | { |
| 336 | "status": 200, |
| 337 | "type": "table", |
| 338 | "has_history": true, |
| 339 | "help": "System journal with real-time updates", |
| 340 | "accepted_params": [ |
| 341 | "info", "after", "before", "direction", "last", "anchor", |
| 342 | "query", "facets", "histogram", "if_modified_since", |
| 343 | "data_only", "delta", "tail", "sampling" |
| 344 | ], |
| 345 | "table": {"id": "journal", "has_history": true}, |
| 346 | "columns": { |
| 347 | "timestamp": { |
| 348 | "index": 0, |
| 349 | "name": "Time", |
| 350 | "type": "timestamp", |
| 351 | "transform": "datetime_usec", |
| 352 | "sort": "descending|fixed", |
| 353 | "sticky": true |
| 354 | }, |
| 355 | "priority": { |
| 356 | "index": 1, |
| 357 | "name": "Level", |
| 358 | "type": "string", |
| 359 | "visualization": "pill", |
| 360 | "filter": "facet", |
| 361 | "options": ["facet", "visible", "sticky"] |
| 362 | }, |
| 363 | "message": { |
| 364 | "index": 2, |
| 365 | "name": "Message", |
| 366 | "type": "string", |
| 367 | "full_width": true, |
| 368 | "options": ["full_width", "wrap", "visible", "main_text", "fts"] |
| 369 | } |
| 370 | }, |
| 371 | "data": [ |
| 372 | [1697644320000000, {"rowOptions": {"severity": "error"}}, "ERROR", "Service failed"], |
| 373 | [1697644319000000, {"rowOptions": {"severity": "normal"}}, "INFO", "Service started"] |
| 374 | ], |
| 375 | "facets": [ |
| 376 | { |
| 377 | "id": "priority", |
| 378 | "name": "Log Level", |
| 379 | "order": 1, |
| 380 | "options": [ |
| 381 | {"id": "ERROR", "name": "ERROR", "count": 45, "order": 1}, |
| 382 | {"id": "INFO", "name": "INFO", "count": 234, "order": 3} |
| 383 | ] |
| 384 | } |
| 385 | ], |
| 386 | "items": { |
| 387 | "evaluated": 50000, |
| 388 | "matched": 2520, |
| 389 | "returned": 100, |
| 390 | "max_to_return": 100 |
| 391 | }, |
| 392 | "anchor": { |
| 393 | "last_modified": 1697644320000000, |
| 394 | "direction": "backward" |
| 395 | } |
| 396 | } |
| 397 | ``` |
| 398 | |
| 399 | **Key Features Demonstrated:** |
| 400 | - `has_history: true` - Enables log explorer UI with infinite scroll |
| 401 | - `accepted_params` - Full parameter support for advanced features |
| 402 | - **Faceted search** - Real-time counts computed by backend |
| 403 | - **Anchor pagination** - Efficient navigation through large datasets |
| 404 | - **Microsecond timestamps** - High precision for log entries |
| 405 | - **Full-text search** - Backend pattern matching with `"fts"` fields |
| 406 | - **Row coloring** - Severity-based visual indicators |
| 407 | |
| 408 | ## Essential Log Explorer Features |
| 409 | |
| 410 | ### 1. Faceted Search Sidebar |
| 411 | |
| 412 | Facets provide dynamic filtering with real-time counts computed by the backend: |
| 413 | |
| 414 | ```json |
| 415 | { |
| 416 | "facets": [ |
| 417 | { |
| 418 | "id": "level", |
| 419 | "name": "Log Level", |
| 420 | "order": 1, |
| 421 | "options": [ |
| 422 | {"id": "ERROR", "name": "ERROR", "count": 45, "order": 1}, |
| 423 | {"id": "WARN", "name": "WARN", "count": 123, "order": 2}, |
| 424 | {"id": "INFO", "name": "INFO", "count": 2341, "order": 3} |
| 425 | ] |
| 426 | } |
| 427 | ] |
| 428 | } |
| 429 | ``` |
| 430 | |
| 431 | **Important**: These counts are calculated by the backend facets library during query execution, not by the frontend. The backend scans through matching records and maintains counters for each facet value. |
| 432 | |
| 433 | ### 2. Full-Text Search |
| 434 | |
| 435 | Enable powerful query search across all fields (processed by backend): |
| 436 | |
| 437 | ```json |
| 438 | { |
| 439 | "columns": { |
| 440 | "message": { |
| 441 | "options": ["fts"], // Make field full-text searchable |
| 442 | // ... other options |
| 443 | } |
| 444 | } |
| 445 | } |
| 446 | ``` |
| 447 | |
| 448 | **Important**: Unlike simple tables, the query parameter is sent to the backend and processed server-side using the facets library. The backend performs pattern matching and only returns matching records. |
| 449 | |
| 450 | **Query Patterns** (using Netdata simple patterns): |
| 451 | - `"error"` - Find "error" anywhere (substring match) |
| 452 | - `"error|warning"` - Find "error" OR "warning" (pipe separator) |
| 453 | - `"!debug"` - Exclude logs containing "debug" |
| 454 | - `"!*debugging*|*debug*"` - Include "debug" but exclude "debugging" |
| 455 | - `"connection failed"` - Find exact phrase (spaces are literal) |
| 456 | |
| 457 | **Pattern Evaluation Rules:** |
| 458 | 1. **Within field**: Left-to-right, first match wins |
| 459 | 2. **Across fields**: ALL fields evaluated, ANY negative match excludes row |
| 460 | 3. **Case-insensitive** matching throughout |
| 461 | |
| 462 | ### 3. Time-Based Histograms |
| 463 | |
| 464 | Visualize log distribution over time (log explorers only): |
| 465 | |
| 466 | ```json |
| 467 | { |
| 468 | "available_histograms": [ |
| 469 | {"id": "level", "name": "Log Level", "order": 1}, |
| 470 | {"id": "source", "name": "Source", "order": 2} |
| 471 | ], |
| 472 | "histogram": { |
| 473 | "id": "level", |
| 474 | "name": "Log Level", |
| 475 | "chart": { |
| 476 | "result": { |
| 477 | "labels": ["time", "ERROR", "WARN", "INFO"], |
| 478 | "data": [ |
| 479 | [1697644200, 5, 12, 234], |
| 480 | [1697644260, 3, 8, 198] |
| 481 | ] |
| 482 | } |
| 483 | } |
| 484 | } |
| 485 | } |
| 486 | ``` |
| 487 | |
| 488 | **Important**: Histograms are NOT supported for simple tables (`has_history: false`). They are optional for log explorers and use the same format as Netdata's `/api/v3/data` endpoint. The backend generates histogram data using the facets library. |
| 489 | |
| 490 | ### 4. Anchor-Based Navigation |
| 491 | |
| 492 | Handle large datasets with efficient pagination: |
| 493 | |
| 494 | ```json |
| 495 | { |
| 496 | "items": { |
| 497 | "evaluated": 50000, // Total scanned |
| 498 | "matched": 2520, // Match filters |
| 499 | "returned": 100, // In this response |
| 500 | "max_to_return": 100 |
| 501 | }, |
| 502 | "anchor": { |
| 503 | "last_modified": 1697644320000000, |
| 504 | "direction": "backward" |
| 505 | } |
| 506 | } |
| 507 | ``` |
| 508 | |
| 509 | ### Parameter Integration Patterns |
| 510 | |
| 511 | **Basic Monitoring Function:** |
| 512 | ```json |
| 513 | { |
| 514 | "accepted_params": ["info", "after", "before"] |
| 515 | } |
| 516 | ``` |
| 517 | |
| 518 | **Advanced Log Function:** |
| 519 | ```json |
| 520 | { |
| 521 | "accepted_params": [ |
| 522 | "info", "after", "before", "direction", "last", "anchor", |
| 523 | "query", "facets", "histogram", "if_modified_since", |
| 524 | "data_only", "delta", "tail", "sampling", "slice" |
| 525 | ] |
| 526 | } |
| 527 | ``` |
| 528 | |
| 529 | **UI Feature Enablement:** |
| 530 | - `"slice"` → Enables "Full data queries" toggle |
| 531 | - `"direction"` → Enables bidirectional pagination |
| 532 | - `"tail"` → Enables streaming mode |
| 533 | - `"delta"` → Enables incremental updates |
| 534 | - `"query"` → Enables full-text search |
| 535 | |
| 536 | ## Advanced Log Explorer Features |
| 537 | |
| 538 | ### Column Options for Logs |
| 539 | |
| 540 | Log explorers support special column options: |
| 541 | |
| 542 | | Option | Effect | |
| 543 | |--------|--------| |
| 544 | | `"fts"` | Full-text searchable by query | |
| 545 | | `"facet"` | Appears in sidebar filters with counts | |
| 546 | | `"main_text"` | Primary content (usually message) | |
| 547 | | `"rich_text"` | May contain formatting | |
| 548 | | `"hidden"` | Hide by default | |
| 549 | |
| 550 | Example: |
| 551 | ```json |
| 552 | { |
| 553 | "message": { |
| 554 | "type": "string", |
| 555 | "full_width": true, |
| 556 | "options": ["full_width", "wrap", "visible", "main_text", "fts", "rich_text"] |
| 557 | } |
| 558 | } |
| 559 | ``` |
| 560 | |
| 561 | ### Log-Specific UI Behavior |
| 562 | |
| 563 | When `has_history: true`: |
| 564 | - **Sidebar**: Shows faceted filters instead of simple filters |
| 565 | - **Search box**: Queries all `fts` fields using pattern matching |
| 566 | - **Infinite scroll**: Loads more data as you scroll |
| 567 | - **Time navigation**: Jump to specific time periods |
| 568 | - **Sampling**: For very large datasets |
| 569 | |
| 570 | --- |
| 571 | |
| 572 | # Part 3: Complete Options Reference |
| 573 | |
| 574 | This section documents every field type, option, and behavior for quick reference while developing. |
| 575 | |
| 576 | ## Field Types and UI Rendering |
| 577 | |
| 578 | ### Text and Categories |
| 579 | |
| 580 | ```json |
| 581 | { |
| 582 | "type": "string", |
| 583 | "visualization": "value" // Default: plain text, left-aligned |
| 584 | } |
| 585 | ``` |
| 586 | |
| 587 | ```json |
| 588 | { |
| 589 | "type": "string", |
| 590 | "visualization": "pill" // Colored badges for status/categories |
| 591 | } |
| 592 | ``` |
| 593 | |
| 594 | ### Numbers and Metrics |
| 595 | |
| 596 | ```json |
| 597 | { |
| 598 | "type": "integer", // Right-aligned numbers |
| 599 | "transform": "number", // Respect decimal_points |
| 600 | "decimal_points": 2 |
| 601 | } |
| 602 | ``` |
| 603 | |
| 604 | ```json |
| 605 | { |
| 606 | "type": "bar-with-integer", // Progress bars with values |
| 607 | "max": 100, // Required for bars |
| 608 | "units": "%" |
| 609 | } |
| 610 | ``` |
| 611 | |
| 612 | ### Timestamp Format Requirements |
| 613 | |
| 614 | **Simple Tables**: Use milliseconds with `datetime` transform |
| 615 | ```json |
| 616 | { |
| 617 | "type": "timestamp", |
| 618 | "transform": "datetime", // Expects milliseconds |
| 619 | "data": [1697644320000] // JavaScript Date format |
| 620 | } |
| 621 | ``` |
| 622 | |
| 623 | **Log Explorers**: Use microseconds with `datetime_usec` transform |
| 624 | ```json |
| 625 | { |
| 626 | "type": "timestamp", |
| 627 | "transform": "datetime_usec", // Expects microseconds |
| 628 | "data": [1697644320000000] // Microsecond precision |
| 629 | } |
| 630 | ``` |
| 631 | |
| 632 | **Frontend Conversion:** |
| 633 | ```javascript |
| 634 | // datetime_usec automatically converts to milliseconds |
| 635 | if (usec) { |
| 636 | epoch = epoch ? Math.floor(epoch / 1000) : epoch |
| 637 | } |
| 638 | ``` |
| 639 | |
| 640 | **API Parameters**: `after` and `before` are automatically converted from milliseconds to seconds when sent to functions. |
| 641 | |
| 642 | ```json |
| 643 | { |
| 644 | "type": "duration", |
| 645 | "transform": "duration_s" // Formats seconds as "1d 2h 3m" |
| 646 | } |
| 647 | ``` |
| 648 | |
| 649 | ### Rich Content |
| 650 | |
| 651 | ```json |
| 652 | { |
| 653 | "type": "feedTemplate", // Full-width rich content |
| 654 | "full_width": true // Automatically applied |
| 655 | } |
| 656 | ``` |
| 657 | |
| 658 | ## Essential Column Options |
| 659 | |
| 660 | ### Layout Control |
| 661 | |
| 662 | ```json |
| 663 | { |
| 664 | "unique_key": true, // Row identifier (exactly one required) |
| 665 | "sticky": true, // Pin when scrolling horizontally |
| 666 | "visible": true, // Show by default |
| 667 | "full_width": true, // Expand to fill available space |
| 668 | "wrap": true // Enable text wrapping |
| 669 | } |
| 670 | ``` |
| 671 | |
| 672 | ### Filtering Options |
| 673 | |
| 674 | ```json |
| 675 | { |
| 676 | "filter": "multiselect" // Checkboxes (default for simple tables) |
| 677 | } |
| 678 | ``` |
| 679 | |
| 680 | ```json |
| 681 | { |
| 682 | "filter": "range" // Min/max sliders for numbers |
| 683 | } |
| 684 | ``` |
| 685 | |
| 686 | ```json |
| 687 | { |
| 688 | "filter": "facet" // Sidebar with counts (log explorers only) |
| 689 | } |
| 690 | ``` |
| 691 | |
| 692 | ### Sorting Options |
| 693 | |
| 694 | ```json |
| 695 | { |
| 696 | "sort": "descending", // Default sort direction |
| 697 | "sortable": true // Allow user sorting (default) |
| 698 | } |
| 699 | ``` |
| 700 | |
| 701 | ```json |
| 702 | { |
| 703 | "sort": "descending|fixed", // Prevent user from changing sort |
| 704 | "sortable": false |
| 705 | } |
| 706 | ``` |
| 707 | |
| 708 | ## Row-Level Features |
| 709 | |
| 710 | ### Row Coloring by Severity |
| 711 | |
| 712 | Add row coloring by including `rowOptions` as the last element: |
| 713 | |
| 714 | ```json |
| 715 | { |
| 716 | "data": [ |
| 717 | ["normal data", "values", {"rowOptions": {"severity": "normal"}}], |
| 718 | ["warning data", "values", {"rowOptions": {"severity": "warning"}}], |
| 719 | ["error data", "values", {"rowOptions": {"severity": "error"}}], |
| 720 | ["notice data", "values", {"rowOptions": {"severity": "notice"}}] |
| 721 | ] |
| 722 | } |
| 723 | ``` |
| 724 | |
| 725 | **Severity Levels:** |
| 726 | - `"normal"` - Default appearance |
| 727 | - `"warning"` - Yellow background |
| 728 | - `"error"` - Red background |
| 729 | - `"notice"` - Blue background |
| 730 | |
| 731 | ## Chart and Grouping Configuration |
| 732 | |
| 733 | ### Chart Definition |
| 734 | |
| 735 | ```json |
| 736 | { |
| 737 | "charts": { |
| 738 | "resource_usage": { |
| 739 | "name": "Resource Usage", |
| 740 | "type": "stacked-bar", |
| 741 | "columns": ["cpu", "memory", "disk"], |
| 742 | "groupBy": "column", |
| 743 | "aggregation": "sum" |
| 744 | }, |
| 745 | "status_distribution": { |
| 746 | "name": "Status Distribution", |
| 747 | "type": "doughnut", |
| 748 | "columns": ["count"], |
| 749 | "groupBy": "all", |
| 750 | "aggregation": "count" |
| 751 | } |
| 752 | }, |
| 753 | "default_charts": [ |
| 754 | ["resource_usage", "service_type"], |
| 755 | ["status_distribution", "status"] |
| 756 | ] |
| 757 | } |
| 758 | ``` |
| 759 | |
| 760 | ### Grouping Configuration |
| 761 | |
| 762 | ```json |
| 763 | { |
| 764 | "group_by": { |
| 765 | "aggregated": [ |
| 766 | { |
| 767 | "id": "by_service", |
| 768 | "name": "By Service Type", |
| 769 | "column": "service_type" |
| 770 | }, |
| 771 | { |
| 772 | "id": "by_status", |
| 773 | "name": "By Status", |
| 774 | "column": "status" |
| 775 | } |
| 776 | ] |
| 777 | } |
| 778 | } |
| 779 | ``` |
| 780 | |
| 781 | ## Backend Implementation Examples |
| 782 | |
| 783 | ### C Code for Simple Tables |
| 784 | |
| 785 | ```c |
| 786 | // Add a progress bar column |
| 787 | buffer_rrdf_table_add_field( |
| 788 | wb, field_id++, "cpu", "CPU Usage", |
| 789 | RRDF_FIELD_TYPE_BAR_WITH_INTEGER, |
| 790 | RRDF_FIELD_VISUAL_BAR, |
| 791 | RRDF_FIELD_TRANSFORM_NUMBER, |
| 792 | 2, "%", 100.0, RRDF_FIELD_SORT_DESCENDING, NULL, |
| 793 | RRDF_FIELD_SUMMARY_SUM, |
| 794 | RRDF_FIELD_FILTER_RANGE, |
| 795 | RRDF_FIELD_OPTS_VISIBLE, NULL |
| 796 | ); |
| 797 | |
| 798 | // Add row coloring |
| 799 | buffer_json_add_array_item_object(wb); |
| 800 | buffer_json_member_add_object(wb, "rowOptions"); |
| 801 | buffer_json_member_add_string(wb, "severity", "error"); |
| 802 | buffer_json_object_close(wb); |
| 803 | buffer_json_object_close(wb); |
| 804 | ``` |
| 805 | |
| 806 | ### Using the Facets Library (Log Explorers) |
| 807 | |
| 808 | ```c |
| 809 | // Initialize facets for log exploration |
| 810 | FACETS *facets = facets_create(...); |
| 811 | |
| 812 | // Set up query search |
| 813 | facets_set_query(facets, query_string); |
| 814 | |
| 815 | // Add a faceted field |
| 816 | facets_register_facet_id(facets, "level", |
| 817 | FACET_KEY_OPTION_FACET | FACET_KEY_OPTION_FTS | FACET_KEY_OPTION_REORDER); |
| 818 | |
| 819 | // Generate the response |
| 820 | facets_table_config(facets, wb); |
| 821 | ``` |
| 822 | |
| 823 | ## Error Handling |
| 824 | |
| 825 | Always handle the info request first: |
| 826 | |
| 827 | ```json |
| 828 | { |
| 829 | "v": 3, // Enable POST requests |
| 830 | "status": 200, |
| 831 | "type": "table", |
| 832 | "has_history": false, // or true for log explorers |
| 833 | "help": "Function description", |
| 834 | "accepted_params": [...], |
| 835 | "required_params": [...] |
| 836 | } |
| 837 | ``` |
| 838 | |
| 839 | ### Complete Info Response Example |
| 840 | |
| 841 | ```json |
| 842 | { |
| 843 | "v": 3, |
| 844 | "status": 200, |
| 845 | "type": "table", |
| 846 | "has_history": true, |
| 847 | "help": "System log explorer with faceted search", |
| 848 | "accepted_params": [ |
| 849 | "info", "after", "before", "direction", "last", "anchor", |
| 850 | "query", "facets", "histogram", "if_modified_since", |
| 851 | "data_only", "delta", "tail", "sampling" |
| 852 | ], |
| 853 | "required_params": [ |
| 854 | { |
| 855 | "id": "unit", |
| 856 | "name": "System Unit", |
| 857 | "type": "select", |
| 858 | "options": [ |
| 859 | {"id": "nginx.service", "name": "Nginx"}, |
| 860 | {"id": "mysql.service", "name": "MySQL"} |
| 861 | ] |
| 862 | } |
| 863 | ] |
| 864 | } |
| 865 | ``` |
| 866 | |
| 867 | **Critical Fields:** |
| 868 | - `"v": 3` enables POST requests with JSON payloads |
| 869 | - `accepted_params` determines which parameters the function accepts |
| 870 | - `required_params` generates filter UI and validates execution |
| 871 | |
| 872 | For errors, return: |
| 873 | |
| 874 | ```json |
| 875 | { |
| 876 | "status": 400, |
| 877 | "errorMessage": "Descriptive error message" |
| 878 | } |
| 879 | ``` |
| 880 | |
| 881 | **Compatibility note (cloud-frontend):** |
| 882 | - The Functions UI expects `errorMessage` (camelCase) and does **not** camelize error payloads. |
| 883 | |
| 884 | ### Performance Optimization |
| 885 | |
| 886 | **For Large Datasets:** |
| 887 | |
| 888 | 1. **Enable Sampling**: |
| 889 | ```json |
| 890 | { |
| 891 | "accepted_params": ["sampling"], |
| 892 | "sampling": 10 // 1 in 10 sampling |
| 893 | } |
| 894 | ``` |
| 895 | |
| 896 | 2. **Use Delta Updates**: |
| 897 | ```json |
| 898 | { |
| 899 | "if_modified_since": 1697644320000000, |
| 900 | "delta": true, |
| 901 | "data_only": true |
| 902 | } |
| 903 | ``` |
| 904 | |
| 905 | 3. **Implement Tail Limiting**: |
| 906 | ```c |
| 907 | // Limit tail data to prevent memory issues |
| 908 | if (tail_mode) { |
| 909 | limit_results_to(500); |
| 910 | } |
| 911 | ``` |
| 912 | |
| 913 | 4. **Support Anchor Pagination**: |
| 914 | ```json |
| 915 | { |
| 916 | "pagination": { |
| 917 | "enabled": true, |
| 918 | "column": "timestamp", |
| 919 | "key": "anchor", |
| 920 | "units": "timestamp_usec" |
| 921 | } |
| 922 | } |
| 923 | ``` |
| 924 | |
| 925 | ## Best Practices Summary |
| 926 | |
| 927 | ### For Simple Tables |
| 928 | 1. Always include one `unique_key` column |
| 929 | 2. Use `bar-with-integer` for metrics with `max` values |
| 930 | 3. Add `filter: "range"` for numbers, `"multiselect"` for categories |
| 931 | 4. Use `rowOptions` for status indication |
| 932 | 5. Set meaningful `default_sort_column` |
| 933 | 6. Define column `summary` types for grouping (SUM for metrics, COUNT for processes) |
| 934 | 7. Add charts for key metrics with appropriate `groupBy` and `aggregation` |
| 935 | 8. Include `group_by` options for common analysis patterns |
| 936 | |
| 937 | ### For Log Explorers |
| 938 | 1. Set `has_history: true` |
| 939 | 2. Use microsecond timestamps with `datetime_usec` |
| 940 | 3. Mark important fields with `"fts"` for search |
| 941 | 4. Use `filter: "facet"` instead of `"multiselect"` |
| 942 | 5. Include proper facets with counts |
| 943 | 6. Implement anchor-based pagination |
| 944 | |
| 945 | ### General |
| 946 | 1. Test with `?info` requests first |
| 947 | 2. Include helpful `help` text |
| 948 | 3. Handle errors gracefully |
| 949 | 4. Use appropriate `units` for clarity |
| 950 | 5. Follow existing function patterns in your codebase |
| 951 | |
| 952 | |
| 953 | --- |
| 954 | |
| 955 | This guide covers everything you need to build both simple monitoring functions and advanced log explorers. For implementation details and edge cases, see [FUNCTIONS_REFERENCE.md](/src/plugins.d/FUNCTION_UI_REFERENCE.md). |