421c60bb0 · cryptotaxi247/netdata-1

@@ -1,193 +1,1036 @@

1 -# charttpl

1 +# Chart Template Format

2

3 -`charttpl` defines the chart-template DSL used by `ModuleV2` collectors.

4 -It is the input schema consumed by `chartengine`.

3 +## Overview

4

6 -**Audience**: `ModuleV2` collector authors and framework contributors.

5 +A **chart template** defines _how a collector's metrics are organized into charts_ in the Netdata dashboard.

6

8 -**See also**: [metrix](/src/go/pkg/metrix/README.md) (metrics storage and read API),

9 -[chartengine](/src/go/plugin/framework/chartengine/README.md) (compile + plan).

7 +> [!NOTE]

8 +> Chart templates are declarative YAML files. You describe **what** to chart, and the engine handles creating, updating, and removing chart instances at runtime.

9

11 -## Purpose

10 +It tells the chart engine:

11

13 -| Package | Role |

14 -|---------------|------------------------------------------------------------------------------|

15 -| `charttpl` | Parse + default + semantic validation of chart-template YAML |

16 -| `chartengine` | Compile template into immutable program and build create/update/remove plans |

12 +- which **metrics** to include

13 +- how to **group** them into charts and families

14 +- how to **name** and **scale** dimensions

15 +- how to create **per-instance charts** (e.g., one chart per host, per disk, per user)

16

18 -## Processing Pipeline

17 +Each collector has a single `charts.yaml` file that describes all its charts.

18

20 -1. **Decode** — Strict YAML decode (`yaml.UnmarshalStrict`). Unknown fields fail decoding.

21 -2. **Defaults** — `version` defaults to `v1` (top-level only); chart `type` defaults to `line`; inheritable `group.chart_defaults` are applied recursively using nearest-group-wins replace semantics.

22 -3. **Semantic validation** — Field-level checks with path-aware errors (metric scoping, selectors, dimension rules, lifecycle bounds).

19 +> [!TIP]

20 +> Groups can be nested to **any depth**. Family paths, context namespaces, and metric scopes compose automatically as you nest — no need to repeat prefixes. See [groups](#4-groups) for the full composition rules and examples.

21

24 -## Syntax (v1)

22 +### How Chart Templates Work

23

26 -### Top-level fields

24 +When a collector runs, the chart engine:

25

29 -|---------------------|--------|----------|---------------------------------------------------------------|

26 +1. Reads the collector's `charts.yaml` file.

27 +2. Compiles it into an immutable program (validates, resolves defaults, infers algorithms).

28 +3. On each collection cycle, matches incoming metrics against dimension selectors.

29 +4. Creates chart instances dynamically based on instance identity labels.

30 +5. Updates dimension values every cycle; removes stale instances based on lifecycle policy.

31

35 -### `groups[]`

32 +**Template Lifecycle**

33

38 -|---------------------|---------------|----------|--------------------------------------------------------------------------------------|

34 +```text

35 + charts.yaml

36 + |

37 + v

38 + ┌─────────────────────┐

39 + │ Decode & Validate │ strict YAML parse + semantic checks

40 + └──────────┬──────────┘

41 + v

42 + ┌─────────────────────┐

43 + │ Compile (engine) │ selector parsing, algorithm inference,

44 + │ │ context/family/ID composition

45 + └──────────┬──────────┘

46 + v

47 + ┌─────────────────────┐

48 + │ Runtime (per cycle)│ match series → create/update/remove

49 + │ │ charts and dimensions

50 + └─────────────────────┘

51 +```

52 +

53 +### Example: Complete Chart Template

54 +

55 +The example below shows a single template that covers all common metric kinds:

56 +gauge, counter, histogram, summary, and stateset. Inline comments explain each field.

57 +

58 +```yaml

59 +version: v1 # schema version (only "v1" supported)

60 +context_namespace: myapp # prefix for all chart contexts → myapp.<group context>.<chart context>

61 +

62 +groups:

63 + # ── Gauge: point-in-time values (algorithm: absolute) ──────────────

64 + - family: Resources

65 + metrics: # metrics visible to dimension selectors in this group

66 + - memory_used_bytes

67 + - memory_total_bytes

68 + - cpu_usage_percent

69 + charts:

70 + - id: memory_usage

71 + title: Memory Usage

72 + context: memory_usage # final context: myapp.memory_usage

73 + units: bytes

74 + type: stacked # line (default), area, stacked, heatmap

75 + algorithm: absolute # absolute = raw value; incremental = rate (value - prev) / interval

76 + instances:

77 + by_labels: [host] # one chart per unique "host" label value

78 + dimensions:

79 + - selector: memory_used_bytes

80 + name: used # static dimension name

81 + - selector: memory_total_bytes

82 + name: total

83 + options:

84 + hidden: true # collected but not drawn (useful for % calculations)

85 +

86 + - id: cpu_usage

87 + title: CPU Usage

88 + context: cpu_usage

89 + units: percentage

90 + instances:

91 + by_labels: [host]

92 + dimensions:

93 + - selector: cpu_usage_percent

94 + name: used

95 + options:

96 + divisor: 100 # raw value in basis points → divide by 100 for percent

97 + float: true # use floating-point precision

98 +

99 + # ── Counter: monotonically increasing values (algorithm: incremental) ──

100 + - family: Traffic

101 + metrics:

102 + - http_requests_total

103 + - bytes_received

104 + - bytes_sent

105 + charts:

106 + - id: http_requests

107 + title: HTTP Requests

108 + context: http_requests

109 + units: requests/s

110 + algorithm: incremental # engine computes rate: (current - previous) / interval

111 + instances:

112 + by_labels: [host]

113 + dimensions:

114 + - selector: http_requests_total

115 + name_from_label: method # dynamic name: each unique label value becomes a dimension

116 + # e.g., method="GET" → dim "GET", method="POST" → dim "POST"

117 +

118 + - id: bandwidth

119 + title: Network Bandwidth

120 + context: bandwidth

121 + units: kilobits/s

122 + type: area

123 + algorithm: incremental

124 + instances:

125 + by_labels: [host]

126 + dimensions:

127 + - selector: bytes_received

128 + name: in

129 + options:

130 + multiplier: 8 # bytes → bits

131 + divisor: 1000 # bits → kilobits

132 + - selector: bytes_sent

133 + name: out

134 + options:

135 + multiplier: -8 # negative = drawn below zero line (bidirectional chart)

136 + divisor: 1000

137 +

138 + # ── Histogram: bucketed distribution (flattened into _bucket, _count, _sum) ──

139 + - family: Latency

140 + metrics:

141 + - request_duration_seconds_bucket

142 + - request_duration_seconds_count

143 + - request_duration_seconds_sum

144 + charts:

145 + - id: request_duration_buckets

146 + title: Request Duration Buckets

147 + context: request_duration_buckets

148 + units: observations/s

149 + type: stacked

150 + algorithm: incremental # histogram buckets are counters

151 + instances:

152 + by_labels: [host]

153 + dimensions:

154 + - selector: request_duration_seconds_bucket

155 + # no name, no name_from_label → engine infers dimension names

156 + # from the "le" (less-than-or-equal) label automatically:

157 + # le="0.005" → dim "0.005", le="0.01" → dim "0.01", etc.

158 +

159 + - id: request_rate

160 + title: Request Rate

161 + context: request_rate

162 + units: requests/s

163 + algorithm: incremental

164 + instances:

165 + by_labels: [host]

166 + dimensions:

167 + - selector: request_duration_seconds_count

168 + name: requests

169 +

170 + # ── Summary: quantile distribution (flattened into quantile values, _count, _sum) ──

171 + - family: Response Time

172 + metrics:

173 + - response_time_seconds

174 + - response_time_seconds_count

175 + - response_time_seconds_sum

176 + charts:

177 + - id: response_time_quantiles

178 + title: Response Time Quantiles

179 + context: response_time_quantiles

180 + units: seconds

181 + algorithm: absolute # quantile values are gauges, not counters

182 + instances:

183 + by_labels: [host]

184 + dimensions:

185 + - selector: response_time_seconds

186 + # no name, no name_from_label → engine infers dimension names

187 + # from the "quantile" label automatically:

188 + # quantile="0.5" → dim "0.5", quantile="0.99" → dim "0.99", etc.

189 + options:

190 + float: true

191 +

192 + # ── StateSet: named boolean states (exactly one active at a time) ──

193 + - family: Health

194 + metrics:

195 + - service_status

196 + charts:

197 + - id: service_health

198 + title: Service Health Status

199 + context: service_health

200 + units: state

201 + instances:

202 + by_labels: [host]

203 + dimensions:

204 + - selector: service_status

205 + # no name, no name_from_label → engine infers dimension names

206 + # from the metric-name label (service_status=<value>) automatically:

207 + # service_status="ready" → dim "ready", service_status="degraded" → dim "degraded", etc.

208 +```

209 +

210 +**What this template produces** — if the collector reports metrics for 2 hosts (`host="web-1"`, `host="web-2"`), the engine creates **2 instances of every chart** (one per host). The histogram bucket chart gets one dimension per `le` boundary, the summary chart gets one per quantile, and the stateset chart gets one per state — all named automatically by the engine.

211 +

212 +## Template Structure

213 +

214 +Every `charts.yaml` follows this structure:

215 +

216 +```yaml

217 +version: <schema version>

218 +context_namespace: <context prefix>

219 +engine: <engine policy>

220 +groups:

221 + - family: <family name>

222 + context_namespace: <context segment>

223 + metrics: <available metrics>

224 + chart_defaults: <inheritable defaults>

225 + charts: <chart definitions>

226 + groups: <nested groups>

227 +```

228 +

229 +| Section | Purpose |

230 +|-----------------------------------------------|----------------------------------------------------|

231 +| [**version**](#1-version) | Schema version (must be `v1`). |

232 +| [**context_namespace**](#2-context_namespace) | Top-level prefix for chart context paths. |

233 +| [**engine**](#3-engine) | Engine-level policy (selectors, autogeneration). |

234 +| [**groups**](#4-groups) | Recursive chart groups — the core of the template. |

235 +

236 +---

237

46 -**Context composition**: The final chart context is built by joining all context parts with `.`:

47 -`<top context_namespace>.<group context_namespace>...<chart.context>`.

48 -For example, with top-level `context_namespace: netdata.go.plugin`, group `context_namespace: mysql`,

49 -and chart `context: queries`, the resulting context is `netdata.go.plugin.mysql.queries`.

238 +## Field Reference

239 +

240 +### 1. version

241 +

242 +Schema version. Currently only `v1` is supported. Defaults to `v1` if omitted.

243 +

244 +```yaml

245 +version: v1

246 +```

247 +

248 +### 2. context_namespace

249 +

250 +Top-level prefix for all chart context paths in the template. Combined with group-level `context_namespace` and chart `context` to form the final context.

251 +

252 +```yaml

253 +context_namespace: mysql

254 +```

255 +

256 +**Context composition** — the final chart context is built by joining all context parts with `.`:

257 +

258 +```

259 +<top context_namespace>.<group context_namespace>...<chart.context>

260 +```

261

51 -### `chart_defaults`

262 +For example:

263

53 -`chart_defaults` applies only to descendant charts of the current group. Supported fields are:

264 +| Level | Value |

265 +|-------------------------------|---------------------|

266 +| Top-level `context_namespace` | `mysql` |

267 +| Group `context_namespace` | _(empty)_ |

268 +| Chart `context` | `queries` |

269 +| **Resulting context** | **`mysql.queries`** |

270 +

271 +### 3. engine

272 +

273 +Template-level policy that controls metric filtering and autogeneration.

274 +

275 +```yaml

276 +engine:

277 + selector:

278 + allow: ["cpu_*", "memory_*"]

279 + deny: ["cpu_guest_*"]

280 + autogen:

281 + enabled: true

282 + expire_after_success_cycles: 50

283 +```

284 +

286 +|---------------------------------------|---------------|-------------|----------------------------------------------------------------------------------------|

290 +| `autogen.max_type_id_len` | int | `0` (=1200) | Max full `type.id` length. Must be `0` or `>= 4`. |

292 +

293 +**When to use autogen**: For collectors like Nagios plugins where the set of metrics is unpredictable and user-defined. The engine creates a chart for every unmatched metric automatically.

294 +

295 +**Example: Nagios collector with autogeneration**

296 +

297 +```yaml

298 +version: v1

299 +context_namespace: nagios

300 +engine:

301 + autogen:

302 + enabled: true

303 + expire_after_success_cycles: 50

304 +groups:

305 + - family: Job

306 + context_namespace: job

307 + groups:

308 + - family: Execution

309 + metrics:

310 + - nagios.job.execution_state

311 + - nagios.job.execution_duration

312 + charts:

313 + - id: job_execution_state

314 + title: Job Execution State

315 + context: execution_state

316 + units: state

317 + instances:

318 + by_labels: [nagios_job]

319 + dimensions:

320 + - selector: nagios.job.execution_state

321 + - id: job_execution_duration

322 + title: Execution Duration

323 + context: execution_duration

324 + units: seconds

325 + instances:

326 + by_labels: [nagios_job]

327 + dimensions:

328 + - selector: nagios.job.execution_duration

329 + name: duration

330 + options:

331 + float: true

332 +```

333 +

334 +Explicitly defined charts (like `execution_state`) use the template. Any _other_ metrics the Nagios plugin emits get auto-charted by the engine.

335 +

336 +### 4. groups

337 +

338 +Groups organize charts into a hierarchy that can be nested to **any depth**. Each group defines a **family** segment, can declare **metrics** in scope, and contains **charts** and/or nested **groups**.

339 +

340 +Nesting serves three purposes:

341 +

342 +1. **Family composition** — each level's `family` is joined with `/`, producing Netdata's hierarchical family structure automatically (the UI renders `/`-separated families as navigable levels).

343 +2. **Context composition** — each level's `context_namespace` is joined with `.`, so you write short context leaves instead of long prefixed strings.

344 +3. **Metric scoping** — metrics declared in a group are inherited by all descendants, so you declare once at the appropriate level.

345 +

346 +```yaml

347 +groups:

348 + - family: <family name>

349 + context_namespace: <optional context segment>

350 + metrics:

351 + - <metric_name>

352 + chart_defaults:

353 + label_promotion: [<label>, ...]

354 + instances:

355 + by_labels: [<label>, ...]

356 + charts:

357 + - <chart definition>

358 + groups:

359 + - <nested group>

360 +```

361 +

363 +|---------------------|---------------|----------|-------------------------------------------------------------------------------------|

370 +

371 +**Family composition** — group families compose hierarchically. The final chart family is built by joining all group `family` segments and the chart's own `family` (if set) with `/`:

372 +

373 +| Level | Family value |

374 +|----------------------|-----------------------------------------|

375 +| Root group | `Storage Engine` |

376 +| Nested group | `InnoDB` |

377 +| Nested group | `Buffer Pool` |

378 +| **Resulting family** | **`Storage Engine/InnoDB/Buffer Pool`** |

379 +

380 +Here is a real-world nesting example showing how family and context compose at each level:

381 +

382 +```yaml

383 +# context_namespace: mysql (set at top level)

384 +groups: # family context

385 + - family: Storage Engine # Storage Engine (inherited)

386 + groups:

387 + - family: InnoDB # Storage Engine/InnoDB (inherited)

388 + groups:

389 + - family: Buffer Pool # Storage Engine/InnoDB/Buffer Pool

390 + charts:

391 + - context: pages # → mysql.pages

392 + - family: I/O # Storage Engine/InnoDB/I/O

393 + charts:

394 + - context: bandwidth # → mysql.bandwidth

395 + - family: MyISAM # Storage Engine/MyISAM

396 + charts:

397 + - context: key_blocks # → mysql.key_blocks

398 +```

399 +

400 +Without nesting, you would repeat `Storage Engine/InnoDB/` in every chart's family and `mysql.` in every context. Nesting eliminates that repetition and makes the structure self-documenting.

401 +

402 +> [!WARNING]

403 +> Dimensions can only reference metrics declared in their group or any ancestor group. Referencing a metric not in scope produces a validation error.

404 +

405 +**Metric scoping** — this prevents accidental cross-references and keeps templates self-documenting:

406 +

407 +```yaml

408 +groups:

409 + - family: Database

410 + metrics:

411 + - queries_total # visible to all charts in this group and nested groups

412 + groups:

413 + - family: Cache

414 + metrics:

415 + - cache_hits # visible only in this group and its descendants

416 + charts:

417 + - title: Cache Performance

418 + context: cache

419 + units: hits/s

420 + dimensions:

421 + - selector: cache_hits # OK — declared in this group

422 + name: hits

423 + - selector: queries_total # OK — inherited from parent group

424 + name: queries

425 +```

426 +

427 +#### chart_defaults

428 +

429 +Inheritable chart configuration applied to all descendant charts in the group subtree. Useful when many charts share the same instance identity or label promotion policy.

430

431 | Field | Type | Description |

432 |-------------------|---------------|------------------------------------------|

57 -| `label_promotion` | array[string] | Default chart label promotion policy |

58 -| `instances` | object | Default chart instance identity policy |

59 -

60 -Inheritance rules:

61 -

62 -- nearest group default wins

63 -- chart-local field overrides inherited default

64 -- list/object fields replace the inherited field wholesale

65 -- no deep merge or append semantics

66 -

67 -### `charts[]`

68 -

70 -|-------------------|---------------|----------|----------------------------------------------------------------------------------|

71 -| `id` | string | no | Base chart ID template (if omitted, derived from `context`) |

78 -| `priority` | int | no | Chart priority |

83 -

84 -### `instances.by_labels`

85 -

86 -Instance identity determines how series are grouped into chart instances.

87 -When multiple series share the same instance identity labels, they appear as dimensions on the same chart.

88 -

89 -| Token | Meaning |

90 -|--------------|-------------------------------------------------|

91 -| `label_key` | Include explicit label key in instance identity |

92 -| `*` | Include all labels |

93 -| `!label_key` | Exclude label key |

94 -

95 -### `lifecycle`

96 -

98 -|----------------------------------|------|----------------|---------------------------------------------------------|

99 -| `max_instances` | int | `0` | Best-effort cap (`0` = disabled) |

101 -| `dimensions.max_dims` | int | `0` | Best-effort dimension cap (`0` = disabled) |

102 -| `dimensions.expire_after_cycles` | int | `0` | Expire dimensions not seen for N successful cycles |

103 -

104 -### `dimensions[]`

105 -

107 -|----------------------|--------|----------|----------------------------------------------------------------|

111 -| `options.multiplier` | int | no | DIM multiplier (`0` means default `1`) |

112 -| `options.divisor` | int | no | DIM divisor (`0` means default `1`) |

115 -

116 -**Selector syntax**: A selector takes the form `metric_name` or `metric_name{label=value, ...}`.

117 -The metric name prefix is required; label-only selectors like `{label=value}` are rejected.

118 -

119 -### `engine` (template-level policy)

120 -

121 -| Field | Type | Description |

122 -|---------------------------------------|---------------|---------------------------------------------------------------------------|

123 -| `selector.allow` | array[string] | Global include selectors |

124 -| `selector.deny` | array[string] | Global exclude selectors |

125 -| `autogen.enabled` | bool | Enable unmatched-series autogen fallback |

126 -| `autogen.max_type_id_len` | int | Max full `type.id` length (`0` = default; must be `0` or `>= 4` when set) |

127 -| `autogen.expire_after_success_cycles` | uint64 | Autogen lifecycle expiry |

128 -

129 -## Minimal Example

433 +| `label_promotion` | array[string] | Default labels to promote on all charts. |

434 +| `instances` | object | Default instance identity policy. |

435 +

436 +> [!NOTE]

437 +> **Inheritance rules**: nearest group default wins (child overrides parent), chart-local field overrides inherited default, and list/object fields replace the inherited field wholesale — there is no deep merge or append.

438 +

439 +**Example: Azure Monitor — all charts share the same instance identity**

440 +

441 +```yaml

442 +groups:

443 + - family: Azure Key Vault

444 + context_namespace: key_vault

445 + chart_defaults:

446 + label_promotion: [resource_name, resource_group, region]

447 + instances:

448 + by_labels: [resource_uid]

449 + charts:

450 + # Every chart below inherits instances and label_promotion

451 + # without repeating them.

452 + - id: availability

453 + title: Azure Key Vault Availability

454 + context: availability

455 + units: percentage

456 + dimensions:

457 + - selector: key_vault.availability_average

458 + name: average

459 + - id: api_latency

460 + title: Azure Key Vault API Latency

461 + context: api_latency

462 + units: milliseconds

463 + dimensions:

464 + - selector: key_vault.service_api_latency_average

465 + name: average

466 +```

467 +

468 +Without `chart_defaults`, you would need to repeat `instances` and `label_promotion` on every chart.

469 +

470 +### 5. charts

471 +

472 +A chart defines a single visualization in the Netdata dashboard.

473 +

474 +```yaml

475 +charts:

476 + - id: <chart ID>

477 + title: <chart title>

478 + family: <optional family leaf>

479 + context: <chart context>

480 + units: <units string>

481 + algorithm: <absolute|incremental>

482 + type: <line|area|stacked|heatmap>

483 + priority: <int>

484 + label_promotion: [<label>, ...]

485 + instances:

486 + by_labels: [<label>, ...]

487 + lifecycle:

488 + max_instances: <int>

489 + expire_after_cycles: <int>

490 + dimensions:

491 + max_dims: <int>

492 + expire_after_cycles: <int>

493 + dimensions:

494 + - <dimension definition>

495 +```

496 +

498 +|-------------------|---------------|----------|------------------------|------------------------------------------------------------------------------|

499 +| `id` | string | no | derived from `context` | Base chart ID. If omitted, derived by replacing `.` with `_` in `context`. |

506 +| `priority` | int | no | `70000` | Chart ordering priority in the dashboard (`0` = use engine default `70000`). |

511 +

512 +> [!TIP]

513 +> When `algorithm` is omitted, the engine infers it from metric name suffixes. You only need to set it explicitly when the suffix doesn't match the intended behavior (e.g., a gauge metric named `*_total`).

514 +

515 +| Suffix | Inferred algorithm |

516 +|-------------------------------------------|--------------------|

517 +| `*_total`, `*_count`, `*_sum`, `*_bucket` | `incremental` |

518 +| Everything else | `absolute` |

519 +

520 +> [!WARNING]

521 +> If a chart's dimensions mix counter-like metrics (e.g., `requests_total`) with gauge-like metrics (e.g., `temperature`) and `algorithm` is omitted, the engine fails with a compile error: _"algorithm inference is ambiguous for mixed metric kinds; set algorithm explicitly"_. Set `algorithm` on the chart to resolve this.

522 +

523 +**Example: MySQL queries — incremental counters displayed as rates**

524 +

525 +```yaml

526 +charts:

527 + - id: queries

528 + title: Queries

529 + context: queries

530 + units: queries/s

531 + algorithm: incremental

532 + dimensions:

533 + - selector: queries

534 + name: queries

535 + - selector: questions

536 + name: questions

537 + - selector: slow_queries

538 + name: slow_queries

539 +```

540 +

541 +**Example: MySQL bandwidth — bidirectional area chart with unit conversion**

542 +

543 +```yaml

544 +charts:

545 + - id: net

546 + title: Bandwidth

547 + context: net

548 + units: kilobits/s

549 + type: area

550 + algorithm: incremental

551 + dimensions:

552 + - selector: bytes_received

553 + name: in

554 + options:

555 + multiplier: 8

556 + divisor: 1000

557 + - selector: bytes_sent

558 + name: out

559 + options:

560 + multiplier: -8 # negative = below zero line

561 + divisor: 1000

562 +```

563 +

564 +#### instances

565 +

566 +Instance identity determines how series are grouped into chart instances. When multiple series share the same instance identity label values, they appear as dimensions on the same chart instance.

567 +

568 +> [!TIP]

569 +> Without `instances`, there is one chart instance (all matching series land on the same chart). With `instances`, the engine creates one chart instance per unique combination of the specified label values.

570 +

571 +```yaml

572 +instances:

573 + by_labels: [host]

574 +```

575 +

576 +| Token | Meaning |

577 +|--------------|---------------------------------------------------------------|

578 +| `label_key` | Include this label in instance identity. |

579 +| `*` | Include all labels. |

580 +| `!label_key` | Exclude this label (use with `*` to include all _except_...). |

581 +

582 +**Example: One chart per host**

583 +

584 +```yaml

585 +instances:

586 + by_labels: [host]

587 +```

588 +

589 +If the collector reports metrics for hosts `server-1`, `server-2`, `server-3`, the engine creates 3 separate chart instances — each showing only that host's dimensions.

590 +

591 +**Example: One chart per unique (job, instance) combination**

592 +

593 +```yaml

594 +instances:

595 + by_labels: [nagios_job, perfdata_value]

596 +```

597 +

598 +**Example: All labels except one**

599 +

600 +```yaml

601 +instances:

602 + by_labels: ["*", "!_collect_job"]

603 +```

604 +

605 +#### lifecycle

606 +

607 +Controls cardinality limits and expiry for chart instances and dimensions.

608 +

610 +|----------------------------------|------|----------------|--------------------------------------------------------------------------------------|

612 +| `expire_after_cycles` | int | `5` | Remove chart instances not seen for N successful collection cycles. |

615 +

616 +**How lifecycle caps work**:

617 +

618 +- Caps are **best-effort** — instances/dimensions actively seen in the current cycle are never evicted.

619 +- Oldest inactive entries are evicted first (by last-seen time).

620 +- Expiry counters only advance on **successful** collection cycles.

621 +

622 +### 6. dimensions

623 +

624 +A dimension binds a metric from the collector's metric store to a line on the chart.

625 +

626 +```yaml

627 +dimensions:

628 + - selector: <metric selector>

629 + name: <static name>

630 + name_from_label: <label key>

631 + options:

632 + multiplier: <int>

633 + divisor: <int>

634 + hidden: <bool>

635 + float: <bool>

636 +```

637 +

639 +|----------------------|--------|----------|---------|------------------------------------------------------------------|

643 +| `options.multiplier` | int | no | `1` | Multiply the raw value by this factor. |

644 +| `options.divisor` | int | no | `1` | Divide the raw value by this factor. |

647 +

648 +> [!IMPORTANT]

649 +> There are three ways to name a dimension — pick **exactly one**:

650 +> - `name` — static name you choose (e.g., `name: read`).

651 +> - `name_from_label` — dynamic name from a label value (e.g., `name_from_label: method` → dimensions "GET", "POST", ...).

652 +> - **Omit both** — the engine infers the name automatically for histogram buckets (`le`), summary quantiles (`quantile`), and statesets.

653 +>

654 +> `name` and `name_from_label` are mutually exclusive. Duplicate static `name` values within the same chart are rejected.

655 +

656 +#### selectors

657 +

658 +A selector specifies which metric(s) a dimension should match.

659 +

660 +**Syntax:**

661 +

662 +```

663 +metric_name

664 +metric_name{label_key=label_value, ...}

665 +```

666 +

667 +- The metric name prefix is **required** — label-only selectors like `{label=value}` are rejected.

668 +- The metric must be declared in the current group's `metrics` list (or inherited from an ancestor group).

669 +- Label filters narrow which series match. Without labels, all series of that metric match.

670 +

671 +**Examples:**

672 +

673 +```yaml

674 +# Match all series of the "queries" metric

675 +- selector: queries

676 +

677 +# Match only series where method="GET"

678 +- selector: http_requests_total{method="GET"}

679 +

680 +# Match a specific histogram bucket

681 +- selector: request_duration_seconds_bucket{le="0.5"}

682 +```

683 +

684 +#### Common dimension patterns

685 +

686 +**Unit conversion** — convert bytes to kilobits per second:

687 +

688 +```yaml

689 +dimensions:

690 + - selector: bytes_received

691 + name: in

692 + options:

693 + multiplier: 8

694 + divisor: 1000

695 +```

696 +

697 +**Bidirectional charts** — use a negative multiplier to display below zero:

698 +

699 +```yaml

700 +dimensions:

701 + - selector: bytes_received

702 + name: in

703 + options:

704 + multiplier: 8

705 + divisor: 1000

706 + - selector: bytes_sent

707 + name: out

708 + options:

709 + multiplier: -8

710 + divisor: 1000

711 +```

712 +

713 +**Float precision** — for ratios or small decimal values:

714 +

715 +```yaml

716 +dimensions:

717 + - selector: efficiency_ratio

718 + name: efficiency

719 + options:

720 + float: true

721 +```

722 +

723 +**Dynamic naming from labels** — each unique label value becomes a separate dimension:

724 +

725 +```yaml

726 +dimensions:

727 + - selector: http_requests_total

728 + name_from_label: method

729 +```

730 +

731 +If the metric has series with `method="GET"`, `method="POST"`, etc., each becomes its own dimension on the chart.

732 +

733 +## Examples

734 +

735 +### Simple: static metrics, no instances

736 +

737 +A collector that monitors a single MySQL server. Each chart has a fixed set of dimensions.

738

739 ```yaml

740 version: v1

133 -context_namespace: netdata.go.plugin.example

741 +context_namespace: mysql

742 groups:

135 - - family: HTTP

743 + - family: Queries

744 + groups:

745 + - family: Statistics

746 + metrics:

747 + - queries

748 + - questions

749 + - slow_queries

750 + - com_delete

751 + - com_insert

752 + - com_select

753 + - com_update

754 + charts:

755 + - id: queries

756 + title: Queries

757 + context: queries

758 + units: queries/s

759 + algorithm: incremental

760 + dimensions:

761 + - selector: queries

762 + name: queries

763 + - selector: questions

764 + name: questions

765 + - selector: slow_queries

766 + name: slow_queries

767 + - id: queries_type

768 + title: Queries By Type

769 + context: queries_type

770 + units: queries/s

771 + type: stacked

772 + algorithm: incremental

773 + dimensions:

774 + - selector: com_delete

775 + name: delete

776 + - selector: com_insert

777 + name: insert

778 + - selector: com_select

779 + name: select

780 + - selector: com_update

781 + name: update

782 +```

783 +

784 +### Per-instance: one chart per host

785 +

786 +A ping collector that monitors multiple hosts. Each host gets its own set of charts.

787 +

788 +```yaml

789 +version: v1

790 +context_namespace: ping

791 +groups:

792 + - family: latency

793 metrics:

137 - - app.requests_total

138 - - app.latency_seconds_bucket

794 + - min_rtt

795 + - max_rtt

796 + - avg_rtt

797 charts:

140 - - id: requests

141 - title: Requests

142 - context: requests

798 + - id: host_rtt

799 + title: Ping round-trip time

800 + context: host_rtt

801 + units: milliseconds

802 + type: area

803 + instances:

804 + by_labels: [host]

805 + dimensions:

806 + - selector: min_rtt

807 + name: min

808 + options:

809 + divisor: 1000

810 + - selector: max_rtt

811 + name: max

812 + options:

813 + divisor: 1000

814 + - selector: avg_rtt

815 + name: avg

816 + options:

817 + divisor: 1000

818 +```

819 +

820 +### Per-instance with multiple labels

821 +

822 +MySQL replication monitoring creates one chart per replication connection.

823 +

824 +```yaml

825 +groups:

826 + - family: Replication

827 + groups:

828 + - family: Slave Status

829 + metrics:

830 + - seconds_behind_master

831 + - slave_io_running

832 + - slave_sql_running

833 + charts:

834 + - id: slave_behind

835 + title: Slave Behind Seconds

836 + context: slave_behind

837 + units: seconds

838 + instances:

839 + by_labels: [connection]

840 + dimensions:

841 + - selector: seconds_behind_master

842 + name: seconds

843 + - id: slave_thread_running

844 + title: I/O / SQL Thread Running State

845 + context: slave_status

846 + units: boolean

847 + instances:

848 + by_labels: [connection]

849 + dimensions:

850 + - selector: slave_io_running

851 + name: io_running

852 + - selector: slave_sql_running

853 + name: sql_running

854 +```

855 +

856 +### Deeply nested groups

857 +

858 +MySQL's InnoDB storage engine metrics organized in a deep hierarchy.

859 +

860 +```yaml

861 +groups:

862 + - family: Storage Engine

863 + groups:

864 + - family: InnoDB

865 + groups:

866 + - family: Buffer Pool

867 + metrics:

868 + - innodb_buffer_pool_pages_data

869 + - innodb_buffer_pool_pages_dirty

870 + - innodb_buffer_pool_pages_free

871 + - innodb_buffer_pool_pages_misc

872 + - innodb_buffer_pool_pages_total

873 + charts:

874 + - id: innodb_buffer_pool_pages

875 + title: InnoDB Buffer Pool Pages

876 + context: innodb_buffer_pool_pages

877 + units: pages

878 + dimensions:

879 + - selector: innodb_buffer_pool_pages_data

880 + name: data

881 + - selector: innodb_buffer_pool_pages_dirty

882 + name: dirty

883 + options:

884 + multiplier: -1

885 + - selector: innodb_buffer_pool_pages_free

886 + name: free

887 + - selector: innodb_buffer_pool_pages_misc

888 + name: misc

889 + options:

890 + multiplier: -1

891 + - selector: innodb_buffer_pool_pages_total

892 + name: total

893 + - family: I/O

894 + metrics:

895 + - innodb_data_read

896 + - innodb_data_written

897 + charts:

898 + - id: innodb_io

899 + title: InnoDB I/O Bandwidth

900 + context: innodb_io

901 + units: KiB/s

902 + type: area

903 + algorithm: incremental

904 + dimensions:

905 + - selector: innodb_data_read

906 + name: read

907 + options:

908 + divisor: 1024

909 + - selector: innodb_data_written

910 + name: write

911 + options:

912 + divisor: 1024

913 +```

914 +

915 +The resulting chart families are `Storage Engine/InnoDB/Buffer Pool` and `Storage Engine/InnoDB/I/O`.

916 +

917 +### chart_defaults: reducing repetition

918 +

919 +When monitoring a cloud resource that has many charts, all sharing the same instance identity.

920 +

921 +```yaml

922 +groups:

923 + - family: Azure PostgreSQL

924 + context_namespace: postgres_flexible

925 + chart_defaults:

926 + label_promotion: [resource_name, resource_group, region]

927 + instances:

928 + by_labels: [resource_uid]

929 + charts:

930 + - title: CPU Percent

931 + context: cpu_percent

932 + units: percentage

933 + dimensions:

934 + - selector: postgres_flexible.cpu_percent_average

935 + name: average

936 + - title: Memory Percent

937 + context: memory_percent

938 + units: percentage

939 + dimensions:

940 + - selector: postgres_flexible.memory_percent_average

941 + name: average

942 + - title: Storage Percent

943 + context: storage_percent

944 + units: percentage

945 + dimensions:

946 + - selector: postgres_flexible.storage_percent_average

947 + name: average

948 +```

949 +

950 +All three charts inherit `instances` and `label_promotion` from `chart_defaults` — no repetition needed.

951 +

952 +### Autogeneration: handling unpredictable metrics

953 +

954 +For collectors where the metric set is user-defined or discovered at runtime, use autogen to catch metrics that don't match any explicit chart template.

955 +

956 +```yaml

957 +version: v1

958 +context_namespace: prometheus_scraper

959 +engine:

960 + autogen:

961 + enabled: true

962 + expire_after_success_cycles: 30

963 + selector:

964 + deny: ["go_*", "promhttp_*"] # exclude internal Go/Prometheus metrics

965 +groups:

966 + - family: Application

967 + metrics:

968 + - app_http_requests_total

969 + - app_http_response_time_seconds

970 + charts:

971 + - id: app_requests

972 + title: Application HTTP Requests

973 + context: http_requests

974 units: requests/s

975 + algorithm: incremental

976 + instances:

977 + by_labels: [instance]

978 dimensions:

145 - - selector: app.requests_total

146 - name: requests

147 - - id: latency

148 - title: Request latency

149 - context: latency

979 + - selector: app_http_requests_total

980 + name_from_label: status_code

981 + - id: app_response_time

982 + title: Application Response Time

983 + context: response_time

984 units: seconds

985 + instances:

986 + by_labels: [instance]

987 dimensions:

152 - - selector: app.latency_seconds_bucket{le="0.5"}

153 - name: le_0_5

988 + - selector: app_http_response_time_seconds

989 + name: p99

990 + options:

991 + float: true

992 ```

993

994 +The explicitly defined charts handle `app_http_requests_total` and `app_http_response_time_seconds`. Any _other_ application metrics the scraper discovers are automatically charted by the engine, and removed after 30 cycles of inactivity. The `selector.deny` filter excludes noisy internal metrics from autogeneration.

995 +

996 ## Validation Rules

997

998 +> [!CAUTION]

999 +> Unknown YAML fields cause an immediate decode error (strict unmarshal). Double-check field names for typos — a misspelled field like `demensions` will be caught at parse time with an unmarshal error, not at runtime with a descriptive message pointing to the affected chart.

1000 +

1001 All rules below produce semantic validation errors unless noted:

1002

160 -- `version` must be `v1`

161 -- `groups[]` must be non-empty

162 -- `group.family` must not be empty or whitespace-only

163 -- `group.metrics[]` entries must not be empty or whitespace-only; no duplicates within same group

164 -- `group.chart_defaults.instances.by_labels` follows the same validation rules as `chart.instances.by_labels`

165 -- `chart.title`, `chart.context`, `chart.units` must be non-empty

166 -- `dimension.selector` must include explicit metric name (prefix before `{`)

167 -- Selector metric must be visible in current group metric scope

168 -- `name` and `name_from_label` are mutually exclusive

169 -- `name` and `name_from_label` must not be whitespace-only

170 -- Duplicate dimension `name` values within the same chart are rejected

171 -- `instances.by_labels` must contain at least one token when `instances` is set

172 -- `instances.by_labels` exclude token must include label key (e.g., `!key`, not bare `!`)

173 -- `instances.by_labels` tokens must not be duplicated

174 -- Lifecycle numeric fields must be `>= 0`

175 -- `engine.autogen.max_type_id_len` must be `0` or `>= 4`

176 -- Unknown YAML field — decode error (strict unmarshal)

177 -

178 -## Current Phase-1 Constraints

179 -

180 -- **Placeholders in `chart.id` and `dimension.name`** — Parsed by template parser but intentionally rejected in phase-1 syntax.

181 -- **Runtime inferred dimension naming** — Allowed only for inferable selectors (histogram bucket / summary quantile / stateset-like sources) when both `name` and `name_from_label` are omitted.

182 -- **Selector parse stage** — Full selector parsing happens in the chartengine compile stage, not here.

1003 +| Rule | Error type |

1004 +|-----------------------------------------------------------------------------------------|---------------------------------|

1005 +| `version` must be `v1` | semantic |

1006 +| `groups[]` must be non-empty | semantic |

1007 +| `group.family` must not be empty or whitespace-only | semantic |

1008 +| `group.metrics[]` entries must not be empty; no duplicates within same group | semantic |

1009 +| `chart.title`, `chart.context`, `chart.units` must be non-empty | semantic |

1010 +| `chart.algorithm` must be `absolute` or `incremental` (when specified) | semantic |

1011 +| `chart.type` must be `line`, `area`, `stacked`, or `heatmap` (when specified) | semantic |

1012 +| `dimension.selector` must include explicit metric name (prefix before `{`) | semantic |

1013 +| Selector metric must be visible in current group metric scope | semantic |

1014 +| `name` and `name_from_label` are mutually exclusive | semantic |

1015 +| `name` and `name_from_label` must not be whitespace-only | semantic |

1016 +| Duplicate dimension `name` values within the same chart are rejected | semantic |

1017 +| `instances.by_labels` must contain at least one token when `instances` is set | semantic |

1018 +| `instances.by_labels` exclude token must include label key (e.g., `!key`, not bare `!`) | semantic |

1019 +| `instances.by_labels` tokens must not be duplicated | semantic |

1020 +| Lifecycle numeric fields must be `>= 0` | semantic |

1021 +| `engine.autogen.max_type_id_len` must be `0` or `>= 4` | semantic |

1022 +| Unknown YAML fields | decode error (strict unmarshal) |

1023

1024 ## Compiler-Derived Behavior

1025

186 -The following behaviors are applied by `chartengine` during compilation, not by `charttpl`.

187 -They affect how you write templates:

1026 +> [!NOTE]

1027 +> These behaviors are applied by `chartengine` during compilation, not by the template parser. You don't need to configure them — they happen automatically, but knowing about them helps you write simpler templates.

1028

189 -| Input | Derived behavior |

190 -|-----------------------------------------|-------------------------------------------------------------------------------------------|

191 -| Missing `chart.id` | `id` derived from `context` (`.` replaced with `_`) |

192 -| Missing `chart.algorithm` | Inferred from metric suffixes (`*_total`, `*_count`, `*_sum`, `*_bucket` => counter-like) |

193 -| Group family hierarchy + `chart.family` | Composed into slash-separated chart family |

1029 +| Input | Derived behavior |

1030 +|-----------------------------------------|------------------------------------------------------------------------------------------|

1031 +| Missing `chart.id` | `id` derived from `context` (`.` replaced with `_`). |

1032 +| Missing `chart.algorithm` | Inferred from metric suffixes (`*_total`, `*_count`, `*_sum`, `*_bucket` = incremental). |

1033 +| `chart.priority = 0` | Treated as `70000` (engine default). |

1034 +| Group family hierarchy + `chart.family` | Composed into `/`-separated chart family. |

1035 +| `options.multiplier = 0` | Treated as `1`. |

1036 +| `options.divisor = 0` | Treated as `1`. |

docs: user-oriented charttpl README rewrite (#22049)