@cryptotaxi247 / netdata-1 / commits / c71dcfc60

chore(go.d): add topology overlay helpers for vSphere (#22625)

Ilya Mashchenko committed Jun 4, 2026 at 05:28 UTC c71dcfc6037ad9901173c77beecd5bc188ff8c71
13 files changed +1228 -86
.agents/skills/project-create-topology/SKILL.md
+22
@@ -102,6 +102,23 @@ developer-facing and must stay in this project skill, not under
102 - Use overlay templates once per payload or type.
103 - Links and actors carry compact refs and parameters only.
104 - Do not put full metric query payloads on every row.
105 + - Build Go producer refs with `topologyv1.NewActorOverlayRefsBuilder` or
106 + `topologyv1.NewLinkOverlayRefsBuilder` instead of hand-assembling compact
107 + tables.
108 + - Overlay refs use a `template` column, exactly one convention owner column
109 + (`actor` with type `actor_ref` or `link` with type `link_ref`), and one
110 + column for each selector param required by the referenced template. Do not
111 + add any other `actor_ref` or `link_ref` columns to overlay refs. Every row
112 + must have a non-null owner value.
113 + - The `template` column and required selector-param columns must be `string`
114 + or `string_ref`; required selector-param row values must resolve to
115 + non-empty strings.
116 + - Do not use `template`, `actor`, or `link` as selector params; those names
117 + are reserved refs-table convention columns.
118 + - For `netdata.metrics`, `node_id` means node scope, `collect_job` maps to
119 + chart label `_collect_job`, and other selector params map to same-named
120 + chart labels. In go.d producers, pass `job.Name()` for `collect_job`; do
121 + not use `job.FullName()`.
122
123 7. Define correlation semantics when actors can be resolved across payloads.
124 - Declare whether the topology needs loose-side resolution, actor
@@ -400,6 +417,11 @@ For SNMP/L2 managed device actor modals:
417 high-cardinality evidence rows.
418 - Raw JSON columns are hidden/debug-only unless a schema-declared projection
419 renders a scalar value.
420 +- Overlay templates and refs validate global template references, required
421 + selector-param columns, selector-param string types and values, reserved
422 + selector-param names, exactly one convention actor/link owner column, no extra
423 + actor/link ref owner columns, non-null owner row values, provider/merge enum
424 + membership, and link-type `overlay_templates` references.
425 - Payload size is measured on realistic or captured data.
426 - Raw sensitive captures remain under `.local/`.
427
.agents/sow/specs/topology-function-schema.md
+38 -2
@@ -426,8 +426,44 @@ graph relationships.
426 Refreshable traffic, state, error, packet, or utilization data is represented by
427 overlay templates and per-actor/per-link refs.
428
429 -Templates define the query mechanism once. Refs provide only template ids and
430 -parameters. Aggregated links merge refs according to the template merge policy.
429 +Templates define the query mechanism once. Refs provide only template ids, one
430 +owner reference, and selector parameters. The refs table uses schema ids for
431 +column names, so template ids, selector params, and refs column ids must match
432 +the topology `$defs/id` contract: start with a letter and then use only letters,
433 +digits, `_`, `.`, `:`, or `-`.
434 +
435 +The refs-table convention is:
436 +
437 +- `template`: string or string_ref template name resolving to
438 + `data.types.overlay_templates`;
439 +- exactly one convention owner column: `actor` with type `actor_ref` or `link`
440 + with type `link_ref`; every row must have a non-null owner value;
441 +- one column for each selector param required by the referenced template.
442 +
443 +Selector params must not use reserved refs-table convention column names:
444 +`template`, `actor`, or `link`.
445 +
446 +No other `actor_ref` or `link_ref` columns are valid in overlay refs. Consumers
447 +can identify ownership from the fixed `actor` / `link` column ids instead of
448 +scanning all columns by type.
449 +
450 +The `template` column and all selector-param columns required by a row's
451 +resolved template must be `string` or `string_ref`. Required selector-param row
452 +values must resolve to non-empty strings. Selector-param columns used by other
453 +templates may be nullable and null on rows whose template does not require them.
454 +Future selectors that need `ip_ref`, `mac_ref`, or empty-string matching must
455 +relax this contract explicitly.
456 +
457 +For `provider: "netdata.metrics"`, selector params are interpreted as:
458 +
459 +- `node_id`: node-scope selector, not a chart label;
460 +- `collect_job`: chart label `_collect_job`;
461 +- other params: same-named chart labels.
462 +
463 +Aggregated actors or links merge overlay refs according to the template merge
464 +policy. `merge.refs` controls ref-list handling, currently `append` or `set`.
465 +`merge.values` controls how multiple matching metric values collapse, currently
466 +`sum`, `min`, `max`, `avg`, `last`, or `none`.
467
468 ## Compatibility
469
src/go/pkg/topology/v1/overlay.go new
+134
@@ -0,0 +1,134 @@
1 +// SPDX-License-Identifier: GPL-3.0-or-later
2 +
3 +package topologyv1
4 +
5 +import "fmt"
6 +
7 +const (
8 + OverlayProviderNetdataMetrics = "netdata.metrics"
9 + OverlayProviderNetdataFunction = "netdata.function"
10 + OverlayProviderExternal = "external"
11 +
12 + OverlayMergeRefsAppend = "append"
13 + OverlayMergeRefsSet = "set"
14 +
15 + OverlayMergeValuesSum = "sum"
16 + OverlayMergeValuesMin = "min"
17 + OverlayMergeValuesMax = "max"
18 + OverlayMergeValuesAvg = "avg"
19 + OverlayMergeValuesLast = "last"
20 + OverlayMergeValuesNone = "none"
21 +
22 + OverlayRefsTemplateColumn = "template"
23 + OverlayRefsActorColumn = "actor"
24 + OverlayRefsLinkColumn = "link"
25 +)
26 +
27 +type OverlayTemplateOption func(*OverlayTemplate)
28 +
29 +func NewOverlayTemplate(provider string, merge OverlayMerge, opts ...OverlayTemplateOption) OverlayTemplate {
30 + template := OverlayTemplate{
31 + Provider: provider,
32 + Merge: merge,
33 + }
34 + for _, opt := range opts {
35 + opt(&template)
36 + }
37 + return template
38 +}
39 +
40 +func WithOverlayContexts(contexts ...string) OverlayTemplateOption {
41 + return func(template *OverlayTemplate) {
42 + template.Contexts = append([]string(nil), contexts...)
43 + }
44 +}
45 +
46 +func WithOverlayDimensions(dimensions ...string) OverlayTemplateOption {
47 + return func(template *OverlayTemplate) {
48 + template.Dimensions = append([]string(nil), dimensions...)
49 + }
50 +}
51 +
52 +func WithOverlaySelectorParams(params ...string) OverlayTemplateOption {
53 + return func(template *OverlayTemplate) {
54 + template.SelectorParams = append([]string(nil), params...)
55 + }
56 +}
57 +
58 +func NewOverlayMerge(refs, values string) OverlayMerge {
59 + return OverlayMerge{
60 + Refs: refs,
61 + Values: values,
62 + }
63 +}
64 +
65 +type OverlayRefsBuilder struct {
66 + strings *StringDictionary
67 + selectorNames []string
68 + builder *TableBuilder
69 + err error
70 +}
71 +
72 +func NewActorOverlayRefsBuilder(strings *StringDictionary, selectorNames ...string) *OverlayRefsBuilder {
73 + return newOverlayRefsBuilder(strings, NewColumn(OverlayRefsActorColumn, "actor_ref", WithRole("reference")), selectorNames...)
74 +}
75 +
76 +func NewLinkOverlayRefsBuilder(strings *StringDictionary, selectorNames ...string) *OverlayRefsBuilder {
77 + return newOverlayRefsBuilder(strings, NewColumn(OverlayRefsLinkColumn, "link_ref", WithRole("reference")), selectorNames...)
78 +}
79 +
80 +func newOverlayRefsBuilder(strings *StringDictionary, ownerColumn Column, selectorNames ...string) *OverlayRefsBuilder {
81 + columns := []Column{
82 + NewColumn(OverlayRefsTemplateColumn, "string_ref", WithDictionary("strings")),
83 + ownerColumn,
84 + }
85 + for _, name := range selectorNames {
86 + columns = append(columns, NewColumn(name, "string_ref", WithDictionary("strings")))
87 + }
88 +
89 + return &OverlayRefsBuilder{
90 + strings: strings,
91 + selectorNames: append([]string(nil), selectorNames...),
92 + builder: NewTableBuilder(columns...),
93 + }
94 +}
95 +
96 +func (b *OverlayRefsBuilder) Add(template string, owner int, selectorValues ...string) int {
97 + row := b.builder.Rows()
98 + if b.err != nil {
99 + return row
100 + }
101 + if b.strings == nil {
102 + b.err = fmt.Errorf("overlay refs builder requires string dictionary")
103 + return row
104 + }
105 + if len(selectorValues) != len(b.selectorNames) {
106 + b.err = fmt.Errorf("overlay ref has %d selector values for %d selector params", len(selectorValues), len(b.selectorNames))
107 + return row
108 + }
109 +
110 + values := make([]any, 0, len(selectorValues)+2)
111 + values = append(values, b.strings.Ref(template), owner)
112 + for _, value := range selectorValues {
113 + values = append(values, b.strings.Ref(value))
114 + }
115 + return b.builder.Add(values...)
116 +}
117 +
118 +func (b *OverlayRefsBuilder) Rows() int {
119 + return b.builder.Rows()
120 +}
121 +
122 +func (b *OverlayRefsBuilder) OverlayRefs() (*OverlayRefs, error) {
123 + if b.err != nil {
124 + return nil, b.err
125 + }
126 + if b.Rows() == 0 {
127 + return nil, nil
128 + }
129 + table, err := b.builder.Table()
130 + if err != nil {
131 + return nil, err
132 + }
133 + return &OverlayRefs{Refs: &table}, nil
134 +}
src/go/pkg/topology/v1/response_test.go
+468 -3
@@ -1413,6 +1413,436 @@ func TestValidateDecodedResponseAllowsUnusedCorrelationRuleColumns(t *testing.T)
1413 require.NoError(t, err)
1414 }
1415
1416 +func TestValidateDecodedResponseRejectsInvalidOverlaySemantics(t *testing.T) {
1417 + cases := map[string]struct {
1418 + mutate func(*Data)
1419 + want string
1420 + }{
1421 + "unknown refs template": {
1422 + mutate: func(data *Data) {
1423 + data.Types.OverlayTemplates = testOverlayTemplates()
1424 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1425 + []Column{
1426 + NewColumn(OverlayRefsTemplateColumn, "string"),
1427 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1428 + NewColumn("collect_job", "string"),
1429 + NewColumn("id", "string"),
1430 + },
1431 + []ColumnEncoding{
1432 + Const("missing"),
1433 + Const(0),
1434 + Const("job"),
1435 + Const("node-a"),
1436 + },
1437 + )}
1438 + },
1439 + want: "references unknown overlay template",
1440 + },
1441 + "missing selector param column": {
1442 + mutate: func(data *Data) {
1443 + data.Types.OverlayTemplates = testOverlayTemplates()
1444 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1445 + []Column{
1446 + NewColumn(OverlayRefsTemplateColumn, "string"),
1447 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1448 + NewColumn("id", "string"),
1449 + },
1450 + []ColumnEncoding{
1451 + Const("node_cpu"),
1452 + Const(0),
1453 + Const("node-a"),
1454 + },
1455 + )}
1456 + },
1457 + want: "missing selector param column \"collect_job\"",
1458 + },
1459 + "reserved selector param column": {
1460 + mutate: func(data *Data) {
1461 + data.Types.OverlayTemplates = testOverlayTemplates()
1462 + template := data.Types.OverlayTemplates["node_cpu"]
1463 + template.SelectorParams = []string{OverlayRefsTemplateColumn}
1464 + data.Types.OverlayTemplates["node_cpu"] = template
1465 + },
1466 + want: "uses reserved overlay refs column",
1467 + },
1468 + "missing template column": {
1469 + mutate: func(data *Data) {
1470 + data.Types.OverlayTemplates = testOverlayTemplates()
1471 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1472 + []Column{
1473 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1474 + NewColumn("collect_job", "string"),
1475 + NewColumn("id", "string"),
1476 + },
1477 + []ColumnEncoding{
1478 + Const(0),
1479 + Const("job"),
1480 + Const("node-a"),
1481 + },
1482 + )}
1483 + },
1484 + want: "missing required template column",
1485 + },
1486 + "template column non-string type": {
1487 + mutate: func(data *Data) {
1488 + data.Types.OverlayTemplates = testOverlayTemplates()
1489 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1490 + []Column{
1491 + NewColumn(OverlayRefsTemplateColumn, "uint"),
1492 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1493 + NewColumn("collect_job", "string"),
1494 + NewColumn("id", "string"),
1495 + },
1496 + []ColumnEncoding{
1497 + Const(0),
1498 + Const(0),
1499 + Const("job"),
1500 + Const("node-a"),
1501 + },
1502 + )}
1503 + },
1504 + want: "data.overlays.refs.template column must be string or string_ref",
1505 + },
1506 + "missing owner column": {
1507 + mutate: func(data *Data) {
1508 + data.Types.OverlayTemplates = testOverlayTemplates()
1509 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1510 + []Column{
1511 + NewColumn(OverlayRefsTemplateColumn, "string"),
1512 + NewColumn("collect_job", "string"),
1513 + NewColumn("id", "string"),
1514 + },
1515 + []ColumnEncoding{
1516 + Const("node_cpu"),
1517 + Const("job"),
1518 + Const("node-a"),
1519 + },
1520 + )}
1521 + },
1522 + want: "must contain exactly one owner column: actor actor_ref or link link_ref",
1523 + },
1524 + "non-convention owner column": {
1525 + mutate: func(data *Data) {
1526 + data.Types.OverlayTemplates = testOverlayTemplates()
1527 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1528 + []Column{
1529 + NewColumn(OverlayRefsTemplateColumn, "string"),
1530 + NewColumn("owner", "actor_ref"),
1531 + NewColumn("collect_job", "string"),
1532 + NewColumn("id", "string"),
1533 + },
1534 + []ColumnEncoding{
1535 + Const("node_cpu"),
1536 + Const(0),
1537 + Const("job"),
1538 + Const("node-a"),
1539 + },
1540 + )}
1541 + },
1542 + want: "data.overlays.refs.owner uses non-convention actor_ref owner column",
1543 + },
1544 + "actor owner column wrong type": {
1545 + mutate: func(data *Data) {
1546 + data.Types.OverlayTemplates = testOverlayTemplates()
1547 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1548 + []Column{
1549 + NewColumn(OverlayRefsTemplateColumn, "string"),
1550 + NewColumn(OverlayRefsActorColumn, "link_ref"),
1551 + NewColumn("collect_job", "string"),
1552 + NewColumn("id", "string"),
1553 + },
1554 + []ColumnEncoding{
1555 + Const("node_cpu"),
1556 + Const(0),
1557 + Const("job"),
1558 + Const("node-a"),
1559 + },
1560 + )}
1561 + },
1562 + want: "data.overlays.refs.actor column must be actor_ref",
1563 + },
1564 + "link owner column wrong type": {
1565 + mutate: func(data *Data) {
1566 + data.Types.OverlayTemplates = testOverlayTemplates()
1567 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1568 + []Column{
1569 + NewColumn(OverlayRefsTemplateColumn, "string"),
1570 + NewColumn(OverlayRefsLinkColumn, "actor_ref"),
1571 + NewColumn("collect_job", "string"),
1572 + NewColumn("id", "string"),
1573 + },
1574 + []ColumnEncoding{
1575 + Const("node_cpu"),
1576 + Const(0),
1577 + Const("job"),
1578 + Const("node-a"),
1579 + },
1580 + )}
1581 + },
1582 + want: "data.overlays.refs.link column must be link_ref",
1583 + },
1584 + "multiple owner columns": {
1585 + mutate: func(data *Data) {
1586 + data.Types.OverlayTemplates = testOverlayTemplates()
1587 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1588 + []Column{
1589 + NewColumn(OverlayRefsTemplateColumn, "string"),
1590 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1591 + NewColumn(OverlayRefsLinkColumn, "link_ref"),
1592 + NewColumn("collect_job", "string"),
1593 + NewColumn("id", "string"),
1594 + },
1595 + []ColumnEncoding{
1596 + Const("node_cpu"),
1597 + Const(0),
1598 + Const(0),
1599 + Const("job"),
1600 + Const("node-a"),
1601 + },
1602 + )}
1603 + },
1604 + want: "must contain exactly one owner column: actor actor_ref or link link_ref",
1605 + },
1606 + "null actor owner value": {
1607 + mutate: func(data *Data) {
1608 + data.Types.OverlayTemplates = testOverlayTemplates()
1609 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1610 + []Column{
1611 + NewColumn(OverlayRefsTemplateColumn, "string"),
1612 + NewColumn(OverlayRefsActorColumn, "actor_ref", WithNullable()),
1613 + NewColumn("collect_job", "string"),
1614 + NewColumn("id", "string"),
1615 + },
1616 + []ColumnEncoding{
1617 + Const("node_cpu"),
1618 + Const(nil),
1619 + Const("job"),
1620 + Const("node-a"),
1621 + },
1622 + )}
1623 + },
1624 + want: "data.overlays.refs.actor[0] is not a non-null owner reference",
1625 + },
1626 + "null link owner value": {
1627 + mutate: func(data *Data) {
1628 + data.Types.OverlayTemplates = testOverlayTemplates()
1629 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1630 + []Column{
1631 + NewColumn(OverlayRefsTemplateColumn, "string"),
1632 + NewColumn(OverlayRefsLinkColumn, "link_ref", WithNullable()),
1633 + NewColumn("collect_job", "string"),
1634 + NewColumn("id", "string"),
1635 + },
1636 + []ColumnEncoding{
1637 + Const("node_cpu"),
1638 + Const(nil),
1639 + Const("job"),
1640 + Const("node-a"),
1641 + },
1642 + )}
1643 + },
1644 + want: "data.overlays.refs.link[0] is not a non-null owner reference",
1645 + },
1646 + "selector column non-string type": {
1647 + mutate: func(data *Data) {
1648 + data.Types.OverlayTemplates = testOverlayTemplates()
1649 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1650 + []Column{
1651 + NewColumn(OverlayRefsTemplateColumn, "string"),
1652 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1653 + NewColumn("collect_job", "string"),
1654 + NewColumn("id", "uint"),
1655 + },
1656 + []ColumnEncoding{
1657 + Const("node_cpu"),
1658 + Const(0),
1659 + Const("job"),
1660 + Const(7),
1661 + },
1662 + )}
1663 + },
1664 + want: "data.overlays.refs.id column must be string or string_ref",
1665 + },
1666 + "empty selector value": {
1667 + mutate: func(data *Data) {
1668 + data.Types.OverlayTemplates = testOverlayTemplates()
1669 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1670 + []Column{
1671 + NewColumn(OverlayRefsTemplateColumn, "string"),
1672 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1673 + NewColumn("collect_job", "string"),
1674 + NewColumn("id", "string"),
1675 + },
1676 + []ColumnEncoding{
1677 + Const("node_cpu"),
1678 + Const(0),
1679 + Const("job"),
1680 + Const(""),
1681 + },
1682 + )}
1683 + },
1684 + want: "data.overlays.refs.id[0] is not a non-empty string",
1685 + },
1686 + "null selector value": {
1687 + mutate: func(data *Data) {
1688 + data.Types.OverlayTemplates = testOverlayTemplates()
1689 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1690 + []Column{
1691 + NewColumn(OverlayRefsTemplateColumn, "string"),
1692 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1693 + NewColumn("collect_job", "string"),
1694 + NewColumn("id", "string", WithNullable()),
1695 + },
1696 + []ColumnEncoding{
1697 + Const("node_cpu"),
1698 + Const(0),
1699 + Const("job"),
1700 + Const(nil),
1701 + },
1702 + )}
1703 + },
1704 + want: "data.overlays.refs.id[0] is not a non-empty string",
1705 + },
1706 + "link type references unknown template": {
1707 + mutate: func(data *Data) {
1708 + data.Types.LinkTypes["dependency"] = LinkType{
1709 + Orientation: "directed",
1710 + DirectionRole: "dependency",
1711 + Aggregation: LinkAggregation{Direction: "preserve"},
1712 + OverlayTemplates: []string{"missing"},
1713 + }
1714 + },
1715 + want: "references unknown overlay template",
1716 + },
1717 + "invalid provider": {
1718 + mutate: func(data *Data) {
1719 + data.Types.OverlayTemplates = testOverlayTemplates()
1720 + template := data.Types.OverlayTemplates["node_cpu"]
1721 + template.Provider = "unknown"
1722 + data.Types.OverlayTemplates["node_cpu"] = template
1723 + },
1724 + want: "provider has unsupported value",
1725 + },
1726 + "invalid merge refs": {
1727 + mutate: func(data *Data) {
1728 + data.Types.OverlayTemplates = testOverlayTemplates()
1729 + template := data.Types.OverlayTemplates["node_cpu"]
1730 + template.Merge.Refs = "replace"
1731 + data.Types.OverlayTemplates["node_cpu"] = template
1732 + },
1733 + want: "merge.refs has unsupported value",
1734 + },
1735 + "invalid merge values": {
1736 + mutate: func(data *Data) {
1737 + data.Types.OverlayTemplates = testOverlayTemplates()
1738 + template := data.Types.OverlayTemplates["node_cpu"]
1739 + template.Merge.Values = "median"
1740 + data.Types.OverlayTemplates["node_cpu"] = template
1741 + },
1742 + want: "merge.values has unsupported value",
1743 + },
1744 + }
1745 +
1746 + for name, tc := range cases {
1747 + t.Run(name, func(t *testing.T) {
1748 + data := minimalValidationData(nil)
1749 + tc.mutate(&data)
1750 +
1751 + err := validateResponseData(t, data)
1752 +
1753 + require.Error(t, err)
1754 + assert.Contains(t, err.Error(), tc.want)
1755 + })
1756 + }
1757 +}
1758 +
1759 +func TestValidateDecodedResponseAcceptsValidOverlaySemantics(t *testing.T) {
1760 + data := minimalValidationData(nil)
1761 + data.Types.OverlayTemplates = testOverlayTemplates()
1762 + data.Types.LinkTypes["dependency"] = LinkType{
1763 + Orientation: "directed",
1764 + DirectionRole: "dependency",
1765 + Aggregation: LinkAggregation{Direction: "preserve"},
1766 + OverlayTemplates: []string{"node_cpu"},
1767 + }
1768 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1769 + []Column{
1770 + NewColumn(OverlayRefsTemplateColumn, "string"),
1771 + NewColumn(OverlayRefsLinkColumn, "link_ref"),
1772 + NewColumn("collect_job", "string"),
1773 + NewColumn("id", "string"),
1774 + },
1775 + []ColumnEncoding{
1776 + Const("node_cpu"),
1777 + Const(0),
1778 + Const("job"),
1779 + Const("node-a"),
1780 + },
1781 + )}
1782 +
1783 + err := validateResponseData(t, data)
1784 +
1785 + require.NoError(t, err)
1786 +}
1787 +
1788 +func TestValidateDecodedResponseAcceptsZeroSelectorOverlaySemantics(t *testing.T) {
1789 + data := minimalValidationData(nil)
1790 + data.Types.OverlayTemplates = map[string]OverlayTemplate{
1791 + "node_state": NewOverlayTemplate(
1792 + OverlayProviderNetdataMetrics,
1793 + NewOverlayMerge(OverlayMergeRefsSet, OverlayMergeValuesLast),
1794 + ),
1795 + }
1796 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTable(
1797 + []Column{
1798 + NewColumn(OverlayRefsTemplateColumn, "string"),
1799 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1800 + },
1801 + []ColumnEncoding{
1802 + Const("node_state"),
1803 + Const(0),
1804 + },
1805 + )}
1806 +
1807 + err := validateResponseData(t, data)
1808 +
1809 + require.NoError(t, err)
1810 +}
1811 +
1812 +func TestValidateDecodedResponseAcceptsMultiTemplateOverlaySelectorScoping(t *testing.T) {
1813 + data := minimalValidationData(nil)
1814 + data.Types.OverlayTemplates = map[string]OverlayTemplate{
1815 + "node_cpu": NewOverlayTemplate(
1816 + OverlayProviderNetdataMetrics,
1817 + NewOverlayMerge(OverlayMergeRefsSet, OverlayMergeValuesLast),
1818 + WithOverlaySelectorParams("cpu_id"),
1819 + ),
1820 + "node_mem": NewOverlayTemplate(
1821 + OverlayProviderNetdataMetrics,
1822 + NewOverlayMerge(OverlayMergeRefsSet, OverlayMergeValuesLast),
1823 + WithOverlaySelectorParams("mem_id"),
1824 + ),
1825 + }
1826 + data.Overlays = &OverlayRefs{Refs: testOverlayRefsTableRows(2,
1827 + []Column{
1828 + NewColumn(OverlayRefsTemplateColumn, "string"),
1829 + NewColumn(OverlayRefsActorColumn, "actor_ref"),
1830 + NewColumn("cpu_id", "string", WithNullable()),
1831 + NewColumn("mem_id", "string", WithNullable()),
1832 + },
1833 + []ColumnEncoding{
1834 + Values("node_cpu", "node_mem"),
1835 + Const(0),
1836 + Values("cpu0", nil),
1837 + Values(nil, "mem0"),
1838 + },
1839 + )}
1840 +
1841 + err := validateResponseData(t, data)
1842 +
1843 + require.NoError(t, err)
1844 +}
1845 +
1846 func TestPresentationTokenEnumsMatchSchema(t *testing.T) {
1847 schemaDoc := loadTopologySchema(t)
1848
@@ -1424,6 +1854,9 @@ func TestPresentationTokenEnumsMatchSchema(t *testing.T) {
1854 assert.ElementsMatch(t, actorSizeScaleTokens, schemaEnum(t, schemaDoc, "actor_size_scale_token"))
1855 assert.ElementsMatch(t, linkSemanticRoleTokens, schemaEnum(t, schemaDoc, "link_semantic_role"))
1856 assert.ElementsMatch(t, iconTokens, schemaEnum(t, schemaDoc, "icon_token"))
1857 + assert.ElementsMatch(t, overlayProviderTokens, schemaEnumAtPath(t, schemaDoc, "$defs", "overlay_template", "properties", "provider"))
1858 + assert.ElementsMatch(t, overlayMergeRefsTokens, schemaEnumAtPath(t, schemaDoc, "$defs", "overlay_merge", "properties", "refs"))
1859 + assert.ElementsMatch(t, overlayMergeValuesTokens, schemaEnumAtPath(t, schemaDoc, "$defs", "overlay_merge", "properties", "values"))
1860 }
1861
1862 func TestValidateDecodedResponseRejectsInvalidActorReference(t *testing.T) {
@@ -1565,6 +1998,27 @@ func dependencyLinkTableWith(rows int, extraColumns []Column, extraValues []Colu
1998 return MustTable(rows, columns, values)
1999 }
2000
2001 +func testOverlayTemplates() map[string]OverlayTemplate {
2002 + return map[string]OverlayTemplate{
2003 + "node_cpu": NewOverlayTemplate(
2004 + OverlayProviderNetdataMetrics,
2005 + NewOverlayMerge(OverlayMergeRefsSet, OverlayMergeValuesLast),
2006 + WithOverlayContexts("system.cpu"),
2007 + WithOverlayDimensions("user"),
2008 + WithOverlaySelectorParams("collect_job", "id"),
2009 + ),
2010 + }
2011 +}
2012 +
2013 +func testOverlayRefsTable(columns []Column, values []ColumnEncoding) *Table {
2014 + return testOverlayRefsTableRows(1, columns, values)
2015 +}
2016 +
2017 +func testOverlayRefsTableRows(rows int, columns []Column, values []ColumnEncoding) *Table {
2018 + table := MustTable(rows, columns, values)
2019 + return &table
2020 +}
2021 +
2022 func withActors(actors Table) func(*Data) {
2023 return func(data *Data) {
2024 data.Actors = actors
@@ -1625,9 +2079,20 @@ func loadTopologySchema(t *testing.T) map[string]any {
2079 func schemaEnum(t *testing.T, schemaDoc map[string]any, defName string) []string {
2080 t.Helper()
2081
1628 - defs, ok := schemaDoc["$defs"].(map[string]any)
1629 - require.True(t, ok)
1630 - definition, ok := defs[defName].(map[string]any)
2082 + return schemaEnumAtPath(t, schemaDoc, "$defs", defName)
2083 +}
2084 +
2085 +func schemaEnumAtPath(t *testing.T, schemaDoc map[string]any, path ...string) []string {
2086 + t.Helper()
2087 +
2088 + var current any = schemaDoc
2089 + for _, element := range path {
2090 + obj, ok := current.(map[string]any)
2091 + require.True(t, ok)
2092 + current, ok = obj[element]
2093 + require.True(t, ok)
2094 + }
2095 + definition, ok := current.(map[string]any)
2096 require.True(t, ok)
2097 rawEnum, ok := definition["enum"].([]any)
2098 require.True(t, ok)
src/go/pkg/topology/v1/validate.go
+302 -7
@@ -28,9 +28,14 @@ type topologyShape struct {
28 tableTypeOwners map[string]string
29 actorTables map[string]map[string]string
30 relationshipTables map[string]map[string]string
31 + overlayTemplates map[string]overlayTemplateShape
32 scaleKeys map[string]struct{}
33 }
34
35 +type overlayTemplateShape struct {
36 + selectorParams []string
37 +}
38 +
39 // ValidateDecodedResponse validates a full topology v1 Function response.
40 // Metadata-only Function info responses intentionally omit data and must not be
41 // passed here.
@@ -98,6 +103,9 @@ func ValidateDecodedData(data map[string]any) error {
103 if err != nil {
104 return err
105 }
106 + if err := validateOverlaySemantics(data, shape, ctx); err != nil {
107 + return err
108 + }
109 if err := validateTypeColumns(data, shape); err != nil {
110 return err
111 }
@@ -272,6 +280,193 @@ func validateOverlayRefs(raw any, ctx validationContext) error {
280 return nil
281 }
282
283 +func validateOverlaySemantics(data map[string]any, shape topologyShape, ctx validationContext) error {
284 + types, _ := data["types"].(map[string]any)
285 + if err := validateLinkTypeOverlayTemplates(types["link_types"], shape.overlayTemplates); err != nil {
286 + return err
287 + }
288 + return validateOverlayRefRows(data["overlays"], shape.overlayTemplates, ctx.dictionaries)
289 +}
290 +
291 +func validateLinkTypeOverlayTemplates(raw any, templates map[string]overlayTemplateShape) error {
292 + linkTypes, ok := raw.(map[string]any)
293 + if !ok {
294 + return fmt.Errorf("data.types.link_types is not an object")
295 + }
296 + known := make(map[string]struct{}, len(templates))
297 + for id := range templates {
298 + known[id] = struct{}{}
299 + }
300 + for typeID, rawType := range linkTypes {
301 + linkType, ok := rawType.(map[string]any)
302 + if !ok {
303 + return fmt.Errorf("data.types.link_types.%s is not an object", typeID)
304 + }
305 + if err := validateOptionalIDArrayRefs("data.types.link_types."+typeID+".overlay_templates", linkType["overlay_templates"], known, "overlay template"); err != nil {
306 + return err
307 + }
308 + }
309 + return nil
310 +}
311 +
312 +func validateOverlayRefRows(raw any, templates map[string]overlayTemplateShape, dictionaries map[string]any) error {
313 + if raw == nil {
314 + return nil
315 + }
316 + overlays, ok := raw.(map[string]any)
317 + if !ok {
318 + return fmt.Errorf("data.overlays is not an object")
319 + }
320 + refs, ok := overlays["refs"]
321 + if !ok {
322 + return nil
323 + }
324 +
325 + rows, columns, err := decodedColumnsFromCompactTable("data.overlays.refs", refs)
326 + if err != nil {
327 + return err
328 + }
329 + if rows == 0 {
330 + return nil
331 + }
332 +
333 + templateColumn, ok := columns[OverlayRefsTemplateColumn]
334 + if !ok {
335 + return fmt.Errorf("data.overlays.refs is missing required %s column", OverlayRefsTemplateColumn)
336 + }
337 + if !isOverlayRefsStringColumn(templateColumn) {
338 + return fmt.Errorf("data.overlays.refs.%s column must be string or string_ref", OverlayRefsTemplateColumn)
339 + }
340 + ownerColumn, err := validateOverlayOwnerColumns(columns)
341 + if err != nil {
342 + return err
343 + }
344 +
345 + for row := range rows {
346 + if ownerColumn.values[row] == nil {
347 + return fmt.Errorf("data.overlays.refs.%s[%d] is not a non-null owner reference", ownerColumn.id, row)
348 + }
349 + templateID, ok := resolveStringValue(templateColumn.values[row], templateColumn.columnType, templateColumn.dictionary, dictionaries)
350 + if !ok || templateID == "" {
351 + return fmt.Errorf("data.overlays.refs.template[%d] is not a non-empty string", row)
352 + }
353 + template, ok := templates[templateID]
354 + if !ok {
355 + return fmt.Errorf("data.overlays.refs.template[%d] references unknown overlay template %q", row, templateID)
356 + }
357 + for _, param := range template.selectorParams {
358 + column, ok := columns[param]
359 + if !ok {
360 + return fmt.Errorf("data.overlays.refs row %d template %q is missing selector param column %q", row, templateID, param)
361 + }
362 + if !isOverlayRefsStringColumn(column) {
363 + return fmt.Errorf("data.overlays.refs.%s column must be string or string_ref", param)
364 + }
365 + value, ok := resolveStringValue(column.values[row], column.columnType, column.dictionary, dictionaries)
366 + if !ok || value == "" {
367 + return fmt.Errorf("data.overlays.refs.%s[%d] is not a non-empty string", param, row)
368 + }
369 + }
370 + }
371 +
372 + return nil
373 +}
374 +
375 +func validateOverlayOwnerColumns(columns map[string]decodedCompactColumn) (decodedCompactColumn, error) {
376 + owners := 0
377 + ownerColumn := decodedCompactColumn{}
378 + if column, ok := columns[OverlayRefsActorColumn]; ok {
379 + if column.columnType != "actor_ref" {
380 + return decodedCompactColumn{}, fmt.Errorf("data.overlays.refs.%s column must be actor_ref", OverlayRefsActorColumn)
381 + }
382 + ownerColumn = column
383 + owners++
384 + }
385 + if column, ok := columns[OverlayRefsLinkColumn]; ok {
386 + if column.columnType != "link_ref" {
387 + return decodedCompactColumn{}, fmt.Errorf("data.overlays.refs.%s column must be link_ref", OverlayRefsLinkColumn)
388 + }
389 + ownerColumn = column
390 + owners++
391 + }
392 + for id, column := range columns {
393 + if id == OverlayRefsActorColumn || id == OverlayRefsLinkColumn {
394 + continue
395 + }
396 + if column.columnType == "actor_ref" || column.columnType == "link_ref" {
397 + return decodedCompactColumn{}, fmt.Errorf("data.overlays.refs.%s uses non-convention %s owner column", id, column.columnType)
398 + }
399 + }
400 + if owners != 1 {
401 + return decodedCompactColumn{}, fmt.Errorf("data.overlays.refs must contain exactly one owner column: actor actor_ref or link link_ref")
402 + }
403 + return ownerColumn, nil
404 +}
405 +
406 +func isOverlayRefsStringColumn(column decodedCompactColumn) bool {
407 + return column.columnType == "string" || column.columnType == "string_ref"
408 +}
409 +
410 +type decodedCompactColumn struct {
411 + id string
412 + columnType string
413 + dictionary string
414 + values []any
415 +}
416 +
417 +func decodedColumnsFromCompactTable(path string, raw any) (int, map[string]decodedCompactColumn, error) {
418 + table, ok := raw.(map[string]any)
419 + if !ok {
420 + return 0, nil, fmt.Errorf("%s is not an object", path)
421 + }
422 + rows, err := decodedTableRows(table)
423 + if err != nil {
424 + return 0, nil, fmt.Errorf("%s: %w", path, err)
425 + }
426 + rawColumns, ok := table["columns"].([]any)
427 + if !ok {
428 + return 0, nil, fmt.Errorf("%s.columns is not an array", path)
429 + }
430 + rawValues, ok := table["values"].([]any)
431 + if !ok {
432 + return 0, nil, fmt.Errorf("%s.values is not an array", path)
433 + }
434 + if len(rawColumns) != len(rawValues) {
435 + return 0, nil, fmt.Errorf("%s columns/values length mismatch: %d columns, %d values", path, len(rawColumns), len(rawValues))
436 + }
437 +
438 + columns := make(map[string]decodedCompactColumn, len(rawColumns))
439 + for i, rawColumn := range rawColumns {
440 + column, ok := rawColumn.(map[string]any)
441 + if !ok {
442 + return 0, nil, fmt.Errorf("%s.columns[%d] is not an object", path, i)
443 + }
444 + columnID, _ := column["id"].(string)
445 + if columnID == "" {
446 + return 0, nil, fmt.Errorf("%s.columns[%d].id is required", path, i)
447 + }
448 + if _, ok := columns[columnID]; ok {
449 + return 0, nil, fmt.Errorf("%s.columns[%d].id duplicates column %q", path, i, columnID)
450 + }
451 + columnType, _ := column["type"].(string)
452 + if columnType == "" {
453 + return 0, nil, fmt.Errorf("%s.columns[%d].type is required", path, i)
454 + }
455 + values, err := decodeColumn(path, i, rows, rawValues[i])
456 + if err != nil {
457 + return 0, nil, err
458 + }
459 + dictionary, _ := column["dictionary"].(string)
460 + columns[columnID] = decodedCompactColumn{
461 + id: columnID,
462 + columnType: columnType,
463 + dictionary: dictionary,
464 + values: values,
465 + }
466 + }
467 + return rows, columns, nil
468 +}
469 +
470 func validateCompactTable(path string, raw any, ctx validationContext) (int, error) {
471 table, ok := raw.(map[string]any)
472 if !ok {
@@ -501,6 +696,10 @@ func collectTopologyShape(data map[string]any) (topologyShape, error) {
696 if err != nil {
697 return topologyShape{}, err
698 }
699 + overlayTemplates, err := collectOverlayTemplates(types["overlay_templates"])
700 + if err != nil {
701 + return topologyShape{}, err
702 + }
703
704 return topologyShape{
705 actorColumns: actorColumns,
@@ -513,6 +712,7 @@ func collectTopologyShape(data map[string]any) (topologyShape, error) {
712 tableTypeOwners: tableTypeOwners,
713 actorTables: actorTables,
714 relationshipTables: relationshipTables,
715 + overlayTemplates: overlayTemplates,
716 scaleKeys: scaleKeys,
717 }, nil
718 }
@@ -2162,6 +2362,96 @@ func collectScaleKeys(raw any) (map[string]struct{}, error) {
2362 return keys, nil
2363 }
2364
2365 +func collectOverlayTemplates(raw any) (map[string]overlayTemplateShape, error) {
2366 + templates := make(map[string]overlayTemplateShape)
2367 + if raw == nil {
2368 + return templates, nil
2369 + }
2370 + rawTemplates, ok := raw.(map[string]any)
2371 + if !ok {
2372 + return nil, fmt.Errorf("data.types.overlay_templates is not an object")
2373 + }
2374 + for templateID, rawTemplate := range rawTemplates {
2375 + template, ok := rawTemplate.(map[string]any)
2376 + if !ok {
2377 + return nil, fmt.Errorf("data.types.overlay_templates.%s is not an object", templateID)
2378 + }
2379 + if _, err := requiredEnum("data.types.overlay_templates."+templateID+".provider", template["provider"], overlayProviderTokens...); err != nil {
2380 + return nil, err
2381 + }
2382 + if err := validateStringList("data.types.overlay_templates."+templateID+".contexts", template["contexts"], true); err != nil {
2383 + return nil, err
2384 + }
2385 + if err := validateStringList("data.types.overlay_templates."+templateID+".dimensions", template["dimensions"], true); err != nil {
2386 + return nil, err
2387 + }
2388 + selectorParams, err := collectStringList("data.types.overlay_templates."+templateID+".selector_params", template["selector_params"], true)
2389 + if err != nil {
2390 + return nil, err
2391 + }
2392 + for i, param := range selectorParams {
2393 + if isOverlayRefsConventionColumn(param) {
2394 + return nil, fmt.Errorf("data.types.overlay_templates.%s.selector_params[%d] uses reserved overlay refs column %q", templateID, i, param)
2395 + }
2396 + }
2397 + merge, ok := template["merge"].(map[string]any)
2398 + if !ok {
2399 + return nil, fmt.Errorf("data.types.overlay_templates.%s.merge is not an object", templateID)
2400 + }
2401 + if _, err := requiredEnum("data.types.overlay_templates."+templateID+".merge.refs", merge["refs"], overlayMergeRefsTokens...); err != nil {
2402 + return nil, err
2403 + }
2404 + if _, err := requiredEnum("data.types.overlay_templates."+templateID+".merge.values", merge["values"], overlayMergeValuesTokens...); err != nil {
2405 + return nil, err
2406 + }
2407 + templates[templateID] = overlayTemplateShape{
2408 + selectorParams: selectorParams,
2409 + }
2410 + }
2411 + return templates, nil
2412 +}
2413 +
2414 +func isOverlayRefsConventionColumn(column string) bool {
2415 + switch column {
2416 + case OverlayRefsTemplateColumn, OverlayRefsActorColumn, OverlayRefsLinkColumn:
2417 + return true
2418 + default:
2419 + return false
2420 + }
2421 +}
2422 +
2423 +func validateStringList(path string, raw any, optional bool) error {
2424 + _, err := collectStringList(path, raw, optional)
2425 + return err
2426 +}
2427 +
2428 +func collectStringList(path string, raw any, optional bool) ([]string, error) {
2429 + if raw == nil {
2430 + if optional {
2431 + return nil, nil
2432 + }
2433 + return nil, fmt.Errorf("%s is not an array", path)
2434 + }
2435 + rawValues, ok := raw.([]any)
2436 + if !ok {
2437 + return nil, fmt.Errorf("%s is not an array", path)
2438 + }
2439 + values := make([]string, 0, len(rawValues))
2440 + seen := make(map[string]struct{}, len(rawValues))
2441 + for i, rawValue := range rawValues {
2442 + value, ok := rawValue.(string)
2443 + if !ok || value == "" {
2444 + return nil, fmt.Errorf("%s[%d] is not a non-empty string", path, i)
2445 + }
2446 + if _, ok := seen[value]; ok {
2447 + return nil, fmt.Errorf("%s[%d] duplicates value %q", path, i, value)
2448 + }
2449 + seen[value] = struct{}{}
2450 + values = append(values, value)
2451 + }
2452 + return values, nil
2453 +}
2454 +
2455 func requiredEnum(path string, raw any, allowed ...string) (string, error) {
2456 value, ok := raw.(string)
2457 if !ok || value == "" {
@@ -2223,13 +2513,18 @@ var (
2513 "info", "structural", "warning", "success", "danger", "blue", "green", "orange",
2514 "purple", "cyan", "yellow", "teal", "gray",
2515 }
2226 - opacityTokens = []string{"normal", "muted", "faded"}
2227 - widthTokens = []string{"thin", "normal", "thick", "emphasis"}
2228 - layoutStrengthTokens = []string{"weakest", "weaker", "normal", "stronger", "strongest"}
2229 - layoutDistanceTokens = []string{"closest", "closer", "normal", "farther", "farthest"}
2230 - actorSizeScaleTokens = []string{"compact", "normal", "emphasized"}
2231 - linkSemanticRoleTokens = []string{"normal", "discovery", "ownership", "traffic", "correlation", "control"}
2232 - iconTokens = []string{
2516 + opacityTokens = []string{"normal", "muted", "faded"}
2517 + widthTokens = []string{"thin", "normal", "thick", "emphasis"}
2518 + layoutStrengthTokens = []string{"weakest", "weaker", "normal", "stronger", "strongest"}
2519 + layoutDistanceTokens = []string{"closest", "closer", "normal", "farther", "farthest"}
2520 + actorSizeScaleTokens = []string{"compact", "normal", "emphasized"}
2521 + linkSemanticRoleTokens = []string{"normal", "discovery", "ownership", "traffic", "correlation", "control"}
2522 + overlayProviderTokens = []string{OverlayProviderNetdataMetrics, OverlayProviderNetdataFunction, OverlayProviderExternal}
2523 + overlayMergeRefsTokens = []string{OverlayMergeRefsAppend, OverlayMergeRefsSet}
2524 + overlayMergeValuesTokens = []string{
2525 + OverlayMergeValuesSum, OverlayMergeValuesMin, OverlayMergeValuesMax, OverlayMergeValuesAvg, OverlayMergeValuesLast, OverlayMergeValuesNone,
2526 + }
2527 + iconTokens = []string{
2528 "router", "switch", "firewall", "access_point", "server", "storage", "load_balancer",
2529 "printer", "phone", "ups", "camera", "process", "agent", "netdata-agent", "parent",
2530 "remote-endpoint", "local-endpoint", "segment", "self", "ip", "cloud", "container",
src/go/plugin/go.d/collector/vsphere/func_readiness.go
+1 -1
@@ -125,7 +125,7 @@ func vsphereMethodHandler(job collectorapi.RuntimeJob) funcapi.MethodHandler {
125 }
126 return &funcVSphere{
127 readiness: &funcReadiness{collector: c},
128 - topology: &funcTopology{collector: c, agentID: job.FullName()},
128 + topology: &funcTopology{collector: c, agentID: job.FullName(), jobName: job.Name()},
129 }
130 }
131
src/go/plugin/go.d/collector/vsphere/func_topology.go
+10 -4
@@ -27,11 +27,16 @@ const (
27 vsphereOwnershipEvidenceType = "vsphere_ownership_evidence"
28 vsphereRunsOnEvidenceType = "vsphere_runs_on_evidence"
29 vsphereNetworkConnectionEvidence = "vsphere_network_connection_evidence"
30 +
31 + vsphereDatastoreUtilizationOverlay = "datastore_space_utilization"
32 + vsphereOverlaySelectorCollectJob = "collect_job"
33 + vsphereOverlaySelectorID = "id"
34 )
35
36 type funcTopology struct {
37 collector *Collector
38 agentID string
39 + jobName string
40 }
41
42 var _ funcapi.MethodHandler = (*funcTopology)(nil)
@@ -60,7 +65,7 @@ func (f *funcTopology) Handle(_ context.Context, method string, _ funcapi.Resolv
65 return funcapi.UnavailableResponse("collector is not initialized")
66 }
67
63 - data, ok := f.collector.topologyData(f.agentID)
68 + data, ok := f.collector.topologyData(f.agentID, f.jobName)
69 if !ok {
70 return funcapi.UnavailableResponse("topology data not available yet, please retry after discovery")
71 }
@@ -77,7 +82,7 @@ func (f *funcTopology) Cleanup(context.Context) {
82 // No per-invocation resources are allocated by the topology function.
83 }
84
80 -func (c *Collector) topologyData(agentID string) (topologyv1.Data, bool) {
85 +func (c *Collector) topologyData(agentID, jobName string) (topologyv1.Data, bool) {
86 c.collectionLock.RLock()
87 defer c.collectionLock.RUnlock()
88
@@ -85,7 +90,7 @@ func (c *Collector) topologyData(agentID string) (topologyv1.Data, bool) {
90 return topologyv1.Data{}, false
91 }
92
88 - builder := newVSphereTopologyBuilder()
93 + builder := newVSphereTopologyBuilder(jobName)
94
95 for _, dc := range sortedDatacenters(c.resources.DataCenters) {
96 builder.addActor("vsphere_datacenter", dc.ID, dc.Name, "", vsphereActorDetail{
@@ -175,7 +180,7 @@ func (c *Collector) topologyData(agentID string) (topologyv1.Data, bool) {
180 }
181 }
182 for _, datastore := range sortedDatastores(c.resources.Datastores) {
178 - builder.addActor("vsphere_datastore", datastore.ID, datastore.Name, datastore.Hier.DC.ID, vsphereActorDetail{
183 + actor := builder.addActor("vsphere_datastore", datastore.ID, datastore.Name, datastore.Hier.DC.ID, vsphereActorDetail{
184 objectType: "datastore",
185 datacenter: datastore.Hier.DC.Name,
186 overallStatus: datastore.OverallStatus,
@@ -190,6 +195,7 @@ func (c *Collector) topologyData(agentID string) (topologyv1.Data, bool) {
195 "datacenter": datastore.Hier.DC.Name,
196 "type": datastore.Type,
197 }))
198 + builder.addDatastoreUtilizationOverlay(actor, datastore.ID)
199 if c.resources.DataCenters.Get(datastore.Hier.DC.ID) != nil {
200 builder.addLink(datastore.Hier.DC.ID, datastore.ID, vsphereTopologyOwnershipLink, "contains")
201 }
src/go/plugin/go.d/collector/vsphere/func_topology_test.go
+37 -3
@@ -42,8 +42,10 @@ func TestFuncTopology_Handle(t *testing.T) {
42 validateVSphereTopologyV1Data(t, data)
43 require.Equal(t, 0, data.Actors.Rows)
44 require.Equal(t, 0, data.Links.Rows)
45 + require.Nil(t, data.Overlays)
46 require.EqualValues(t, 0, data.Stats["hosts"])
47 require.EqualValues(t, 0, data.Stats["vms"])
48 + require.EqualValues(t, 0, data.Stats["overlay_refs"])
49 },
50 },
51 "unknown method": {
@@ -55,7 +57,7 @@ func TestFuncTopology_Handle(t *testing.T) {
57
58 for name, tc := range tests {
59 t.Run(name, func(t *testing.T) {
58 - handler := &funcTopology{collector: tc.collector(), agentID: "vsphere_vcenter1"}
60 + handler := &funcTopology{collector: tc.collector(), agentID: "vsphere_vcenter1", jobName: "vsphere_vcenter1"}
61
62 resp := handler.Handle(context.Background(), tc.method, nil)
63
@@ -151,7 +153,7 @@ func newVSphereTopologyTestCollector() *Collector {
153
154 func TestFuncTopology_HandleWithInventoryCache(t *testing.T) {
155 collr := newVSphereTopologyTestCollector()
154 - handler := &funcTopology{collector: collr, agentID: "vsphere_vcenter1"}
156 + handler := &funcTopology{collector: collr, agentID: "vsphere_vcenter1", jobName: "vsphere_vcenter1"}
157
158 resp := handler.Handle(context.Background(), "topology:vsphere", nil)
159
@@ -167,12 +169,14 @@ func TestFuncTopology_HandleWithInventoryCache(t *testing.T) {
169 require.Equal(t, "virtualization", data.Types.ActorTypes["vsphere_vm"].Layer)
170 require.Equal(t, 8, data.Actors.Rows)
171 require.Equal(t, 9, data.Links.Rows)
172 + require.EqualValues(t, 1, data.Stats["overlay_refs"])
173
174 actors := topologyTableRows(t, data.Actors, data.Dictionaries)
175 requireTopologyRow(t, actors, "vsphere_moid", "datacenter-1")
176 requireTopologyRow(t, actors, "vsphere_moid", "domain-c1")
177 requireTopologyRow(t, actors, "vsphere_moid", "host-1")
178 vm := requireTopologyRow(t, actors, "vsphere_moid", "vm-1")
179 + datastore := requireTopologyRow(t, actors, "vsphere_moid", "datastore-1")
180 network := requireTopologyRow(t, actors, "vsphere_moid", "network-1")
181 require.Equal(t, "vsphere_vm", vm["type"])
182 require.Equal(t, "vm", vm["object_type"])
@@ -200,6 +204,21 @@ func TestFuncTopology_HandleWithInventoryCache(t *testing.T) {
204 require.Equal(t, "vm-1", runEvidence["source_moid"])
205 require.Equal(t, "host", runEvidence["target_type"])
206 require.Equal(t, "host-1", runEvidence["target_moid"])
207 +
208 + template := data.Types.OverlayTemplates[vsphereDatastoreUtilizationOverlay]
209 + require.Equal(t, topologyv1.OverlayProviderNetdataMetrics, template.Provider)
210 + require.Equal(t, []string{"vsphere.datastore_space_utilization"}, template.Contexts)
211 + require.Equal(t, []string{"used"}, template.Dimensions)
212 + require.Equal(t, []string{vsphereOverlaySelectorCollectJob, vsphereOverlaySelectorID}, template.SelectorParams)
213 + require.Equal(t, topologyv1.NewOverlayMerge(topologyv1.OverlayMergeRefsSet, topologyv1.OverlayMergeValuesLast), template.Merge)
214 +
215 + require.NotNil(t, data.Overlays)
216 + require.NotNil(t, data.Overlays.Refs)
217 + refs := topologyTableRows(t, *data.Overlays.Refs, data.Dictionaries)
218 + ref := requireTopologyRow(t, refs, vsphereOverlaySelectorID, "datastore-1")
219 + require.Equal(t, vsphereDatastoreUtilizationOverlay, ref[topologyv1.OverlayRefsTemplateColumn])
220 + require.Equal(t, topologyIntValue(t, datastore["_row"]), topologyIntValue(t, ref[topologyv1.OverlayRefsActorColumn]))
221 + require.Equal(t, "vsphere_vcenter1", ref[vsphereOverlaySelectorCollectJob])
222 }
223
224 func TestFuncTopology_DoesNotLinkToFilteredActors(t *testing.T) {
@@ -232,7 +251,7 @@ func TestFuncTopology_DoesNotLinkToFilteredActors(t *testing.T) {
251 },
252 }
253
235 - data, ok := collr.topologyData("agent")
254 + data, ok := collr.topologyData("agent", "job")
255
256 require.True(t, ok)
257 validateVSphereTopologyV1Data(t, data)
@@ -245,6 +264,21 @@ func TestFuncTopology_DoesNotLinkToFilteredActors(t *testing.T) {
264 require.NotContains(t, keys, "vm:vm-1->host:host-filtered:vsphere_runs_on")
265 }
266
267 +func TestFuncTopology_SanitizesOverlayCollectJobLikeChartLabels(t *testing.T) {
268 + collr := newVSphereTopologyTestCollector()
269 +
270 + data, ok := collr.topologyData("vsphere_vcenter1", "job'\nname\x00")
271 +
272 + require.True(t, ok)
273 + validateVSphereTopologyV1Data(t, data)
274 + require.Equal(t, "vsphere_vcenter1", data.Producer.Instance)
275 + require.NotNil(t, data.Overlays)
276 + require.NotNil(t, data.Overlays.Refs)
277 + refs := topologyTableRows(t, *data.Overlays.Refs, data.Dictionaries)
278 + ref := requireTopologyRow(t, refs, vsphereOverlaySelectorID, "datastore-1")
279 + require.Equal(t, "job name", ref[vsphereOverlaySelectorCollectJob])
280 +}
281 +
282 func validateVSphereTopologyV1Data(t *testing.T, data topologyv1.Data) {
283 t.Helper()
284
src/go/plugin/go.d/collector/vsphere/func_topology_v1_builder.go
+48 -5
@@ -58,6 +58,7 @@ type vsphereActorDetail struct {
58
59 type vsphereTopologyBuilder struct {
60 strings *topologyv1.StringDictionary
61 + collectJob string
62 actorIndexes map[string]int
63 actorObjectTypes map[string]string
64 actors *topologyv1.TableBuilder
@@ -65,11 +66,14 @@ type vsphereTopologyBuilder struct {
66 evidence map[string]*topologyv1.TableBuilder
67 details *topologyv1.TableBuilder
68 labels *topologyv1.TableBuilder
69 + overlays *topologyv1.OverlayRefsBuilder
70 }
71
70 -func newVSphereTopologyBuilder() *vsphereTopologyBuilder {
72 +func newVSphereTopologyBuilder(jobName string) *vsphereTopologyBuilder {
73 + strings := topologyv1.NewStringDictionary()
74 return &vsphereTopologyBuilder{
72 - strings: topologyv1.NewStringDictionary(),
75 + strings: strings,
76 + collectJob: sanitizeTopologyLabelValue(cleanTopologyString(jobName)),
77 actorIndexes: make(map[string]int),
78 actorObjectTypes: make(map[string]string),
79 actors: topologyv1.NewTableBuilder(vsphereTopologyActorColumns()...),
@@ -79,8 +83,9 @@ func newVSphereTopologyBuilder() *vsphereTopologyBuilder {
83 vsphereRunsOnEvidenceType: topologyv1.NewTableBuilder(vsphereTopologyEvidenceColumns()...),
84 vsphereNetworkConnectionEvidence: topologyv1.NewTableBuilder(vsphereTopologyEvidenceColumns()...),
85 },
82 - details: topologyv1.NewTableBuilder(vsphereTopologyDetailColumns()...),
83 - labels: topologyv1.NewTableBuilder(vsphereTopologyLabelColumns()...),
86 + details: topologyv1.NewTableBuilder(vsphereTopologyDetailColumns()...),
87 + labels: topologyv1.NewTableBuilder(vsphereTopologyLabelColumns()...),
88 + overlays: topologyv1.NewActorOverlayRefsBuilder(strings, vsphereOverlaySelectorCollectJob, vsphereOverlaySelectorID),
89 }
90 }
91
@@ -194,6 +199,10 @@ func (b *vsphereTopologyBuilder) data(agentID string, collectedAt time.Time, res
199 if err != nil {
200 return topologyv1.Data{}, fmt.Errorf("build vSphere topology labels table: %w", err)
201 }
202 + overlays, err := b.overlays.OverlayRefs()
203 + if err != nil {
204 + return topologyv1.Data{}, fmt.Errorf("build vSphere topology overlay refs: %w", err)
205 + }
206
207 evidence, evidenceRows, err := b.evidenceMap()
208 if err != nil {
@@ -213,6 +222,10 @@ func (b *vsphereTopologyBuilder) data(agentID string, collectedAt time.Time, res
222 stats["evidence_rows"] = evidenceRows
223 stats["detail_rows"] = detailTable.Rows
224 stats["label_rows"] = labelTable.Rows
225 + stats["overlay_refs"] = 0
226 + if overlays != nil && overlays.Refs != nil {
227 + stats["overlay_refs"] = overlays.Refs.Rows
228 + }
229
230 return topologyv1.Data{
231 SchemaVersion: topologyv1.SchemaVersion,
@@ -250,7 +263,8 @@ func (b *vsphereTopologyBuilder) data(agentID string, collectedAt time.Time, res
263 },
264 },
265 },
253 - Stats: stats,
266 + Stats: stats,
267 + Overlays: overlays,
268 }, nil
269 }
270
@@ -356,6 +370,14 @@ func (b *vsphereTopologyBuilder) addActorLabel(actor int, key, value, source, ki
370 )
371 }
372
373 +func (b *vsphereTopologyBuilder) addDatastoreUtilizationOverlay(actor int, moid string) {
374 + moid = cleanTopologyString(moid)
375 + if actor < 0 || b.collectJob == "" || moid == "" {
376 + return
377 + }
378 + b.overlays.Add(vsphereDatastoreUtilizationOverlay, actor, b.collectJob, moid)
379 +}
380 +
381 func (b *vsphereTopologyBuilder) stringRef(value string) int {
382 return b.strings.Ref(cleanTopologyString(value))
383 }
@@ -507,6 +529,15 @@ func vsphereTopologyTypes() topologyv1.TypeRegistry {
529 vsphereTopologyDetailTable: vsphereDetailTableType(),
530 vsphereTopologyLabelsTable: vsphereLabelsTableType(),
531 },
532 + OverlayTemplates: map[string]topologyv1.OverlayTemplate{
533 + vsphereDatastoreUtilizationOverlay: topologyv1.NewOverlayTemplate(
534 + topologyv1.OverlayProviderNetdataMetrics,
535 + topologyv1.NewOverlayMerge(topologyv1.OverlayMergeRefsSet, topologyv1.OverlayMergeValuesLast),
536 + topologyv1.WithOverlayContexts("vsphere.datastore_space_utilization"),
537 + topologyv1.WithOverlayDimensions("used"),
538 + topologyv1.WithOverlaySelectorParams(vsphereOverlaySelectorCollectJob, vsphereOverlaySelectorID),
539 + ),
540 + },
541 }
542 }
543
@@ -646,6 +677,18 @@ func cleanTopologyString(value string) string {
677 return strings.TrimSpace(value)
678 }
679
680 +// Keep in sync with chartemit.wireValueReplacer so overlay refs match chart labels.
681 +var topologyLabelValueReplacer = strings.NewReplacer(
682 + "'", "",
683 + "\n", " ",
684 + "\r", " ",
685 + "\x00", "",
686 +)
687 +
688 +func sanitizeTopologyLabelValue(value string) string {
689 + return topologyLabelValueReplacer.Replace(value)
690 +}
691 +
692 func nullableBool(value any) any {
693 switch v := value.(type) {
694 case nil:
src/go/plugin/go.d/collector/vsphere/integrations/vmware_vcenter_server.md
+3 -2
@@ -881,7 +881,7 @@ No additional configuration is required.
881
882 #### Returns
883
884 -Cached vSphere inventory topology payload using the netdata.topology.v1 schema. Actors represent discovered inventory objects and links represent inventory ownership, VM-to-host runtime placement, and host/VM network attachment relationships.
884 +Cached vSphere inventory topology payload using the netdata.topology.v1 schema. Actors represent discovered inventory objects, links represent inventory ownership, VM-to-host runtime placement, and host/VM network attachment relationships, and overlays expose refreshable datastore utilization metrics.
885
886 | Column | Type | Unit | Visibility | Description |
887 |:-------|:-----|:-----|:-----------|:------------|
@@ -896,7 +896,8 @@ Cached vSphere inventory topology payload using the netdata.topology.v1 schema.
896 | links | object | | | Compact link table for relationships between vSphere inventory actors. |
897 | evidence | object | | | Relationship evidence tables backing the rendered links. |
898 | tables | object | | | Actor detail and label tables used by topology modals. |
899 -| stats | object | | | Counts of discovered inventory objects, actors, and links included in the response. |
899 +| overlays | object | | | Telemetry overlay refs for refreshable metrics, including datastore used-space utilization selected by collector job and vSphere managed object ID. |
900 +| stats | object | | | Counts of discovered inventory objects, actors, links, and overlay refs included in the response. |
901
902
903
src/go/plugin/go.d/collector/vsphere/metadata.yaml
+6 -2
@@ -503,7 +503,7 @@ modules:
503 default: ""
504 options: []
505 returns:
506 - description: Cached vSphere inventory topology payload using the netdata.topology.v1 schema. Actors represent discovered inventory objects and links represent inventory ownership, VM-to-host runtime placement, and host/VM network attachment relationships.
506 + description: Cached vSphere inventory topology payload using the netdata.topology.v1 schema. Actors represent discovered inventory objects, links represent inventory ownership, VM-to-host runtime placement, and host/VM network attachment relationships, and overlays expose refreshable datastore utilization metrics.
507 columns:
508 - name: schema_version
509 type: string
@@ -549,10 +549,14 @@ modules:
549 type: object
550 unit: ""
551 description: Actor detail and label tables used by topology modals.
552 + - name: overlays
553 + type: object
554 + unit: ""
555 + description: Telemetry overlay refs for refreshable metrics, including datastore used-space utilization selected by collector job and vSphere managed object ID.
556 - name: stats
557 type: object
558 unit: ""
555 - description: Counts of discovered inventory objects, actors, and links included in the response.
559 + description: Counts of discovered inventory objects, actors, links, and overlay refs included in the response.
560 performance: |
561 Uses cached collector state only:<br/>• No additional vCenter or ESXi API requests are triggered by the function<br/>• Response size grows with discovered inventory object count<br/>• `collect_network_topology` adds Network discovery during normal collector discovery cycles when enabled
562 security: |
src/go/tools/functions-validation/fixtures/topology-v1/vsphere.json
+125 -50
@@ -64,6 +64,8 @@
64 "normal",
65 "VMFS",
66 "type",
67 + "datastore_space_utilization",
68 + "vsphere_vcenter1",
69 "vsphere_network",
70 "network",
71 "network-1",
@@ -5426,6 +5428,25 @@
5428 ]
5429 }
5430 }
5431 + },
5432 + "overlay_templates": {
5433 + "datastore_space_utilization": {
5434 + "provider": "netdata.metrics",
5435 + "contexts": [
5436 + "vsphere.datastore_space_utilization"
5437 + ],
5438 + "dimensions": [
5439 + "used"
5440 + ],
5441 + "selector_params": [
5442 + "collect_job",
5443 + "id"
5444 + ],
5445 + "merge": {
5446 + "refs": "set",
5447 + "values": "last"
5448 + }
5449 + }
5450 }
5451 },
5452 "presentation": {
@@ -5551,9 +5572,9 @@
5572 18,
5573 24,
5574 30,
5554 - 37,
5555 - 44,
5556 - 48
5575 + 39,
5576 + 46,
5577 + 50
5578 ]
5579 },
5580 {
@@ -5564,9 +5585,9 @@
5585 19,
5586 25,
5587 31,
5567 - 38,
5568 - 45,
5569 - 49
5588 + 40,
5589 + 47,
5590 + 51
5591 ]
5592 },
5593 {
@@ -5577,9 +5598,9 @@
5598 20,
5599 26,
5600 32,
5580 - 39,
5581 - 46,
5582 - 50
5601 + 41,
5602 + 48,
5603 + 52
5604 ]
5605 },
5606 {
@@ -5590,9 +5611,9 @@
5611 21,
5612 27,
5613 33,
5593 - 40,
5594 - 47,
5595 - 51
5614 + 42,
5615 + 49,
5616 + 53
5617 ]
5618 },
5619 {
@@ -5718,8 +5739,8 @@
5739 28,
5740 16,
5741 16,
5721 - 42,
5722 - 42,
5742 + 44,
5743 + 44,
5744 16,
5745 16
5746 ]
@@ -5732,8 +5753,8 @@
5753 29,
5754 17,
5755 17,
5735 - 43,
5736 - 43,
5756 + 45,
5757 + 45,
5758 17,
5759 17
5760 ]
@@ -5821,22 +5842,22 @@
5842 {
5843 "codec": "values",
5844 "values": [
5824 - 38,
5825 - 38
5845 + 40,
5846 + 40
5847 ]
5848 },
5849 {
5850 "codec": "values",
5851 "values": [
5831 - 39,
5832 - 39
5852 + 41,
5853 + 41
5854 ]
5855 },
5856 {
5857 "codec": "values",
5858 "values": [
5838 - 43,
5839 - 43
5859 + 45,
5860 + 45
5861 ]
5862 }
5863 ]
@@ -5923,9 +5944,9 @@
5944 10,
5945 19,
5946 31,
5926 - 38,
5927 - 45,
5928 - 49
5947 + 40,
5948 + 47,
5949 + 51
5950 ]
5951 },
5952 {
@@ -5934,9 +5955,9 @@
5955 11,
5956 20,
5957 32,
5937 - 39,
5938 - 46,
5939 - 50
5958 + 41,
5959 + 48,
5960 + 52
5961 ]
5962 },
5963 {
@@ -6185,10 +6206,10 @@
6206 8,
6207 1,
6208 10,
6188 - 49,
6209 + 51,
6210 10,
6211 1,
6191 - 49
6212 + 51
6213 ]
6214 },
6215 {
@@ -6224,26 +6245,26 @@
6245 3,
6246 3,
6247 35,
6248 + 42,
6249 40,
6228 - 38,
6229 - 39,
6250 + 41,
6251 3,
6252 3,
6232 - 41,
6253 + 43,
6254 + 49,
6255 47,
6234 - 45,
6235 - 46,
6256 + 48,
6257 3,
6258 3,
6259 + 53,
6260 51,
6239 - 49,
6240 - 50,
6261 + 52,
6262 3,
6263 12,
6243 - 51,
6264 + 53,
6265 12,
6266 3,
6246 - 51
6267 + 53
6268 ]
6269 },
6270 {
@@ -6733,9 +6754,9 @@
6754 19,
6755 25,
6756 31,
6736 - 38,
6737 - 45,
6738 - 49
6757 + 40,
6758 + 47,
6759 + 51
6760 ]
6761 },
6762 {
@@ -6746,9 +6767,9 @@
6767 20,
6768 26,
6769 32,
6749 - 39,
6750 - 46,
6751 - 50
6770 + 41,
6771 + 48,
6772 + 52
6773 ]
6774 },
6775 {
@@ -6759,9 +6780,9 @@
6780 21,
6781 27,
6782 33,
6762 - 40,
6763 - 47,
6764 - 51
6783 + 42,
6784 + 49,
6785 + 53
6786 ]
6787 },
6788 {
@@ -6839,7 +6860,7 @@
6860 null,
6861 null,
6862 null,
6842 - 51
6863 + 53
6864 ]
6865 },
6866 {
@@ -6941,7 +6962,7 @@
6962 null,
6963 null,
6964 null,
6944 - 41,
6965 + 43,
6966 null,
6967 null
6968 ]
@@ -7263,6 +7284,59 @@
7284 }
7285 }
7286 },
7287 + "overlays": {
7288 + "refs": {
7289 + "rows": 1,
7290 + "columns": [
7291 + {
7292 + "id": "template",
7293 + "type": "string_ref",
7294 + "dictionary": "strings"
7295 + },
7296 + {
7297 + "id": "actor",
7298 + "type": "actor_ref",
7299 + "role": "reference"
7300 + },
7301 + {
7302 + "id": "collect_job",
7303 + "type": "string_ref",
7304 + "dictionary": "strings"
7305 + },
7306 + {
7307 + "id": "id",
7308 + "type": "string_ref",
7309 + "dictionary": "strings"
7310 + }
7311 + ],
7312 + "values": [
7313 + {
7314 + "codec": "values",
7315 + "values": [
7316 + 37
7317 + ]
7318 + },
7319 + {
7320 + "codec": "values",
7321 + "values": [
7322 + 4
7323 + ]
7324 + },
7325 + {
7326 + "codec": "values",
7327 + "values": [
7328 + 38
7329 + ]
7330 + },
7331 + {
7332 + "codec": "values",
7333 + "values": [
7334 + 32
7335 + ]
7336 + }
7337 + ]
7338 + }
7339 + },
7340 "stats": {
7341 "actor_rows": 8,
7342 "clusters": 1,
@@ -7275,6 +7349,7 @@
7349 "label_rows": 50,
7350 "link_rows": 9,
7351 "networks": 1,
7352 + "overlay_refs": 1,
7353 "resource_pools": 1,
7354 "vms": 1
7355 }
src/plugins.d/FUNCTION_TOPOLOGY_DEVELOPER_GUIDE.md
+34 -7
@@ -952,8 +952,15 @@ underlying canonical table.
952 Topology should be refreshable without recomputing topology. Overlay templates
953 define how the UI or Cloud can query metrics for an actor or link.
954
955 -Templates live once in the type registry. Overlay refs carry only template ids
956 -and parameters.
955 +Templates live once in the type registry. Overlay refs carry only template ids,
956 +one owner reference, and selector parameters.
957 +
958 +For `provider: "netdata.metrics"`, selector params are interpreted by the
959 +consumer:
960 +
961 +- `node_id` scopes the metric query to a node;
962 +- `collect_job` maps to the chart label `_collect_job`;
963 +- other params map to same-named chart labels.
964
965 Example:
966
@@ -965,7 +972,7 @@ Example:
972 "provider": "netdata.metrics",
973 "contexts": ["snmp.interface_traffic"],
974 "dimensions": ["received", "sent"],
968 - "selector_params": ["node_id", "interface_id"],
975 + "selector_params": ["node_id", "if_name"],
976 "merge": {
977 "refs": "set",
978 "values": "sum"
@@ -977,18 +984,38 @@ Example:
984 "refs": {
985 "rows": 1,
986 "columns": [
980 - {"id": "owner_kind", "type": "string_ref", "dictionary": "strings"},
981 - {"id": "owner", "type": "link_ref"},
987 {"id": "template", "type": "string_ref", "dictionary": "strings"},
988 + {"id": "link", "type": "link_ref", "role": "reference"},
989 {"id": "node_id", "type": "string_ref", "dictionary": "strings"},
984 - {"id": "interface_id", "type": "string_ref", "dictionary": "strings"}
990 + {"id": "if_name", "type": "string_ref", "dictionary": "strings"}
991 ],
986 - "values": []
992 + "values": [
993 + {"codec": "const", "value": 0},
994 + {"codec": "const", "value": 0},
995 + {"codec": "const", "value": 1},
996 + {"codec": "const", "value": 2}
997 + ]
998 }
999 }
1000 }
1001 ```
1002
1003 +Overlay refs use schema ids for column names, so selector params and ref column
1004 +ids must start with a letter and contain only letters, digits, `_`, `.`, `:`,
1005 +or `-`. The `template` column identifies the template by name. Exactly one
1006 +convention owner column must be present: `actor` with type `actor_ref` or `link`
1007 +with type `link_ref`. No other `actor_ref` or `link_ref` columns are valid in
1008 +overlay refs. Every row must have a non-null value in the owner column. Each
1009 +selector param used by a referenced template must have a same-named refs-table
1010 +column.
1011 +Selector params must not use the reserved refs-table convention column names
1012 +`template`, `actor`, or `link`.
1013 +
1014 +The `template` column and selector-param columns required by a row's resolved
1015 +template must be `string` or `string_ref`. Required selector-param values must
1016 +resolve to non-empty strings. Selector-param columns used by other templates may
1017 +be nullable and null on rows whose template does not require them.
1018 +
1019 Overlay refs are optional. Do not fabricate per-link bandwidth if the producer
1020 does not have it. For network sockets, current evidence may include snapshot
1021 metrics such as RTT or retransmissions, but that is not a time-series overlay