| 1 | # SNMP Profile Projection |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | SNMP profiles are one catalog with explicit projections for their consumers. |
| 6 | Regular SNMP metric collection, SNMP topology, SNMP licensing, and SNMP BGP |
| 7 | monitoring use the same profile loading, matching, inheritance, metadata, and |
| 8 | tag machinery, but they consume different profile views. |
| 9 | |
| 10 | ## Consumers |
| 11 | |
| 12 | The supported profile consumers are: |
| 13 | |
| 14 | - `metrics` - regular SNMP charted metrics and virtual metrics. |
| 15 | - `topology` - SNMP topology observations. |
| 16 | - `licensing` - typed SNMP network-device license rows. |
| 17 | - `bgp` - typed SNMP BGP device, peer, and peer-family rows. |
| 18 | |
| 19 | Profile metadata fields and top-level `metric_tags` default to all consumers. |
| 20 | They may narrow their visibility with: |
| 21 | |
| 22 | ```yaml |
| 23 | consumers: [metrics] |
| 24 | consumers: [topology] |
| 25 | consumers: [licensing] |
| 26 | consumers: [bgp] |
| 27 | ``` |
| 28 | |
| 29 | Metadata resource `id_tags` do not carry `consumers` today. They inherit the |
| 30 | metadata defaults used by charted metrics and topology and are not included in a |
| 31 | licensing-only projection. |
| 32 | |
| 33 | Metric rows under top-level `metrics:` are regular metric rows. They are |
| 34 | metrics-only. |
| 35 | |
| 36 | Topology rows live under top-level `topology:` and must declare a closed |
| 37 | `kind`. |
| 38 | |
| 39 | Licensing rows live under top-level `licensing:` and emit typed license rows, |
| 40 | not chart metrics and not hidden underscore-prefixed metrics. |
| 41 | |
| 42 | BGP rows live under top-level `bgp:` and emit typed BGP rows, not chart metrics |
| 43 | and not hidden underscore-prefixed metrics. |
| 44 | |
| 45 | ## Topology Rows |
| 46 | |
| 47 | Topology rows reuse the regular `MetricsConfig` scalar/table shape: |
| 48 | |
| 49 | ```yaml |
| 50 | topology: |
| 51 | - kind: lldp_rem |
| 52 | table: |
| 53 | OID: 1.0.8802.1.1.2.1.4.1 |
| 54 | name: lldpRemTable |
| 55 | symbols: |
| 56 | - OID: 1.0.8802.1.1.2.1.4.1.1.6 |
| 57 | name: lldp_rem |
| 58 | metric_tags: |
| 59 | - tag: lldp_loc_port_num |
| 60 | index: 2 |
| 61 | ``` |
| 62 | |
| 63 | Topology row symbol names must not be underscore-prefixed. The historical |
| 64 | `_topology_*` naming convention is not a classifier. |
| 65 | |
| 66 | Topology rows must not set regular metric chart/export fields on the row value |
| 67 | symbol: `chart_meta`, `metric_type`, `mapping`, `transform`, `scale_factor`, |
| 68 | `format`, or `constant_value_one`. |
| 69 | |
| 70 | `systemUptime` remains in `metrics:` for regular SNMP collection. It is not a |
| 71 | topology kind. Topology-specific uptime acquisition is collector code, not |
| 72 | profile topology schema. |
| 73 | |
| 74 | ## Topology Kinds |
| 75 | |
| 76 | The closed topology kind set is: |
| 77 | |
| 78 | - `lldp_loc_port` |
| 79 | - `lldp_loc_man_addr` |
| 80 | - `lldp_rem` |
| 81 | - `lldp_rem_man_addr` |
| 82 | - `lldp_rem_man_addr_compat` |
| 83 | - `cdp_cache` |
| 84 | - `if_name` |
| 85 | - `if_status` |
| 86 | - `if_duplex` |
| 87 | - `ip_if_index` |
| 88 | - `bridge_port_if_index` |
| 89 | - `fdb_entry` |
| 90 | - `qbridge_fdb_entry` |
| 91 | - `qbridge_vlan_entry` |
| 92 | - `stp_port` |
| 93 | - `vtp_vlan` |
| 94 | - `arp_entry` |
| 95 | - `arp_legacy_entry` |
| 96 | |
| 97 | ## Licensing Rows |
| 98 | |
| 99 | Licensing rows are row-centric because one license row can aggregate identity, |
| 100 | descriptors, state, timers, and usage signals: |
| 101 | |
| 102 | ```yaml |
| 103 | licensing: |
| 104 | - id: vendor-license-group |
| 105 | table: |
| 106 | OID: 1.3.6.1.4.1.example.1 |
| 107 | name: vendorLicenseTable |
| 108 | identity: |
| 109 | id: { index: 1 } |
| 110 | name: |
| 111 | symbol: |
| 112 | OID: 1.3.6.1.4.1.example.1.2 |
| 113 | name: vendorLicenseName |
| 114 | state: |
| 115 | symbol: |
| 116 | OID: 1.3.6.1.4.1.example.1.3 |
| 117 | name: vendorLicenseState |
| 118 | mapping: |
| 119 | 1: "0" |
| 120 | 2: "1" |
| 121 | 3: "2" |
| 122 | signals: |
| 123 | expiry: |
| 124 | timestamp: |
| 125 | symbol: |
| 126 | OID: 1.3.6.1.4.1.example.1.4 |
| 127 | name: vendorLicenseExpiry |
| 128 | format: snmp_dateandtime |
| 129 | ``` |
| 130 | |
| 131 | Scalar-only licensing rows are allowed. If a scalar row combines multiple |
| 132 | scalar signal OIDs into one license row, it must declare an explicit `id:` so |
| 133 | the grouping is stable. Otherwise scalar structural identity defaults to the |
| 134 | single scalar signal OID. |
| 135 | |
| 136 | `from: <oid>` lets a typed licensing value read a sibling OID directly. For |
| 137 | table rows, `from` must be a peer column under the same table OID. For scalar |
| 138 | rows, schema validation only checks OID syntax and the row's explicit identity |
| 139 | rules; there is no cross-profile reference path in profile validation. |
| 140 | |
| 141 | Supported licensing signal fields are: |
| 142 | |
| 143 | - state severity: `state` |
| 144 | - timers: `expiry`, `authorization`, `certificate`, `grace` |
| 145 | - usage: `used`, `capacity`, `available`, `percent` |
| 146 | |
| 147 | Timer signals may declare exactly one of the shorthand timer value, |
| 148 | `timestamp`, or `remaining`. Sentinel policies are closed names and are |
| 149 | evaluated by the typed licensing producer before runtime consumers see the row. |
| 150 | |
| 151 | ## BGP Rows |
| 152 | |
| 153 | BGP rows are row-centric because one BGP row can aggregate identity, |
| 154 | descriptors, connection state, counters, timers, route counts, limits, and |
| 155 | error fields: |
| 156 | |
| 157 | ```yaml |
| 158 | bgp: |
| 159 | - id: std-peer |
| 160 | MIB: BGP4-MIB |
| 161 | kind: peer |
| 162 | table: |
| 163 | OID: 1.3.6.1.2.1.15.3 |
| 164 | name: bgpPeerTable |
| 165 | identity: |
| 166 | neighbor: |
| 167 | symbol: |
| 168 | OID: 1.3.6.1.2.1.15.3.1.7 |
| 169 | name: bgpPeerRemoteAddr |
| 170 | format: ip_address |
| 171 | remote_as: |
| 172 | symbol: |
| 173 | OID: 1.3.6.1.2.1.15.3.1.9 |
| 174 | name: bgpPeerRemoteAs |
| 175 | format: uint32 |
| 176 | state: |
| 177 | symbol: |
| 178 | OID: 1.3.6.1.2.1.15.3.1.2 |
| 179 | name: bgpPeerState |
| 180 | mapping: |
| 181 | items: |
| 182 | 1: idle |
| 183 | 2: connect |
| 184 | 3: active |
| 185 | 4: opensent |
| 186 | 5: openconfirm |
| 187 | 6: established |
| 188 | connection: |
| 189 | established_uptime: |
| 190 | symbol: |
| 191 | OID: 1.3.6.1.2.1.15.3.1.16 |
| 192 | name: bgpPeerFsmEstablishedTime |
| 193 | ``` |
| 194 | |
| 195 | BGP row kinds are closed: |
| 196 | |
| 197 | - `device` |
| 198 | - `peer` |
| 199 | - `peer_family` |
| 200 | |
| 201 | Device rows are identity-free device summaries. Supported `device_counts` |
| 202 | fields are `peers`, `ibgp_peers`, `ebgp_peers`, and per-state fields under |
| 203 | `states`. `ibgp_peers` and `ebgp_peers` map to the public |
| 204 | `bgp.devices.peer_counts` dimensions `ibgp` and `ebgp`; `peers` maps to |
| 205 | `configured`. |
| 206 | |
| 207 | Peer rows must declare stable `neighbor` and `remote_as` identity fields. |
| 208 | Peer-family rows must additionally declare canonical `address_family` and |
| 209 | `subsequent_address_family` identity fields. |
| 210 | |
| 211 | The BGP state mapping is a closed RFC 4271 state contract. Rows that declare |
| 212 | state must map all six states by default: |
| 213 | |
| 214 | - `idle` |
| 215 | - `connect` |
| 216 | - `active` |
| 217 | - `opensent` |
| 218 | - `openconfirm` |
| 219 | - `established` |
| 220 | |
| 221 | Partial state coverage is allowed only when the row explicitly declares |
| 222 | `partial: true`. That escape hatch records that the profile author knowingly |
| 223 | accepts a partial source MIB. `partial_states: [...]` records the canonical |
| 224 | states that the source can represent. |
| 225 | |
| 226 | AFI/SAFI values are normalized to closed canonical strings at profile load. |
| 227 | Known address families include `ipv4`, `ipv6`, and `l2vpn`. Known subsequent |
| 228 | address families include `unicast`, `multicast`, `labeled_unicast`, `mvpn`, |
| 229 | `vpls`, `evpn`, and `vpn`. Vendor-private values require an explicit |
| 230 | `allow_private: true` on that identity value. |
| 231 | |
| 232 | BGP keeps normal ddsnmp table-row chart behavior. A typed BGP table row can |
| 233 | still produce one chart per row; the typed projection controls domain identity, |
| 234 | validation, function output, and old-protocol deletion, not a separate |
| 235 | BGP-specific chart cap. |
| 236 | |
| 237 | BGP value fields can read from the current row, from row indexes, from literal |
| 238 | values, or from a related table in the same resolved profile. Row-index value |
| 239 | sources can use `index: N`, `index_transform:`, or `index_from_end: N`. |
| 240 | `index_from_end` selects one OID index component from the tail and is for MIBs |
| 241 | where the wanted trailing INDEX component follows a variable-length component |
| 242 | such as `InetAddress`. Profiles should declare only one row-index selector per |
| 243 | typed BGP value. Validation rejects configs that set more than one of |
| 244 | `index`, `index_from_end`, and `index_transform` on the same typed BGP value. |
| 245 | |
| 246 | A cross-table typed value uses `table: <table_name>` with `index_transform:` |
| 247 | to derive the referenced row index: |
| 248 | |
| 249 | ```yaml |
| 250 | identity: |
| 251 | remote_as: |
| 252 | table: vendorBgpPeerTable |
| 253 | index_transform: |
| 254 | - start: 0 |
| 255 | drop_right: 2 |
| 256 | symbol: |
| 257 | OID: 1.3.6.1.4.1.example.2.1 |
| 258 | name: vendorBgpPeerRemoteAs |
| 259 | ``` |
| 260 | |
| 261 | If the referenced table is keyed differently, a typed BGP value may also use |
| 262 | `lookup_symbol:`. The collector first derives a lookup value from the current |
| 263 | row index with `index_transform:`, scans the referenced table for a row whose |
| 264 | `lookup_symbol` column has that value, and then reads the requested typed |
| 265 | `symbol` from the matched row. This is required for MIBs such as Juniper |
| 266 | BGP4-V2 where peer-family counters are indexed by peer ID, AFI, and SAFI while |
| 267 | peer identity is stored in a peer table keyed by routing-instance/local/remote |
| 268 | address components. |
| 269 | |
| 270 | Cross-table typed values are internal BGP row sources. They do not require |
| 271 | synthetic metric labels and must not reintroduce underscore-prefixed side |
| 272 | protocols. Scalar BGP rows cannot use `table:` sources. |
| 273 | |
| 274 | When a peer or peer-family row does not expose a routing instance, public chart |
| 275 | and function labels normalize the empty value to `default`. |
| 276 | |
| 277 | BGP function cache freshness is tracked per typed source profile. A failed BGP |
| 278 | source preserves its stale rows during the bounded stale window without |
| 279 | blocking other BGP sources from refreshing. After the stale window expires, the |
| 280 | function omits expired stale rows and returns 503 when no usable rows remain. |
| 281 | |
| 282 | ## Resolve And Projection |
| 283 | |
| 284 | `ddsnmp.Catalog.Resolve()` resolves profiles by `sysObjectID`, `sysDescr`, and |
| 285 | manual profile policy. The regular SNMP collector uses manual-profile fallback |
| 286 | semantics. The topology collector uses manual-profile augment semantics. |
| 287 | |
| 288 | `ResolvedProfileSet.Project(metrics)` returns the regular metrics view: |
| 289 | |
| 290 | - keeps `metrics`; |
| 291 | - keeps `virtual_metrics`; |
| 292 | - drops `topology`; |
| 293 | - filters metadata and top-level metric tags by `consumers`. |
| 294 | |
| 295 | `ResolvedProfileSet.Project(topology)` returns the topology view: |
| 296 | |
| 297 | - keeps `topology`; |
| 298 | - drops regular `metrics`; |
| 299 | - drops `virtual_metrics`; |
| 300 | - filters metadata and top-level metric tags by `consumers`. |
| 301 | |
| 302 | `ResolvedProfileSet.Project(licensing)` returns the licensing view: |
| 303 | |
| 304 | - keeps `licensing`; |
| 305 | - drops regular `metrics`; |
| 306 | - drops `topology`; |
| 307 | - drops `virtual_metrics`; |
| 308 | - filters metadata and top-level metric tags by `consumers`. |
| 309 | |
| 310 | `ResolvedProfileSet.Project(bgp)` returns the BGP view: |
| 311 | |
| 312 | - keeps `bgp`; |
| 313 | - drops regular `metrics`; |
| 314 | - drops `topology`; |
| 315 | - drops `licensing`; |
| 316 | - drops `virtual_metrics`; |
| 317 | - filters metadata and top-level metric tags by `consumers`. |
| 318 | |
| 319 | The regular SNMP collector uses `Project(metrics, licensing)` so one SNMP pass |
| 320 | can produce charted metrics and typed license rows. Single-consumer projections |
| 321 | remain pure. |
| 322 | |
| 323 | The regular SNMP collector can use `Project(metrics, licensing, bgp)` when BGP |
| 324 | typed rows are produced alongside ordinary charted metrics and licensing rows. |
| 325 | Single-consumer projections remain pure. |
| 326 | |
| 327 | `ProjectedView.FilterByKind()` is a topology view filter. VLAN-context topology |
| 328 | uses it with the VLAN-scopable kind set instead of hardcoded topology mixin |
| 329 | filenames. |
| 330 | |
| 331 | ## Inheritance And Merge Rules |
| 332 | |
| 333 | Profile inheritance must merge `topology:`, `licensing:`, and `bgp:` rows in |
| 334 | addition to `metrics:`, `virtual_metrics`, metadata, global metric tags, and |
| 335 | static tags. |
| 336 | |
| 337 | Topology row identity is: |
| 338 | |
| 339 | ```text |
| 340 | kind + table identity + symbol name |
| 341 | ``` |
| 342 | |
| 343 | The table identity is the table name when set, otherwise the table OID. Scalar |
| 344 | topology rows use kind plus scalar symbol name and OID. |
| 345 | |
| 346 | When a derived topology row overrides an inherited row with the same identity, |
| 347 | the derived row wins. Conflicting topology kinds for the same table/symbol |
| 348 | identity are load errors. |
| 349 | |
| 350 | Cross-profile deduplication runs after profile matching because it depends on |
| 351 | matched-profile specificity. It must deduplicate both regular metrics and |
| 352 | topology/licensing rows in the resolved matched set. |
| 353 | |
| 354 | Runtime licensing row structural identity is: |
| 355 | |
| 356 | ```text |
| 357 | origin profile id + table OID + row index |
| 358 | origin profile id + scalar signal OID |
| 359 | origin profile id + explicit scalar group id |
| 360 | ``` |
| 361 | |
| 362 | `origin profile id` is the logical profile file that declared the licensing row, |
| 363 | including mixin-origin rows after `extends:` merge. It is not the root matched |
| 364 | profile name and not an absolute workstation path. Repeated `(structural |
| 365 | identity, signal kind)` entries are load errors unless a valid inheritance |
| 366 | override handles them. |
| 367 | |
| 368 | Profile inheritance merge identity is the pre-collection form of that identity: |
| 369 | |
| 370 | ```text |
| 371 | table OID |
| 372 | scalar signal OID |
| 373 | explicit scalar group id |
| 374 | ``` |
| 375 | |
| 376 | Derived `licensing:` rows with the same merge identity replace inherited rows. |
| 377 | This keeps intentional `extends:` overrides valid while duplicate signal kinds |
| 378 | inside one resolved profile remain load errors. |
| 379 | |
| 380 | Runtime BGP row structural identity is: |
| 381 | |
| 382 | ```text |
| 383 | origin profile id + row kind + typed config id + table OID + row index |
| 384 | origin profile id + row kind + scalar signal OID |
| 385 | origin profile id + row kind + explicit scalar group id |
| 386 | ``` |
| 387 | |
| 388 | The typed config id is part of table-row structural identity because a single |
| 389 | SNMP table row can legitimately produce multiple typed BGP rows with different |
| 390 | semantics, such as separate peer-family views over one operational table. The |
| 391 | runtime key still length-prefixes every component, including an empty config id. |
| 392 | |
| 393 | Within that structural identity, display fields such as peer description, |
| 394 | local address, and peer identifier never determine storage identity. During the |
| 395 | legacy-to-typed BGP migration, typed structural identity owns typed row and |
| 396 | function row identity; chart and alert identity remain bound to the existing |
| 397 | public chart context and label contract until the legacy BGP path is removed. |
| 398 | |
| 399 | Profile inheritance merge identity is the pre-collection form of that identity: |
| 400 | |
| 401 | ```text |
| 402 | row kind + table OID + optional row id |
| 403 | row kind + scalar signal OID |
| 404 | row kind + explicit scalar group id |
| 405 | ``` |
| 406 | |
| 407 | Derived `bgp:` rows with the same merge identity replace inherited rows. Repeated |
| 408 | `(structural identity, signal kind)` entries are load errors unless a valid |
| 409 | inheritance override handles them. |
| 410 | |
| 411 | ## Delivery |
| 412 | |
| 413 | Regular metrics are emitted through `ProfileMetrics.Metrics`. |
| 414 | |
| 415 | Topology rows are emitted through `ProfileMetrics.TopologyMetrics` and carry |
| 416 | `Metric.TopologyKind`. |
| 417 | |
| 418 | Licensing rows are emitted through `ProfileMetrics.LicenseRows` and carry |
| 419 | typed grouped fields for identity, descriptors, state, timers, usage, tags, |
| 420 | origin profile id, table OID, row key, and structural id. |
| 421 | |
| 422 | BGP rows are emitted through `ProfileMetrics.BGPRows` and carry typed grouped |
| 423 | fields for row kind, identity, descriptors, admin/state, connection, traffic, |
| 424 | transitions, timers, last-error, last-notification, reason, graceful-restart, |
| 425 | route, route-limit, device-count, tags, origin profile id, table OID, row key, |
| 426 | and structural id. BGP does not use a generic signal-kind map; each field has |
| 427 | its own numeric, boolean, text, or state shape. |
| 428 | |
| 429 | `ProfileMetrics.HiddenMetrics` remains a generic delivery container for |
| 430 | underscore-prefixed non-topology and non-licensing metrics. SNMP topology and |
| 431 | SNMP licensing must not depend on hidden metrics. SNMP BGP must also not depend |
| 432 | on hidden metrics or underscore-prefixed tag protocols. |
| 433 | |
| 434 | Licensing row counts are reported through `Stats.Metrics.Licensing`. Ordinary |
| 435 | `Stats.Metrics.Tables` and `Stats.Metrics.Rows` remain regular chart-metric |
| 436 | table counters. Licensing timing and processing failures use their own |
| 437 | licensing fields in timing and processing-error stats. |
| 438 | |
| 439 | Top-level `metric_tags` on topology projections are profile/device labels. They |
| 440 | are applied through topology profile-tag ingestion and are not topology row |
| 441 | dispatch keys. |
| 442 | |
| 443 | ## Validation Guarantees |
| 444 | |
| 445 | Profile validation rejects: |
| 446 | |
| 447 | - unknown topology kinds; |
| 448 | - underscore-prefixed topology row value symbol names; |
| 449 | - regular metric chart/export-only fields on topology row value symbols; |
| 450 | - unknown licensing signal, sentinel, and state policy names; |
| 451 | - licensing rows without state or signals; |
| 452 | - scalar licensing rows that group multiple scalar signal OIDs without an |
| 453 | explicit `id`; |
| 454 | - scalar licensing rows with only literal values and no explicit `id`; |
| 455 | - repeated licensing signal kinds for the same structural identity; |
| 456 | - licensing table `from` OIDs outside the row table; |
| 457 | - underscore-prefixed licensing value names; |
| 458 | - regular metric chart/export-only fields, transforms, scale factors, and |
| 459 | constant-value hacks on licensing row value symbols; |
| 460 | - `extract_value`, `match_pattern`, or `match_value` on licensing row value |
| 461 | symbols; |
| 462 | - timer slots that set both timestamp-style and remaining-style values; |
| 463 | - unsupported licensing value formats; |
| 464 | - unknown BGP row kinds; |
| 465 | - BGP peer rows without neighbor or remote AS identity; |
| 466 | - BGP peer-family rows without neighbor, remote AS, AFI, or SAFI identity; |
| 467 | - BGP state mappings that do not cover all six RFC 4271 states unless explicit |
| 468 | partial coverage is declared; |
| 469 | - unknown BGP peer-state, AFI, or SAFI values; |
| 470 | - repeated BGP typed fields for the same structural identity; |
| 471 | - device-count fields on non-device BGP rows; |
| 472 | - route and route-limit fields on non-peer-family BGP rows; |
| 473 | - BGP table `from` OIDs outside the row table; |
| 474 | - BGP scalar values that use `table:` sources; |
| 475 | - BGP scalar values that use row-index sources such as `index:`, |
| 476 | `index_from_end:`, or `index_transform:`; |
| 477 | - BGP values that set more than one row-index selector (`index`, |
| 478 | `index_from_end`, or `index_transform`); |
| 479 | - BGP cross-table value sources without a source OID; |
| 480 | - BGP cross-table value sources whose source or lookup OID is outside the |
| 481 | referenced table when that referenced table is declared by another BGP row in |
| 482 | the resolved profile; |
| 483 | - BGP `lookup_symbol` value sources without a referenced table; |
| 484 | - underscore-prefixed BGP value names; |
| 485 | - regular metric chart/export-only fields, transforms, scale factors, and |
| 486 | constant-value hacks on BGP row value symbols; |
| 487 | - invalid `consumers` values; |
| 488 | - virtual metrics whose sources resolve to topology rows. |