@cryptotaxi247 / netdata-1 / commits / 3a55e1526

ibm.d: various fixes (#21204)

* ibm.d: guard odbc bridge from runaway allocations * removed obsolete todo doc * ibm.d: bound mq/db2 cardinality with selectors and group charts

Costa Tsaousis committed Oct 25, 2025 at 12:49 UTC 3a55e15262b53b772601653c634d28d39b418ac7

32 files changed +4639 -822

TODO-AS400-SPEED.md deleted

-112

@@ -1,112 +0,0 @@

1 -# IBM i AS400 Collector – Query Performance Roadmap

2 -

3 -## 1. Current Situation

4 -- The latency chart (`netdata.plugin_ibm.as400_query_latency`) for the last 10 minutes shows sustained slowdowns from three collectors:

5 - - `job_queues` / `count_job_queues`: average ~2.9 s, peaks >4.0 s.

6 - - `output_queue_info`: average ~2.8 s, peaks >4.3 s.

7 - - `other`: residual bucket averaging ~1.2 s (needs attribution).

8 -- Message queue collectors are currently disabled in production (0 ms latency), but the SQL still performs full scans when enabled.

9 -- The collector remains single-threaded; any slow query blocks all 5 s metrics.

10 -

11 -## 2. Immediate Offenders (must-fix)

12 -The following collectors must be reworked first; they combine high latency with unbounded scans:

13 -

15 -| --- | --- | --- | --- |

16 -| `message_queue_aggregates`, `count_message_queues` | `QSYS2.MESSAGE_QUEUE_INFO` (detail table, full scan) | Full table scan, aggregation, `FETCH FIRST` post-scan | ✅ done (table function per queue, documented fallback + defaults) |

19 -| `collectActiveJobs` (`buildActiveJobQuery`) | `TABLE(QSYS2.ACTIVE_JOB_INFO(...))` per job | ✅ done (requires explicit `active_jobs` list; one table-function call per job with user/job filters, skips when not found) |

22 -

23 -## 3. Full Query Inventory & Table Scan Risk

24 -

25 -Legend: ✅ (fine), ⚠️ (needs filtering/guardrails), ❗️ (full scan / heavy).

26 -

28 -| --- | --- | --- | --- | --- | --- |

29 -| `querySystemStatus*`, `querySystemActivity*`, `queryConfiguredCPUs`, `queryAverageCPU`, `querySystemASP`, `queryActiveJobs`, `querySerialNumber`, `querySystemModel` | `TABLE(QSYS2.SYSTEM_STATUS(...))` | Table function | No additional filters | ✅ | Table function already scoped by IBM; negligible overhead |

44 -| `queryIBMiVersion`, `queryIBMiVersionDataArea`, `queryTechnologyRefresh` | Metadata views | — | — | ✅ | One-off |

45 -| `callAnalyzePlanCache`, `queryPlanCacheSummary` | Stored proc + temp table | — | None | ❗️ | Run asynchronously with throttling |

46 -

47 -## 4. Strategy (phased)

48 -

49 -### Phase 1 – Query Rewrite & Filtering

50 -1. **Explicit queue targets only:** retire generic selectors and `max_*` limits for message/job/output queues; accept a list of fully-qualified queue names (library/queue) and run one table-function call per entry. Provide sensible defaults (e.g., `QSYS/QSYSOPR`, `QSYS/QSYSMSG`, `QSYS/QHST`) and keep a simple `collect_*` enable/disable switch for each feature.

51 -2. **Prefer table functions (7.4+):** when available, use IBM’s table functions (`TABLE(QSYS2.MESSAGE_QUEUE_INFO(...))`, `TABLE(QSYS2.ACTIVE_JOB_INFO(...))`, etc.) so the filtering happens via parameters instead of scanning the entire detail view. On older releases fall back to views but construct strict `WHERE library/name IN (...)` clauses from the explicit lists.

52 -3. **Helper for list expansion:** implement shared helpers that turn config lists into SQL fragments and per-collector worklists. If glob support is introduced, expand to explicit names on a periodic cadence and reuse the list so the optimized SQL always receives concrete values (never `LIKE`).

53 -4. **Lightweight counts:** reuse the explicit target list for cardinality checks (length of the list) instead of running `COUNT(*)` scans where possible; otherwise cache the last-known count and refresh less frequently.

54 -5. **Output queues:** ✅ implemented via `OUTPUT_QUEUE_ENTRIES` with view fallback.

55 -

56 -### Phase 2 – Background Producers & Staging

57 -1. Introduce a dedicated slow-path worker (“producer”) that runs on aligned beats (default 30 s) to execute heavy collectors (`message_queue_*`, `job_queue_*`, `output_queue_*`, `count_subsystems`/`collectSubsystems`, plan cache). Producer keeps its own `context.Context` so it stops when the job stops or configuration reloads.

58 -2. Expose new configuration knobs:

59 - - `slow_path` (bool, default `true`) to enable/disable the background worker.

60 - - `slow_path_update_every` (duration, default `30s`, must be ≥ main `update_every`) to control beat cadence.

61 - - `slow_path_max_connections` (int, default `1`) to set `db.SetMaxOpenConns` for the slow worker so queries can run concurrently when increased.

62 -3. Convert the existing sequential collectors into “consumers” for the heavy datasets: on each fast loop they read from the slow-path cache (values + timestamps + error state) and export data only when fresh. Fast loop stays synchronous for lightweight collectors (system status, memory pools, disks, temp storage, job info, active jobs, network stats, HTTP servers, etc.).

63 -4. Implement the cache layer with per-feature entries (latest value, timestamp, last error) and proper locking. Bring query-latency logging inside the slow producer so timing reflects the asynchronous work.

64 -5. Ensure lifecycle management: producer goroutines respect `slow_path` changes and job shutdown (cancel + wait); structure the code so we can register additional slow tracks in the future without refactoring.

65 -

66 -### Phase 3 – Instrumentation & Validation

67 -1. Extend query latency tracking to name every query (remove “other” bucket).

68 -2. Add debug telemetry to confirm selector expansion and SQL rewriting.

69 -3. Establish regression test fixtures (mock DB responses) for selector translation and per-queue sampling.

70 -

71 -## 5. Selector Handling & Glob Patterns

72 -- Current usage does not rely on glob patterns, but long-term usability may. Proposed approach:

73 - 1. Accept plain lists (preferred). For globs, expand to explicit queue/library names on a cadence (e.g., refresh every minute using targeted list queries).

74 - 2. Build per-feature caches that hold resolved names and feed them into the optimized SQL (`WHERE library/name IN (...)`) or per-queue execution list.

75 - 3. When expansion yields no matches, skip the heavy query entirely to avoid wasted scans.

76 -

77 -## 6. Action Plan

78 -

79 -1. **Message queues:** (✅ done) explicit list + table function (7.4+) / documented behaviour on older releases.

80 -2. **Job queues:** keep the `QSYS2.JOB_QUEUE_INFO` per-queue WHERE filtering, but execute it from the slow producer cache and, when present, enrich results with `SYSTOOLS.JOB_QUEUE_ENTRIES` (7.4 TR9+/7.5 TR3+) without increasing call volume.

81 -3. **Output queues:** mirror job queue treatment, including spool file considerations. (Research complete: use `QSYS2.OUTPUT_QUEUE_ENTRIES(library, queue, detail)` table function on 7.2+; fall back to view if unavailable.)

82 -4. **Active jobs:** ✅ complete — configuration now requires explicit `active_jobs` list; per-job table-function queries replace global scan.

83 -5. **Subsystem inventory:** migrate to slow producer/cache, publishing active subsystem metrics only when refreshed.

84 -6. **Plan cache:** migrate to the slow producer (dedicated cadence & timeout); only publish when fresh data arrives.

85 -7. **Producer/consumer cache:** implement the background fetchers, shared caches (values + timestamps + errors), and consumer-side merge logic so fast loops simply read whichever data is ready. Slow worker should support concurrent query fan-out per `slow_path_max_connections`.

86 -8. **SetUpdateEvery plumbing:** after the cache layer exists, wire per-context update intervals so slow charts report their natural cadence.

87 -9. **Selector infrastructure:** build reusable helper that expands globs (if any) into explicit lists and share it across queues/disks/interfaces.

88 -10. **Instrumentation:** name all queries in latency map, add logging for large result sets, and monitor improvements via `netdata.plugin_ibm.as400_query_latency`, splitting fast/slow instances so both timings are visible.

89 -

90 -## 7. Outstanding Questions

91 -- Confirm IBM i versions we must support (table functions require 7.4+). Decide on fallback mechanism for older releases.

92 -- Determine acceptable concurrency level per partition (number of simultaneous ODBC connections).

93 -- Validate whether additional IBM services (e.g., HTTP server info) provide table functions or existing views can be filtered by name list.

94 -- Decide cadence for selector refresh (e.g., once per minute vs. per iteration) balancing staleness vs. overhead.

95 -- Document how to reintroduce queue-total counts if a customer insists.

96 -

97 -## 8. Customer Request: Optional Queue Totals

98 -- Customer asked to restore `count_message_queues`, `count_job_queues`, `count_output_queues`, acknowledging high cost but wants opt-in capability.

99 -- Plan of record (pending research validation):

100 - - **Configuration flags:** per-query booleans – `collect_message_queue_totals`, `collect_job_queue_totals`, `collect_output_queue_totals` – default `false` with explicit warnings about full scans.

101 - - **Batch path:** mirror the slow-path configuration style (`batch_path_enabled`, `batch_path_update_every`, `batch_path_max_connections`). Default cadence 60 s for testing (enforce ≥60 s at load time, document ≥600 s for production). The worker starts only if at least one total flag is enabled.

102 - - **Metrics/contexts:** emit two contexts when totals are enabled:

103 - * `as400.queues_count` with a single `queues` dimension and instances labeled by `queue_type` (`message_queue`, `job_queue`, `output_queue`) and `item_type` (matching `message`, `job`, `spooled_file` for consistency).

104 - * `as400.queued_items` with a single `items` dimension using the same label scheme.

105 - Only instances corresponding to enabled flags are published.

106 - - **Families:** reorganize queue-related families under a unified `queues/` namespace before release:

107 - * rename existing `messaging/message_queues` → `queues/message`, `messaging/output_queues` → `queues/output`.

108 - * move per-job-queue charts from `workloads/job_queues` into `queues/job`.

109 - * add the new aggregate charts under `queues/overview`.

110 - - **Error handling:** reuse existing slow-path behaviour—skip iteration on failure, log once per key, clear after a success.

111 - - **Testing:** verify against pub400.com with flags on/off, measure latency impact, confirm throttled logging, and adjust the batch interval before handing off to production.

112 -- Deep research in progress to see if IBM i offers lighter-weight alternatives (aggregated services, APIs, event feeds). We'll adjust implementation if better options surface.

src/go/plugin/ibm.d/config/ibm.d/db2.conf

+60 -51

@@ -102,73 +102,82 @@ jobs:

102 # # Default: '' (all databases)

103 # collect_databases_matching: ''

104 #

105 - # # Collect buffer pool metrics

106 - # # Default: true

107 - # collect_bufferpool_metrics: true

108 - #

109 - # # Maximum number of buffer pools to monitor

110 - # # Default: 20 (0 = unlimited)

111 - # max_bufferpools: 20

112 - #

113 - # # Pattern to filter buffer pools. Supports wildcards (*, ?) and multiple patterns separated by |

114 - # # Default: '' (all buffer pools)

115 - # collect_bufferpools_matching: ''

116 - #

117 - # # Collect tablespace metrics

118 - # # Default: true

119 - # collect_tablespace_metrics: true

120 - #

121 - # # Maximum number of tablespaces to monitor

122 - # # Default: 100 (0 = unlimited)

123 - # max_tablespaces: 50

105 +# # Collect buffer pool metrics

106 +# # Default: true

107 +# collect_bufferpool_metrics: true

108 +#

109 +# # Maximum number of buffer pools to chart individually (others go to __other__)

110 +# # Default: 20 (0 = disable per-instance charts)

111 +# max_bufferpools: 20

112 +#

113 +# # Buffer pool selectors (wildcards supported)

114 +# include_bufferpools:

115 +# - 'IBMDEFAULTBP'

116 +# - 'IBMSYSTEMBP*'

117 +# exclude_bufferpools:

118 +# - 'TMP*'

119 #

125 - # # Pattern to filter tablespaces. Supports wildcards (*, ?) and multiple patterns separated by |

126 - # # Example: 'USER*' to monitor only user tablespaces, '*TEMP*' for temporary tablespaces

127 - # # Default: '' (all tablespaces)

128 - # collect_tablespaces_matching: ''

120 +# # Collect tablespace metrics

121 +# # Default: true

122 +# collect_tablespace_metrics: true

123 +#

124 +# # Maximum number of tablespaces to chart individually (others go to __other__)

125 +# # Default: 50 (0 = disable per-instance charts)

126 +# max_tablespaces: 50

127 +#

128 +# # Tablespace selectors

129 +# include_tablespaces:

130 +# - 'SYSCATSPACE'

131 +# - 'TEMPSPACE*'

132 +# exclude_tablespaces:

133 +# - '*TEMP2*'

134 #

135 # # Collect connection metrics

136 # # Default: true

137 # collect_connection_metrics: true

138 #

134 - # # Maximum number of connections to monitor

135 - # # Default: 200 (0 = unlimited)

136 - # max_connections: 100

137 - #

138 - # # Pattern to filter connections by application name. Supports wildcards (*, ?) and multiple patterns separated by |

139 - # # Example: '*jdbc*' to monitor only JDBC connections, '*java*|*python*' for Java or Python

140 - # # Default: '' (all connections)

141 - # collect_connections_matching: ''

142 - #

143 - # # Collect table metrics (size, activity)

144 - # # Default: false

145 - # collect_table_metrics: false

146 - #

147 - # # Maximum number of tables to monitor

148 - # # Default: 50 (0 = unlimited)

149 - # max_tables: 50

139 +# # Maximum number of connections to chart individually

140 +# # Default: 50 (0 = disable per-connection charts)

141 +# max_connections: 50

142 +#

143 +# # Connection selectors (matched against application_id, application_name, hostname)

144 +# include_connections:

145 +# - 'db2sysc*'

146 +# - 'db2agent*'

147 +# exclude_connections:

148 +# - '*TEMP*'

149 #

151 - # # Pattern to filter tables (schema.table). Supports wildcards (*, ?) and multiple patterns.

152 - # # Example: 'MYSCHEMA.*' to monitor all tables in MYSCHEMA

153 - # # Default: '' (all tables)

154 - # collect_tables_matching: ''

150 +# # Collect table metrics (size, activity)

151 +# # Default: false

152 +# collect_table_metrics: false

153 +#

154 +# # Maximum number of tables to chart individually

155 +# # Default: 25 (0 = disable per-table charts)

156 +# max_tables: 25

157 +#

158 +# # Table selectors (schema.table syntax)

159 +# include_tables:

160 +# - 'MYSCHEMA.*'

161 +# exclude_tables:

162 +# - '*.TMP*'

163 #

164 # # Collect index metrics (scans, fragmentation)

165 # # Default: false

166 # collect_index_metrics: false

167 #

160 - # # Maximum number of indexes to monitor

161 - # # Default: 100 (0 = unlimited)

162 - # max_indexes: 100

168 +# # Maximum number of indexes to chart individually

169 +# # Default: 50 (0 = disable per-index charts)

170 +# max_indexes: 50

171 #

172 # # Number of days to look back for backup history

173 # # Default: 30

174 # backup_history_days: 30

175 #

168 - # # Pattern to filter indexes (schema.index). Supports wildcards (*, ?) and multiple patterns.

169 - # # Example: 'MYSCHEMA.IDX_*' to monitor specific indexes in MYSCHEMA

170 - # # Default: '' (all indexes)

171 - # collect_indexes_matching: ''

176 +# # Index selectors (schema.index syntax)

177 +# include_indexes:

178 +# - 'MYSCHEMA.IDX_*'

179 +# exclude_indexes:

180 +# - '*.TMP*'

181 #

182 # # Collect SQL statement cache metrics (DB2 9.7+ LUW)

183 # # Default: true

@@ -373,4 +382,4 @@ jobs:

382 ## 4. Network latency: For remote databases, consider:

383 ## - Increasing timeout values

384 ## - Running a local netdata node closer to the database

376 -## - Using connection compression if supported

\ No newline at end of file

385 +## - Using connection compression if supported

src/go/plugin/ibm.d/config/ibm.d/mq.conf

+20 -13

@@ -121,14 +121,17 @@

121 # collect_sys_topics: false

122 #

123 # # === FILTERING AND SELECTORS ===

124 -# # Pattern to filter queues (wildcards supported: *, ?)

125 -# # Empty means collect all queues

126 -# # Default: "" (all queues)

127 -# # Examples:

128 -# # "APP.*" - All queues starting with APP

129 -# # "*.REQUEST" - All request queues

130 -# # "DEV.*|TEST.*" - DEV or TEST queues

131 -# queue_selector: ""

124 +# # Queue include patterns (wildcards supported). Empty list means all queues are eligible.

125 +# # Default: monitors critical system queues required for healthy operations.

126 +# include_queues:

127 +# - "SYSTEM.DEAD.LETTER.QUEUE"

128 +# - "SYSTEM.ADMIN.COMMAND.QUEUE"

129 +# - "SYSTEM.ADMIN.STATISTICS.QUEUE"

130 +#

131 +# # Queue exclude patterns applied after includes. Defaults drop churn-heavy namespaces.

132 +# exclude_queues:

133 +# - "SYSTEM.*"

134 +# - "AMQ.*"

135 #

136 # # Pattern to filter channels (wildcards supported)

137 # # Default: "" (all channels)

@@ -149,8 +152,8 @@

152 # # === CARDINALITY LIMITS ===

153 # # Maximum number of queues to monitor (0 = unlimited)

154 # # Use this to prevent excessive memory usage on large systems

152 -# # Default: 100

153 -# max_queues: 100

155 +# # Default: 50

156 +# max_queues: 50

157 #

158 # # Maximum number of channels to monitor (0 = unlimited)

159 # # Default: 100

@@ -190,7 +193,11 @@

193 # collect_system_topics: false

194 #

195 # # Filter to monitor only production queues

193 -# queue_selector: "PROD.*|APP.*"

196 +# include_queues:

197 +# - "PROD.*"

198 +# - "APP.*"

199 +# exclude_queues:

200 +# - "PROD.TEMP.*"

201 # max_queues: 200

202 #

203 # # Monitor only active channels

@@ -263,10 +270,10 @@

270 ## - For production, configure proper channel authentication records

271 ##

272 ## 5. High cardinality warnings

266 -## - Use selectors to filter objects: queue_selector: "APP.*"

273 +## - Use selectors to filter objects: include_queues: ["APP.*"]

274 ## - Set max limits: max_queues: 50

275 ## - Disable system object collection if not needed

276 ##

277 ## To test configuration:

278 ## cd /usr/libexec/netdata/plugins.d/

272 -## sudo -u netdata ./ibm.d.plugin -d -m mq --dump=3s --dump-summary

\ No newline at end of file

279 +## sudo -u netdata ./ibm.d.plugin -d -m mq --dump=3s --dump-summary

src/go/plugin/ibm.d/modules/db2/README.md

+120 -9

@@ -6,6 +6,18 @@ Monitors IBM DB2 databases using system catalog views and MON_GET_* table

6 functions to expose connections, locking, buffer pool efficiency, tablespace

7 capacity, and workload performance metrics.

9 +Detailed charts are opt-in per object family through include/exclude lists.

10 +Defaults focus on engine activity (system connections, core buffer pools,

11 +catalog tablespaces). Matching uses glob patterns that can target schema or

12 +application names, with include rules taking precedence over excludes.

13 +

14 +When the number of matching objects exceeds the configured `max_*` limits,

15 +the collector publishes deterministic top-N per-instance charts, aggregates

16 +the remainder under `group="__other__"`, and logs a throttled warning so you

17 +can refine selectors before cardinality runs away. Group charts (by schema,

18 +application prefix, or buffer pool family) are always emitted so high-level

19 +visibility is preserved even when individual instances are trimmed.

20 +

22 This collector is part of the [Netdata](https://github.com/netdata/netdata) monitoring solution.

@@ -90,6 +102,28 @@ Metrics:

102 | db2.bufferpool_instance_pages | used, total | pages |

103 | db2.bufferpool_instance_writes | writes | writes/s |

104

105 +### Per bufferpoolgroup

106 +

107 +These metrics refer to individual bufferpoolgroup instances.

108 +

109 +Labels:

110 +

111 +| Label | Description |

112 +|:------|:------------|

113 +| group | Group identifier |

114 +

115 +Metrics:

116 +

117 +| Metric | Dimensions | Unit |

118 +|:-------|:-----------|:-----|

119 +| db2.bufferpool_group_hit_ratio | overall | percentage |

120 +| db2.bufferpool_group_detailed_hit_ratio | data, index, xda, column | percentage |

121 +| db2.bufferpool_group_reads | logical, physical | reads/s |

122 +| db2.bufferpool_group_data_reads | logical, physical | reads/s |

123 +| db2.bufferpool_group_index_reads | logical, physical | reads/s |

124 +| db2.bufferpool_group_pages | used, total | pages |

125 +| db2.bufferpool_group_writes | writes | writes/s |

126 +

127 ### Per connection

128

129 These metrics refer to individual connection instances.

@@ -114,6 +148,26 @@ Metrics:

148 | db2.connection_wait_time | lock, log_disk, log_buffer, pool_read, pool_write, direct_read, direct_write, fcm_recv, fcm_send | milliseconds |

149 | db2.connection_processing_time | routine, compile, section, commit, rollback | milliseconds |

150

151 +### Per connectiongroup

152 +

153 +These metrics refer to individual connectiongroup instances.

154 +

155 +Labels:

156 +

157 +| Label | Description |

158 +|:------|:------------|

159 +| group | Group identifier |

160 +

161 +Metrics:

162 +

163 +| Metric | Dimensions | Unit |

164 +|:-------|:-----------|:-----|

165 +| db2.connection_group.count | count | connections |

166 +| db2.connection_group.state | state | state |

167 +| db2.connection_group.activity | read, written | rows/s |

168 +| db2.connection_group.wait_time | lock, log_disk, log_buffer, pool_read, pool_write, direct_read, direct_write, fcm_recv, fcm_send | milliseconds |

169 +| db2.connection_group.processing_time | routine, compile, section, commit, rollback | milliseconds |

170 +

171 ### Per database

172

173 These metrics refer to individual database instances.

@@ -148,6 +202,22 @@ Metrics:

202 |:-------|:-----------|:-----|

203 | db2.index_usage | index, full | scans/s |

204

205 +### Per indexgroup

206 +

207 +These metrics refer to individual indexgroup instances.

208 +

209 +Labels:

210 +

211 +| Label | Description |

212 +|:------|:------------|

213 +| group | Group identifier |

214 +

215 +Metrics:

216 +

217 +| Metric | Dimensions | Unit |

218 +|:-------|:-----------|:-----|

219 +| db2.index_group_usage | index, full | scans/s |

220 +

221 ### Per memorypool

222

223 These metrics refer to individual memorypool instances.

@@ -226,6 +296,23 @@ Metrics:

296 | db2.table_size | data, index, long_obj | bytes |

297 | db2.table_activity | read, written | rows/s |

298

299 +### Per tablegroup

300 +

301 +These metrics refer to individual tablegroup instances.

302 +

303 +Labels:

304 +

305 +| Label | Description |

306 +|:------|:------------|

307 +| group | Group identifier |

308 +

309 +Metrics:

310 +

311 +| Metric | Dimensions | Unit |

312 +|:-------|:-----------|:-----|

313 +| db2.table_group_size | data, index, long_obj | bytes |

314 +| db2.table_group_activity | read, written | rows/s |

315 +

316 ### Per tableio

317

318 These metrics refer to individual tableio instances.

@@ -267,6 +354,25 @@ Metrics:

354 | db2.tablespace_usable_size | total, usable | bytes |

355 | db2.tablespace_state | state | state |

356

357 +### Per tablespacegroup

358 +

359 +These metrics refer to individual tablespacegroup instances.

360 +

361 +Labels:

362 +

363 +| Label | Description |

364 +|:------|:------------|

365 +| group | Group identifier |

366 +

367 +Metrics:

368 +

369 +| Metric | Dimensions | Unit |

370 +|:-------|:-----------|:-----|

371 +| db2.tablespace_group_usage | used | percentage |

372 +| db2.tablespace_group_size | used, free | bytes |

373 +| db2.tablespace_group_usable_size | total, usable | bytes |

374 +| db2.tablespace_group_state | state | state |

375 +

376

377 ## Configuration

378

@@ -303,20 +409,25 @@ The following options can be defined globally or per job.

409 | CollectIndexMetrics | CollectIndexMetrics toggles index usage metrics. | `auto` | no | - | - |

410 | MaxDatabases | MaxDatabases caps the number of databases charted. | `10` | no | - | - |

411 | MaxBufferpools | MaxBufferpools caps the number of buffer pools charted. | `20` | no | - | - |

306 -| MaxTablespaces | MaxTablespaces caps the number of tablespaces charted. | `100` | no | - | - |

307 -| MaxConnections | MaxConnections caps the number of connection instances charted. | `200` | no | - | - |

308 -| MaxTables | MaxTables caps the number of tables charted. | `50` | no | - | - |

309 -| MaxIndexes | MaxIndexes caps the number of indexes charted. | `100` | no | - | - |

412 +| MaxTablespaces | MaxTablespaces caps the number of tablespaces charted. | `50` | no | - | - |

413 +| MaxConnections | MaxConnections caps the number of connection instances charted. | `50` | no | - | - |

414 +| MaxTables | MaxTables caps the number of tables charted. | `25` | no | - | - |

415 +| MaxIndexes | MaxIndexes caps the number of indexes charted. | `50` | no | - | - |

416 | BackupHistoryDays | BackupHistoryDays controls how many days of backup history are retrieved. | `30` | no | - | - |

417 | CollectMemoryMetrics | CollectMemoryMetrics enables memory pool statistics. | `true` | no | - | - |

418 | CollectWaitMetrics | CollectWaitMetrics enables wait time statistics (locks, logs, I/O). | `true` | no | - | - |

419 | CollectTableIOMetrics | CollectTableIOMetrics enables table I/O statistics when available. | `true` | no | - | - |

420 | CollectDatabasesMatching | CollectDatabasesMatching filters databases by name using glob patterns. | `` | no | - | - |

315 -| CollectBufferpoolsMatching | CollectBufferpoolsMatching filters buffer pools by name using glob patterns. | `` | no | - | - |

316 -| CollectTablespacesMatching | CollectTablespacesMatching filters tablespaces by name using glob patterns. | `` | no | - | - |

317 -| CollectConnectionsMatching | CollectConnectionsMatching filters monitored connections by application ID. | `` | no | - | - |

318 -| CollectTablesMatching | CollectTablesMatching filters tables by schema/name. | `` | no | - | - |

319 -| CollectIndexesMatching | CollectIndexesMatching filters indexes by schema/name. | `` | no | - | - |

421 +| IncludeConnections | IncludeConnections filters monitored connections by application ID or application name (wildcards supported). | `[db2sysc* db2agent* db2hadr* db2acd* db2bmgr*]` | no | - | - |

422 +| ExcludeConnections | ExcludeConnections excludes connections after inclusion matching. | `[*TEMP*]` | no | - | - |

423 +| IncludeBufferpools | IncludeBufferpools filters buffer pools by name. | `[IBMDEFAULTBP IBMSYSTEMBP* IBMHADRBP*]` | no | - | - |

424 +| ExcludeBufferpools | ExcludeBufferpools excludes buffer pools after inclusion. | `nil` | no | - | - |

425 +| IncludeTablespaces | IncludeTablespaces filters tablespaces by name. | `[SYSCATSPACE TEMPSPACE* SYSTOOLSPACE]` | no | - | - |

426 +| ExcludeTablespaces | ExcludeTablespaces excludes tablespaces after inclusion. | `[TEMPSPACE2]` | no | - | - |

427 +| IncludeTables | IncludeTables filters tables by schema/name. | `nil` | no | - | - |

428 +| ExcludeTables | ExcludeTables excludes tables after inclusion. | `nil` | no | - | - |

429 +| IncludeIndexes | IncludeIndexes filters indexes by schema/name. | `nil` | no | - | - |

430 +| ExcludeIndexes | ExcludeIndexes excludes indexes after inclusion. | `nil` | no | - | - |

431

432 ### Examples

433

src/go/plugin/ibm.d/modules/db2/collector.go

+131 -12

@@ -8,7 +8,9 @@ import (

8 "context"

9 "database/sql"

10 "errors"

11 + "strings"

12 "sync"

13 + "time"

15 "github.com/netdata/netdata/go/plugins/pkg/matcher"

16 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/framework"

@@ -45,12 +47,22 @@ type Collector struct {

47 prefetchers map[string]*prefetcherInstanceMetrics

49 // Selectors

48 - databaseSelector matcher.Matcher

49 - bufferpoolSelector matcher.Matcher

50 - tablespaceSelector matcher.Matcher

51 - connectionSelector matcher.Matcher

52 - tableSelector matcher.Matcher

53 - indexSelector matcher.Matcher

50 + databaseSelector matcher.Matcher

51 +

52 + connectionInclude matcher.Matcher

53 + connectionExclude matcher.Matcher

54 +

55 + bufferpoolInclude matcher.Matcher

56 + bufferpoolExclude matcher.Matcher

57 +

58 + tablespaceInclude matcher.Matcher

59 + tablespaceExclude matcher.Matcher

60 +

61 + tableInclude matcher.Matcher

62 + tableExclude matcher.Matcher

63 +

64 + indexInclude matcher.Matcher

65 + indexExclude matcher.Matcher

67 // DB2 version info

68 version string

@@ -60,12 +72,7 @@ type Collector struct {

72 serverInfo serverInfo

74 // Filtering mode flags

63 - databaseFilterMode bool

64 - bufferpoolFilterMode bool

65 - tablespaceFilterMode bool

66 - connectionFilterMode bool

67 - tableFilterMode bool

68 - indexFilterMode bool

75 + databaseFilterMode bool

77 // Resilience tracking

78 disabledMetrics map[string]bool

@@ -87,6 +94,51 @@ type Collector struct {

95 once sync.Once

96 metaOnce sync.Once

97 +

98 + warnMu sync.Mutex

99 + warns map[string]time.Time

100 +}

101 +

102 +const warnThrottleInterval = 10 * time.Minute

103 +

104 +func compileMatcher(patterns []string) (matcher.Matcher, error) {

105 + if len(patterns) == 0 {

106 + return nil, nil

107 + }

108 +

109 + expr := strings.TrimSpace(strings.Join(patterns, " "))

110 + if expr == "" {

111 + return nil, nil

112 + }

113 +

114 + return matcher.NewSimplePatternsMatcher(expr)

115 +}

116 +

117 +func (c *Collector) warnOnce(key string, format string, args ...interface{}) {

118 + c.warnMu.Lock()

119 + defer c.warnMu.Unlock()

120 +

121 + if c.warns == nil {

122 + c.warns = make(map[string]time.Time)

123 + }

124 +

125 + now := time.Now()

126 + if last, ok := c.warns[key]; ok && now.Sub(last) < warnThrottleInterval {

127 + return

128 + }

129 +

130 + c.Warningf(format, args...)

131 + c.warns[key] = now

132 +}

133 +

134 +func (c *Collector) clearWarnOnce(key string) {

135 + c.warnMu.Lock()

136 + defer c.warnMu.Unlock()

137 +

138 + if c.warns == nil {

139 + return

140 + }

141 + delete(c.warns, key)

142 }

143

144 func (c *Collector) initOnce() {

@@ -122,6 +174,73 @@ func (c *Collector) resetCaches() {

174 c.prefetchers = make(map[string]*prefetcherInstanceMetrics)

175 }

176

177 +func (c *Collector) matchIncludeExclude(include, exclude matcher.Matcher, values ...string) bool {

178 + matched := false

179 + matchedMap := make(map[string]bool)

180 +

181 + if include == nil {

182 + matched = true

183 + } else {

184 + for _, v := range values {

185 + if v == "" {

186 + continue

187 + }

188 + if include.MatchString(v) {

189 + matched = true

190 + matchedMap[v] = true

191 + }

192 + }

193 + }

194 +

195 + if !matched {

196 + return false

197 + }

198 +

199 + if exclude != nil {

200 + for _, v := range values {

201 + if v == "" {

202 + continue

203 + }

204 + if exclude.MatchString(v) {

205 + if include != nil && matchedMap[v] {

206 + continue

207 + }

208 + return false

209 + }

210 + }

211 + }

212 +

213 + return true

214 +}

215 +

216 +func (c *Collector) allowConnection(id string, meta *connectionMetrics) bool {

217 + appName := ""

218 + host := ""

219 + ip := ""

220 + if meta != nil {

221 + appName = meta.applicationName

222 + host = meta.clientHostname

223 + ip = meta.clientIP

224 + }

225 + return c.matchIncludeExclude(c.connectionInclude, c.connectionExclude, id, appName, host, ip)

226 +}

227 +

228 +func (c *Collector) allowBufferpool(name string) bool {

229 + return c.matchIncludeExclude(c.bufferpoolInclude, c.bufferpoolExclude, name)

230 +}

231 +

232 +func (c *Collector) allowTablespace(name, contentType, state string) bool {

233 + return c.matchIncludeExclude(c.tablespaceInclude, c.tablespaceExclude, name, contentType, state)

234 +}

235 +

236 +func (c *Collector) allowTable(key string) bool {

237 + return c.matchIncludeExclude(c.tableInclude, c.tableExclude, key)

238 +}

239 +

240 +func (c *Collector) allowIndex(key string) bool {

241 + return c.matchIncludeExclude(c.indexInclude, c.indexExclude, key)

242 +}

243 +

244 // CollectOnce implements framework.CollectorImpl.

245 func (c *Collector) CollectOnce() error {

246 c.initOnce()

src/go/plugin/ibm.d/modules/db2/config.go

+24 -14

@@ -78,18 +78,28 @@ type Config struct {

78 // CollectDatabasesMatching filters databases by name using glob patterns.

79 CollectDatabasesMatching string `yaml:"collect_databases_matching,omitempty" json:"collect_databases_matching" ui:"group:Databases"`

81 - // CollectBufferpoolsMatching filters buffer pools by name using glob patterns.

82 - CollectBufferpoolsMatching string `yaml:"collect_bufferpools_matching,omitempty" json:"collect_bufferpools_matching" ui:"group:Buffer Pools"`

83 -

84 - // CollectTablespacesMatching filters tablespaces by name using glob patterns.

85 - CollectTablespacesMatching string `yaml:"collect_tablespaces_matching,omitempty" json:"collect_tablespaces_matching" ui:"group:Tablespaces"`

86 -

87 - // CollectConnectionsMatching filters monitored connections by application ID.

88 - CollectConnectionsMatching string `yaml:"collect_connections_matching,omitempty" json:"collect_connections_matching" ui:"group:Connections Monitoring"`

89 -

90 - // CollectTablesMatching filters tables by schema/name.

91 - CollectTablesMatching string `yaml:"collect_tables_matching,omitempty" json:"collect_tables_matching" ui:"group:Tables"`

92 -

93 - // CollectIndexesMatching filters indexes by schema/name.

94 - CollectIndexesMatching string `yaml:"collect_indexes_matching,omitempty" json:"collect_indexes_matching" ui:"group:Indexes"`

81 + // IncludeConnections filters monitored connections by application ID or application name (wildcards supported).

82 + IncludeConnections []string `yaml:"include_connections,omitempty" json:"include_connections" ui:"group:Connections Monitoring"`

83 + // ExcludeConnections excludes connections after inclusion matching.

84 + ExcludeConnections []string `yaml:"exclude_connections,omitempty" json:"exclude_connections" ui:"group:Connections Monitoring"`

85 +

86 + // IncludeBufferpools filters buffer pools by name.

87 + IncludeBufferpools []string `yaml:"include_bufferpools,omitempty" json:"include_bufferpools" ui:"group:Buffer Pools"`

88 + // ExcludeBufferpools excludes buffer pools after inclusion.

89 + ExcludeBufferpools []string `yaml:"exclude_bufferpools,omitempty" json:"exclude_bufferpools" ui:"group:Buffer Pools"`

90 +

91 + // IncludeTablespaces filters tablespaces by name.

92 + IncludeTablespaces []string `yaml:"include_tablespaces,omitempty" json:"include_tablespaces" ui:"group:Tablespaces"`

93 + // ExcludeTablespaces excludes tablespaces after inclusion.

94 + ExcludeTablespaces []string `yaml:"exclude_tablespaces,omitempty" json:"exclude_tablespaces" ui:"group:Tablespaces"`

95 +

96 + // IncludeTables filters tables by schema/name.

97 + IncludeTables []string `yaml:"include_tables,omitempty" json:"include_tables" ui:"group:Tables"`

98 + // ExcludeTables excludes tables after inclusion.

99 + ExcludeTables []string `yaml:"exclude_tables,omitempty" json:"exclude_tables" ui:"group:Tables"`

100 +

101 + // IncludeIndexes filters indexes by schema/name.

102 + IncludeIndexes []string `yaml:"include_indexes,omitempty" json:"include_indexes" ui:"group:Indexes"`

103 + // ExcludeIndexes excludes indexes after inclusion.

104 + ExcludeIndexes []string `yaml:"exclude_indexes,omitempty" json:"exclude_indexes" ui:"group:Indexes"`

105 }

src/go/plugin/ibm.d/modules/db2/config_schema.json

+152 -39

@@ -19,12 +19,6 @@

19 "title": "Collect Bufferpool Metrics",

20 "type": "string"

21 },

22 - "collect_bufferpools_matching": {

23 - "default": "",

24 - "description": "CollectBufferpoolsMatching filters buffer pools by name using glob patterns.",

25 - "title": "Collect Bufferpools Matching",

26 - "type": "string"

27 - },

22 "collect_connection_metrics": {

23 "default": "auto",

24 "description": "CollectConnectionMetrics toggles per-connection activity metrics.",

@@ -36,12 +30,6 @@

30 "title": "Collect Connection Metrics",

31 "type": "string"

32 },

39 - "collect_connections_matching": {

40 - "default": "",

41 - "description": "CollectConnectionsMatching filters monitored connections by application ID.",

42 - "title": "Collect Connections Matching",

43 - "type": "string"

44 - },

33 "collect_database_metrics": {

34 "default": "auto",

35 "description": "CollectDatabaseMetrics toggles high-level database status metrics.",

@@ -70,12 +58,6 @@

58 "title": "Collect Index Metrics",

59 "type": "string"

60 },

73 - "collect_indexes_matching": {

74 - "default": "",

75 - "description": "CollectIndexesMatching filters indexes by schema/name.",

76 - "title": "Collect Indexes Matching",

77 - "type": "string"

78 - },

61 "collect_lock_metrics": {

62 "default": "auto",

63 "description": "CollectLockMetrics toggles lock contention metrics.",

@@ -110,12 +92,6 @@

92 "title": "Collect Table Metrics",

93 "type": "string"

94 },

113 - "collect_tables_matching": {

114 - "default": "",

115 - "description": "CollectTablesMatching filters tables by schema/name.",

116 - "title": "Collect Tables Matching",

117 - "type": "string"

118 - },

95 "collect_tablespace_metrics": {

96 "default": "auto",

97 "description": "CollectTablespaceMetrics toggles tablespace capacity metrics.",

@@ -127,12 +103,6 @@

103 "title": "Collect Tablespace Metrics",

104 "type": "string"

105 },

130 - "collect_tablespaces_matching": {

131 - "default": "",

132 - "description": "CollectTablespacesMatching filters tablespaces by name using glob patterns.",

133 - "title": "Collect Tablespaces Matching",

134 - "type": "string"

135 - },

106 "collect_wait_metrics": {

107 "default": true,

108 "description": "CollectWaitMetrics enables wait time statistics (locks, logs, I/O).",

@@ -145,6 +115,114 @@

115 "title": "DSN",

116 "type": "string"

117 },

118 + "exclude_bufferpools": {

119 + "default": "nil",

120 + "description": "ExcludeBufferpools excludes buffer pools after inclusion.",

121 + "items": {

122 + "type": "string"

123 + },

124 + "title": "Exclude Bufferpools",

125 + "type": "array"

126 + },

127 + "exclude_connections": {

128 + "default": [

129 + "*TEMP*"

130 + ],

131 + "description": "ExcludeConnections excludes connections after inclusion matching.",

132 + "items": {

133 + "type": "string"

134 + },

135 + "title": "Exclude Connections",

136 + "type": "array"

137 + },

138 + "exclude_indexes": {

139 + "default": "nil",

140 + "description": "ExcludeIndexes excludes indexes after inclusion.",

141 + "items": {

142 + "type": "string"

143 + },

144 + "title": "Exclude Indexes",

145 + "type": "array"

146 + },

147 + "exclude_tables": {

148 + "default": "nil",

149 + "description": "ExcludeTables excludes tables after inclusion.",

150 + "items": {

151 + "type": "string"

152 + },

153 + "title": "Exclude Tables",

154 + "type": "array"

155 + },

156 + "exclude_tablespaces": {

157 + "default": [

158 + "TEMPSPACE2"

159 + ],

160 + "description": "ExcludeTablespaces excludes tablespaces after inclusion.",

161 + "items": {

162 + "type": "string"

163 + },

164 + "title": "Exclude Tablespaces",

165 + "type": "array"

166 + },

167 + "include_bufferpools": {

168 + "default": [

169 + "IBMDEFAULTBP",

170 + "IBMSYSTEMBP*",

171 + "IBMHADRBP*"

172 + ],

173 + "description": "IncludeBufferpools filters buffer pools by name.",

174 + "items": {

175 + "type": "string"

176 + },

177 + "title": "Include Bufferpools",

178 + "type": "array"

179 + },

180 + "include_connections": {

181 + "default": [

182 + "db2sysc*",

183 + "db2agent*",

184 + "db2hadr*",

185 + "db2acd*",

186 + "db2bmgr*"

187 + ],

188 + "description": "IncludeConnections filters monitored connections by application ID or application name (wildcards supported).",

189 + "items": {

190 + "type": "string"

191 + },

192 + "title": "Include Connections",

193 + "type": "array"

194 + },

195 + "include_indexes": {

196 + "default": "nil",

197 + "description": "IncludeIndexes filters indexes by schema/name.",

198 + "items": {

199 + "type": "string"

200 + },

201 + "title": "Include Indexes",

202 + "type": "array"

203 + },

204 + "include_tables": {

205 + "default": "nil",

206 + "description": "IncludeTables filters tables by schema/name.",

207 + "items": {

208 + "type": "string"

209 + },

210 + "title": "Include Tables",

211 + "type": "array"

212 + },

213 + "include_tablespaces": {

214 + "default": [

215 + "SYSCATSPACE",

216 + "TEMPSPACE*",

217 + "SYSTOOLSPACE"

218 + ],

219 + "description": "IncludeTablespaces filters tablespaces by name.",

220 + "items": {

221 + "type": "string"

222 + },

223 + "title": "Include Tablespaces",

224 + "type": "array"

225 + },

226 "max_bufferpools": {

227 "default": 20,

228 "description": "MaxBufferpools caps the number of buffer pools charted.",

@@ -152,7 +230,7 @@

230 "type": "integer"

231 },

232 "max_connections": {

155 - "default": 200,

233 + "default": 50,

234 "description": "MaxConnections caps the number of connection instances charted.",

235 "title": "Max Connections",

236 "type": "integer"

@@ -176,19 +254,19 @@

254 "type": "integer"

255 },

256 "max_indexes": {

179 - "default": 100,

257 + "default": 50,

258 "description": "MaxIndexes caps the number of indexes charted.",

259 "title": "Max Indexes",

260 "type": "integer"

261 },

262 "max_tables": {

185 - "default": 50,

263 + "default": 25,

264 "description": "MaxTables caps the number of tables charted.",

265 "title": "Max Tables",

266 "type": "integer"

267 },

268 "max_tablespaces": {

191 - "default": 100,

269 + "default": 50,

270 "description": "MaxTablespaces caps the number of tablespaces charted.",

271 "title": "Max Tablespaces",

272 "type": "integer"

@@ -217,6 +295,36 @@

295 "type": "object"

296 },

297 "uiSchema": {

298 + "exclude_bufferpools": {

299 + "ui:listFlavour": "list"

300 + },

301 + "exclude_connections": {

302 + "ui:listFlavour": "list"

303 + },

304 + "exclude_indexes": {

305 + "ui:listFlavour": "list"

306 + },

307 + "exclude_tables": {

308 + "ui:listFlavour": "list"

309 + },

310 + "exclude_tablespaces": {

311 + "ui:listFlavour": "list"

312 + },

313 + "include_bufferpools": {

314 + "ui:listFlavour": "list"

315 + },

316 + "include_connections": {

317 + "ui:listFlavour": "list"

318 + },

319 + "include_indexes": {

320 + "ui:listFlavour": "list"

321 + },

322 + "include_tables": {

323 + "ui:listFlavour": "list"

324 + },

325 + "include_tablespaces": {

326 + "ui:listFlavour": "list"

327 + },

328 "ui:flavour": "tabs",

329 "ui:options": {

330 "tabs": [

@@ -249,7 +357,8 @@

357 "fields": [

358 "collect_bufferpool_metrics",

359 "max_bufferpools",

252 - "collect_bufferpools_matching"

360 + "include_bufferpools",

361 + "exclude_bufferpools"

362 ],

363 "title": "Buffer Pools"

364 },

@@ -257,7 +366,8 @@

366 "fields": [

367 "collect_tablespace_metrics",

368 "max_tablespaces",

260 - "collect_tablespaces_matching"

369 + "include_tablespaces",

370 + "exclude_tablespaces"

371 ],

372 "title": "Tablespaces"

373 },

@@ -265,7 +375,8 @@

375 "fields": [

376 "collect_connection_metrics",

377 "max_connections",

268 - "collect_connections_matching"

378 + "include_connections",

379 + "exclude_connections"

380 ],

381 "title": "Connections Monitoring"

382 },

@@ -282,7 +393,8 @@

393 "fields": [

394 "collect_table_metrics",

395 "max_tables",

285 - "collect_tables_matching"

396 + "include_tables",

397 + "exclude_tables"

398 ],

399 "title": "Tables"

400 },

@@ -290,7 +402,8 @@

402 "fields": [

403 "collect_index_metrics",

404 "max_indexes",

293 - "collect_indexes_matching"

405 + "include_indexes",

406 + "exclude_indexes"

407 ],

408 "title": "Indexes"

409 }

src/go/plugin/ibm.d/modules/db2/contexts/contexts.yaml

+222

@@ -496,6 +496,81 @@ Bufferpool:

496 dimensions:

497 - {name: writes, algo: incremental}

498

499 +BufferpoolGroup:

500 + labels:

501 + - group

502 + contexts:

503 + - name: HitRatio

504 + context: db2.bufferpool_group_hit_ratio

505 + title: Buffer Pool Group Hit Ratio

506 + family: bufferpools/groups

507 + units: percentage

508 + type: line

509 + priority: 1117

510 + dimensions:

511 + - {name: overall, algo: absolute, precision: 1000, div: 1000}

512 + - name: DetailedHitRatio

513 + context: db2.bufferpool_group_detailed_hit_ratio

514 + title: Buffer Pool Group Detailed Hit Ratios

515 + family: bufferpools/groups

516 + units: percentage

517 + type: line

518 + priority: 1118

519 + dimensions:

520 + - {name: data, algo: absolute, precision: 1000, div: 1000}

521 + - {name: index, algo: absolute, precision: 1000, div: 1000}

522 + - {name: xda, algo: absolute, precision: 1000, div: 1000}

523 + - {name: column, algo: absolute, precision: 1000, div: 1000}

524 + - name: Reads

525 + context: db2.bufferpool_group_reads

526 + title: Buffer Pool Group Reads

527 + family: bufferpools/groups

528 + units: reads/s

529 + type: stacked

530 + priority: 1119

531 + dimensions:

532 + - {name: logical, algo: incremental}

533 + - {name: physical, algo: incremental}

534 + - name: DataReads

535 + context: db2.bufferpool_group_data_reads

536 + title: Buffer Pool Group Data Reads

537 + family: bufferpools/groups

538 + units: reads/s

539 + type: stacked

540 + priority: 1120

541 + dimensions:

542 + - {name: logical, algo: incremental}

543 + - {name: physical, algo: incremental}

544 + - name: IndexReads

545 + context: db2.bufferpool_group_index_reads

546 + title: Buffer Pool Group Index Reads

547 + family: bufferpools/groups

548 + units: reads/s

549 + type: stacked

550 + priority: 1121

551 + dimensions:

552 + - {name: logical, algo: incremental}

553 + - {name: physical, algo: incremental}

554 + - name: Pages

555 + context: db2.bufferpool_group_pages

556 + title: Buffer Pool Group Pages

557 + family: bufferpools/groups

558 + units: pages

559 + type: stacked

560 + priority: 1122

561 + dimensions:

562 + - {name: used, algo: absolute}

563 + - {name: total, algo: absolute}

564 + - name: Writes

565 + context: db2.bufferpool_group_writes

566 + title: Buffer Pool Group Writes

567 + family: bufferpools/groups

568 + units: writes/s

569 + type: line

570 + priority: 1123

571 + dimensions:

572 + - {name: writes, algo: incremental}

573 +

574 Tablespace:

575 labels:

576 - tablespace

@@ -542,6 +617,49 @@ Tablespace:

617 dimensions:

618 - {name: state, algo: absolute}

619

620 +TablespaceGroup:

621 + labels:

622 + - group

623 + contexts:

624 + - name: Usage

625 + context: db2.tablespace_group_usage

626 + title: Tablespace Group Usage

627 + family: tablespaces/groups

628 + units: percentage

629 + type: line

630 + priority: 1124

631 + dimensions:

632 + - {name: used, algo: absolute, precision: 1000, div: 1000}

633 + - name: Size

634 + context: db2.tablespace_group_size

635 + title: Tablespace Group Size

636 + family: tablespaces/groups

637 + units: bytes

638 + type: stacked

639 + priority: 1125

640 + dimensions:

641 + - {name: used, algo: absolute}

642 + - {name: free, algo: absolute}

643 + - name: UsableSize

644 + context: db2.tablespace_group_usable_size

645 + title: Tablespace Group Usable Size

646 + family: tablespaces/groups

647 + units: bytes

648 + type: line

649 + priority: 1126

650 + dimensions:

651 + - {name: total, algo: absolute}

652 + - {name: usable, algo: absolute}

653 + - name: State

654 + context: db2.tablespace_group_state

655 + title: Tablespace Group State

656 + family: tablespaces/groups

657 + units: state

658 + type: line

659 + priority: 1127

660 + dimensions:

661 + - {name: state, algo: absolute}

662 +

663 Connection:

664 labels:

665 - application_id

@@ -601,6 +719,69 @@ Connection:

719 - {name: commit, algo: incremental}

720 - {name: rollback, algo: incremental}

721

722 +ConnectionGroup:

723 + labels:

724 + - group

725 + contexts:

726 + - name: Count

727 + context: db2.connection_group.count

728 + title: Connection Group Count

729 + family: connections/groups

730 + units: connections

731 + type: line

732 + priority: 1134

733 + dimensions:

734 + - {name: count, algo: absolute}

735 + - name: State

736 + context: db2.connection_group.state

737 + title: Connection Group State Sum

738 + family: connections/groups

739 + units: state

740 + type: line

741 + priority: 1135

742 + dimensions:

743 + - {name: state, algo: absolute}

744 + - name: Activity

745 + context: db2.connection_group.activity

746 + title: Connection Group Row Activity

747 + family: connections/groups

748 + units: rows/s

749 + type: area

750 + priority: 1136

751 + dimensions:

752 + - {name: read, algo: incremental}

753 + - {name: written, algo: incremental}

754 + - name: WaitTime

755 + context: db2.connection_group.wait_time

756 + title: Connection Group Wait Time

757 + family: connections/groups

758 + units: milliseconds

759 + type: stacked

760 + priority: 1137

761 + dimensions:

762 + - {name: lock, algo: incremental}

763 + - {name: log_disk, algo: incremental}

764 + - {name: log_buffer, algo: incremental}

765 + - {name: pool_read, algo: incremental}

766 + - {name: pool_write, algo: incremental}

767 + - {name: direct_read, algo: incremental}

768 + - {name: direct_write, algo: incremental}

769 + - {name: fcm_recv, algo: incremental}

770 + - {name: fcm_send, algo: incremental}

771 + - name: ProcessingTime

772 + context: db2.connection_group.processing_time

773 + title: Connection Group Processing Time

774 + family: connections/groups

775 + units: milliseconds

776 + type: stacked

777 + priority: 1138

778 + dimensions:

779 + - {name: routine, algo: incremental}

780 + - {name: compile, algo: incremental}

781 + - {name: section, algo: incremental}

782 + - {name: commit, algo: incremental}

783 + - {name: rollback, algo: incremental}

784 +

785 Table:

786 labels:

787 - table

@@ -627,6 +808,32 @@ Table:

808 - {name: read, algo: incremental}

809 - {name: written, algo: incremental}

810

811 +TableGroup:

812 + labels:

813 + - group

814 + contexts:

815 + - name: Size

816 + context: db2.table_group_size

817 + title: Table Group Size

818 + family: tables/groups

819 + units: bytes

820 + type: stacked

821 + priority: 1142

822 + dimensions:

823 + - {name: data, algo: absolute}

824 + - {name: index, algo: absolute}

825 + - {name: long_obj, algo: absolute}

826 + - name: Activity

827 + context: db2.table_group_activity

828 + title: Table Group Activity

829 + family: tables/groups

830 + units: rows/s

831 + type: area

832 + priority: 1143

833 + dimensions:

834 + - {name: read, algo: incremental}

835 + - {name: written, algo: incremental}

836 +

837 Index:

838 labels:

839 - index

@@ -642,6 +849,21 @@ Index:

849 - {name: index, algo: incremental}

850 - {name: full, algo: incremental}

851

852 +IndexGroup:

853 + labels:

854 + - group

855 + contexts:

856 + - name: Usage

857 + context: db2.index_group_usage

858 + title: Index Group Usage

859 + family: indexes/groups

860 + units: scans/s

861 + type: area

862 + priority: 1151

863 + dimensions:

864 + - {name: index, algo: incremental}

865 + - {name: full, algo: incremental}

866 +

867 MemoryPool:

868 labels:

869 - pool_type

src/go/plugin/ibm.d/modules/db2/contexts/zz_generated_contexts.go

+1329 -112

@@ -445,43 +445,812 @@ var Bufferpool = struct {

445 },

446 }

447

448 +// --- BufferpoolGroup ---

449 +

450 +// BufferpoolGroupHitRatioValues defines the type-safe values for BufferpoolGroup.HitRatio context

451 +type BufferpoolGroupHitRatioValues struct {

452 + Overall int64

453 +}

454 +

455 +// BufferpoolGroupHitRatioContext provides type-safe operations for BufferpoolGroup.HitRatio context

456 +type BufferpoolGroupHitRatioContext struct {

457 + framework.Context[BufferpoolGroupLabels]

458 +}

459 +

460 +// Set provides type-safe dimension setting for BufferpoolGroup.HitRatio context

461 +func (c BufferpoolGroupHitRatioContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupHitRatioValues) {

462 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

463 + "overall": values.Overall,

464 + })

465 +}

466 +

467 +// SetUpdateEvery sets the update interval for this instance

468 +func (c BufferpoolGroupHitRatioContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

469 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

470 +}

471 +

472 +// BufferpoolGroupDetailedHitRatioValues defines the type-safe values for BufferpoolGroup.DetailedHitRatio context

473 +type BufferpoolGroupDetailedHitRatioValues struct {

474 + Data int64

475 + Index int64

476 + Xda int64

477 + Column int64

478 +}

479 +

480 +// BufferpoolGroupDetailedHitRatioContext provides type-safe operations for BufferpoolGroup.DetailedHitRatio context

481 +type BufferpoolGroupDetailedHitRatioContext struct {

482 + framework.Context[BufferpoolGroupLabels]

483 +}

484 +

485 +// Set provides type-safe dimension setting for BufferpoolGroup.DetailedHitRatio context

486 +func (c BufferpoolGroupDetailedHitRatioContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupDetailedHitRatioValues) {

487 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

488 + "data": values.Data,

489 + "index": values.Index,

490 + "xda": values.Xda,

491 + "column": values.Column,

492 + })

493 +}

494 +

495 +// SetUpdateEvery sets the update interval for this instance

496 +func (c BufferpoolGroupDetailedHitRatioContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

497 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

498 +}

499 +

500 +// BufferpoolGroupReadsValues defines the type-safe values for BufferpoolGroup.Reads context

501 +type BufferpoolGroupReadsValues struct {

502 + Logical int64

503 + Physical int64

504 +}

505 +

506 +// BufferpoolGroupReadsContext provides type-safe operations for BufferpoolGroup.Reads context

507 +type BufferpoolGroupReadsContext struct {

508 + framework.Context[BufferpoolGroupLabels]

509 +}

510 +

511 +// Set provides type-safe dimension setting for BufferpoolGroup.Reads context

512 +func (c BufferpoolGroupReadsContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupReadsValues) {

513 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

514 + "logical": values.Logical,

515 + "physical": values.Physical,

516 + })

517 +}

518 +

519 +// SetUpdateEvery sets the update interval for this instance

520 +func (c BufferpoolGroupReadsContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

521 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

522 +}

523 +

524 +// BufferpoolGroupDataReadsValues defines the type-safe values for BufferpoolGroup.DataReads context

525 +type BufferpoolGroupDataReadsValues struct {

526 + Logical int64

527 + Physical int64

528 +}

529 +

530 +// BufferpoolGroupDataReadsContext provides type-safe operations for BufferpoolGroup.DataReads context

531 +type BufferpoolGroupDataReadsContext struct {

532 + framework.Context[BufferpoolGroupLabels]

533 +}

534 +

535 +// Set provides type-safe dimension setting for BufferpoolGroup.DataReads context

536 +func (c BufferpoolGroupDataReadsContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupDataReadsValues) {

537 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

538 + "logical": values.Logical,

539 + "physical": values.Physical,

540 + })

541 +}

542 +

543 +// SetUpdateEvery sets the update interval for this instance

544 +func (c BufferpoolGroupDataReadsContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

545 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

546 +}

547 +

548 +// BufferpoolGroupIndexReadsValues defines the type-safe values for BufferpoolGroup.IndexReads context

549 +type BufferpoolGroupIndexReadsValues struct {

550 + Logical int64

551 + Physical int64

552 +}

553 +

554 +// BufferpoolGroupIndexReadsContext provides type-safe operations for BufferpoolGroup.IndexReads context

555 +type BufferpoolGroupIndexReadsContext struct {

556 + framework.Context[BufferpoolGroupLabels]

557 +}

558 +

559 +// Set provides type-safe dimension setting for BufferpoolGroup.IndexReads context

560 +func (c BufferpoolGroupIndexReadsContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupIndexReadsValues) {

561 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

562 + "logical": values.Logical,

563 + "physical": values.Physical,

564 + })

565 +}

566 +

567 +// SetUpdateEvery sets the update interval for this instance

568 +func (c BufferpoolGroupIndexReadsContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

569 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

570 +}

571 +

572 +// BufferpoolGroupPagesValues defines the type-safe values for BufferpoolGroup.Pages context

573 +type BufferpoolGroupPagesValues struct {

574 + Used int64

575 + Total int64

576 +}

577 +

578 +// BufferpoolGroupPagesContext provides type-safe operations for BufferpoolGroup.Pages context

579 +type BufferpoolGroupPagesContext struct {

580 + framework.Context[BufferpoolGroupLabels]

581 +}

582 +

583 +// Set provides type-safe dimension setting for BufferpoolGroup.Pages context

584 +func (c BufferpoolGroupPagesContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupPagesValues) {

585 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

586 + "used": values.Used,

587 + "total": values.Total,

588 + })

589 +}

590 +

591 +// SetUpdateEvery sets the update interval for this instance

592 +func (c BufferpoolGroupPagesContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

593 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

594 +}

595 +

596 +// BufferpoolGroupWritesValues defines the type-safe values for BufferpoolGroup.Writes context

597 +type BufferpoolGroupWritesValues struct {

598 + Writes int64

599 +}

600 +

601 +// BufferpoolGroupWritesContext provides type-safe operations for BufferpoolGroup.Writes context

602 +type BufferpoolGroupWritesContext struct {

603 + framework.Context[BufferpoolGroupLabels]

604 +}

605 +

606 +// Set provides type-safe dimension setting for BufferpoolGroup.Writes context

607 +func (c BufferpoolGroupWritesContext) Set(state *framework.CollectorState, labels BufferpoolGroupLabels, values BufferpoolGroupWritesValues) {

608 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

609 + "writes": values.Writes,

610 + })

611 +}

612 +

613 +// SetUpdateEvery sets the update interval for this instance

614 +func (c BufferpoolGroupWritesContext) SetUpdateEvery(state *framework.CollectorState, labels BufferpoolGroupLabels, updateEvery int) {

615 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

616 +}

617 +

618 +// BufferpoolGroupLabels defines the required labels for BufferpoolGroup contexts

619 +type BufferpoolGroupLabels struct {

620 + Group string

621 +}

622 +

623 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

624 +func (l BufferpoolGroupLabels) InstanceID(contextName string) string {

625 + // Label order from YAML: group

626 + return contextName + "." + cleanLabelValue(l.Group)

627 +}

628 +

629 +// BufferpoolGroup contains all metric contexts for BufferpoolGroup

630 +var BufferpoolGroup = struct {

631 + HitRatio BufferpoolGroupHitRatioContext

632 + DetailedHitRatio BufferpoolGroupDetailedHitRatioContext

633 + Reads BufferpoolGroupReadsContext

634 + DataReads BufferpoolGroupDataReadsContext

635 + IndexReads BufferpoolGroupIndexReadsContext

636 + Pages BufferpoolGroupPagesContext

637 + Writes BufferpoolGroupWritesContext

638 +}{

639 + HitRatio: BufferpoolGroupHitRatioContext{

640 + Context: framework.Context[BufferpoolGroupLabels]{

641 + Name: "db2.bufferpool_group_hit_ratio",

642 + Family: "bufferpools/groups",

643 + Title: "Buffer Pool Group Hit Ratio",

644 + Units: "percentage",

645 + Type: module.Line,

646 + Priority: 1117,

647 + UpdateEvery: 1,

648 + Dimensions: []framework.Dimension{

649 + {

650 + Name: "overall",

651 + Algorithm: module.Absolute,

652 + Mul: 1,

653 + Div: 1000,

654 + Precision: 1000,

655 + },

656 + },

657 + LabelKeys: []string{

658 + "group",

659 + },

660 + },

661 + },

662 + DetailedHitRatio: BufferpoolGroupDetailedHitRatioContext{

663 + Context: framework.Context[BufferpoolGroupLabels]{

664 + Name: "db2.bufferpool_group_detailed_hit_ratio",

665 + Family: "bufferpools/groups",

666 + Title: "Buffer Pool Group Detailed Hit Ratios",

667 + Units: "percentage",

668 + Type: module.Line,

669 + Priority: 1118,

670 + UpdateEvery: 1,

671 + Dimensions: []framework.Dimension{

672 + {

673 + Name: "data",

674 + Algorithm: module.Absolute,

675 + Mul: 1,

676 + Div: 1000,

677 + Precision: 1000,

678 + },

679 + {

680 + Name: "index",

681 + Algorithm: module.Absolute,

682 + Mul: 1,

683 + Div: 1000,

684 + Precision: 1000,

685 + },

686 + {

687 + Name: "xda",

688 + Algorithm: module.Absolute,

689 + Mul: 1,

690 + Div: 1000,

691 + Precision: 1000,

692 + },

693 + {

694 + Name: "column",

695 + Algorithm: module.Absolute,

696 + Mul: 1,

697 + Div: 1000,

698 + Precision: 1000,

699 + },

700 + },

701 + LabelKeys: []string{

702 + "group",

703 + },

704 + },

705 + },

706 + Reads: BufferpoolGroupReadsContext{

707 + Context: framework.Context[BufferpoolGroupLabels]{

708 + Name: "db2.bufferpool_group_reads",

709 + Family: "bufferpools/groups",

710 + Title: "Buffer Pool Group Reads",

711 + Units: "reads/s",

712 + Type: module.Stacked,

713 + Priority: 1119,

714 + UpdateEvery: 1,

715 + Dimensions: []framework.Dimension{

716 + {

717 + Name: "logical",

718 + Algorithm: module.Incremental,

719 + Mul: 1,

720 + Div: 1,

721 + Precision: 1,

722 + },

723 + {

724 + Name: "physical",

725 + Algorithm: module.Incremental,

726 + Mul: 1,

727 + Div: 1,

728 + Precision: 1,

729 + },

730 + },

731 + LabelKeys: []string{

732 + "group",

733 + },

734 + },

735 + },

736 + DataReads: BufferpoolGroupDataReadsContext{

737 + Context: framework.Context[BufferpoolGroupLabels]{

738 + Name: "db2.bufferpool_group_data_reads",

739 + Family: "bufferpools/groups",

740 + Title: "Buffer Pool Group Data Reads",

741 + Units: "reads/s",

742 + Type: module.Stacked,

743 + Priority: 1120,

744 + UpdateEvery: 1,

745 + Dimensions: []framework.Dimension{

746 + {

747 + Name: "logical",

748 + Algorithm: module.Incremental,

749 + Mul: 1,

750 + Div: 1,

751 + Precision: 1,

752 + },

753 + {

754 + Name: "physical",

755 + Algorithm: module.Incremental,

756 + Mul: 1,

757 + Div: 1,

758 + Precision: 1,

759 + },

760 + },

761 + LabelKeys: []string{

762 + "group",

763 + },

764 + },

765 + },

766 + IndexReads: BufferpoolGroupIndexReadsContext{

767 + Context: framework.Context[BufferpoolGroupLabels]{

768 + Name: "db2.bufferpool_group_index_reads",

769 + Family: "bufferpools/groups",

770 + Title: "Buffer Pool Group Index Reads",

771 + Units: "reads/s",

772 + Type: module.Stacked,

773 + Priority: 1121,

774 + UpdateEvery: 1,

775 + Dimensions: []framework.Dimension{

776 + {

777 + Name: "logical",

778 + Algorithm: module.Incremental,

779 + Mul: 1,

780 + Div: 1,

781 + Precision: 1,

782 + },

783 + {

784 + Name: "physical",

785 + Algorithm: module.Incremental,

786 + Mul: 1,

787 + Div: 1,

788 + Precision: 1,

789 + },

790 + },

791 + LabelKeys: []string{

792 + "group",

793 + },

794 + },

795 + },

796 + Pages: BufferpoolGroupPagesContext{

797 + Context: framework.Context[BufferpoolGroupLabels]{

798 + Name: "db2.bufferpool_group_pages",

799 + Family: "bufferpools/groups",

800 + Title: "Buffer Pool Group Pages",

801 + Units: "pages",

802 + Type: module.Stacked,

803 + Priority: 1122,

804 + UpdateEvery: 1,

805 + Dimensions: []framework.Dimension{

806 + {

807 + Name: "used",

808 + Algorithm: module.Absolute,

809 + Mul: 1,

810 + Div: 1,

811 + Precision: 1,

812 + },

813 + {

814 + Name: "total",

815 + Algorithm: module.Absolute,

816 + Mul: 1,

817 + Div: 1,

818 + Precision: 1,

819 + },

820 + },

821 + LabelKeys: []string{

822 + "group",

823 + },

824 + },

825 + },

826 + Writes: BufferpoolGroupWritesContext{

827 + Context: framework.Context[BufferpoolGroupLabels]{

828 + Name: "db2.bufferpool_group_writes",

829 + Family: "bufferpools/groups",

830 + Title: "Buffer Pool Group Writes",

831 + Units: "writes/s",

832 + Type: module.Line,

833 + Priority: 1123,

834 + UpdateEvery: 1,

835 + Dimensions: []framework.Dimension{

836 + {

837 + Name: "writes",

838 + Algorithm: module.Incremental,

839 + Mul: 1,

840 + Div: 1,

841 + Precision: 1,

842 + },

843 + },

844 + LabelKeys: []string{

845 + "group",

846 + },

847 + },

848 + },

849 +}

850 +

851 // --- Connection ---

852

450 -// ConnectionStateValues defines the type-safe values for Connection.State context

451 -type ConnectionStateValues struct {

853 +// ConnectionStateValues defines the type-safe values for Connection.State context

854 +type ConnectionStateValues struct {

855 + State int64

856 +}

857 +

858 +// ConnectionStateContext provides type-safe operations for Connection.State context

859 +type ConnectionStateContext struct {

860 + framework.Context[ConnectionLabels]

861 +}

862 +

863 +// Set provides type-safe dimension setting for Connection.State context

864 +func (c ConnectionStateContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionStateValues) {

865 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

866 + "state": values.State,

867 + })

868 +}

869 +

870 +// SetUpdateEvery sets the update interval for this instance

871 +func (c ConnectionStateContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

872 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

873 +}

874 +

875 +// ConnectionActivityValues defines the type-safe values for Connection.Activity context

876 +type ConnectionActivityValues struct {

877 + Read int64

878 + Written int64

879 +}

880 +

881 +// ConnectionActivityContext provides type-safe operations for Connection.Activity context

882 +type ConnectionActivityContext struct {

883 + framework.Context[ConnectionLabels]

884 +}

885 +

886 +// Set provides type-safe dimension setting for Connection.Activity context

887 +func (c ConnectionActivityContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionActivityValues) {

888 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

889 + "read": values.Read,

890 + "written": values.Written,

891 + })

892 +}

893 +

894 +// SetUpdateEvery sets the update interval for this instance

895 +func (c ConnectionActivityContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

896 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

897 +}

898 +

899 +// ConnectionWaitTimeValues defines the type-safe values for Connection.WaitTime context

900 +type ConnectionWaitTimeValues struct {

901 + Lock int64

902 + Log_disk int64

903 + Log_buffer int64

904 + Pool_read int64

905 + Pool_write int64

906 + Direct_read int64

907 + Direct_write int64

908 + Fcm_recv int64

909 + Fcm_send int64

910 +}

911 +

912 +// ConnectionWaitTimeContext provides type-safe operations for Connection.WaitTime context

913 +type ConnectionWaitTimeContext struct {

914 + framework.Context[ConnectionLabels]

915 +}

916 +

917 +// Set provides type-safe dimension setting for Connection.WaitTime context

918 +func (c ConnectionWaitTimeContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionWaitTimeValues) {

919 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

920 + "lock": values.Lock,

921 + "log_disk": values.Log_disk,

922 + "log_buffer": values.Log_buffer,

923 + "pool_read": values.Pool_read,

924 + "pool_write": values.Pool_write,

925 + "direct_read": values.Direct_read,

926 + "direct_write": values.Direct_write,

927 + "fcm_recv": values.Fcm_recv,

928 + "fcm_send": values.Fcm_send,

929 + })

930 +}

931 +

932 +// SetUpdateEvery sets the update interval for this instance

933 +func (c ConnectionWaitTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

934 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

935 +}

936 +

937 +// ConnectionProcessingTimeValues defines the type-safe values for Connection.ProcessingTime context

938 +type ConnectionProcessingTimeValues struct {

939 + Routine int64

940 + Compile int64

941 + Section int64

942 + Commit int64

943 + Rollback int64

944 +}

945 +

946 +// ConnectionProcessingTimeContext provides type-safe operations for Connection.ProcessingTime context

947 +type ConnectionProcessingTimeContext struct {

948 + framework.Context[ConnectionLabels]

949 +}

950 +

951 +// Set provides type-safe dimension setting for Connection.ProcessingTime context

952 +func (c ConnectionProcessingTimeContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionProcessingTimeValues) {

953 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

954 + "routine": values.Routine,

955 + "compile": values.Compile,

956 + "section": values.Section,

957 + "commit": values.Commit,

958 + "rollback": values.Rollback,

959 + })

960 +}

961 +

962 +// SetUpdateEvery sets the update interval for this instance

963 +func (c ConnectionProcessingTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

964 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

965 +}

966 +

967 +// ConnectionLabels defines the required labels for Connection contexts

968 +type ConnectionLabels struct {

969 + Application_id string

970 + Application_name string

971 + Client_hostname string

972 + Client_ip string

973 + Client_user string

974 + State string

975 +}

976 +

977 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

978 +func (l ConnectionLabels) InstanceID(contextName string) string {

979 + // Label order from YAML: application_id, application_name, client_hostname, client_ip, client_user, state

980 + return contextName + "." + cleanLabelValue(l.Application_id) + "_" + cleanLabelValue(l.Application_name) + "_" + cleanLabelValue(l.Client_hostname) + "_" + cleanLabelValue(l.Client_ip) + "_" + cleanLabelValue(l.Client_user) + "_" + cleanLabelValue(l.State)

981 +}

982 +

983 +// Connection contains all metric contexts for Connection

984 +var Connection = struct {

985 + State ConnectionStateContext

986 + Activity ConnectionActivityContext

987 + WaitTime ConnectionWaitTimeContext

988 + ProcessingTime ConnectionProcessingTimeContext

989 +}{

990 + State: ConnectionStateContext{

991 + Context: framework.Context[ConnectionLabels]{

992 + Name: "db2.connection_state",

993 + Family: "connections/instances",

994 + Title: "Connection State",

995 + Units: "state",

996 + Type: module.Line,

997 + Priority: 1130,

998 + UpdateEvery: 1,

999 + Dimensions: []framework.Dimension{

1000 + {

1001 + Name: "state",

1002 + Algorithm: module.Absolute,

1003 + Mul: 1,

1004 + Div: 1,

1005 + Precision: 1,

1006 + },

1007 + },

1008 + LabelKeys: []string{

1009 + "application_id",

1010 + "application_name",

1011 + "client_hostname",

1012 + "client_ip",

1013 + "client_user",

1014 + "state",

1015 + },

1016 + },

1017 + },

1018 + Activity: ConnectionActivityContext{

1019 + Context: framework.Context[ConnectionLabels]{

1020 + Name: "db2.connection_activity",

1021 + Family: "connections/instances",

1022 + Title: "Connection Row Activity",

1023 + Units: "rows/s",

1024 + Type: module.Area,

1025 + Priority: 1131,

1026 + UpdateEvery: 1,

1027 + Dimensions: []framework.Dimension{

1028 + {

1029 + Name: "read",

1030 + Algorithm: module.Incremental,

1031 + Mul: 1,

1032 + Div: 1,

1033 + Precision: 1,

1034 + },

1035 + {

1036 + Name: "written",

1037 + Algorithm: module.Incremental,

1038 + Mul: 1,

1039 + Div: 1,

1040 + Precision: 1,

1041 + },

1042 + },

1043 + LabelKeys: []string{

1044 + "application_id",

1045 + "application_name",

1046 + "client_hostname",

1047 + "client_ip",

1048 + "client_user",

1049 + "state",

1050 + },

1051 + },

1052 + },

1053 + WaitTime: ConnectionWaitTimeContext{

1054 + Context: framework.Context[ConnectionLabels]{

1055 + Name: "db2.connection_wait_time",

1056 + Family: "connections/instances",

1057 + Title: "Connection Wait Time",

1058 + Units: "milliseconds",

1059 + Type: module.Stacked,

1060 + Priority: 1132,

1061 + UpdateEvery: 1,

1062 + Dimensions: []framework.Dimension{

1063 + {

1064 + Name: "lock",

1065 + Algorithm: module.Incremental,

1066 + Mul: 1,

1067 + Div: 1,

1068 + Precision: 1,

1069 + },

1070 + {

1071 + Name: "log_disk",

1072 + Algorithm: module.Incremental,

1073 + Mul: 1,

1074 + Div: 1,

1075 + Precision: 1,

1076 + },

1077 + {

1078 + Name: "log_buffer",

1079 + Algorithm: module.Incremental,

1080 + Mul: 1,

1081 + Div: 1,

1082 + Precision: 1,

1083 + },

1084 + {

1085 + Name: "pool_read",

1086 + Algorithm: module.Incremental,

1087 + Mul: 1,

1088 + Div: 1,

1089 + Precision: 1,

1090 + },

1091 + {

1092 + Name: "pool_write",

1093 + Algorithm: module.Incremental,

1094 + Mul: 1,

1095 + Div: 1,

1096 + Precision: 1,

1097 + },

1098 + {

1099 + Name: "direct_read",

1100 + Algorithm: module.Incremental,

1101 + Mul: 1,

1102 + Div: 1,

1103 + Precision: 1,

1104 + },

1105 + {

1106 + Name: "direct_write",

1107 + Algorithm: module.Incremental,

1108 + Mul: 1,

1109 + Div: 1,

1110 + Precision: 1,

1111 + },

1112 + {

1113 + Name: "fcm_recv",

1114 + Algorithm: module.Incremental,

1115 + Mul: 1,

1116 + Div: 1,

1117 + Precision: 1,

1118 + },

1119 + {

1120 + Name: "fcm_send",

1121 + Algorithm: module.Incremental,

1122 + Mul: 1,

1123 + Div: 1,

1124 + Precision: 1,

1125 + },

1126 + },

1127 + LabelKeys: []string{

1128 + "application_id",

1129 + "application_name",

1130 + "client_hostname",

1131 + "client_ip",

1132 + "client_user",

1133 + "state",

1134 + },

1135 + },

1136 + },

1137 + ProcessingTime: ConnectionProcessingTimeContext{

1138 + Context: framework.Context[ConnectionLabels]{

1139 + Name: "db2.connection_processing_time",

1140 + Family: "connections/instances",

1141 + Title: "Connection Processing Time",

1142 + Units: "milliseconds",

1143 + Type: module.Stacked,

1144 + Priority: 1133,

1145 + UpdateEvery: 1,

1146 + Dimensions: []framework.Dimension{

1147 + {

1148 + Name: "routine",

1149 + Algorithm: module.Incremental,

1150 + Mul: 1,

1151 + Div: 1,

1152 + Precision: 1,

1153 + },

1154 + {

1155 + Name: "compile",

1156 + Algorithm: module.Incremental,

1157 + Mul: 1,

1158 + Div: 1,

1159 + Precision: 1,

1160 + },

1161 + {

1162 + Name: "section",

1163 + Algorithm: module.Incremental,

1164 + Mul: 1,

1165 + Div: 1,

1166 + Precision: 1,

1167 + },

1168 + {

1169 + Name: "commit",

1170 + Algorithm: module.Incremental,

1171 + Mul: 1,

1172 + Div: 1,

1173 + Precision: 1,

1174 + },

1175 + {

1176 + Name: "rollback",

1177 + Algorithm: module.Incremental,

1178 + Mul: 1,

1179 + Div: 1,

1180 + Precision: 1,

1181 + },

1182 + },

1183 + LabelKeys: []string{

1184 + "application_id",

1185 + "application_name",

1186 + "client_hostname",

1187 + "client_ip",

1188 + "client_user",

1189 + "state",

1190 + },

1191 + },

1192 + },

1193 +}

1194 +

1195 +// --- ConnectionGroup ---

1196 +

1197 +// ConnectionGroupCountValues defines the type-safe values for ConnectionGroup.Count context

1198 +type ConnectionGroupCountValues struct {

1199 + Count int64

1200 +}

1201 +

1202 +// ConnectionGroupCountContext provides type-safe operations for ConnectionGroup.Count context

1203 +type ConnectionGroupCountContext struct {

1204 + framework.Context[ConnectionGroupLabels]

1205 +}

1206 +

1207 +// Set provides type-safe dimension setting for ConnectionGroup.Count context

1208 +func (c ConnectionGroupCountContext) Set(state *framework.CollectorState, labels ConnectionGroupLabels, values ConnectionGroupCountValues) {

1209 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1210 + "count": values.Count,

1211 + })

1212 +}

1213 +

1214 +// SetUpdateEvery sets the update interval for this instance

1215 +func (c ConnectionGroupCountContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionGroupLabels, updateEvery int) {

1216 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1217 +}

1218 +

1219 +// ConnectionGroupStateValues defines the type-safe values for ConnectionGroup.State context

1220 +type ConnectionGroupStateValues struct {

1221 State int64

1222 }

1223

455 -// ConnectionStateContext provides type-safe operations for Connection.State context

456 -type ConnectionStateContext struct {

457 - framework.Context[ConnectionLabels]

1224 +// ConnectionGroupStateContext provides type-safe operations for ConnectionGroup.State context

1225 +type ConnectionGroupStateContext struct {

1226 + framework.Context[ConnectionGroupLabels]

1227 }

1228

460 -// Set provides type-safe dimension setting for Connection.State context

461 -func (c ConnectionStateContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionStateValues) {

1229 +// Set provides type-safe dimension setting for ConnectionGroup.State context

1230 +func (c ConnectionGroupStateContext) Set(state *framework.CollectorState, labels ConnectionGroupLabels, values ConnectionGroupStateValues) {

1231 state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1232 "state": values.State,

1233 })

1234 }

1235

1236 // SetUpdateEvery sets the update interval for this instance

468 -func (c ConnectionStateContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

1237 +func (c ConnectionGroupStateContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionGroupLabels, updateEvery int) {

1238 state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1239 }

1240

472 -// ConnectionActivityValues defines the type-safe values for Connection.Activity context

473 -type ConnectionActivityValues struct {

1241 +// ConnectionGroupActivityValues defines the type-safe values for ConnectionGroup.Activity context

1242 +type ConnectionGroupActivityValues struct {

1243 Read int64

1244 Written int64

1245 }

1246

478 -// ConnectionActivityContext provides type-safe operations for Connection.Activity context

479 -type ConnectionActivityContext struct {

480 - framework.Context[ConnectionLabels]

1247 +// ConnectionGroupActivityContext provides type-safe operations for ConnectionGroup.Activity context

1248 +type ConnectionGroupActivityContext struct {

1249 + framework.Context[ConnectionGroupLabels]

1250 }

1251

483 -// Set provides type-safe dimension setting for Connection.Activity context

484 -func (c ConnectionActivityContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionActivityValues) {

1252 +// Set provides type-safe dimension setting for ConnectionGroup.Activity context

1253 +func (c ConnectionGroupActivityContext) Set(state *framework.CollectorState, labels ConnectionGroupLabels, values ConnectionGroupActivityValues) {

1254 state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1255 "read": values.Read,

1256 "written": values.Written,

@@ -489,12 +1258,12 @@ func (c ConnectionActivityContext) Set(state *framework.CollectorState, labels C

1258 }

1259

1260 // SetUpdateEvery sets the update interval for this instance

492 -func (c ConnectionActivityContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

1261 +func (c ConnectionGroupActivityContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionGroupLabels, updateEvery int) {

1262 state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1263 }

1264

496 -// ConnectionWaitTimeValues defines the type-safe values for Connection.WaitTime context

497 -type ConnectionWaitTimeValues struct {

1265 +// ConnectionGroupWaitTimeValues defines the type-safe values for ConnectionGroup.WaitTime context

1266 +type ConnectionGroupWaitTimeValues struct {

1267 Lock int64

1268 Log_disk int64

1269 Log_buffer int64

@@ -506,13 +1275,13 @@ type ConnectionWaitTimeValues struct {

1275 Fcm_send int64

1276 }

1277

509 -// ConnectionWaitTimeContext provides type-safe operations for Connection.WaitTime context

510 -type ConnectionWaitTimeContext struct {

511 - framework.Context[ConnectionLabels]

1278 +// ConnectionGroupWaitTimeContext provides type-safe operations for ConnectionGroup.WaitTime context

1279 +type ConnectionGroupWaitTimeContext struct {

1280 + framework.Context[ConnectionGroupLabels]

1281 }

1282

514 -// Set provides type-safe dimension setting for Connection.WaitTime context

515 -func (c ConnectionWaitTimeContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionWaitTimeValues) {

1283 +// Set provides type-safe dimension setting for ConnectionGroup.WaitTime context

1284 +func (c ConnectionGroupWaitTimeContext) Set(state *framework.CollectorState, labels ConnectionGroupLabels, values ConnectionGroupWaitTimeValues) {

1285 state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1286 "lock": values.Lock,

1287 "log_disk": values.Log_disk,

@@ -527,12 +1296,12 @@ func (c ConnectionWaitTimeContext) Set(state *framework.CollectorState, labels C

1296 }

1297

1298 // SetUpdateEvery sets the update interval for this instance

530 -func (c ConnectionWaitTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

1299 +func (c ConnectionGroupWaitTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionGroupLabels, updateEvery int) {

1300 state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1301 }

1302

534 -// ConnectionProcessingTimeValues defines the type-safe values for Connection.ProcessingTime context

535 -type ConnectionProcessingTimeValues struct {

1303 +// ConnectionGroupProcessingTimeValues defines the type-safe values for ConnectionGroup.ProcessingTime context

1304 +type ConnectionGroupProcessingTimeValues struct {

1305 Routine int64

1306 Compile int64

1307 Section int64

@@ -540,13 +1309,13 @@ type ConnectionProcessingTimeValues struct {

1309 Rollback int64

1310 }

1311

543 -// ConnectionProcessingTimeContext provides type-safe operations for Connection.ProcessingTime context

544 -type ConnectionProcessingTimeContext struct {

545 - framework.Context[ConnectionLabels]

1312 +// ConnectionGroupProcessingTimeContext provides type-safe operations for ConnectionGroup.ProcessingTime context

1313 +type ConnectionGroupProcessingTimeContext struct {

1314 + framework.Context[ConnectionGroupLabels]

1315 }

1316

548 -// Set provides type-safe dimension setting for Connection.ProcessingTime context

549 -func (c ConnectionProcessingTimeContext) Set(state *framework.CollectorState, labels ConnectionLabels, values ConnectionProcessingTimeValues) {

1317 +// Set provides type-safe dimension setting for ConnectionGroup.ProcessingTime context

1318 +func (c ConnectionGroupProcessingTimeContext) Set(state *framework.CollectorState, labels ConnectionGroupLabels, values ConnectionGroupProcessingTimeValues) {

1319 state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1320 "routine": values.Routine,

1321 "compile": values.Compile,

@@ -557,41 +1326,60 @@ func (c ConnectionProcessingTimeContext) Set(state *framework.CollectorState, la

1326 }

1327

1328 // SetUpdateEvery sets the update interval for this instance

560 -func (c ConnectionProcessingTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionLabels, updateEvery int) {

1329 +func (c ConnectionGroupProcessingTimeContext) SetUpdateEvery(state *framework.CollectorState, labels ConnectionGroupLabels, updateEvery int) {

1330 state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1331 }

1332

564 -// ConnectionLabels defines the required labels for Connection contexts

565 -type ConnectionLabels struct {

566 - Application_id string

567 - Application_name string

568 - Client_hostname string

569 - Client_ip string

570 - Client_user string

571 - State string

1333 +// ConnectionGroupLabels defines the required labels for ConnectionGroup contexts

1334 +type ConnectionGroupLabels struct {

1335 + Group string

1336 }

1337

1338 // InstanceID generates a unique instance ID using the hardcoded label order from YAML

575 -func (l ConnectionLabels) InstanceID(contextName string) string {

576 - // Label order from YAML: application_id, application_name, client_hostname, client_ip, client_user, state

577 - return contextName + "." + cleanLabelValue(l.Application_id) + "_" + cleanLabelValue(l.Application_name) + "_" + cleanLabelValue(l.Client_hostname) + "_" + cleanLabelValue(l.Client_ip) + "_" + cleanLabelValue(l.Client_user) + "_" + cleanLabelValue(l.State)

578 -}

579 -

580 -// Connection contains all metric contexts for Connection

581 -var Connection = struct {

582 - State ConnectionStateContext

583 - Activity ConnectionActivityContext

584 - WaitTime ConnectionWaitTimeContext

585 - ProcessingTime ConnectionProcessingTimeContext

1339 +func (l ConnectionGroupLabels) InstanceID(contextName string) string {

1340 + // Label order from YAML: group

1341 + return contextName + "." + cleanLabelValue(l.Group)

1342 +}

1343 +

1344 +// ConnectionGroup contains all metric contexts for ConnectionGroup

1345 +var ConnectionGroup = struct {

1346 + Count ConnectionGroupCountContext

1347 + State ConnectionGroupStateContext

1348 + Activity ConnectionGroupActivityContext

1349 + WaitTime ConnectionGroupWaitTimeContext

1350 + ProcessingTime ConnectionGroupProcessingTimeContext

1351 }{

587 - State: ConnectionStateContext{

588 - Context: framework.Context[ConnectionLabels]{

589 - Name: "db2.connection_state",

590 - Family: "connections/instances",

591 - Title: "Connection State",

1352 + Count: ConnectionGroupCountContext{

1353 + Context: framework.Context[ConnectionGroupLabels]{

1354 + Name: "db2.connection_group.count",

1355 + Family: "connections/groups",

1356 + Title: "Connection Group Count",

1357 + Units: "connections",

1358 + Type: module.Line,

1359 + Priority: 1134,

1360 + UpdateEvery: 1,

1361 + Dimensions: []framework.Dimension{

1362 + {

1363 + Name: "count",

1364 + Algorithm: module.Absolute,

1365 + Mul: 1,

1366 + Div: 1,

1367 + Precision: 1,

1368 + },

1369 + },

1370 + LabelKeys: []string{

1371 + "group",

1372 + },

1373 + },

1374 + },

1375 + State: ConnectionGroupStateContext{

1376 + Context: framework.Context[ConnectionGroupLabels]{

1377 + Name: "db2.connection_group.state",

1378 + Family: "connections/groups",

1379 + Title: "Connection Group State Sum",

1380 Units: "state",

1381 Type: module.Line,

594 - Priority: 1130,

1382 + Priority: 1135,

1383 UpdateEvery: 1,

1384 Dimensions: []framework.Dimension{

1385 {

@@ -603,23 +1391,18 @@ var Connection = struct {

1391 },

1392 },

1393 LabelKeys: []string{

606 - "application_id",

607 - "application_name",

608 - "client_hostname",

609 - "client_ip",

610 - "client_user",

611 - "state",

1394 + "group",

1395 },

1396 },

1397 },

615 - Activity: ConnectionActivityContext{

616 - Context: framework.Context[ConnectionLabels]{

617 - Name: "db2.connection_activity",

618 - Family: "connections/instances",

619 - Title: "Connection Row Activity",

1398 + Activity: ConnectionGroupActivityContext{

1399 + Context: framework.Context[ConnectionGroupLabels]{

1400 + Name: "db2.connection_group.activity",

1401 + Family: "connections/groups",

1402 + Title: "Connection Group Row Activity",

1403 Units: "rows/s",

1404 Type: module.Area,

622 - Priority: 1131,

1405 + Priority: 1136,

1406 UpdateEvery: 1,

1407 Dimensions: []framework.Dimension{

1408 {

@@ -638,23 +1421,18 @@ var Connection = struct {

1421 },

1422 },

1423 LabelKeys: []string{

641 - "application_id",

642 - "application_name",

643 - "client_hostname",

644 - "client_ip",

645 - "client_user",

646 - "state",

1424 + "group",

1425 },

1426 },

1427 },

650 - WaitTime: ConnectionWaitTimeContext{

651 - Context: framework.Context[ConnectionLabels]{

652 - Name: "db2.connection_wait_time",

653 - Family: "connections/instances",

654 - Title: "Connection Wait Time",

1428 + WaitTime: ConnectionGroupWaitTimeContext{

1429 + Context: framework.Context[ConnectionGroupLabels]{

1430 + Name: "db2.connection_group.wait_time",

1431 + Family: "connections/groups",

1432 + Title: "Connection Group Wait Time",

1433 Units: "milliseconds",

1434 Type: module.Stacked,

657 - Priority: 1132,

1435 + Priority: 1137,

1436 UpdateEvery: 1,

1437 Dimensions: []framework.Dimension{

1438 {

@@ -722,23 +1500,18 @@ var Connection = struct {

1500 },

1501 },

1502 LabelKeys: []string{

725 - "application_id",

726 - "application_name",

727 - "client_hostname",

728 - "client_ip",

729 - "client_user",

730 - "state",

1503 + "group",

1504 },

1505 },

1506 },

734 - ProcessingTime: ConnectionProcessingTimeContext{

735 - Context: framework.Context[ConnectionLabels]{

736 - Name: "db2.connection_processing_time",

737 - Family: "connections/instances",

738 - Title: "Connection Processing Time",

1507 + ProcessingTime: ConnectionGroupProcessingTimeContext{

1508 + Context: framework.Context[ConnectionGroupLabels]{

1509 + Name: "db2.connection_group.processing_time",

1510 + Family: "connections/groups",

1511 + Title: "Connection Group Processing Time",

1512 Units: "milliseconds",

1513 Type: module.Stacked,

741 - Priority: 1133,

1514 + Priority: 1138,

1515 UpdateEvery: 1,

1516 Dimensions: []framework.Dimension{

1517 {

@@ -778,12 +1551,7 @@ var Connection = struct {

1551 },

1552 },

1553 LabelKeys: []string{

781 - "application_id",

782 - "application_name",

783 - "client_hostname",

784 - "client_ip",

785 - "client_user",

786 - "state",

1554 + "group",

1555 },

1556 },

1557 },

@@ -934,23 +1702,96 @@ type IndexLabels struct {

1702 }

1703

1704 // InstanceID generates a unique instance ID using the hardcoded label order from YAML

937 -func (l IndexLabels) InstanceID(contextName string) string {

938 - // Label order from YAML: index

939 - return contextName + "." + cleanLabelValue(l.Index)

1705 +func (l IndexLabels) InstanceID(contextName string) string {

1706 + // Label order from YAML: index

1707 + return contextName + "." + cleanLabelValue(l.Index)

1708 +}

1709 +

1710 +// Index contains all metric contexts for Index

1711 +var Index = struct {

1712 + Usage IndexUsageContext

1713 +}{

1714 + Usage: IndexUsageContext{

1715 + Context: framework.Context[IndexLabels]{

1716 + Name: "db2.index_usage",

1717 + Family: "indexes",

1718 + Title: "Index Usage",

1719 + Units: "scans/s",

1720 + Type: module.Area,

1721 + Priority: 1150,

1722 + UpdateEvery: 1,

1723 + Dimensions: []framework.Dimension{

1724 + {

1725 + Name: "index",

1726 + Algorithm: module.Incremental,

1727 + Mul: 1,

1728 + Div: 1,

1729 + Precision: 1,

1730 + },

1731 + {

1732 + Name: "full",

1733 + Algorithm: module.Incremental,

1734 + Mul: 1,

1735 + Div: 1,

1736 + Precision: 1,

1737 + },

1738 + },

1739 + LabelKeys: []string{

1740 + "index",

1741 + },

1742 + },

1743 + },

1744 +}

1745 +

1746 +// --- IndexGroup ---

1747 +

1748 +// IndexGroupUsageValues defines the type-safe values for IndexGroup.Usage context

1749 +type IndexGroupUsageValues struct {

1750 + Index int64

1751 + Full int64

1752 +}

1753 +

1754 +// IndexGroupUsageContext provides type-safe operations for IndexGroup.Usage context

1755 +type IndexGroupUsageContext struct {

1756 + framework.Context[IndexGroupLabels]

1757 +}

1758 +

1759 +// Set provides type-safe dimension setting for IndexGroup.Usage context

1760 +func (c IndexGroupUsageContext) Set(state *framework.CollectorState, labels IndexGroupLabels, values IndexGroupUsageValues) {

1761 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

1762 + "index": values.Index,

1763 + "full": values.Full,

1764 + })

1765 +}

1766 +

1767 +// SetUpdateEvery sets the update interval for this instance

1768 +func (c IndexGroupUsageContext) SetUpdateEvery(state *framework.CollectorState, labels IndexGroupLabels, updateEvery int) {

1769 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

1770 +}

1771 +

1772 +// IndexGroupLabels defines the required labels for IndexGroup contexts

1773 +type IndexGroupLabels struct {

1774 + Group string

1775 +}

1776 +

1777 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

1778 +func (l IndexGroupLabels) InstanceID(contextName string) string {

1779 + // Label order from YAML: group

1780 + return contextName + "." + cleanLabelValue(l.Group)

1781 }

1782

942 -// Index contains all metric contexts for Index

943 -var Index = struct {

944 - Usage IndexUsageContext

1783 +// IndexGroup contains all metric contexts for IndexGroup

1784 +var IndexGroup = struct {

1785 + Usage IndexGroupUsageContext

1786 }{

946 - Usage: IndexUsageContext{

947 - Context: framework.Context[IndexLabels]{

948 - Name: "db2.index_usage",

949 - Family: "indexes",

950 - Title: "Index Usage",

1787 + Usage: IndexGroupUsageContext{

1788 + Context: framework.Context[IndexGroupLabels]{

1789 + Name: "db2.index_group_usage",

1790 + Family: "indexes/groups",

1791 + Title: "Index Group Usage",

1792 Units: "scans/s",

1793 Type: module.Area,

953 - Priority: 1150,

1794 + Priority: 1151,

1795 UpdateEvery: 1,

1796 Dimensions: []framework.Dimension{

1797 {

@@ -969,7 +1810,7 @@ var Index = struct {

1810 },

1811 },

1812 LabelKeys: []string{

972 - "index",

1813 + "group",

1814 },

1815 },

1816 },

@@ -3902,6 +4743,143 @@ var Table = struct {

4743 },

4744 }

4745

4746 +// --- TableGroup ---

4747 +

4748 +// TableGroupSizeValues defines the type-safe values for TableGroup.Size context

4749 +type TableGroupSizeValues struct {

4750 + Data int64

4751 + Index int64

4752 + Long_obj int64

4753 +}

4754 +

4755 +// TableGroupSizeContext provides type-safe operations for TableGroup.Size context

4756 +type TableGroupSizeContext struct {

4757 + framework.Context[TableGroupLabels]

4758 +}

4759 +

4760 +// Set provides type-safe dimension setting for TableGroup.Size context

4761 +func (c TableGroupSizeContext) Set(state *framework.CollectorState, labels TableGroupLabels, values TableGroupSizeValues) {

4762 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

4763 + "data": values.Data,

4764 + "index": values.Index,

4765 + "long_obj": values.Long_obj,

4766 + })

4767 +}

4768 +

4769 +// SetUpdateEvery sets the update interval for this instance

4770 +func (c TableGroupSizeContext) SetUpdateEvery(state *framework.CollectorState, labels TableGroupLabels, updateEvery int) {

4771 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

4772 +}

4773 +

4774 +// TableGroupActivityValues defines the type-safe values for TableGroup.Activity context

4775 +type TableGroupActivityValues struct {

4776 + Read int64

4777 + Written int64

4778 +}

4779 +

4780 +// TableGroupActivityContext provides type-safe operations for TableGroup.Activity context

4781 +type TableGroupActivityContext struct {

4782 + framework.Context[TableGroupLabels]

4783 +}

4784 +

4785 +// Set provides type-safe dimension setting for TableGroup.Activity context

4786 +func (c TableGroupActivityContext) Set(state *framework.CollectorState, labels TableGroupLabels, values TableGroupActivityValues) {

4787 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

4788 + "read": values.Read,

4789 + "written": values.Written,

4790 + })

4791 +}

4792 +

4793 +// SetUpdateEvery sets the update interval for this instance

4794 +func (c TableGroupActivityContext) SetUpdateEvery(state *framework.CollectorState, labels TableGroupLabels, updateEvery int) {

4795 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

4796 +}

4797 +

4798 +// TableGroupLabels defines the required labels for TableGroup contexts

4799 +type TableGroupLabels struct {

4800 + Group string

4801 +}

4802 +

4803 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

4804 +func (l TableGroupLabels) InstanceID(contextName string) string {

4805 + // Label order from YAML: group

4806 + return contextName + "." + cleanLabelValue(l.Group)

4807 +}

4808 +

4809 +// TableGroup contains all metric contexts for TableGroup

4810 +var TableGroup = struct {

4811 + Size TableGroupSizeContext

4812 + Activity TableGroupActivityContext

4813 +}{

4814 + Size: TableGroupSizeContext{

4815 + Context: framework.Context[TableGroupLabels]{

4816 + Name: "db2.table_group_size",

4817 + Family: "tables/groups",

4818 + Title: "Table Group Size",

4819 + Units: "bytes",

4820 + Type: module.Stacked,

4821 + Priority: 1142,

4822 + UpdateEvery: 1,

4823 + Dimensions: []framework.Dimension{

4824 + {

4825 + Name: "data",

4826 + Algorithm: module.Absolute,

4827 + Mul: 1,

4828 + Div: 1,

4829 + Precision: 1,

4830 + },

4831 + {

4832 + Name: "index",

4833 + Algorithm: module.Absolute,

4834 + Mul: 1,

4835 + Div: 1,

4836 + Precision: 1,

4837 + },

4838 + {

4839 + Name: "long_obj",

4840 + Algorithm: module.Absolute,

4841 + Mul: 1,

4842 + Div: 1,

4843 + Precision: 1,

4844 + },

4845 + },

4846 + LabelKeys: []string{

4847 + "group",

4848 + },

4849 + },

4850 + },

4851 + Activity: TableGroupActivityContext{

4852 + Context: framework.Context[TableGroupLabels]{

4853 + Name: "db2.table_group_activity",

4854 + Family: "tables/groups",

4855 + Title: "Table Group Activity",

4856 + Units: "rows/s",

4857 + Type: module.Area,

4858 + Priority: 1143,

4859 + UpdateEvery: 1,

4860 + Dimensions: []framework.Dimension{

4861 + {

4862 + Name: "read",

4863 + Algorithm: module.Incremental,

4864 + Mul: 1,

4865 + Div: 1,

4866 + Precision: 1,

4867 + },

4868 + {

4869 + Name: "written",

4870 + Algorithm: module.Incremental,

4871 + Mul: 1,

4872 + Div: 1,

4873 + Precision: 1,

4874 + },

4875 + },

4876 + LabelKeys: []string{

4877 + "group",

4878 + },

4879 + },

4880 + },

4881 +}

4882 +

4883 // --- TableIO ---

4884

4885 // TableIOScansValues defines the type-safe values for TableIO.Scans context

@@ -4357,6 +5335,226 @@ var Tablespace = struct {

5335 },

5336 }

5337

5338 +// --- TablespaceGroup ---

5339 +

5340 +// TablespaceGroupUsageValues defines the type-safe values for TablespaceGroup.Usage context

5341 +type TablespaceGroupUsageValues struct {

5342 + Used int64

5343 +}

5344 +

5345 +// TablespaceGroupUsageContext provides type-safe operations for TablespaceGroup.Usage context

5346 +type TablespaceGroupUsageContext struct {

5347 + framework.Context[TablespaceGroupLabels]

5348 +}

5349 +

5350 +// Set provides type-safe dimension setting for TablespaceGroup.Usage context

5351 +func (c TablespaceGroupUsageContext) Set(state *framework.CollectorState, labels TablespaceGroupLabels, values TablespaceGroupUsageValues) {

5352 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

5353 + "used": values.Used,

5354 + })

5355 +}

5356 +

5357 +// SetUpdateEvery sets the update interval for this instance

5358 +func (c TablespaceGroupUsageContext) SetUpdateEvery(state *framework.CollectorState, labels TablespaceGroupLabels, updateEvery int) {

5359 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

5360 +}

5361 +

5362 +// TablespaceGroupSizeValues defines the type-safe values for TablespaceGroup.Size context

5363 +type TablespaceGroupSizeValues struct {

5364 + Used int64

5365 + Free int64

5366 +}

5367 +

5368 +// TablespaceGroupSizeContext provides type-safe operations for TablespaceGroup.Size context

5369 +type TablespaceGroupSizeContext struct {

5370 + framework.Context[TablespaceGroupLabels]

5371 +}

5372 +

5373 +// Set provides type-safe dimension setting for TablespaceGroup.Size context

5374 +func (c TablespaceGroupSizeContext) Set(state *framework.CollectorState, labels TablespaceGroupLabels, values TablespaceGroupSizeValues) {

5375 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

5376 + "used": values.Used,

5377 + "free": values.Free,

5378 + })

5379 +}

5380 +

5381 +// SetUpdateEvery sets the update interval for this instance

5382 +func (c TablespaceGroupSizeContext) SetUpdateEvery(state *framework.CollectorState, labels TablespaceGroupLabels, updateEvery int) {

5383 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

5384 +}

5385 +

5386 +// TablespaceGroupUsableSizeValues defines the type-safe values for TablespaceGroup.UsableSize context

5387 +type TablespaceGroupUsableSizeValues struct {

5388 + Total int64

5389 + Usable int64

5390 +}

5391 +

5392 +// TablespaceGroupUsableSizeContext provides type-safe operations for TablespaceGroup.UsableSize context

5393 +type TablespaceGroupUsableSizeContext struct {

5394 + framework.Context[TablespaceGroupLabels]

5395 +}

5396 +

5397 +// Set provides type-safe dimension setting for TablespaceGroup.UsableSize context

5398 +func (c TablespaceGroupUsableSizeContext) Set(state *framework.CollectorState, labels TablespaceGroupLabels, values TablespaceGroupUsableSizeValues) {

5399 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

5400 + "total": values.Total,

5401 + "usable": values.Usable,

5402 + })

5403 +}

5404 +

5405 +// SetUpdateEvery sets the update interval for this instance

5406 +func (c TablespaceGroupUsableSizeContext) SetUpdateEvery(state *framework.CollectorState, labels TablespaceGroupLabels, updateEvery int) {

5407 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

5408 +}

5409 +

5410 +// TablespaceGroupStateValues defines the type-safe values for TablespaceGroup.State context

5411 +type TablespaceGroupStateValues struct {

5412 + State int64

5413 +}

5414 +

5415 +// TablespaceGroupStateContext provides type-safe operations for TablespaceGroup.State context

5416 +type TablespaceGroupStateContext struct {

5417 + framework.Context[TablespaceGroupLabels]

5418 +}

5419 +

5420 +// Set provides type-safe dimension setting for TablespaceGroup.State context

5421 +func (c TablespaceGroupStateContext) Set(state *framework.CollectorState, labels TablespaceGroupLabels, values TablespaceGroupStateValues) {

5422 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

5423 + "state": values.State,

5424 + })

5425 +}

5426 +

5427 +// SetUpdateEvery sets the update interval for this instance

5428 +func (c TablespaceGroupStateContext) SetUpdateEvery(state *framework.CollectorState, labels TablespaceGroupLabels, updateEvery int) {

5429 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

5430 +}

5431 +

5432 +// TablespaceGroupLabels defines the required labels for TablespaceGroup contexts

5433 +type TablespaceGroupLabels struct {

5434 + Group string

5435 +}

5436 +

5437 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

5438 +func (l TablespaceGroupLabels) InstanceID(contextName string) string {

5439 + // Label order from YAML: group

5440 + return contextName + "." + cleanLabelValue(l.Group)

5441 +}

5442 +

5443 +// TablespaceGroup contains all metric contexts for TablespaceGroup

5444 +var TablespaceGroup = struct {

5445 + Usage TablespaceGroupUsageContext

5446 + Size TablespaceGroupSizeContext

5447 + UsableSize TablespaceGroupUsableSizeContext

5448 + State TablespaceGroupStateContext

5449 +}{

5450 + Usage: TablespaceGroupUsageContext{

5451 + Context: framework.Context[TablespaceGroupLabels]{

5452 + Name: "db2.tablespace_group_usage",

5453 + Family: "tablespaces/groups",

5454 + Title: "Tablespace Group Usage",

5455 + Units: "percentage",

5456 + Type: module.Line,

5457 + Priority: 1124,

5458 + UpdateEvery: 1,

5459 + Dimensions: []framework.Dimension{

5460 + {

5461 + Name: "used",

5462 + Algorithm: module.Absolute,

5463 + Mul: 1,

5464 + Div: 1000,

5465 + Precision: 1000,

5466 + },

5467 + },

5468 + LabelKeys: []string{

5469 + "group",

5470 + },

5471 + },

5472 + },

5473 + Size: TablespaceGroupSizeContext{

5474 + Context: framework.Context[TablespaceGroupLabels]{

5475 + Name: "db2.tablespace_group_size",

5476 + Family: "tablespaces/groups",

5477 + Title: "Tablespace Group Size",

5478 + Units: "bytes",

5479 + Type: module.Stacked,

5480 + Priority: 1125,

5481 + UpdateEvery: 1,

5482 + Dimensions: []framework.Dimension{

5483 + {

5484 + Name: "used",

5485 + Algorithm: module.Absolute,

5486 + Mul: 1,

5487 + Div: 1,

5488 + Precision: 1,

5489 + },

5490 + {

5491 + Name: "free",

5492 + Algorithm: module.Absolute,

5493 + Mul: 1,

5494 + Div: 1,

5495 + Precision: 1,

5496 + },

5497 + },

5498 + LabelKeys: []string{

5499 + "group",

5500 + },

5501 + },

5502 + },

5503 + UsableSize: TablespaceGroupUsableSizeContext{

5504 + Context: framework.Context[TablespaceGroupLabels]{

5505 + Name: "db2.tablespace_group_usable_size",

5506 + Family: "tablespaces/groups",

5507 + Title: "Tablespace Group Usable Size",

5508 + Units: "bytes",

5509 + Type: module.Line,

5510 + Priority: 1126,

5511 + UpdateEvery: 1,

5512 + Dimensions: []framework.Dimension{

5513 + {

5514 + Name: "total",

5515 + Algorithm: module.Absolute,

5516 + Mul: 1,

5517 + Div: 1,

5518 + Precision: 1,

5519 + },

5520 + {

5521 + Name: "usable",

5522 + Algorithm: module.Absolute,

5523 + Mul: 1,

5524 + Div: 1,

5525 + Precision: 1,

5526 + },

5527 + },

5528 + LabelKeys: []string{

5529 + "group",

5530 + },

5531 + },

5532 + },

5533 + State: TablespaceGroupStateContext{

5534 + Context: framework.Context[TablespaceGroupLabels]{

5535 + Name: "db2.tablespace_group_state",

5536 + Family: "tablespaces/groups",

5537 + Title: "Tablespace Group State",

5538 + Units: "state",

5539 + Type: module.Line,

5540 + Priority: 1127,

5541 + UpdateEvery: 1,

5542 + Dimensions: []framework.Dimension{

5543 + {

5544 + Name: "state",

5545 + Algorithm: module.Absolute,

5546 + Mul: 1,

5547 + Div: 1,

5548 + Precision: 1,

5549 + },

5550 + },

5551 + LabelKeys: []string{

5552 + "group",

5553 + },

5554 + },

5555 + },

5556 +}

5557 +

5558 // GetAllContexts returns all contexts for framework registration

5559 func GetAllContexts() []interface{} {

5560 return []interface{}{

@@ -4367,13 +5565,26 @@ func GetAllContexts() []interface{} {

5565 &Bufferpool.IndexReads.Context,

5566 &Bufferpool.Pages.Context,

5567 &Bufferpool.Writes.Context,

5568 + &BufferpoolGroup.HitRatio.Context,

5569 + &BufferpoolGroup.DetailedHitRatio.Context,

5570 + &BufferpoolGroup.Reads.Context,

5571 + &BufferpoolGroup.DataReads.Context,

5572 + &BufferpoolGroup.IndexReads.Context,

5573 + &BufferpoolGroup.Pages.Context,

5574 + &BufferpoolGroup.Writes.Context,

5575 &Connection.State.Context,

5576 &Connection.Activity.Context,

5577 &Connection.WaitTime.Context,

5578 &Connection.ProcessingTime.Context,

5579 + &ConnectionGroup.Count.Context,

5580 + &ConnectionGroup.State.Context,

5581 + &ConnectionGroup.Activity.Context,

5582 + &ConnectionGroup.WaitTime.Context,

5583 + &ConnectionGroup.ProcessingTime.Context,

5584 &Database.Status.Context,

5585 &Database.Applications.Context,

5586 &Index.Usage.Context,

5587 + &IndexGroup.Usage.Context,

5588 &MemoryPool.Usage.Context,

5589 &MemoryPool.HighWaterMark.Context,

5590 &MemorySet.Usage.Context,

@@ -4427,6 +5638,8 @@ func GetAllContexts() []interface{} {

5638 &System.TimeSpent.Context,

5639 &Table.Size.Context,

5640 &Table.Activity.Context,

5641 + &TableGroup.Size.Context,

5642 + &TableGroup.Activity.Context,

5643 &TableIO.Scans.Context,

5644 &TableIO.Rows.Context,

5645 &TableIO.Activity.Context,

@@ -4435,5 +5648,9 @@ func GetAllContexts() []interface{} {

5648 &Tablespace.Size.Context,

5649 &Tablespace.UsableSize.Context,

5650 &Tablespace.State.Context,

5651 + &TablespaceGroup.Usage.Context,

5652 + &TablespaceGroup.Size.Context,

5653 + &TablespaceGroup.UsableSize.Context,

5654 + &TablespaceGroup.State.Context,

5655 }

5656 }

src/go/plugin/ibm.d/modules/db2/export_bufferpool.go

+297 -35

@@ -4,89 +4,351 @@ package db2

5 import (

6 "fmt"

7 + "sort"

8 + "strings"

10 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

11 )

13 +type bufferpoolEntry struct {

14 + name string

15 + meta *bufferpoolMetrics

16 + metrics bufferpoolInstanceMetrics

17 +}

18 +

19 +type bufferpoolGroupAggregate struct {

20 + Hits, Misses int64

21 + DataHits, DataMisses int64

22 + IndexHits, IndexMisses int64

23 + XDAHits, XDAMisses int64

24 + ColumnHits, ColumnMisses int64

25 +

26 + LogicalReads, PhysicalReads int64

27 + DataLogicalReads, DataPhysicalReads int64

28 + IndexLogicalReads, IndexPhysicalReads int64

29 +

30 + UsedPages, TotalPages int64

31 + Writes int64

32 +}

33 +

34 +func bufferpoolGroupKey(name string) string {

35 + if name == "" {

36 + return "__unknown__"

37 + }

38 + up := strings.ToUpper(name)

39 + if idx := strings.Index(up, "_"); idx > 0 {

40 + return up[:idx]

41 + }

42 + return up

43 +}

44 +

45 +func (a *bufferpoolGroupAggregate) add(m bufferpoolInstanceMetrics) {

46 + a.Hits += m.Hits

47 + a.Misses += m.Misses

48 + a.DataHits += m.DataHits

49 + a.DataMisses += m.DataMisses

50 + a.IndexHits += m.IndexHits

51 + a.IndexMisses += m.IndexMisses

52 + a.XDAHits += m.XDAHits

53 + a.XDAMisses += m.XDAMisses

54 + a.ColumnHits += m.ColumnHits

55 + a.ColumnMisses += m.ColumnMisses

56 +

57 + a.LogicalReads += m.LogicalReads

58 + a.PhysicalReads += m.PhysicalReads

59 + a.DataLogicalReads += m.DataLogicalReads

60 + a.DataPhysicalReads += m.DataPhysicalReads

61 + a.IndexLogicalReads += m.IndexLogicalReads

62 + a.IndexPhysicalReads += m.IndexPhysicalReads

63 +

64 + a.UsedPages += m.UsedPages

65 + a.TotalPages += m.TotalPages

66 + a.Writes += m.Writes

67 +}

68 +

69 func (c *Collector) exportBufferpoolMetrics(mx metricsData) {

70 + entries := make([]bufferpoolEntry, 0, len(mx.bufferpools))

71 for name, metrics := range mx.bufferpools {

13 - meta := c.bufferpools[name]

14 - pageSizeLabel := "unknown"

15 - if meta != nil && meta.pageSize > 0 {

16 - pageSizeLabel = fmt.Sprintf("%d", meta.pageSize)

72 + entries = append(entries, bufferpoolEntry{

73 + name: name,

74 + meta: c.bufferpools[name],

75 + metrics: metrics,

76 + })

77 + }

78 +

79 + if len(entries) == 0 {

80 + c.clearWarnOnce("db2_bufferpool_overflow")

81 + return

82 + }

83 +

84 + sort.Slice(entries, func(i, j int) bool {

85 + return entries[i].name < entries[j].name

86 + })

87 +

88 + limit := c.MaxBufferpools

89 + if limit <= 0 || limit > len(entries) {

90 + limit = len(entries)

91 + }

92 +

93 + groupAgg := make(map[string]*bufferpoolGroupAggregate)

94 + overflowAgg := &bufferpoolGroupAggregate{}

95 + overflowCount := 0

96 + overflowGroups := make(map[string]int)

97 + overflowExample := make(map[string]string)

98 +

99 + for idx, entry := range entries {

100 + groupKey := bufferpoolGroupKey(entry.name)

101 + agg := groupAgg[groupKey]

102 + if agg == nil {

103 + agg = &bufferpoolGroupAggregate{}

104 + groupAgg[groupKey] = agg

105 }

106 + agg.add(entry.metrics)

107

19 - labels := contexts.BufferpoolLabels{

20 - Bufferpool: name,

21 - Page_size: pageSizeLabel,

108 + if idx < limit {

109 + c.emitPerBufferpoolMetrics(entry)

110 + continue

111 }

112

24 - totalReads := metrics.Hits + metrics.Misses

113 + overflowAgg.add(entry.metrics)

114 + overflowCount++

115 + overflowGroups[groupKey]++

116 + if _, ok := overflowExample[groupKey]; !ok {

117 + overflowExample[groupKey] = entry.name

118 + }

119 + }

120 +

121 + c.emitBufferpoolGroupMetrics(groupAgg, overflowAgg, overflowCount)

122 +

123 + if overflowCount > 0 {

124 + parts := make([]string, 0, len(overflowGroups))

125 + for group, count := range overflowGroups {

126 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, overflowExample[group]))

127 + }

128 + sort.Strings(parts)

129 + c.warnOnce("db2_bufferpool_overflow", "too many buffer pools for per-instance charts (MaxBufferpools=%d). Aggregated %d additional pools: %s", c.MaxBufferpools, overflowCount, strings.Join(parts, ", "))

130 + } else {

131 + c.clearWarnOnce("db2_bufferpool_overflow")

132 + }

133 +}

134 +

135 +func (c *Collector) emitPerBufferpoolMetrics(entry bufferpoolEntry) {

136 + pageSizeLabel := "unknown"

137 + if entry.meta != nil && entry.meta.pageSize > 0 {

138 + pageSizeLabel = fmt.Sprintf("%d", entry.meta.pageSize)

139 + }

140 +

141 + labels := contexts.BufferpoolLabels{

142 + Bufferpool: entry.name,

143 + Page_size: pageSizeLabel,

144 + }

145 +

146 + totalReads := entry.metrics.Hits + entry.metrics.Misses

147 + overallRatio := int64(0)

148 + if totalReads > 0 {

149 + overallRatio = entry.metrics.Hits * 100 * Precision / totalReads

150 + }

151 +

152 + dataReads := entry.metrics.DataHits + entry.metrics.DataMisses

153 + dataRatio := int64(0)

154 + if dataReads > 0 {

155 + dataRatio = entry.metrics.DataHits * 100 * Precision / dataReads

156 + }

157 +

158 + indexReads := entry.metrics.IndexHits + entry.metrics.IndexMisses

159 + indexRatio := int64(0)

160 + if indexReads > 0 {

161 + indexRatio = entry.metrics.IndexHits * 100 * Precision / indexReads

162 + }

163 +

164 + xdaReads := entry.metrics.XDAHits + entry.metrics.XDAMisses

165 + xdaRatio := int64(0)

166 + if xdaReads > 0 {

167 + xdaRatio = entry.metrics.XDAHits * 100 * Precision / xdaReads

168 + }

169 +

170 + columnReads := entry.metrics.ColumnHits + entry.metrics.ColumnMisses

171 + columnRatio := int64(0)

172 + if columnReads > 0 {

173 + columnRatio = entry.metrics.ColumnHits * 100 * Precision / columnReads

174 + }

175 +

176 + contexts.Bufferpool.HitRatio.Set(c.State, labels, contexts.BufferpoolHitRatioValues{

177 + Overall: overallRatio,

178 + })

179 +

180 + contexts.Bufferpool.DetailedHitRatio.Set(c.State, labels, contexts.BufferpoolDetailedHitRatioValues{

181 + Data: dataRatio,

182 + Index: indexRatio,

183 + Xda: xdaRatio,

184 + Column: columnRatio,

185 + })

186 +

187 + contexts.Bufferpool.Reads.Set(c.State, labels, contexts.BufferpoolReadsValues{

188 + Logical: entry.metrics.LogicalReads,

189 + Physical: entry.metrics.PhysicalReads,

190 + })

191 +

192 + contexts.Bufferpool.DataReads.Set(c.State, labels, contexts.BufferpoolDataReadsValues{

193 + Logical: entry.metrics.DataLogicalReads,

194 + Physical: entry.metrics.DataPhysicalReads,

195 + })

196 +

197 + contexts.Bufferpool.IndexReads.Set(c.State, labels, contexts.BufferpoolIndexReadsValues{

198 + Logical: entry.metrics.IndexLogicalReads,

199 + Physical: entry.metrics.IndexPhysicalReads,

200 + })

201 +

202 + contexts.Bufferpool.Pages.Set(c.State, labels, contexts.BufferpoolPagesValues{

203 + Used: entry.metrics.UsedPages,

204 + Total: entry.metrics.TotalPages,

205 + })

206 +

207 + contexts.Bufferpool.Writes.Set(c.State, labels, contexts.BufferpoolWritesValues{

208 + Writes: entry.metrics.Writes,

209 + })

210 +

211 + contexts.Bufferpool.DataReads.SetUpdateEvery(c.State, labels, c.Config.UpdateEvery)

212 + contexts.Bufferpool.IndexReads.SetUpdateEvery(c.State, labels, c.Config.UpdateEvery)

213 +}

214 +

215 +func (c *Collector) emitBufferpoolGroupMetrics(groups map[string]*bufferpoolGroupAggregate, overflow *bufferpoolGroupAggregate, overflowCount int) {

216 + keys := make([]string, 0, len(groups))

217 + for k := range groups {

218 + keys = append(keys, k)

219 + }

220 + sort.Strings(keys)

221 +

222 + for _, key := range keys {

223 + agg := groups[key]

224 + labels := contexts.BufferpoolGroupLabels{Group: key}

225 +

226 + totalReads := agg.Hits + agg.Misses

227 overallRatio := int64(0)

228 if totalReads > 0 {

27 - overallRatio = metrics.Hits * 100 * Precision / totalReads

229 + overallRatio = agg.Hits * 100 * Precision / totalReads

230 }

231

30 - dataReads := metrics.DataHits + metrics.DataMisses

232 + dataReads := agg.DataHits + agg.DataMisses

233 dataRatio := int64(0)

234 if dataReads > 0 {

33 - dataRatio = metrics.DataHits * 100 * Precision / dataReads

235 + dataRatio = agg.DataHits * 100 * Precision / dataReads

236 }

237

36 - indexReads := metrics.IndexHits + metrics.IndexMisses

238 + indexReads := agg.IndexHits + agg.IndexMisses

239 indexRatio := int64(0)

240 if indexReads > 0 {

39 - indexRatio = metrics.IndexHits * 100 * Precision / indexReads

241 + indexRatio = agg.IndexHits * 100 * Precision / indexReads

242 }

243

42 - xdaReads := metrics.XDAHits + metrics.XDAMisses

244 + xdaReads := agg.XDAHits + agg.XDAMisses

245 xdaRatio := int64(0)

246 if xdaReads > 0 {

45 - xdaRatio = metrics.XDAHits * 100 * Precision / xdaReads

247 + xdaRatio = agg.XDAHits * 100 * Precision / xdaReads

248 }

249

48 - columnReads := metrics.ColumnHits + metrics.ColumnMisses

250 + columnReads := agg.ColumnHits + agg.ColumnMisses

251 columnRatio := int64(0)

252 if columnReads > 0 {

51 - columnRatio = metrics.ColumnHits * 100 * Precision / columnReads

253 + columnRatio = agg.ColumnHits * 100 * Precision / columnReads

254 }

255

54 - contexts.Bufferpool.HitRatio.Set(c.State, labels, contexts.BufferpoolHitRatioValues{

256 + contexts.BufferpoolGroup.HitRatio.Set(c.State, labels, contexts.BufferpoolGroupHitRatioValues{

257 Overall: overallRatio,

258 })

259

58 - contexts.Bufferpool.DetailedHitRatio.Set(c.State, labels, contexts.BufferpoolDetailedHitRatioValues{

260 + contexts.BufferpoolGroup.DetailedHitRatio.Set(c.State, labels, contexts.BufferpoolGroupDetailedHitRatioValues{

261 Data: dataRatio,

262 Index: indexRatio,

263 Xda: xdaRatio,

264 Column: columnRatio,

265 })

266

65 - contexts.Bufferpool.Reads.Set(c.State, labels, contexts.BufferpoolReadsValues{

66 - Logical: metrics.LogicalReads,

67 - Physical: metrics.PhysicalReads,

267 + contexts.BufferpoolGroup.Reads.Set(c.State, labels, contexts.BufferpoolGroupReadsValues{

268 + Logical: agg.LogicalReads,

269 + Physical: agg.PhysicalReads,

270 })

271

70 - contexts.Bufferpool.DataReads.Set(c.State, labels, contexts.BufferpoolDataReadsValues{

71 - Logical: metrics.DataLogicalReads,

72 - Physical: metrics.DataPhysicalReads,

272 + contexts.BufferpoolGroup.DataReads.Set(c.State, labels, contexts.BufferpoolGroupDataReadsValues{

273 + Logical: agg.DataLogicalReads,

274 + Physical: agg.DataPhysicalReads,

275 })

276

75 - contexts.Bufferpool.IndexReads.Set(c.State, labels, contexts.BufferpoolIndexReadsValues{

76 - Logical: metrics.IndexLogicalReads,

77 - Physical: metrics.IndexPhysicalReads,

277 + contexts.BufferpoolGroup.IndexReads.Set(c.State, labels, contexts.BufferpoolGroupIndexReadsValues{

278 + Logical: agg.IndexLogicalReads,

279 + Physical: agg.IndexPhysicalReads,

280 })

281

80 - contexts.Bufferpool.Pages.Set(c.State, labels, contexts.BufferpoolPagesValues{

81 - Used: metrics.UsedPages,

82 - Total: metrics.TotalPages,

282 + contexts.BufferpoolGroup.Pages.Set(c.State, labels, contexts.BufferpoolGroupPagesValues{

283 + Used: agg.UsedPages,

284 + Total: agg.TotalPages,

285 })

286

85 - contexts.Bufferpool.Writes.Set(c.State, labels, contexts.BufferpoolWritesValues{

86 - Writes: metrics.Writes,

287 + contexts.BufferpoolGroup.Writes.Set(c.State, labels, contexts.BufferpoolGroupWritesValues{

288 + Writes: agg.Writes,

289 })

290 + }

291

89 - contexts.Bufferpool.DataReads.SetUpdateEvery(c.State, labels, c.Config.UpdateEvery)

90 - contexts.Bufferpool.IndexReads.SetUpdateEvery(c.State, labels, c.Config.UpdateEvery)

292 + if overflowCount > 0 && overflow != nil {

293 + labels := contexts.BufferpoolGroupLabels{Group: "__other__"}

294 +

295 + totalReads := overflow.Hits + overflow.Misses

296 + overallRatio := int64(0)

297 + if totalReads > 0 {

298 + overallRatio = overflow.Hits * 100 * Precision / totalReads

299 + }

300 +

301 + dataReads := overflow.DataHits + overflow.DataMisses

302 + dataRatio := int64(0)

303 + if dataReads > 0 {

304 + dataRatio = overflow.DataHits * 100 * Precision / dataReads

305 + }

306 +

307 + indexReads := overflow.IndexHits + overflow.IndexMisses

308 + indexRatio := int64(0)

309 + if indexReads > 0 {

310 + indexRatio = overflow.IndexHits * 100 * Precision / indexReads

311 + }

312 +

313 + xdaReads := overflow.XDAHits + overflow.XDAMisses

314 + xdaRatio := int64(0)

315 + if xdaReads > 0 {

316 + xdaRatio = overflow.XDAHits * 100 * Precision / xdaReads

317 + }

318 +

319 + columnReads := overflow.ColumnHits + overflow.ColumnMisses

320 + columnRatio := int64(0)

321 + if columnReads > 0 {

322 + columnRatio = overflow.ColumnHits * 100 * Precision / columnReads

323 + }

324 +

325 + contexts.BufferpoolGroup.HitRatio.Set(c.State, labels, contexts.BufferpoolGroupHitRatioValues{

326 + Overall: overallRatio,

327 + })

328 + contexts.BufferpoolGroup.DetailedHitRatio.Set(c.State, labels, contexts.BufferpoolGroupDetailedHitRatioValues{

329 + Data: dataRatio,

330 + Index: indexRatio,

331 + Xda: xdaRatio,

332 + Column: columnRatio,

333 + })

334 + contexts.BufferpoolGroup.Reads.Set(c.State, labels, contexts.BufferpoolGroupReadsValues{

335 + Logical: overflow.LogicalReads,

336 + Physical: overflow.PhysicalReads,

337 + })

338 + contexts.BufferpoolGroup.DataReads.Set(c.State, labels, contexts.BufferpoolGroupDataReadsValues{

339 + Logical: overflow.DataLogicalReads,

340 + Physical: overflow.DataPhysicalReads,

341 + })

342 + contexts.BufferpoolGroup.IndexReads.Set(c.State, labels, contexts.BufferpoolGroupIndexReadsValues{

343 + Logical: overflow.IndexLogicalReads,

344 + Physical: overflow.IndexPhysicalReads,

345 + })

346 + contexts.BufferpoolGroup.Pages.Set(c.State, labels, contexts.BufferpoolGroupPagesValues{

347 + Used: overflow.UsedPages,

348 + Total: overflow.TotalPages,

349 + })

350 + contexts.BufferpoolGroup.Writes.Set(c.State, labels, contexts.BufferpoolGroupWritesValues{

351 + Writes: overflow.Writes,

352 + })

353 }

354 }

src/go/plugin/ibm.d/modules/db2/export_connection.go

+267 -50

@@ -2,60 +2,277 @@

3 package db2

5 -import "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

5 +import (

6 + "fmt"

7 + "sort"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

11 +)

12 +

13 +type connectionEntry struct {

14 + id string

15 + meta *connectionMetrics

16 + metrics connectionInstanceMetrics

17 +}

18 +

19 +type connectionGroupAggregate struct {

20 + Count int64

21 +

22 + State int64

23 + Executing int64

24 + RowsRead int64

25 + RowsWritten int64

26 + TotalCPUTime int64

27 + LockWaitTime int64

28 + LogDiskWaitTime int64

29 + LogBufferWaitTime int64

30 + PoolReadTime int64

31 + PoolWriteTime int64

32 + DirectReadTime int64

33 + DirectWriteTime int64

34 + FCMRecvWaitTime int64

35 + FCMSendWaitTime int64

36 + RoutineTime int64

37 + CompileTime int64

38 + SectionTime int64

39 + CommitTime int64

40 + RollbackTime int64

41 +}

42 +

43 +func connectionGroupKey(meta *connectionMetrics) string {

44 + if meta == nil {

45 + return "__unknown__"

46 + }

47 + name := strings.TrimSpace(meta.applicationName)

48 + if name == "" || name == "-" {

49 + name = meta.applicationID

50 + }

51 + if name == "" {

52 + name = "UNKNOWN"

53 + }

54 + fields := strings.Fields(name)

55 + key := strings.ToUpper(fields[0])

56 + if key == "" {

57 + key = "UNKNOWN"

58 + }

59 + return key

60 +}

61 +

62 +func (a *connectionGroupAggregate) add(m connectionInstanceMetrics) {

63 + a.Count++

64 + a.State += m.State

65 + a.Executing += m.ExecutingQueries

66 + a.RowsRead += m.RowsRead

67 + a.RowsWritten += m.RowsWritten

68 + a.TotalCPUTime += m.TotalCPUTime

69 + a.LockWaitTime += m.LockWaitTime

70 + a.LogDiskWaitTime += m.LogDiskWaitTime

71 + a.LogBufferWaitTime += m.LogBufferWaitTime

72 + a.PoolReadTime += m.PoolReadTime

73 + a.PoolWriteTime += m.PoolWriteTime

74 + a.DirectReadTime += m.DirectReadTime

75 + a.DirectWriteTime += m.DirectWriteTime

76 + a.FCMRecvWaitTime += m.FCMRecvWaitTime

77 + a.FCMSendWaitTime += m.FCMSendWaitTime

78 + a.RoutineTime += m.TotalRoutineTime

79 + a.CompileTime += m.TotalCompileTime

80 + a.SectionTime += m.TotalSectionTime

81 + a.CommitTime += m.TotalCommitTime

82 + a.RollbackTime += m.TotalRollbackTime

83 +}

85 func (c *Collector) exportConnectionMetrics() {

86 + entries := make([]connectionEntry, 0, len(c.mx.connections))

87 for id, metrics := range c.mx.connections {

88 meta := c.connections[id]

10 - labels := contexts.ConnectionLabels{

11 - Application_id: id,

12 - }

13 -

14 - if meta != nil {

15 - if meta.applicationName != "" && meta.applicationName != "-" {

16 - labels.Application_name = meta.applicationName

17 - }

18 - if meta.clientHostname != "" && meta.clientHostname != "-" {

19 - labels.Client_hostname = meta.clientHostname

20 - }

21 - if meta.clientIP != "" && meta.clientIP != "-" {

22 - labels.Client_ip = meta.clientIP

23 - }

24 - if meta.clientUser != "" && meta.clientUser != "-" {

25 - labels.Client_user = meta.clientUser

26 - }

27 - if meta.connectionState != "" {

28 - labels.State = meta.connectionState

29 - }

30 - }

31 -

32 - contexts.Connection.State.Set(c.State, labels, contexts.ConnectionStateValues{

33 - State: metrics.State,

34 - })

35 -

36 - contexts.Connection.Activity.Set(c.State, labels, contexts.ConnectionActivityValues{

37 - Read: metrics.RowsRead,

38 - Written: metrics.RowsWritten,

39 - })

40 -

41 - contexts.Connection.WaitTime.Set(c.State, labels, contexts.ConnectionWaitTimeValues{

42 - Lock: metrics.LockWaitTime,

43 - Log_disk: metrics.LogDiskWaitTime,

44 - Log_buffer: metrics.LogBufferWaitTime,

45 - Pool_read: metrics.PoolReadTime,

46 - Pool_write: metrics.PoolWriteTime,

47 - Direct_read: metrics.DirectReadTime,

48 - Direct_write: metrics.DirectWriteTime,

49 - Fcm_recv: metrics.FCMRecvWaitTime,

50 - Fcm_send: metrics.FCMSendWaitTime,

51 - })

52 -

53 - contexts.Connection.ProcessingTime.Set(c.State, labels, contexts.ConnectionProcessingTimeValues{

54 - Routine: metrics.TotalRoutineTime,

55 - Compile: metrics.TotalCompileTime,

56 - Section: metrics.TotalSectionTime,

57 - Commit: metrics.TotalCommitTime,

58 - Rollback: metrics.TotalRollbackTime,

89 + entries = append(entries, connectionEntry{

90 + id: id,

91 + meta: meta,

92 + metrics: metrics,

93 + })

94 + }

95 +

96 + if len(entries) == 0 {

97 + c.clearWarnOnce("db2_connection_overflow")

98 + return

99 + }

100 +

101 + sort.Slice(entries, func(i, j int) bool {

102 + return entries[i].id < entries[j].id

103 + })

104 +

105 + limit := c.MaxConnections

106 + if limit <= 0 || limit > len(entries) {

107 + limit = len(entries)

108 + }

109 +

110 + groupAgg := make(map[string]*connectionGroupAggregate)

111 + overflowAgg := &connectionGroupAggregate{}

112 + overflowCount := 0

113 + overflowGroups := make(map[string]int)

114 + overflowExample := make(map[string]string)

115 +

116 + for idx, entry := range entries {

117 + key := connectionGroupKey(entry.meta)

118 + agg := groupAgg[key]

119 + if agg == nil {

120 + agg = &connectionGroupAggregate{}

121 + groupAgg[key] = agg

122 + }

123 + agg.add(entry.metrics)

124 +

125 + if idx < limit {

126 + c.emitPerConnectionMetrics(entry)

127 + continue

128 + }

129 +

130 + overflowAgg.add(entry.metrics)

131 + overflowCount++

132 + overflowGroups[key]++

133 + if _, ok := overflowExample[key]; !ok {

134 + overflowExample[key] = entry.id

135 + }

136 + }

137 +

138 + c.emitConnectionGroupMetrics(groupAgg, overflowAgg, overflowCount)

139 +

140 + if overflowCount > 0 {

141 + parts := make([]string, 0, len(overflowGroups))

142 + for group, count := range overflowGroups {

143 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, overflowExample[group]))

144 + }

145 + sort.Strings(parts)

146 + c.warnOnce("db2_connection_overflow", "too many connections for per-connection charts (MaxConnections=%d). Aggregated %d additional connections: %s", c.MaxConnections, overflowCount, strings.Join(parts, ", "))

147 + } else {

148 + c.clearWarnOnce("db2_connection_overflow")

149 + }

150 +}

151 +

152 +func (c *Collector) emitPerConnectionMetrics(entry connectionEntry) {

153 + labels := contexts.ConnectionLabels{

154 + Application_id: entry.id,

155 + }

156 +

157 + if entry.meta != nil {

158 + if entry.meta.applicationName != "" && entry.meta.applicationName != "-" {

159 + labels.Application_name = entry.meta.applicationName

160 + }

161 + if entry.meta.clientHostname != "" && entry.meta.clientHostname != "-" {

162 + labels.Client_hostname = entry.meta.clientHostname

163 + }

164 + if entry.meta.clientIP != "" && entry.meta.clientIP != "-" {

165 + labels.Client_ip = entry.meta.clientIP

166 + }

167 + if entry.meta.clientUser != "" && entry.meta.clientUser != "-" {

168 + labels.Client_user = entry.meta.clientUser

169 + }

170 + if entry.meta.connectionState != "" {

171 + labels.State = entry.meta.connectionState

172 + }

173 + }

174 +

175 + contexts.Connection.State.Set(c.State, labels, contexts.ConnectionStateValues{

176 + State: entry.metrics.State,

177 + })

178 +

179 + contexts.Connection.Activity.Set(c.State, labels, contexts.ConnectionActivityValues{

180 + Read: entry.metrics.RowsRead,

181 + Written: entry.metrics.RowsWritten,

182 + })

183 +

184 + contexts.Connection.WaitTime.Set(c.State, labels, contexts.ConnectionWaitTimeValues{

185 + Lock: entry.metrics.LockWaitTime,

186 + Log_disk: entry.metrics.LogDiskWaitTime,

187 + Log_buffer: entry.metrics.LogBufferWaitTime,

188 + Pool_read: entry.metrics.PoolReadTime,

189 + Pool_write: entry.metrics.PoolWriteTime,

190 + Direct_read: entry.metrics.DirectReadTime,

191 + Direct_write: entry.metrics.DirectWriteTime,

192 + Fcm_recv: entry.metrics.FCMRecvWaitTime,

193 + Fcm_send: entry.metrics.FCMSendWaitTime,

194 + })

195 +

196 + contexts.Connection.ProcessingTime.Set(c.State, labels, contexts.ConnectionProcessingTimeValues{

197 + Routine: entry.metrics.TotalRoutineTime,

198 + Compile: entry.metrics.TotalCompileTime,

199 + Section: entry.metrics.TotalSectionTime,

200 + Commit: entry.metrics.TotalCommitTime,

201 + Rollback: entry.metrics.TotalRollbackTime,

202 + })

203 +}

204 +

205 +func (c *Collector) emitConnectionGroupMetrics(groups map[string]*connectionGroupAggregate, overflow *connectionGroupAggregate, overflowCount int) {

206 + keys := make([]string, 0, len(groups))

207 + for k := range groups {

208 + keys = append(keys, k)

209 + }

210 + sort.Strings(keys)

211 +

212 + for _, key := range keys {

213 + agg := groups[key]

214 + labels := contexts.ConnectionGroupLabels{Group: key}

215 +

216 + contexts.ConnectionGroup.Count.Set(c.State, labels, contexts.ConnectionGroupCountValues{

217 + Count: agg.Count,

218 + })

219 +

220 + contexts.ConnectionGroup.State.Set(c.State, labels, contexts.ConnectionGroupStateValues{

221 + State: agg.State,

222 + })

223 +

224 + contexts.ConnectionGroup.Activity.Set(c.State, labels, contexts.ConnectionGroupActivityValues{

225 + Read: agg.RowsRead,

226 + Written: agg.RowsWritten,

227 + })

228 +

229 + contexts.ConnectionGroup.WaitTime.Set(c.State, labels, contexts.ConnectionGroupWaitTimeValues{

230 + Lock: agg.LockWaitTime,

231 + Log_disk: agg.LogDiskWaitTime,

232 + Log_buffer: agg.LogBufferWaitTime,

233 + Pool_read: agg.PoolReadTime,

234 + Pool_write: agg.PoolWriteTime,

235 + Direct_read: agg.DirectReadTime,

236 + Direct_write: agg.DirectWriteTime,

237 + Fcm_recv: agg.FCMRecvWaitTime,

238 + Fcm_send: agg.FCMSendWaitTime,

239 + })

240 +

241 + contexts.ConnectionGroup.ProcessingTime.Set(c.State, labels, contexts.ConnectionGroupProcessingTimeValues{

242 + Routine: agg.RoutineTime,

243 + Compile: agg.CompileTime,

244 + Section: agg.SectionTime,

245 + Commit: agg.CommitTime,

246 + Rollback: agg.RollbackTime,

247 + })

248 + }

249 +

250 + if overflowCount > 0 && overflow != nil && overflow.Count > 0 {

251 + labels := contexts.ConnectionGroupLabels{Group: "__other__"}

252 +

253 + contexts.ConnectionGroup.Count.Set(c.State, labels, contexts.ConnectionGroupCountValues{Count: overflow.Count})

254 + contexts.ConnectionGroup.State.Set(c.State, labels, contexts.ConnectionGroupStateValues{State: overflow.State})

255 + contexts.ConnectionGroup.Activity.Set(c.State, labels, contexts.ConnectionGroupActivityValues{

256 + Read: overflow.RowsRead,

257 + Written: overflow.RowsWritten,

258 + })

259 + contexts.ConnectionGroup.WaitTime.Set(c.State, labels, contexts.ConnectionGroupWaitTimeValues{

260 + Lock: overflow.LockWaitTime,

261 + Log_disk: overflow.LogDiskWaitTime,

262 + Log_buffer: overflow.LogBufferWaitTime,

263 + Pool_read: overflow.PoolReadTime,

264 + Pool_write: overflow.PoolWriteTime,

265 + Direct_read: overflow.DirectReadTime,

266 + Direct_write: overflow.DirectWriteTime,

267 + Fcm_recv: overflow.FCMRecvWaitTime,

268 + Fcm_send: overflow.FCMSendWaitTime,

269 + })

270 + contexts.ConnectionGroup.ProcessingTime.Set(c.State, labels, contexts.ConnectionGroupProcessingTimeValues{

271 + Routine: overflow.RoutineTime,

272 + Compile: overflow.CompileTime,

273 + Section: overflow.SectionTime,

274 + Commit: overflow.CommitTime,

275 + Rollback: overflow.RollbackTime,

276 })

277 }

278 }

src/go/plugin/ibm.d/modules/db2/export_index.go

+115 -6

@@ -2,17 +2,126 @@

3 package db2

5 -import "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

5 +import (

6 + "fmt"

7 + "sort"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

11 +)

12 +

13 +type indexEntry struct {

14 + name string

15 + metrics indexInstanceMetrics

16 +}

17 +

18 +type indexGroupAggregate struct {

19 + LeafNodes int64

20 + IndexScans int64

21 + FullScans int64

22 +}

23 +

24 +func indexGroupKey(name string) string {

25 + if idx := strings.Index(name, "."); idx > 0 {

26 + return name[:idx]

27 + }

28 + return name

29 +}

30 +

31 +func (a *indexGroupAggregate) add(m indexInstanceMetrics) {

32 + a.LeafNodes += m.LeafNodes

33 + a.IndexScans += m.IndexScans

34 + a.FullScans += m.FullScans

35 +}

37 func (c *Collector) exportIndexMetrics() {

38 + entries := make([]indexEntry, 0, len(c.mx.indexes))

39 for name, metrics := range c.mx.indexes {

9 - labels := contexts.IndexLabels{

10 - Index: name,

40 + entries = append(entries, indexEntry{name: name, metrics: metrics})

41 + }

42 +

43 + if len(entries) == 0 {

44 + c.clearWarnOnce("db2_index_overflow")

45 + return

46 + }

47 +

48 + sort.Slice(entries, func(i, j int) bool {

49 + return entries[i].name < entries[j].name

50 + })

51 +

52 + limit := c.MaxIndexes

53 + if limit <= 0 || limit > len(entries) {

54 + limit = len(entries)

55 + }

56 +

57 + groupAgg := make(map[string]*indexGroupAggregate)

58 + overflowAgg := &indexGroupAggregate{}

59 + overflowCount := 0

60 + overflowGroups := make(map[string]int)

61 + overflowExample := make(map[string]string)

62 +

63 + for idx, entry := range entries {

64 + key := indexGroupKey(entry.name)

65 + agg := groupAgg[key]

66 + if agg == nil {

67 + agg = &indexGroupAggregate{}

68 + groupAgg[key] = agg

69 + }

70 + agg.add(entry.metrics)

71 +

72 + if idx < limit {

73 + labels := contexts.IndexLabels{Index: entry.name}

74 + contexts.Index.Usage.Set(c.State, labels, contexts.IndexUsageValues{

75 + Index: entry.metrics.IndexScans,

76 + Full: entry.metrics.FullScans,

77 + })

78 + continue

79 + }

80 +

81 + overflowAgg.add(entry.metrics)

82 + overflowCount++

83 + overflowGroups[key]++

84 + if _, ok := overflowExample[key]; !ok {

85 + overflowExample[key] = entry.name

86 + }

87 + }

88 +

89 + c.emitIndexGroupMetrics(groupAgg, overflowAgg, overflowCount)

90 +

91 + if overflowCount > 0 {

92 + parts := make([]string, 0, len(overflowGroups))

93 + for group, count := range overflowGroups {

94 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, overflowExample[group]))

95 }

96 + sort.Strings(parts)

97 + c.warnOnce("db2_index_overflow", "too many indexes for per-instance charts (MaxIndexes=%d). Aggregated %d additional indexes: %s", c.MaxIndexes, overflowCount, strings.Join(parts, ", "))

98 + } else {

99 + c.clearWarnOnce("db2_index_overflow")

100 + }

101 +}

102 +

103 +func (c *Collector) emitIndexGroupMetrics(groups map[string]*indexGroupAggregate, overflow *indexGroupAggregate, overflowCount int) {

104 + keys := make([]string, 0, len(groups))

105 + for k := range groups {

106 + keys = append(keys, k)

107 + }

108 + sort.Strings(keys)

109 +

110 + for _, key := range keys {

111 + agg := groups[key]

112 + labels := contexts.IndexGroupLabels{Group: key}

113 +

114 + contexts.IndexGroup.Usage.Set(c.State, labels, contexts.IndexGroupUsageValues{

115 + Index: agg.IndexScans,

116 + Full: agg.FullScans,

117 + })

118 + }

119

13 - contexts.Index.Usage.Set(c.State, labels, contexts.IndexUsageValues{

14 - Index: metrics.IndexScans,

15 - Full: metrics.FullScans,

120 + if overflowCount > 0 && overflow != nil {

121 + labels := contexts.IndexGroupLabels{Group: "__other__"}

122 + contexts.IndexGroup.Usage.Set(c.State, labels, contexts.IndexGroupUsageValues{

123 + Index: overflow.IndexScans,

124 + Full: overflow.FullScans,

125 })

126 }

127 }

src/go/plugin/ibm.d/modules/db2/export_system.go

+8 -3

@@ -7,6 +7,11 @@ import "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

7 func (c *Collector) exportSystemMetrics() {

8 labels := contexts.EmptyLabels{}

10 + interval := int64(c.Config.UpdateEvery)

11 + if interval <= 0 {

12 + interval = 1

13 + }

14 +

15 contexts.System.ServiceHealth.Set(c.State, labels, contexts.SystemServiceHealthValues{

16 Connection: c.mx.CanConnect,

17 Database: c.mx.DatabaseStatus,

@@ -46,9 +51,9 @@ func (c *Collector) exportSystemMetrics() {

51 })

53 contexts.System.RowActivity.Set(c.State, labels, contexts.SystemRowActivityValues{

49 - Read: c.mx.RowsRead,

50 - Returned: c.mx.RowsReturned,

51 - Modified: c.mx.RowsModified,

54 + Read: c.mx.RowsRead / interval,

55 + Returned: c.mx.RowsReturned / interval,

56 + Modified: c.mx.RowsModified / interval,

57 })

59 contexts.System.BufferpoolHitRatio.Set(c.State, labels, contexts.SystemBufferpoolHitRatioValues{

src/go/plugin/ibm.d/modules/db2/export_table.go

+135 -10

@@ -2,23 +2,148 @@

3 package db2

5 -import "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

5 +import (

6 + "fmt"

7 + "sort"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

11 +)

12 +

13 +type tableEntry struct {

14 + name string

15 + metrics tableInstanceMetrics

16 +}

17 +

18 +type tableGroupAggregate struct {

19 + DataSize int64

20 + IndexSize int64

21 + LongObjSize int64

22 + RowsRead int64

23 + RowsWritten int64

24 + Count int64

25 +}

26 +

27 +func tableGroupKey(name string) string {

28 + if idx := strings.Index(name, "."); idx > 0 {

29 + return name[:idx]

30 + }

31 + return name

32 +}

33 +

34 +func (a *tableGroupAggregate) add(m tableInstanceMetrics) {

35 + a.Count++

36 + a.DataSize += m.DataSize

37 + a.IndexSize += m.IndexSize

38 + a.LongObjSize += m.LongObjSize

39 + a.RowsRead += m.RowsRead

40 + a.RowsWritten += m.RowsWritten

41 +}

43 func (c *Collector) exportTableMetrics() {

44 + entries := make([]tableEntry, 0, len(c.mx.tables))

45 for name, metrics := range c.mx.tables {

9 - labels := contexts.TableLabels{

10 - Table: name,

46 + entries = append(entries, tableEntry{name: name, metrics: metrics})

47 + }

48 +

49 + if len(entries) == 0 {

50 + c.clearWarnOnce("db2_table_overflow")

51 + return

52 + }

53 +

54 + sort.Slice(entries, func(i, j int) bool {

55 + return entries[i].name < entries[j].name

56 + })

57 +

58 + limit := c.MaxTables

59 + if limit <= 0 || limit > len(entries) {

60 + limit = len(entries)

61 + }

62 +

63 + groupAgg := make(map[string]*tableGroupAggregate)

64 + overflowAgg := &tableGroupAggregate{}

65 + overflowCount := 0

66 + overflowGroups := make(map[string]int)

67 + overflowExample := make(map[string]string)

68 +

69 + for idx, entry := range entries {

70 + key := tableGroupKey(entry.name)

71 + agg := groupAgg[key]

72 + if agg == nil {

73 + agg = &tableGroupAggregate{}

74 + groupAgg[key] = agg

75 }

76 + agg.add(entry.metrics)

13 - contexts.Table.Size.Set(c.State, labels, contexts.TableSizeValues{

14 - Data: metrics.DataSize,

15 - Index: metrics.IndexSize,

16 - Long_obj: metrics.LongObjSize,

78 + if idx < limit {

79 + labels := contexts.TableLabels{Table: entry.name}

80 + contexts.Table.Size.Set(c.State, labels, contexts.TableSizeValues{

81 + Data: entry.metrics.DataSize,

82 + Index: entry.metrics.IndexSize,

83 + Long_obj: entry.metrics.LongObjSize,

84 + })

85 + contexts.Table.Activity.Set(c.State, labels, contexts.TableActivityValues{

86 + Read: entry.metrics.RowsRead,

87 + Written: entry.metrics.RowsWritten,

88 + })

89 + continue

90 + }

91 +

92 + overflowAgg.add(entry.metrics)

93 + overflowCount++

94 + overflowGroups[key]++

95 + if _, ok := overflowExample[key]; !ok {

96 + overflowExample[key] = entry.name

97 + }

98 + }

99 +

100 + c.emitTableGroupMetrics(groupAgg, overflowAgg, overflowCount)

101 +

102 + if overflowCount > 0 {

103 + parts := make([]string, 0, len(overflowGroups))

104 + for group, count := range overflowGroups {

105 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, overflowExample[group]))

106 + }

107 + sort.Strings(parts)

108 + c.warnOnce("db2_table_overflow", "too many tables for per-instance charts (MaxTables=%d). Aggregated %d additional tables: %s", c.MaxTables, overflowCount, strings.Join(parts, ", "))

109 + } else {

110 + c.clearWarnOnce("db2_table_overflow")

111 + }

112 +}

113 +

114 +func (c *Collector) emitTableGroupMetrics(groups map[string]*tableGroupAggregate, overflow *tableGroupAggregate, overflowCount int) {

115 + keys := make([]string, 0, len(groups))

116 + for k := range groups {

117 + keys = append(keys, k)

118 + }

119 + sort.Strings(keys)

120 +

121 + for _, key := range keys {

122 + agg := groups[key]

123 + labels := contexts.TableGroupLabels{Group: key}

124 +

125 + contexts.TableGroup.Size.Set(c.State, labels, contexts.TableGroupSizeValues{

126 + Data: agg.DataSize,

127 + Index: agg.IndexSize,

128 + Long_obj: agg.LongObjSize,

129 })

130

19 - contexts.Table.Activity.Set(c.State, labels, contexts.TableActivityValues{

20 - Read: metrics.RowsRead,

21 - Written: metrics.RowsWritten,

131 + contexts.TableGroup.Activity.Set(c.State, labels, contexts.TableGroupActivityValues{

132 + Read: agg.RowsRead,

133 + Written: agg.RowsWritten,

134 + })

135 + }

136 +

137 + if overflowCount > 0 && overflow != nil {

138 + labels := contexts.TableGroupLabels{Group: "__other__"}

139 + contexts.TableGroup.Size.Set(c.State, labels, contexts.TableGroupSizeValues{

140 + Data: overflow.DataSize,

141 + Index: overflow.IndexSize,

142 + Long_obj: overflow.LongObjSize,

143 + })

144 + contexts.TableGroup.Activity.Set(c.State, labels, contexts.TableGroupActivityValues{

145 + Read: overflow.RowsRead,

146 + Written: overflow.RowsWritten,

147 })

148 }

149 }

src/go/plugin/ibm.d/modules/db2/export_tablespace.go

+189 -34

@@ -2,49 +2,204 @@

3 package db2

5 -import "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

5 +import (

6 + "fmt"

7 + "sort"

8 + "strings"

9 +

10 + "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/db2/contexts"

11 +)

12 +

13 +type tablespaceEntry struct {

14 + name string

15 + meta *tablespaceMetrics

16 + metrics tablespaceInstanceMetrics

17 +}

18 +

19 +type tablespaceGroupAggregate struct {

20 + UsedSize int64

21 + FreeSize int64

22 + TotalSize int64

23 + UsableSize int64

24 + State int64

25 +}

26 +

27 +func tablespaceGroupKey(meta *tablespaceMetrics) string {

28 + if meta == nil {

29 + return "UNKNOWN"

30 + }

31 + parts := []string{}

32 + if meta.tbspType != "" {

33 + parts = append(parts, strings.ToUpper(meta.tbspType))

34 + }

35 + if meta.contentType != "" {

36 + parts = append(parts, strings.ToUpper(meta.contentType))

37 + }

38 + if len(parts) == 0 {

39 + return "UNKNOWN"

40 + }

41 + return strings.Join(parts, "/")

42 +}

43 +

44 +func (a *tablespaceGroupAggregate) add(m tablespaceInstanceMetrics) {

45 + a.UsedSize += m.UsedSize

46 + a.FreeSize += m.FreeSize

47 + a.TotalSize += m.TotalSize

48 + a.UsableSize += m.UsableSize

49 + a.State += m.State

50 +}

52 func (c *Collector) exportTablespaceMetrics() {

53 + entries := make([]tablespaceEntry, 0, len(c.mx.tablespaces))

54 for name, metrics := range c.mx.tablespaces {

9 - meta := c.tablespaces[name]

10 - tspType := "unknown"

11 - contentType := "unknown"

12 - stateLabel := "unknown"

13 - if meta != nil {

14 - if meta.tbspType != "" {

15 - tspType = meta.tbspType

16 - }

17 - if meta.contentType != "" {

18 - contentType = meta.contentType

19 - }

20 - if meta.state != "" {

21 - stateLabel = meta.state

22 - }

23 - }

24 -

25 - labels := contexts.TablespaceLabels{

26 - Tablespace: name,

27 - Type: tspType,

28 - Content_type: contentType,

29 - State: stateLabel,

30 - }

31 -

32 - contexts.Tablespace.Usage.Set(c.State, labels, contexts.TablespaceUsageValues{

33 - Used: metrics.UsedPercent,

55 + entries = append(entries, tablespaceEntry{

56 + name: name,

57 + meta: c.tablespaces[name],

58 + metrics: metrics,

59 })

60 + }

61 +

62 + if len(entries) == 0 {

63 + c.clearWarnOnce("db2_tablespace_overflow")

64 + return

65 + }

66 +

67 + sort.Slice(entries, func(i, j int) bool {

68 + return entries[i].name < entries[j].name

69 + })

70 +

71 + limit := c.MaxTablespaces

72 + if limit <= 0 || limit > len(entries) {

73 + limit = len(entries)

74 + }

75 +

76 + groupAgg := make(map[string]*tablespaceGroupAggregate)

77 + overflowAgg := &tablespaceGroupAggregate{}

78 + overflowCount := 0

79 + overflowGroups := make(map[string]int)

80 + overflowExample := make(map[string]string)

81 +

82 + for idx, entry := range entries {

83 + key := tablespaceGroupKey(entry.meta)

84 + agg := groupAgg[key]

85 + if agg == nil {

86 + agg = &tablespaceGroupAggregate{}

87 + groupAgg[key] = agg

88 + }

89 + agg.add(entry.metrics)

90 +

91 + if idx < limit {

92 + c.emitPerTablespaceMetrics(entry)

93 + continue

94 + }

95 +

96 + overflowAgg.add(entry.metrics)

97 + overflowCount++

98 + overflowGroups[key]++

99 + if _, ok := overflowExample[key]; !ok {

100 + overflowExample[key] = entry.name

101 + }

102 + }

103

36 - contexts.Tablespace.Size.Set(c.State, labels, contexts.TablespaceSizeValues{

37 - Used: metrics.UsedSize,

38 - Free: metrics.FreeSize,

104 + c.emitTablespaceGroupMetrics(groupAgg, overflowAgg, overflowCount)

105 +

106 + if overflowCount > 0 {

107 + parts := make([]string, 0, len(overflowGroups))

108 + for group, count := range overflowGroups {

109 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, overflowExample[group]))

110 + }

111 + sort.Strings(parts)

112 + c.warnOnce("db2_tablespace_overflow", "too many tablespaces for per-instance charts (MaxTablespaces=%d). Aggregated %d additional tablespaces: %s", c.MaxTablespaces, overflowCount, strings.Join(parts, ", "))

113 + } else {

114 + c.clearWarnOnce("db2_tablespace_overflow")

115 + }

116 +}

117 +

118 +func (c *Collector) emitPerTablespaceMetrics(entry tablespaceEntry) {

119 + tspType := "unknown"

120 + contentType := "unknown"

121 + stateLabel := "unknown"

122 + if entry.meta != nil {

123 + if entry.meta.tbspType != "" {

124 + tspType = entry.meta.tbspType

125 + }

126 + if entry.meta.contentType != "" {

127 + contentType = entry.meta.contentType

128 + }

129 + if entry.meta.state != "" {

130 + stateLabel = entry.meta.state

131 + }

132 + }

133 +

134 + labels := contexts.TablespaceLabels{

135 + Tablespace: entry.name,

136 + Type: tspType,

137 + Content_type: contentType,

138 + State: stateLabel,

139 + }

140 +

141 + contexts.Tablespace.Usage.Set(c.State, labels, contexts.TablespaceUsageValues{

142 + Used: entry.metrics.UsedPercent,

143 + })

144 +

145 + contexts.Tablespace.Size.Set(c.State, labels, contexts.TablespaceSizeValues{

146 + Used: entry.metrics.UsedSize,

147 + Free: entry.metrics.FreeSize,

148 + })

149 +

150 + contexts.Tablespace.UsableSize.Set(c.State, labels, contexts.TablespaceUsableSizeValues{

151 + Total: entry.metrics.TotalSize,

152 + Usable: entry.metrics.UsableSize,

153 + })

154 +

155 + contexts.Tablespace.State.Set(c.State, labels, contexts.TablespaceStateValues{

156 + State: entry.metrics.State,

157 + })

158 +}

159 +

160 +func (c *Collector) emitTablespaceGroupMetrics(groups map[string]*tablespaceGroupAggregate, overflow *tablespaceGroupAggregate, overflowCount int) {

161 + keys := make([]string, 0, len(groups))

162 + for k := range groups {

163 + keys = append(keys, k)

164 + }

165 + sort.Strings(keys)

166 +

167 + for _, key := range keys {

168 + agg := groups[key]

169 + labels := contexts.TablespaceGroupLabels{Group: key}

170 +

171 + usedPercent := int64(0)

172 + if agg.TotalSize > 0 {

173 + usedPercent = agg.UsedSize * 100 * Precision / agg.TotalSize

174 + }

175 + contexts.TablespaceGroup.Usage.Set(c.State, labels, contexts.TablespaceGroupUsageValues{

176 + Used: usedPercent,

177 + })

178 +

179 + contexts.TablespaceGroup.Size.Set(c.State, labels, contexts.TablespaceGroupSizeValues{

180 + Used: agg.UsedSize,

181 + Free: agg.FreeSize,

182 })

183

41 - contexts.Tablespace.UsableSize.Set(c.State, labels, contexts.TablespaceUsableSizeValues{

42 - Total: metrics.TotalSize,

43 - Usable: metrics.UsableSize,

184 + contexts.TablespaceGroup.UsableSize.Set(c.State, labels, contexts.TablespaceGroupUsableSizeValues{

185 + Total: agg.TotalSize,

186 + Usable: agg.UsableSize,

187 })

188

46 - contexts.Tablespace.State.Set(c.State, labels, contexts.TablespaceStateValues{

47 - State: metrics.State,

189 + contexts.TablespaceGroup.State.Set(c.State, labels, contexts.TablespaceGroupStateValues{

190 + State: agg.State,

191 })

192 }

193 +

194 + if overflowCount > 0 && overflow != nil {

195 + labels := contexts.TablespaceGroupLabels{Group: "__other__"}

196 + usedPercent := int64(0)

197 + if overflow.TotalSize > 0 {

198 + usedPercent = overflow.UsedSize * 100 * Precision / overflow.TotalSize

199 + }

200 + contexts.TablespaceGroup.Usage.Set(c.State, labels, contexts.TablespaceGroupUsageValues{Used: usedPercent})

201 + contexts.TablespaceGroup.Size.Set(c.State, labels, contexts.TablespaceGroupSizeValues{Used: overflow.UsedSize, Free: overflow.FreeSize})

202 + contexts.TablespaceGroup.UsableSize.Set(c.State, labels, contexts.TablespaceGroupUsableSizeValues{Total: overflow.TotalSize, Usable: overflow.UsableSize})

203 + contexts.TablespaceGroup.State.Set(c.State, labels, contexts.TablespaceGroupStateValues{State: overflow.State})

204 + }

205 }

src/go/plugin/ibm.d/modules/db2/indexes.go

+7 -1

@@ -25,7 +25,7 @@ func (c *Collector) collectIndexInstances(ctx context.Context) error {

25 currentIndex = value

26 key = fmt.Sprintf("%s.%s", currentSchema, currentIndex)

28 - if c.indexSelector != nil && !c.indexSelector.MatchString(key) {

28 + if !c.allowIndex(key) {

29 key = ""

30 return

31 }

@@ -59,6 +59,12 @@ func (c *Collector) collectIndexInstances(ctx context.Context) error {

59 }

60 }

61 }

62 +

63 + if lineEnd {

64 + key = ""

65 + currentIndex = ""

66 + currentSchema = ""

67 + }

68 })

70 return err

src/go/plugin/ibm.d/modules/db2/init.go

+88 -40

@@ -43,19 +43,47 @@ func defaultConfig() Config {

44 MaxDatabases: 10,

45 MaxBufferpools: 20,

46 - MaxTablespaces: 100,

47 - MaxConnections: 200,

48 - MaxTables: 50,

49 - MaxIndexes: 100,

46 + MaxTablespaces: 50,

47 + MaxConnections: 50,

48 + MaxTables: 25,

49 + MaxIndexes: 50,

51 BackupHistoryDays: 30,

53 - CollectDatabasesMatching: "",

54 - CollectBufferpoolsMatching: "",

55 - CollectTablespacesMatching: "",

56 - CollectConnectionsMatching: "",

57 - CollectTablesMatching: "",

58 - CollectIndexesMatching: "",

53 + CollectDatabasesMatching: "",

54 +

55 + IncludeConnections: []string{

56 + "db2sysc*",

57 + "db2agent*",

58 + "db2hadr*",

59 + "db2acd*",

60 + "db2bmgr*",

61 + },

62 + ExcludeConnections: []string{

63 + "*TEMP*",

64 + },

65 +

66 + IncludeBufferpools: []string{

67 + "IBMDEFAULTBP",

68 + "IBMSYSTEMBP*",

69 + "IBMHADRBP*",

70 + },

71 + ExcludeBufferpools: nil,

72 +

73 + IncludeTablespaces: []string{

74 + "SYSCATSPACE",

75 + "TEMPSPACE*",

76 + "SYSTOOLSPACE",

77 + },

78 + ExcludeTablespaces: []string{

79 + "TEMPSPACE2",

80 + },

81 +

82 + IncludeTables: nil,

83 + ExcludeTables: nil,

84 +

85 + IncludeIndexes: nil,

86 + ExcludeIndexes: nil,

87 }

88 }

@@ -105,41 +133,61 @@ func (c *Collector) Init(ctx context.Context) error {

133 }

134 c.databaseSelector = m

135 }

108 - if c.CollectBufferpoolsMatching != "" {

109 - m, err := matcher.NewSimplePatternsMatcher(c.CollectBufferpoolsMatching)

110 - if err != nil {

111 - return fmt.Errorf("invalid bufferpool selector pattern '%s': %v", c.CollectBufferpoolsMatching, err)

112 - }

113 - c.bufferpoolSelector = m

136 +

137 + connInclude, err := compileMatcher(c.IncludeConnections)

138 + if err != nil {

139 + return fmt.Errorf("invalid include_connections patterns: %w", err)

140 }

115 - if c.CollectTablespacesMatching != "" {

116 - m, err := matcher.NewSimplePatternsMatcher(c.CollectTablespacesMatching)

117 - if err != nil {

118 - return fmt.Errorf("invalid tablespace selector pattern '%s': %v", c.CollectTablespacesMatching, err)

119 - }

120 - c.tablespaceSelector = m

141 + connExclude, err := compileMatcher(c.ExcludeConnections)

142 + if err != nil {

143 + return fmt.Errorf("invalid exclude_connections patterns: %w", err)

144 }

122 - if c.CollectConnectionsMatching != "" {

123 - m, err := matcher.NewSimplePatternsMatcher(c.CollectConnectionsMatching)

124 - if err != nil {

125 - return fmt.Errorf("invalid connection selector pattern '%s': %v", c.CollectConnectionsMatching, err)

126 - }

127 - c.connectionSelector = m

145 + c.connectionInclude = connInclude

146 + c.connectionExclude = connExclude

147 +

148 + bpInclude, err := compileMatcher(c.IncludeBufferpools)

149 + if err != nil {

150 + return fmt.Errorf("invalid include_bufferpools patterns: %w", err)

151 }

129 - if c.CollectTablesMatching != "" {

130 - m, err := matcher.NewSimplePatternsMatcher(c.CollectTablesMatching)

131 - if err != nil {

132 - return fmt.Errorf("invalid table selector pattern '%s': %v", c.CollectTablesMatching, err)

133 - }

134 - c.tableSelector = m

152 + bpExclude, err := compileMatcher(c.ExcludeBufferpools)

153 + if err != nil {

154 + return fmt.Errorf("invalid exclude_bufferpools patterns: %w", err)

155 }

136 - if c.CollectIndexesMatching != "" {

137 - m, err := matcher.NewSimplePatternsMatcher(c.CollectIndexesMatching)

138 - if err != nil {

139 - return fmt.Errorf("invalid index selector pattern '%s': %v", c.CollectIndexesMatching, err)

140 - }

141 - c.indexSelector = m

156 + c.bufferpoolInclude = bpInclude

157 + c.bufferpoolExclude = bpExclude

158 +

159 + tspInclude, err := compileMatcher(c.IncludeTablespaces)

160 + if err != nil {

161 + return fmt.Errorf("invalid include_tablespaces patterns: %w", err)

162 + }

163 + tspExclude, err := compileMatcher(c.ExcludeTablespaces)

164 + if err != nil {

165 + return fmt.Errorf("invalid exclude_tablespaces patterns: %w", err)

166 + }

167 + c.tablespaceInclude = tspInclude

168 + c.tablespaceExclude = tspExclude

169 +

170 + tblInclude, err := compileMatcher(c.IncludeTables)

171 + if err != nil {

172 + return fmt.Errorf("invalid include_tables patterns: %w", err)

173 + }

174 + tblExclude, err := compileMatcher(c.ExcludeTables)

175 + if err != nil {

176 + return fmt.Errorf("invalid exclude_tables patterns: %w", err)

177 + }

178 + c.tableInclude = tblInclude

179 + c.tableExclude = tblExclude

180 +

181 + idxInclude, err := compileMatcher(c.IncludeIndexes)

182 + if err != nil {

183 + return fmt.Errorf("invalid include_indexes patterns: %w", err)

184 + }

185 + idxExclude, err := compileMatcher(c.ExcludeIndexes)

186 + if err != nil {

187 + return fmt.Errorf("invalid exclude_indexes patterns: %w", err)

188 }

189 + c.indexInclude = idxInclude

190 + c.indexExclude = idxExclude

191

192 if err := c.ensureConnected(ctx); err != nil {

193 return err

src/go/plugin/ibm.d/modules/db2/instances.go

+32 -17

@@ -146,22 +146,11 @@ func (c *Collector) doCollectDatabaseInstances(ctx context.Context, applySelecto

146 }

147

148 func (c *Collector) collectBufferpoolInstances(ctx context.Context) error {

149 - if c.MaxBufferpools <= 0 {

150 - return nil

151 - }

152 -

153 - // Always use MON_GET_BUFFERPOOL for bufferpool instances

154 - // Note: MON_GET_BUFFERPOOL doesn't support FETCH FIRST, so we'll handle limit in post-processing

149 query := queryMonGetBufferpool

150 c.Debugf("using MON_GET_BUFFERPOOL for bufferpool instances")

151

152 var currentBP string

159 - count := 0

153 err := c.doQuery(ctx, query, func(column, value string, lineEnd bool) {

161 - // Handle limit for MON_GET queries

162 - if count >= c.MaxBufferpools {

163 - return

164 - }

154 switch column {

155 case "BP_NAME":

156 currentBP = strings.TrimSpace(value)

@@ -169,15 +158,13 @@ func (c *Collector) collectBufferpoolInstances(ctx context.Context) error {

158 return

159 }

160

172 - // Apply selector if configured

173 - if c.bufferpoolSelector != nil && !c.bufferpoolSelector.MatchString(currentBP) {

174 - currentBP = "" // Skip this bufferpool

161 + if !c.allowBufferpool(currentBP) {

162 + currentBP = ""

163 return

164 }

165

166 if _, exists := c.bufferpools[currentBP]; !exists {

167 c.bufferpools[currentBP] = &bufferpoolMetrics{name: currentBP}

180 - count++

168 }

169 if _, exists := c.mx.bufferpools[currentBP]; !exists {

170 c.mx.bufferpools[currentBP] = bufferpoolInstanceMetrics{}

@@ -457,6 +444,11 @@ func (c *Collector) collectTablespaceInstances(ctx context.Context) error {

444 return

445 }

446

447 + if !c.allowTablespace(currentTbsp, "", "") {

448 + currentTbsp = ""

449 + return

450 + }

451 +

452 if _, exists := c.tablespaces[currentTbsp]; !exists {

453 c.tablespaces[currentTbsp] = &tablespaceMetrics{name: currentTbsp}

454 }

@@ -544,17 +536,30 @@ func (c *Collector) collectTablespaceInstances(ctx context.Context) error {

536 }

537 }

538 }

539 +

540 + if currentTbsp != "" {

541 + meta := c.tablespaces[currentTbsp]

542 + if !c.allowTablespace(currentTbsp, meta.contentType, meta.state) {

543 + delete(c.tablespaces, currentTbsp)

544 + delete(c.mx.tablespaces, currentTbsp)

545 + currentTbsp = ""

546 + }

547 + }

548 +

549 + if lineEnd {

550 + currentTbsp = ""

551 + }

552 })

553

554 return err

555 }

556

557 func (c *Collector) collectConnectionInstances(ctx context.Context) error {

558 + // MaxConnections <=0 disables per-connection collection entirely.

559 if c.MaxConnections <= 0 {

560 return nil

561 }

562

557 - // Always use MON_GET_CONNECTION for connection instances

563 query := queryMonGetConnectionDetails

564 c.Debugf("using MON_GET_CONNECTION for connection instances")

565

@@ -570,6 +575,7 @@ func (c *Collector) collectConnectionInstances(ctx context.Context) error {

575 if _, exists := c.connections[currentAppID]; !exists {

576 c.connections[currentAppID] = &connectionMetrics{applicationID: currentAppID}

577 }

578 +

579 c.mx.connections[currentAppID] = connectionInstanceMetrics{}

580

581 case "APPLICATION_NAME":

@@ -595,7 +601,6 @@ func (c *Collector) collectConnectionInstances(ctx context.Context) error {

601 case "APPL_STATUS":

602 if currentAppID != "" {

603 c.connections[currentAppID].connectionState = value

598 - // Map state to numeric

604 stateValue := int64(0)

605 execQueries := int64(0)

606 switch strings.ToUpper(value) {

@@ -638,6 +643,16 @@ func (c *Collector) collectConnectionInstances(ctx context.Context) error {

643 }

644 }

645 }

646 +

647 + if lineEnd && currentAppID != "" {

648 + meta := c.connections[currentAppID]

649 + if !c.allowConnection(currentAppID, meta) {

650 + delete(c.connections, currentAppID)

651 + delete(c.mx.connections, currentAppID)

652 + }

653 + currentAppID = ""

654 + }

655 +

656 })

657

658 return err

src/go/plugin/ibm.d/modules/db2/metadata.yaml

+182

@@ -25,6 +25,18 @@ modules:

25 functions to expose connections, locking, buffer pool efficiency, tablespace

26 capacity, and workload performance metrics.

28 + Detailed charts are opt-in per object family through include/exclude lists.

29 + Defaults focus on engine activity (system connections, core buffer pools,

30 + catalog tablespaces). Matching uses glob patterns that can target schema or

31 + application names, with include rules taking precedence over excludes.

32 +

33 + When the number of matching objects exceeds the configured `max_*` limits,

34 + the collector publishes deterministic top-N per-instance charts, aggregates

35 + the remainder under `group="__other__"`, and logs a throttled warning so you

36 + can refine selectors before cardinality runs away. Group charts (by schema,

37 + application prefix, or buffer pool family) are always emitted so high-level

38 + visibility is preserved even when individual instances are trimmed.

39 +

40 method_description: |

41 The collector connects to IBM DB2 and collects metrics via its monitoring interface.

42 supported_platforms:

@@ -136,6 +148,61 @@ modules:

148 chart_type: line

149 dimensions:

150 - name: writes

151 + - name: bufferpoolgroup

152 + description: These metrics refer to bufferpoolgroup instances.

153 + labels:

154 + - name: group

155 + description: Group identifier

156 + metrics:

157 + - name: db2.bufferpool_group_hit_ratio

158 + description: Buffer Pool Group Hit Ratio

159 + unit: percentage

160 + chart_type: line

161 + dimensions:

162 + - name: overall

163 + - name: db2.bufferpool_group_detailed_hit_ratio

164 + description: Buffer Pool Group Detailed Hit Ratios

165 + unit: percentage

166 + chart_type: line

167 + dimensions:

168 + - name: data

169 + - name: index

170 + - name: xda

171 + - name: column

172 + - name: db2.bufferpool_group_reads

173 + description: Buffer Pool Group Reads

174 + unit: reads/s

175 + chart_type: stacked

176 + dimensions:

177 + - name: logical

178 + - name: physical

179 + - name: db2.bufferpool_group_data_reads

180 + description: Buffer Pool Group Data Reads

181 + unit: reads/s

182 + chart_type: stacked

183 + dimensions:

184 + - name: logical

185 + - name: physical

186 + - name: db2.bufferpool_group_index_reads

187 + description: Buffer Pool Group Index Reads

188 + unit: reads/s

189 + chart_type: stacked

190 + dimensions:

191 + - name: logical

192 + - name: physical

193 + - name: db2.bufferpool_group_pages

194 + description: Buffer Pool Group Pages

195 + unit: pages

196 + chart_type: stacked

197 + dimensions:

198 + - name: used

199 + - name: total

200 + - name: db2.bufferpool_group_writes

201 + description: Buffer Pool Group Writes

202 + unit: writes/s

203 + chart_type: line

204 + dimensions:

205 + - name: writes

206 - name: connection

207 description: These metrics refer to connection instances.

208 labels:

@@ -189,6 +256,55 @@ modules:

256 - name: section

257 - name: commit

258 - name: rollback

259 + - name: connectiongroup

260 + description: These metrics refer to connectiongroup instances.

261 + labels:

262 + - name: group

263 + description: Group identifier

264 + metrics:

265 + - name: db2.connection_group.count

266 + description: Connection Group Count

267 + unit: connections

268 + chart_type: line

269 + dimensions:

270 + - name: count

271 + - name: db2.connection_group.state

272 + description: Connection Group State Sum

273 + unit: state

274 + chart_type: line

275 + dimensions:

276 + - name: state

277 + - name: db2.connection_group.activity

278 + description: Connection Group Row Activity

279 + unit: rows/s

280 + chart_type: area

281 + dimensions:

282 + - name: read

283 + - name: written

284 + - name: db2.connection_group.wait_time

285 + description: Connection Group Wait Time

286 + unit: milliseconds

287 + chart_type: stacked

288 + dimensions:

289 + - name: lock

290 + - name: log_disk

291 + - name: log_buffer

292 + - name: pool_read

293 + - name: pool_write

294 + - name: direct_read

295 + - name: direct_write

296 + - name: fcm_recv

297 + - name: fcm_send

298 + - name: db2.connection_group.processing_time

299 + description: Connection Group Processing Time

300 + unit: milliseconds

301 + chart_type: stacked

302 + dimensions:

303 + - name: routine

304 + - name: compile

305 + - name: section

306 + - name: commit

307 + - name: rollback

308 - name: database

309 description: These metrics refer to database instances.

310 labels:

@@ -222,6 +338,19 @@ modules:

338 dimensions:

339 - name: index

340 - name: full

341 + - name: indexgroup

342 + description: These metrics refer to indexgroup instances.

343 + labels:

344 + - name: group

345 + description: Group identifier

346 + metrics:

347 + - name: db2.index_group_usage

348 + description: Index Group Usage

349 + unit: scans/s

350 + chart_type: area

351 + dimensions:

352 + - name: index

353 + - name: full

354 - name: memorypool

355 description: These metrics refer to memorypool instances.

356 labels:

@@ -626,6 +755,27 @@ modules:

755 dimensions:

756 - name: read

757 - name: written

758 + - name: tablegroup

759 + description: These metrics refer to tablegroup instances.

760 + labels:

761 + - name: group

762 + description: Group identifier

763 + metrics:

764 + - name: db2.table_group_size

765 + description: Table Group Size

766 + unit: bytes

767 + chart_type: stacked

768 + dimensions:

769 + - name: data

770 + - name: index

771 + - name: long_obj

772 + - name: db2.table_group_activity

773 + description: Table Group Activity

774 + unit: rows/s

775 + chart_type: area

776 + dimensions:

777 + - name: read

778 + - name: written

779 - name: tableio

780 description: These metrics refer to tableio instances.

781 labels:

@@ -696,3 +846,35 @@ modules:

846 chart_type: line

847 dimensions:

848 - name: state

849 + - name: tablespacegroup

850 + description: These metrics refer to tablespacegroup instances.

851 + labels:

852 + - name: group

853 + description: Group identifier

854 + metrics:

855 + - name: db2.tablespace_group_usage

856 + description: Tablespace Group Usage

857 + unit: percentage

858 + chart_type: line

859 + dimensions:

860 + - name: used

861 + - name: db2.tablespace_group_size

862 + description: Tablespace Group Size

863 + unit: bytes

864 + chart_type: stacked

865 + dimensions:

866 + - name: used

867 + - name: free

868 + - name: db2.tablespace_group_usable_size

869 + description: Tablespace Group Usable Size

870 + unit: bytes

871 + chart_type: line

872 + dimensions:

873 + - name: total

874 + - name: usable

875 + - name: db2.tablespace_group_state

876 + description: Tablespace Group State

877 + unit: state

878 + chart_type: line

879 + dimensions:

880 + - name: state

src/go/plugin/ibm.d/modules/db2/module.yaml

+12

@@ -4,6 +4,18 @@ description: |

4 Monitors IBM DB2 databases using system catalog views and MON_GET_* table

5 functions to expose connections, locking, buffer pool efficiency, tablespace

6 capacity, and workload performance metrics.

7 +

8 + Detailed charts are opt-in per object family through include/exclude lists.

9 + Defaults focus on engine activity (system connections, core buffer pools,

10 + catalog tablespaces). Matching uses glob patterns that can target schema or

11 + application names, with include rules taking precedence over excludes.

12 +

13 + When the number of matching objects exceeds the configured `max_*` limits,

14 + the collector publishes deterministic top-N per-instance charts, aggregates

15 + the remainder under `group="__other__"`, and logs a throttled warning so you

16 + can refine selectors before cardinality runs away. Group charts (by schema,

17 + application prefix, or buffer pool family) are always emitted so high-level

18 + visibility is preserved even when individual instances are trimmed.

19 icon: ibm.svg

20 categories:

21 - data-collection.database-servers

src/go/plugin/ibm.d/modules/db2/tables.go

+7 -1

@@ -25,7 +25,7 @@ func (c *Collector) collectTableInstances(ctx context.Context) error {

25 currentTable = value

26 key = fmt.Sprintf("%s.%s", currentSchema, currentTable)

28 - if c.tableSelector != nil && !c.tableSelector.MatchString(key) {

28 + if !c.allowTable(key) {

29 key = ""

30 return

31 }

@@ -75,6 +75,12 @@ func (c *Collector) collectTableInstances(ctx context.Context) error {

75 }

76 }

77 }

78 +

79 + if lineEnd {

80 + key = ""

81 + currentTable = ""

82 + currentSchema = ""

83 + }

84 })

86 return err

src/go/plugin/ibm.d/modules/mq/README.md

+38 -2

@@ -5,6 +5,19 @@

5 Monitors IBM MQ queue managers, queues, channels, and topics

6 using the PCF (Programmable Command Format) protocol.

8 +By default the collector tracks the critical system queues `SYSTEM.DEAD.LETTER.QUEUE`,

9 +`SYSTEM.ADMIN.COMMAND.QUEUE`, and `SYSTEM.ADMIN.STATISTICS.QUEUE`. All other queues are

10 +opt-in via the `include_queues` list, with `exclude_queues` removing noisy patterns such as

11 +`SYSTEM.*` or `AMQ.*`. Include patterns take precedence over excludes so you can safely

12 +monitor individual system queues while dropping the broader wildcard.

13 +

14 +Per-queue charts are bounded by `max_queues` (default 50). When more queues are discovered,

15 +the collector exports the busiest ones individually, rolls the remainder into an

16 +aggregated `__other__` dimension, and logs a throttled warning listing the overflowed

17 +groups. Parallel queue-group charts summarise depth, traffic, and backlog per naming

18 +prefix (first two dot-separated segments, collapsing all `SYSTEM.*` queues together), so

19 +high-level visibility is never lost even when detailed charts are trimmed.

20 +

22 This collector is part of the [Netdata](https://github.com/netdata/netdata) monitoring solution.

@@ -184,6 +197,28 @@ Metrics:

197 | mq.queue.msg_delivery_sequence | priority, fifo | boolean |

198 | mq.queue.harden_get_backout | enabled, disabled | boolean |

199

200 +### Per queuegroup

201 +

202 +These metrics refer to individual queuegroup instances.

203 +

204 +Labels:

205 +

206 +| Label | Description |

207 +|:------|:------------|

208 +| group | Group identifier |

209 +

210 +Metrics:

211 +

212 +| Metric | Dimensions | Unit |

213 +|:-------|:-----------|:-----|

214 +| mq.queue_group.depth | current, max | messages |

215 +| mq.queue_group.depth_percentage | percentage | percentage |

216 +| mq.queue_group.messages | enqueued, dequeued | messages/s |

217 +| mq.queue_group.connections | input, output | connections |

218 +| mq.queue_group.uncommitted_msgs | uncommitted | messages |

219 +| mq.queue_group.file_size | current, max | bytes |

220 +| mq.queue_group.oldest_msg_age | oldest_msg_age | seconds |

221 +

222 ### Per queuestatistics

223

224 These metrics refer to individual queuestatistics instances.

@@ -283,12 +318,13 @@ The following options can be defined globally or per job.

318 | CollectSystemListeners | Enable collection of system listener metrics (SYSTEM.* listeners show internal connectivity) | `true` | no | - | - |

319 | CollectChannelConfig | Enable collection of channel configuration metrics | `true` | no | - | - |

320 | CollectQueueConfig | Enable collection of queue configuration metrics | `true` | no | - | - |

286 -| QueueSelector | Pattern to filter queues (wildcards supported) | `` | no | - | - |

321 +| IncludeQueues | Patterns to include queues (wildcards supported). Empty means include everything. | `[SYSTEM.DEAD.LETTER.QUEUE SYSTEM.ADMIN.COMMAND.QUEUE SYSTEM.ADMIN.STATISTICS.QUEUE]` | no | - | - |

322 +| ExcludeQueues | Patterns to exclude queues after inclusion (wildcards supported). | `[SYSTEM.* AMQ.*]` | no | - | - |

323 | ChannelSelector | Pattern to filter channels (wildcards supported) | `` | no | - | - |

324 | TopicSelector | Pattern to filter topics (wildcards supported) | `` | no | - | - |

325 | ListenerSelector | Pattern to filter listeners (wildcards supported) | `` | no | - | - |

326 | SubscriptionSelector | Pattern to filter subscriptions (wildcards supported) | `` | no | - | - |

291 -| MaxQueues | Maximum number of queues to collect (0 = no limit) | `100` | no | - | - |

327 +| MaxQueues | Maximum number of queues to collect (0 = no limit) | `50` | no | - | - |

328 | MaxChannels | Maximum number of channels to collect (0 = no limit) | `100` | no | - | - |

329 | MaxTopics | Maximum number of topics to collect (0 = no limit) | `100` | no | - | - |

330 | MaxListeners | Maximum number of listeners to collect (0 = no limit) | `100` | no | - | - |

src/go/plugin/ibm.d/modules/mq/collect_queues.go

+482 -230

@@ -2,12 +2,98 @@ package mq

3 import (

4 "fmt"

5 + "sort"

6 + "strings"

7 "time"

9 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/mq/contexts"

10 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/pcf"

11 )

13 +type queueGroupAggregate struct {

14 + DepthCurrent int64

15 + DepthMax int64

16 +

17 + MessagesEnqueued int64

18 + MessagesDequeued int64

19 + HasMessages bool

20 +

21 + ConnectionsInput int64

22 + ConnectionsOutput int64

23 + HasConnections bool

24 +

25 + Uncommitted int64

26 + HasUncommitted bool

27 +

28 + FileSizeCurrent int64

29 + FileSizeMax int64

30 + HasFileSize bool

31 +

32 + OldestMessageAge int64

33 + HasOldest bool

34 +}

35 +

36 +func (a *queueGroupAggregate) add(queue *pcf.QueueMetrics) {

37 + a.DepthCurrent += queue.CurrentDepth

38 + a.DepthMax += queue.MaxDepth

39 +

40 + if queue.HasResetStats {

41 + a.MessagesEnqueued += queue.EnqueueCount

42 + a.MessagesDequeued += queue.DequeueCount

43 + a.HasMessages = true

44 + }

45 +

46 + if queue.HasStatusMetrics {

47 + if queue.OpenInputCount.IsCollected() {

48 + a.ConnectionsInput += queue.OpenInputCount.Int64()

49 + a.HasConnections = true

50 + }

51 + if queue.OpenOutputCount.IsCollected() {

52 + a.ConnectionsOutput += queue.OpenOutputCount.Int64()

53 + a.HasConnections = true

54 + }

55 + if queue.UncommittedMsgs.IsCollected() {

56 + a.Uncommitted += queue.UncommittedMsgs.Int64()

57 + a.HasUncommitted = true

58 + }

59 + if queue.CurrentFileSize.IsCollected() {

60 + a.FileSizeCurrent += queue.CurrentFileSize.Int64()

61 + a.HasFileSize = true

62 + }

63 + if queue.CurrentMaxFileSize.IsCollected() {

64 + a.FileSizeMax += queue.CurrentMaxFileSize.Int64()

65 + a.HasFileSize = true

66 + }

67 + if queue.OldestMsgAge.IsCollected() {

68 + age := queue.OldestMsgAge.Int64()

69 + if age >= 0 {

70 + if !a.HasOldest || age > a.OldestMessageAge {

71 + a.OldestMessageAge = age

72 + }

73 + a.HasOldest = true

74 + }

75 + }

76 + }

77 +}

78 +

79 +func queueGroupKey(name string) string {

80 + if name == "" {

81 + return "__unknown__"

82 + }

83 + if strings.HasPrefix(name, "SYSTEM.") {

84 + return "SYSTEM"

85 + }

86 + parts := strings.Split(name, ".")

87 + switch len(parts) {

88 + case 0:

89 + return "__unknown__"

90 + case 1:

91 + return parts[0]

92 + default:

93 + return strings.Join(parts[:2], ".")

94 + }

95 +}

96 +

97 // convertMQDateTimeToSecondsSince converts MQ date (YYYYMMDD) and time (HHMMSSSS) to seconds since that time

98 // Returns -1 if the date/time is invalid or not collected

99 func convertMQDateTimeToSecondsSince(date, timeVal pcf.AttributeValue) int64 {

@@ -45,294 +131,460 @@ func convertMQDateTimeToSecondsSince(date, timeVal pcf.AttributeValue) int64 {

131 return secondsSince

132 }

133

134 +func (c *Collector) shouldCollectQueue(name string) bool {

135 + included := true

136 + if len(c.Config.IncludeQueues) > 0 {

137 + if c.queueIncludeMatcher == nil {

138 + return false

139 + }

140 + included = c.queueIncludeMatcher.MatchString(name)

141 + if !included {

142 + return false

143 + }

144 + }

145 +

146 + if c.queueExcludeMatcher != nil && c.queueExcludeMatcher.MatchString(name) {

147 + if len(c.Config.IncludeQueues) > 0 {

148 + return included

149 + }

150 + return false

151 + }

152 +

153 + return true

154 +}

155 +

156 func (c *Collector) collectQueueMetrics() error {

49 - c.Debugf("Collecting queues with selector '%s', config: %v, reset_stats: %v, system: %v",

50 - c.Config.QueueSelector, c.Config.CollectQueueConfig, c.Config.CollectResetQueueStats, c.Config.CollectSystemQueues)

157 + c.Debugf("Collecting queues include=%v exclude=%v config=%v reset_stats=%v system=%v",

158 + c.Config.IncludeQueues, c.Config.ExcludeQueues, c.Config.CollectQueueConfig, c.Config.CollectResetQueueStats, c.Config.CollectSystemQueues)

159

52 - // Use new GetQueues with transparency

160 result, err := c.client.GetQueues(

54 - c.Config.CollectQueueConfig, // collectConfig

55 - true, // collectMetrics (always)

56 - c.Config.CollectResetQueueStats, // collectReset

57 - c.Config.MaxQueues, // maxQueues (0 = no limit)

58 - c.Config.QueueSelector, // selector pattern

59 - c.Config.CollectSystemQueues, // collectSystem

161 + c.Config.CollectQueueConfig,

162 + true,

163 + c.Config.CollectResetQueueStats,

164 + 0, // fetch everything; we enforce limits locally

165 + "*", // selector - we perform filtering ourselves

166 + c.Config.CollectSystemQueues,

167 )

168 if err != nil {

169 return fmt.Errorf("failed to collect queue metrics: %w", err)

170 }

171

65 - // Check discovery success

172 if !result.Stats.Discovery.Success {

173 c.Errorf("Queue discovery failed completely")

174 return fmt.Errorf("queue discovery failed")

175 }

176

71 - // Map transparency counters to user-facing semantics

72 - monitored := int64(0)

73 - if result.Stats.Metrics != nil {

74 - monitored = result.Stats.Metrics.OkItems

75 - }

76 -

177 failed := result.Stats.Discovery.UnparsedItems

178 if result.Stats.Metrics != nil {

179 failed += result.Stats.Metrics.FailedItems

180 }

81 - // Note: Config and Reset failures are not counted as they're optional

181

83 - // Update overview metrics with correct semantics

182 + filtered := make([]*pcf.QueueMetrics, 0, len(result.Queues))

183 + for i := range result.Queues {

184 + queue := result.Queues[i]

185 + if c.shouldCollectQueue(queue.Name) {

186 + queueCopy := queue

187 + filtered = append(filtered, &queueCopy)

188 + }

189 + }

190 +

191 + excludedByFilter := int64(len(result.Queues) - len(filtered))

192 + monitored := int64(len(filtered))

193 +

194 c.setQueueOverviewMetrics(

85 - monitored, // monitored (successfully enriched)

86 - result.Stats.Discovery.ExcludedItems, // excluded (filtered by user)

87 - result.Stats.Discovery.InvisibleItems, // invisible (discovery errors)

88 - failed, // failed (unparsed + enrichment failures)

195 + monitored,

196 + result.Stats.Discovery.ExcludedItems+excludedByFilter,

197 + result.Stats.Discovery.InvisibleItems,

198 + failed,

199 )

200

91 - // Log collection summary

92 - c.Debugf("Queue collection complete - discovered:%d visible:%d included:%d collected:%d failed:%d",

93 - result.Stats.Discovery.AvailableItems,

94 - result.Stats.Discovery.AvailableItems-result.Stats.Discovery.InvisibleItems,

95 - result.Stats.Discovery.IncludedItems,

96 - len(result.Queues),

97 - failed)

98 -

99 - // Process collected queue metrics

100 - for _, queue := range result.Queues {

101 - // Note: Protocol already applied selector and system queue filtering

102 - // We just need to set the metrics

103 -

104 - labels := contexts.QueueLabels{

105 - Queue: queue.Name,

106 - Type: pcf.QueueTypeString(int32(queue.Type)),

107 - }

201 + if len(filtered) == 0 {

202 + c.Debugf("No queues matched the include/exclude patterns")

203 + c.clearWarnOnce("queue_overflow")

204 + return nil

205 + }

206

109 - // Basic metrics (always available)

110 - contexts.Queue.Depth.Set(c.State, labels, contexts.QueueDepthValues{

111 - Current: queue.CurrentDepth,

112 - Max: queue.MaxDepth,

113 - })

207 + sort.Slice(filtered, func(i, j int) bool {

208 + return filtered[i].Name < filtered[j].Name

209 + })

210

115 - // Calculate and set depth percentage

116 - if queue.MaxDepth > 0 {

117 - percentage := float64(queue.CurrentDepth) / float64(queue.MaxDepth) * 100.0

118 - contexts.Queue.DepthPercentage.Set(c.State, labels, contexts.QueueDepthPercentageValues{

119 - Percentage: int64(percentage * 1000), // Pre-multiply for precision

120 - })

211 + limit := c.Config.MaxQueues

212 + if limit < 0 {

213 + limit = 0

214 + }

215 +

216 + aggregated := make(map[string]*queueGroupAggregate)

217 + overflowTotals := &queueGroupAggregate{}

218 + overflowCount := 0

219 + overflowGroups := make(map[string]int)

220 + overflowExamples := make(map[string]string)

221 +

222 + for idx, queue := range filtered {

223 + groupKey := queueGroupKey(queue.Name)

224 + agg := aggregated[groupKey]

225 + if agg == nil {

226 + agg = &queueGroupAggregate{}

227 + aggregated[groupKey] = agg

228 }

229 + agg.add(queue)

230

123 - // Status metrics (if status collection succeeded)

124 - if queue.HasStatusMetrics {

125 - // Only send connections if both values are collected

126 - if queue.OpenInputCount.IsCollected() && queue.OpenOutputCount.IsCollected() {

127 - contexts.Queue.Connections.Set(c.State, labels, contexts.QueueConnectionsValues{

128 - Input: queue.OpenInputCount.Int64(),

129 - Output: queue.OpenOutputCount.Int64(),

130 - })

131 - }

231 + if limit == 0 || idx < limit {

232 + c.emitPerQueueMetrics(queue)

233 + continue

234 + }

235

133 - // Oldest message age - only send if collected and not -1 (which means no messages)

134 - if queue.OldestMsgAge.IsCollected() && queue.OldestMsgAge.Int64() != -1 {

135 - contexts.Queue.OldestMessageAge.Set(c.State, labels, contexts.QueueOldestMessageAgeValues{

136 - Oldest_msg_age: queue.OldestMsgAge.Int64(),

137 - })

138 - }

236 + overflowTotals.add(queue)

237 + overflowCount++

238 + overflowGroups[groupKey]++

239 + if _, exists := overflowExamples[groupKey]; !exists {

240 + overflowExamples[groupKey] = queue.Name

241 + }

242 + }

243

140 - // Uncommitted messages - only send if collected

141 - if queue.UncommittedMsgs.IsCollected() {

142 - contexts.Queue.UncommittedMessages.Set(c.State, labels, contexts.QueueUncommittedMessagesValues{

143 - Uncommitted: queue.UncommittedMsgs.Int64(),

144 - })

145 - }

244 + c.emitQueueGroupMetrics(aggregated)

245

147 - // File size metrics - only send if at least one is collected (IBM MQ 9.1.5+)

148 - if queue.CurrentFileSize.IsCollected() || queue.CurrentMaxFileSize.IsCollected() {

149 - fileSizeValues := contexts.QueueFileSizeValues{}

150 - hasAnyFileSize := false

246 + if overflowCount > 0 {

247 + c.emitPerQueueOverflowMetrics(overflowTotals)

248

152 - if queue.CurrentFileSize.IsCollected() {

153 - fileSizeValues.Current = queue.CurrentFileSize.Int64()

154 - hasAnyFileSize = true

155 - }

156 - if queue.CurrentMaxFileSize.IsCollected() {

157 - fileSizeValues.Max = queue.CurrentMaxFileSize.Int64()

158 - hasAnyFileSize = true

159 - }

249 + parts := make([]string, 0, len(overflowGroups))

250 + for group, count := range overflowGroups {

251 + sample := overflowExamples[group]

252 + parts = append(parts, fmt.Sprintf("%s:%d (e.g. %s)", group, count, sample))

253 + }

254 + sort.Strings(parts)

255 + c.warnOnce("queue_overflow", "too many queues for per-queue charts (MaxQueues=%d). Aggregated %d additional queues: %s", limit, overflowCount, strings.Join(parts, ", "))

256 + } else {

257 + c.clearWarnOnce("queue_overflow")

258 + }

259

161 - if hasAnyFileSize {

162 - contexts.Queue.FileSize.Set(c.State, labels, fileSizeValues)

163 - }

164 - }

260 + c.Debugf("queue collection complete - discovered:%d matched:%d overflow:%d groups:%d",

261 + len(result.Queues), len(filtered), overflowCount, len(aggregated))

262

166 - // Queue time indicators (short/long period) - only send if both collected and not -1

167 - if queue.QTimeShort.IsCollected() && queue.QTimeLong.IsCollected() &&

168 - queue.QTimeShort.Int64() != -1 && queue.QTimeLong.Int64() != -1 {

169 - contexts.Queue.QueueTimeIndicators.Set(c.State, labels, contexts.QueueQueueTimeIndicatorsValues{

170 - Short_period: queue.QTimeShort.Int64(),

171 - Long_period: queue.QTimeLong.Int64(),

172 - })

173 - }

263 + return nil

264 +}

265

175 - // Last activity times - calculate seconds since last get/put

176 - sinceLastGet := convertMQDateTimeToSecondsSince(queue.LastGetDate, queue.LastGetTime)

177 - sinceLastPut := convertMQDateTimeToSecondsSince(queue.LastPutDate, queue.LastPutTime)

266 +func (c *Collector) emitPerQueueMetrics(queue *pcf.QueueMetrics) {

267 + labels := contexts.QueueLabels{

268 + Queue: queue.Name,

269 + Type: pcf.QueueTypeString(int32(queue.Type)),

270 + }

271

179 - // Only send if we have at least one valid time

180 - if sinceLastGet >= 0 || sinceLastPut >= 0 {

181 - // If one is invalid, use -1 to indicate no activity

182 - if sinceLastGet < 0 {

183 - sinceLastGet = -1

184 - }

185 - if sinceLastPut < 0 {

186 - sinceLastPut = -1

187 - }

272 + contexts.Queue.Depth.Set(c.State, labels, contexts.QueueDepthValues{

273 + Current: queue.CurrentDepth,

274 + Max: queue.MaxDepth,

275 + })

276

189 - contexts.Queue.LastActivity.Set(c.State, labels, contexts.QueueLastActivityValues{

190 - Since_last_get: sinceLastGet,

191 - Since_last_put: sinceLastPut,

192 - })

193 - }

277 + if queue.MaxDepth > 0 {

278 + percentage := float64(queue.CurrentDepth) / float64(queue.MaxDepth) * 100.0

279 + contexts.Queue.DepthPercentage.Set(c.State, labels, contexts.QueueDepthPercentageValues{

280 + Percentage: int64(percentage * 1000),

281 + })

282 + }

283 +

284 + if queue.HasStatusMetrics {

285 + if queue.OpenInputCount.IsCollected() && queue.OpenOutputCount.IsCollected() {

286 + contexts.Queue.Connections.Set(c.State, labels, contexts.QueueConnectionsValues{

287 + Input: queue.OpenInputCount.Int64(),

288 + Output: queue.OpenOutputCount.Int64(),

289 + })

290 }

291

196 - // Message count metrics (if reset stats were collected)

197 - if queue.HasResetStats {

198 - contexts.Queue.Messages.Set(c.State, labels, contexts.QueueMessagesValues{

199 - Enqueued: queue.EnqueueCount,

200 - Dequeued: queue.DequeueCount,

292 + if queue.OldestMsgAge.IsCollected() && queue.OldestMsgAge.Int64() != -1 {

293 + contexts.Queue.OldestMessageAge.Set(c.State, labels, contexts.QueueOldestMessageAgeValues{

294 + Oldest_msg_age: queue.OldestMsgAge.Int64(),

295 })

296 + }

297

203 - contexts.Queue.HighDepth.Set(c.State, labels, contexts.QueueHighDepthValues{

204 - High_depth: queue.HighDepth,

298 + if queue.UncommittedMsgs.IsCollected() {

299 + contexts.Queue.UncommittedMessages.Set(c.State, labels, contexts.QueueUncommittedMessagesValues{

300 + Uncommitted: queue.UncommittedMsgs.Int64(),

301 })

302 + }

303

207 - // TODO: Add time since reset context if needed

208 - // TimeSinceReset: queue.TimeSinceReset

304 + if queue.CurrentFileSize.IsCollected() || queue.CurrentMaxFileSize.IsCollected() {

305 + fileSizeValues := contexts.QueueFileSizeValues{}

306 + hasAny := false

307 + if queue.CurrentFileSize.IsCollected() {

308 + fileSizeValues.Current = queue.CurrentFileSize.Int64()

309 + hasAny = true

310 + }

311 + if queue.CurrentMaxFileSize.IsCollected() {

312 + fileSizeValues.Max = queue.CurrentMaxFileSize.Int64()

313 + hasAny = true

314 + }

315 + if hasAny {

316 + contexts.Queue.FileSize.Set(c.State, labels, fileSizeValues)

317 + }

318 }

319

211 - // Configuration metrics - only send when actually collected

212 - // Basic configuration (inhibit status, max message length)

213 - if queue.InhibitGet.IsCollected() && queue.InhibitPut.IsCollected() {

214 - contexts.Queue.InhibitStatus.Set(c.State, labels, contexts.QueueInhibitStatusValues{

215 - Inhibit_get: queue.InhibitGet.Int64(),

216 - Inhibit_put: queue.InhibitPut.Int64(),

320 + if queue.QTimeShort.IsCollected() && queue.QTimeLong.IsCollected() &&

321 + queue.QTimeShort.Int64() != -1 && queue.QTimeLong.Int64() != -1 {

322 + contexts.Queue.QueueTimeIndicators.Set(c.State, labels, contexts.QueueQueueTimeIndicatorsValues{

323 + Short_period: queue.QTimeShort.Int64(),

324 + Long_period: queue.QTimeLong.Int64(),

325 })

326 }

327

220 - if queue.MaxMsgLength.IsCollected() {

221 - contexts.Queue.MaxMessageLength.Set(c.State, labels, contexts.QueueMaxMessageLengthValues{

222 - Max_msg_length: queue.MaxMsgLength.Int64(),

328 + sinceLastGet := convertMQDateTimeToSecondsSince(queue.LastGetDate, queue.LastGetTime)

329 + sinceLastPut := convertMQDateTimeToSecondsSince(queue.LastPutDate, queue.LastPutTime)

330 + if sinceLastGet >= 0 || sinceLastPut >= 0 {

331 + if sinceLastGet < 0 {

332 + sinceLastGet = -1

333 + }

334 + if sinceLastPut < 0 {

335 + sinceLastPut = -1

336 + }

337 + contexts.Queue.LastActivity.Set(c.State, labels, contexts.QueueLastActivityValues{

338 + Since_last_get: sinceLastGet,

339 + Since_last_put: sinceLastPut,

340 })

341 }

342 + }

343

226 - // Detailed configuration only if explicitly enabled

227 - if c.Config.CollectQueueConfig {

228 - if queue.DefPriority.IsCollected() {

229 - contexts.Queue.Priority.Set(c.State, labels, contexts.QueuePriorityValues{

230 - Def_priority: queue.DefPriority.Int64(),

231 - })

232 - }

344 + if queue.HasResetStats {

345 + contexts.Queue.Messages.Set(c.State, labels, contexts.QueueMessagesValues{

346 + Enqueued: queue.EnqueueCount,

347 + Dequeued: queue.DequeueCount,

348 + })

349 + }

350

234 - if queue.TriggerDepth.IsCollected() && queue.TriggerType.IsCollected() {

235 - contexts.Queue.Triggers.Set(c.State, labels, contexts.QueueTriggersValues{

236 - Trigger_depth: queue.TriggerDepth.Int64(),

237 - Trigger_type: queue.TriggerType.Int64(),

238 - })

239 - }

351 + contexts.Queue.HighDepth.Set(c.State, labels, contexts.QueueHighDepthValues{

352 + High_depth: queue.HighDepth,

353 + })

354

241 - if queue.BackoutThreshold.IsCollected() {

242 - contexts.Queue.BackoutThreshold.Set(c.State, labels, contexts.QueueBackoutThresholdValues{

243 - Backout_threshold: queue.BackoutThreshold.Int64(),

244 - })

245 - }

355 + if queue.InhibitGet.IsCollected() && queue.InhibitPut.IsCollected() {

356 + contexts.Queue.InhibitStatus.Set(c.State, labels, contexts.QueueInhibitStatusValues{

357 + Inhibit_get: queue.InhibitGet.Int64(),

358 + Inhibit_put: queue.InhibitPut.Int64(),

359 + })

360 + }

361

247 - // New configuration metrics

248 - if queue.ServiceInterval.IsCollected() {

249 - contexts.Queue.ServiceInterval.Set(c.State, labels, contexts.QueueServiceIntervalValues{

250 - Service_interval: queue.ServiceInterval.Int64(),

251 - })

252 - }

362 + if queue.MaxMsgLength.IsCollected() {

363 + contexts.Queue.MaxMessageLength.Set(c.State, labels, contexts.QueueMaxMessageLengthValues{

364 + Max_msg_length: queue.MaxMsgLength.Int64(),

365 + })

366 + }

367

254 - if queue.RetentionInterval.IsCollected() {

255 - contexts.Queue.RetentionInterval.Set(c.State, labels, contexts.QueueRetentionIntervalValues{

256 - Retention_interval: queue.RetentionInterval.Int64(),

257 - })

258 - }

368 + if !c.Config.CollectQueueConfig {

369 + return

370 + }

371

260 - // Message persistence configuration

261 - if queue.DefPersistence.IsCollected() {

262 - persistent := int64(0)

263 - nonPersistent := int64(0)

264 - if queue.DefPersistence.Int64() == 1 {

265 - persistent = 1

266 - } else {

267 - nonPersistent = 1

268 - }

269 - contexts.Queue.MessagePersistence.Set(c.State, labels, contexts.QueueMessagePersistenceValues{

270 - Persistent: persistent,

271 - Non_persistent: nonPersistent,

272 - })

273 - }

372 + if queue.DefPriority.IsCollected() {

373 + contexts.Queue.Priority.Set(c.State, labels, contexts.QueuePriorityValues{

374 + Def_priority: queue.DefPriority.Int64(),

375 + })

376 + }

377

275 - // Queue Scope (0=MQSCO_Q_MGR, 1=MQSCO_CELL)

276 - if queue.Scope.IsCollected() {

277 - queueManager := int64(0)

278 - cell := int64(0)

279 - if queue.Scope.Int64() == 0 {

280 - queueManager = 1

281 - } else {

282 - cell = 1

283 - }

284 - contexts.Queue.QueueScope.Set(c.State, labels, contexts.QueueQueueScopeValues{

285 - Queue_manager: queueManager,

286 - Cell: cell,

287 - })

288 - }

378 + if queue.TriggerDepth.IsCollected() && queue.TriggerType.IsCollected() {

379 + contexts.Queue.Triggers.Set(c.State, labels, contexts.QueueTriggersValues{

380 + Trigger_depth: queue.TriggerDepth.Int64(),

381 + Trigger_type: queue.TriggerType.Int64(),

382 + })

383 + }

384

290 - // Queue Usage (0=MQUS_NORMAL, 1=MQUS_TRANSMISSION)

291 - if queue.Usage.IsCollected() {

292 - normal := int64(0)

293 - transmission := int64(0)

294 - if queue.Usage.Int64() == 0 {

295 - normal = 1

296 - } else {

297 - transmission = 1

298 - }

299 - contexts.Queue.QueueUsage.Set(c.State, labels, contexts.QueueQueueUsageValues{

300 - Normal: normal,

301 - Transmission: transmission,

302 - })

303 - }

385 + if queue.BackoutThreshold.IsCollected() {

386 + contexts.Queue.BackoutThreshold.Set(c.State, labels, contexts.QueueBackoutThresholdValues{

387 + Backout_threshold: queue.BackoutThreshold.Int64(),

388 + })

389 + }

390

305 - // Message Delivery Sequence (0=MQMDS_PRIORITY, 1=MQMDS_FIFO)

306 - if queue.MsgDeliverySequence.IsCollected() {

307 - priority := int64(0)

308 - fifo := int64(0)

309 - if queue.MsgDeliverySequence.Int64() == 0 {

310 - priority = 1

311 - } else {

312 - fifo = 1

313 - }

314 - contexts.Queue.MessageDeliverySequence.Set(c.State, labels, contexts.QueueMessageDeliverySequenceValues{

315 - Priority: priority,

316 - Fifo: fifo,

317 - })

318 - }

391 + if queue.ServiceInterval.IsCollected() {

392 + contexts.Queue.ServiceInterval.Set(c.State, labels, contexts.QueueServiceIntervalValues{

393 + Service_interval: queue.ServiceInterval.Int64(),

394 + })

395 + }

396

320 - // Harden Get Backout (0=disabled, 1=enabled)

321 - if queue.HardenGetBackout.IsCollected() {

322 - enabled := int64(0)

323 - disabled := int64(0)

324 - if queue.HardenGetBackout.Int64() == 1 {

325 - enabled = 1

326 - } else {

327 - disabled = 1

328 - }

329 - contexts.Queue.HardenGetBackout.Set(c.State, labels, contexts.QueueHardenGetBackoutValues{

330 - Enabled: enabled,

331 - Disabled: disabled,

332 - })

333 - }

397 + if queue.RetentionInterval.IsCollected() {

398 + contexts.Queue.RetentionInterval.Set(c.State, labels, contexts.QueueRetentionIntervalValues{

399 + Retention_interval: queue.RetentionInterval.Int64(),

400 + })

401 + }

402 +

403 + if queue.DefPersistence.IsCollected() {

404 + persistent := int64(0)

405 + nonPersistent := int64(0)

406 + if queue.DefPersistence.Int64() == 1 {

407 + persistent = 1

408 + } else {

409 + nonPersistent = 1

410 }

411 + contexts.Queue.MessagePersistence.Set(c.State, labels, contexts.QueueMessagePersistenceValues{

412 + Persistent: persistent,

413 + Non_persistent: nonPersistent,

414 + })

415 }

416

337 - return nil

417 + if queue.Scope.IsCollected() {

418 + queueManager := int64(0)

419 + cell := int64(0)

420 + if queue.Scope.Int64() == 0 {

421 + queueManager = 1

422 + } else {

423 + cell = 1

424 + }

425 + contexts.Queue.QueueScope.Set(c.State, labels, contexts.QueueQueueScopeValues{

426 + Queue_manager: queueManager,

427 + Cell: cell,

428 + })

429 + }

430 +

431 + if queue.Usage.IsCollected() {

432 + normal := int64(0)

433 + transmission := int64(0)

434 + if queue.Usage.Int64() == 0 {

435 + normal = 1

436 + } else {

437 + transmission = 1

438 + }

439 + contexts.Queue.QueueUsage.Set(c.State, labels, contexts.QueueQueueUsageValues{

440 + Normal: normal,

441 + Transmission: transmission,

442 + })

443 + }

444 +

445 + if queue.MsgDeliverySequence.IsCollected() {

446 + priority := int64(0)

447 + fifo := int64(0)

448 + if queue.MsgDeliverySequence.Int64() == 0 {

449 + priority = 1

450 + } else {

451 + fifo = 1

452 + }

453 + contexts.Queue.MessageDeliverySequence.Set(c.State, labels, contexts.QueueMessageDeliverySequenceValues{

454 + Priority: priority,

455 + Fifo: fifo,

456 + })

457 + }

458 +

459 + if queue.HardenGetBackout.IsCollected() {

460 + enabled := int64(0)

461 + disabled := int64(0)

462 + if queue.HardenGetBackout.Int64() == 1 {

463 + enabled = 1

464 + } else {

465 + disabled = 1

466 + }

467 + contexts.Queue.HardenGetBackout.Set(c.State, labels, contexts.QueueHardenGetBackoutValues{

468 + Enabled: enabled,

469 + Disabled: disabled,

470 + })

471 + }

472 +}

473 +

474 +func (c *Collector) emitPerQueueOverflowMetrics(total *queueGroupAggregate) {

475 + if total == nil {

476 + return

477 + }

478 +

479 + labels := contexts.QueueLabels{

480 + Queue: "__other__",

481 + Type: "aggregated",

482 + }

483 +

484 + contexts.Queue.Depth.Set(c.State, labels, contexts.QueueDepthValues{

485 + Current: total.DepthCurrent,

486 + Max: total.DepthMax,

487 + })

488 +

489 + if total.DepthMax > 0 {

490 + percentage := float64(total.DepthCurrent) / float64(total.DepthMax) * 100.0

491 + contexts.Queue.DepthPercentage.Set(c.State, labels, contexts.QueueDepthPercentageValues{

492 + Percentage: int64(percentage * 1000),

493 + })

494 + }

495 +

496 + if total.HasMessages {

497 + contexts.Queue.Messages.Set(c.State, labels, contexts.QueueMessagesValues{

498 + Enqueued: total.MessagesEnqueued,

499 + Dequeued: total.MessagesDequeued,

500 + })

501 + }

502 +

503 + if total.HasConnections {

504 + contexts.Queue.Connections.Set(c.State, labels, contexts.QueueConnectionsValues{

505 + Input: total.ConnectionsInput,

506 + Output: total.ConnectionsOutput,

507 + })

508 + }

509 +

510 + if total.HasUncommitted {

511 + contexts.Queue.UncommittedMessages.Set(c.State, labels, contexts.QueueUncommittedMessagesValues{

512 + Uncommitted: total.Uncommitted,

513 + })

514 + }

515 +

516 + if total.HasFileSize {

517 + contexts.Queue.FileSize.Set(c.State, labels, contexts.QueueFileSizeValues{

518 + Current: total.FileSizeCurrent,

519 + Max: total.FileSizeMax,

520 + })

521 + }

522 +

523 + if total.HasOldest {

524 + contexts.Queue.OldestMessageAge.Set(c.State, labels, contexts.QueueOldestMessageAgeValues{

525 + Oldest_msg_age: total.OldestMessageAge,

526 + })

527 + }

528 +}

529 +

530 +func (c *Collector) emitQueueGroupMetrics(groups map[string]*queueGroupAggregate) {

531 + if len(groups) == 0 {

532 + return

533 + }

534 +

535 + keys := make([]string, 0, len(groups))

536 + for k := range groups {

537 + keys = append(keys, k)

538 + }

539 + sort.Strings(keys)

540 +

541 + for _, key := range keys {

542 + agg := groups[key]

543 + labels := contexts.QueueGroupLabels{Group: key}

544 +

545 + contexts.QueueGroup.Depth.Set(c.State, labels, contexts.QueueGroupDepthValues{

546 + Current: agg.DepthCurrent,

547 + Max: agg.DepthMax,

548 + })

549 +

550 + if agg.DepthMax > 0 {

551 + percentage := float64(agg.DepthCurrent) / float64(agg.DepthMax) * 100.0

552 + contexts.QueueGroup.DepthPercentage.Set(c.State, labels, contexts.QueueGroupDepthPercentageValues{

553 + Percentage: int64(percentage * 1000),

554 + })

555 + }

556 +

557 + if agg.HasMessages {

558 + contexts.QueueGroup.Messages.Set(c.State, labels, contexts.QueueGroupMessagesValues{

559 + Enqueued: agg.MessagesEnqueued,

560 + Dequeued: agg.MessagesDequeued,

561 + })

562 + }

563 +

564 + if agg.HasConnections {

565 + contexts.QueueGroup.Connections.Set(c.State, labels, contexts.QueueGroupConnectionsValues{

566 + Input: agg.ConnectionsInput,

567 + Output: agg.ConnectionsOutput,

568 + })

569 + }

570 +

571 + if agg.HasUncommitted {

572 + contexts.QueueGroup.UncommittedMessages.Set(c.State, labels, contexts.QueueGroupUncommittedMessagesValues{

573 + Uncommitted: agg.Uncommitted,

574 + })

575 + }

576 +

577 + if agg.HasFileSize {

578 + contexts.QueueGroup.FileSize.Set(c.State, labels, contexts.QueueGroupFileSizeValues{

579 + Current: agg.FileSizeCurrent,

580 + Max: agg.FileSizeMax,

581 + })

582 + }

583 +

584 + if agg.HasOldest {

585 + contexts.QueueGroup.OldestMessageAge.Set(c.State, labels, contexts.QueueGroupOldestMessageAgeValues{

586 + Oldest_msg_age: agg.OldestMessageAge,

587 + })

588 + }

589 + }

590 }

src/go/plugin/ibm.d/modules/mq/collector.go

@@ -4,6 +4,7 @@ import (

4 "sync"

5 "time"

7 + "github.com/netdata/netdata/go/plugins/pkg/matcher"

8 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/framework"

9 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/mq/contexts"

10 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/pcf"

@@ -15,6 +16,9 @@ type Collector struct {

16 Config `yaml:",inline" json:",inline"` // Embed config to receive YAML unmarshal

17 client *pcf.Client

19 + queueIncludeMatcher matcher.Matcher

20 + queueExcludeMatcher matcher.Matcher

21 +

22 // Resolved effective intervals (auto-detected or user-configured)

23 effectiveStatisticsInterval int

24 effectiveSysTopicInterval int

src/go/plugin/ibm.d/modules/mq/config.go

+4 -2

@@ -44,8 +44,10 @@ type Config struct {

44 // Enable collection of queue configuration metrics

45 CollectQueueConfig bool `yaml:"collect_queue_config" json:"collect_queue_config"`

47 - // Pattern to filter queues (wildcards supported)

48 - QueueSelector string `yaml:"queue_selector" json:"queue_selector"`

47 + // Patterns to include queues (wildcards supported). Empty means include everything.

48 + IncludeQueues []string `yaml:"include_queues" json:"include_queues"`

49 + // Patterns to exclude queues after inclusion (wildcards supported).

50 + ExcludeQueues []string `yaml:"exclude_queues" json:"exclude_queues"`

51 // Pattern to filter channels (wildcards supported)

52 ChannelSelector string `yaml:"channel_selector" json:"channel_selector"`

53 // Pattern to filter topics (wildcards supported)

src/go/plugin/ibm.d/modules/mq/config_schema.json

+34 -8

@@ -98,12 +98,37 @@

98 "title": "Collect Topics",

99 "type": "boolean"

100 },

101 + "exclude_queues": {

102 + "default": [

103 + "SYSTEM.*",

104 + "AMQ.*"

105 + ],

106 + "description": "Patterns to exclude queues after inclusion (wildcards supported).",

107 + "items": {

108 + "type": "string"

109 + },

110 + "title": "Exclude Queues",

111 + "type": "array"

112 + },

113 "host": {

114 "default": "localhost",

115 "description": "IBM MQ server hostname or IP address",

116 "title": "Host",

117 "type": "string"

118 },

119 + "include_queues": {

120 + "default": [

121 + "SYSTEM.DEAD.LETTER.QUEUE",

122 + "SYSTEM.ADMIN.COMMAND.QUEUE",

123 + "SYSTEM.ADMIN.STATISTICS.QUEUE"

124 + ],

125 + "description": "Patterns to include queues (wildcards supported). Empty means include everything.",

126 + "items": {

127 + "type": "string"

128 + },

129 + "title": "Include Queues",

130 + "type": "array"

131 + },

132 "listener_selector": {

133 "default": "",

134 "description": "Pattern to filter listeners (wildcards supported)",

@@ -123,7 +148,7 @@

148 "type": "integer"

149 },

150 "max_queues": {

126 - "default": 100,

151 + "default": 50,

152 "description": "Maximum number of queues to collect (0 = no limit)",

153 "title": "Max Queues",

154 "type": "integer"

@@ -155,12 +180,6 @@

180 "title": "Queue Manager",

181 "type": "string"

182 },

158 - "queue_selector": {

159 - "default": "",

160 - "description": "Pattern to filter queues (wildcards supported)",

161 - "title": "Queue Selector",

162 - "type": "string"

163 - },

183 "statistics_interval,omitempty": {

184 "default": 60,

185 "description": "Statistics collection interval in seconds (auto-detected STATINT overwrites this value)",

@@ -203,6 +222,12 @@

222 "type": "object"

223 },

224 "uiSchema": {

225 + "exclude_queues": {

226 + "ui:listFlavour": "list"

227 + },

228 + "include_queues": {

229 + "ui:listFlavour": "list"

230 + },

231 "password": {

232 "ui:widget": "password"

233 },

@@ -222,6 +247,8 @@

247 "host",

248 "port",

249 "user",

250 + "include_queues",

251 + "exclude_queues",

252 "statistics_interval,omitempty",

253 "sys_topic_interval,omitempty"

254 ],

@@ -254,7 +281,6 @@

281 },

282 {

283 "fields": [

257 - "queue_selector",

284 "channel_selector",

285 "topic_selector",

286 "listener_selector",

src/go/plugin/ibm.d/modules/mq/contexts/contexts.yaml

+72

@@ -357,6 +357,78 @@ Queue:

357 - {name: enabled, algo: absolute}

358 - {name: disabled, algo: absolute}

359

360 +QueueGroup:

361 + labels:

362 + - group

363 + contexts:

364 + - name: Depth

365 + context: mq.queue_group.depth

366 + title: Queue Group Depth

367 + units: messages

368 + family: queues/group

369 + type: line

370 + priority: 2450

371 + dimensions:

372 + - {name: current, algo: absolute}

373 + - {name: max, algo: absolute}

374 + - name: DepthPercentage

375 + context: mq.queue_group.depth_percentage

376 + title: Queue Group Depth Percentage

377 + units: percentage

378 + family: queues/group

379 + type: line

380 + priority: 2451

381 + dimensions:

382 + - {name: percentage, algo: absolute, precision: 1000, div: 1000}

383 + - name: Messages

384 + context: mq.queue_group.messages

385 + title: Queue Group Messages

386 + units: messages/s

387 + family: queues/group

388 + type: line

389 + priority: 2452

390 + dimensions:

391 + - {name: enqueued, algo: incremental}

392 + - {name: dequeued, algo: incremental}

393 + - name: Connections

394 + context: mq.queue_group.connections

395 + title: Queue Group Connections

396 + units: connections

397 + family: queues/group

398 + type: line

399 + priority: 2453

400 + dimensions:

401 + - {name: input, algo: absolute}

402 + - {name: output, algo: absolute}

403 + - name: UncommittedMessages

404 + context: mq.queue_group.uncommitted_msgs

405 + title: Queue Group Uncommitted Messages

406 + units: messages

407 + family: queues/group

408 + type: line

409 + priority: 2454

410 + dimensions:

411 + - {name: uncommitted, algo: absolute}

412 + - name: FileSize

413 + context: mq.queue_group.file_size

414 + title: Queue Group File Size

415 + units: bytes

416 + family: queues/group

417 + type: line

418 + priority: 2455

419 + dimensions:

420 + - {name: current, algo: absolute}

421 + - {name: max, algo: absolute}

422 + - name: OldestMessageAge

423 + context: mq.queue_group.oldest_msg_age

424 + title: Queue Group Oldest Message Age

425 + units: seconds

426 + family: queues/group

427 + type: line

428 + priority: 2456

429 + dimensions:

430 + - {name: oldest_msg_age, algo: absolute}

431 +

432 Channel:

433 labels:

434 - channel

src/go/plugin/ibm.d/modules/mq/contexts/zz_generated_contexts.go

+383

@@ -2990,6 +2990,382 @@ var Queue = struct {

2990 },

2991 }

2992

2993 +// --- QueueGroup ---

2994 +

2995 +// QueueGroupDepthValues defines the type-safe values for QueueGroup.Depth context

2996 +type QueueGroupDepthValues struct {

2997 + Current int64

2998 + Max int64

2999 +}

3000 +

3001 +// QueueGroupDepthContext provides type-safe operations for QueueGroup.Depth context

3002 +type QueueGroupDepthContext struct {

3003 + framework.Context[QueueGroupLabels]

3004 +}

3005 +

3006 +// Set provides type-safe dimension setting for QueueGroup.Depth context

3007 +func (c QueueGroupDepthContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupDepthValues) {

3008 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3009 + "current": values.Current,

3010 + "max": values.Max,

3011 + })

3012 +}

3013 +

3014 +// SetUpdateEvery sets the update interval for this instance

3015 +func (c QueueGroupDepthContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3016 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3017 +}

3018 +

3019 +// QueueGroupDepthPercentageValues defines the type-safe values for QueueGroup.DepthPercentage context

3020 +type QueueGroupDepthPercentageValues struct {

3021 + Percentage int64

3022 +}

3023 +

3024 +// QueueGroupDepthPercentageContext provides type-safe operations for QueueGroup.DepthPercentage context

3025 +type QueueGroupDepthPercentageContext struct {

3026 + framework.Context[QueueGroupLabels]

3027 +}

3028 +

3029 +// Set provides type-safe dimension setting for QueueGroup.DepthPercentage context

3030 +func (c QueueGroupDepthPercentageContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupDepthPercentageValues) {

3031 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3032 + "percentage": values.Percentage,

3033 + })

3034 +}

3035 +

3036 +// SetUpdateEvery sets the update interval for this instance

3037 +func (c QueueGroupDepthPercentageContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3038 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3039 +}

3040 +

3041 +// QueueGroupMessagesValues defines the type-safe values for QueueGroup.Messages context

3042 +type QueueGroupMessagesValues struct {

3043 + Enqueued int64

3044 + Dequeued int64

3045 +}

3046 +

3047 +// QueueGroupMessagesContext provides type-safe operations for QueueGroup.Messages context

3048 +type QueueGroupMessagesContext struct {

3049 + framework.Context[QueueGroupLabels]

3050 +}

3051 +

3052 +// Set provides type-safe dimension setting for QueueGroup.Messages context

3053 +func (c QueueGroupMessagesContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupMessagesValues) {

3054 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3055 + "enqueued": values.Enqueued,

3056 + "dequeued": values.Dequeued,

3057 + })

3058 +}

3059 +

3060 +// SetUpdateEvery sets the update interval for this instance

3061 +func (c QueueGroupMessagesContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3062 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3063 +}

3064 +

3065 +// QueueGroupConnectionsValues defines the type-safe values for QueueGroup.Connections context

3066 +type QueueGroupConnectionsValues struct {

3067 + Input int64

3068 + Output int64

3069 +}

3070 +

3071 +// QueueGroupConnectionsContext provides type-safe operations for QueueGroup.Connections context

3072 +type QueueGroupConnectionsContext struct {

3073 + framework.Context[QueueGroupLabels]

3074 +}

3075 +

3076 +// Set provides type-safe dimension setting for QueueGroup.Connections context

3077 +func (c QueueGroupConnectionsContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupConnectionsValues) {

3078 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3079 + "input": values.Input,

3080 + "output": values.Output,

3081 + })

3082 +}

3083 +

3084 +// SetUpdateEvery sets the update interval for this instance

3085 +func (c QueueGroupConnectionsContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3086 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3087 +}

3088 +

3089 +// QueueGroupUncommittedMessagesValues defines the type-safe values for QueueGroup.UncommittedMessages context

3090 +type QueueGroupUncommittedMessagesValues struct {

3091 + Uncommitted int64

3092 +}

3093 +

3094 +// QueueGroupUncommittedMessagesContext provides type-safe operations for QueueGroup.UncommittedMessages context

3095 +type QueueGroupUncommittedMessagesContext struct {

3096 + framework.Context[QueueGroupLabels]

3097 +}

3098 +

3099 +// Set provides type-safe dimension setting for QueueGroup.UncommittedMessages context

3100 +func (c QueueGroupUncommittedMessagesContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupUncommittedMessagesValues) {

3101 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3102 + "uncommitted": values.Uncommitted,

3103 + })

3104 +}

3105 +

3106 +// SetUpdateEvery sets the update interval for this instance

3107 +func (c QueueGroupUncommittedMessagesContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3108 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3109 +}

3110 +

3111 +// QueueGroupFileSizeValues defines the type-safe values for QueueGroup.FileSize context

3112 +type QueueGroupFileSizeValues struct {

3113 + Current int64

3114 + Max int64

3115 +}

3116 +

3117 +// QueueGroupFileSizeContext provides type-safe operations for QueueGroup.FileSize context

3118 +type QueueGroupFileSizeContext struct {

3119 + framework.Context[QueueGroupLabels]

3120 +}

3121 +

3122 +// Set provides type-safe dimension setting for QueueGroup.FileSize context

3123 +func (c QueueGroupFileSizeContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupFileSizeValues) {

3124 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3125 + "current": values.Current,

3126 + "max": values.Max,

3127 + })

3128 +}

3129 +

3130 +// SetUpdateEvery sets the update interval for this instance

3131 +func (c QueueGroupFileSizeContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3132 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3133 +}

3134 +

3135 +// QueueGroupOldestMessageAgeValues defines the type-safe values for QueueGroup.OldestMessageAge context

3136 +type QueueGroupOldestMessageAgeValues struct {

3137 + Oldest_msg_age int64

3138 +}

3139 +

3140 +// QueueGroupOldestMessageAgeContext provides type-safe operations for QueueGroup.OldestMessageAge context

3141 +type QueueGroupOldestMessageAgeContext struct {

3142 + framework.Context[QueueGroupLabels]

3143 +}

3144 +

3145 +// Set provides type-safe dimension setting for QueueGroup.OldestMessageAge context

3146 +func (c QueueGroupOldestMessageAgeContext) Set(state *framework.CollectorState, labels QueueGroupLabels, values QueueGroupOldestMessageAgeValues) {

3147 + state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{

3148 + "oldest_msg_age": values.Oldest_msg_age,

3149 + })

3150 +}

3151 +

3152 +// SetUpdateEvery sets the update interval for this instance

3153 +func (c QueueGroupOldestMessageAgeContext) SetUpdateEvery(state *framework.CollectorState, labels QueueGroupLabels, updateEvery int) {

3154 + state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)

3155 +}

3156 +

3157 +// QueueGroupLabels defines the required labels for QueueGroup contexts

3158 +type QueueGroupLabels struct {

3159 + Group string

3160 +}

3161 +

3162 +// InstanceID generates a unique instance ID using the hardcoded label order from YAML

3163 +func (l QueueGroupLabels) InstanceID(contextName string) string {

3164 + // Label order from YAML: group

3165 + return contextName + "." + cleanLabelValue(l.Group)

3166 +}

3167 +

3168 +// QueueGroup contains all metric contexts for QueueGroup

3169 +var QueueGroup = struct {

3170 + Depth QueueGroupDepthContext

3171 + DepthPercentage QueueGroupDepthPercentageContext

3172 + Messages QueueGroupMessagesContext

3173 + Connections QueueGroupConnectionsContext

3174 + UncommittedMessages QueueGroupUncommittedMessagesContext

3175 + FileSize QueueGroupFileSizeContext

3176 + OldestMessageAge QueueGroupOldestMessageAgeContext

3177 +}{

3178 + Depth: QueueGroupDepthContext{

3179 + Context: framework.Context[QueueGroupLabels]{

3180 + Name: "mq.queue_group.depth",

3181 + Family: "queues/group",

3182 + Title: "Queue Group Depth",

3183 + Units: "messages",

3184 + Type: module.Line,

3185 + Priority: 2450,

3186 + UpdateEvery: 1,

3187 + Dimensions: []framework.Dimension{

3188 + {

3189 + Name: "current",

3190 + Algorithm: module.Absolute,

3191 + Mul: 1,

3192 + Div: 1,

3193 + Precision: 1,

3194 + },

3195 + {

3196 + Name: "max",

3197 + Algorithm: module.Absolute,

3198 + Mul: 1,

3199 + Div: 1,

3200 + Precision: 1,

3201 + },

3202 + },

3203 + LabelKeys: []string{

3204 + "group",

3205 + },

3206 + },

3207 + },

3208 + DepthPercentage: QueueGroupDepthPercentageContext{

3209 + Context: framework.Context[QueueGroupLabels]{

3210 + Name: "mq.queue_group.depth_percentage",

3211 + Family: "queues/group",

3212 + Title: "Queue Group Depth Percentage",

3213 + Units: "percentage",

3214 + Type: module.Line,

3215 + Priority: 2451,

3216 + UpdateEvery: 1,

3217 + Dimensions: []framework.Dimension{

3218 + {

3219 + Name: "percentage",

3220 + Algorithm: module.Absolute,

3221 + Mul: 1,

3222 + Div: 1000,

3223 + Precision: 1000,

3224 + },

3225 + },

3226 + LabelKeys: []string{

3227 + "group",

3228 + },

3229 + },

3230 + },

3231 + Messages: QueueGroupMessagesContext{

3232 + Context: framework.Context[QueueGroupLabels]{

3233 + Name: "mq.queue_group.messages",

3234 + Family: "queues/group",

3235 + Title: "Queue Group Messages",

3236 + Units: "messages/s",

3237 + Type: module.Line,

3238 + Priority: 2452,

3239 + UpdateEvery: 1,

3240 + Dimensions: []framework.Dimension{

3241 + {

3242 + Name: "enqueued",

3243 + Algorithm: module.Incremental,

3244 + Mul: 1,

3245 + Div: 1,

3246 + Precision: 1,

3247 + },

3248 + {

3249 + Name: "dequeued",

3250 + Algorithm: module.Incremental,

3251 + Mul: 1,

3252 + Div: 1,

3253 + Precision: 1,

3254 + },

3255 + },

3256 + LabelKeys: []string{

3257 + "group",

3258 + },

3259 + },

3260 + },

3261 + Connections: QueueGroupConnectionsContext{

3262 + Context: framework.Context[QueueGroupLabels]{

3263 + Name: "mq.queue_group.connections",

3264 + Family: "queues/group",

3265 + Title: "Queue Group Connections",

3266 + Units: "connections",

3267 + Type: module.Line,

3268 + Priority: 2453,

3269 + UpdateEvery: 1,

3270 + Dimensions: []framework.Dimension{

3271 + {

3272 + Name: "input",

3273 + Algorithm: module.Absolute,

3274 + Mul: 1,

3275 + Div: 1,

3276 + Precision: 1,

3277 + },

3278 + {

3279 + Name: "output",

3280 + Algorithm: module.Absolute,

3281 + Mul: 1,

3282 + Div: 1,

3283 + Precision: 1,

3284 + },

3285 + },

3286 + LabelKeys: []string{

3287 + "group",

3288 + },

3289 + },

3290 + },

3291 + UncommittedMessages: QueueGroupUncommittedMessagesContext{

3292 + Context: framework.Context[QueueGroupLabels]{

3293 + Name: "mq.queue_group.uncommitted_msgs",

3294 + Family: "queues/group",

3295 + Title: "Queue Group Uncommitted Messages",

3296 + Units: "messages",

3297 + Type: module.Line,

3298 + Priority: 2454,

3299 + UpdateEvery: 1,

3300 + Dimensions: []framework.Dimension{

3301 + {

3302 + Name: "uncommitted",

3303 + Algorithm: module.Absolute,

3304 + Mul: 1,

3305 + Div: 1,

3306 + Precision: 1,

3307 + },

3308 + },

3309 + LabelKeys: []string{

3310 + "group",

3311 + },

3312 + },

3313 + },

3314 + FileSize: QueueGroupFileSizeContext{

3315 + Context: framework.Context[QueueGroupLabels]{

3316 + Name: "mq.queue_group.file_size",

3317 + Family: "queues/group",

3318 + Title: "Queue Group File Size",

3319 + Units: "bytes",

3320 + Type: module.Line,

3321 + Priority: 2455,

3322 + UpdateEvery: 1,

3323 + Dimensions: []framework.Dimension{

3324 + {

3325 + Name: "current",

3326 + Algorithm: module.Absolute,

3327 + Mul: 1,

3328 + Div: 1,

3329 + Precision: 1,

3330 + },

3331 + {

3332 + Name: "max",

3333 + Algorithm: module.Absolute,

3334 + Mul: 1,

3335 + Div: 1,

3336 + Precision: 1,

3337 + },

3338 + },

3339 + LabelKeys: []string{

3340 + "group",

3341 + },

3342 + },

3343 + },

3344 + OldestMessageAge: QueueGroupOldestMessageAgeContext{

3345 + Context: framework.Context[QueueGroupLabels]{

3346 + Name: "mq.queue_group.oldest_msg_age",

3347 + Family: "queues/group",

3348 + Title: "Queue Group Oldest Message Age",

3349 + Units: "seconds",

3350 + Type: module.Line,

3351 + Priority: 2456,

3352 + UpdateEvery: 1,

3353 + Dimensions: []framework.Dimension{

3354 + {

3355 + Name: "oldest_msg_age",

3356 + Algorithm: module.Absolute,

3357 + Mul: 1,

3358 + Div: 1,

3359 + Precision: 1,

3360 + },

3361 + },

3362 + LabelKeys: []string{

3363 + "group",

3364 + },

3365 + },

3366 + },

3367 +}

3368 +

3369 // --- QueueManager ---

3370

3371 // QueueManagerStatusValues defines the type-safe values for QueueManager.Status context

@@ -4566,6 +4942,13 @@ func GetAllContexts() []interface{} {

4942 &Queue.QueueUsage.Context,

4943 &Queue.MessageDeliverySequence.Context,

4944 &Queue.HardenGetBackout.Context,

4945 + &QueueGroup.Depth.Context,

4946 + &QueueGroup.DepthPercentage.Context,

4947 + &QueueGroup.Messages.Context,

4948 + &QueueGroup.Connections.Context,

4949 + &QueueGroup.UncommittedMessages.Context,

4950 + &QueueGroup.FileSize.Context,

4951 + &QueueGroup.OldestMessageAge.Context,

4952 &QueueManager.Status.Context,

4953 &QueueManager.ConnectionCount.Context,

4954 &QueueManager.Uptime.Context,

src/go/plugin/ibm.d/modules/mq/init.go

+47 -9

@@ -4,11 +4,27 @@ import (

4 "context"

5 "encoding/json"

6 "fmt"

7 + "strings"

9 + "github.com/netdata/netdata/go/plugins/pkg/matcher"

10 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/mq/contexts"

11 "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/pcf"

12 )

14 +func compileMatcher(patterns []string) (matcher.Matcher, error) {

15 + if len(patterns) == 0 {

16 + return nil, nil

17 + }

18 +

19 + expr := strings.Join(patterns, " ")

20 + expr = strings.TrimSpace(expr)

21 + if expr == "" {

22 + return nil, nil

23 + }

24 +

25 + return matcher.NewSimplePatternsMatcher(expr)

26 +}

27 +

28 // defaultConfig returns a new Config with all default values set.

29 // This is used both for New() and for the module registration to ensure

30 // consistency and single source of truth for defaults.

@@ -46,15 +62,24 @@ func defaultConfig() Config {

62 StatisticsInterval: 60, // Default 60s, auto-detected STATINT overwrites if available

63 SysTopicInterval: 10, // Default 10s per IBM docs, user can override if customized

49 - // Selector defaults - empty means collect nothing (user must explicitly configure)

50 - QueueSelector: "",

65 + // Queue selection defaults: monitor core system queues, ignore churny patterns

66 + IncludeQueues: []string{

67 + "SYSTEM.DEAD.LETTER.QUEUE",

68 + "SYSTEM.ADMIN.COMMAND.QUEUE",

69 + "SYSTEM.ADMIN.STATISTICS.QUEUE",

70 + },

71 + ExcludeQueues: []string{

72 + "SYSTEM.*",

73 + "AMQ.*",

74 + },

75 +

76 ChannelSelector: "",

77 TopicSelector: "",

78 ListenerSelector: "",

79 SubscriptionSelector: "",

81 // Cardinality control defaults

57 - MaxQueues: 100,

82 + MaxQueues: 50,

83 MaxChannels: 100,

84 MaxTopics: 100,

85 MaxListeners: 100,

@@ -106,13 +131,26 @@ func (c *Collector) Init(ctx context.Context) error {

131 Password: c.Config.Password,

132 }, c.State)

133

109 - // Log the selectors - these are applied locally after discovery

110 - if c.Config.QueueSelector == "" {

111 - c.Infof("Queue selector: empty (no queues will be collected)")

112 - } else if c.Config.QueueSelector == "*" {

113 - c.Infof("Queue selector: all queues will be collected")

134 + includeMatcher, err := compileMatcher(c.Config.IncludeQueues)

135 + if err != nil {

136 + return fmt.Errorf("invalid include_queues patterns: %w", err)

137 + }

138 + excludeMatcher, err := compileMatcher(c.Config.ExcludeQueues)

139 + if err != nil {

140 + return fmt.Errorf("invalid exclude_queues patterns: %w", err)

141 + }

142 + c.queueIncludeMatcher = includeMatcher

143 + c.queueExcludeMatcher = excludeMatcher

144 +

145 + if len(c.Config.IncludeQueues) == 0 {

146 + c.Infof("Queue include patterns: none (all queues eligible)")

147 + } else {

148 + c.Infof("Queue include patterns: %v", c.Config.IncludeQueues)

149 + }

150 + if len(c.Config.ExcludeQueues) == 0 {

151 + c.Infof("Queue exclude patterns: none")

152 } else {

115 - c.Infof("Queue selector configured: %s (applied after discovery)", c.Config.QueueSelector)

153 + c.Infof("Queue exclude patterns: %v", c.Config.ExcludeQueues)

154 }

155

156 if c.Config.ChannelSelector == "" {

src/go/plugin/ibm.d/modules/mq/metadata.yaml

+65

@@ -24,6 +24,19 @@ modules:

24 Monitors IBM MQ queue managers, queues, channels, and topics

25 using the PCF (Programmable Command Format) protocol.

27 + By default the collector tracks the critical system queues `SYSTEM.DEAD.LETTER.QUEUE`,

28 + `SYSTEM.ADMIN.COMMAND.QUEUE`, and `SYSTEM.ADMIN.STATISTICS.QUEUE`. All other queues are

29 + opt-in via the `include_queues` list, with `exclude_queues` removing noisy patterns such as

30 + `SYSTEM.*` or `AMQ.*`. Include patterns take precedence over excludes so you can safely

31 + monitor individual system queues while dropping the broader wildcard.

32 +

33 + Per-queue charts are bounded by `max_queues` (default 50). When more queues are discovered,

34 + the collector exports the busiest ones individually, rolls the remainder into an

35 + aggregated `__other__` dimension, and logs a throttled warning listing the overflowed

36 + groups. Parallel queue-group charts summarise depth, traffic, and backlog per naming

37 + prefix (first two dot-separated segments, collapsing all `SYSTEM.*` queues together), so

38 + high-level visibility is never lost even when detailed charts are trimmed.

39 +

40 method_description: |

41 The collector connects to IBM MQ and collects metrics via its monitoring interface.

42 supported_platforms:

@@ -478,6 +491,58 @@ modules:

491 dimensions:

492 - name: enabled

493 - name: disabled

494 + - name: queuegroup

495 + description: These metrics refer to queuegroup instances.

496 + labels:

497 + - name: group

498 + description: Group identifier

499 + metrics:

500 + - name: mq.queue_group.depth

501 + description: Queue Group Depth

502 + unit: messages

503 + chart_type: line

504 + dimensions:

505 + - name: current

506 + - name: max

507 + - name: mq.queue_group.depth_percentage

508 + description: Queue Group Depth Percentage

509 + unit: percentage

510 + chart_type: line

511 + dimensions:

512 + - name: percentage

513 + - name: mq.queue_group.messages

514 + description: Queue Group Messages

515 + unit: messages/s

516 + chart_type: line

517 + dimensions:

518 + - name: enqueued

519 + - name: dequeued

520 + - name: mq.queue_group.connections

521 + description: Queue Group Connections

522 + unit: connections

523 + chart_type: line

524 + dimensions:

525 + - name: input

526 + - name: output

527 + - name: mq.queue_group.uncommitted_msgs

528 + description: Queue Group Uncommitted Messages

529 + unit: messages

530 + chart_type: line

531 + dimensions:

532 + - name: uncommitted

533 + - name: mq.queue_group.file_size

534 + description: Queue Group File Size

535 + unit: bytes

536 + chart_type: line

537 + dimensions:

538 + - name: current

539 + - name: max

540 + - name: mq.queue_group.oldest_msg_age

541 + description: Queue Group Oldest Message Age

542 + unit: seconds

543 + chart_type: line

544 + dimensions:

545 + - name: oldest_msg_age

546 - name: global

547 description: These metrics refer to the entire monitored instance.

548 labels: []

src/go/plugin/ibm.d/modules/mq/module.yaml

+13

@@ -3,6 +3,19 @@ display_name: IBM MQ

3 description: |

4 Monitors IBM MQ queue managers, queues, channels, and topics

5 using the PCF (Programmable Command Format) protocol.

6 +

7 + By default the collector tracks the critical system queues `SYSTEM.DEAD.LETTER.QUEUE`,

8 + `SYSTEM.ADMIN.COMMAND.QUEUE`, and `SYSTEM.ADMIN.STATISTICS.QUEUE`. All other queues are

9 + opt-in via the `include_queues` list, with `exclude_queues` removing noisy patterns such as

10 + `SYSTEM.*` or `AMQ.*`. Include patterns take precedence over excludes so you can safely

11 + monitor individual system queues while dropping the broader wildcard.

12 +

13 + Per-queue charts are bounded by `max_queues` (default 50). When more queues are discovered,

14 + the collector exports the busiest ones individually, rolls the remainder into an

15 + aggregated `__other__` dimension, and logs a throttled warning listing the overflowed

16 + groups. Parallel queue-group charts summarise depth, traffic, and backlog per naming

17 + prefix (first two dot-separated segments, collapsing all `SYSTEM.*` queues together), so

18 + high-level visibility is never lost even when detailed charts are trimmed.

19 icon: ibm-mq.svg

20 categories:

21 - data-collection.message-brokers

src/go/plugin/ibm.d/pkg/odbcbridge/bridge.c

+100 -12

@@ -5,6 +5,9 @@

5 #include <string.h>

6 #include <stdio.h>

8 +#define ODBC_DEFAULT_BUFFER_SIZE 4096

9 +#define ODBC_MAX_BUFFER_SIZE (16 * 1024 * 1024)

10 +

11 // Connection structure with optimizations

12 typedef struct {

13 SQLHENV env;

@@ -379,28 +382,113 @@ int odbc_get_value(odbc_conn_t conn_handle, int column_index, odbc_value_t* valu

382 break;

383 }

384

385 + case ODBC_TYPE_BINARY: {

386 + // Determine required buffer size

387 + char dummy[1];

388 + SQLLEN binary_len = 0;

389 +

390 + ret = SQLGetData(conn->stmt, column_index + 1, SQL_C_BINARY,

391 + dummy, 0, &binary_len);

392 +

393 + if (!SQL_SUCCEEDED(ret) && ret != SQL_SUCCESS_WITH_INFO) {

394 + return ODBC_ERROR;

395 + }

396 +

397 + if (binary_len == SQL_NULL_DATA) {

398 + value->is_null = true;

399 + return ODBC_SUCCESS;

400 + }

401 +

402 + size_t buffer_size = ODBC_DEFAULT_BUFFER_SIZE;

403 + if (binary_len > 0 && binary_len != SQL_NO_TOTAL) {

404 + size_t required = (size_t)binary_len;

405 + if (required > buffer_size) {

406 + buffer_size = required;

407 + }

408 + }

409 +

410 + if (buffer_size > ODBC_MAX_BUFFER_SIZE) {

411 + buffer_size = ODBC_MAX_BUFFER_SIZE;

412 + }

413 +

414 + if (buffer_size == 0) {

415 + buffer_size = ODBC_DEFAULT_BUFFER_SIZE;

416 + }

417 +

418 + void* buffer = malloc(buffer_size);

419 + if (!buffer) {

420 + return ODBC_ERROR;

421 + }

422 +

423 + ret = SQLGetData(conn->stmt, column_index + 1, SQL_C_BINARY,

424 + buffer, buffer_size, &indicator);

425 +

426 + if (SQL_SUCCEEDED(ret) || ret == SQL_SUCCESS_WITH_INFO) {

427 + if (indicator == SQL_NULL_DATA) {

428 + value->is_null = true;

429 + free(buffer);

430 + } else {

431 + value->type = ODBC_TYPE_BINARY;

432 + value->is_null = false;

433 + if (indicator >= 0 && indicator <= (SQLLEN)buffer_size) {

434 + value->data.binary_val.len = (size_t)indicator;

435 + } else if (indicator == SQL_NO_TOTAL || indicator > (SQLLEN)buffer_size) {

436 + value->data.binary_val.len = buffer_size;

437 + } else {

438 + value->data.binary_val.len = 0;

439 + }

440 + value->data.binary_val.data = buffer;

441 + }

442 + return ODBC_SUCCESS;

443 + }

444 +

445 + free(buffer);

446 + break;

447 + }

448 +

449 case ODBC_TYPE_STRING:

450 default: {

451 // First call to get the required buffer size

452 char dummy[1];

386 - SQLLEN str_len_or_ind;

387 -

453 + SQLLEN str_len_or_ind = 0;

454 +

455 ret = SQLGetData(conn->stmt, column_index + 1, SQL_C_CHAR,

456 dummy, 0, &str_len_or_ind);

390 -

457 +

458 + if (!SQL_SUCCEEDED(ret) && ret != SQL_SUCCESS_WITH_INFO) {

459 + return ODBC_ERROR;

460 + }

461 +

462 if (str_len_or_ind == SQL_NULL_DATA) {

463 value->is_null = true;

464 return ODBC_SUCCESS;

465 }

395 -

396 - // Allocate buffer and get the actual data

397 - size_t buffer_size = (str_len_or_ind > 0) ? str_len_or_ind + 1 : 4096;

466 +

467 + size_t buffer_size = ODBC_DEFAULT_BUFFER_SIZE;

468 + if (str_len_or_ind > 0 && str_len_or_ind != SQL_NO_TOTAL) {

469 + size_t required = (size_t)str_len_or_ind + 1;

470 + if (required > buffer_size) {

471 + buffer_size = required;

472 + }

473 + }

474 +

475 + if (buffer_size > ODBC_MAX_BUFFER_SIZE) {

476 + buffer_size = ODBC_MAX_BUFFER_SIZE;

477 + }

478 +

479 + if (buffer_size == 0) {

480 + buffer_size = ODBC_DEFAULT_BUFFER_SIZE;

481 + }

482 +

483 char* buffer = malloc(buffer_size);

399 -

484 + if (!buffer) {

485 + return ODBC_ERROR;

486 + }

487 +

488 ret = SQLGetData(conn->stmt, column_index + 1, SQL_C_CHAR,

489 buffer, buffer_size, &indicator);

402 -

403 - if (SQL_SUCCEEDED(ret)) {

490 +

491 + if (SQL_SUCCEEDED(ret) || ret == SQL_SUCCESS_WITH_INFO) {

492 if (indicator == SQL_NULL_DATA) {

493 value->is_null = true;

494 free(buffer);

@@ -410,13 +498,13 @@ int odbc_get_value(odbc_conn_t conn_handle, int column_index, odbc_value_t* valu

498 value->data.string_val = buffer;

499 }

500 return ODBC_SUCCESS;

413 - } else {

414 - free(buffer);

501 }

502 +

503 + free(buffer);

504 break;

505 }

506 }

419 -

507 +

508 return ODBC_ERROR;

509 }

510