As400 part3: performance optimizations (#21164)
* optimize message queues * reworked the output_queues * as400: require explicit active job targets * removed old todo * Refactor AS400 latency counters and slow-path cache * as400: read system name from SYSTEM_STATUS_INFO * loggin imrovements across all modules * Align AS400 batch path latency handling with slow path
Costa Tsaousis committed
Oct 23, 2025 at 20:30 UTC
2300493f22fa868fcf9a4bb0b1257de2670b7094
43 files changed
+4323
-1855
TODO-AS400-SPEED.md
new
+112
@@ -0,0 +1,112 @@
1
+# IBM i AS400 Collector – Query Performance Roadmap
2
+
3
+## 1. Current Situation
4
+- The latency chart (`netdata.plugin_ibm.as400_query_latency`) for the last 10 minutes shows sustained slowdowns from three collectors:
5
+ - `job_queues` / `count_job_queues`: average ~2.9 s, peaks >4.0 s.
6
+ - `output_queue_info`: average ~2.8 s, peaks >4.3 s.
7
+ - `other`: residual bucket averaging ~1.2 s (needs attribution).
8
+- Message queue collectors are currently disabled in production (0 ms latency), but the SQL still performs full scans when enabled.
9
+- The collector remains single-threaded; any slow query blocks all 5 s metrics.
10
+
11
+## 2. Immediate Offenders (must-fix)
12
+The following collectors must be reworked first; they combine high latency with unbounded scans:
13
+
14
+| Query | Source | Problem | Required Fix |
15
+| --- | --- | --- | --- |
16
+| `message_queue_aggregates`, `count_message_queues` | `QSYS2.MESSAGE_QUEUE_INFO` (detail table, full scan) | Full table scan, aggregation, `FETCH FIRST` post-scan | ✅ done (table function per queue, documented fallback + defaults) |
17
+| `collectJobQueues` (`queryJobQueues`) | `QSYS2.JOB_QUEUE_INFO` | View per queue with WHERE filter; still executes every fast loop | ✅ done (slow-path cache with per-queue fan-out and concurrency) |
18
+| `collectOutputQueues` (`queryOutputQueueInfo`) | `QSYS2.OUTPUT_QUEUE_INFO` | View-based per queue | ✅ done (table function per queue with view fallback; counts derive from entries) |
19
+| `collectActiveJobs` (`buildActiveJobQuery`) | `TABLE(QSYS2.ACTIVE_JOB_INFO(...))` per job | ✅ done (requires explicit `active_jobs` list; one table-function call per job with user/job filters, skips when not found) |
20
+| `collectSubsystems` (`querySubsystems` + `queryCountSubsystems`) | `QSYS2.SUBSYSTEM_INFO` | View | Aggregates active subsystems every 15 s | ✅ done (slow-path cache with interval override) |
21
+| `collectPlanCache` (`CALL QSYS2.ANALYZE_PLAN_CACHE('03', …)`) | Stored procedure | Walks the entire plan cache every cycle, blocking fast loop | ✅ done (runs on slow-path worker; latency separated) |
22
+
23
+## 3. Full Query Inventory & Table Scan Risk
24
+
25
+Legend: ✅ (fine), ⚠️ (needs filtering/guardrails), ❗️ (full scan / heavy).
26
+
27
+| Query Constant | Source | Type | Filters today | Risk | Notes |
28
+| --- | --- | --- | --- | --- | --- |
29
+| `querySystemStatus*`, `querySystemActivity*`, `queryConfiguredCPUs`, `queryAverageCPU`, `querySystemASP`, `queryActiveJobs`, `querySerialNumber`, `querySystemModel` | `TABLE(QSYS2.SYSTEM_STATUS(...))` | Table function | No additional filters | ✅ | Table function already scoped by IBM; negligible overhead |
30
+| `queryMemoryPools*` | `TABLE(QSYS2.MEMORY_POOL(...))` | Table function | Hard-coded pool names | ✅ | Already filtered |
31
+| `queryDiskStatus` | `QSYS2.SYSDISKSTAT` | View | No filter | ⚠️ | Should add unit selector `WHERE UNIT_NUMBER IN (...)` when available |
32
+| `queryDiskInstances`, `queryDiskInstancesEnhanced` | `QSYS2.SYSDISKSTAT` | View | No filter other than Go-side matcher | ⚠️ | Translate selector to `WHERE UNIT_NUMBER IN (...)` when matcher configured |
33
+| `queryCountDisks` | `QSYS2.SYSDISKSTAT` | View | DISTINCT, no filter | ⚠️ | Same as above; cache counts if possible |
34
+| `queryJobInfo` | `TABLE(QSYS2.JOB_INFO(JOB_STATUS_FILTER => '*JOBQ'))` | Table function | Filter built-in | ✅ | Already narrow |
35
+| `queryCountJobQueues`, `queryJobQueues` | `QSYS2.JOB_QUEUE_INFO` | View | Per-queue WHERE clause | ⚠️ | Live data only; keep filtered view but shift work to slow producer and optionally enrich with `SYSTOOLS.JOB_QUEUE_ENTRIES` (7.4 TR9+) |
36
+| `queryCountMessageQueues`, `queryMessageQueueAggregates` | `QSYS2.MESSAGE_QUEUE_INFO` | View (detail rows) | No SQL filter | ❗️ | Must switch to table function w/ per-queue sampling |
37
+| `queryCountOutputQueues`, `queryOutputQueueInfo` | `QSYS2.OUTPUT_QUEUE_INFO` | View | No SQL filter | ❗️ | Rewrite like message/job queues |
38
+| `queryNetworkConnections` | `QSYS2.NETSTAT_INFO` | View | Excludes loopback | ⚠️ | Consider host filter support if selectors introduced |
39
+| `queryNetworkInterfaces`, `queryCountNetworkInterfaces` | `QSYS2.NETSTAT_INTERFACE_INFO` | View | `WHERE LINE_DESCRIPTION != '*LOOPBACK'` | ⚠️ | Add optional selector list |
40
+| `queryTempStorageTotal`, `queryTempStorageNamed` | `QSYS2.SYSTMPSTG` | View | Filters global bucket null/not null | ⚠️ | Acceptable cost today; keep under review |
41
+| `querySubsystems`, `queryCountSubsystems` | `QSYS2.SUBSYSTEM_INFO` | View | `WHERE STATUS = 'ACTIVE'` | ✅ | Already filtered |
42
+| `buildActiveJobQuery` | `TABLE(QSYS2.ACTIVE_JOB_INFO(...))` | Table function | Filters by job name/user; per-job query | ⚠️ | Executes once per configured job; still tied to number of tracked jobs but avoids full system scan |
43
+| `queryHTTPServerInfo`, `queryCountHTTPServers` | `QSYS2.HTTP_SERVER_INFO` | View | No filter | ⚠️ | Consider selector per server name |
44
+| `queryIBMiVersion`, `queryIBMiVersionDataArea`, `queryTechnologyRefresh` | Metadata views | — | — | ✅ | One-off |
45
+| `callAnalyzePlanCache`, `queryPlanCacheSummary` | Stored proc + temp table | — | None | ❗️ | Run asynchronously with throttling |
46
+
47
+## 4. Strategy (phased)
48
+
49
+### Phase 1 – Query Rewrite & Filtering
50
+1. **Explicit queue targets only:** retire generic selectors and `max_*` limits for message/job/output queues; accept a list of fully-qualified queue names (library/queue) and run one table-function call per entry. Provide sensible defaults (e.g., `QSYS/QSYSOPR`, `QSYS/QSYSMSG`, `QSYS/QHST`) and keep a simple `collect_*` enable/disable switch for each feature.
51
+2. **Prefer table functions (7.4+):** when available, use IBM’s table functions (`TABLE(QSYS2.MESSAGE_QUEUE_INFO(...))`, `TABLE(QSYS2.ACTIVE_JOB_INFO(...))`, etc.) so the filtering happens via parameters instead of scanning the entire detail view. On older releases fall back to views but construct strict `WHERE library/name IN (...)` clauses from the explicit lists.
52
+3. **Helper for list expansion:** implement shared helpers that turn config lists into SQL fragments and per-collector worklists. If glob support is introduced, expand to explicit names on a periodic cadence and reuse the list so the optimized SQL always receives concrete values (never `LIKE`).
53
+4. **Lightweight counts:** reuse the explicit target list for cardinality checks (length of the list) instead of running `COUNT(*)` scans where possible; otherwise cache the last-known count and refresh less frequently.
54
+5. **Output queues:** ✅ implemented via `OUTPUT_QUEUE_ENTRIES` with view fallback.
55
+
56
+### Phase 2 – Background Producers & Staging
57
+1. Introduce a dedicated slow-path worker (“producer”) that runs on aligned beats (default 30 s) to execute heavy collectors (`message_queue_*`, `job_queue_*`, `output_queue_*`, `count_subsystems`/`collectSubsystems`, plan cache). Producer keeps its own `context.Context` so it stops when the job stops or configuration reloads.
58
+2. Expose new configuration knobs:
59
+ - `slow_path` (bool, default `true`) to enable/disable the background worker.
60
+ - `slow_path_update_every` (duration, default `30s`, must be ≥ main `update_every`) to control beat cadence.
61
+ - `slow_path_max_connections` (int, default `1`) to set `db.SetMaxOpenConns` for the slow worker so queries can run concurrently when increased.
62
+3. Convert the existing sequential collectors into “consumers” for the heavy datasets: on each fast loop they read from the slow-path cache (values + timestamps + error state) and export data only when fresh. Fast loop stays synchronous for lightweight collectors (system status, memory pools, disks, temp storage, job info, active jobs, network stats, HTTP servers, etc.).
63
+4. Implement the cache layer with per-feature entries (latest value, timestamp, last error) and proper locking. Bring query-latency logging inside the slow producer so timing reflects the asynchronous work.
64
+5. Ensure lifecycle management: producer goroutines respect `slow_path` changes and job shutdown (cancel + wait); structure the code so we can register additional slow tracks in the future without refactoring.
65
+
66
+### Phase 3 – Instrumentation & Validation
67
+1. Extend query latency tracking to name every query (remove “other” bucket).
68
+2. Add debug telemetry to confirm selector expansion and SQL rewriting.
69
+3. Establish regression test fixtures (mock DB responses) for selector translation and per-queue sampling.
70
+
71
+## 5. Selector Handling & Glob Patterns
72
+- Current usage does not rely on glob patterns, but long-term usability may. Proposed approach:
73
+ 1. Accept plain lists (preferred). For globs, expand to explicit queue/library names on a cadence (e.g., refresh every minute using targeted list queries).
74
+ 2. Build per-feature caches that hold resolved names and feed them into the optimized SQL (`WHERE library/name IN (...)`) or per-queue execution list.
75
+ 3. When expansion yields no matches, skip the heavy query entirely to avoid wasted scans.
76
+
77
+## 6. Action Plan
78
+
79
+1. **Message queues:** (✅ done) explicit list + table function (7.4+) / documented behaviour on older releases.
80
+2. **Job queues:** keep the `QSYS2.JOB_QUEUE_INFO` per-queue WHERE filtering, but execute it from the slow producer cache and, when present, enrich results with `SYSTOOLS.JOB_QUEUE_ENTRIES` (7.4 TR9+/7.5 TR3+) without increasing call volume.
81
+3. **Output queues:** mirror job queue treatment, including spool file considerations. (Research complete: use `QSYS2.OUTPUT_QUEUE_ENTRIES(library, queue, detail)` table function on 7.2+; fall back to view if unavailable.)
82
+4. **Active jobs:** ✅ complete — configuration now requires explicit `active_jobs` list; per-job table-function queries replace global scan.
83
+5. **Subsystem inventory:** migrate to slow producer/cache, publishing active subsystem metrics only when refreshed.
84
+6. **Plan cache:** migrate to the slow producer (dedicated cadence & timeout); only publish when fresh data arrives.
85
+7. **Producer/consumer cache:** implement the background fetchers, shared caches (values + timestamps + errors), and consumer-side merge logic so fast loops simply read whichever data is ready. Slow worker should support concurrent query fan-out per `slow_path_max_connections`.
86
+8. **SetUpdateEvery plumbing:** after the cache layer exists, wire per-context update intervals so slow charts report their natural cadence.
87
+9. **Selector infrastructure:** build reusable helper that expands globs (if any) into explicit lists and share it across queues/disks/interfaces.
88
+10. **Instrumentation:** name all queries in latency map, add logging for large result sets, and monitor improvements via `netdata.plugin_ibm.as400_query_latency`, splitting fast/slow instances so both timings are visible.
89
+
90
+## 7. Outstanding Questions
91
+- Confirm IBM i versions we must support (table functions require 7.4+). Decide on fallback mechanism for older releases.
92
+- Determine acceptable concurrency level per partition (number of simultaneous ODBC connections).
93
+- Validate whether additional IBM services (e.g., HTTP server info) provide table functions or existing views can be filtered by name list.
94
+- Decide cadence for selector refresh (e.g., once per minute vs. per iteration) balancing staleness vs. overhead.
95
+- Document how to reintroduce queue-total counts if a customer insists.
96
+
97
+## 8. Customer Request: Optional Queue Totals
98
+- Customer asked to restore `count_message_queues`, `count_job_queues`, `count_output_queues`, acknowledging high cost but wants opt-in capability.
99
+- Plan of record (pending research validation):
100
+ - **Configuration flags:** per-query booleans – `collect_message_queue_totals`, `collect_job_queue_totals`, `collect_output_queue_totals` – default `false` with explicit warnings about full scans.
101
+ - **Batch path:** mirror the slow-path configuration style (`batch_path_enabled`, `batch_path_update_every`, `batch_path_max_connections`). Default cadence 60 s for testing (enforce ≥60 s at load time, document ≥600 s for production). The worker starts only if at least one total flag is enabled.
102
+ - **Metrics/contexts:** emit two contexts when totals are enabled:
103
+ * `as400.queues_count` with a single `queues` dimension and instances labeled by `queue_type` (`message_queue`, `job_queue`, `output_queue`) and `item_type` (matching `message`, `job`, `spooled_file` for consistency).
104
+ * `as400.queued_items` with a single `items` dimension using the same label scheme.
105
+ Only instances corresponding to enabled flags are published.
106
+ - **Families:** reorganize queue-related families under a unified `queues/` namespace before release:
107
+ * rename existing `messaging/message_queues` → `queues/message`, `messaging/output_queues` → `queues/output`.
108
+ * move per-job-queue charts from `workloads/job_queues` into `queues/job`.
109
+ * add the new aggregate charts under `queues/overview`.
110
+ - **Error handling:** reuse existing slow-path behaviour—skip iteration on failure, log once per key, clear after a success.
111
+ - **Testing:** verify against pub400.com with flags on/off, measure latency impact, confirm throttled logging, and adjust the batch interval before handing off to production.
112
+- Deep research in progress to see if IBM i offers lighter-weight alternatives (aggregated services, APIs, event feeds). We'll adjust implementation if better options surface.
TODO-WEBSPHERE.md
deleted
-124
@@ -1,124 +0,0 @@
1
-# TODO – WebSphere Collectors Framework Migration
2
-
3
-## Goals
4
-- Port the three legacy WebSphere collectors (`websphere_pmi`, `websphere_mp`, `websphere_jmx`) onto the ibm.d framework so they follow the same structure as the new AS400/DB2 modules.
5
-- Split low-level data acquisition into reusable protocols (PMI/XML, MicroProfile/OpenMetrics, and JMX helper) so collectors only orchestrate business logic and metric shaping.
6
-- Preserve the current breadth of metrics, filtering, and cardinality controls while improving maintainability, chart lifecycle handling, and type safety.
7
-- Lay groundwork for reusing the new protocols in future IBM or Java-centric collectors (e.g. other OpenMetrics endpoints, generic JMX targets).
8
-
9
-## Current State Snapshot
10
-- **PMI collector**: migrated to `modules/websphere/pmi` with framework contexts; legacy `collector/websphere_pmi` code has been removed.
11
-- **MicroProfile collector**: migrated to `modules/websphere/mp` on top of the reusable OpenMetrics protocol; the go.d implementation has been retired.
12
-- **JMX collector**: migrated to `modules/websphere/jmx` using the new JMX bridge; JVM, thread pool, JDBC/JCA, JMS, and application domains are exported through framework contexts while advanced domains (clusters, servlets, EJB) remain on the roadmap.
13
-- The legacy WebSphere collectors (`collector/websphere_*`) have all been removed in favour of the framework modules.
14
-- Tests now cover the PMI, MP, and JMX protocol adapters plus the new JMX collector flows.
15
-
16
-## Target Architecture
17
-```
18
-modules/
19
- websphere/
20
- common/ # shared config structs, label builders, helpers
21
- pmi/
22
- collector.go # framework Collector implementation
23
- collect_*.go
24
- config.go
25
- contexts/
26
- module.go / module_stub.go
27
- mp/
28
- ... (same pattern)
29
- jmx/
30
- ... (same pattern)
31
-protocols/
32
- websphere/
33
- pmi/
34
- client.go # HTTP + XML streaming + caching helpers
35
- parser.go # typed table-like access into PMI XML
36
- jmxbridge/
37
- client.go # generic helper process manager + command plumbing
38
- helper_stub.go # !cgo stub emitting explanatory error
39
- websphere/
40
- jmx/
41
- adapter.go # maps WebSphere domains onto the generic bridge
42
- types.go
43
- openmetrics/
44
- client.go # generic OpenMetrics/Prometheus fetch + parse
45
- parser.go
46
-```
47
-- Collectors depend on their protocol client and the generated `contexts` package; they expose only orchestration/domain logic.
48
-- Protocol packages expose typed data structures (e.g. PMI nodes/servers/stats, OpenMetrics samples grouped by scope, JMX responses by domain) and utility iterators so collectors can map them to contexts deterministically.
49
-
50
-## Protocol Design Notes
51
-### PMI (PerfServlet XML)
52
-- Provide an API that abstracts the XML traversal into table-like collections: e.g. `Fetch(ctx)` returning a snapshot containing hierarchical lookups (`Servers`, `ThreadPools`, etc.) plus helpers to enumerate stats.
53
-- Handle PMI refresh windows (`pmi_refresh_rate`) and caching inside the protocol so collectors simply request logical datasets and leave delta/integral calculations to reusable helpers.
54
-- Reuse/extend existing delta caches (time stat, average stat, integral) but move them into protocol-level utility structs so multiple collectors (future) can leverage them.
55
-- Consider streaming parsing with `encoding/xml` decoder to avoid holding the whole document when not needed, while still producing predictable structures.
56
-
57
-### MicroProfile / OpenMetrics
58
-- Build a generic `protocols/openmetrics` client that can:
59
- - Fetch metrics text over HTTP (respecting auth/TLS config) with context cancellation.
60
- - Parse Prometheus/OpenMetrics into a typed representation (families, samples, labels) using existing `go.d/pkg/prometheus` parsers under the hood.
61
- - Provide convenience filters (by scope: base, vendor, application) and helpers to coerce numeric values and units.
62
-- The WebSphere MP collector consumes this protocol to classify metrics into framework contexts (JVM, thread pools, REST endpoints, generic “other”).
63
-- Ensure the protocol is reusable for other collectors that need OpenMetrics ingestion.
64
-
65
-### JMX Helper / Java Bridge
66
-- Encapsulate the helper process lifecycle in a generic `protocols/jmxbridge` package:
67
- - Manage helper jar extraction, process start/stop, command marshaling, and response decoding.
68
- - Surface a domain-agnostic command API (INIT/SCRAPE/etc.) so adapters can plug in command builders and response decoders.
69
- - Implement resilience logic (restarts, circuit breaker state) within the bridge so collectors just surface health metrics and orchestrate data mapping.
70
- - Provide a `!cgo` stub module that registers an informative error when CGO is disabled, matching the framework contract.
71
-- Layer a WebSphere-specific adapter on top of the bridge that translates between high-level fetch routines (`FetchJVM`, `FetchThreadPools`, `FetchJDBC`, …) and the underlying helper commands, returning strongly typed structs instead of `map[string]interface{}`.
72
-
73
-## Migration Phases & Tasks
74
-### Phase 1 – Protocol Foundations & Shared Utilities
75
-- [x] Stand up `protocols/websphere/pmi` with HTTP client setup, PMI XML parsing, caching primitives, and unit tests using existing sample payloads from `collect_test.go`.
76
-- [x] Extract a reusable OpenMetrics client (`protocols/openmetrics`) leveraging the existing Prometheus parser; cover fetch + parse with fixtures from current MP tests.
77
-- [x] Stand up `protocols/jmxbridge` to host the helper lifecycle + command plumbing, then add a WebSphere adapter that ports logic from `jmx.go`/`resilience.go`; include lifecycle tests using stub responses.
78
-- [x] Create `modules/websphere/common` with shared config structs (cluster labels, cardinality defaults), label builders, and helpers for consistent context labeling across the three modules.
79
-- [x] Provide CGO stubs and ensure go vet/build succeed when CGO is disabled.
80
-
81
-### Phase 2 – WebSphere PMI Module Migration
82
-- [x] Define `modules/websphere/pmi/contexts/contexts.yaml` covering existing metric families (JVM, thread pools, JDBC/JCA, JMS, web apps, APM, cluster, etc.) and regenerate code via `go generate`.
83
-- [x] Port configuration schema/validation into `config.go` and `module.yaml`, preserving feature flags, selectors, and cardinality controls from the legacy collector.
84
-- [x] Re-implement `CollectOnce` using the new PMI protocol: iterate typed datasets, apply selector filters, and populate contexts with type-safe setters.
85
-- [x] Move delta/time-average logic onto protocol utilities or framework state helpers to keep collector loops minimal.
86
-- [x] Recreate tests (unit + integration) using the new module structure; adapt existing fixtures to assert context output rather than raw chart IDs.
87
-- [x] Update documentation (`README.md`, stock config) to reference the new module path.
88
-- [x] Delete the legacy `collector/websphere_pmi` package once parity tests pass.
89
-
90
-### Phase 3 – WebSphere MicroProfile Module Migration
91
-- [x] Author contexts for JVM, vendor/thread-pool, REST endpoint metrics, and fallback “other” metrics with sensible families/priorities mirroring current dashboards.
92
-- [x] Implement the framework collector (`modules/websphere/mp`) that:
93
- * Uses the OpenMetrics protocol to fetch samples.
94
- * Classifies metrics via regex/prefix helpers (moved from legacy code) housed in `modules/websphere/common`.
95
- * Applies REST endpoint filtering/cardinality limits before exporting contexts.
96
-- [x] Port configuration (URL rules, TLS options, selectors) and regenerate schema.
97
-- [x] Add tests verifying metrics classification and context emission using sample metric payloads.
98
-- [x] Remove legacy `collector/websphere_mp` after verification.
99
-
100
-### Phase 4 – WebSphere JMX Module Migration
101
-- [x] Build framework contexts for JVM, thread pools, JDBC/JCA pools, JMS destinations, and web applications in `modules/websphere/jmx`.
102
-- [ ] Add contexts for advanced domains (clusters, servlets, EJBs, JDBC advanced stats) once representative data sets are available.
103
-- [x] Implement the framework collector using the new bridge + adapter for the covered domains.
104
- * Manage helper health metrics via protocol signals (connection state, circuit breaker).
105
- * Keep cardinality management and selectors (applications, pools, servlets, ejbs) but leverage framework state for instance lifecycle.
106
- * Translate protocol structs into context setters, handling precision scaling consistently.
107
-- [x] Port configuration handling (JMX URL, auth, classpath, feature toggles) and regenerate schema/stock config.
108
-- [x] Ensure helper jar embedding + extraction still works under the new layout (update build scripts if necessary).
109
-- [x] Rework unit/integration tests to exercise protocol mocks and verify context output; include helper process restart scenarios.
110
-- [x] Drop `collector/websphere_jmx` once migration is validated.
111
-
112
-### Phase 5 – Integration, QA, and Cleanup
113
-- [ ] Update CI workflows (fmt/vet/build/test) if new packages or go:generate steps require adjustments.
114
-- [ ] Run `./build-ibm.sh` and smoke-test all three modules with existing configs (`/etc/netdata/ibm.d/websphere_*.conf`) to verify runtime behavior and dashboards.
115
-- [ ] Refresh documentation index (e.g., `WEBSPHERE-MONITORING.md`) to point to new modules and protocols.
116
-- [ ] Confirm framework dashboards render with the reorganized families and priorities; align with Netdata Cloud expectations.
117
-- [ ] Archive/remove leftover fixtures or helper scripts from legacy collectors.
118
-
119
-## Open Questions / Research Items
120
-- Should PMI parsing expose a generic “table” abstraction reusable by future XML-based collectors? Investigate whether building a lightweight typed traversal helper is worth the effort vs. bespoke structs.
121
-- For OpenMetrics, can we upstream the protocol into a shared location (`shared/prom`) for reuse beyond WebSphere? Evaluate scope before locking in API.
122
-- The JMX helper currently embeds a WebSphere-focused JAR; consider generalizing it (or supporting multiple helper jars) to serve other Java platforms once the bridge is in place, and track helper versions during build.
123
-- Determine how much of the existing resilience logic (circuit breaker, cached metrics) belongs in the protocol vs. collector (especially for surfacing health charts).
124
-- Verify whether PMI and JMX modules can share cluster labeling/state helpers to avoid divergence in future features.
modified-files
packaging/tools/agent-events/server.go
+109
-60
@@ -68,8 +68,8 @@ var (
68
meter metric.Meter
69
70
// Counters
71
- requestsCounter metric.Int64Counter // Unified counter with status label
72
- bytesReceived metric.Int64Counter
71
+ requestsCounter metric.Int64Counter // Unified counter with status label
72
+ bytesReceived metric.Int64Counter
73
74
// Gauges
75
dedupCacheSize metric.Int64ObservableGauge
@@ -112,7 +112,7 @@ func initMetrics() (*prometheus.Exporter, error) {
112
// Helper to create counters with status labels for consolidated metrics
113
createLabeledCounter := func(name, desc string) (metric.Int64Counter, error) {
114
counter, err := meter.Int64Counter(
115
- name,
115
+ name,
116
metric.WithDescription(desc),
117
metric.WithUnit("1"),
118
)
@@ -142,7 +142,7 @@ func initMetrics() (*prometheus.Exporter, error) {
142
// Create unified counters with status label
143
requestsCounter, _ = createLabeledCounter("agent_events_requests", "Number of requests by status")
144
bytesReceived, _ = createLabeledCounter("agent_events_received_bytes", "Number of bytes received in request bodies by status")
145
-
145
+
146
// Pre-initialize counters with all status labels set to zero
147
ctx := context.Background()
148
statusLabels := []string{"success", "duplicate", "invalid_json", "method_not_allowed", "body_too_large", "failed_to_read", "cant_marshal_output"}
@@ -150,18 +150,17 @@ func initMetrics() (*prometheus.Exporter, error) {
150
requestsCounter.Add(ctx, 0, metric.WithAttributes(attribute.String("status", status)))
151
bytesReceived.Add(ctx, 0, metric.WithAttributes(attribute.String("status", status)))
152
}
153
-
153
+
154
dedupCacheSize, _ = createGauge("agent_events_dedup_cache_entries", "Current number of entries in the deduplication cache")
155
activeConnectionsGauge, _ = createGauge("agent_events_active_connections", "Number of currently active connections")
156
uptimeGauge, _ = createGauge("agent_events_uptime_seconds", "How long the server has been running in seconds")
157
requestDuration, _ = createHistogram("agent_events_request_duration_seconds",
158
"Histogram of request processing times in seconds",
159
- []float64{0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1 },
159
+ []float64{0.00001, 0.000025, 0.00005, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1},
160
)
161
// Basic check if any metric failed (optional, depends on how critical individual metrics are)
162
// if requestsTotal == nil || ... { return exporter, fmt.Errorf("one or more metrics failed to initialize") }
163
164
-
164
// Register callbacks for observable metrics
165
_, err = meter.RegisterCallback(
166
func(_ context.Context, observer metric.Observer) error {
@@ -233,7 +232,7 @@ func cleanupExpiredEntries(interval time.Duration) {
232
}
233
mapMutex.Unlock()
234
236
- // Log hourly summary if any were cleaned in the last hour
235
+ // Log hourly summary if any were cleaned in the last hour
236
if cleanedCount > 0 && time.Since(lastCleanupLogTime) >= time.Hour {
237
slog.Debug("cleaned up expired entries",
238
"count_past_hour", cleanedCount,
@@ -241,9 +240,9 @@ func cleanupExpiredEntries(interval time.Duration) {
240
cleanedCount = 0 // Reset hourly count
241
lastCleanupLogTime = time.Now()
242
} else if deletedInCycle > 0 {
244
- // Optional: Log every cycle if debugging cleanup
245
- // slog.Debug("cleanup cycle completed", "deleted", deletedInCycle, "remaining", currentMapSize-deletedInCycle)
246
- }
243
+ // Optional: Log every cycle if debugging cleanup
244
+ // slog.Debug("cleanup cycle completed", "deleted", deletedInCycle, "remaining", currentMapSize-deletedInCycle)
245
+ }
246
}
247
}
248
@@ -294,34 +293,47 @@ func handler(w http.ResponseWriter, r *http.Request) {
293
var fullData map[string]interface{}
294
if err := json.Unmarshal(body, &fullData); err != nil {
295
http.Error(w, "Invalid JSON", http.StatusBadRequest)
297
- bodyDetail := ""; if slog.Default().Enabled(context.Background(), slog.LevelDebug) { bodyDetail = fmt.Sprintf(", Body: %s", string(body)) } else { bodyDetail = fmt.Sprintf(", Body snippet: %s", limitString(string(body), 100)) }
296
+ bodyDetail := ""
297
+ if slog.Default().Enabled(context.Background(), slog.LevelDebug) {
298
+ bodyDetail = fmt.Sprintf(", Body: %s", string(body))
299
+ } else {
300
+ bodyDetail = fmt.Sprintf(", Body snippet: %s", limitString(string(body), 100))
301
+ }
302
slog.Warn("request discarded", "reason", "invalid_json", "error", err.Error(), "body_detail", bodyDetail)
303
bytesReceived.Add(ctx, int64(len(body)), metric.WithAttributes(attribute.String("status", "invalid_json")))
304
requestsCounter.Add(ctx, 1, metric.WithAttributes(attribute.String("status", "invalid_json")))
305
return
306
}
303
-
307
+
308
bytesReceived.Add(ctx, int64(len(body)), metric.WithAttributes(attribute.String("status", "success")))
309
310
// Add Cloudflare Headers to all requests, excluding IP addresses for GDPR compliance
311
cfHeaders := make(map[string]string)
312
cfHeaderPrefixes := []string{"CF-IPCountry", "CF-Ray", "CF-IPCity", "CF-IPContinent", "CF-IPRegion", "CF-IPTimeZone", "CF-IPCOLO"}
313
// Explicitly excluding IP-related headers: CF-Connecting-IP, CF-IPLatitude, CF-IPLongitude, CF-Visitor
310
- for _, name := range cfHeaderPrefixes { if value := r.Header.Get(name); value != "" { key := strings.TrimPrefix(name, "CF-"); cfHeaders[key] = value } }
311
- for name, values := range r.Header {
312
- if strings.HasPrefix(name, "CF-") && len(values) > 0 {
314
+ for _, name := range cfHeaderPrefixes {
315
+ if value := r.Header.Get(name); value != "" {
316
+ key := strings.TrimPrefix(name, "CF-")
317
+ cfHeaders[key] = value
318
+ }
319
+ }
320
+ for name, values := range r.Header {
321
+ if strings.HasPrefix(name, "CF-") && len(values) > 0 {
322
// Skip IP-related headers for GDPR compliance
323
if name == "CF-Connecting-IP" || name == "CF-IPLatitude" || name == "CF-IPLongitude" || name == "CF-Visitor" {
324
continue
325
}
317
- key := strings.TrimPrefix(name, "CF-");
318
- if _, exists := cfHeaders[key]; !exists {
319
- cfHeaders[key] = values[0]
320
- }
321
- }
322
- }
323
- if len(cfHeaders) > 0 { fullData["cf"] = cfHeaders; slog.Debug("added cloudflare headers", "count", len(cfHeaders)) }
324
-
326
+ key := strings.TrimPrefix(name, "CF-")
327
+ if _, exists := cfHeaders[key]; !exists {
328
+ cfHeaders[key] = values[0]
329
+ }
330
+ }
331
+ }
332
+ if len(cfHeaders) > 0 {
333
+ fullData["cf"] = cfHeaders
334
+ slog.Debug("added cloudflare headers", "count", len(cfHeaders))
335
+ }
336
+
337
// Deduplication Logic
338
shouldProcess := true
339
var finalKeyString string
@@ -331,15 +343,17 @@ func handler(w http.ResponseWriter, r *http.Request) {
343
for i, path := range keyPaths {
344
result := gjson.GetBytes(body, path)
345
keyBuilder.WriteString(result.String()) // gjson returns "" for non-existent paths
334
- if i < len(keyPaths)-1 { keyBuilder.WriteString(dedupSeparator) }
346
+ if i < len(keyPaths)-1 {
347
+ keyBuilder.WriteString(dedupSeparator)
348
+ }
349
}
350
finalKeyString = keyBuilder.String()
351
dedupHash = sha256.Sum256([]byte(finalKeyString))
352
slog.Debug("generated dedup key", "key_string", finalKeyString, "hash", fmt.Sprintf("%x", dedupHash))
339
-
353
+
354
// Add _dedup key to all requests (regardless of duplicate status)
355
fullData["_dedup"] = map[string]interface{}{
342
- "key": finalKeyString,
356
+ "key": finalKeyString,
357
"hash": fmt.Sprintf("%x", dedupHash),
358
}
359
@@ -383,7 +397,7 @@ func handler(w http.ResponseWriter, r *http.Request) {
397
slog.Error("error writing response", "context", "after_duplicate_discard", "error", err)
398
}
399
}
386
-
400
+
401
// For duplicates, return early
402
if !shouldProcess {
403
return
@@ -400,30 +414,37 @@ func healthHandler(w http.ResponseWriter, r *http.Request) {
414
"connections": atomic.LoadInt32(&activeConnections),
415
"version": "agent-events v1.0",
416
}
403
- memStats := &runtime.MemStats{}; runtime.ReadMemStats(memStats); status["memory"] = map[string]interface{}{"alloc": memStats.Alloc, "total_alloc": memStats.TotalAlloc, "sys": memStats.Sys, "heap_alloc": memStats.HeapAlloc, "gc_cycles": memStats.NumGC}
404
- mapMutex.Lock(); mapSize := len(seenIDs); mapMutex.Unlock();
417
+ memStats := &runtime.MemStats{}
418
+ runtime.ReadMemStats(memStats)
419
+ status["memory"] = map[string]interface{}{"alloc": memStats.Alloc, "total_alloc": memStats.TotalAlloc, "sys": memStats.Sys, "heap_alloc": memStats.HeapAlloc, "gc_cycles": memStats.NumGC}
420
+ mapMutex.Lock()
421
+ mapSize := len(seenIDs)
422
+ mapMutex.Unlock()
423
dedupLogEnabled := dedupLogger != nil
424
dedupLogPath := ""
425
if dedupLogEnabled {
426
dedupLogPath = dedupLogFile
427
}
428
status["deduplication"] = map[string]interface{}{
411
- "enabled": len(keyPaths) > 0,
412
- "keys": keyPaths,
413
- "window": dedupWindow.String(),
414
- "map_size": mapSize,
429
+ "enabled": len(keyPaths) > 0,
430
+ "keys": keyPaths,
431
+ "window": dedupWindow.String(),
432
+ "map_size": mapSize,
433
"log_enabled": dedupLogEnabled,
416
- "log_file": dedupLogPath,
434
+ "log_file": dedupLogPath,
435
+ }
436
+ w.Header().Set("Content-Type", "application/json")
437
+ w.WriteHeader(http.StatusOK)
438
+ if err := json.NewEncoder(w).Encode(status); err != nil {
439
+ slog.Error("error encoding health check response", "error", err)
440
}
418
- w.Header().Set("Content-Type", "application/json"); w.WriteHeader(http.StatusOK);
419
- if err := json.NewEncoder(w).Encode(status); err != nil { slog.Error("error encoding health check response", "error", err) }
441
}
442
443
// main is the entry point of the application
444
func main() {
445
// Setup initial logger before flag parsing
446
initialLogLevel := slog.LevelInfo
426
- initialLogHandler := slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{ Level: initialLogLevel, AddSource: true })
447
+ initialLogHandler := slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{Level: initialLogLevel, AddSource: true})
448
slog.SetDefault(slog.New(initialLogHandler))
449
450
startTime = time.Now()
@@ -445,22 +466,30 @@ func main() {
466
// Configure final logger based on flags
467
var level slog.Level
468
switch strings.ToLower(*logLevelFlag) {
448
- case "debug": level = slog.LevelDebug
449
- case "info": level = slog.LevelInfo
450
- case "warn": level = slog.LevelWarn
451
- case "error": level = slog.LevelError
469
+ case "debug":
470
+ level = slog.LevelDebug
471
+ case "info":
472
+ level = slog.LevelInfo
473
+ case "warn":
474
+ level = slog.LevelWarn
475
+ case "error":
476
+ level = slog.LevelError
477
default:
478
slog.Warn("invalid log level specified, defaulting to info", "value", *logLevelFlag)
479
level = slog.LevelInfo
480
}
481
var logHandler slog.Handler
457
- if *logFormat == "text" { logHandler = slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{ Level: level, AddSource: true }) } else { logHandler = slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{ Level: level, AddSource: true }) }
482
+ if *logFormat == "text" {
483
+ logHandler = slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: level, AddSource: true})
484
+ } else {
485
+ logHandler = slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{Level: level, AddSource: true})
486
+ }
487
slog.SetDefault(slog.New(logHandler))
488
489
// Initialize core components
490
seenIDs = make(map[[32]byte]seenEntry)
491
dedupWindow = time.Duration(*dedupSeconds) * time.Second
463
-
492
+
493
// Open deduplication log file if specified
494
if dedupLogFile != "" {
495
var err error
@@ -471,18 +500,25 @@ func main() {
500
}
501
slog.Info("deduplication log file opened", "path", dedupLogFile)
502
}
474
-
475
- if _, err := initMetrics(); err != nil { // Handle potential error from initMetrics
476
- slog.Error("failed to initialize metrics", "error", err)
477
- os.Exit(1)
478
- }
503
504
+ if _, err := initMetrics(); err != nil { // Handle potential error from initMetrics
505
+ slog.Error("failed to initialize metrics", "error", err)
506
+ os.Exit(1)
507
+ }
508
509
// Start background tasks
510
if dedupWindow > 0 && len(keyPaths) > 0 {
483
- cleanupInterval := dedupWindow / 10; if cleanupInterval < 1*time.Minute { cleanupInterval = 1 * time.Minute } else if cleanupInterval > 15*time.Minute { cleanupInterval = 15 * time.Minute }
484
- slog.Info("cleanup goroutine started", "interval", cleanupInterval); go cleanupExpiredEntries(cleanupInterval)
485
- } else if dedupWindow <= 0 && len(keyPaths) > 0 { slog.Warn("deduplication keys provided, but window is zero or negative", "keys", keyPaths, "window", dedupWindow) }
511
+ cleanupInterval := dedupWindow / 10
512
+ if cleanupInterval < 1*time.Minute {
513
+ cleanupInterval = 1 * time.Minute
514
+ } else if cleanupInterval > 15*time.Minute {
515
+ cleanupInterval = 15 * time.Minute
516
+ }
517
+ slog.Info("cleanup goroutine started", "interval", cleanupInterval)
518
+ go cleanupExpiredEntries(cleanupInterval)
519
+ } else if dedupWindow <= 0 && len(keyPaths) > 0 {
520
+ slog.Warn("deduplication keys provided, but window is zero or negative", "keys", keyPaths, "window", dedupWindow)
521
+ }
522
523
// Configure HTTP server
524
server := &http.Server{
@@ -494,7 +530,9 @@ func main() {
530
mux := http.NewServeMux()
531
mux.HandleFunc("/", handler)
532
mux.HandleFunc(*healthPath, healthHandler)
497
- if *expvarPath != "" { mux.Handle(*expvarPath, expvar.Handler()) } // Register expvar if path not empty
533
+ if *expvarPath != "" {
534
+ mux.Handle(*expvarPath, expvar.Handler())
535
+ } // Register expvar if path not empty
536
mux.Handle(*metricsPath, promhttp.Handler()) // Use promhttp handler for OTEL metrics
537
// Add pprof handlers to custom mux
538
mux.HandleFunc("/debug/pprof/", http.DefaultServeMux.ServeHTTP)
@@ -518,21 +556,30 @@ func main() {
556
557
select {
558
case err := <-serverErrors:
521
- if err != nil && !errors.Is(err, http.ErrServerClosed) { slog.Error("server error", "error", err) }
559
+ if err != nil && !errors.Is(err, http.ErrServerClosed) {
560
+ slog.Error("server error", "error", err)
561
+ }
562
case sig := <-stop:
563
slog.Info("shutdown initiated", "signal", sig.String())
524
- shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second); defer cancel()
564
+ shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
565
+ defer cancel()
566
567
// Shutdown meter provider first
568
if meterProvider != nil {
569
slog.Info("shutting down OpenTelemetry meter provider")
529
- if err := meterProvider.Shutdown(shutdownCtx); err != nil { slog.Error("meter provider shutdown failed", "error", err) }
570
+ if err := meterProvider.Shutdown(shutdownCtx); err != nil {
571
+ slog.Error("meter provider shutdown failed", "error", err)
572
+ }
573
}
574
575
// Shutdown HTTP server
576
slog.Info("shutting down HTTP server")
534
- if err := server.Shutdown(shutdownCtx); err != nil { slog.Error("server shutdown failed", "error", err) } else { slog.Info("server shutdown completed gracefully") }
535
-
577
+ if err := server.Shutdown(shutdownCtx); err != nil {
578
+ slog.Error("server shutdown failed", "error", err)
579
+ } else {
580
+ slog.Info("server shutdown completed gracefully")
581
+ }
582
+
583
// Close deduplication log file if open
584
if dedupLogger != nil {
585
slog.Info("closing deduplication log file")
@@ -547,6 +594,8 @@ func main() {
594
595
// --- Helper Functions ---
596
func limitString(s string, maxLen int) string {
550
- if len(s) <= maxLen { return s }
597
+ if len(s) <= maxLen {
598
+ return s
599
+ }
600
return s[:maxLen] + "..."
552
-}
\ No newline at end of file
601
+}
packaging/tools/agent-events/server_test.go
+336
-112
@@ -109,26 +109,50 @@ func TestHandler(t *testing.T) {
109
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
110
rr := httptest.NewRecorder()
111
stdout, _ := captureOutput(t, func() { handler(rr, req) })
112
- if status := rr.Code; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
113
- if rr.Body.String() != "OK" { t.Errorf("body: got %v want %v", rr.Body.String(), "OK") }
114
- if !strings.Contains(stdout, `"id":"uuid-1"`) { t.Errorf("stdout missing id: %q", stdout) }
115
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock()
116
- if mapLen != 1 { t.Errorf("map size: got %d want 1", mapLen) }
112
+ if status := rr.Code; status != http.StatusOK {
113
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
114
+ }
115
+ if rr.Body.String() != "OK" {
116
+ t.Errorf("body: got %v want %v", rr.Body.String(), "OK")
117
+ }
118
+ if !strings.Contains(stdout, `"id":"uuid-1"`) {
119
+ t.Errorf("stdout missing id: %q", stdout)
120
+ }
121
+ mapMutex.Lock()
122
+ mapLen := len(seenIDs)
123
+ mapMutex.Unlock()
124
+ if mapLen != 1 {
125
+ t.Errorf("map size: got %d want 1", mapLen)
126
+ }
127
})
128
129
t.Run("DuplicateRequestWithinWindow", func(t *testing.T) {
130
t.Cleanup(resetDedupState)
131
firstJsonBody := `{"id": "uuid-2", "data": "value2"}`
132
firstReq := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(firstJsonBody))
123
- firstRr := httptest.NewRecorder(); captureOutput(t, func() { handler(firstRr, firstReq) })
124
- if firstRr.Code != http.StatusOK { t.Fatalf("Setup failed") }
133
+ firstRr := httptest.NewRecorder()
134
+ captureOutput(t, func() { handler(firstRr, firstReq) })
135
+ if firstRr.Code != http.StatusOK {
136
+ t.Fatalf("Setup failed")
137
+ }
138
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(firstJsonBody))
126
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
127
- if status := rr.Code; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
128
- if rr.Body.String() != "OK" { t.Errorf("body: got %v want %v", rr.Body.String(), "OK") }
129
- if stdout != "" { t.Errorf("stdout not empty: %q", stdout) }
130
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock()
131
- if mapLen != 1 { t.Errorf("map size: got %d want 1", mapLen) }
139
+ rr := httptest.NewRecorder()
140
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
141
+ if status := rr.Code; status != http.StatusOK {
142
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
143
+ }
144
+ if rr.Body.String() != "OK" {
145
+ t.Errorf("body: got %v want %v", rr.Body.String(), "OK")
146
+ }
147
+ if stdout != "" {
148
+ t.Errorf("stdout not empty: %q", stdout)
149
+ }
150
+ mapMutex.Lock()
151
+ mapLen := len(seenIDs)
152
+ mapMutex.Unlock()
153
+ if mapLen != 1 {
154
+ t.Errorf("map size: got %d want 1", mapLen)
155
+ }
156
})
157
158
t.Run("InvalidJSON", func(t *testing.T) {
@@ -137,8 +161,12 @@ func TestHandler(t *testing.T) {
161
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
162
rr := httptest.NewRecorder()
163
stdout, _ := captureOutput(t, func() { handler(rr, req) })
140
- if status := rr.Code; status != http.StatusBadRequest { t.Errorf("status: got %v want %v", status, http.StatusBadRequest) }
141
- if stdout != "" { t.Errorf("stdout not empty: %q", stdout) }
164
+ if status := rr.Code; status != http.StatusBadRequest {
165
+ t.Errorf("status: got %v want %v", status, http.StatusBadRequest)
166
+ }
167
+ if stdout != "" {
168
+ t.Errorf("stdout not empty: %q", stdout)
169
+ }
170
})
171
172
t.Run("MissingDedupKey", func(t *testing.T) {
@@ -147,10 +175,18 @@ func TestHandler(t *testing.T) {
175
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
176
rr := httptest.NewRecorder()
177
stdout, _ := captureOutput(t, func() { handler(rr, req) })
150
- if status := rr.Code; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
151
- if !strings.Contains(stdout, `"other_id":"uuid-4"`) { t.Errorf("stdout missing other_id: %q", stdout) }
152
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock()
153
- if mapLen != 1 { t.Errorf("map size: got %d want 1", mapLen) }
178
+ if status := rr.Code; status != http.StatusOK {
179
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
180
+ }
181
+ if !strings.Contains(stdout, `"other_id":"uuid-4"`) {
182
+ t.Errorf("stdout missing other_id: %q", stdout)
183
+ }
184
+ mapMutex.Lock()
185
+ mapLen := len(seenIDs)
186
+ mapMutex.Unlock()
187
+ if mapLen != 1 {
188
+ t.Errorf("map size: got %d want 1", mapLen)
189
+ }
190
})
191
192
t.Run("CloudflareHeaders", func(t *testing.T) {
@@ -158,20 +194,40 @@ func TestHandler(t *testing.T) {
194
jsonBody := `{"id": "uuid-cf", "data": "value-cf"}`
195
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
196
// Set allowed headers (non-IP related)
161
- req.Header.Set("CF-IPCountry", "US"); req.Header.Set("CF-Ray", "123"); req.Header.Set("CF-IPCity", "Testville")
197
+ req.Header.Set("CF-IPCountry", "US")
198
+ req.Header.Set("CF-Ray", "123")
199
+ req.Header.Set("CF-IPCity", "Testville")
200
// Set IP-related headers that should be excluded for GDPR compliance
163
- req.Header.Set("CF-Connecting-IP", "1.2.3.4"); req.Header.Set("CF-IPLatitude", "12.34"); req.Header.Set("CF-IPLongitude", "-56.78"); req.Header.Set("CF-Visitor", "{\"ip\":\"1.2.3.4\"}")
164
-
165
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
166
- if status := rr.Code; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
167
- if !strings.Contains(stdout, `"cf":`) { t.Errorf("stdout missing cf object") }
168
- if !strings.Contains(stdout, `"IPCountry":"US"`) { t.Errorf("stdout missing cf header IPCountry") }
169
-
201
+ req.Header.Set("CF-Connecting-IP", "1.2.3.4")
202
+ req.Header.Set("CF-IPLatitude", "12.34")
203
+ req.Header.Set("CF-IPLongitude", "-56.78")
204
+ req.Header.Set("CF-Visitor", "{\"ip\":\"1.2.3.4\"}")
205
+
206
+ rr := httptest.NewRecorder()
207
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
208
+ if status := rr.Code; status != http.StatusOK {
209
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
210
+ }
211
+ if !strings.Contains(stdout, `"cf":`) {
212
+ t.Errorf("stdout missing cf object")
213
+ }
214
+ if !strings.Contains(stdout, `"IPCountry":"US"`) {
215
+ t.Errorf("stdout missing cf header IPCountry")
216
+ }
217
+
218
// Verify IP-related headers are excluded for GDPR compliance
171
- if strings.Contains(stdout, `"Connecting-IP"`) { t.Errorf("stdout should not contain IP address: Connecting-IP") }
172
- if strings.Contains(stdout, `"IPLatitude"`) { t.Errorf("stdout should not contain IP geolocation: IPLatitude") }
173
- if strings.Contains(stdout, `"IPLongitude"`) { t.Errorf("stdout should not contain IP geolocation: IPLongitude") }
174
- if strings.Contains(stdout, `"Visitor"`) { t.Errorf("stdout should not contain Visitor which includes IP") }
219
+ if strings.Contains(stdout, `"Connecting-IP"`) {
220
+ t.Errorf("stdout should not contain IP address: Connecting-IP")
221
+ }
222
+ if strings.Contains(stdout, `"IPLatitude"`) {
223
+ t.Errorf("stdout should not contain IP geolocation: IPLatitude")
224
+ }
225
+ if strings.Contains(stdout, `"IPLongitude"`) {
226
+ t.Errorf("stdout should not contain IP geolocation: IPLongitude")
227
+ }
228
+ if strings.Contains(stdout, `"Visitor"`) {
229
+ t.Errorf("stdout should not contain Visitor which includes IP")
230
+ }
231
})
232
233
t.Run("MethodNotAllowed", func(t *testing.T) {
@@ -179,8 +235,12 @@ func TestHandler(t *testing.T) {
235
req := httptest.NewRequest(http.MethodGet, "/", nil)
236
rr := httptest.NewRecorder()
237
stdout, _ := captureOutput(t, func() { handler(rr, req) })
182
- if status := rr.Code; status != http.StatusMethodNotAllowed { t.Errorf("status: got %v want %v", status, http.StatusMethodNotAllowed) }
183
- if stdout != "" { t.Errorf("stdout not empty: %q", stdout) }
238
+ if status := rr.Code; status != http.StatusMethodNotAllowed {
239
+ t.Errorf("status: got %v want %v", status, http.StatusMethodNotAllowed)
240
+ }
241
+ if stdout != "" {
242
+ t.Errorf("stdout not empty: %q", stdout)
243
+ }
244
})
245
246
t.Run("RequestEntityTooLarge", func(t *testing.T) {
@@ -189,25 +249,45 @@ func TestHandler(t *testing.T) {
249
req := httptest.NewRequest(http.MethodPost, "/", bytes.NewReader(largeBody))
250
rr := httptest.NewRecorder()
251
stdout, _ := captureOutput(t, func() { handler(rr, req) })
192
- if status := rr.Code; status != http.StatusRequestEntityTooLarge { t.Errorf("status: got %v want %v", status, http.StatusRequestEntityTooLarge) }
193
- if stdout != "" { t.Errorf("stdout not empty: %q", stdout) }
252
+ if status := rr.Code; status != http.StatusRequestEntityTooLarge {
253
+ t.Errorf("status: got %v want %v", status, http.StatusRequestEntityTooLarge)
254
+ }
255
+ if stdout != "" {
256
+ t.Errorf("stdout not empty: %q", stdout)
257
+ }
258
})
259
260
t.Run("MultiKeyDeduplication", func(t *testing.T) {
261
t.Cleanup(resetDedupState)
198
- keyPaths = []string{"id", "source"}; dedupSeparator = "|"
262
+ keyPaths = []string{"id", "source"}
263
+ dedupSeparator = "|"
264
req1 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "1", "source": "A", "data": "v1"}`))
200
- rr1 := httptest.NewRecorder(); stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
201
- if rr1.Code != http.StatusOK || !strings.Contains(stdout1, `"id":"1"`) { t.Errorf("Request 1 failed") }
265
+ rr1 := httptest.NewRecorder()
266
+ stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
267
+ if rr1.Code != http.StatusOK || !strings.Contains(stdout1, `"id":"1"`) {
268
+ t.Errorf("Request 1 failed")
269
+ }
270
req2 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "1", "source": "B", "data": "v2"}`))
203
- rr2 := httptest.NewRecorder(); stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
204
- if rr2.Code != http.StatusOK || !strings.Contains(stdout2, `"source":"B"`) { t.Errorf("Request 2 failed") }
271
+ rr2 := httptest.NewRecorder()
272
+ stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
273
+ if rr2.Code != http.StatusOK || !strings.Contains(stdout2, `"source":"B"`) {
274
+ t.Errorf("Request 2 failed")
275
+ }
276
req3 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "1", "source": "A", "data": "v3"}`))
206
- rr3 := httptest.NewRecorder(); stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
207
- if rr3.Code != http.StatusOK { t.Errorf("Request 3 status wrong") }
208
- if stdout3 != "" { t.Errorf("Request 3 produced output") }
209
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock()
210
- if mapLen != 2 { t.Errorf("map size: got %d want 2", mapLen) }
277
+ rr3 := httptest.NewRecorder()
278
+ stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
279
+ if rr3.Code != http.StatusOK {
280
+ t.Errorf("Request 3 status wrong")
281
+ }
282
+ if stdout3 != "" {
283
+ t.Errorf("Request 3 produced output")
284
+ }
285
+ mapMutex.Lock()
286
+ mapLen := len(seenIDs)
287
+ mapMutex.Unlock()
288
+ if mapLen != 2 {
289
+ t.Errorf("map size: got %d want 2", mapLen)
290
+ }
291
})
292
293
t.Run("MetricsDelta", func(t *testing.T) {
@@ -217,12 +297,27 @@ func TestHandler(t *testing.T) {
297
t.Run("HealthEndpoint", func(t *testing.T) {
298
t.Cleanup(resetDedupState)
299
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
220
- rr := httptest.NewRecorder(); healthHandler(rr, req)
221
- if status := rr.Code; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
222
- if contentType := rr.Header().Get("Content-Type"); contentType != "application/json" { t.Errorf("content type: got %v want %v", contentType, "application/json") }
223
- var result map[string]interface{}; if err := json.Unmarshal(rr.Body.Bytes(), &result); err != nil { t.Fatalf("invalid JSON: %v", err) }
224
- requiredFields := []string{"status", "timestamp", "uptime", "goroutines", "memory", "deduplication"}; for _, field := range requiredFields { if _, ok := result[field]; !ok { t.Errorf("missing field: %s", field) } }
225
- if status, ok := result["status"].(string); !ok || status != "ok" { t.Errorf("status field: got %v want ok", result["status"]) }
300
+ rr := httptest.NewRecorder()
301
+ healthHandler(rr, req)
302
+ if status := rr.Code; status != http.StatusOK {
303
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
304
+ }
305
+ if contentType := rr.Header().Get("Content-Type"); contentType != "application/json" {
306
+ t.Errorf("content type: got %v want %v", contentType, "application/json")
307
+ }
308
+ var result map[string]interface{}
309
+ if err := json.Unmarshal(rr.Body.Bytes(), &result); err != nil {
310
+ t.Fatalf("invalid JSON: %v", err)
311
+ }
312
+ requiredFields := []string{"status", "timestamp", "uptime", "goroutines", "memory", "deduplication"}
313
+ for _, field := range requiredFields {
314
+ if _, ok := result[field]; !ok {
315
+ t.Errorf("missing field: %s", field)
316
+ }
317
+ }
318
+ if status, ok := result["status"].(string); !ok || status != "ok" {
319
+ t.Errorf("status field: got %v want ok", result["status"])
320
+ }
321
})
322
323
t.Run("MetricsEndpoint", func(t *testing.T) {
@@ -246,14 +341,22 @@ func TestHandler(t *testing.T) {
341
342
// Fetch metrics
343
resp, err := http.Get(metricsServer.URL)
249
- if err != nil { t.Fatalf("failed to get metrics: %v", err) }
344
+ if err != nil {
345
+ t.Fatalf("failed to get metrics: %v", err)
346
+ }
347
defer resp.Body.Close()
348
252
- if status := resp.StatusCode; status != http.StatusOK { t.Errorf("status: got %v want %v", status, http.StatusOK) }
253
- if contentType := resp.Header.Get("Content-Type"); !strings.HasPrefix(contentType, "text/plain") { t.Errorf("content type: got %q want prefix text/plain", contentType) }
349
+ if status := resp.StatusCode; status != http.StatusOK {
350
+ t.Errorf("status: got %v want %v", status, http.StatusOK)
351
+ }
352
+ if contentType := resp.Header.Get("Content-Type"); !strings.HasPrefix(contentType, "text/plain") {
353
+ t.Errorf("content type: got %q want prefix text/plain", contentType)
354
+ }
355
356
metricsBodyBytes, err := io.ReadAll(resp.Body)
256
- if err != nil { t.Fatalf("failed to read metrics body: %v", err) }
357
+ if err != nil {
358
+ t.Fatalf("failed to read metrics body: %v", err)
359
+ }
360
metricsContent := string(metricsBodyBytes)
361
t.Logf("Metrics Output for Verification:\n%s", metricsContent) // Log for manual inspection if needed
362
@@ -264,7 +367,7 @@ func TestHandler(t *testing.T) {
367
description string
368
}{
369
{
267
- namePatterns: []string{"agent_events_requests", "agent_events_requests_ratio_total"},
370
+ namePatterns: []string{"agent_events_requests", "agent_events_requests_ratio_total"},
371
description: "Requests counter",
372
},
373
{
@@ -284,7 +387,7 @@ func TestHandler(t *testing.T) {
387
description: "Go runtime metrics",
388
},
389
}
287
-
390
+
391
for _, check := range metricChecks {
392
found := false
393
for _, pattern := range check.namePatterns {
@@ -297,7 +400,7 @@ func TestHandler(t *testing.T) {
400
t.Errorf("metrics response missing expected metric: %s (patterns: %v)", check.description, check.namePatterns)
401
}
402
}
300
-
403
+
404
// OpenTelemetry histogram metrics have this pattern in the output:
405
// agent_events_request_duration_seconds_bucket{...
406
// agent_events_request_duration_seconds_sum{...
@@ -309,41 +412,70 @@ func TestHandler(t *testing.T) {
412
})
413
414
t.Run("DedupWindowExpiration", func(t *testing.T) {
312
- t.Cleanup(resetDedupState); oldWindow := dedupWindow; dedupWindow = 50 * time.Millisecond; t.Cleanup(func() { dedupWindow = oldWindow })
415
+ t.Cleanup(resetDedupState)
416
+ oldWindow := dedupWindow
417
+ dedupWindow = 50 * time.Millisecond
418
+ t.Cleanup(func() { dedupWindow = oldWindow })
419
req1 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "exp1"}`))
314
- rr1 := httptest.NewRecorder(); stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
315
- if rr1.Code != http.StatusOK || !strings.Contains(stdout1, "exp1") { t.Errorf("Req 1 failed") }
420
+ rr1 := httptest.NewRecorder()
421
+ stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
422
+ if rr1.Code != http.StatusOK || !strings.Contains(stdout1, "exp1") {
423
+ t.Errorf("Req 1 failed")
424
+ }
425
req2 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "exp1"}`))
317
- rr2 := httptest.NewRecorder(); stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
318
- if stdout2 != "" { t.Errorf("Immediate duplicate not suppressed") }
426
+ rr2 := httptest.NewRecorder()
427
+ stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
428
+ if stdout2 != "" {
429
+ t.Errorf("Immediate duplicate not suppressed")
430
+ }
431
time.Sleep(100 * time.Millisecond)
432
req3 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "exp1"}`))
321
- rr3 := httptest.NewRecorder(); stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
322
- if rr3.Code != http.StatusOK || !strings.Contains(stdout3, "exp1") { t.Errorf("Req 3 after expiry failed") }
433
+ rr3 := httptest.NewRecorder()
434
+ stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
435
+ if rr3.Code != http.StatusOK || !strings.Contains(stdout3, "exp1") {
436
+ t.Errorf("Req 3 after expiry failed")
437
+ }
438
})
439
440
t.Run("LongKeyValues", func(t *testing.T) {
326
- t.Cleanup(resetDedupState); longId := strings.Repeat("a", 500)
441
+ t.Cleanup(resetDedupState)
442
+ longId := strings.Repeat("a", 500)
443
jsonBody := fmt.Sprintf(`{"id": "%s", "data": "long"}`, longId)
444
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
329
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
330
- if rr.Code != http.StatusOK || !strings.Contains(stdout, "long") { t.Errorf("Long key req failed") }
445
+ rr := httptest.NewRecorder()
446
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
447
+ if rr.Code != http.StatusOK || !strings.Contains(stdout, "long") {
448
+ t.Errorf("Long key req failed")
449
+ }
450
reqDup := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
332
- rrDup := httptest.NewRecorder(); stdoutDup, _ := captureOutput(t, func() { handler(rrDup, reqDup) })
333
- if stdoutDup != "" { t.Errorf("Long key duplicate not suppressed") }
451
+ rrDup := httptest.NewRecorder()
452
+ stdoutDup, _ := captureOutput(t, func() { handler(rrDup, reqDup) })
453
+ if stdoutDup != "" {
454
+ t.Errorf("Long key duplicate not suppressed")
455
+ }
456
})
457
458
t.Run("OptionsMethod", func(t *testing.T) {
459
t.Cleanup(resetDedupState)
460
req := httptest.NewRequest(http.MethodOptions, "/", nil)
339
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
340
- if status := rr.Code; status != http.StatusMethodNotAllowed { t.Errorf("status: got %v want %v", status, http.StatusMethodNotAllowed) }
341
- if stdout != "" { t.Errorf("stdout not empty: %q", stdout) }
461
+ rr := httptest.NewRecorder()
462
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
463
+ if status := rr.Code; status != http.StatusMethodNotAllowed {
464
+ t.Errorf("status: got %v want %v", status, http.StatusMethodNotAllowed)
465
+ }
466
+ if stdout != "" {
467
+ t.Errorf("stdout not empty: %q", stdout)
468
+ }
469
})
470
471
t.Run("VariousJSONFormats", func(t *testing.T) {
472
t.Cleanup(resetDedupState)
346
- testCases := []struct{ name string; body string; expectStatus int; expectOutput bool }{
473
+ testCases := []struct {
474
+ name string
475
+ body string
476
+ expectStatus int
477
+ expectOutput bool
478
+ }{
479
{"EmptyObject", `{}`, http.StatusOK, true}, {"ValidJSON", `{"id": "valid"}`, http.StatusOK, true},
480
{"SingleQuotes", `{'id': 'invalid'}`, http.StatusBadRequest, false}, {"TrailingComma", `{"id": "comma",}`, http.StatusBadRequest, false},
481
{"UnquotedKey", `{id: "unquoted"}`, http.StatusBadRequest, false},
@@ -351,51 +483,122 @@ func TestHandler(t *testing.T) {
483
for _, tc := range testCases {
484
t.Run(tc.name, func(t *testing.T) {
485
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(tc.body))
354
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
355
- if status := rr.Code; status != tc.expectStatus { t.Errorf("status: got %v want %v", status, tc.expectStatus) }
356
- hasOutput := stdout != ""; if hasOutput != tc.expectOutput { t.Errorf("Output mismatch: expected %t, got %t (stdout: %q)", tc.expectOutput, hasOutput, stdout) }
486
+ rr := httptest.NewRecorder()
487
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
488
+ if status := rr.Code; status != tc.expectStatus {
489
+ t.Errorf("status: got %v want %v", status, tc.expectStatus)
490
+ }
491
+ hasOutput := stdout != ""
492
+ if hasOutput != tc.expectOutput {
493
+ t.Errorf("Output mismatch: expected %t, got %t (stdout: %q)", tc.expectOutput, hasOutput, stdout)
494
+ }
495
})
496
}
497
})
498
499
t.Run("ConcurrentRequests", func(t *testing.T) {
362
- t.Cleanup(resetDedupState); numRequests := 50; var wg sync.WaitGroup; wg.Add(numRequests)
500
+ t.Cleanup(resetDedupState)
501
+ numRequests := 50
502
+ var wg sync.WaitGroup
503
+ wg.Add(numRequests)
504
process := func(id int) {
364
- defer wg.Done(); jsonBody := fmt.Sprintf(`{"id": "conc-%d"}`, id); req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
365
- rr := httptest.NewRecorder(); captureOutput(t, func() { handler(rr, req) })
366
- if status := rr.Code; status != http.StatusOK { t.Logf("conc req %d status: got %v want %v", id, status, http.StatusOK); t.Fail() }
505
+ defer wg.Done()
506
+ jsonBody := fmt.Sprintf(`{"id": "conc-%d"}`, id)
507
+ req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
508
+ rr := httptest.NewRecorder()
509
+ captureOutput(t, func() { handler(rr, req) })
510
+ if status := rr.Code; status != http.StatusOK {
511
+ t.Logf("conc req %d status: got %v want %v", id, status, http.StatusOK)
512
+ t.Fail()
513
+ }
514
+ }
515
+ for i := 0; i < numRequests; i++ {
516
+ go process(i)
517
+ }
518
+ wg.Wait()
519
+ mapMutex.Lock()
520
+ mapLen := len(seenIDs)
521
+ mapMutex.Unlock()
522
+ if mapLen != numRequests {
523
+ t.Errorf("map size: got %d want %d", mapLen, numRequests)
524
}
368
- for i := 0; i < numRequests; i++ { go process(i) }; wg.Wait()
369
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock()
370
- if mapLen != numRequests { t.Errorf("map size: got %d want %d", mapLen, numRequests) }
525
})
526
527
t.Run("CleanupExpiredEntries", func(t *testing.T) {
374
- t.Cleanup(resetDedupState); oldWindow := dedupWindow; dedupWindow = 50 * time.Millisecond; t.Cleanup(func() { dedupWindow = oldWindow }); numEntries := 5
375
- for i := 0; i < numEntries; i++ { req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(fmt.Sprintf(`{"id": "clean-%d"}`, i))); rr := httptest.NewRecorder(); captureOutput(t, func() { handler(rr, req) }); if rr.Code != http.StatusOK { t.Fatalf("Setup failed entry %d", i)} }
376
- mapMutex.Lock(); if got := len(seenIDs); got != numEntries { t.Fatalf("Entries after add: %d != %d", got, numEntries) }; mapMutex.Unlock()
377
- time.Sleep(100 * time.Millisecond); now := time.Now(); mapMutex.Lock()
378
- for h, entry := range seenIDs { if now.Sub(entry.timestamp) >= dedupWindow { delete(seenIDs, h) } }
379
- count := len(seenIDs); mapMutex.Unlock(); if count != 0 { t.Errorf("Entries after cleanup: %d != 0", count) }
528
+ t.Cleanup(resetDedupState)
529
+ oldWindow := dedupWindow
530
+ dedupWindow = 50 * time.Millisecond
531
+ t.Cleanup(func() { dedupWindow = oldWindow })
532
+ numEntries := 5
533
+ for i := 0; i < numEntries; i++ {
534
+ req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(fmt.Sprintf(`{"id": "clean-%d"}`, i)))
535
+ rr := httptest.NewRecorder()
536
+ captureOutput(t, func() { handler(rr, req) })
537
+ if rr.Code != http.StatusOK {
538
+ t.Fatalf("Setup failed entry %d", i)
539
+ }
540
+ }
541
+ mapMutex.Lock()
542
+ if got := len(seenIDs); got != numEntries {
543
+ t.Fatalf("Entries after add: %d != %d", got, numEntries)
544
+ }
545
+ mapMutex.Unlock()
546
+ time.Sleep(100 * time.Millisecond)
547
+ now := time.Now()
548
+ mapMutex.Lock()
549
+ for h, entry := range seenIDs {
550
+ if now.Sub(entry.timestamp) >= dedupWindow {
551
+ delete(seenIDs, h)
552
+ }
553
+ }
554
+ count := len(seenIDs)
555
+ mapMutex.Unlock()
556
+ if count != 0 {
557
+ t.Errorf("Entries after cleanup: %d != 0", count)
558
+ }
559
})
560
561
t.Run("MixedKeyTypes", func(t *testing.T) {
383
- t.Cleanup(resetDedupState); keyPaths = []string{"id", "count", "enabled"}; dedupSeparator = "|"
562
+ t.Cleanup(resetDedupState)
563
+ keyPaths = []string{"id", "count", "enabled"}
564
+ dedupSeparator = "|"
565
req1 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "mix", "count": 1, "enabled": true}`))
385
- rr1 := httptest.NewRecorder(); stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
386
- if rr1.Code != http.StatusOK || !strings.Contains(stdout1, `"id":"mix"`) { t.Errorf("Req 1 failed") }
566
+ rr1 := httptest.NewRecorder()
567
+ stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
568
+ if rr1.Code != http.StatusOK || !strings.Contains(stdout1, `"id":"mix"`) {
569
+ t.Errorf("Req 1 failed")
570
+ }
571
req2 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"enabled": true, "count": 1.0, "id": "mix"}`))
388
- rr2 := httptest.NewRecorder(); stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
389
- if rr2.Code != http.StatusOK { t.Errorf("Req 2 status wrong") }; if stdout2 != "" { t.Errorf("Req 2 produced output") }
572
+ rr2 := httptest.NewRecorder()
573
+ stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
574
+ if rr2.Code != http.StatusOK {
575
+ t.Errorf("Req 2 status wrong")
576
+ }
577
+ if stdout2 != "" {
578
+ t.Errorf("Req 2 produced output")
579
+ }
580
req3 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"id": "mix", "count": 1, "enabled": false}`))
391
- rr3 := httptest.NewRecorder(); stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
392
- if rr3.Code != http.StatusOK || !strings.Contains(stdout3, `"enabled":false`) { t.Errorf("Req 3 failed") }
393
- mapMutex.Lock(); mapLen := len(seenIDs); mapMutex.Unlock(); if mapLen != 2 { t.Errorf("map size: got %d want 2", mapLen) }
581
+ rr3 := httptest.NewRecorder()
582
+ stdout3, _ := captureOutput(t, func() { handler(rr3, req3) })
583
+ if rr3.Code != http.StatusOK || !strings.Contains(stdout3, `"enabled":false`) {
584
+ t.Errorf("Req 3 failed")
585
+ }
586
+ mapMutex.Lock()
587
+ mapLen := len(seenIDs)
588
+ mapMutex.Unlock()
589
+ if mapLen != 2 {
590
+ t.Errorf("map size: got %d want 2", mapLen)
591
+ }
592
})
593
594
t.Run("JsonFormatTests", func(t *testing.T) {
595
t.Cleanup(resetDedupState)
398
- testCases := []struct{ name string; body string; expectStatus int; expectOutput bool }{
596
+ testCases := []struct {
597
+ name string
598
+ body string
599
+ expectStatus int
600
+ expectOutput bool
601
+ }{
602
{"EmptyObject", `{}`, http.StatusOK, true}, {"ValidJSON", `{"id": "valid"}`, http.StatusOK, true},
603
{"CompletelyInvalid", `not json`, http.StatusBadRequest, false}, {"IncompleteJSON", `{"id": "inc`, http.StatusBadRequest, false},
604
{"ArrayAsRoot", `[1, 2]`, http.StatusBadRequest, false}, // Expect 400 now
@@ -403,9 +606,15 @@ func TestHandler(t *testing.T) {
606
for _, tc := range testCases {
607
t.Run(tc.name, func(t *testing.T) {
608
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(tc.body))
406
- rr := httptest.NewRecorder(); stdout, _ := captureOutput(t, func() { handler(rr, req) })
407
- if status := rr.Code; status != tc.expectStatus { t.Errorf("status: got %v want %v", status, tc.expectStatus) }
408
- hasOutput := stdout != ""; if hasOutput != tc.expectOutput { t.Errorf("Output mismatch: expected %t, got %t", tc.expectOutput, hasOutput) }
609
+ rr := httptest.NewRecorder()
610
+ stdout, _ := captureOutput(t, func() { handler(rr, req) })
611
+ if status := rr.Code; status != tc.expectStatus {
612
+ t.Errorf("status: got %v want %v", status, tc.expectStatus)
613
+ }
614
+ hasOutput := stdout != ""
615
+ if hasOutput != tc.expectOutput {
616
+ t.Errorf("Output mismatch: expected %t, got %t", tc.expectOutput, hasOutput)
617
+ }
618
})
619
}
620
})
@@ -414,21 +623,36 @@ func TestHandler(t *testing.T) {
623
// Verifies server returns BadRequest for invalid JSON
624
t.Cleanup(resetDedupState)
625
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`not json`))
417
- rr := httptest.NewRecorder(); captureOutput(t, func() { handler(rr, req) })
418
- if status := rr.Code; status != http.StatusBadRequest { t.Errorf("status invalid: got %v want %v", status, http.StatusBadRequest) }
626
+ rr := httptest.NewRecorder()
627
+ captureOutput(t, func() { handler(rr, req) })
628
+ if status := rr.Code; status != http.StatusBadRequest {
629
+ t.Errorf("status invalid: got %v want %v", status, http.StatusBadRequest)
630
+ }
631
req2 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"open":`))
420
- rr2 := httptest.NewRecorder(); captureOutput(t, func() { handler(rr2, req2) })
421
- if status := rr2.Code; status != http.StatusBadRequest { t.Errorf("status incomplete: got %v want %v", status, http.StatusBadRequest) }
632
+ rr2 := httptest.NewRecorder()
633
+ captureOutput(t, func() { handler(rr2, req2) })
634
+ if status := rr2.Code; status != http.StatusBadRequest {
635
+ t.Errorf("status incomplete: got %v want %v", status, http.StatusBadRequest)
636
+ }
637
})
638
639
t.Run("ZeroLengthDedupWindow", func(t *testing.T) {
425
- t.Cleanup(resetDedupState); oldWindow := dedupWindow; dedupWindow = 0; t.Cleanup(func() { dedupWindow = oldWindow })
640
+ t.Cleanup(resetDedupState)
641
+ oldWindow := dedupWindow
642
+ dedupWindow = 0
643
+ t.Cleanup(func() { dedupWindow = oldWindow })
644
jsonBody := `{"id": "zero"}`
645
req1 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
428
- rr1 := httptest.NewRecorder(); stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
429
- if rr1.Code != http.StatusOK || !strings.Contains(stdout1, "zero") { t.Errorf("Req 1 failed") }
646
+ rr1 := httptest.NewRecorder()
647
+ stdout1, _ := captureOutput(t, func() { handler(rr1, req1) })
648
+ if rr1.Code != http.StatusOK || !strings.Contains(stdout1, "zero") {
649
+ t.Errorf("Req 1 failed")
650
+ }
651
req2 := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(jsonBody))
431
- rr2 := httptest.NewRecorder(); stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
432
- if rr2.Code != http.StatusOK || !strings.Contains(stdout2, "zero") { t.Errorf("Req 2 (duplicate) failed") }
652
+ rr2 := httptest.NewRecorder()
653
+ stdout2, _ := captureOutput(t, func() { handler(rr2, req2) })
654
+ if rr2.Code != http.StatusOK || !strings.Contains(stdout2, "zero") {
655
+ t.Errorf("Req 2 (duplicate) failed")
656
+ }
657
})
434
-}
\ No newline at end of file
658
+}
src/go/pkg/confopt/autobool.go
+55
-16
@@ -4,8 +4,6 @@ import (
4
"encoding/json"
5
"fmt"
6
"strings"
7
-
8
- "gopkg.in/yaml.v3"
7
)
8
9
// AutoBool represents a tri-state boolean with explicit auto/enable/disable semantics.
@@ -96,27 +94,59 @@ func (a AutoBool) MarshalYAML() (interface{}, error) {
94
return a.String(), nil
95
}
96
99
-// UnmarshalYAML accepts string scalars (case insensitive) and defaults to auto
100
-// when empty. Any other value results in an error to ensure early feedback.
101
-func (a *AutoBool) UnmarshalYAML(node *yaml.Node) error {
102
- if node == nil {
97
+// UnmarshalYAML accepts literal booleans and strings (case insensitive) and
98
+// defaults to auto when empty. Any other value results in an error to ensure
99
+// early feedback. The signature matches the yaml.v2 marshaler interface so the
100
+// same implementation works for both yaml.v2 and yaml.v3 consumers.
101
+func (a *AutoBool) UnmarshalYAML(unmarshal func(interface{}) error) error {
102
+ if unmarshal == nil {
103
*a = AutoBoolAuto
104
return nil
105
}
106
- if node.Kind != yaml.ScalarNode {
107
- return fmt.Errorf("autobool: expected scalar value, got %v", node.Kind)
106
+
107
+ var raw interface{}
108
+ if err := unmarshal(&raw); err != nil {
109
+ return err
110
}
109
- value := strings.TrimSpace(node.Value)
110
- if value == "" {
111
+
112
+ switch v := raw.(type) {
113
+ case nil:
114
*a = AutoBoolAuto
115
return nil
116
+ case bool:
117
+ if v {
118
+ *a = AutoBoolEnabled
119
+ } else {
120
+ *a = AutoBoolDisabled
121
+ }
122
+ return nil
123
+ case string:
124
+ value := strings.TrimSpace(v)
125
+ if value == "" {
126
+ *a = AutoBoolAuto
127
+ return nil
128
+ }
129
+ parsed, err := parseAutoBool(value)
130
+ if err != nil {
131
+ return err
132
+ }
133
+ *a = parsed
134
+ return nil
135
+ case []byte:
136
+ value := strings.TrimSpace(string(v))
137
+ if value == "" {
138
+ *a = AutoBoolAuto
139
+ return nil
140
+ }
141
+ parsed, err := parseAutoBool(value)
142
+ if err != nil {
143
+ return err
144
+ }
145
+ *a = parsed
146
+ return nil
147
+ default:
148
+ return fmt.Errorf("autobool: expected boolean or string value, got %T", raw)
149
}
114
- parsed, err := parseAutoBool(value)
115
- if err != nil {
116
- return err
117
- }
118
- *a = parsed
119
- return nil
150
}
151
152
// MarshalJSON writes the canonical string representation.
@@ -126,6 +156,15 @@ func (a AutoBool) MarshalJSON() ([]byte, error) {
156
157
// UnmarshalJSON accepts string values (case insensitive).
158
func (a *AutoBool) UnmarshalJSON(data []byte) error {
159
+ var rawBool bool
160
+ if err := json.Unmarshal(data, &rawBool); err == nil {
161
+ if rawBool {
162
+ *a = AutoBoolEnabled
163
+ } else {
164
+ *a = AutoBoolDisabled
165
+ }
166
+ return nil
167
+ }
168
var raw string
169
if err := json.Unmarshal(data, &raw); err != nil {
170
return fmt.Errorf("autobool: expected string value: %w", err)
src/go/pkg/confopt/autobool_test.go
+16
@@ -107,6 +107,22 @@ func TestAutoBoolInvalidInputs(t *testing.T) {
107
}
108
}
109
110
+func TestAutoBoolYAMLBooleanLiteral(t *testing.T) {
111
+ var a AutoBool
112
+ if err := yaml.Unmarshal([]byte("true\n"), &a); err != nil {
113
+ t.Fatalf("expected yaml.Unmarshal to accept boolean literal: %v", err)
114
+ }
115
+ if a != AutoBoolEnabled {
116
+ t.Fatalf("yaml bool literal true => %q, want %q", a, AutoBoolEnabled)
117
+ }
118
+ if err := yaml.Unmarshal([]byte("false\n"), &a); err != nil {
119
+ t.Fatalf("expected yaml.Unmarshal to accept boolean literal: %v", err)
120
+ }
121
+ if a != AutoBoolDisabled {
122
+ t.Fatalf("yaml bool literal false => %q, want %q", a, AutoBoolDisabled)
123
+ }
124
+}
125
+
126
func TestAutoBoolWithDefault(t *testing.T) {
127
if got := AutoBoolAuto.WithDefault(true); got != AutoBoolEnabled {
128
t.Fatalf("AutoBoolAuto.WithDefault(true) => %q, want %q", got, AutoBoolEnabled)
src/go/plugin/ibm.d/README.md
+5
-2
@@ -146,7 +146,8 @@ jobs:
146
dsn: AS400_PROD # ODBC DSN name from /etc/odbc.ini
147
update_every: 10 # Collection frequency in seconds
148
collect_active_jobs: yes # Enable job monitoring
149
- max_active_jobs: 100 # Limit number of jobs tracked
149
+ active_jobs:
150
+ - 123456/QSYS/QSPCJOB # Fully qualified JOB_NUMBER/USER/JOB_NAME
151
reset_statistics: no # Reset cumulative counters
152
```
153
@@ -285,7 +286,9 @@ jobs:
286
- name: tuned_as400
287
dsn: AS400_PROD
288
update_every: 30 # Reduce frequency
288
- max_active_jobs: 50 # Limit job collection
289
+ collect_active_jobs: yes # Enable job metrics
290
+ active_jobs:
291
+ - 123456/QSYS/QSPCJOB # Explicit job list controls collection
292
connection_timeout: 30 # Increase timeout
293
query_timeout: 25 # Increase query timeout
294
```
src/go/plugin/ibm.d/config/ibm.d/as400.conf
+54
-45
@@ -45,20 +45,24 @@ jobs:
45
# # ssl_server_cert_path: /path/to/cert.pem
46
#
47
# # Update interval in seconds
48
- # # Default: 10
49
- # update_every: 10
48
+ # # Default: 5
49
+ # update_every: 5
50
#
51
# # Connection timeout
52
# # Default: 2
53
# timeout: 2
54
#
55
- # # Maximum database connections
56
- # # Default: 1
57
- # max_db_conns: 1
58
- #
59
- # # Maximum database connection lifetime
60
- # # Default: 600
61
- # max_db_life_time: 600
55
+ # # Slow-path worker for expensive queries
56
+ # # Defaults: slow_path=true, slow_path_update_every=10, slow_path_max_connections=1
57
+ # slow_path: true
58
+ # slow_path_update_every: 10
59
+ # slow_path_max_connections: 1
60
+ #
61
+ # # Batch-path worker for aggregate queue totals (disabled by default)
62
+ # # Recommended: enable only when queue totals are required and use a long interval (>=600s in production).
63
+ # batch_path: false
64
+ # batch_path_update_every: 60
65
+ # batch_path_max_connections: 1
66
67
## Example: ODBC DSN string configuration
68
# - name: as400_dsn
@@ -94,13 +98,12 @@ jobs:
98
# password: secret
99
# database: '*SYSBAS'
100
#
97
- # # Collect top active jobs by CPU (requires IBM i V7R3+)
98
- # # Default: false
99
- # collect_active_jobs: true
100
- #
101
- # # Maximum active jobs to collect
102
- # # Default: 100
103
- # max_active_jobs: 20
101
+ # # Active jobs to monitor (fully-qualified identifiers: JOB_NUMBER/USER/JOB_NAME).
102
+ # # Provide concrete job names to enable active job metrics; leave empty to disable.
103
+ # # Example:
104
+ # # active_jobs:
105
+ # # - 123456/QSYS/QSPCJOB
106
+ # active_jobs: []
107
#
108
# # Per-instance metric collection settings
109
# # These control which detailed metrics are collected
@@ -126,32 +129,24 @@ jobs:
129
# # Default: 100 (0 = unlimited)
130
# max_subsystems: 10
131
#
129
- # # Pattern to filter subsystems. Supports wildcards (*, ?) and multiple patterns separated by |
130
- # # Example: 'QINTER' for interactive subsystem only, 'Q*' for all Q subsystems
131
- # # Default: '' (all subsystems)
132
- # collect_subsystems_matching: ''
133
- #
134
- # # Collect job queue metrics (per-queue statistics)
135
- # # Default: true
136
- # collect_job_queue_metrics: true
137
- #
138
- # # Maximum number of job queues to monitor
139
- # # Default: 100 (0 = unlimited)
140
- # max_job_queues: 20
132
+ # # Message queues to monitor (explicit list). Leave empty to disable collection.
133
+ # message_queues:
134
+ # - QSYS/QSYSOPR
135
+ # - QSYS/QSYSMSG
136
+ # - QSYS/QHST
137
#
142
- # # Pattern to filter job queues. Supports wildcards (*, ?) and multiple patterns separated by |
143
- # # Example: 'QBATCH*' for batch queues only, '*PROD*|*TEST*' for prod or test queues
144
- # # Default: '' (all queues)
145
- # collect_job_queues_matching: ''
138
+ # # Job queues to monitor (explicit list). Leave empty to disable collection.
139
+ # job_queues: []
140
#
147
- # # Collect metrics for top active jobs by CPU usage
148
- # # Default: false
149
- # collect_active_jobs: false
141
+ # # Output queues to monitor (explicit list). Leave empty to disable collection.
142
+ # output_queues: []
143
#
151
- # # Maximum number of active jobs to monitor
152
- # # Default: 100
153
- # max_active_jobs: 10
154
- #
144
+ # # Active jobs to monitor (fully-qualified identifiers: JOB_NUMBER/USER/JOB_NAME).
145
+ # # Provide concrete job names to enable active job metrics; leave empty to disable.
146
+ # # Example:
147
+ # # active_jobs:
148
+ # # - 123456/QUSER/QPADEV0001
149
+ # active_jobs: []
150
#
151
# # Virtual node (vnode) assignment
152
# # Associates this job with a Virtual Node in Netdata Cloud
@@ -171,7 +166,9 @@ jobs:
166
# # Disable all per-instance metrics
167
# collect_disk_metrics: false
168
# collect_subsystem_metrics: false
174
- # collect_job_queue_metrics: false
169
+ # message_queues: []
170
+ # job_queues: []
171
+ # output_queues: []
172
173
## Example: Filtered collection
174
## Monitor specific resources using SQL LIKE patterns
@@ -190,8 +187,10 @@ jobs:
187
# # Monitor only interactive and batch subsystems
188
# collect_subsystems_matching: 'Q*'
189
#
193
- # # Monitor only batch job queues
194
- # collect_job_queues_matching: 'QBATCH*'
190
+ # # Monitor specific job queues
191
+ # job_queues:
192
+ # - QSYS/QBATCH
193
+ # - MYLIB/MYJOBQ
194
195
#------------------------------------------------------------------------------
196
# COLLECTED METRICS
@@ -226,12 +225,22 @@ jobs:
225
## - Jobs held on job queues
226
## - Storage used (MB)
227
229
-## Per-job-queue metrics (when collect_job_queue_metrics is true):
228
+## Per-job-queue metrics (when job_queues list is non-empty):
229
## - Jobs waiting
230
## - Jobs held
231
## - Jobs scheduled
232
234
-## Per-job metrics (when collect_active_jobs is true):
233
+## Per-message-queue metrics (when message_queues list is non-empty):
234
+## - Total messages
235
+## - Message counts by type (informational, inquiry, diagnostic, escape, notify, sender copy)
236
+## - Maximum severity in queue
237
+
238
+## Per-output-queue metrics (when output_queues list is non-empty):
239
+## - Number of files waiting
240
+## - Active writers
241
+## - Released status
242
+
243
+## Per-job metrics (when active_jobs list is non-empty and collect_active_jobs is true):
244
## - CPU time used
245
## - Temporary storage used
246
## - Job active time
@@ -291,4 +300,4 @@ jobs:
300
## ./ibm.d.plugin -d -m as400 --dump=3s --dump-summary
301
##
302
## Test ODBC connection separately:
294
-## isql -v "Driver={IBM i Access ODBC Driver};System=hostname;Uid=user;Pwd=pass;"
\ No newline at end of file
303
+## isql -v "Driver={IBM i Access ODBC Driver};System=hostname;Uid=user;Pwd=pass;"
src/go/plugin/ibm.d/docgen/config_parser.go
+107
-37
@@ -144,17 +144,30 @@ func extractConfigField(fieldName string, field *ast.Field, comments []*ast.Comm
144
// Convert Go type to JSON Schema type
145
isPointer := strings.HasPrefix(fieldType, "*")
146
jsonType := convertToJSONType(fieldType)
147
+ var itemsType string
148
+ if sliceElem, ok := getSliceElementGoType(fieldType); ok {
149
+ itemsType = convertToJSONType(sliceElem)
150
+ if itemsType == "array" {
151
+ itemsType = "string"
152
+ }
153
+ }
154
155
// Extract documentation from comments
156
description := extractFieldDescription(fieldName, field, comments)
157
+ title := formatTitleFromJSONName(jsonName)
158
+ if title == "" {
159
+ title = camelToWords(fieldName)
160
+ }
161
162
// Create field
163
configField := &ConfigField{
164
Name: fieldName,
165
JSONName: jsonName,
166
Type: jsonType,
167
+ Title: title,
168
Required: !strings.Contains(yamlName, "omitempty"),
169
Description: description,
170
+ ItemsType: itemsType,
171
Pointer: isPointer,
172
GoType: fieldType,
173
}
@@ -176,6 +189,12 @@ func extractGoType(expr ast.Expr) string {
189
if ident, ok := t.X.(*ast.Ident); ok {
190
return ident.Name + "." + t.Sel.Name
191
}
192
+ case *ast.ArrayType:
193
+ elemType := extractGoType(t.Elt)
194
+ if elemType == "" {
195
+ elemType = "interface{}"
196
+ }
197
+ return "[]" + elemType
198
case *ast.StarExpr:
199
inner := extractGoType(t.X)
200
if inner == "" {
@@ -217,6 +236,9 @@ func convertToSnakeCase(s string) string {
236
}
237
238
func convertToJSONType(goType string) string {
239
+ if strings.HasPrefix(goType, "[]") {
240
+ return "array"
241
+ }
242
baseType := strings.TrimPrefix(goType, "*")
243
switch goType {
244
case "string":
@@ -248,33 +270,26 @@ func convertToJSONType(goType string) string {
270
}
271
272
func extractFieldDescription(fieldName string, field *ast.Field, comments []*ast.CommentGroup) string {
251
- // First, try to extract description from field comment
273
+ extractCommentText := func(list []*ast.Comment) string {
274
+ lines := make([]string, 0, len(list))
275
+ for _, c := range list {
276
+ clean := strings.TrimSpace(strings.TrimPrefix(strings.TrimPrefix(strings.TrimSuffix(c.Text, "*/"), "/*"), "//"))
277
+ if clean != "" {
278
+ lines = append(lines, clean)
279
+ }
280
+ }
281
+ return strings.Join(lines, " ")
282
+ }
283
+
284
if field.Comment != nil && len(field.Comment.List) > 0 {
253
- // Use the first comment line as description
254
- comment := field.Comment.List[0].Text
255
- // Clean up the comment (remove // or /*)
256
- comment = strings.TrimSpace(comment)
257
- comment = strings.TrimPrefix(comment, "//")
258
- comment = strings.TrimPrefix(comment, "/*")
259
- comment = strings.TrimSuffix(comment, "*/")
260
- comment = strings.TrimSpace(comment)
261
- if comment != "" {
262
- return comment
285
+ if text := extractCommentText(field.Comment.List); text != "" {
286
+ return text
287
}
288
}
289
266
- // If no field comment, try to find comment before the field
290
if field.Doc != nil && len(field.Doc.List) > 0 {
268
- // Use the last doc comment line as description
269
- comment := field.Doc.List[len(field.Doc.List)-1].Text
270
- // Clean up the comment
271
- comment = strings.TrimSpace(comment)
272
- comment = strings.TrimPrefix(comment, "//")
273
- comment = strings.TrimPrefix(comment, "/*")
274
- comment = strings.TrimSuffix(comment, "*/")
275
- comment = strings.TrimSpace(comment)
276
- if comment != "" {
277
- return comment
291
+ if text := extractCommentText(field.Doc.List); text != "" {
292
+ return text
293
}
294
}
295
@@ -297,58 +312,58 @@ func extractFieldDescription(fieldName string, field *ast.Field, comments []*ast
312
313
// Generic connection-related fields
314
if strings.Contains(fieldLower, "host") {
300
- return "Server hostname or IP address"
315
+ return "Hostname"
316
}
317
if strings.Contains(fieldLower, "port") {
303
- return "Server port number"
318
+ return "Port"
319
}
320
if strings.Contains(fieldLower, "user") || strings.Contains(fieldLower, "username") {
306
- return "Username for authentication"
321
+ return "Username"
322
}
323
if strings.Contains(fieldLower, "password") || strings.Contains(fieldLower, "pass") {
309
- return "Password for authentication"
324
+ return "Password"
325
}
326
327
// Collection control fields
328
if strings.HasPrefix(fieldLower, "collect") {
314
- resource := extractResourceFromFieldName(fieldName)
315
- return fmt.Sprintf("Enable collection of %s metrics", resource)
329
+ resource := camelToWords(strings.TrimPrefix(fieldName, "Collect"))
330
+ return fmt.Sprintf("Collect %s", resource)
331
}
332
333
// Selector fields
334
if strings.HasSuffix(fieldLower, "selector") {
320
- resource := extractResourceFromFieldName(fieldName)
321
- return fmt.Sprintf("Pattern to filter %s (wildcards supported)", resource)
335
+ resource := camelToWords(strings.TrimSuffix(fieldName, "Selector"))
336
+ return fmt.Sprintf("Filter %s", resource)
337
}
338
339
// Timeout fields
340
if strings.Contains(fieldLower, "timeout") {
326
- return "Connection timeout duration in seconds"
341
+ return "Timeout"
342
}
343
344
// SSL/TLS fields
345
if strings.Contains(fieldLower, "ssl") || strings.Contains(fieldLower, "tls") {
331
- return "Enable SSL/TLS encrypted connection"
346
+ return "SSL/TLS"
347
}
348
349
// URL/URI fields
350
if strings.Contains(fieldLower, "url") || strings.Contains(fieldLower, "uri") {
336
- return "Connection URL or URI"
351
+ return "URL"
352
}
353
354
// DSN fields
355
if strings.Contains(fieldLower, "dsn") {
341
- return "Data Source Name (DSN) for connection"
356
+ return "DSN"
357
}
358
359
// Max/limit fields
360
if strings.HasPrefix(fieldLower, "max") {
346
- resource := extractResourceFromFieldName(fieldName)
347
- return fmt.Sprintf("Maximum number of %s to monitor", resource)
361
+ resource := camelToWords(strings.TrimPrefix(fieldName, "Max"))
362
+ return fmt.Sprintf("Max %s", resource)
363
}
364
365
// Default to a more descriptive generic description
351
- return fmt.Sprintf("%s", camelToWords(fieldName))
366
+ return camelToWords(fieldName)
367
}
368
369
// Helper function to extract resource name from field name
@@ -394,6 +409,33 @@ func camelToWords(s string) string {
409
return result.String()
410
}
411
412
+var titleAcronyms = map[string]string{
413
+ "dsn": "DSN",
414
+ "ssl": "SSL",
415
+ "tls": "TLS",
416
+ "ibm": "IBM",
417
+ "odbc": "ODBC",
418
+}
419
+
420
+func formatTitleFromJSONName(name string) string {
421
+ if name == "" {
422
+ return ""
423
+ }
424
+ parts := strings.Split(name, "_")
425
+ for i, part := range parts {
426
+ if part == "" {
427
+ continue
428
+ }
429
+ lower := strings.ToLower(part)
430
+ if upper, ok := titleAcronyms[lower]; ok {
431
+ parts[i] = upper
432
+ continue
433
+ }
434
+ parts[i] = strings.ToUpper(lower[:1]) + lower[1:]
435
+ }
436
+ return strings.Join(parts, " ")
437
+}
438
+
439
func setFieldDefaults(field *ConfigField) {
440
// Set defaults based on field name patterns
441
switch field.Name {
@@ -630,6 +672,16 @@ func (g *DocGenerator) extractDefaultsFromLiteral(lit *ast.CompositeLit, default
672
fieldName := ident.Name
673
switch val := kv.Value.(type) {
674
case *ast.CompositeLit:
675
+ if isArrayLiteral(val) {
676
+ values := make([]interface{}, 0, len(val.Elts))
677
+ for _, elt := range val.Elts {
678
+ if value := g.extractValue(elt); value != nil {
679
+ values = append(values, value)
680
+ }
681
+ }
682
+ defaults[fieldName] = values
683
+ continue
684
+ }
685
nested := make(map[string]interface{})
686
g.extractDefaultsFromLiteral(val, nested)
687
// Flatten nested composite literals for embedded configs such as framework.Config.
@@ -740,6 +792,24 @@ func exprToString(expr ast.Expr) string {
792
return strings.TrimSpace(buf.String())
793
}
794
795
+func getSliceElementGoType(goType string) (string, bool) {
796
+ if !strings.HasPrefix(goType, "[]") {
797
+ return "", false
798
+ }
799
+ elem := strings.TrimPrefix(goType, "[]")
800
+ return elem, true
801
+}
802
+
803
+func isArrayLiteral(lit *ast.CompositeLit) bool {
804
+ if lit == nil {
805
+ return false
806
+ }
807
+ if _, ok := lit.Type.(*ast.ArrayType); ok {
808
+ return true
809
+ }
810
+ return false
811
+}
812
+
813
func (g *DocGenerator) extractConstValues(file *ast.File) map[string]interface{} {
814
consts := make(map[string]interface{})
815
if file == nil {
src/go/plugin/ibm.d/docgen/main.go
+41
-13
@@ -59,6 +59,8 @@ type ConfigField struct {
59
Name string
60
JSONName string
61
Type string
62
+ Title string
63
+ ItemsType string
64
Required bool
65
Default interface{}
66
Description string
@@ -237,6 +239,7 @@ func (g *DocGenerator) parseConfig() ([]ConfigField, error) {
239
Name: name,
240
JSONName: name,
241
Type: fieldType,
242
+ Title: formatTitleFromJSONName(name),
243
Required: false,
244
Default: defaultVal,
245
Description: desc,
@@ -275,6 +278,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
278
Name: "update_every",
279
JSONName: "update_every",
280
Type: "integer",
281
+ Title: formatTitleFromJSONName("update_every"),
282
Required: false,
283
Default: 10,
284
Description: "Data collection frequency",
@@ -284,6 +288,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
288
Name: "reset_statistics",
289
JSONName: "reset_statistics",
290
Type: "boolean",
291
+ Title: formatTitleFromJSONName("reset_statistics"),
292
Required: false,
293
Default: false,
294
Description: "ResetStatistics enables SQL calls that reset IBM i system statistics on each run.",
@@ -292,6 +297,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
297
Name: "endpoint",
298
JSONName: "endpoint",
299
Type: "string",
300
+ Title: formatTitleFromJSONName("endpoint"),
301
Required: false,
302
Default: "dummy://localhost",
303
Description: "Connection endpoint",
@@ -301,6 +307,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
307
Name: "connect_timeout",
308
JSONName: "connect_timeout",
309
Type: "integer",
310
+ Title: formatTitleFromJSONName("connect_timeout"),
311
Required: false,
312
Default: 5,
313
Description: "Connection timeout in seconds",
@@ -311,6 +318,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
318
Name: "collect_items",
319
JSONName: "collect_items",
320
Type: "boolean",
321
+ Title: formatTitleFromJSONName("collect_items"),
322
Required: false,
323
Default: true,
324
Description: "Enable collection of item metrics",
@@ -319,6 +327,7 @@ func (g *DocGenerator) getFallbackConfigFields() []ConfigField {
327
Name: "max_items",
328
JSONName: "max_items",
329
Type: "integer",
330
+ Title: formatTitleFromJSONName("max_items"),
331
Required: false,
332
Default: 10,
333
Description: "Maximum number of items to collect",
@@ -395,9 +404,22 @@ func (g *DocGenerator) generateConfigSchema(fields []ConfigField) error {
404
405
for _, field := range fields {
406
prop := map[string]interface{}{
398
- "title": field.Description,
407
+ "title": field.Title,
408
"type": field.Type,
409
}
410
+ if field.Description != "" {
411
+ prop["description"] = field.Description
412
+ }
413
+
414
+ if field.Type == "array" {
415
+ itemsType := field.ItemsType
416
+ if itemsType == "" {
417
+ itemsType = "string"
418
+ }
419
+ prop["items"] = map[string]interface{}{
420
+ "type": itemsType,
421
+ }
422
+ }
423
424
if field.Default != nil {
425
prop["default"] = field.Default
@@ -438,17 +460,23 @@ func (g *DocGenerator) generateConfigSchema(fields []ConfigField) error {
460
}
461
groupFields[group] = append(groupFields[group], field.JSONName)
462
441
- if field.UIWidget != "" || field.UIHelp != "" || field.UIPlaceholder != "" {
442
- opts := make(map[string]interface{})
443
- if field.UIWidget != "" {
444
- opts["ui:widget"] = field.UIWidget
445
- }
446
- if field.UIHelp != "" {
447
- opts["ui:help"] = field.UIHelp
448
- }
449
- if field.UIPlaceholder != "" {
450
- opts["ui:placeholder"] = field.UIPlaceholder
451
- }
463
+ opts, exists := fieldUIOptions[field.JSONName]
464
+ if !exists {
465
+ opts = make(map[string]interface{})
466
+ }
467
+ if field.UIWidget != "" {
468
+ opts["ui:widget"] = field.UIWidget
469
+ }
470
+ if field.UIHelp != "" {
471
+ opts["ui:help"] = field.UIHelp
472
+ }
473
+ if field.UIPlaceholder != "" {
474
+ opts["ui:placeholder"] = field.UIPlaceholder
475
+ }
476
+ if field.Type == "array" {
477
+ opts["ui:listFlavour"] = "list"
478
+ }
479
+ if len(opts) > 0 {
480
fieldUIOptions[field.JSONName] = opts
481
}
482
}
@@ -463,6 +491,7 @@ func (g *DocGenerator) generateConfigSchema(fields []ConfigField) error {
491
"fullPage": true,
492
},
493
}
494
+ uiSchema["ui:flavour"] = "tabs"
495
496
if len(groupOrder) > 0 {
497
tabs := make([]map[string]interface{}, 0, len(groupOrder))
@@ -477,7 +506,6 @@ func (g *DocGenerator) generateConfigSchema(fields []ConfigField) error {
506
})
507
}
508
if len(tabs) > 0 {
480
- uiSchema["ui:flavour"] = "tabs"
509
uiSchema["ui:options"] = map[string]interface{}{
510
"tabs": tabs,
511
}
src/go/plugin/ibm.d/modules/as400/README.md
+87
-12
@@ -13,6 +13,14 @@ expose CPU, memory, storage, job, and subsystem activity.
13
- libodbc.so (provided by unixODBC)
14
- IBM i Access Client Solutions
15
16
+**Collection paths**
17
+
18
+The collector executes queries in multiple tracks:
19
+
20
+- **Fast path (5s)**: lightweight system status queries remain sequential on the main plugin thread.
21
+- **Slow path (10s beat)**: heavier queries (per-queue metrics, subsystems, plan cache, etc.) run in a background worker with bounded concurrency.
22
+- **Batch path (≥60s beat)**: optional long-period worker used for expensive aggregate queries such as queue totals. Disabled by default unless queue totals are explicitly enabled.
23
+
24
**CPU Collection Methods:**
25
26
The collector uses a hybrid approach for CPU utilization metrics to handle IBM i 7.4+ where
@@ -58,6 +66,13 @@ statistics on each query via `SYSTEM_STATUS(RESET_STATISTICS=>'YES')`. When enab
66
67
Default: `false` (statistics are not reset, using `RESET_STATISTICS=>'NO'`)
68
69
+**Chart Gaps During Baseline Resets:**
70
+
71
+The `as400.system_activity_cpu_rate` and `as400.system_activity_cpu_utilization` charts rely on
72
+delta calculations. When the collector detects that IBM i reset these statistics—or when it is
73
+still establishing the initial baseline—it intentionally skips a sample instead of emitting a zero
74
+or spike. Netdata renders those skipped samples as small gaps, which is expected behaviour.
75
+
76
**Cardinality Management:**
77
78
To prevent performance issues from excessive metric creation, the collector enforces cardinality
@@ -81,11 +96,25 @@ Use **both** limit and selector options together to manage high-cardinality envi
96
| `max_job_queues` | Maximum job queues to monitor | 100 |
97
| `max_message_queues` | Maximum message queues to monitor | 100 |
98
| `max_output_queues` | Maximum output queues to monitor | 100 |
84
-| `max_active_jobs` | Maximum active jobs to monitor | 100 |
99
+| `active_jobs` | Fully qualified active jobs to monitor (`JOB_NUMBER/USER/JOB_NAME`) | `[]` |
100
| `collect_disks_matching` | Glob pattern to filter disks (e.g., `"001* 002*"`) | `""` (match all) |
101
| `collect_subsystems_matching` | Glob pattern to filter subsystems (e.g., `"QINTER QBATCH"`) | `""` (match all) |
102
| `collect_job_queues_matching` | Glob pattern to filter job queues (e.g., `"QSYS/*"`) | `""` (match all) |
103
104
+Optional batch-path controls:
105
+
106
+| Option | Purpose | Default |
107
+|--------|---------|---------|
108
+| `batch_path` | Enables the long-period batch worker for aggregate queries | `false` |
109
+| `batch_path_update_every` | Batch worker cadence (minimum 60s, recommend ≥600s in production) | `60s` |
110
+| `batch_path_max_connections` | Maximum concurrent connections for batch queries | `1` |
111
+| `collect_message_queue_totals` | Enables full-scan counting of all message queues and messages | `auto` (off) |
112
+| `collect_job_queue_totals` | Enables aggregate counting of job queues and queued jobs | `auto` (off) |
113
+| `collect_output_queue_totals` | Enables aggregate counting of output queues and spooled files | `auto` (off) |
114
+
115
+> **Warning:** queue totals require scanning IBM i catalog views and can be very expensive on large systems. Leave these options disabled unless aggregate counts are absolutely necessary.
116
+
117
+
118
**Example Workflow:**
119
120
1. System has 500 disks, collector skips disk metrics (exceeds default limit of 100)
@@ -99,6 +128,15 @@ Use **both** limit and selector options together to manage high-cardinality envi
128
- Set limits based on your Netdata server's capacity (each instance = multiple charts)
129
- Start with defaults and adjust based on actual usage patterns
130
131
+**IBM i 7.2–7.3 Behavior Note (Message Queues):**
132
+
133
+IBM i 7.4 introduced a message-queue table function that returns only the live backlog. On
134
+7.2–7.3 systems we fall back to the `QSYS2.MESSAGE_QUEUE_INFO` view, which includes *all*
135
+recorded messages (even those already processed/cleared from the queue). Aggregations—especially
136
+`MAX(SEVERITY)`—therefore reflect the historical log, not just the outstanding backlog. This
137
+behaviour is inherent to the IBM SQL service and can lead to higher-than-expected max severity
138
+values on pre-7.4 systems.
139
+
140
Network interface metrics have a fixed internal limit of 50 instances, and HTTP server metrics are capped at 200 instances; these limits are currently not configurable.
141
142
@@ -275,6 +313,22 @@ Metrics:
313
| as400.network_interface_status | active | status |
314
| as400.network_interface_mtu | mtu | bytes |
315
316
+### Per observability
317
+
318
+These metrics refer to individual observability instances.
319
+
320
+Labels:
321
+
322
+| Label | Description |
323
+|:------|:------------|
324
+| path | Path identifier |
325
+
326
+Metrics:
327
+
328
+| Metric | Dimensions | Unit |
329
+|:-------|:-----------|:-----|
330
+| netdata.plugin_ibm.as400_query_latency | analyze_plan_cache, count_disks, count_http_servers, count_network_interfaces, count_subsystems, detect_ibmi_version_primary, detect_ibmi_version_fallback, disk_instances, disk_instances_enhanced, disk_status, http_server_info, job_info, job_queues, job_queue_totals, memory_pools, message_queue_aggregates, message_queue_totals, network_connections, network_interfaces, output_queue_info, output_queue_totals, plan_cache_summary, serial_number, system_activity, system_model, system_status, temp_storage_named, temp_storage_total, technology_refresh_level, active_job, other | ms |
331
+
332
### Per outputqueue
333
334
These metrics refer to individual outputqueue instances.
@@ -311,6 +365,24 @@ Metrics:
365
|:-------|:-----------|:-----|
366
| as400.plan_cache_summary | value | value |
367
368
+### Per queueoverview
369
+
370
+These metrics refer to individual queueoverview instances.
371
+
372
+Labels:
373
+
374
+| Label | Description |
375
+|:------|:------------|
376
+| queue_type | Queue_type identifier |
377
+| item_type | Item_type identifier |
378
+
379
+Metrics:
380
+
381
+| Metric | Dimensions | Unit |
382
+|:-------|:-----------|:-----|
383
+| as400.queues_count | queues | queues |
384
+| as400.queued_items | items | items |
385
+
386
### Per subsystem
387
388
These metrics refer to individual subsystem instances.
@@ -366,12 +438,10 @@ The following options can be defined globally or per job.
438
439
| Name | Description | Default | Required | Min | Max |
440
|:-----|:------------|:--------|:---------|:----|:----|
369
-| update_every | Data collection frequency | `10` | no | 1 | - |
441
+| update_every | Data collection frequency | `5` | no | 1 | - |
442
| Vnode | Vnode allows binding the collector to a virtual node. | `` | no | - | - |
443
| DSN | DSN provides a full IBM i ODBC connection string if manual override is needed. | `` | no | - | - |
444
| Timeout | Timeout controls how long to wait for SQL statements and RPCs. | `2000000000` | no | - | - |
373
-| MaxDbConns | MaxDbConns restricts the maximum number of open ODBC connections. | `1` | no | - | - |
374
-| MaxDbLifeTime | MaxDbLifeTime limits how long a pooled connection may live before being recycled. | `600000000000` | no | - | - |
445
| Hostname | Hostname is the remote IBM i host to monitor. | `` | no | - | - |
446
| Port | Port is the TCP port for the IBM i Access ODBC server. | `8471` | no | 1 | 65535 |
447
| Username | Username supplies the credentials used for authentication. | `` | no | - | - |
@@ -383,21 +453,26 @@ The following options can be defined globally or per job.
453
| ResetStatistics | ResetStatistics toggles destructive SQL services that reset system statistics on each query. | `false` | no | - | - |
454
| CollectDiskMetrics | CollectDiskMetrics toggles collection of disk unit statistics. | `auto` | no | - | - |
455
| CollectSubsystemMetrics | CollectSubsystemMetrics toggles collection of subsystem activity metrics. | `auto` | no | - | - |
386
-| CollectJobQueueMetrics | CollectJobQueueMetrics toggles collection of job queue backlog metrics. | `auto` | no | - | - |
456
| CollectActiveJobs | CollectActiveJobs toggles collection of detailed per-job metrics. | `auto` | no | - | - |
457
| CollectHTTPServerMetrics | CollectHTTPServerMetrics toggles collection of IBM HTTP Server statistics. | `auto` | no | - | - |
389
-| CollectMessageQueueMetrics | CollectMessageQueueMetrics toggles collection of IBM i message queue metrics. | `auto` | no | - | - |
390
-| CollectOutputQueueMetrics | CollectOutputQueueMetrics toggles collection of IBM i output queue metrics. | `auto` | no | - | - |
458
| CollectPlanCacheMetrics | CollectPlanCacheMetrics toggles collection of plan cache analysis metrics. | `auto` | no | - | - |
459
+| CollectMessageQueueTotals | CollectMessageQueueTotals enables expensive aggregate counting across all message queues. | `auto` | no | - | - |
460
+| CollectJobQueueTotals | CollectJobQueueTotals enables expensive aggregate counting across all job queues. | `auto` | no | - | - |
461
+| CollectOutputQueueTotals | CollectOutputQueueTotals enables expensive aggregate counting across all output queues. | `auto` | no | - | - |
462
+| SlowPath | SlowPath enables the asynchronous slow-path worker for heavy queries. | `true` | no | - | - |
463
+| SlowPathUpdateEvery | SlowPathUpdateEvery controls the beat interval for the slow-path worker. | `10000000000` | no | - | - |
464
+| SlowPathMaxConnections | SlowPathMaxConnections caps the number of concurrent queries the slow-path worker may run. | `1` | no | - | - |
465
+| BatchPath | BatchPath enables the long-period batch worker for expensive queue aggregates. | `true` | no | - | - |
466
+| BatchPathUpdateEvery | BatchPathUpdateEvery controls the beat interval for the batch worker. | `60000000000` | no | - | - |
467
+| BatchPathMaxConnections | BatchPathMaxConnections caps concurrent queries for the batch worker. | `1` | no | - | - |
468
| MaxDisks | MaxDisks caps how many disk units may be charted. | `100` | no | - | - |
469
| MaxSubsystems | MaxSubsystems caps how many subsystems may be charted. | `100` | no | - | - |
394
-| MaxJobQueues | MaxJobQueues caps how many job queues may be charted. | `100` | no | - | - |
395
-| MaxMessageQueues | MaxMessageQueues caps how many message queues may be charted. | `100` | no | - | - |
396
-| MaxOutputQueues | MaxOutputQueues caps how many output queues may be charted. | `100` | no | - | - |
397
-| MaxActiveJobs | MaxActiveJobs caps how many active jobs may be charted. | `100` | no | - | - |
470
| DiskSelector | DiskSelector filters disk units by name using glob-style patterns. | `` | no | - | - |
471
| SubsystemSelector | SubsystemSelector filters subsystems by name using glob-style patterns. | `` | no | - | - |
400
-| JobQueueSelector | JobQueueSelector filters job queues by name using glob-style patterns. | `` | no | - | - |
472
+| ActiveJobs | ActiveJobs lists active jobs to monitor, using fully-qualified job identifiers (JOB_NUMBER/USER/JOB_NAME). When empty, active job collection is disabled. | `nil` | no | - | - |
473
+| MessageQueues | MessageQueues lists message queues to collect, formatted as LIBRARY/QUEUE strings. When empty, message queue collection is disabled. The default configuration monitors QSYS/QSYSOPR, QSYS/QSYSMSG, and QSYS/QHST. | `[QSYS/QSYSOPR QSYS/QSYSMSG QSYS/QHST]` | no | - | - |
474
+| JobQueues | JobQueues lists job queues to collect, formatted as LIBRARY/QUEUE strings. When empty, job queue collection is disabled. | `nil` | no | - | - |
475
+| OutputQueues | OutputQueues lists output queues to collect, formatted as LIBRARY/QUEUE strings. When empty, output queue collection is disabled. | `nil` | no | - | - |
476
477
### Examples
478
src/go/plugin/ibm.d/modules/as400/batch_path.go
new
+342
@@ -0,0 +1,342 @@
1
+//go:build cgo
2
+// +build cgo
3
+
4
+package as400
5
+
6
+// SPDX-License-Identifier: GPL-3.0-or-later
7
+
8
+import (
9
+ "context"
10
+ "errors"
11
+ "fmt"
12
+ "sync"
13
+ "time"
14
+
15
+ as400proto "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/as400"
16
+)
17
+
18
+const (
19
+ queryNameMessageQueueTotals = "message_queue_totals"
20
+ queryNameJobQueueTotals = "job_queue_totals"
21
+ queryNameOutputQueueTotals = "output_queue_totals"
22
+)
23
+
24
+type batchPathConfig struct {
25
+ enabled bool
26
+ interval time.Duration
27
+ maxConnections int
28
+}
29
+
30
+type queueTotalsSnapshot struct {
31
+ timestamp time.Time
32
+ err error
33
+ queues map[string]int64
34
+ items map[string]int64
35
+}
36
+
37
+type batchCache struct {
38
+ mu sync.RWMutex
39
+ totals queueTotalsSnapshot
40
+ latency latencyCache
41
+}
42
+
43
+func (c *Collector) batchTotalsEnabled() bool {
44
+ return c != nil && (c.CollectMessageQueueTotals.IsEnabled() || c.CollectJobQueueTotals.IsEnabled() || c.CollectOutputQueueTotals.IsEnabled())
45
+}
46
+
47
+func (c *Collector) startBatchPath() error {
48
+ c.stopBatchPath()
49
+
50
+ totalsEnabled := c.batchTotalsEnabled()
51
+ if !c.BatchPath && totalsEnabled {
52
+ c.Infof("batch path not started: batch_path is disabled while totals are enabled (message=%s job=%s output=%s)",
53
+ c.CollectMessageQueueTotals.String(), c.CollectJobQueueTotals.String(), c.CollectOutputQueueTotals.String())
54
+ }
55
+
56
+ cfg := batchPathConfig{
57
+ enabled: c.BatchPath && totalsEnabled,
58
+ interval: time.Duration(c.BatchPathUpdateEvery),
59
+ maxConnections: c.BatchPathMaxConnections,
60
+ }
61
+
62
+ if cfg.interval <= 0 {
63
+ cfg.interval = time.Minute
64
+ }
65
+ if cfg.interval < time.Minute {
66
+ c.Warningf("batch path update every %s is shorter than 1m; using 1m", cfg.interval)
67
+ cfg.interval = time.Minute
68
+ }
69
+ if cfg.maxConnections <= 0 {
70
+ cfg.maxConnections = 1
71
+ }
72
+
73
+ c.batch.config = cfg
74
+
75
+ if !cfg.enabled {
76
+ c.Infof("batch path not started: totals-enabled=%t (message=%s job=%s output=%s) batch_path=%t",
77
+ totalsEnabled,
78
+ c.CollectMessageQueueTotals.String(), c.CollectJobQueueTotals.String(), c.CollectOutputQueueTotals.String(),
79
+ c.BatchPath)
80
+ return nil
81
+ }
82
+
83
+ clientCfg := as400proto.Config{
84
+ DSN: c.DSN,
85
+ Timeout: time.Duration(c.Timeout),
86
+ MaxOpenConns: cfg.maxConnections,
87
+ }
88
+
89
+ client := as400proto.NewClient(clientCfg)
90
+ ctx := context.Background()
91
+ if err := client.Connect(ctx); err != nil {
92
+ return fmt.Errorf("batch path: connect failed: %w", err)
93
+ }
94
+ if err := client.Ping(ctx); err != nil {
95
+ _ = client.Close()
96
+ return fmt.Errorf("batch path: ping failed: %w", err)
97
+ }
98
+
99
+ runCtx, cancel := context.WithCancel(context.Background())
100
+ c.batch.client = client
101
+ c.batch.cancel = cancel
102
+ c.batch.wg.Add(1)
103
+ go c.runBatchPath(runCtx)
104
+ c.Infof("batch path worker started (interval=%s, max_conns=%d)", cfg.interval, cfg.maxConnections)
105
+ return nil
106
+}
107
+
108
+func (c *Collector) stopBatchPath() {
109
+ if c.batch.cancel != nil {
110
+ c.batch.cancel()
111
+ }
112
+ c.batch.wg.Wait()
113
+ if c.batch.client != nil {
114
+ if err := c.batch.client.Close(); err != nil {
115
+ c.Errorf("batch path: closing client failed: %v", err)
116
+ }
117
+ }
118
+ c.batch.cancel = nil
119
+ c.batch.client = nil
120
+ c.batch.config = batchPathConfig{}
121
+}
122
+
123
+func (c *Collector) runBatchPath(ctx context.Context) {
124
+ defer c.batch.wg.Done()
125
+
126
+ interval := c.batch.config.interval
127
+ if interval <= 0 {
128
+ interval = time.Minute
129
+ }
130
+
131
+ now := time.Now()
132
+ beat := now
133
+ c.runBatchCollectors(ctx, beat)
134
+ nextBeat := beat.Add(interval)
135
+
136
+ for {
137
+ sleep := time.Until(nextBeat)
138
+ if sleep > 0 {
139
+ timer := time.NewTimer(sleep)
140
+ select {
141
+ case <-ctx.Done():
142
+ timer.Stop()
143
+ return
144
+ case <-timer.C:
145
+ }
146
+ } else {
147
+ select {
148
+ case <-ctx.Done():
149
+ return
150
+ default:
151
+ }
152
+ }
153
+
154
+ beat = nextBeat
155
+ c.runBatchCollectors(ctx, beat)
156
+
157
+ nextBeat = nextBeat.Add(interval)
158
+ now = time.Now()
159
+ for nextBeat.Before(now) {
160
+ nextBeat = nextBeat.Add(interval)
161
+ }
162
+ }
163
+}
164
+
165
+func (c *Collector) runBatchCollectors(ctx context.Context, beat time.Time) {
166
+ if ctx.Err() != nil {
167
+ return
168
+ }
169
+
170
+ c.batch.cache.beginLatencyCycle(beat)
171
+
172
+ if !c.batchTotalsEnabled() {
173
+ c.batch.cache.setTotals(queueTotalsSnapshot{timestamp: beat})
174
+ return
175
+ }
176
+
177
+ snapshot, err := c.fetchQueueTotals(ctx, beat, c.batchDoQueryRow)
178
+ c.batch.cache.setTotals(snapshot)
179
+ if err != nil && !errors.Is(err, context.Canceled) {
180
+ c.logErrorOnce("batch_path_error", "batch path: %s", trimDriverMessage(err))
181
+ } else if err == nil {
182
+ c.clearErrorOnce("batch_path_error")
183
+ }
184
+}
185
+
186
+func (c *Collector) fetchQueueTotals(ctx context.Context, beat time.Time, do queryRowFunc) (queueTotalsSnapshot, error) {
187
+ snapshot := queueTotalsSnapshot{
188
+ timestamp: beat,
189
+ queues: make(map[string]int64),
190
+ items: make(map[string]int64),
191
+ }
192
+
193
+ var firstErr error
194
+
195
+ if c.CollectMessageQueueTotals.IsEnabled() {
196
+ var messageCount, queueCount int64
197
+ err := do(ctx, queryNameMessageQueueTotals, queryMessageQueueTotals, func(column, value string) {
198
+ switch column {
199
+ case "MESSAGE_COUNT":
200
+ messageCount = parseInt64OrZero(value)
201
+ case "QUEUE_COUNT":
202
+ queueCount = parseInt64OrZero(value)
203
+ }
204
+ })
205
+ if err != nil {
206
+ c.logQueryErrorOnce("batch_message_queue_totals", queryMessageQueueTotals, err)
207
+ if firstErr == nil {
208
+ firstErr = fmt.Errorf("message queue totals: %w", err)
209
+ }
210
+ } else {
211
+ c.clearErrorOnce("batch_message_queue_totals")
212
+ snapshot.queues["message_queue"] = queueCount
213
+ snapshot.items["message_queue"] = messageCount
214
+ }
215
+ }
216
+
217
+ if c.CollectJobQueueTotals.IsEnabled() {
218
+ var queueCount, jobCount int64
219
+ err := do(ctx, queryNameJobQueueTotals, queryJobQueueTotals, func(column, value string) {
220
+ switch column {
221
+ case "QUEUE_COUNT":
222
+ queueCount = parseInt64OrZero(value)
223
+ case "JOB_COUNT":
224
+ jobCount = parseInt64OrZero(value)
225
+ }
226
+ })
227
+ if err != nil {
228
+ c.logQueryErrorOnce("batch_job_queue_totals", queryJobQueueTotals, err)
229
+ if firstErr == nil {
230
+ firstErr = fmt.Errorf("job queue totals: %w", err)
231
+ }
232
+ } else {
233
+ c.clearErrorOnce("batch_job_queue_totals")
234
+ snapshot.queues["job_queue"] = queueCount
235
+ snapshot.items["job_queue"] = jobCount
236
+ }
237
+ }
238
+
239
+ if c.CollectOutputQueueTotals.IsEnabled() {
240
+ var queueCount, fileCount int64
241
+ err := do(ctx, queryNameOutputQueueTotals, queryOutputQueueTotals, func(column, value string) {
242
+ switch column {
243
+ case "QUEUE_COUNT":
244
+ queueCount = parseInt64OrZero(value)
245
+ case "FILE_COUNT":
246
+ fileCount = parseInt64OrZero(value)
247
+ }
248
+ })
249
+ if err != nil {
250
+ c.logQueryErrorOnce("batch_output_queue_totals", queryOutputQueueTotals, err)
251
+ if firstErr == nil {
252
+ firstErr = fmt.Errorf("output queue totals: %w", err)
253
+ }
254
+ } else {
255
+ c.clearErrorOnce("batch_output_queue_totals")
256
+ snapshot.queues["output_queue"] = queueCount
257
+ snapshot.items["output_queue"] = fileCount
258
+ }
259
+ }
260
+
261
+ snapshot.err = firstErr
262
+ return snapshot, snapshot.err
263
+}
264
+
265
+func (c *Collector) batchDoQueryRow(ctx context.Context, queryName, query string, assign func(column, value string)) error {
266
+ if c.batch.client == nil {
267
+ return errors.New("batch path client not initialised")
268
+ }
269
+
270
+ start := time.Now()
271
+ err := c.queryRowWithClient(ctx, c.batch.client, queryName, query, assign)
272
+ elapsed := time.Since(start)
273
+ latency := elapsed.Microseconds()
274
+ if latency == 0 {
275
+ latency = 1
276
+ }
277
+ c.batch.cache.addLatency(queryName, latency)
278
+ c.Debugf("batch recorded %s=%dµs", queryName, latency)
279
+ return err
280
+}
281
+
282
+func (c *Collector) batchPathActive() bool {
283
+ return c != nil && c.batch.config.enabled && c.batch.client != nil
284
+}
285
+
286
+func (c *Collector) batchPathIntervalSeconds() int {
287
+ if c == nil {
288
+ return 0
289
+ }
290
+ if !c.batchPathActive() {
291
+ return 0
292
+ }
293
+ interval := int(c.batch.config.interval / time.Second)
294
+ if interval < 1 {
295
+ interval = 60
296
+ }
297
+ return interval
298
+}
299
+
300
+func (c *batchCache) beginLatencyCycle(ts time.Time) {
301
+ c.latency.beginCycle(ts)
302
+}
303
+
304
+func (c *batchCache) addLatency(name string, value int64) {
305
+ c.latency.add(name, value)
306
+}
307
+
308
+func (c *batchCache) setTotals(snapshot queueTotalsSnapshot) {
309
+ c.mu.Lock()
310
+ c.totals = snapshot
311
+ c.mu.Unlock()
312
+}
313
+
314
+func (c *batchCache) getTotals() queueTotalsSnapshot {
315
+ c.mu.RLock()
316
+ defer c.mu.RUnlock()
317
+ return cloneQueueTotalsSnapshot(c.totals)
318
+}
319
+
320
+func (c *batchCache) getLatencies() (map[string]int64, time.Time) {
321
+ return c.latency.snapshot()
322
+}
323
+
324
+func cloneQueueTotalsSnapshot(src queueTotalsSnapshot) queueTotalsSnapshot {
325
+ dst := queueTotalsSnapshot{
326
+ timestamp: src.timestamp,
327
+ err: src.err,
328
+ }
329
+ if src.queues != nil {
330
+ dst.queues = make(map[string]int64, len(src.queues))
331
+ for k, v := range src.queues {
332
+ dst.queues[k] = v
333
+ }
334
+ }
335
+ if src.items != nil {
336
+ dst.items = make(map[string]int64, len(src.items))
337
+ for k, v := range src.items {
338
+ dst.items[k] = v
339
+ }
340
+ }
341
+ return dst
342
+}
src/go/plugin/ibm.d/modules/as400/cardinality.go
-2
@@ -5,8 +5,6 @@ import "context"
5
const (
6
networkInterfaceLimit = 50
7
httpServerLimit = 200
8
- messageQueueLimit = 200
9
- outputQueueLimit = 200
8
)
9
10
type cardinalityGuard struct {
src/go/plugin/ibm.d/modules/as400/collect_activejobs.go
+78
-134
@@ -8,165 +8,109 @@ package as400
8
import (
9
"context"
10
"fmt"
11
- "strconv"
11
+ "math"
12
+ "strings"
13
)
14
14
-// countActiveJobs returns the number of active jobs for cardinality check
15
-func (a *Collector) countActiveJobs(ctx context.Context) (int, error) {
16
- var count int
17
- err := a.doQueryRow(ctx, "count_active_jobs", queryCountActiveJobs, func(column, value string) {
18
- if column == "COUNT" {
19
- if v, err := strconv.Atoi(value); err == nil {
20
- count = v
21
- }
22
- }
23
- })
24
- return count, err
25
-}
26
-
27
-// collectActiveJobs collects metrics for top CPU-consuming active jobs
15
+// collectActiveJobs collects metrics for explicitly configured active jobs
16
func (a *Collector) collectActiveJobs(ctx context.Context) error {
29
- // Check if feature is enabled
30
- if !a.CollectActiveJobs.IsEnabled() {
17
+ if len(a.activeJobTargets) == 0 {
18
return nil
19
}
33
-
34
- allowed, count, err := a.activeJobsCardinality.Allow(ctx, a.countActiveJobs)
35
- if err != nil {
36
- return fmt.Errorf("failed to count active jobs: %v", err)
37
- }
38
- if !allowed {
39
- a.logOnce("active_jobs_cardinality", "active jobs (%d) exceed configured limit (%d), skipping detailed collection", count, a.MaxActiveJobs)
20
+ if !a.CollectActiveJobs.IsEnabled() {
21
return nil
22
}
42
- a.Debugf("Found %d active jobs in system", count)
23
44
- // Query for top active jobs by CPU usage
45
- query := fmt.Sprintf(queryTopActiveJobs, a.MaxActiveJobs)
24
+ var firstErr error
25
47
- var (
48
- currentJobName string
49
- currentJob *activeJobMetrics
50
- )
51
- err = a.doQuery(ctx, "top_active_jobs", query, func(column, value string, lineEnd bool) {
52
- switch column {
53
- case "JOB_NAME":
54
- currentJobName = value
55
- currentJob = a.getActiveJobMetrics(currentJobName)
56
- if currentJob != nil {
57
- currentJob.jobName = value
58
- }
59
-
60
- case "JOB_STATUS":
61
- if currentJob != nil {
62
- currentJob.jobStatus = value
63
- }
26
+ for _, target := range a.activeJobTargets {
27
+ key := target.ID()
28
+ meta := a.getActiveJobMetrics(key)
29
+ meta.qualifiedName = key
30
+ meta.jobNumber = target.Number
31
+ meta.jobUser = target.User
32
+ meta.jobName = target.Name
33
65
- case "SUBSYSTEM":
66
- if currentJob != nil {
67
- currentJob.subsystem = value
68
- }
34
+ metrics := activeJobInstanceMetrics{}
35
+ found := false
36
70
- case "JOB_TYPE":
71
- if currentJob != nil {
72
- currentJob.jobType = value
73
- }
37
+ queryName := fmt.Sprintf("active_job_%s_%s_%s", target.Number, target.User, target.Name)
38
+ query := buildActiveJobQuery(target)
39
75
- case "ELAPSED_CPU_TIME":
76
- if currentJob != nil {
77
- if v, err := strconv.ParseFloat(value, 64); err == nil {
78
- currentJob.elapsedCPUTime = int64(v)
79
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
80
- m.ElapsedCPUTime = int64(v)
81
- a.mx.activeJobs[currentJobName] = m
82
- } else {
83
- a.mx.activeJobs[currentJobName] = activeJobInstanceMetrics{
84
- ElapsedCPUTime: int64(v),
85
- }
86
- }
40
+ err := a.doQuery(ctx, queryName, query, func(column, value string, lineEnd bool) {
41
+ switch column {
42
+ case "JOB_NAME":
43
+ qualified := strings.TrimSpace(value)
44
+ if qualified != "" {
45
+ meta.qualifiedName = qualified
46
}
88
- }
89
-
90
- case "ELAPSED_TIME":
91
- if currentJob != nil {
92
- if v, err := strconv.ParseFloat(value, 64); err == nil {
93
- currentJob.elapsedTime = int64(v)
94
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
95
- m.ElapsedTime = int64(v)
96
- a.mx.activeJobs[currentJobName] = m
97
- }
47
+ case "JOB_USER":
48
+ user := strings.TrimSpace(value)
49
+ if user != "" {
50
+ meta.jobUser = strings.ToUpper(user)
51
}
99
- }
100
-
101
- case "TEMPORARY_STORAGE":
102
- if currentJob != nil {
103
- if v, err := strconv.ParseInt(value, 10, 64); err == nil {
104
- // Convert KB to MB
105
- vMB := v / 1024
106
- currentJob.temporaryStorage = vMB
107
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
108
- m.TemporaryStorage = vMB
109
- a.mx.activeJobs[currentJobName] = m
110
- }
52
+ case "JOB_NUMBER":
53
+ number := strings.TrimSpace(value)
54
+ if number != "" {
55
+ meta.jobNumber = number
56
}
112
- }
113
-
114
- case "CPU_PERCENTAGE":
115
- if currentJob != nil {
116
- if v, err := strconv.ParseFloat(value, 64); err == nil {
117
- currentJob.cpuPercentage = int64(v * precision)
118
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
119
- m.CPUPercentage = int64(v * precision)
120
- a.mx.activeJobs[currentJobName] = m
121
- }
57
+ case "JOB_STATUS":
58
+ meta.jobStatus = strings.TrimSpace(value)
59
+ case "SUBSYSTEM":
60
+ meta.subsystem = strings.TrimSpace(value)
61
+ case "JOB_TYPE":
62
+ meta.jobType = strings.TrimSpace(value)
63
+ case "ELAPSED_CPU_TIME":
64
+ if v, ok := a.parseInt64Value(value, 1); ok {
65
+ metrics.ElapsedCPUTime = v
66
}
123
- }
124
-
125
- case "ELAPSED_INTERACTIVE_TRANSACTIONS":
126
- if currentJob != nil {
127
- if v, err := strconv.ParseInt(value, 10, 64); err == nil {
128
- currentJob.interactiveTransactions = v
129
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
130
- m.ElapsedInteractiveTransactions = v
131
- a.mx.activeJobs[currentJobName] = m
132
- }
67
+ case "ELAPSED_TIME":
68
+ if v, ok := a.parseInt64Value(value, 1); ok {
69
+ metrics.ElapsedTime = v
70
}
134
- }
135
-
136
- case "ELAPSED_TOTAL_DISK_IO_COUNT":
137
- if currentJob != nil {
138
- if v, err := strconv.ParseInt(value, 10, 64); err == nil {
139
- currentJob.diskIO = v
140
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
141
- m.ElapsedDiskIO = v
142
- a.mx.activeJobs[currentJobName] = m
143
- }
71
+ case "TEMPORARY_STORAGE":
72
+ if v, ok := a.parseInt64Value(value, 1); ok {
73
+ metrics.TemporaryStorage = v / 1024 // Convert KB to MB
74
+ }
75
+ case "CPU_PERCENTAGE":
76
+ if f, ok := a.parseFloat64Value(value); ok {
77
+ metrics.CPUPercentage = int64(math.Round(f * float64(precision)))
78
+ }
79
+ case "ELAPSED_INTERACTIVE_TRANSACTIONS":
80
+ if v, ok := a.parseInt64Value(value, 1); ok {
81
+ metrics.ElapsedInteractiveTransactions = v
82
+ }
83
+ case "ELAPSED_TOTAL_DISK_IO_COUNT":
84
+ if v, ok := a.parseInt64Value(value, 1); ok {
85
+ metrics.ElapsedDiskIO = v
86
+ }
87
+ case "THREAD_COUNT":
88
+ if v, ok := a.parseInt64Value(value, 1); ok {
89
+ metrics.ThreadCount = v
90
}
91
}
92
147
- case "THREAD_COUNT":
148
- if currentJob != nil {
149
- if v, err := strconv.ParseInt(value, 10, 64); err == nil {
150
- currentJob.threadCount = v
151
- if m, ok := a.mx.activeJobs[currentJobName]; ok {
152
- m.ThreadCount = v
153
- a.mx.activeJobs[currentJobName] = m
154
- }
155
- }
93
+ if lineEnd {
94
+ found = true
95
}
96
+ })
97
158
- case "RUN_PRIORITY":
159
- if currentJob != nil {
160
- if v, err := strconv.ParseInt(value, 10, 64); err == nil {
161
- currentJob.runPriority = v
162
- }
98
+ if err != nil {
99
+ if firstErr == nil {
100
+ firstErr = fmt.Errorf("active job %s: %w", key, err)
101
}
102
+ continue
103
+ }
104
+
105
+ if !found {
106
+ meta.jobStatus = "NOT FOUND"
107
+ meta.subsystem = ""
108
+ meta.jobType = ""
109
+ metrics = activeJobInstanceMetrics{}
110
}
165
- })
111
167
- if err != nil {
168
- return err
112
+ a.mx.activeJobs[key] = metrics
113
}
114
171
- return nil
115
+ return firstErr
116
}
src/go/plugin/ibm.d/modules/as400/collect_data.go
+257
-226
@@ -98,6 +98,27 @@ func (a *Collector) computeEntitledCPUPercentage(cpuUtilization float64) int64 {
98
return int64(math.Round(entitled * float64(precision)))
99
}
100
101
+func (a *Collector) applyCPUUtilization(method string, cpuUtilization float64) {
102
+ adjusted := cpuUtilization
103
+ if adjusted < 0 {
104
+ a.Warningf("CPU collection (%s): interval utilization negative (%.2f%%), clamping to 0", method, adjusted/float64(precision))
105
+ adjusted = 0
106
+ }
107
+ if cpus := a.mx.ConfiguredCPUs; cpus > 0 {
108
+ maxAllowed := float64(cpus) * 100.0 * float64(precision)
109
+ if adjusted > maxAllowed {
110
+ a.Warningf("CPU collection (%s): interval utilization (%.2f%%) exceeds configured capacity (%d CPUs), clamping to %.2f%%",
111
+ method, adjusted/float64(precision), cpus, maxAllowed/float64(precision))
112
+ adjusted = maxAllowed
113
+ }
114
+ }
115
+ value := int64(math.Round(adjusted))
116
+ a.mx.systemActivity.AverageCPUUtilization = value
117
+ a.mx.systemActivity.AverageCPURate = value
118
+ a.mx.CPUPercentage = value
119
+ a.mx.EntitledCPUPercentage = a.computeEntitledCPUPercentage(adjusted)
120
+}
121
+
122
func (a *Collector) parseFloat64Value(value string) (float64, bool) {
123
cleaned := cleanNumericString(value)
124
if cleaned == "" || cleaned == "-" || cleaned == "." || cleaned == "+" {
@@ -220,9 +241,6 @@ func (a *Collector) collect(ctx context.Context) error {
241
}
242
243
func (a *Collector) recordQueryLatency(queryName string, duration time.Duration) {
223
- if a.mx == nil {
224
- return
225
- }
244
if queryName == "" {
245
queryName = "unknown_query"
246
}
@@ -232,16 +250,15 @@ func (a *Collector) recordQueryLatency(queryName string, duration time.Duration)
250
sanitized = "unknown_query"
251
}
252
235
- if a.mx.queryLatencies == nil {
236
- a.mx.queryLatencies = make(map[string]int64)
237
- }
238
-
253
latency := duration.Microseconds()
254
if latency == 0 && duration > 0 {
255
latency = 1
256
}
257
244
- a.mx.queryLatencies[sanitized] += latency
258
+ if a.fastQueryLatencyCounters == nil {
259
+ a.fastQueryLatencyCounters = make(map[string]int64)
260
+ }
261
+ a.fastQueryLatencyCounters[sanitized] += latency
262
}
263
264
func (a *Collector) collectSystemStatus(ctx context.Context) error {
@@ -428,137 +445,172 @@ func (a *Collector) collectJobInfo(ctx context.Context) error {
445
}
446
447
func (a *Collector) collectMessageQueues(ctx context.Context) error {
431
- if !a.CollectMessageQueueMetrics.IsEnabled() {
448
+ if len(a.messageQueueTargets) == 0 {
449
return nil
450
}
451
435
- allowed, count, err := a.messageQueuesCardinality.Allow(ctx, a.countMessageQueues)
436
- if err != nil {
437
- return fmt.Errorf("failed to count message queues: %w", err)
438
- }
439
- if !allowed {
440
- limit := a.MaxMessageQueues
441
- if limit <= 0 {
442
- limit = messageQueueLimit
452
+ if a.slowPathActive() {
453
+ snapshot := a.slow.cache.getMessageQueues()
454
+ for _, target := range a.messageQueueTargets {
455
+ key := target.ID()
456
+ meta := a.getMessageQueueMetrics(key)
457
+ *meta = messageQueueMetrics{
458
+ library: target.Library,
459
+ name: target.Name,
460
+ }
461
+ if snapshotMeta, ok := snapshot.meta[key]; ok {
462
+ if snapshotMeta.library != "" {
463
+ meta.library = snapshotMeta.library
464
+ }
465
+ if snapshotMeta.name != "" {
466
+ meta.name = snapshotMeta.name
467
+ }
468
+ }
469
+ a.messageQueues[key] = meta
470
+ a.mx.messageQueues[key] = snapshot.metrics[key]
471
}
444
- a.logOnce("message_queue_cardinality", "message queue count (%d) exceeds limit (%d), skipping collection", count, limit)
445
- return nil
472
+ return snapshot.err
473
}
474
448
- limit := a.MaxMessageQueues
449
- if limit <= 0 {
450
- limit = messageQueueLimit
451
- }
452
- query := fmt.Sprintf(queryMessageQueueAggregates, limit)
475
+ var firstErr error
476
454
- var (
455
- library string
456
- queue string
457
- metrics messageQueueInstanceMetrics
458
- )
477
+ for _, target := range a.messageQueueTargets {
478
+ key := target.ID()
479
+ meta := a.getMessageQueueMetrics(key)
480
+ meta.library = target.Library
481
+ meta.name = target.Name
482
+ a.messageQueues[key] = meta
483
460
- return a.doQuery(ctx, "message_queue_aggregates", query, func(column, value string, lineEnd bool) {
461
- switch column {
462
- case "MESSAGE_QUEUE_LIBRARY":
463
- library = normalizeValue(value)
464
- case "MESSAGE_QUEUE_NAME":
465
- queue = normalizeValue(value)
466
- case "MESSAGE_COUNT":
467
- metrics.Total = parseInt64OrZero(value)
468
- case "INFORMATIONAL_MESSAGES":
469
- metrics.Informational = parseInt64OrZero(value)
470
- case "INQUIRY_MESSAGES":
471
- metrics.Inquiry = parseInt64OrZero(value)
472
- case "DIAGNOSTIC_MESSAGES":
473
- metrics.Diagnostic = parseInt64OrZero(value)
474
- case "ESCAPE_MESSAGES":
475
- metrics.Escape = parseInt64OrZero(value)
476
- case "NOTIFY_MESSAGES":
477
- metrics.Notify = parseInt64OrZero(value)
478
- case "SENDER_COPY_MESSAGES":
479
- metrics.SenderCopy = parseInt64OrZero(value)
480
- case "MAX_SEVERITY":
481
- metrics.MaxSeverity = parseInt64OrZero(value)
484
+ metrics := messageQueueInstanceMetrics{}
485
+ found := false
486
+
487
+ queryName := fmt.Sprintf("message_queue_%s_%s", target.Library, target.Name)
488
+ query := buildMessageQueueQuery(target, a.supportsMessageQueueTableFunction())
489
+
490
+ err := a.doQuery(ctx, queryName, query, func(column, value string, lineEnd bool) {
491
+ switch column {
492
+ case "MESSAGE_COUNT":
493
+ metrics.Total = parseInt64OrZero(value)
494
+ case "INFORMATIONAL_MESSAGES":
495
+ metrics.Informational = parseInt64OrZero(value)
496
+ case "INQUIRY_MESSAGES":
497
+ metrics.Inquiry = parseInt64OrZero(value)
498
+ case "DIAGNOSTIC_MESSAGES":
499
+ metrics.Diagnostic = parseInt64OrZero(value)
500
+ case "ESCAPE_MESSAGES":
501
+ metrics.Escape = parseInt64OrZero(value)
502
+ case "NOTIFY_MESSAGES":
503
+ metrics.Notify = parseInt64OrZero(value)
504
+ case "SENDER_COPY_MESSAGES":
505
+ metrics.SenderCopy = parseInt64OrZero(value)
506
+ case "MAX_SEVERITY":
507
+ metrics.MaxSeverity = parseInt64OrZero(value)
508
+ }
509
+
510
+ if lineEnd {
511
+ found = true
512
+ }
513
+ })
514
+
515
+ if err != nil {
516
+ if firstErr == nil {
517
+ firstErr = fmt.Errorf("message queue %s: %w", key, err)
518
+ }
519
+ continue
520
}
521
484
- if lineEnd {
485
- if queue != "" {
486
- key := library + "/" + queue
487
- meta := a.getMessageQueueMetrics(key)
488
- meta.library = library
489
- meta.name = queue
490
- a.messageQueues[key] = meta
491
- a.mx.messageQueues[key] = metrics
492
- }
493
- library = ""
494
- queue = ""
522
+ if !found {
523
metrics = messageQueueInstanceMetrics{}
524
}
497
- })
525
+
526
+ a.mx.messageQueues[key] = metrics
527
+ }
528
+
529
+ return firstErr
530
}
531
532
func (a *Collector) collectOutputQueues(ctx context.Context) error {
501
- if !a.CollectOutputQueueMetrics.IsEnabled() {
533
+ if len(a.outputQueueTargets) == 0 {
534
return nil
535
}
536
505
- allowed, count, err := a.outputQueuesCardinality.Allow(ctx, a.countOutputQueues)
506
- if err != nil {
507
- return fmt.Errorf("failed to count output queues: %w", err)
508
- }
509
- if !allowed {
510
- limit := a.MaxOutputQueues
511
- if limit <= 0 {
512
- limit = outputQueueLimit
537
+ if a.slowPathActive() {
538
+ snapshot := a.slow.cache.getOutputQueues()
539
+ for _, target := range a.outputQueueTargets {
540
+ key := target.ID()
541
+ metaPtr := a.getOutputQueueMetrics(key)
542
+ defaultMeta := outputQueueMetrics{
543
+ library: target.Library,
544
+ name: target.Name,
545
+ status: "UNKNOWN",
546
+ }
547
+ if snapshotMeta, ok := snapshot.meta[key]; ok {
548
+ *metaPtr = snapshotMeta
549
+ } else {
550
+ *metaPtr = defaultMeta
551
+ }
552
+ a.outputQueues[key] = metaPtr
553
+ a.mx.outputQueues[key] = snapshot.metrics[key]
554
}
514
- a.logOnce("output_queue_cardinality", "output queue count (%d) exceeds limit (%d), skipping collection", count, limit)
515
- return nil
555
+ return snapshot.err
556
}
557
518
- limit := a.MaxOutputQueues
519
- if limit <= 0 {
520
- limit = outputQueueLimit
521
- }
522
- query := fmt.Sprintf(queryOutputQueueInfo, limit)
558
+ var firstErr error
559
524
- var (
525
- library string
526
- queue string
527
- status string
528
- metrics outputQueueInstanceMetrics
529
- )
560
+ for _, target := range a.outputQueueTargets {
561
+ key := target.ID()
562
+ meta := a.getOutputQueueMetrics(key)
563
+ meta.library = target.Library
564
+ meta.name = target.Name
565
+ meta.status = "UNKNOWN"
566
+ a.outputQueues[key] = meta
567
531
- return a.doQuery(ctx, "output_queue_info", query, func(column, value string, lineEnd bool) {
532
- switch column {
533
- case "OUTPUT_QUEUE_LIBRARY_NAME":
534
- library = normalizeValue(value)
535
- case "OUTPUT_QUEUE_NAME":
536
- queue = normalizeValue(value)
537
- case "OUTPUT_QUEUE_STATUS":
538
- status = normalizeValue(value)
539
- case "NUMBER_OF_FILES":
540
- metrics.Files = parseInt64OrZero(value)
541
- case "NUMBER_OF_WRITERS":
542
- metrics.Writers = parseInt64OrZero(value)
568
+ metrics := outputQueueInstanceMetrics{}
569
+ entriesCount := int64(0)
570
+ entriesUsed := false
571
+
572
+ queryName := fmt.Sprintf("output_queue_%s_%s", target.Library, target.Name)
573
+ err := a.doQuery(ctx, queryName, buildOutputQueueEntriesQuery(target), func(column, value string, lineEnd bool) {
574
+ if lineEnd {
575
+ entriesCount++
576
+ }
577
+ })
578
+ if err != nil {
579
+ if isSQLFeatureError(err) {
580
+ a.Debugf("output queue entries function unavailable for %s/%s, falling back to view", target.Library, target.Name)
581
+ } else if firstErr == nil {
582
+ firstErr = fmt.Errorf("output queue %s (entries): %w", key, err)
583
+ }
584
+ } else {
585
+ entriesUsed = true
586
+ metrics.Files = entriesCount
587
}
588
545
- if lineEnd {
546
- if queue != "" {
547
- key := library + "/" + queue
548
- meta := a.getOutputQueueMetrics(key)
549
- meta.library = library
550
- meta.name = queue
551
- meta.status = status
552
- a.outputQueues[key] = meta
553
- metrics.Released = boolToInt(strings.EqualFold(status, "RELEASED"))
554
- a.mx.outputQueues[key] = metrics
555
- }
556
- library = ""
557
- queue = ""
558
- status = ""
559
- metrics = outputQueueInstanceMetrics{}
589
+ viewErr := a.doQuery(ctx, queryName+"_view", buildOutputQueueInfoQuery(target), func(column, value string, lineEnd bool) {
590
+ switch column {
591
+ case "OUTPUT_QUEUE_STATUS":
592
+ meta.status = strings.TrimSpace(value)
593
+ case "NUMBER_OF_WRITERS":
594
+ metrics.Writers = parseInt64OrZero(value)
595
+ case "NUMBER_OF_FILES":
596
+ if !entriesUsed {
597
+ metrics.Files = parseInt64OrZero(value)
598
+ }
599
+ }
600
+ })
601
+ if viewErr != nil {
602
+ if firstErr == nil {
603
+ firstErr = fmt.Errorf("output queue %s (info): %w", key, viewErr)
604
+ }
605
+ continue
606
}
561
- })
607
+
608
+ metrics.Released = boolToInt(strings.EqualFold(meta.status, "RELEASED"))
609
+ a.outputQueues[key] = meta
610
+ a.mx.outputQueues[key] = metrics
611
+ }
612
+
613
+ return firstErr
614
}
615
616
func (a *Collector) doQuery(ctx context.Context, queryName, query string, assign func(column, value string, lineEnd bool)) error {
@@ -941,30 +993,6 @@ func (a *Collector) countNetworkInterfaces(ctx context.Context) (int, error) {
993
return count, err
994
}
995
944
-func (a *Collector) countMessageQueues(ctx context.Context) (int, error) {
945
- var count int
946
- err := a.doQueryRow(ctx, "count_message_queues", queryCountMessageQueues, func(column, value string) {
947
- if column == "COUNT" {
948
- if v, ok := a.parseInt64Value(value, 1); ok {
949
- count = int(v)
950
- }
951
- }
952
- })
953
- return count, err
954
-}
955
-
956
-func (a *Collector) countOutputQueues(ctx context.Context) (int, error) {
957
- var count int
958
- err := a.doQueryRow(ctx, "count_output_queues", queryCountOutputQueues, func(column, value string) {
959
- if column == "COUNT" {
960
- if v, ok := a.parseInt64Value(value, 1); ok {
961
- count = int(v)
962
- }
963
- }
964
- })
965
- return count, err
966
-}
967
-
996
func (a *Collector) countHTTPServers(ctx context.Context) (int, error) {
997
var count int64
998
err := a.doQueryRow(ctx, "count_http_servers", queryCountHTTPServers, func(column, value string) {
@@ -1001,18 +1029,6 @@ func (a *Collector) countSubsystems(ctx context.Context) (int, error) {
1029
return int(count), err
1030
}
1031
1004
-func (a *Collector) countJobQueues(ctx context.Context) (int, error) {
1005
- var count int64
1006
- err := a.doQueryRow(ctx, "count_job_queues", queryCountJobQueues, func(column, value string) {
1007
- if column == "COUNT" {
1008
- if v, ok := a.parseInt64Value(value, 1); ok {
1009
- count = v
1010
- }
1011
- }
1012
- })
1013
- return int(count), err
1014
-}
1015
-
1032
// Temporary storage collection
1033
func (a *Collector) collectTempStorage(ctx context.Context) error {
1034
// Collect total temp storage
@@ -1068,6 +1084,19 @@ func (a *Collector) collectTempStorage(ctx context.Context) error {
1084
1085
// Subsystems collection
1086
func (a *Collector) collectSubsystems(ctx context.Context) error {
1087
+ if a.slowPathActive() {
1088
+ snapshot := a.slow.cache.getSubsystems()
1089
+ for key, meta := range snapshot.meta {
1090
+ ptr := a.getSubsystemMetrics(key)
1091
+ *ptr = meta
1092
+ a.subsystems[key] = ptr
1093
+ }
1094
+ for key, metrics := range snapshot.metrics {
1095
+ a.mx.subsystems[key] = metrics
1096
+ }
1097
+ return snapshot.err
1098
+ }
1099
+
1100
query := querySubsystems
1101
if a.MaxSubsystems > 0 {
1102
if total, err := a.countSubsystems(ctx); err != nil {
@@ -1136,61 +1165,79 @@ func (a *Collector) collectSubsystems(ctx context.Context) error {
1165
1166
// Job queues collection
1167
func (a *Collector) collectJobQueues(ctx context.Context) error {
1139
- query := queryJobQueues
1140
- if a.MaxJobQueues > 0 {
1141
- if total, err := a.countJobQueues(ctx); err != nil {
1142
- a.logOnce("job_queue_count_failed", "failed to count job queues before applying limit: %v", err)
1143
- } else if total > a.MaxJobQueues {
1144
- a.logOnce("job_queue_limit", "job queue count (%d) exceeds limit (%d); truncating results", total, a.MaxJobQueues)
1145
- }
1146
- query = withFetchLimit(query, a.MaxJobQueues)
1168
+ if len(a.jobQueueTargets) == 0 {
1169
+ return nil
1170
}
1171
1149
- var currentQueue string
1150
- return a.doQuery(ctx, "job_queues", query, func(column, value string, lineEnd bool) {
1151
- switch column {
1152
- case "QUEUE_NAME":
1153
- name := strings.TrimSpace(value)
1154
- if name == "" {
1155
- currentQueue = ""
1156
- return
1172
+ if a.slowPathActive() {
1173
+ snapshot := a.slow.cache.getJobQueues()
1174
+ for _, target := range a.jobQueueTargets {
1175
+ key := target.ID()
1176
+ metaPtr := a.getJobQueueMetrics(key)
1177
+ defaultMeta := jobQueueMetrics{
1178
+ library: target.Library,
1179
+ name: target.Name,
1180
+ status: "NOT_FOUND",
1181
}
1158
- if a.jobQueueSelector != nil && !a.jobQueueSelector.MatchString(name) {
1159
- currentQueue = ""
1160
- return
1161
- }
1162
- currentQueue = name
1163
- queue := a.getJobQueueMetrics(currentQueue)
1164
- parts := strings.SplitN(name, "/", 2)
1165
- if len(parts) == 2 {
1166
- queue.library = parts[0]
1167
- queue.name = parts[1]
1182
+ if snapshotMeta, ok := snapshot.meta[key]; ok {
1183
+ *metaPtr = snapshotMeta
1184
} else {
1169
- queue.name = name
1170
- queue.library = ""
1185
+ *metaPtr = defaultMeta
1186
}
1172
- queue.status = "RELEASED"
1187
+ a.jobQueues[key] = metaPtr
1188
+ a.mx.jobQueues[key] = snapshot.metrics[key]
1189
+ }
1190
+ return snapshot.err
1191
+ }
1192
1174
- case "NUMBER_OF_JOBS":
1175
- if currentQueue != "" && a.jobQueues[currentQueue] != nil {
1176
- if v, ok := a.parseInt64Value(value, 1); ok {
1177
- if m, ok := a.mx.jobQueues[currentQueue]; ok {
1178
- m.NumberOfJobs = v
1179
- a.mx.jobQueues[currentQueue] = m
1180
- } else {
1181
- a.mx.jobQueues[currentQueue] = jobQueueInstanceMetrics{
1182
- NumberOfJobs: v,
1183
- }
1184
- }
1185
- }
1193
+ var firstErr error
1194
+
1195
+ for _, target := range a.jobQueueTargets {
1196
+ key := target.ID()
1197
+ queue := a.getJobQueueMetrics(key)
1198
+ queue.library = target.Library
1199
+ queue.name = target.Name
1200
+ queue.status = "UNKNOWN"
1201
+ a.jobQueues[key] = queue
1202
+
1203
+ metrics := jobQueueInstanceMetrics{}
1204
+ found := false
1205
+
1206
+ queryName := fmt.Sprintf("job_queue_%s_%s", target.Library, target.Name)
1207
+ err := a.doQuery(ctx, queryName, buildJobQueueQuery(target), func(column, value string, lineEnd bool) {
1208
+ switch column {
1209
+ case "JOB_QUEUE_STATUS":
1210
+ queue.status = strings.TrimSpace(value)
1211
+ case "NUMBER_OF_JOBS":
1212
+ metrics.NumberOfJobs = parseInt64OrZero(value)
1213
+ case "RELEASED_JOBS":
1214
+ queue.jobsWaiting = parseInt64OrZero(value)
1215
+ case "SCHEDULED_JOBS":
1216
+ queue.jobsScheduled = parseInt64OrZero(value)
1217
+ case "HELD_JOBS":
1218
+ queue.jobsHeld = parseInt64OrZero(value)
1219
+ }
1220
+
1221
+ if lineEnd {
1222
+ found = true
1223
+ }
1224
+ })
1225
+
1226
+ if err != nil {
1227
+ if firstErr == nil {
1228
+ firstErr = fmt.Errorf("job queue %s: %w", key, err)
1229
}
1187
- // Note: HELD_JOB_COUNT column removed - it doesn't exist in JOB_QUEUE_INFO table
1230
+ continue
1231
}
1232
1190
- if lineEnd {
1191
- currentQueue = ""
1233
+ if !found {
1234
+ queue.status = "NOT_FOUND"
1235
}
1193
- })
1236
+
1237
+ a.mx.jobQueues[key] = metrics
1238
+ }
1239
+
1240
+ return firstErr
1241
}
1242
1243
// Enhanced disk collection with all metrics
@@ -1611,6 +1658,19 @@ func (a *Collector) collectPlanCache(ctx context.Context) error {
1658
return nil
1659
}
1660
1661
+ if a.slowPathActive() {
1662
+ snapshot := a.slow.cache.getPlanCache()
1663
+ for key, meta := range snapshot.meta {
1664
+ if ptr := a.getPlanCacheMetrics(key, meta.heading); ptr != nil {
1665
+ a.planCache[key] = ptr
1666
+ }
1667
+ }
1668
+ for key, values := range snapshot.values {
1669
+ a.mx.planCache[key] = values
1670
+ }
1671
+ return snapshot.err
1672
+ }
1673
+
1674
start := time.Now()
1675
if err := a.client.Exec(ctx, callAnalyzePlanCache); err != nil {
1676
return fmt.Errorf("failed to analyze plan cache: %w", err)
@@ -1715,23 +1775,8 @@ func (a *Collector) collectSystemActivity(ctx context.Context) error {
1775
if a.UpdateEvery > 0 {
1776
deltaSeconds := float64(deltaNanos) / 1e9
1777
intervalSeconds := float64(a.UpdateEvery)
1718
-
1719
- // TOTAL_CPU_TIME is naturally in per-core scale - do NOT divide by ConfiguredCPUs
1720
- cpuUtilization := (deltaSeconds / intervalSeconds) * 100.0 * precision
1721
- maxAllowed := float64(a.mx.ConfiguredCPUs) * 100.0 * precision
1722
- if cpuUtilization >= 0 && (a.mx.ConfiguredCPUs <= 0 || cpuUtilization <= maxAllowed) {
1723
- a.mx.systemActivity.AverageCPUUtilization = int64(cpuUtilization)
1724
- a.mx.systemActivity.AverageCPURate = int64(cpuUtilization)
1725
- a.mx.CPUPercentage = int64(cpuUtilization)
1726
- a.mx.EntitledCPUPercentage = a.computeEntitledCPUPercentage(cpuUtilization)
1727
- } else {
1728
- if cpuUtilization < 0 {
1729
- a.Warningf("CPU collection: calculated utilization negative (%.2f%%), skipping this sample", cpuUtilization/precision)
1730
- } else {
1731
- a.Warningf("CPU collection: calculated utilization (%.2f%%) exceeds configured capacity (%d CPUs), skipping this sample",
1732
- cpuUtilization/precision, a.mx.ConfiguredCPUs)
1733
- }
1734
- }
1778
+ cpuUtilization := (deltaSeconds / intervalSeconds) * 100.0 * float64(precision)
1779
+ a.applyCPUUtilization("TOTAL_CPU_TIME", cpuUtilization)
1780
}
1781
} else {
1782
a.Debugf("CPU collection: establishing baseline for TOTAL_CPU_TIME method")
@@ -1771,21 +1816,7 @@ func (a *Collector) collectSystemActivity(ctx context.Context) error {
1816
if deltaTime > 0 {
1817
// ELAPSED_CPU_USED is already in per-core scaling
1818
intervalCPU := float64(deltaProduct) / float64(deltaTime)
1774
- cpuUtilization := intervalCPU
1775
- maxAllowed := float64(a.mx.ConfiguredCPUs) * 100.0 * precision
1776
- if cpuUtilization >= 0 && (a.mx.ConfiguredCPUs <= 0 || cpuUtilization <= maxAllowed) {
1777
- a.mx.systemActivity.AverageCPUUtilization = int64(cpuUtilization)
1778
- a.mx.systemActivity.AverageCPURate = int64(cpuUtilization)
1779
- a.mx.CPUPercentage = int64(cpuUtilization)
1780
- a.mx.EntitledCPUPercentage = a.computeEntitledCPUPercentage(cpuUtilization)
1781
- } else {
1782
- if cpuUtilization < 0 {
1783
- a.Warningf("CPU collection: interval utilization negative (%.2f%%), skipping this sample", cpuUtilization/precision)
1784
- } else {
1785
- a.Warningf("CPU collection: interval utilization (%.2f%%) exceeds configured capacity (%d CPUs), skipping this sample",
1786
- cpuUtilization/precision, a.mx.ConfiguredCPUs)
1787
- }
1788
- }
1819
+ a.applyCPUUtilization("ELAPSED_CPU_USED", intervalCPU)
1820
}
1821
} else {
1822
a.Debugf("CPU collection: re-establishing baseline after reset")
src/go/plugin/ibm.d/modules/as400/collector.go
+295
-66
@@ -11,6 +11,7 @@ import (
11
"fmt"
12
"strings"
13
"sync"
14
+ "time"
15
16
"github.com/netdata/netdata/go/plugins/pkg/matcher"
17
"github.com/netdata/netdata/go/plugins/pkg/stm"
@@ -31,6 +32,10 @@ type Collector struct {
32
// Per-iteration metrics
33
mx *metricsData
34
35
+ fastQueryLatencyCounters map[string]int64
36
+ batchLatencyValues contexts.ObservabilityQueryLatencyBatchValues
37
+ batchLatencyValid bool
38
+
39
// Metadata caches (reset every iteration)
40
disks map[string]*diskMetrics
41
subsystems map[string]*subsystemMetrics
@@ -46,7 +51,22 @@ type Collector struct {
51
// Selectors
52
diskSelector matcher.Matcher
53
subsystemSelector matcher.Matcher
49
- jobQueueSelector matcher.Matcher
54
+
55
+ slow struct {
56
+ client *as400proto.Client
57
+ cancel context.CancelFunc
58
+ wg sync.WaitGroup
59
+ config slowPathConfig
60
+ cache slowCache
61
+ }
62
+
63
+ batch struct {
64
+ client *as400proto.Client
65
+ cancel context.CancelFunc
66
+ wg sync.WaitGroup
67
+ config batchPathConfig
68
+ cache batchCache
69
+ }
70
71
// System identity
72
systemName string
@@ -59,19 +79,23 @@ type Collector struct {
79
versionMod int
80
81
// Feature flags and logging guards
62
- disabled map[string]bool
82
+ disabled map[string]bool
83
+ errorLogged map[string]bool
84
+ muErrorLog sync.Mutex
85
86
// Cardinality guards to avoid repeated expensive counts
87
diskCardinality cardinalityGuard
66
- activeJobsCardinality cardinalityGuard
88
networkInterfacesCardinality cardinalityGuard
89
httpServersCardinality cardinalityGuard
69
- messageQueuesCardinality cardinalityGuard
70
- outputQueuesCardinality cardinalityGuard
90
91
dump *dumpContext
92
groups []collectionGroup
93
94
+ messageQueueTargets []queueTarget
95
+ jobQueueTargets []queueTarget
96
+ outputQueueTargets []queueTarget
97
+ activeJobTargets []activeJobTarget
98
+
99
// CPU collection state for delta-based calculation
100
cpuCollectionMethod string // "total_cpu_time" or "elapsed_cpu_used"
101
prevTotalCPUTime int64 // Previous TOTAL_CPU_TIME value (nanoseconds)
@@ -85,6 +109,7 @@ type Collector struct {
109
func (c *Collector) initOnce() {
110
c.once.Do(func() {
111
c.disabled = make(map[string]bool)
112
+ c.errorLogged = make(map[string]bool)
113
c.mx = &metricsData{}
114
c.resetInstanceCaches()
115
c.initGroups()
@@ -112,7 +137,6 @@ func (c *Collector) resetInstanceCaches() {
137
c.mx.networkInterfaces = make(map[string]networkInterfaceInstanceMetrics)
138
c.mx.httpServers = make(map[string]httpServerInstanceMetrics)
139
c.mx.planCache = make(map[string]planCacheInstanceMetrics)
115
- c.mx.queryLatencies = make(map[string]int64)
140
}
141
142
func (c *Collector) prepareIterationState() {
@@ -121,11 +145,8 @@ func (c *Collector) prepareIterationState() {
145
}
146
c.resetInstanceCaches()
147
c.diskCardinality.Configure(c.MaxDisks)
124
- c.activeJobsCardinality.Configure(c.MaxActiveJobs)
148
c.networkInterfacesCardinality.Configure(networkInterfaceLimit)
149
c.httpServersCardinality.Configure(httpServerLimit)
127
- c.messageQueuesCardinality.Configure(c.MaxMessageQueues)
128
- c.outputQueuesCardinality.Configure(c.MaxOutputQueues)
150
}
151
152
func (c *Collector) initGroups() {
@@ -179,6 +200,7 @@ func (c *Collector) CollectOnce() error {
200
c.exportJobQueueMetrics()
201
c.exportMessageQueueMetrics()
202
c.exportOutputQueueMetrics()
203
+ c.exportQueueTotalsMetrics()
204
c.exportTempStorageMetrics()
205
c.exportActiveJobMetrics()
206
c.exportNetworkInterfaceMetrics()
@@ -548,6 +570,9 @@ func (c *Collector) exportSubsystemMetrics() {
570
Library: library,
571
Status: status,
572
}
573
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
574
+ contexts.Subsystem.Jobs.SetUpdateEvery(c.State, labels, interval)
575
+ }
576
contexts.Subsystem.Jobs.Set(c.State, labels, contexts.SubsystemJobsValues{
577
Active: values.CurrentActiveJobs,
578
Maximum: values.MaximumActiveJobs,
@@ -575,6 +600,9 @@ func (c *Collector) exportJobQueueMetrics() {
600
Library: library,
601
Status: status,
602
}
603
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
604
+ contexts.JobQueue.Length.SetUpdateEvery(c.State, labels, interval)
605
+ }
606
contexts.JobQueue.Length.Set(c.State, labels, contexts.JobQueueLengthValues{
607
Jobs: values.NumberOfJobs,
608
})
@@ -606,6 +634,10 @@ func (c *Collector) exportMessageQueueMetrics() {
634
Library: library,
635
Queue: queue,
636
}
637
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
638
+ contexts.MessageQueue.Messages.SetUpdateEvery(c.State, labels, interval)
639
+ contexts.MessageQueue.Severity.SetUpdateEvery(c.State, labels, interval)
640
+ }
641
contexts.MessageQueue.Messages.Set(c.State, labels, contexts.MessageQueueMessagesValues{
642
Total: values.Total,
643
Informational: values.Informational,
@@ -639,6 +671,11 @@ func (c *Collector) exportOutputQueueMetrics() {
671
Queue: queue,
672
Status: status,
673
}
674
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
675
+ contexts.OutputQueue.Files.SetUpdateEvery(c.State, labels, interval)
676
+ contexts.OutputQueue.Writers.SetUpdateEvery(c.State, labels, interval)
677
+ contexts.OutputQueue.Status.SetUpdateEvery(c.State, labels, interval)
678
+ }
679
contexts.OutputQueue.Files.Set(c.State, labels, contexts.OutputQueueFilesValues{
680
Files: values.Files,
681
})
@@ -651,6 +688,49 @@ func (c *Collector) exportOutputQueueMetrics() {
688
}
689
}
690
691
+func (c *Collector) exportQueueTotalsMetrics() {
692
+ if !c.batchPathActive() {
693
+ return
694
+ }
695
+
696
+ snapshot := c.batch.cache.getTotals()
697
+ if snapshot.timestamp.IsZero() && len(snapshot.queues) == 0 && len(snapshot.items) == 0 && snapshot.err == nil {
698
+ return
699
+ }
700
+
701
+ interval := c.batchPathIntervalSeconds()
702
+ types := []struct {
703
+ queueType string
704
+ itemType string
705
+ enabled bool
706
+ }{
707
+ {"message_queue", "message", c.CollectMessageQueueTotals.IsEnabled()},
708
+ {"job_queue", "job", c.CollectJobQueueTotals.IsEnabled()},
709
+ {"output_queue", "spooled_file", c.CollectOutputQueueTotals.IsEnabled()},
710
+ }
711
+
712
+ for _, entry := range types {
713
+ if !entry.enabled {
714
+ continue
715
+ }
716
+
717
+ labels := contexts.QueueOverviewLabels{
718
+ Queue_type: entry.queueType,
719
+ Item_type: entry.itemType,
720
+ }
721
+ if interval > 0 {
722
+ contexts.QueueOverview.Count.SetUpdateEvery(c.State, labels, interval)
723
+ contexts.QueueOverview.Items.SetUpdateEvery(c.State, labels, interval)
724
+ }
725
+ contexts.QueueOverview.Count.Set(c.State, labels, contexts.QueueOverviewCountValues{
726
+ Queues: snapshot.queues[entry.queueType],
727
+ })
728
+ contexts.QueueOverview.Items.Set(c.State, labels, contexts.QueueOverviewItemsValues{
729
+ Items: snapshot.items[entry.queueType],
730
+ })
731
+ }
732
+}
733
+
734
func (c *Collector) exportActiveJobMetrics() {
735
for jobName, values := range c.mx.activeJobs {
736
meta := c.activeJobs[jobName]
@@ -659,7 +739,11 @@ func (c *Collector) exportActiveJobMetrics() {
739
subsystem := ""
740
jobType := ""
741
if meta != nil {
662
- if meta.jobName != "" {
742
+ if meta.qualifiedName != "" {
743
+ jobNameLabel = meta.qualifiedName
744
+ } else if meta.jobNumber != "" && meta.jobUser != "" && meta.jobName != "" {
745
+ jobNameLabel = fmt.Sprintf("%s/%s/%s", meta.jobNumber, meta.jobUser, meta.jobName)
746
+ } else if meta.jobName != "" {
747
jobNameLabel = meta.jobName
748
}
749
jobStatus = meta.jobStatus
@@ -772,82 +856,225 @@ func (c *Collector) exportPlanCacheMetrics() {
856
metricLabel = meta.heading
857
}
858
labels := contexts.PlanCacheLabels{Metric: metricLabel}
859
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
860
+ contexts.PlanCache.Summary.SetUpdateEvery(c.State, labels, interval)
861
+ }
862
contexts.PlanCache.Summary.Set(c.State, labels, contexts.PlanCacheSummaryValues{
863
Value: values.Value,
864
})
865
}
866
}
867
781
-func (c *Collector) exportQueryLatencyMetrics() {
782
- if c.mx == nil || len(c.mx.queryLatencies) == 0 {
783
- return
868
+func (c *Collector) logUnknownQueryLatency(path, name string) {
869
+ c.logOnce("unknown_query_latency_"+path+"_"+name, "%s path query latency not mapped to chart dimension: %s", path, name)
870
+}
871
+
872
+func (c *Collector) splitLatencyCounters(counters map[string]int64) (
873
+ contexts.ObservabilityQueryLatencyFastValues,
874
+ contexts.ObservabilityQueryLatencySlowValues,
875
+ contexts.ObservabilityQueryLatencyBatchValues,
876
+ bool, bool, bool,
877
+) {
878
+ var (
879
+ fast contexts.ObservabilityQueryLatencyFastValues
880
+ slow contexts.ObservabilityQueryLatencySlowValues
881
+ batch contexts.ObservabilityQueryLatencyBatchValues
882
+ )
883
+
884
+ if len(counters) == 0 {
885
+ return fast, slow, batch, false, false, false
886
+ }
887
+
888
+ fastDirect := map[string]*int64{
889
+ "count_disks": &fast.Count_disks,
890
+ "count_http_servers": &fast.Count_http_servers,
891
+ "count_network_interfaces": &fast.Count_network_interfaces,
892
+ "detect_ibmi_version_primary": &fast.Detect_ibmi_version_primary,
893
+ "detect_ibmi_version_fallback": &fast.Detect_ibmi_version_fallback,
894
+ "disk_instances": &fast.Disk_instances,
895
+ "disk_instances_enhanced": &fast.Disk_instances_enhanced,
896
+ "disk_status": &fast.Disk_status,
897
+ "http_server_info": &fast.Http_server_info,
898
+ "job_info": &fast.Job_info,
899
+ "memory_pools": &fast.Memory_pools,
900
+ "network_connections": &fast.Network_connections,
901
+ "network_interfaces": &fast.Network_interfaces,
902
+ "serial_number": &fast.Serial_number,
903
+ "system_activity": &fast.System_activity,
904
+ "system_model": &fast.System_model,
905
+ "system_status": &fast.System_status,
906
+ "system_name_metric": &fast.System_name,
907
+ "temp_storage_named": &fast.Temp_storage_named,
908
+ "temp_storage_total": &fast.Temp_storage_total,
909
+ "technology_refresh_level": &fast.Technology_refresh_level,
910
+ }
911
+
912
+ slowDirect := map[string]*int64{
913
+ "analyze_plan_cache": &slow.Analyze_plan_cache,
914
+ "count_subsystems": &slow.Count_subsystems,
915
+ "subsystems": &slow.Subsystems,
916
+ "plan_cache_summary": &slow.Plan_cache_summary,
917
}
918
786
- values := contexts.ObservabilityQueryLatencyValues{}
787
- fieldMap := map[string]*int64{
788
- "analyze_plan_cache": &values.Analyze_plan_cache,
789
- "count_active_jobs": &values.Count_active_jobs,
790
- "count_disks": &values.Count_disks,
791
- "count_http_servers": &values.Count_http_servers,
792
- "count_job_queues": &values.Count_job_queues,
793
- "count_message_queues": &values.Count_message_queues,
794
- "count_network_interfaces": &values.Count_network_interfaces,
795
- "count_output_queues": &values.Count_output_queues,
796
- "count_subsystems": &values.Count_subsystems,
797
- "detect_ibmi_version_primary": &values.Detect_ibmi_version_primary,
798
- "detect_ibmi_version_fallback": &values.Detect_ibmi_version_fallback,
799
- "disk_instances": &values.Disk_instances,
800
- "disk_instances_enhanced": &values.Disk_instances_enhanced,
801
- "disk_status": &values.Disk_status,
802
- "http_server_info": &values.Http_server_info,
803
- "job_info": &values.Job_info,
804
- "job_queues": &values.Job_queues,
805
- "memory_pools": &values.Memory_pools,
806
- "message_queue_aggregates": &values.Message_queue_aggregates,
807
- "network_connections": &values.Network_connections,
808
- "network_interfaces": &values.Network_interfaces,
809
- "output_queue_info": &values.Output_queue_info,
810
- "plan_cache_summary": &values.Plan_cache_summary,
811
- "serial_number": &values.Serial_number,
812
- "system_activity": &values.System_activity,
813
- "system_model": &values.System_model,
814
- "system_status": &values.System_status,
815
- "temp_storage_named": &values.Temp_storage_named,
816
- "temp_storage_total": &values.Temp_storage_total,
817
- "technology_refresh_level": &values.Technology_refresh_level,
818
- "top_active_jobs": &values.Top_active_jobs,
819
- }
820
-
821
- var otherTotal int64
822
-
823
- for name, latency := range c.mx.queryLatencies {
824
- if latency == 0 {
919
+ var (
920
+ fastSet bool
921
+ slowSet bool
922
+ batchSet bool
923
+ )
924
+
925
+ for name, total := range counters {
926
+ if total <= 0 {
927
+ continue
928
+ }
929
+
930
+ if target, ok := fastDirect[name]; ok {
931
+ *target += total
932
+ fastSet = true
933
continue
934
}
827
- if target, ok := fieldMap[name]; ok {
828
- *target += latency
829
- } else {
830
- otherTotal += latency
935
+
936
+ if target, ok := slowDirect[name]; ok {
937
+ *target += total
938
+ slowSet = true
939
+ continue
940
+ }
941
+
942
+ switch {
943
+ case name == queryNameMessageQueueTotals:
944
+ batch.Message_queue_totals += total
945
+ batchSet = true
946
+ case name == queryNameJobQueueTotals:
947
+ batch.Job_queue_totals += total
948
+ batchSet = true
949
+ case name == queryNameOutputQueueTotals:
950
+ batch.Output_queue_totals += total
951
+ batchSet = true
952
+ case strings.HasPrefix(name, "message_queue_"):
953
+ slow.Message_queue_aggregates += total
954
+ slowSet = true
955
+ case strings.HasPrefix(name, "job_queue_"):
956
+ slow.Job_queues += total
957
+ slowSet = true
958
+ case strings.HasPrefix(name, "output_queue_"):
959
+ slow.Output_queue_info += total
960
+ slowSet = true
961
+ case strings.HasPrefix(name, "active_job_"):
962
+ fast.Active_job += total
963
+ fastSet = true
964
+ default:
965
+ c.logOnce("unknown_query_latency_"+name, "query latency not mapped to chart dimension: %s", name)
966
+ }
967
+ }
968
+
969
+ return fast, slow, batch, fastSet, slowSet, batchSet
970
+}
971
+
972
+func (c *Collector) exportQueryLatencyMetrics() {
973
+ fastValues, slowFallback, batchFallback, fastHasData, slowFallbackHasData, batchFallbackHasData := c.splitLatencyCounters(c.fastQueryLatencyCounters)
974
+
975
+ if fastHasData {
976
+ labels := contexts.EmptyLabels{}
977
+ if interval := c.fastPathIntervalSeconds(); interval > 0 {
978
+ contexts.Observability.QueryLatencyFast.SetUpdateEvery(c.State, labels, interval)
979
}
980
+ contexts.Observability.QueryLatencyFast.Set(c.State, labels, fastValues)
981
}
982
834
- if otherTotal > 0 {
835
- values.Other = otherTotal
983
+ var slowSet bool
984
+ if c.slowPathActive() {
985
+ if counters, _ := c.slow.cache.getLatencies(); len(counters) > 0 {
986
+ _, slowValues, _, _, slowHasData, _ := c.splitLatencyCounters(counters)
987
+ if slowHasData {
988
+ labels := contexts.EmptyLabels{}
989
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
990
+ contexts.Observability.QueryLatencySlow.SetUpdateEvery(c.State, labels, interval)
991
+ }
992
+ contexts.Observability.QueryLatencySlow.Set(c.State, labels, slowValues)
993
+ slowSet = true
994
+ }
995
+ }
996
+ }
997
+ if !slowSet && slowFallbackHasData {
998
+ labels := contexts.EmptyLabels{}
999
+ if interval := c.fastPathIntervalSeconds(); interval > 0 {
1000
+ contexts.Observability.QueryLatencySlow.SetUpdateEvery(c.State, labels, interval)
1001
+ }
1002
+ contexts.Observability.QueryLatencySlow.Set(c.State, labels, slowFallback)
1003
+ } else if !slowSet && c.slowPathActive() {
1004
+ labels := contexts.EmptyLabels{}
1005
+ if interval := c.slowPathIntervalSeconds(); interval > 0 {
1006
+ contexts.Observability.QueryLatencySlow.SetUpdateEvery(c.State, labels, interval)
1007
+ }
1008
+ contexts.Observability.QueryLatencySlow.Set(c.State, labels, contexts.ObservabilityQueryLatencySlowValues{})
1009
}
1010
838
- var total int64
839
- for _, ptr := range fieldMap {
840
- if ptr != nil {
841
- total += *ptr
1011
+ var batchSet bool
1012
+ if c.batchPathActive() {
1013
+ if counters, _ := c.batch.cache.getLatencies(); len(counters) > 0 {
1014
+ _, _, batchValues, _, _, batchHasData := c.splitLatencyCounters(counters)
1015
+ if batchHasData {
1016
+ c.batchLatencyValues = batchValues
1017
+ c.batchLatencyValid = true
1018
+ labels := contexts.EmptyLabels{}
1019
+ if interval := c.batchPathIntervalSeconds(); interval > 0 {
1020
+ contexts.Observability.QueryLatencyBatch.SetUpdateEvery(c.State, labels, interval)
1021
+ }
1022
+ contexts.Observability.QueryLatencyBatch.Set(c.State, labels, c.batchLatencyValues)
1023
+ batchSet = true
1024
+ }
1025
+ }
1026
+ }
1027
+ if !batchSet && c.batchLatencyValid {
1028
+ labels := contexts.EmptyLabels{}
1029
+ if interval := c.batchPathIntervalSeconds(); interval > 0 {
1030
+ contexts.Observability.QueryLatencyBatch.SetUpdateEvery(c.State, labels, interval)
1031
+ }
1032
+ contexts.Observability.QueryLatencyBatch.Set(c.State, labels, c.batchLatencyValues)
1033
+ batchSet = true
1034
+ }
1035
+ if !batchSet && batchFallbackHasData {
1036
+ labels := contexts.EmptyLabels{}
1037
+ if interval := c.fastPathIntervalSeconds(); interval > 0 {
1038
+ contexts.Observability.QueryLatencyBatch.SetUpdateEvery(c.State, labels, interval)
1039
+ }
1040
+ contexts.Observability.QueryLatencyBatch.Set(c.State, labels, batchFallback)
1041
+ } else if c.batchPathActive() && !batchSet {
1042
+ labels := contexts.EmptyLabels{}
1043
+ if interval := c.batchPathIntervalSeconds(); interval > 0 {
1044
+ contexts.Observability.QueryLatencyBatch.SetUpdateEvery(c.State, labels, interval)
1045
}
1046
+ contexts.Observability.QueryLatencyBatch.Set(c.State, labels, contexts.ObservabilityQueryLatencyBatchValues{})
1047
}
844
- total += values.Other
1048
+}
1049
846
- if total == 0 {
847
- return
1050
+func (c *Collector) fastPathIntervalSeconds() int {
1051
+ if c == nil {
1052
+ return 0
1053
+ }
1054
+ if c.Collector.Config.UpdateEvery > 0 {
1055
+ return c.Collector.Config.UpdateEvery
1056
}
1057
+ if c.Config.UpdateEvery > 0 {
1058
+ return c.Config.UpdateEvery
1059
+ }
1060
+ return 1
1061
+}
1062
850
- contexts.Observability.QueryLatency.Set(c.State, contexts.EmptyLabels{}, values)
1063
+func (c *Collector) slowPathIntervalSeconds() int {
1064
+ if c == nil {
1065
+ return 0
1066
+ }
1067
+ if !c.slowPathActive() {
1068
+ return 0
1069
+ }
1070
+ interval := int(c.slow.config.interval / time.Second)
1071
+ if interval < 1 {
1072
+ interval = c.fastPathIntervalSeconds()
1073
+ if interval < 1 {
1074
+ interval = 1
1075
+ }
1076
+ }
1077
+ return interval
1078
}
1079
1080
func (c *Collector) exportSystemActivityMetrics() {
@@ -873,6 +1100,8 @@ func (c *Collector) verifyConfig() error {
1100
}
1101
1102
func (c *Collector) Cleanup(ctx context.Context) {
1103
+ c.stopBatchPath()
1104
+ c.stopSlowPath()
1105
if c.client != nil {
1106
if err := c.client.Close(); err != nil {
1107
c.Errorf("cleanup: error closing database: %v", err)
src/go/plugin/ibm.d/modules/as400/config.go
+43
-29
@@ -18,12 +18,6 @@ type Config struct {
18
// Timeout controls how long to wait for SQL statements and RPCs.
19
Timeout confopt.Duration `yaml:"timeout,omitempty" json:"timeout" ui:"group:Connection"`
20
21
- // MaxDbConns restricts the maximum number of open ODBC connections.
22
- MaxDbConns int `yaml:"max_db_conns,omitempty" json:"max_db_conns" ui:"group:Advanced"`
23
-
24
- // MaxDbLifeTime limits how long a pooled connection may live before being recycled.
25
- MaxDbLifeTime confopt.Duration `yaml:"max_db_life_time,omitempty" json:"max_db_life_time" ui:"group:Advanced"`
26
-
21
// Hostname is the remote IBM i host to monitor.
22
Hostname string `yaml:"hostname,omitempty" json:"hostname" ui:"group:Connection"`
23
@@ -57,41 +51,47 @@ type Config struct {
51
// CollectSubsystemMetrics toggles collection of subsystem activity metrics.
52
CollectSubsystemMetrics confopt.AutoBool `yaml:"collect_subsystem_metrics,omitempty" json:"collect_subsystem_metrics" ui:"group:Subsystems"`
53
60
- // CollectJobQueueMetrics toggles collection of job queue backlog metrics.
61
- CollectJobQueueMetrics confopt.AutoBool `yaml:"collect_job_queue_metrics,omitempty" json:"collect_job_queue_metrics" ui:"group:Job Queues"`
62
-
54
// CollectActiveJobs toggles collection of detailed per-job metrics.
55
CollectActiveJobs confopt.AutoBool `yaml:"collect_active_jobs,omitempty" json:"collect_active_jobs" ui:"group:Active Jobs"`
56
57
// CollectHTTPServerMetrics toggles collection of IBM HTTP Server statistics.
58
CollectHTTPServerMetrics confopt.AutoBool `yaml:"collect_http_server_metrics,omitempty" json:"collect_http_server_metrics" ui:"group:Other Metrics"`
59
69
- // CollectMessageQueueMetrics toggles collection of IBM i message queue metrics.
70
- CollectMessageQueueMetrics confopt.AutoBool `yaml:"collect_message_queue_metrics,omitempty" json:"collect_message_queue_metrics" ui:"group:Message Queues"`
71
-
72
- // CollectOutputQueueMetrics toggles collection of IBM i output queue metrics.
73
- CollectOutputQueueMetrics confopt.AutoBool `yaml:"collect_output_queue_metrics,omitempty" json:"collect_output_queue_metrics" ui:"group:Output Queues"`
74
-
60
// CollectPlanCacheMetrics toggles collection of plan cache analysis metrics.
61
CollectPlanCacheMetrics confopt.AutoBool `yaml:"collect_plan_cache_metrics,omitempty" json:"collect_plan_cache_metrics" ui:"group:Other Metrics"`
62
78
- // MaxDisks caps how many disk units may be charted.
79
- MaxDisks int `yaml:"max_disks,omitempty" json:"max_disks" ui:"group:Disks"`
63
+ // CollectMessageQueueTotals enables expensive aggregate counting across all message queues.
64
+ CollectMessageQueueTotals confopt.AutoBool `yaml:"collect_message_queue_totals,omitempty" json:"collect_message_queue_totals" ui:"group:Queues"`
65
81
- // MaxSubsystems caps how many subsystems may be charted.
82
- MaxSubsystems int `yaml:"max_subsystems,omitempty" json:"max_subsystems" ui:"group:Subsystems"`
66
+ // CollectJobQueueTotals enables expensive aggregate counting across all job queues.
67
+ CollectJobQueueTotals confopt.AutoBool `yaml:"collect_job_queue_totals,omitempty" json:"collect_job_queue_totals" ui:"group:Queues"`
68
+
69
+ // CollectOutputQueueTotals enables expensive aggregate counting across all output queues.
70
+ CollectOutputQueueTotals confopt.AutoBool `yaml:"collect_output_queue_totals,omitempty" json:"collect_output_queue_totals" ui:"group:Queues"`
71
+
72
+ // SlowPath enables the asynchronous slow-path worker for heavy queries.
73
+ SlowPath bool `yaml:"slow_path,omitempty" json:"slow_path" ui:"group:Advanced"`
74
84
- // MaxJobQueues caps how many job queues may be charted.
85
- MaxJobQueues int `yaml:"max_job_queues,omitempty" json:"max_job_queues" ui:"group:Job Queues"`
75
+ // SlowPathUpdateEvery controls the beat interval for the slow-path worker.
76
+ SlowPathUpdateEvery confopt.Duration `yaml:"slow_path_update_every,omitempty" json:"slow_path_update_every" ui:"group:Advanced"`
77
87
- // MaxMessageQueues caps how many message queues may be charted.
88
- MaxMessageQueues int `yaml:"max_message_queues,omitempty" json:"max_message_queues" ui:"group:Message Queues"`
78
+ // SlowPathMaxConnections caps the number of concurrent queries the slow-path worker may run.
79
+ SlowPathMaxConnections int `yaml:"slow_path_max_connections,omitempty" json:"slow_path_max_connections" ui:"group:Advanced"`
80
90
- // MaxOutputQueues caps how many output queues may be charted.
91
- MaxOutputQueues int `yaml:"max_output_queues,omitempty" json:"max_output_queues" ui:"group:Output Queues"`
81
+ // BatchPath enables the long-period batch worker for expensive queue aggregates.
82
+ BatchPath bool `yaml:"batch_path,omitempty" json:"batch_path" ui:"group:Advanced"`
83
93
- // MaxActiveJobs caps how many active jobs may be charted.
94
- MaxActiveJobs int `yaml:"max_active_jobs,omitempty" json:"max_active_jobs" ui:"group:Active Jobs"`
84
+ // BatchPathUpdateEvery controls the beat interval for the batch worker.
85
+ BatchPathUpdateEvery confopt.Duration `yaml:"batch_path_update_every,omitempty" json:"batch_path_update_every" ui:"group:Advanced"`
86
+
87
+ // BatchPathMaxConnections caps concurrent queries for the batch worker.
88
+ BatchPathMaxConnections int `yaml:"batch_path_max_connections,omitempty" json:"batch_path_max_connections" ui:"group:Advanced"`
89
+
90
+ // MaxDisks caps how many disk units may be charted.
91
+ MaxDisks int `yaml:"max_disks,omitempty" json:"max_disks" ui:"group:Disks"`
92
+
93
+ // MaxSubsystems caps how many subsystems may be charted.
94
+ MaxSubsystems int `yaml:"max_subsystems,omitempty" json:"max_subsystems" ui:"group:Subsystems"`
95
96
// DiskSelector filters disk units by name using glob-style patterns.
97
DiskSelector string `yaml:"collect_disks_matching,omitempty" json:"collect_disks_matching" ui:"group:Disks"`
@@ -99,6 +99,20 @@ type Config struct {
99
// SubsystemSelector filters subsystems by name using glob-style patterns.
100
SubsystemSelector string `yaml:"collect_subsystems_matching,omitempty" json:"collect_subsystems_matching" ui:"group:Subsystems"`
101
102
- // JobQueueSelector filters job queues by name using glob-style patterns.
103
- JobQueueSelector string `yaml:"collect_job_queues_matching,omitempty" json:"collect_job_queues_matching" ui:"group:Job Queues"`
102
+ // ActiveJobs lists active jobs to monitor, using fully-qualified job identifiers (JOB_NUMBER/USER/JOB_NAME).
103
+ // When empty, active job collection is disabled.
104
+ ActiveJobs []string `yaml:"active_jobs,omitempty" json:"active_jobs" ui:"group:Active Jobs"`
105
+
106
+ // MessageQueues lists message queues to collect, formatted as LIBRARY/QUEUE strings.
107
+ // When empty, message queue collection is disabled. The default configuration monitors
108
+ // QSYS/QSYSOPR, QSYS/QSYSMSG, and QSYS/QHST.
109
+ MessageQueues []string `yaml:"message_queues,omitempty" json:"message_queues" ui:"group:Queues"`
110
+
111
+ // JobQueues lists job queues to collect, formatted as LIBRARY/QUEUE strings.
112
+ // When empty, job queue collection is disabled.
113
+ JobQueues []string `yaml:"job_queues,omitempty" json:"job_queues" ui:"group:Queues"`
114
+
115
+ // OutputQueues lists output queues to collect, formatted as LIBRARY/QUEUE strings.
116
+ // When empty, output queue collection is disabled.
117
+ OutputQueues []string `yaml:"output_queues,omitempty" json:"output_queues" ui:"group:Queues"`
118
}
src/go/plugin/ibm.d/modules/as400/config_schema.json
+157
-86
@@ -2,208 +2,274 @@
2
"jsonSchema": {
3
"$schema": "http://json-schema.org/draft-07/schema#",
4
"properties": {
5
+ "active_jobs": {
6
+ "default": "nil",
7
+ "description": "ActiveJobs lists active jobs to monitor, using fully-qualified job identifiers (JOB_NUMBER/USER/JOB_NAME). When empty, active job collection is disabled.",
8
+ "items": {
9
+ "type": "string"
10
+ },
11
+ "title": "Active Jobs",
12
+ "type": "array"
13
+ },
14
+ "batch_path": {
15
+ "default": false,
16
+ "description": "BatchPath enables the long-period batch worker for expensive queue aggregates.",
17
+ "title": "Batch Path",
18
+ "type": "boolean"
19
+ },
20
+ "batch_path_max_connections": {
21
+ "default": 1,
22
+ "description": "BatchPathMaxConnections caps concurrent queries for the batch worker.",
23
+ "title": "Batch Path Max Connections",
24
+ "type": "integer"
25
+ },
26
+ "batch_path_update_every": {
27
+ "default": 60000000000,
28
+ "description": "BatchPathUpdateEvery controls the beat interval for the batch worker.",
29
+ "title": "Batch Path Update Every",
30
+ "type": "integer"
31
+ },
32
"collect_active_jobs": {
33
"default": "auto",
34
+ "description": "CollectActiveJobs toggles collection of detailed per-job metrics.",
35
"enum": [
36
"auto",
37
"enabled",
38
"disabled"
39
],
12
- "title": "CollectActiveJobs toggles collection of detailed per-job metrics.",
40
+ "title": "Collect Active Jobs",
41
"type": "string"
42
},
43
"collect_disk_metrics": {
44
"default": "auto",
45
+ "description": "CollectDiskMetrics toggles collection of disk unit statistics.",
46
"enum": [
47
"auto",
48
"enabled",
49
"disabled"
50
],
22
- "title": "CollectDiskMetrics toggles collection of disk unit statistics.",
51
+ "title": "Collect Disk Metrics",
52
"type": "string"
53
},
54
"collect_disks_matching": {
55
"default": "",
27
- "title": "DiskSelector filters disk units by name using glob-style patterns.",
56
+ "description": "DiskSelector filters disk units by name using glob-style patterns.",
57
+ "title": "Collect Disks Matching",
58
"type": "string"
59
},
60
"collect_http_server_metrics": {
61
"default": "auto",
62
+ "description": "CollectHTTPServerMetrics toggles collection of IBM HTTP Server statistics.",
63
"enum": [
64
"auto",
65
"enabled",
66
"disabled"
67
],
37
- "title": "CollectHTTPServerMetrics toggles collection of IBM HTTP Server statistics.",
68
+ "title": "Collect Http Server Metrics",
69
"type": "string"
70
},
40
- "collect_job_queue_metrics": {
71
+ "collect_job_queue_totals": {
72
"default": "auto",
73
+ "description": "CollectJobQueueTotals enables expensive aggregate counting across all job queues.",
74
"enum": [
75
"auto",
76
"enabled",
77
"disabled"
78
],
47
- "title": "CollectJobQueueMetrics toggles collection of job queue backlog metrics.",
79
+ "title": "Collect Job Queue Totals",
80
"type": "string"
81
},
50
- "collect_job_queues_matching": {
51
- "default": "",
52
- "title": "JobQueueSelector filters job queues by name using glob-style patterns.",
53
- "type": "string"
54
- },
55
- "collect_message_queue_metrics": {
82
+ "collect_message_queue_totals": {
83
"default": "auto",
84
+ "description": "CollectMessageQueueTotals enables expensive aggregate counting across all message queues.",
85
"enum": [
86
"auto",
87
"enabled",
88
"disabled"
89
],
62
- "title": "CollectMessageQueueMetrics toggles collection of IBM i message queue metrics.",
90
+ "title": "Collect Message Queue Totals",
91
"type": "string"
92
},
65
- "collect_output_queue_metrics": {
93
+ "collect_output_queue_totals": {
94
"default": "auto",
95
+ "description": "CollectOutputQueueTotals enables expensive aggregate counting across all output queues.",
96
"enum": [
97
"auto",
98
"enabled",
99
"disabled"
100
],
72
- "title": "CollectOutputQueueMetrics toggles collection of IBM i output queue metrics.",
101
+ "title": "Collect Output Queue Totals",
102
"type": "string"
103
},
104
"collect_plan_cache_metrics": {
105
"default": "auto",
106
+ "description": "CollectPlanCacheMetrics toggles collection of plan cache analysis metrics.",
107
"enum": [
108
"auto",
109
"enabled",
110
"disabled"
111
],
82
- "title": "CollectPlanCacheMetrics toggles collection of plan cache analysis metrics.",
112
+ "title": "Collect Plan Cache Metrics",
113
"type": "string"
114
},
115
"collect_subsystem_metrics": {
116
"default": "auto",
117
+ "description": "CollectSubsystemMetrics toggles collection of subsystem activity metrics.",
118
"enum": [
119
"auto",
120
"enabled",
121
"disabled"
122
],
92
- "title": "CollectSubsystemMetrics toggles collection of subsystem activity metrics.",
123
+ "title": "Collect Subsystem Metrics",
124
"type": "string"
125
},
126
"collect_subsystems_matching": {
127
"default": "",
97
- "title": "SubsystemSelector filters subsystems by name using glob-style patterns.",
128
+ "description": "SubsystemSelector filters subsystems by name using glob-style patterns.",
129
+ "title": "Collect Subsystems Matching",
130
"type": "string"
131
},
132
"connection_type": {
133
"default": "odbc",
102
- "title": "ConnectionType selects how the collector connects (currently only \"odbc\").",
134
+ "description": "ConnectionType selects how the collector connects (currently only \"odbc\").",
135
+ "title": "Connection Type",
136
"type": "string"
137
},
138
"database": {
139
"default": "*SYSBAS",
107
- "title": "Database selects the IBM i database (library) to use when building the DSN.",
140
+ "description": "Database selects the IBM i database (library) to use when building the DSN.",
141
+ "title": "Database",
142
"type": "string"
143
},
144
"dsn": {
145
"default": "",
112
- "title": "DSN provides a full IBM i ODBC connection string if manual override is needed.",
146
+ "description": "DSN provides a full IBM i ODBC connection string if manual override is needed.",
147
+ "title": "DSN",
148
"type": "string"
149
},
150
"hostname": {
151
"default": "",
117
- "title": "Hostname is the remote IBM i host to monitor.",
152
+ "description": "Hostname is the remote IBM i host to monitor.",
153
+ "title": "Hostname",
154
"type": "string"
155
},
120
- "max_active_jobs": {
121
- "default": 100,
122
- "title": "MaxActiveJobs caps how many active jobs may be charted.",
123
- "type": "integer"
124
- },
125
- "max_db_conns": {
126
- "default": 1,
127
- "title": "MaxDbConns restricts the maximum number of open ODBC connections.",
128
- "type": "integer"
129
- },
130
- "max_db_life_time": {
131
- "default": 600000000000,
132
- "title": "MaxDbLifeTime limits how long a pooled connection may live before being recycled.",
133
- "type": "integer"
156
+ "job_queues": {
157
+ "default": "nil",
158
+ "description": "JobQueues lists job queues to collect, formatted as LIBRARY/QUEUE strings. When empty, job queue collection is disabled.",
159
+ "items": {
160
+ "type": "string"
161
+ },
162
+ "title": "Job Queues",
163
+ "type": "array"
164
},
165
"max_disks": {
166
"default": 100,
137
- "title": "MaxDisks caps how many disk units may be charted.",
138
- "type": "integer"
139
- },
140
- "max_job_queues": {
141
- "default": 100,
142
- "title": "MaxJobQueues caps how many job queues may be charted.",
143
- "type": "integer"
144
- },
145
- "max_message_queues": {
146
- "default": 100,
147
- "title": "MaxMessageQueues caps how many message queues may be charted.",
148
- "type": "integer"
149
- },
150
- "max_output_queues": {
151
- "default": 100,
152
- "title": "MaxOutputQueues caps how many output queues may be charted.",
167
+ "description": "MaxDisks caps how many disk units may be charted.",
168
+ "title": "Max Disks",
169
"type": "integer"
170
},
171
"max_subsystems": {
172
"default": 100,
157
- "title": "MaxSubsystems caps how many subsystems may be charted.",
173
+ "description": "MaxSubsystems caps how many subsystems may be charted.",
174
+ "title": "Max Subsystems",
175
"type": "integer"
176
},
177
+ "message_queues": {
178
+ "default": [
179
+ "QSYS/QSYSOPR",
180
+ "QSYS/QSYSMSG",
181
+ "QSYS/QHST"
182
+ ],
183
+ "description": "MessageQueues lists message queues to collect, formatted as LIBRARY/QUEUE strings. When empty, message queue collection is disabled. The default configuration monitors QSYS/QSYSOPR, QSYS/QSYSMSG, and QSYS/QHST.",
184
+ "items": {
185
+ "type": "string"
186
+ },
187
+ "title": "Message Queues",
188
+ "type": "array"
189
+ },
190
"odbc_driver": {
191
"default": "IBM i Access ODBC Driver",
162
- "title": "ODBCDriver specifies the driver name registered on the host.",
192
+ "description": "ODBCDriver specifies the driver name registered on the host.",
193
+ "title": "ODBC Driver",
194
"type": "string"
195
},
196
+ "output_queues": {
197
+ "default": "nil",
198
+ "description": "OutputQueues lists output queues to collect, formatted as LIBRARY/QUEUE strings. When empty, output queue collection is disabled.",
199
+ "items": {
200
+ "type": "string"
201
+ },
202
+ "title": "Output Queues",
203
+ "type": "array"
204
+ },
205
"password": {
206
"default": "",
207
+ "description": "Password supplies the password used for authentication.",
208
"format": "password",
168
- "title": "Password supplies the password used for authentication.",
209
+ "title": "Password",
210
"type": "string"
211
},
212
"port": {
213
"default": 8471,
214
+ "description": "Port is the TCP port for the IBM i Access ODBC server.",
215
"maximum": 65535,
216
"minimum": 1,
175
- "title": "Port is the TCP port for the IBM i Access ODBC server.",
217
+ "title": "Port",
218
"type": "integer"
219
},
220
"reset_statistics": {
221
"default": false,
180
- "title": "ResetStatistics toggles destructive SQL services that reset system statistics on each query.",
222
+ "description": "ResetStatistics toggles destructive SQL services that reset system statistics on each query.",
223
+ "title": "Reset Statistics",
224
+ "type": "boolean"
225
+ },
226
+ "slow_path": {
227
+ "default": true,
228
+ "description": "SlowPath enables the asynchronous slow-path worker for heavy queries.",
229
+ "title": "Slow Path",
230
"type": "boolean"
231
},
232
+ "slow_path_max_connections": {
233
+ "default": 1,
234
+ "description": "SlowPathMaxConnections caps the number of concurrent queries the slow-path worker may run.",
235
+ "title": "Slow Path Max Connections",
236
+ "type": "integer"
237
+ },
238
+ "slow_path_update_every": {
239
+ "default": 10000000000,
240
+ "description": "SlowPathUpdateEvery controls the beat interval for the slow-path worker.",
241
+ "title": "Slow Path Update Every",
242
+ "type": "integer"
243
+ },
244
"timeout": {
245
"default": 2000000000,
185
- "title": "Timeout controls how long to wait for SQL statements and RPCs.",
246
+ "description": "Timeout controls how long to wait for SQL statements and RPCs.",
247
+ "title": "Timeout",
248
"type": "integer"
249
},
250
"update_every": {
189
- "default": 10,
251
+ "default": 5,
252
+ "description": "Data collection frequency",
253
"minimum": 1,
191
- "title": "Data collection frequency",
254
+ "title": "Update Every",
255
"type": "integer"
256
},
257
"use_ssl": {
258
"default": false,
196
- "title": "UseSSL enables TLS for the ODBC connection when supported by the driver.",
259
+ "description": "UseSSL enables TLS for the ODBC connection when supported by the driver.",
260
+ "title": "Use SSL",
261
"type": "boolean"
262
},
263
"username": {
264
"default": "",
201
- "title": "Username supplies the credentials used for authentication.",
265
+ "description": "Username supplies the credentials used for authentication.",
266
+ "title": "Username",
267
"type": "string"
268
},
269
"vnode": {
270
"default": "",
206
- "title": "Vnode allows binding the collector to a virtual node.",
271
+ "description": "Vnode allows binding the collector to a virtual node.",
272
+ "title": "Vnode",
273
"type": "string"
274
}
275
},
@@ -211,6 +277,18 @@
277
"type": "object"
278
},
279
"uiSchema": {
280
+ "active_jobs": {
281
+ "ui:listFlavour": "list"
282
+ },
283
+ "job_queues": {
284
+ "ui:listFlavour": "list"
285
+ },
286
+ "message_queues": {
287
+ "ui:listFlavour": "list"
288
+ },
289
+ "output_queues": {
290
+ "ui:listFlavour": "list"
291
+ },
292
"password": {
293
"ui:widget": "password"
294
},
@@ -236,9 +314,13 @@
314
},
315
{
316
"fields": [
239
- "max_db_conns",
240
- "max_db_life_time",
241
- "reset_statistics"
317
+ "reset_statistics",
318
+ "slow_path",
319
+ "slow_path_update_every",
320
+ "slow_path_max_connections",
321
+ "batch_path",
322
+ "batch_path_update_every",
323
+ "batch_path_max_connections"
324
],
325
"title": "Advanced"
326
},
@@ -258,18 +340,10 @@
340
],
341
"title": "Subsystems"
342
},
261
- {
262
- "fields": [
263
- "collect_job_queue_metrics",
264
- "max_job_queues",
265
- "collect_job_queues_matching"
266
- ],
267
- "title": "Job Queues"
268
- },
343
{
344
"fields": [
345
"collect_active_jobs",
272
- "max_active_jobs"
346
+ "active_jobs"
347
],
348
"title": "Active Jobs"
349
},
@@ -282,17 +356,14 @@
356
},
357
{
358
"fields": [
285
- "collect_message_queue_metrics",
286
- "max_message_queues"
287
- ],
288
- "title": "Message Queues"
289
- },
290
- {
291
- "fields": [
292
- "collect_output_queue_metrics",
293
- "max_output_queues"
359
+ "collect_message_queue_totals",
360
+ "collect_job_queue_totals",
361
+ "collect_output_queue_totals",
362
+ "message_queues",
363
+ "job_queues",
364
+ "output_queues"
365
],
295
- "title": "Output Queues"
366
+ "title": "Queues"
367
}
368
]
369
},
@@ -300,4 +371,4 @@
371
"fullPage": true
372
}
373
}
303
-}
\ No newline at end of file
374
+}
src/go/plugin/ibm.d/modules/as400/contexts/contexts.yaml
+104
-64
@@ -388,6 +388,31 @@ Subsystem:
388
algo: absolute
389
- name: maximum
390
algo: absolute
391
+QueueOverview:
392
+ labels:
393
+ - queue_type
394
+ - item_type
395
+ contexts:
396
+ - name: Count
397
+ context: as400.queues_count
398
+ title: Queue Counts
399
+ family: queues/overview
400
+ units: queues
401
+ type: line
402
+ priority: 504
403
+ dimensions:
404
+ - name: queues
405
+ algo: absolute
406
+ - name: Items
407
+ context: as400.queued_items
408
+ title: Queued Items
409
+ family: queues/overview
410
+ units: items
411
+ type: line
412
+ priority: 505
413
+ dimensions:
414
+ - name: items
415
+ algo: absolute
416
JobQueue:
417
labels:
418
- job_queue
@@ -397,7 +422,7 @@ JobQueue:
422
- name: Length
423
context: as400.jobqueue_length
424
title: Job Queue Length
400
- family: workloads/job_queues
425
+ family: queues/job
426
units: jobs
427
type: line
428
priority: 505
@@ -574,7 +599,7 @@ MessageQueue:
599
- name: Messages
600
context: as400.message_queue_messages
601
title: Message Queue Messages
577
- family: messaging/message_queues
602
+ family: queues/message
603
units: messages
604
type: stacked
605
priority: 700
@@ -596,7 +621,7 @@ MessageQueue:
621
- name: Severity
622
context: as400.message_queue_severity
623
title: Message Queue Severity
599
- family: messaging/message_queues
624
+ family: queues/message
625
units: severity
626
type: line
627
priority: 701
@@ -612,7 +637,7 @@ OutputQueue:
637
- name: Files
638
context: as400.output_queue_files
639
title: Output Queue Files
615
- family: messaging/output_queues
640
+ family: queues/output
641
units: files
642
type: line
643
priority: 702
@@ -622,7 +647,7 @@ OutputQueue:
647
- name: Writers
648
context: as400.output_queue_writers
649
title: Output Queue Writers
625
- family: messaging/output_queues
650
+ family: queues/output
651
units: writers
652
type: line
653
priority: 703
@@ -632,7 +657,7 @@ OutputQueue:
657
- name: Status
658
context: as400.output_queue_status
659
title: Output Queue Status
635
- family: messaging/output_queues
660
+ family: queues/output
661
units: state
662
type: line
663
priority: 704
@@ -655,109 +680,124 @@ PlanCache:
680
algo: absolute
681
div: 1000
682
Observability:
658
- labels: []
683
contexts:
660
- - name: QueryLatency
661
- context: netdata.plugin_ibm.as400_query_latency
662
- title: AS400 Query Latency
684
+ - name: QueryLatencyFast
685
+ context: netdata.plugin_ibm.as400_query_latency_fast
686
+ title: AS400 Query Latency (Fast Path)
687
family: plugins/ibm.d/latency
688
units: ms
689
type: stacked
690
priority: 146000
691
dimensions:
668
- - name: analyze_plan_cache
669
- algo: absolute
670
- div: 1000
671
- - name: count_active_jobs
672
- algo: absolute
673
- div: 1000
692
- name: count_disks
675
- algo: absolute
693
+ algo: incremental
694
div: 1000
695
- name: count_http_servers
678
- algo: absolute
679
- div: 1000
680
- - name: count_job_queues
681
- algo: absolute
682
- div: 1000
683
- - name: count_message_queues
684
- algo: absolute
696
+ algo: incremental
697
div: 1000
698
- name: count_network_interfaces
687
- algo: absolute
688
- div: 1000
689
- - name: count_output_queues
690
- algo: absolute
691
- div: 1000
692
- - name: count_subsystems
693
- algo: absolute
699
+ algo: incremental
700
div: 1000
701
- name: detect_ibmi_version_primary
696
- algo: absolute
702
+ algo: incremental
703
div: 1000
704
- name: detect_ibmi_version_fallback
699
- algo: absolute
705
+ algo: incremental
706
div: 1000
707
- name: disk_instances
702
- algo: absolute
708
+ algo: incremental
709
div: 1000
710
- name: disk_instances_enhanced
705
- algo: absolute
711
+ algo: incremental
712
div: 1000
713
- name: disk_status
708
- algo: absolute
714
+ algo: incremental
715
div: 1000
716
- name: http_server_info
711
- algo: absolute
717
+ algo: incremental
718
div: 1000
719
- name: job_info
714
- algo: absolute
715
- div: 1000
716
- - name: job_queues
717
- algo: absolute
720
+ algo: incremental
721
div: 1000
722
- name: memory_pools
720
- algo: absolute
721
- div: 1000
722
- - name: message_queue_aggregates
723
- algo: absolute
723
+ algo: incremental
724
div: 1000
725
- name: network_connections
726
- algo: absolute
726
+ algo: incremental
727
div: 1000
728
- name: network_interfaces
729
- algo: absolute
730
- div: 1000
731
- - name: output_queue_info
732
- algo: absolute
733
- div: 1000
734
- - name: plan_cache_summary
735
- algo: absolute
729
+ algo: incremental
730
div: 1000
731
- name: serial_number
738
- algo: absolute
732
+ algo: incremental
733
+ div: 1000
734
+ - name: system_name
735
+ algo: incremental
736
div: 1000
737
- name: system_activity
741
- algo: absolute
738
+ algo: incremental
739
div: 1000
740
- name: system_model
744
- algo: absolute
741
+ algo: incremental
742
div: 1000
743
- name: system_status
747
- algo: absolute
744
+ algo: incremental
745
div: 1000
746
- name: temp_storage_named
750
- algo: absolute
747
+ algo: incremental
748
div: 1000
749
- name: temp_storage_total
753
- algo: absolute
750
+ algo: incremental
751
div: 1000
752
- name: technology_refresh_level
756
- algo: absolute
753
+ algo: incremental
754
div: 1000
758
- - name: top_active_jobs
759
- algo: absolute
755
+ - name: active_job
756
+ algo: incremental
757
div: 1000
761
- - name: other
762
- algo: absolute
758
+ - name: QueryLatencySlow
759
+ context: netdata.plugin_ibm.as400_query_latency_slow
760
+ title: AS400 Query Latency (Slow Path)
761
+ family: plugins/ibm.d/latency
762
+ units: ms
763
+ type: stacked
764
+ priority: 146010
765
+ dimensions:
766
+ - name: analyze_plan_cache
767
+ algo: incremental
768
+ div: 1000
769
+ - name: count_subsystems
770
+ algo: incremental
771
+ div: 1000
772
+ - name: subsystems
773
+ algo: incremental
774
+ div: 1000
775
+ - name: message_queue_aggregates
776
+ algo: incremental
777
+ div: 1000
778
+ - name: job_queues
779
+ algo: incremental
780
+ div: 1000
781
+ - name: output_queue_info
782
+ algo: incremental
783
+ div: 1000
784
+ - name: plan_cache_summary
785
+ algo: incremental
786
+ div: 1000
787
+ - name: QueryLatencyBatch
788
+ context: netdata.plugin_ibm.as400_query_latency_batch
789
+ title: AS400 Query Latency (Batch Path)
790
+ family: plugins/ibm.d/latency
791
+ units: ms
792
+ type: stacked
793
+ priority: 146020
794
+ dimensions:
795
+ - name: message_queue_totals
796
+ algo: incremental
797
+ div: 1000
798
+ - name: job_queue_totals
799
+ algo: incremental
800
+ div: 1000
801
+ - name: output_queue_totals
802
+ algo: incremental
803
div: 1000
src/go/plugin/ibm.d/modules/as400/contexts/zz_generated_contexts.go
+293
-106
@@ -1013,7 +1013,7 @@ var JobQueue = struct {
1013
Length: JobQueueLengthContext{
1014
Context: framework.Context[JobQueueLabels]{
1015
Name: "as400.jobqueue_length",
1016
- Family: "workloads/job_queues",
1016
+ Family: "queues/job",
1017
Title: "Job Queue Length",
1018
Units: "jobs",
1019
Type: module.Line,
@@ -1115,7 +1115,7 @@ var MessageQueue = struct {
1115
Messages: MessageQueueMessagesContext{
1116
Context: framework.Context[MessageQueueLabels]{
1117
Name: "as400.message_queue_messages",
1118
- Family: "messaging/message_queues",
1118
+ Family: "queues/message",
1119
Title: "Message Queue Messages",
1120
Units: "messages",
1121
Type: module.Stacked,
@@ -1181,7 +1181,7 @@ var MessageQueue = struct {
1181
Severity: MessageQueueSeverityContext{
1182
Context: framework.Context[MessageQueueLabels]{
1183
Name: "as400.message_queue_severity",
1184
- Family: "messaging/message_queues",
1184
+ Family: "queues/message",
1185
Title: "Message Queue Severity",
1186
Units: "severity",
1187
Type: module.Line,
@@ -1331,17 +1331,11 @@ var NetworkInterface = struct {
1331
1332
// --- Observability ---
1333
1334
-// ObservabilityQueryLatencyValues defines the type-safe values for Observability.QueryLatency context
1335
-type ObservabilityQueryLatencyValues struct {
1336
- Analyze_plan_cache int64
1337
- Count_active_jobs int64
1334
+// ObservabilityQueryLatencyFastValues defines the type-safe values for Observability.QueryLatencyFast context
1335
+type ObservabilityQueryLatencyFastValues struct {
1336
Count_disks int64
1337
Count_http_servers int64
1340
- Count_job_queues int64
1341
- Count_message_queues int64
1338
Count_network_interfaces int64
1343
- Count_output_queues int64
1344
- Count_subsystems int64
1339
Detect_ibmi_version_primary int64
1340
Detect_ibmi_version_fallback int64
1341
Disk_instances int64
@@ -1349,41 +1343,31 @@ type ObservabilityQueryLatencyValues struct {
1343
Disk_status int64
1344
Http_server_info int64
1345
Job_info int64
1352
- Job_queues int64
1346
Memory_pools int64
1354
- Message_queue_aggregates int64
1347
Network_connections int64
1348
Network_interfaces int64
1357
- Output_queue_info int64
1358
- Plan_cache_summary int64
1349
Serial_number int64
1350
+ System_name int64
1351
System_activity int64
1352
System_model int64
1353
System_status int64
1354
Temp_storage_named int64
1355
Temp_storage_total int64
1356
Technology_refresh_level int64
1366
- Top_active_jobs int64
1367
- Other int64
1357
+ Active_job int64
1358
}
1359
1370
-// ObservabilityQueryLatencyContext provides type-safe operations for Observability.QueryLatency context
1371
-type ObservabilityQueryLatencyContext struct {
1360
+// ObservabilityQueryLatencyFastContext provides type-safe operations for Observability.QueryLatencyFast context
1361
+type ObservabilityQueryLatencyFastContext struct {
1362
framework.Context[EmptyLabels]
1363
}
1364
1375
-// Set provides type-safe dimension setting for Observability.QueryLatency context
1376
-func (c ObservabilityQueryLatencyContext) Set(state *framework.CollectorState, labels EmptyLabels, values ObservabilityQueryLatencyValues) {
1365
+// Set provides type-safe dimension setting for Observability.QueryLatencyFast context
1366
+func (c ObservabilityQueryLatencyFastContext) Set(state *framework.CollectorState, labels EmptyLabels, values ObservabilityQueryLatencyFastValues) {
1367
state.SetMetricsForGeneratedCode(&c.Context, nil, map[string]int64{
1378
- "analyze_plan_cache": values.Analyze_plan_cache,
1379
- "count_active_jobs": values.Count_active_jobs,
1368
"count_disks": values.Count_disks,
1369
"count_http_servers": values.Count_http_servers,
1382
- "count_job_queues": values.Count_job_queues,
1383
- "count_message_queues": values.Count_message_queues,
1370
"count_network_interfaces": values.Count_network_interfaces,
1385
- "count_output_queues": values.Count_output_queues,
1386
- "count_subsystems": values.Count_subsystems,
1371
"detect_ibmi_version_primary": values.Detect_ibmi_version_primary,
1372
"detect_ibmi_version_fallback": values.Detect_ibmi_version_fallback,
1373
"disk_instances": values.Disk_instances,
@@ -1391,264 +1375,350 @@ func (c ObservabilityQueryLatencyContext) Set(state *framework.CollectorState, l
1375
"disk_status": values.Disk_status,
1376
"http_server_info": values.Http_server_info,
1377
"job_info": values.Job_info,
1394
- "job_queues": values.Job_queues,
1378
"memory_pools": values.Memory_pools,
1396
- "message_queue_aggregates": values.Message_queue_aggregates,
1379
"network_connections": values.Network_connections,
1380
"network_interfaces": values.Network_interfaces,
1399
- "output_queue_info": values.Output_queue_info,
1400
- "plan_cache_summary": values.Plan_cache_summary,
1381
"serial_number": values.Serial_number,
1382
+ "system_name": values.System_name,
1383
"system_activity": values.System_activity,
1384
"system_model": values.System_model,
1385
"system_status": values.System_status,
1386
"temp_storage_named": values.Temp_storage_named,
1387
"temp_storage_total": values.Temp_storage_total,
1388
"technology_refresh_level": values.Technology_refresh_level,
1408
- "top_active_jobs": values.Top_active_jobs,
1409
- "other": values.Other,
1389
+ "active_job": values.Active_job,
1390
})
1391
}
1392
1393
// SetUpdateEvery sets the update interval for this instance
1414
-func (c ObservabilityQueryLatencyContext) SetUpdateEvery(state *framework.CollectorState, labels EmptyLabels, updateEvery int) {
1394
+func (c ObservabilityQueryLatencyFastContext) SetUpdateEvery(state *framework.CollectorState, labels EmptyLabels, updateEvery int) {
1395
+ state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, nil, updateEvery)
1396
+}
1397
+
1398
+// ObservabilityQueryLatencySlowValues defines the type-safe values for Observability.QueryLatencySlow context
1399
+type ObservabilityQueryLatencySlowValues struct {
1400
+ Analyze_plan_cache int64
1401
+ Count_subsystems int64
1402
+ Subsystems int64
1403
+ Message_queue_aggregates int64
1404
+ Job_queues int64
1405
+ Output_queue_info int64
1406
+ Plan_cache_summary int64
1407
+}
1408
+
1409
+// ObservabilityQueryLatencySlowContext provides type-safe operations for Observability.QueryLatencySlow context
1410
+type ObservabilityQueryLatencySlowContext struct {
1411
+ framework.Context[EmptyLabels]
1412
+}
1413
+
1414
+// Set provides type-safe dimension setting for Observability.QueryLatencySlow context
1415
+func (c ObservabilityQueryLatencySlowContext) Set(state *framework.CollectorState, labels EmptyLabels, values ObservabilityQueryLatencySlowValues) {
1416
+ state.SetMetricsForGeneratedCode(&c.Context, nil, map[string]int64{
1417
+ "analyze_plan_cache": values.Analyze_plan_cache,
1418
+ "count_subsystems": values.Count_subsystems,
1419
+ "subsystems": values.Subsystems,
1420
+ "message_queue_aggregates": values.Message_queue_aggregates,
1421
+ "job_queues": values.Job_queues,
1422
+ "output_queue_info": values.Output_queue_info,
1423
+ "plan_cache_summary": values.Plan_cache_summary,
1424
+ })
1425
+}
1426
+
1427
+// SetUpdateEvery sets the update interval for this instance
1428
+func (c ObservabilityQueryLatencySlowContext) SetUpdateEvery(state *framework.CollectorState, labels EmptyLabels, updateEvery int) {
1429
+ state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, nil, updateEvery)
1430
+}
1431
+
1432
+// ObservabilityQueryLatencyBatchValues defines the type-safe values for Observability.QueryLatencyBatch context
1433
+type ObservabilityQueryLatencyBatchValues struct {
1434
+ Message_queue_totals int64
1435
+ Job_queue_totals int64
1436
+ Output_queue_totals int64
1437
+}
1438
+
1439
+// ObservabilityQueryLatencyBatchContext provides type-safe operations for Observability.QueryLatencyBatch context
1440
+type ObservabilityQueryLatencyBatchContext struct {
1441
+ framework.Context[EmptyLabels]
1442
+}
1443
+
1444
+// Set provides type-safe dimension setting for Observability.QueryLatencyBatch context
1445
+func (c ObservabilityQueryLatencyBatchContext) Set(state *framework.CollectorState, labels EmptyLabels, values ObservabilityQueryLatencyBatchValues) {
1446
+ state.SetMetricsForGeneratedCode(&c.Context, nil, map[string]int64{
1447
+ "message_queue_totals": values.Message_queue_totals,
1448
+ "job_queue_totals": values.Job_queue_totals,
1449
+ "output_queue_totals": values.Output_queue_totals,
1450
+ })
1451
+}
1452
+
1453
+// SetUpdateEvery sets the update interval for this instance
1454
+func (c ObservabilityQueryLatencyBatchContext) SetUpdateEvery(state *framework.CollectorState, labels EmptyLabels, updateEvery int) {
1455
state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, nil, updateEvery)
1456
}
1457
1458
// Observability contains all metric contexts for Observability
1459
var Observability = struct {
1420
- QueryLatency ObservabilityQueryLatencyContext
1460
+ QueryLatencyFast ObservabilityQueryLatencyFastContext
1461
+ QueryLatencySlow ObservabilityQueryLatencySlowContext
1462
+ QueryLatencyBatch ObservabilityQueryLatencyBatchContext
1463
}{
1422
- QueryLatency: ObservabilityQueryLatencyContext{
1464
+ QueryLatencyFast: ObservabilityQueryLatencyFastContext{
1465
Context: framework.Context[EmptyLabels]{
1424
- Name: "netdata.plugin_ibm.as400_query_latency",
1466
+ Name: "netdata.plugin_ibm.as400_query_latency_fast",
1467
Family: "plugins/ibm.d/latency",
1426
- Title: "AS400 Query Latency",
1468
+ Title: "AS400 Query Latency (Fast Path)",
1469
Units: "ms",
1470
Type: module.Stacked,
1471
Priority: 146000,
1472
UpdateEvery: 1,
1473
Dimensions: []framework.Dimension{
1474
{
1433
- Name: "analyze_plan_cache",
1434
- Algorithm: module.Absolute,
1475
+ Name: "count_disks",
1476
+ Algorithm: module.Incremental,
1477
Mul: 1,
1478
Div: 1000,
1479
Precision: 1,
1480
},
1481
{
1440
- Name: "count_active_jobs",
1441
- Algorithm: module.Absolute,
1482
+ Name: "count_http_servers",
1483
+ Algorithm: module.Incremental,
1484
Mul: 1,
1485
Div: 1000,
1486
Precision: 1,
1487
},
1488
{
1447
- Name: "count_disks",
1448
- Algorithm: module.Absolute,
1489
+ Name: "count_network_interfaces",
1490
+ Algorithm: module.Incremental,
1491
Mul: 1,
1492
Div: 1000,
1493
Precision: 1,
1494
},
1495
{
1454
- Name: "count_http_servers",
1455
- Algorithm: module.Absolute,
1496
+ Name: "detect_ibmi_version_primary",
1497
+ Algorithm: module.Incremental,
1498
Mul: 1,
1499
Div: 1000,
1500
Precision: 1,
1501
},
1502
{
1461
- Name: "count_job_queues",
1462
- Algorithm: module.Absolute,
1503
+ Name: "detect_ibmi_version_fallback",
1504
+ Algorithm: module.Incremental,
1505
Mul: 1,
1506
Div: 1000,
1507
Precision: 1,
1508
},
1509
{
1468
- Name: "count_message_queues",
1469
- Algorithm: module.Absolute,
1510
+ Name: "disk_instances",
1511
+ Algorithm: module.Incremental,
1512
Mul: 1,
1513
Div: 1000,
1514
Precision: 1,
1515
},
1516
{
1475
- Name: "count_network_interfaces",
1476
- Algorithm: module.Absolute,
1517
+ Name: "disk_instances_enhanced",
1518
+ Algorithm: module.Incremental,
1519
Mul: 1,
1520
Div: 1000,
1521
Precision: 1,
1522
},
1523
{
1482
- Name: "count_output_queues",
1483
- Algorithm: module.Absolute,
1524
+ Name: "disk_status",
1525
+ Algorithm: module.Incremental,
1526
Mul: 1,
1527
Div: 1000,
1528
Precision: 1,
1529
},
1530
{
1489
- Name: "count_subsystems",
1490
- Algorithm: module.Absolute,
1531
+ Name: "http_server_info",
1532
+ Algorithm: module.Incremental,
1533
Mul: 1,
1534
Div: 1000,
1535
Precision: 1,
1536
},
1537
{
1496
- Name: "detect_ibmi_version_primary",
1497
- Algorithm: module.Absolute,
1538
+ Name: "job_info",
1539
+ Algorithm: module.Incremental,
1540
Mul: 1,
1541
Div: 1000,
1542
Precision: 1,
1543
},
1544
{
1503
- Name: "detect_ibmi_version_fallback",
1504
- Algorithm: module.Absolute,
1545
+ Name: "memory_pools",
1546
+ Algorithm: module.Incremental,
1547
Mul: 1,
1548
Div: 1000,
1549
Precision: 1,
1550
},
1551
{
1510
- Name: "disk_instances",
1511
- Algorithm: module.Absolute,
1552
+ Name: "network_connections",
1553
+ Algorithm: module.Incremental,
1554
Mul: 1,
1555
Div: 1000,
1556
Precision: 1,
1557
},
1558
{
1517
- Name: "disk_instances_enhanced",
1518
- Algorithm: module.Absolute,
1559
+ Name: "network_interfaces",
1560
+ Algorithm: module.Incremental,
1561
Mul: 1,
1562
Div: 1000,
1563
Precision: 1,
1564
},
1565
{
1524
- Name: "disk_status",
1525
- Algorithm: module.Absolute,
1566
+ Name: "serial_number",
1567
+ Algorithm: module.Incremental,
1568
Mul: 1,
1569
Div: 1000,
1570
Precision: 1,
1571
},
1572
{
1531
- Name: "http_server_info",
1532
- Algorithm: module.Absolute,
1573
+ Name: "system_name",
1574
+ Algorithm: module.Incremental,
1575
Mul: 1,
1576
Div: 1000,
1577
Precision: 1,
1578
},
1579
{
1538
- Name: "job_info",
1539
- Algorithm: module.Absolute,
1580
+ Name: "system_activity",
1581
+ Algorithm: module.Incremental,
1582
Mul: 1,
1583
Div: 1000,
1584
Precision: 1,
1585
},
1586
{
1545
- Name: "job_queues",
1546
- Algorithm: module.Absolute,
1587
+ Name: "system_model",
1588
+ Algorithm: module.Incremental,
1589
Mul: 1,
1590
Div: 1000,
1591
Precision: 1,
1592
},
1593
{
1552
- Name: "memory_pools",
1553
- Algorithm: module.Absolute,
1594
+ Name: "system_status",
1595
+ Algorithm: module.Incremental,
1596
Mul: 1,
1597
Div: 1000,
1598
Precision: 1,
1599
},
1600
{
1559
- Name: "message_queue_aggregates",
1560
- Algorithm: module.Absolute,
1601
+ Name: "temp_storage_named",
1602
+ Algorithm: module.Incremental,
1603
Mul: 1,
1604
Div: 1000,
1605
Precision: 1,
1606
},
1607
{
1566
- Name: "network_connections",
1567
- Algorithm: module.Absolute,
1608
+ Name: "temp_storage_total",
1609
+ Algorithm: module.Incremental,
1610
Mul: 1,
1611
Div: 1000,
1612
Precision: 1,
1613
},
1614
{
1573
- Name: "network_interfaces",
1574
- Algorithm: module.Absolute,
1615
+ Name: "technology_refresh_level",
1616
+ Algorithm: module.Incremental,
1617
Mul: 1,
1618
Div: 1000,
1619
Precision: 1,
1620
},
1621
{
1580
- Name: "output_queue_info",
1581
- Algorithm: module.Absolute,
1622
+ Name: "active_job",
1623
+ Algorithm: module.Incremental,
1624
Mul: 1,
1625
Div: 1000,
1626
Precision: 1,
1627
},
1628
+ },
1629
+ LabelKeys: []string{},
1630
+ },
1631
+ },
1632
+ QueryLatencySlow: ObservabilityQueryLatencySlowContext{
1633
+ Context: framework.Context[EmptyLabels]{
1634
+ Name: "netdata.plugin_ibm.as400_query_latency_slow",
1635
+ Family: "plugins/ibm.d/latency",
1636
+ Title: "AS400 Query Latency (Slow Path)",
1637
+ Units: "ms",
1638
+ Type: module.Stacked,
1639
+ Priority: 146010,
1640
+ UpdateEvery: 1,
1641
+ Dimensions: []framework.Dimension{
1642
{
1587
- Name: "plan_cache_summary",
1588
- Algorithm: module.Absolute,
1643
+ Name: "analyze_plan_cache",
1644
+ Algorithm: module.Incremental,
1645
Mul: 1,
1646
Div: 1000,
1647
Precision: 1,
1648
},
1649
{
1594
- Name: "serial_number",
1595
- Algorithm: module.Absolute,
1650
+ Name: "count_subsystems",
1651
+ Algorithm: module.Incremental,
1652
Mul: 1,
1653
Div: 1000,
1654
Precision: 1,
1655
},
1656
{
1601
- Name: "system_activity",
1602
- Algorithm: module.Absolute,
1657
+ Name: "subsystems",
1658
+ Algorithm: module.Incremental,
1659
Mul: 1,
1660
Div: 1000,
1661
Precision: 1,
1662
},
1663
{
1608
- Name: "system_model",
1609
- Algorithm: module.Absolute,
1664
+ Name: "message_queue_aggregates",
1665
+ Algorithm: module.Incremental,
1666
Mul: 1,
1667
Div: 1000,
1668
Precision: 1,
1669
},
1670
{
1615
- Name: "system_status",
1616
- Algorithm: module.Absolute,
1671
+ Name: "job_queues",
1672
+ Algorithm: module.Incremental,
1673
Mul: 1,
1674
Div: 1000,
1675
Precision: 1,
1676
},
1677
{
1622
- Name: "temp_storage_named",
1623
- Algorithm: module.Absolute,
1678
+ Name: "output_queue_info",
1679
+ Algorithm: module.Incremental,
1680
Mul: 1,
1681
Div: 1000,
1682
Precision: 1,
1683
},
1684
{
1629
- Name: "temp_storage_total",
1630
- Algorithm: module.Absolute,
1685
+ Name: "plan_cache_summary",
1686
+ Algorithm: module.Incremental,
1687
Mul: 1,
1688
Div: 1000,
1689
Precision: 1,
1690
},
1691
+ },
1692
+ LabelKeys: []string{},
1693
+ },
1694
+ },
1695
+ QueryLatencyBatch: ObservabilityQueryLatencyBatchContext{
1696
+ Context: framework.Context[EmptyLabels]{
1697
+ Name: "netdata.plugin_ibm.as400_query_latency_batch",
1698
+ Family: "plugins/ibm.d/latency",
1699
+ Title: "AS400 Query Latency (Batch Path)",
1700
+ Units: "ms",
1701
+ Type: module.Stacked,
1702
+ Priority: 146020,
1703
+ UpdateEvery: 1,
1704
+ Dimensions: []framework.Dimension{
1705
{
1636
- Name: "technology_refresh_level",
1637
- Algorithm: module.Absolute,
1706
+ Name: "message_queue_totals",
1707
+ Algorithm: module.Incremental,
1708
Mul: 1,
1709
Div: 1000,
1710
Precision: 1,
1711
},
1712
{
1643
- Name: "top_active_jobs",
1644
- Algorithm: module.Absolute,
1713
+ Name: "job_queue_totals",
1714
+ Algorithm: module.Incremental,
1715
Mul: 1,
1716
Div: 1000,
1717
Precision: 1,
1718
},
1719
{
1650
- Name: "other",
1651
- Algorithm: module.Absolute,
1720
+ Name: "output_queue_totals",
1721
+ Algorithm: module.Incremental,
1722
Mul: 1,
1723
Div: 1000,
1724
Precision: 1,
@@ -1749,7 +1819,7 @@ var OutputQueue = struct {
1819
Files: OutputQueueFilesContext{
1820
Context: framework.Context[OutputQueueLabels]{
1821
Name: "as400.output_queue_files",
1752
- Family: "messaging/output_queues",
1822
+ Family: "queues/output",
1823
Title: "Output Queue Files",
1824
Units: "files",
1825
Type: module.Line,
@@ -1774,7 +1844,7 @@ var OutputQueue = struct {
1844
Writers: OutputQueueWritersContext{
1845
Context: framework.Context[OutputQueueLabels]{
1846
Name: "as400.output_queue_writers",
1777
- Family: "messaging/output_queues",
1847
+ Family: "queues/output",
1848
Title: "Output Queue Writers",
1849
Units: "writers",
1850
Type: module.Line,
@@ -1799,7 +1869,7 @@ var OutputQueue = struct {
1869
Status: OutputQueueStatusContext{
1870
Context: framework.Context[OutputQueueLabels]{
1871
Name: "as400.output_queue_status",
1802
- Family: "messaging/output_queues",
1872
+ Family: "queues/output",
1873
Title: "Output Queue Status",
1874
Units: "state",
1875
Type: module.Line,
@@ -1887,6 +1957,119 @@ var PlanCache = struct {
1957
},
1958
}
1959
1960
+// --- QueueOverview ---
1961
+
1962
+// QueueOverviewCountValues defines the type-safe values for QueueOverview.Count context
1963
+type QueueOverviewCountValues struct {
1964
+ Queues int64
1965
+}
1966
+
1967
+// QueueOverviewCountContext provides type-safe operations for QueueOverview.Count context
1968
+type QueueOverviewCountContext struct {
1969
+ framework.Context[QueueOverviewLabels]
1970
+}
1971
+
1972
+// Set provides type-safe dimension setting for QueueOverview.Count context
1973
+func (c QueueOverviewCountContext) Set(state *framework.CollectorState, labels QueueOverviewLabels, values QueueOverviewCountValues) {
1974
+ state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{
1975
+ "queues": values.Queues,
1976
+ })
1977
+}
1978
+
1979
+// SetUpdateEvery sets the update interval for this instance
1980
+func (c QueueOverviewCountContext) SetUpdateEvery(state *framework.CollectorState, labels QueueOverviewLabels, updateEvery int) {
1981
+ state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)
1982
+}
1983
+
1984
+// QueueOverviewItemsValues defines the type-safe values for QueueOverview.Items context
1985
+type QueueOverviewItemsValues struct {
1986
+ Items int64
1987
+}
1988
+
1989
+// QueueOverviewItemsContext provides type-safe operations for QueueOverview.Items context
1990
+type QueueOverviewItemsContext struct {
1991
+ framework.Context[QueueOverviewLabels]
1992
+}
1993
+
1994
+// Set provides type-safe dimension setting for QueueOverview.Items context
1995
+func (c QueueOverviewItemsContext) Set(state *framework.CollectorState, labels QueueOverviewLabels, values QueueOverviewItemsValues) {
1996
+ state.SetMetricsForGeneratedCode(&c.Context, labels, map[string]int64{
1997
+ "items": values.Items,
1998
+ })
1999
+}
2000
+
2001
+// SetUpdateEvery sets the update interval for this instance
2002
+func (c QueueOverviewItemsContext) SetUpdateEvery(state *framework.CollectorState, labels QueueOverviewLabels, updateEvery int) {
2003
+ state.SetUpdateEveryOverrideForGeneratedCode(&c.Context, labels, updateEvery)
2004
+}
2005
+
2006
+// QueueOverviewLabels defines the required labels for QueueOverview contexts
2007
+type QueueOverviewLabels struct {
2008
+ Queue_type string
2009
+ Item_type string
2010
+}
2011
+
2012
+// InstanceID generates a unique instance ID using the hardcoded label order from YAML
2013
+func (l QueueOverviewLabels) InstanceID(contextName string) string {
2014
+ // Label order from YAML: queue_type, item_type
2015
+ return contextName + "." + cleanLabelValue(l.Queue_type) + "_" + cleanLabelValue(l.Item_type)
2016
+}
2017
+
2018
+// QueueOverview contains all metric contexts for QueueOverview
2019
+var QueueOverview = struct {
2020
+ Count QueueOverviewCountContext
2021
+ Items QueueOverviewItemsContext
2022
+}{
2023
+ Count: QueueOverviewCountContext{
2024
+ Context: framework.Context[QueueOverviewLabels]{
2025
+ Name: "as400.queues_count",
2026
+ Family: "queues/overview",
2027
+ Title: "Queue Counts",
2028
+ Units: "queues",
2029
+ Type: module.Line,
2030
+ Priority: 504,
2031
+ UpdateEvery: 1,
2032
+ Dimensions: []framework.Dimension{
2033
+ {
2034
+ Name: "queues",
2035
+ Algorithm: module.Absolute,
2036
+ Mul: 1,
2037
+ Div: 1,
2038
+ Precision: 1,
2039
+ },
2040
+ },
2041
+ LabelKeys: []string{
2042
+ "queue_type",
2043
+ "item_type",
2044
+ },
2045
+ },
2046
+ },
2047
+ Items: QueueOverviewItemsContext{
2048
+ Context: framework.Context[QueueOverviewLabels]{
2049
+ Name: "as400.queued_items",
2050
+ Family: "queues/overview",
2051
+ Title: "Queued Items",
2052
+ Units: "items",
2053
+ Type: module.Line,
2054
+ Priority: 505,
2055
+ UpdateEvery: 1,
2056
+ Dimensions: []framework.Dimension{
2057
+ {
2058
+ Name: "items",
2059
+ Algorithm: module.Absolute,
2060
+ Mul: 1,
2061
+ Div: 1,
2062
+ Precision: 1,
2063
+ },
2064
+ },
2065
+ LabelKeys: []string{
2066
+ "queue_type",
2067
+ "item_type",
2068
+ },
2069
+ },
2070
+ },
2071
+}
2072
+
2073
// --- Subsystem ---
2074
2075
// SubsystemJobsValues defines the type-safe values for Subsystem.Jobs context
@@ -3268,11 +3451,15 @@ func GetAllContexts() []interface{} {
3451
&MessageQueue.Severity.Context,
3452
&NetworkInterface.Status.Context,
3453
&NetworkInterface.MTU.Context,
3271
- &Observability.QueryLatency.Context,
3454
+ &Observability.QueryLatencyFast.Context,
3455
+ &Observability.QueryLatencySlow.Context,
3456
+ &Observability.QueryLatencyBatch.Context,
3457
&OutputQueue.Files.Context,
3458
&OutputQueue.Writers.Context,
3459
&OutputQueue.Status.Context,
3460
&PlanCache.Summary.Context,
3461
+ &QueueOverview.Count.Context,
3462
+ &QueueOverview.Items.Context,
3463
&Subsystem.Jobs.Context,
3464
&System.CPUUtilization.Context,
3465
&System.CPUEntitledUtilization.Context,
src/go/plugin/ibm.d/modules/as400/groups.go
+73
-30
@@ -31,56 +31,68 @@ func (g *systemGroup) Collect(ctx context.Context) error {
31
if isSQLTemporaryError(err) {
32
c.Debugf("system status collection failed with temporary error, will show gaps: %v", err)
33
} else {
34
- c.Errorf("failed to collect system status: %v", err)
34
+ c.logErrorOnce("system_status_error", "failed to collect system status: %s", trimDriverMessage(err))
35
}
36
+ } else {
37
+ c.clearErrorOnce("system_status_error")
38
}
39
40
if err := c.collectMemoryPools(ctx); err != nil {
41
if isSQLTemporaryError(err) {
42
c.Debugf("memory pools collection failed with temporary error, will show gaps: %v", err)
43
} else {
42
- c.Errorf("failed to collect memory pools: %v", err)
44
+ c.logErrorOnce("memory_pools_error", "failed to collect memory pools: %s", trimDriverMessage(err))
45
}
46
+ } else {
47
+ c.clearErrorOnce("memory_pools_error")
48
}
49
50
if err := c.collectDiskStatus(ctx); err != nil {
51
if isSQLFeatureError(err) {
48
- c.Warningf("disk status monitoring not available on this IBM i version: %v", err)
52
+ c.logOnce("disk_status_unavailable", "disk status monitoring not available on this IBM i version: %v", err)
53
} else if isSQLTemporaryError(err) {
54
c.Debugf("disk status collection failed with temporary error, will show gaps: %v", err)
55
} else {
52
- c.Errorf("failed to collect disk status: %v", err)
56
+ c.logErrorOnce("disk_status_error", "failed to collect disk status: %s", trimDriverMessage(err))
57
}
58
+ } else {
59
+ c.clearErrorOnce("disk_status_error")
60
}
61
62
if err := c.collectJobInfo(ctx); err != nil {
63
if isSQLFeatureError(err) {
58
- c.Warningf("job info monitoring not available on this IBM i version: %v", err)
64
+ c.logOnce("job_info_unavailable", "job info monitoring not available on this IBM i version: %v", err)
65
} else if isSQLTemporaryError(err) {
66
c.Debugf("job info collection failed with temporary error, will show gaps: %v", err)
67
} else {
62
- c.Errorf("failed to collect job info: %v", err)
68
+ c.logErrorOnce("job_info_error", "failed to collect job info: %s", trimDriverMessage(err))
69
}
70
+ } else {
71
+ c.clearErrorOnce("job_info_error")
72
}
73
74
if err := c.collectNetworkConnections(ctx); err != nil {
75
if isSQLFeatureError(err) {
68
- c.Warningf("network connections monitoring not available on this IBM i version: %v", err)
76
+ c.logOnce("network_connections_unavailable", "network connections monitoring not available on this IBM i version: %v", err)
77
} else if isSQLTemporaryError(err) {
78
c.Debugf("network connections collection failed with temporary error, will show gaps: %v", err)
79
} else {
72
- c.Errorf("failed to collect network connections: %v", err)
80
+ c.logErrorOnce("network_connections_error", "failed to collect network connections: %s", trimDriverMessage(err))
81
}
82
+ } else {
83
+ c.clearErrorOnce("network_connections_error")
84
}
85
86
if err := c.collectTempStorage(ctx); err != nil {
87
if isSQLFeatureError(err) {
78
- c.Warningf("temporary storage monitoring not available on this IBM i version: %v", err)
88
+ c.logOnce("temp_storage_unavailable", "temporary storage monitoring not available on this IBM i version: %v", err)
89
} else if isSQLTemporaryError(err) {
90
c.Debugf("temporary storage collection failed with temporary error, will show gaps: %v", err)
91
} else {
82
- c.Errorf("failed to collect temporary storage: %v", err)
92
+ c.logErrorOnce("temp_storage_error", "failed to collect temporary storage: %s", trimDriverMessage(err))
93
}
94
+ } else {
95
+ c.clearErrorOnce("temp_storage_error")
96
}
97
98
return nil
@@ -104,8 +116,10 @@ func (g *subsystemGroup) Collect(ctx context.Context) error {
116
g.c.Warningf("subsystem monitoring not available on this IBM i version: %v", err)
117
return nil
118
}
107
- g.c.Errorf("failed to collect subsystems: %v", err)
119
+ g.c.logErrorOnce("subsystems_error", "failed to collect subsystems: %s", trimDriverMessage(err))
120
+ return nil
121
}
122
+ g.c.clearErrorOnce("subsystems_error")
123
return nil
124
}
125
@@ -115,7 +129,7 @@ type jobQueueGroup struct {
129
130
func (g *jobQueueGroup) Name() string { return "job_queues" }
131
func (g *jobQueueGroup) Enabled() bool {
118
- return g.c.CollectJobQueueMetrics.IsEnabled()
132
+ return len(g.c.jobQueueTargets) > 0
133
}
134
135
func (g *jobQueueGroup) Collect(ctx context.Context) error {
@@ -127,8 +141,10 @@ func (g *jobQueueGroup) Collect(ctx context.Context) error {
141
g.c.Warningf("job queue monitoring not available on this IBM i version: %v", err)
142
return nil
143
}
130
- g.c.Errorf("failed to collect job queues: %v", err)
144
+ g.c.logErrorOnce("job_queues_error", "failed to collect job queues: %s", trimDriverMessage(err))
145
+ return nil
146
}
147
+ g.c.clearErrorOnce("job_queues_error")
148
return nil
149
}
150
@@ -138,7 +154,7 @@ type messageQueueGroup struct {
154
155
func (g *messageQueueGroup) Name() string { return "message_queues" }
156
func (g *messageQueueGroup) Enabled() bool {
141
- return g.c.CollectMessageQueueMetrics.IsEnabled()
157
+ return len(g.c.messageQueueTargets) > 0
158
}
159
160
func (g *messageQueueGroup) Collect(ctx context.Context) error {
@@ -148,15 +164,16 @@ func (g *messageQueueGroup) Collect(ctx context.Context) error {
164
if err := g.c.collectMessageQueues(ctx); err != nil {
165
if isSQLFeatureError(err) {
166
g.c.Warningf("message queue metrics not available on this IBM i version: %v", err)
151
- g.c.CollectMessageQueueMetrics = confopt.AutoBoolDisabled
167
return nil
168
}
169
if isSQLTemporaryError(err) {
170
g.c.Debugf("message queue collection failed with temporary error, will show gaps: %v", err)
171
return nil
172
}
158
- g.c.Errorf("failed to collect message queues: %v", err)
173
+ g.c.logErrorOnce("message_queues_error", "failed to collect message queues: %s", trimDriverMessage(err))
174
+ return nil
175
}
176
+ g.c.clearErrorOnce("message_queues_error")
177
return nil
178
}
179
@@ -166,7 +183,7 @@ type outputQueueGroup struct {
183
184
func (g *outputQueueGroup) Name() string { return "output_queues" }
185
func (g *outputQueueGroup) Enabled() bool {
169
- return g.c.CollectOutputQueueMetrics.IsEnabled()
186
+ return len(g.c.outputQueueTargets) > 0
187
}
188
189
func (g *outputQueueGroup) Collect(ctx context.Context) error {
@@ -176,15 +193,16 @@ func (g *outputQueueGroup) Collect(ctx context.Context) error {
193
if err := g.c.collectOutputQueues(ctx); err != nil {
194
if isSQLFeatureError(err) {
195
g.c.Warningf("output queue metrics not available on this IBM i version: %v", err)
179
- g.c.CollectOutputQueueMetrics = confopt.AutoBoolDisabled
196
return nil
197
}
198
if isSQLTemporaryError(err) {
199
g.c.Debugf("output queue collection failed with temporary error, will show gaps: %v", err)
200
return nil
201
}
186
- g.c.Errorf("failed to collect output queues: %v", err)
202
+ g.c.logErrorOnce("output_queues_error", "failed to collect output queues: %s", trimDriverMessage(err))
203
+ return nil
204
}
205
+ g.c.clearErrorOnce("output_queues_error")
206
return nil
207
}
208
@@ -203,13 +221,18 @@ func (g *diskGroup) Collect(ctx context.Context) error {
221
}
222
if err := g.c.collectDiskInstancesEnhanced(ctx); err != nil {
223
if isSQLFeatureError(err) {
206
- g.c.Warningf("enhanced disk metrics not available, using basic collection: %v", err)
224
+ g.c.logOnce("disk_enhanced_unavailable", "enhanced disk metrics not available, using basic collection: %v", err)
225
if basicErr := g.c.collectDiskInstances(ctx); basicErr != nil {
208
- g.c.Errorf("failed to collect disk instances: %v", basicErr)
226
+ g.c.logErrorOnce("disk_instances_error", "failed to collect disk instances: %s", trimDriverMessage(basicErr))
227
}
228
+ } else if isSQLTemporaryError(err) {
229
+ g.c.Debugf("disk instances collection failed with temporary error, will show gaps: %v", err)
230
} else {
211
- g.c.Errorf("failed to collect enhanced disk instances: %v", err)
231
+ g.c.logErrorOnce("disk_instances_enhanced_error", "failed to collect enhanced disk instances: %s", trimDriverMessage(err))
232
}
233
+ } else {
234
+ g.c.clearErrorOnce("disk_instances_enhanced_error")
235
+ g.c.clearErrorOnce("disk_instances_error")
236
}
237
return nil
238
}
@@ -220,7 +243,7 @@ type activeJobGroup struct {
243
244
func (g *activeJobGroup) Name() string { return "active_jobs" }
245
func (g *activeJobGroup) Enabled() bool {
223
- return g.c.CollectActiveJobs.IsEnabled()
246
+ return g.c.CollectActiveJobs.IsEnabled() && len(g.c.activeJobTargets) > 0
247
}
248
249
func (g *activeJobGroup) Collect(ctx context.Context) error {
@@ -233,8 +256,14 @@ func (g *activeJobGroup) Collect(ctx context.Context) error {
256
g.c.CollectActiveJobs = confopt.AutoBoolDisabled
257
return nil
258
}
236
- g.c.Errorf("failed to collect active jobs: %v", err)
259
+ if isSQLTemporaryError(err) {
260
+ g.c.Debugf("active job collection failed with temporary error, will show gaps: %v", err)
261
+ return nil
262
+ }
263
+ g.c.logErrorOnce("active_jobs_error", "failed to collect active jobs: %s", trimDriverMessage(err))
264
+ return nil
265
}
266
+ g.c.clearErrorOnce("active_jobs_error")
267
return nil
268
}
269
@@ -251,8 +280,14 @@ func (g *networkInterfaceGroup) Collect(ctx context.Context) error {
280
g.c.Warningf("network interface monitoring not available on this IBM i version: %v", err)
281
return nil
282
}
254
- g.c.Errorf("failed to collect network interfaces: %v", err)
283
+ if isSQLTemporaryError(err) {
284
+ g.c.Debugf("network interface collection failed with temporary error, will show gaps: %v", err)
285
+ return nil
286
+ }
287
+ g.c.logErrorOnce("network_interfaces_error", "failed to collect network interfaces: %s", trimDriverMessage(err))
288
+ return nil
289
}
290
+ g.c.clearErrorOnce("network_interfaces_error")
291
return nil
292
}
293
@@ -266,15 +301,17 @@ func (g *systemActivityGroup) Enabled() bool { return true }
301
func (g *systemActivityGroup) Collect(ctx context.Context) error {
302
if err := g.c.collectSystemActivity(ctx); err != nil {
303
if isSQLFeatureError(err) {
269
- g.c.Warningf("system activity monitoring not available on this IBM i version: %v", err)
304
+ g.c.logOnce("system_activity_unavailable", "system activity monitoring not available on this IBM i version: %v", err)
305
return nil
306
}
307
if isSQLTemporaryError(err) {
308
g.c.Debugf("system activity collection failed with temporary error, will show gaps: %v", err)
309
return nil
310
}
276
- g.c.Errorf("failed to collect system activity: %v", err)
311
+ g.c.logErrorOnce("system_activity_error", "failed to collect system activity: %s", trimDriverMessage(err))
312
+ return nil
313
}
314
+ g.c.clearErrorOnce("system_activity_error")
315
return nil
316
}
317
@@ -301,8 +338,10 @@ func (g *httpServerGroup) Collect(ctx context.Context) error {
338
g.c.Debugf("HTTP server metrics collection failed with temporary error, will show gaps: %v", err)
339
return nil
340
}
304
- g.c.Errorf("failed to collect HTTP server metrics: %v", err)
341
+ g.c.logErrorOnce("http_server_error", "failed to collect HTTP server metrics: %s", trimDriverMessage(err))
342
+ return nil
343
}
344
+ g.c.clearErrorOnce("http_server_error")
345
return nil
346
}
347
@@ -321,7 +360,8 @@ func (g *planCacheGroup) Collect(ctx context.Context) error {
360
}
361
if err := g.c.collectPlanCache(ctx); err != nil {
362
if isSQLFeatureError(err) {
324
- g.c.Warningf("plan cache analysis not available or requires additional authority: %v", err)
363
+ tmsg := trimDriverMessage(err)
364
+ g.c.logErrorOnce("plan_cache_missing_privs", "plan cache analysis not available or requires additional authority: %s", tmsg)
365
g.c.CollectPlanCacheMetrics = confopt.AutoBoolDisabled
366
return nil
367
}
@@ -329,7 +369,10 @@ func (g *planCacheGroup) Collect(ctx context.Context) error {
369
g.c.Debugf("plan cache metrics collection failed with temporary error, will show gaps: %v", err)
370
return nil
371
}
332
- g.c.Errorf("failed to collect plan cache metrics: %v", err)
372
+ g.c.logErrorOnce("plan_cache_error", "failed to collect plan cache metrics: %s", trimDriverMessage(err))
373
+ return nil
374
}
375
+ g.c.clearErrorOnce("plan_cache_missing_privs")
376
+ g.c.clearErrorOnce("plan_cache_error")
377
return nil
378
}
src/go/plugin/ibm.d/modules/as400/helpers.go
+77
-13
@@ -34,6 +34,25 @@ func (c *Collector) logOnce(key string, format string, args ...interface{}) {
34
c.disabled[key] = true
35
}
36
37
+func (c *Collector) logErrorOnce(key string, format string, args ...interface{}) {
38
+ c.muErrorLog.Lock()
39
+ defer c.muErrorLog.Unlock()
40
+
41
+ if c.errorLogged[key] {
42
+ return
43
+ }
44
+ msg := fmt.Sprintf(format, args...)
45
+ c.Errorf("[%s] %s", key, msg)
46
+ c.errorLogged[key] = true
47
+}
48
+
49
+func (c *Collector) clearErrorOnce(key string) {
50
+ c.muErrorLog.Lock()
51
+ defer c.muErrorLog.Unlock()
52
+
53
+ delete(c.errorLogged, key)
54
+}
55
+
56
func (c *Collector) isDisabled(key string) bool {
57
return c.disabled[key]
58
}
@@ -118,6 +137,38 @@ func (c *Collector) detectAvailableFeatures(ctx context.Context) {
137
c.disabled["ifs_object_statistics"] = true
138
}
139
140
+func trimDriverMessage(err error) string {
141
+ if err == nil {
142
+ return ""
143
+ }
144
+ msg := err.Error()
145
+ msg = strings.TrimRight(msg, "\x00")
146
+ msg = strings.TrimSpace(msg)
147
+ return msg
148
+}
149
+
150
+func sanitizeQuery(q string) string {
151
+ if q == "" {
152
+ return ""
153
+ }
154
+ q = strings.Join(strings.Fields(q), " ")
155
+ const limit = 240
156
+ if len(q) > limit {
157
+ return q[:limit-3] + "..."
158
+ }
159
+ return q
160
+}
161
+
162
+func (c *Collector) logQueryErrorOnce(key, query string, err error) {
163
+ msg := trimDriverMessage(err)
164
+ if query != "" {
165
+ query = sanitizeQuery(query)
166
+ c.logErrorOnce(key, "query failed (%s): %s", query, msg)
167
+ return
168
+ }
169
+ c.logErrorOnce(key, "%s", msg)
170
+}
171
+
172
func (c *Collector) collectSystemInfo(ctx context.Context) {
173
c.systemName = "Unknown"
174
@@ -126,6 +177,14 @@ func (c *Collector) collectSystemInfo(ctx context.Context) {
177
c.Debugf("detected serial number: %s", c.serialNumber)
178
})
179
180
+ _ = c.collectSingleMetric(ctx, "system_name_metric", querySystemName, func(value string) {
181
+ name := strings.TrimSpace(value)
182
+ if name != "" {
183
+ c.systemName = name
184
+ c.Debugf("detected system name: %s", c.systemName)
185
+ }
186
+ })
187
+
188
_ = c.collectSingleMetric(ctx, "system_model", querySystemModel, func(value string) {
189
c.model = strings.TrimSpace(value)
190
c.Debugf("detected system model: %s", c.model)
@@ -252,31 +311,26 @@ func (c *Collector) logVersionInformation() {
311
}
312
313
func (c *Collector) setConfigurationDefaults() {
255
- jobQueuesDefault := c.versionMajor >= 7 && c.versionRelease >= 2
314
c.CollectDiskMetrics = c.CollectDiskMetrics.WithDefault(true)
315
c.CollectSubsystemMetrics = c.CollectSubsystemMetrics.WithDefault(true)
258
- c.CollectJobQueueMetrics = c.CollectJobQueueMetrics.WithDefault(jobQueuesDefault)
259
- c.CollectActiveJobs = c.CollectActiveJobs.WithDefault(false)
260
- c.CollectMessageQueueMetrics = c.CollectMessageQueueMetrics.WithDefault(true)
261
- c.CollectOutputQueueMetrics = c.CollectOutputQueueMetrics.WithDefault(true)
316
c.CollectHTTPServerMetrics = c.CollectHTTPServerMetrics.WithDefault(true)
317
c.CollectPlanCacheMetrics = c.CollectPlanCacheMetrics.WithDefault(true)
318
265
- if c.MaxMessageQueues <= 0 {
266
- c.MaxMessageQueues = messageQueueLimit
319
+ if len(c.ActiveJobs) == 0 {
320
+ c.CollectActiveJobs = c.CollectActiveJobs.WithDefault(false)
321
+ } else {
322
+ c.CollectActiveJobs = c.CollectActiveJobs.WithDefault(true)
323
}
324
269
- if c.MaxOutputQueues <= 0 {
270
- c.MaxOutputQueues = outputQueueLimit
325
+ if c.MessageQueues == nil {
326
+ c.MessageQueues = append([]string{}, "QSYS/QSYSOPR", "QSYS/QSYSMSG", "QSYS/QHST")
327
}
328
273
- c.Infof("Configuration after defaults: DiskMetrics=%t, SubsystemMetrics=%t, JobQueueMetrics=%t, MessageQueues=%t, OutputQueues=%t, ActiveJobs=%t, HTTPServer=%t, PlanCache=%t",
329
+ c.Infof("Configuration after defaults: DiskMetrics=%t, SubsystemMetrics=%t, ActiveJobs=%t (configured=%d), HTTPServer=%t, PlanCache=%t",
330
c.CollectDiskMetrics.IsEnabled(),
331
c.CollectSubsystemMetrics.IsEnabled(),
276
- c.CollectJobQueueMetrics.IsEnabled(),
277
- c.CollectMessageQueueMetrics.IsEnabled(),
278
- c.CollectOutputQueueMetrics.IsEnabled(),
332
c.CollectActiveJobs.IsEnabled(),
333
+ len(c.ActiveJobs),
334
c.CollectHTTPServerMetrics.IsEnabled(),
335
c.CollectPlanCacheMetrics.IsEnabled())
336
}
@@ -301,3 +355,13 @@ func (c *Collector) systemActivityQuery() string {
355
}
356
return querySystemActivityNoReset
357
}
358
+
359
+func (c *Collector) supportsMessageQueueTableFunction() bool {
360
+ if c.versionMajor == 0 {
361
+ return false
362
+ }
363
+ if c.versionMajor > 7 {
364
+ return true
365
+ }
366
+ return c.versionMajor == 7 && c.versionRelease >= 4
367
+}
src/go/plugin/ibm.d/modules/as400/init.go
+43
-32
@@ -20,13 +20,11 @@ import (
20
func defaultConfig() Config {
21
return Config{
22
Config: framework.Config{
23
- UpdateEvery: 10,
23
+ UpdateEvery: 5,
24
},
25
- Vnode: "",
26
- DSN: "",
27
- Timeout: confopt.Duration(2 * time.Second),
28
- MaxDbConns: 1,
29
- MaxDbLifeTime: confopt.Duration(10 * time.Minute),
25
+ Vnode: "",
26
+ DSN: "",
27
+ Timeout: confopt.Duration(2 * time.Second),
28
29
Hostname: "",
30
Port: 8471,
@@ -38,25 +36,35 @@ func defaultConfig() Config {
36
UseSSL: false,
37
ResetStatistics: false,
38
41
- CollectDiskMetrics: confopt.AutoBoolAuto,
42
- CollectSubsystemMetrics: confopt.AutoBoolAuto,
43
- CollectJobQueueMetrics: confopt.AutoBoolAuto,
44
- CollectActiveJobs: confopt.AutoBoolAuto,
45
- CollectHTTPServerMetrics: confopt.AutoBoolAuto,
46
- CollectMessageQueueMetrics: confopt.AutoBoolAuto,
47
- CollectOutputQueueMetrics: confopt.AutoBoolAuto,
48
- CollectPlanCacheMetrics: confopt.AutoBoolAuto,
49
-
50
- MaxDisks: 100,
51
- MaxSubsystems: 100,
52
- MaxJobQueues: 100,
53
- MaxMessageQueues: 100,
54
- MaxOutputQueues: 100,
55
- MaxActiveJobs: 100,
39
+ CollectDiskMetrics: confopt.AutoBoolAuto,
40
+ CollectSubsystemMetrics: confopt.AutoBoolAuto,
41
+ CollectActiveJobs: confopt.AutoBoolAuto,
42
+ CollectHTTPServerMetrics: confopt.AutoBoolAuto,
43
+ CollectPlanCacheMetrics: confopt.AutoBoolAuto,
44
+ CollectMessageQueueTotals: confopt.AutoBoolAuto,
45
+ CollectJobQueueTotals: confopt.AutoBoolAuto,
46
+ CollectOutputQueueTotals: confopt.AutoBoolAuto,
47
+
48
+ SlowPath: true,
49
+ SlowPathUpdateEvery: confopt.Duration(10 * time.Second),
50
+ SlowPathMaxConnections: 1,
51
+ BatchPath: false,
52
+ BatchPathUpdateEvery: confopt.Duration(60 * time.Second),
53
+ BatchPathMaxConnections: 1,
54
+
55
+ MaxDisks: 100,
56
+ MaxSubsystems: 100,
57
58
DiskSelector: "",
59
SubsystemSelector: "",
59
- JobQueueSelector: "",
60
+ MessageQueues: []string{
61
+ "QSYS/QSYSOPR",
62
+ "QSYS/QSYSMSG",
63
+ "QSYS/QHST",
64
+ },
65
+ JobQueues: nil,
66
+ OutputQueues: nil,
67
+ ActiveJobs: nil,
68
}
69
}
70
@@ -98,8 +106,7 @@ func (c *Collector) Init(ctx context.Context) error {
106
clientCfg := as400proto.Config{
107
DSN: c.DSN,
108
Timeout: time.Duration(c.Timeout),
101
- MaxOpenConns: c.MaxDbConns,
102
- ConnMaxLife: time.Duration(c.MaxDbLifeTime),
109
+ MaxOpenConns: 1,
110
}
111
c.client = as400proto.NewClient(clientCfg)
112
@@ -122,14 +129,6 @@ func (c *Collector) Init(ctx context.Context) error {
129
}
130
c.subsystemSelector = m
131
}
125
- if c.JobQueueSelector != "" {
126
- m, err := matcher.NewSimplePatternsMatcher(c.JobQueueSelector)
127
- if err != nil {
128
- return fmt.Errorf("invalid job queue selector pattern '%s': %v", c.JobQueueSelector, err)
129
- }
130
- c.jobQueueSelector = m
131
- }
132
-
132
// Detect IBM i version on first init to drive feature toggles
133
if err := c.client.Connect(ctx); err == nil {
134
if err := c.detectIBMiVersion(ctx); err != nil {
@@ -145,6 +144,18 @@ func (c *Collector) Init(ctx context.Context) error {
144
c.applyGlobalLabels()
145
}
146
147
+ if err := c.configureTargets(); err != nil {
148
+ return err
149
+ }
150
+
151
+ if err := c.startSlowPath(); err != nil {
152
+ return err
153
+ }
154
+
155
+ if err := c.startBatchPath(); err != nil {
156
+ return err
157
+ }
158
+
159
return nil
160
}
161
src/go/plugin/ibm.d/modules/as400/instances.go
+7
-1
@@ -69,6 +69,9 @@ type tempStorageMetrics struct {
69
70
// activeJobMetrics holds metrics for an individual active job
71
type activeJobMetrics struct {
72
+ qualifiedName string
73
+ jobNumber string
74
+ jobUser string
75
jobName string
76
jobStatus string
77
subsystem string
@@ -156,7 +159,10 @@ func (a *Collector) getTempStorageMetrics(name string) *tempStorageMetrics {
159
160
func (a *Collector) getActiveJobMetrics(jobName string) *activeJobMetrics {
161
if _, ok := a.activeJobs[jobName]; !ok {
159
- a.activeJobs[jobName] = &activeJobMetrics{jobName: jobName}
162
+ a.activeJobs[jobName] = &activeJobMetrics{
163
+ qualifiedName: jobName,
164
+ jobName: jobName,
165
+ }
166
}
167
return a.activeJobs[jobName]
168
}
src/go/plugin/ibm.d/modules/as400/latency_cache.go
new
+46
@@ -0,0 +1,46 @@
1
+package as400
2
+
3
+import (
4
+ "sync"
5
+ "time"
6
+)
7
+
8
+type latencyCache struct {
9
+ mu sync.RWMutex
10
+ values map[string]int64
11
+ last time.Time
12
+}
13
+
14
+func (l *latencyCache) beginCycle(ts time.Time) {
15
+ l.mu.Lock()
16
+ if l.values == nil {
17
+ l.values = make(map[string]int64)
18
+ }
19
+ l.last = ts
20
+ l.mu.Unlock()
21
+}
22
+
23
+func (l *latencyCache) add(name string, value int64) {
24
+ if value == 0 {
25
+ return
26
+ }
27
+ l.mu.Lock()
28
+ if l.values == nil {
29
+ l.values = make(map[string]int64)
30
+ }
31
+ l.values[name] += value
32
+ l.mu.Unlock()
33
+}
34
+
35
+func (l *latencyCache) snapshot() (map[string]int64, time.Time) {
36
+ l.mu.RLock()
37
+ defer l.mu.RUnlock()
38
+ if len(l.values) == 0 {
39
+ return nil, l.last
40
+ }
41
+ out := make(map[string]int64, len(l.values))
42
+ for k, v := range l.values {
43
+ out[k] = v
44
+ }
45
+ return out, l.last
46
+}
src/go/plugin/ibm.d/modules/as400/metadata.yaml
+101
-38
@@ -32,6 +32,14 @@ modules:
32
- libodbc.so (provided by unixODBC)
33
- IBM i Access Client Solutions
34
35
+ **Collection paths**
36
+
37
+ The collector executes queries in multiple tracks:
38
+
39
+ - **Fast path (5s)**: lightweight system status queries remain sequential on the main plugin thread.
40
+ - **Slow path (10s beat)**: heavier queries (per-queue metrics, subsystems, plan cache, etc.) run in a background worker with bounded concurrency.
41
+ - **Batch path (≥60s beat)**: optional long-period worker used for expensive aggregate queries such as queue totals. Disabled by default unless queue totals are explicitly enabled.
42
+
43
**CPU Collection Methods:**
44
45
The collector uses a hybrid approach for CPU utilization metrics to handle IBM i 7.4+ where
@@ -77,6 +85,13 @@ modules:
85
86
Default: `false` (statistics are not reset, using `RESET_STATISTICS=>'NO'`)
87
88
+ **Chart Gaps During Baseline Resets:**
89
+
90
+ The `as400.system_activity_cpu_rate` and `as400.system_activity_cpu_utilization` charts rely on
91
+ delta calculations. When the collector detects that IBM i reset these statistics—or when it is
92
+ still establishing the initial baseline—it intentionally skips a sample instead of emitting a zero
93
+ or spike. Netdata renders those skipped samples as small gaps, which is expected behaviour.
94
+
95
**Cardinality Management:**
96
97
To prevent performance issues from excessive metric creation, the collector enforces cardinality
@@ -100,11 +115,25 @@ modules:
115
| `max_job_queues` | Maximum job queues to monitor | 100 |
116
| `max_message_queues` | Maximum message queues to monitor | 100 |
117
| `max_output_queues` | Maximum output queues to monitor | 100 |
103
- | `max_active_jobs` | Maximum active jobs to monitor | 100 |
118
+ | `active_jobs` | Fully qualified active jobs to monitor (`JOB_NUMBER/USER/JOB_NAME`) | `[]` |
119
| `collect_disks_matching` | Glob pattern to filter disks (e.g., `"001* 002*"`) | `""` (match all) |
120
| `collect_subsystems_matching` | Glob pattern to filter subsystems (e.g., `"QINTER QBATCH"`) | `""` (match all) |
121
| `collect_job_queues_matching` | Glob pattern to filter job queues (e.g., `"QSYS/*"`) | `""` (match all) |
122
123
+ Optional batch-path controls:
124
+
125
+ | Option | Purpose | Default |
126
+ |--------|---------|---------|
127
+ | `batch_path` | Enables the long-period batch worker for aggregate queries | `false` |
128
+ | `batch_path_update_every` | Batch worker cadence (minimum 60s, recommend ≥600s in production) | `60s` |
129
+ | `batch_path_max_connections` | Maximum concurrent connections for batch queries | `1` |
130
+ | `collect_message_queue_totals` | Enables full-scan counting of all message queues and messages | `auto` (off) |
131
+ | `collect_job_queue_totals` | Enables aggregate counting of job queues and queued jobs | `auto` (off) |
132
+ | `collect_output_queue_totals` | Enables aggregate counting of output queues and spooled files | `auto` (off) |
133
+
134
+ > **Warning:** queue totals require scanning IBM i catalog views and can be very expensive on large systems. Leave these options disabled unless aggregate counts are absolutely necessary.
135
+
136
+
137
**Example Workflow:**
138
139
1. System has 500 disks, collector skips disk metrics (exceeds default limit of 100)
@@ -118,6 +147,15 @@ modules:
147
- Set limits based on your Netdata server's capacity (each instance = multiple charts)
148
- Start with defaults and adjust based on actual usage patterns
149
150
+ **IBM i 7.2–7.3 Behavior Note (Message Queues):**
151
+
152
+ IBM i 7.4 introduced a message-queue table function that returns only the live backlog. On
153
+ 7.2–7.3 systems we fall back to the `QSYS2.MESSAGE_QUEUE_INFO` view, which includes *all*
154
+ recorded messages (even those already processed/cleared from the queue). Aggregations—especially
155
+ `MAX(SEVERITY)`—therefore reflect the historical log, not just the outstanding backlog. This
156
+ behaviour is inherent to the IBM SQL service and can lead to higher-than-expected max severity
157
+ values on pre-7.4 systems.
158
+
159
Network interface metrics have a fixed internal limit of 50 instances, and HTTP server metrics are capped at 200 instances; these limits are currently not configurable.
160
161
method_description: |
@@ -384,6 +422,48 @@ modules:
422
chart_type: line
423
dimensions:
424
- name: mtu
425
+ - name: observability
426
+ description: These metrics refer to observability instances.
427
+ labels:
428
+ - name: path
429
+ description: Path identifier
430
+ metrics:
431
+ - name: netdata.plugin_ibm.as400_query_latency
432
+ description: AS400 Query Latency
433
+ unit: ms
434
+ chart_type: stacked
435
+ dimensions:
436
+ - name: analyze_plan_cache
437
+ - name: count_disks
438
+ - name: count_http_servers
439
+ - name: count_network_interfaces
440
+ - name: count_subsystems
441
+ - name: detect_ibmi_version_primary
442
+ - name: detect_ibmi_version_fallback
443
+ - name: disk_instances
444
+ - name: disk_instances_enhanced
445
+ - name: disk_status
446
+ - name: http_server_info
447
+ - name: job_info
448
+ - name: job_queues
449
+ - name: memory_pools
450
+ - name: message_queue_aggregates
451
+ - name: network_connections
452
+ - name: network_interfaces
453
+ - name: output_queue_info
454
+ - name: plan_cache_summary
455
+ - name: job_queue_totals
456
+ - name: message_queue_totals
457
+ - name: output_queue_totals
458
+ - name: serial_number
459
+ - name: system_activity
460
+ - name: system_model
461
+ - name: system_status
462
+ - name: temp_storage_named
463
+ - name: temp_storage_total
464
+ - name: technology_refresh_level
465
+ - name: active_job
466
+ - name: other
467
- name: outputqueue
468
description: These metrics refer to outputqueue instances.
469
labels:
@@ -424,6 +504,26 @@ modules:
504
chart_type: line
505
dimensions:
506
- name: value
507
+ - name: queueoverview
508
+ description: These metrics refer to queueoverview instances.
509
+ labels:
510
+ - name: queue_type
511
+ description: Queue_type identifier
512
+ - name: item_type
513
+ description: Item_type identifier
514
+ metrics:
515
+ - name: as400.queues_count
516
+ description: Queue Counts
517
+ unit: queues
518
+ chart_type: line
519
+ dimensions:
520
+ - name: queues
521
+ - name: as400.queued_items
522
+ description: Queued Items
523
+ unit: items
524
+ chart_type: line
525
+ dimensions:
526
+ - name: items
527
- name: subsystem
528
description: These metrics refer to subsystem instances.
529
labels:
@@ -605,43 +705,6 @@ modules:
705
- name: average
706
- name: minimum
707
- name: maximum
608
- - name: netdata.plugin_ibm.as400_query_latency
609
- description: Query Collection Latency
610
- unit: ms
611
- chart_type: stacked
612
- dimensions:
613
- - name: analyze_plan_cache
614
- - name: count_active_jobs
615
- - name: count_disks
616
- - name: count_http_servers
617
- - name: count_job_queues
618
- - name: count_message_queues
619
- - name: count_network_interfaces
620
- - name: count_output_queues
621
- - name: count_subsystems
622
- - name: detect_ibmi_version_primary
623
- - name: detect_ibmi_version_fallback
624
- - name: disk_instances
625
- - name: disk_instances_enhanced
626
- - name: disk_status
627
- - name: http_server_info
628
- - name: job_info
629
- - name: job_queues
630
- - name: memory_pools
631
- - name: message_queue_aggregates
632
- - name: network_connections
633
- - name: network_interfaces
634
- - name: output_queue_info
635
- - name: plan_cache_summary
636
- - name: serial_number
637
- - name: system_activity
638
- - name: system_model
639
- - name: system_status
640
- - name: temp_storage_named
641
- - name: temp_storage_total
642
- - name: technology_refresh_level
643
- - name: top_active_jobs
644
- - name: other
708
- name: tempstoragebucket
709
description: These metrics refer to tempstoragebucket instances.
710
labels:
src/go/plugin/ibm.d/modules/as400/module.yaml
+39
-1
@@ -12,6 +12,14 @@ description: |
12
- libodbc.so (provided by unixODBC)
13
- IBM i Access Client Solutions
14
15
+ **Collection paths**
16
+
17
+ The collector executes queries in multiple tracks:
18
+
19
+ - **Fast path (5s)**: lightweight system status queries remain sequential on the main plugin thread.
20
+ - **Slow path (10s beat)**: heavier queries (per-queue metrics, subsystems, plan cache, etc.) run in a background worker with bounded concurrency.
21
+ - **Batch path (≥60s beat)**: optional long-period worker used for expensive aggregate queries such as queue totals. Disabled by default unless queue totals are explicitly enabled.
22
+
23
**CPU Collection Methods:**
24
25
The collector uses a hybrid approach for CPU utilization metrics to handle IBM i 7.4+ where
@@ -57,6 +65,13 @@ description: |
65
66
Default: `false` (statistics are not reset, using `RESET_STATISTICS=>'NO'`)
67
68
+ **Chart Gaps During Baseline Resets:**
69
+
70
+ The `as400.system_activity_cpu_rate` and `as400.system_activity_cpu_utilization` charts rely on
71
+ delta calculations. When the collector detects that IBM i reset these statistics—or when it is
72
+ still establishing the initial baseline—it intentionally skips a sample instead of emitting a zero
73
+ or spike. Netdata renders those skipped samples as small gaps, which is expected behaviour.
74
+
75
**Cardinality Management:**
76
77
To prevent performance issues from excessive metric creation, the collector enforces cardinality
@@ -80,11 +95,25 @@ description: |
95
| `max_job_queues` | Maximum job queues to monitor | 100 |
96
| `max_message_queues` | Maximum message queues to monitor | 100 |
97
| `max_output_queues` | Maximum output queues to monitor | 100 |
83
- | `max_active_jobs` | Maximum active jobs to monitor | 100 |
98
+ | `active_jobs` | Fully qualified active jobs to monitor (`JOB_NUMBER/USER/JOB_NAME`) | `[]` |
99
| `collect_disks_matching` | Glob pattern to filter disks (e.g., `"001* 002*"`) | `""` (match all) |
100
| `collect_subsystems_matching` | Glob pattern to filter subsystems (e.g., `"QINTER QBATCH"`) | `""` (match all) |
101
| `collect_job_queues_matching` | Glob pattern to filter job queues (e.g., `"QSYS/*"`) | `""` (match all) |
102
103
+ Optional batch-path controls:
104
+
105
+ | Option | Purpose | Default |
106
+ |--------|---------|---------|
107
+ | `batch_path` | Enables the long-period batch worker for aggregate queries | `false` |
108
+ | `batch_path_update_every` | Batch worker cadence (minimum 60s, recommend ≥600s in production) | `60s` |
109
+ | `batch_path_max_connections` | Maximum concurrent connections for batch queries | `1` |
110
+ | `collect_message_queue_totals` | Enables full-scan counting of all message queues and messages | `auto` (off) |
111
+ | `collect_job_queue_totals` | Enables aggregate counting of job queues and queued jobs | `auto` (off) |
112
+ | `collect_output_queue_totals` | Enables aggregate counting of output queues and spooled files | `auto` (off) |
113
+
114
+ > **Warning:** queue totals require scanning IBM i catalog views and can be very expensive on large systems. Leave these options disabled unless aggregate counts are absolutely necessary.
115
+
116
+
117
**Example Workflow:**
118
119
1. System has 500 disks, collector skips disk metrics (exceeds default limit of 100)
@@ -98,6 +127,15 @@ description: |
127
- Set limits based on your Netdata server's capacity (each instance = multiple charts)
128
- Start with defaults and adjust based on actual usage patterns
129
130
+ **IBM i 7.2–7.3 Behavior Note (Message Queues):**
131
+
132
+ IBM i 7.4 introduced a message-queue table function that returns only the live backlog. On
133
+ 7.2–7.3 systems we fall back to the `QSYS2.MESSAGE_QUEUE_INFO` view, which includes *all*
134
+ recorded messages (even those already processed/cleared from the queue). Aggregations—especially
135
+ `MAX(SEVERITY)`—therefore reflect the historical log, not just the outstanding backlog. This
136
+ behaviour is inherent to the IBM SQL service and can lead to higher-than-expected max severity
137
+ values on pre-7.4 systems.
138
+
139
Network interface metrics have a fixed internal limit of 50 instances, and HTTP server metrics are capped at 200 instances; these limits are currently not configurable.
140
icon: ibm-i.svg
141
categories:
src/go/plugin/ibm.d/modules/as400/module_stub.go
+2
-1
@@ -15,6 +15,7 @@ var configSchema string
15
16
type Collector struct {
17
module.Base
18
+ Config
19
}
20
21
func New() *Collector {
@@ -22,7 +23,7 @@ func New() *Collector {
23
}
24
25
func (c *Collector) Configuration() any {
25
- return nil
26
+ return &c.Config
27
}
28
29
func (c *Collector) Init(context.Context) error {
src/go/plugin/ibm.d/modules/as400/queues.go
new
+199
@@ -0,0 +1,199 @@
1
+//go:build cgo
2
+// +build cgo
3
+
4
+package as400
5
+
6
+import (
7
+ "fmt"
8
+ "strings"
9
+)
10
+
11
+type queueTarget struct {
12
+ Library string
13
+ Name string
14
+}
15
+
16
+func (t queueTarget) ID() string {
17
+ return t.Library + "/" + t.Name
18
+}
19
+
20
+type activeJobTarget struct {
21
+ Number string
22
+ User string
23
+ Name string
24
+}
25
+
26
+func (t activeJobTarget) ID() string {
27
+ return t.Number + "/" + t.User + "/" + t.Name
28
+}
29
+
30
+func parseQueueTargets(entries []string) ([]queueTarget, error) {
31
+ var targets []queueTarget
32
+ seen := make(map[string]struct{})
33
+
34
+ for _, raw := range entries {
35
+ trimmed := strings.TrimSpace(raw)
36
+ if trimmed == "" {
37
+ continue
38
+ }
39
+ parts := strings.SplitN(trimmed, "/", 2)
40
+ if len(parts) != 2 {
41
+ return nil, fmt.Errorf("invalid queue identifier %q (expected LIBRARY/QUEUE)", raw)
42
+ }
43
+ lib := strings.ToUpper(strings.TrimSpace(parts[0]))
44
+ name := strings.ToUpper(strings.TrimSpace(parts[1]))
45
+ if lib == "" || name == "" {
46
+ return nil, fmt.Errorf("invalid queue identifier %q (library and queue must be non-empty)", raw)
47
+ }
48
+ if !validObjectName(lib) || !validObjectName(name) {
49
+ return nil, fmt.Errorf("invalid queue identifier %q (only A-Z, 0-9, _, $, #, @ allowed)", raw)
50
+ }
51
+ key := lib + "/" + name
52
+ if _, exists := seen[key]; exists {
53
+ continue
54
+ }
55
+ seen[key] = struct{}{}
56
+ targets = append(targets, queueTarget{
57
+ Library: lib,
58
+ Name: name,
59
+ })
60
+ }
61
+
62
+ return targets, nil
63
+}
64
+
65
+func parseActiveJobTargets(entries []string) ([]activeJobTarget, error) {
66
+ var targets []activeJobTarget
67
+ seen := make(map[string]struct{})
68
+
69
+ for _, raw := range entries {
70
+ trimmed := strings.TrimSpace(raw)
71
+ if trimmed == "" {
72
+ continue
73
+ }
74
+
75
+ parts := strings.Split(trimmed, "/")
76
+ if len(parts) != 3 {
77
+ return nil, fmt.Errorf("invalid active job identifier %q (expected JOB_NUMBER/USER/JOB_NAME)", raw)
78
+ }
79
+
80
+ number := strings.TrimSpace(parts[0])
81
+ user := strings.ToUpper(strings.TrimSpace(parts[1]))
82
+ name := strings.ToUpper(strings.TrimSpace(parts[2]))
83
+
84
+ if !validJobNumber(number) {
85
+ return nil, fmt.Errorf("invalid active job identifier %q (job number must be six digits)", raw)
86
+ }
87
+ if !validObjectName(user) || !validObjectName(name) {
88
+ return nil, fmt.Errorf("invalid active job identifier %q (only A-Z, 0-9, _, $, #, @ allowed)", raw)
89
+ }
90
+
91
+ key := number + "/" + user + "/" + name
92
+ if _, exists := seen[key]; exists {
93
+ continue
94
+ }
95
+ seen[key] = struct{}{}
96
+ targets = append(targets, activeJobTarget{
97
+ Number: number,
98
+ User: user,
99
+ Name: name,
100
+ })
101
+ }
102
+
103
+ return targets, nil
104
+}
105
+
106
+func (c *Collector) configureTargets() error {
107
+ targets, err := parseQueueTargets(c.MessageQueues)
108
+ if err != nil {
109
+ return fmt.Errorf("message_queues configuration error: %w", err)
110
+ }
111
+ c.messageQueueTargets = targets
112
+
113
+ targets, err = parseQueueTargets(c.JobQueues)
114
+ if err != nil {
115
+ return fmt.Errorf("job_queues configuration error: %w", err)
116
+ }
117
+ c.jobQueueTargets = targets
118
+
119
+ targets, err = parseQueueTargets(c.OutputQueues)
120
+ if err != nil {
121
+ return fmt.Errorf("output_queues configuration error: %w", err)
122
+ }
123
+ c.outputQueueTargets = targets
124
+
125
+ jobTargets, err := parseActiveJobTargets(c.ActiveJobs)
126
+ if err != nil {
127
+ return fmt.Errorf("active_jobs configuration error: %w", err)
128
+ }
129
+ c.activeJobTargets = jobTargets
130
+
131
+ if len(c.messageQueueTargets) == 0 {
132
+ c.Infof("message queue metrics disabled: no queues configured")
133
+ } else {
134
+ c.Infof("message queue metrics enabled for %d queue(s): %s", len(c.messageQueueTargets), queueTargetList(c.messageQueueTargets))
135
+ }
136
+
137
+ if len(c.jobQueueTargets) == 0 {
138
+ c.Infof("job queue metrics disabled: no queues configured")
139
+ } else {
140
+ c.Infof("job queue metrics enabled for %d queue(s): %s", len(c.jobQueueTargets), queueTargetList(c.jobQueueTargets))
141
+ }
142
+
143
+ if len(c.outputQueueTargets) == 0 {
144
+ c.Infof("output queue metrics disabled: no queues configured")
145
+ } else {
146
+ c.Infof("output queue metrics enabled for %d queue(s): %s", len(c.outputQueueTargets), queueTargetList(c.outputQueueTargets))
147
+ }
148
+ if len(c.activeJobTargets) == 0 {
149
+ c.Infof("active job metrics disabled: no jobs configured")
150
+ } else {
151
+ c.Infof("active job metrics enabled for %d job(s): %s", len(c.activeJobTargets), activeJobTargetList(c.activeJobTargets))
152
+ }
153
+
154
+ return nil
155
+}
156
+
157
+func queueTargetList(targets []queueTarget) string {
158
+ ids := make([]string, 0, len(targets))
159
+ for _, t := range targets {
160
+ ids = append(ids, t.ID())
161
+ }
162
+ return strings.Join(ids, ", ")
163
+}
164
+
165
+func activeJobTargetList(targets []activeJobTarget) string {
166
+ ids := make([]string, 0, len(targets))
167
+ for _, t := range targets {
168
+ ids = append(ids, t.ID())
169
+ }
170
+ return strings.Join(ids, ", ")
171
+}
172
+
173
+func validObjectName(value string) bool {
174
+ for _, r := range value {
175
+ switch {
176
+ case r >= 'A' && r <= 'Z':
177
+ continue
178
+ case r >= '0' && r <= '9':
179
+ continue
180
+ case r == '_' || r == '$' || r == '#' || r == '@':
181
+ continue
182
+ default:
183
+ return false
184
+ }
185
+ }
186
+ return true
187
+}
188
+
189
+func validJobNumber(value string) bool {
190
+ if len(value) != 6 {
191
+ return false
192
+ }
193
+ for _, r := range value {
194
+ if r < '0' || r > '9' {
195
+ return false
196
+ }
197
+ }
198
+ return true
199
+}
src/go/plugin/ibm.d/modules/as400/slow_path.go
new
+837
@@ -0,0 +1,837 @@
1
+//go:build cgo
2
+// +build cgo
3
+
4
+package as400
5
+
6
+import (
7
+ "context"
8
+ "errors"
9
+ "fmt"
10
+ "strings"
11
+ "sync"
12
+ "time"
13
+
14
+ "golang.org/x/sync/errgroup"
15
+
16
+ as400proto "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/as400"
17
+)
18
+
19
+type slowPathConfig struct {
20
+ enabled bool
21
+ interval time.Duration
22
+ maxConnections int
23
+}
24
+
25
+type messageQueueSnapshot struct {
26
+ metrics map[string]messageQueueInstanceMetrics
27
+ meta map[string]messageQueueMetrics
28
+ timestamp time.Time
29
+ err error
30
+}
31
+
32
+type jobQueueSnapshot struct {
33
+ metrics map[string]jobQueueInstanceMetrics
34
+ meta map[string]jobQueueMetrics
35
+ timestamp time.Time
36
+ err error
37
+}
38
+
39
+type outputQueueSnapshot struct {
40
+ metrics map[string]outputQueueInstanceMetrics
41
+ meta map[string]outputQueueMetrics
42
+ timestamp time.Time
43
+ err error
44
+}
45
+
46
+type subsystemSnapshot struct {
47
+ metrics map[string]subsystemInstanceMetrics
48
+ meta map[string]subsystemMetrics
49
+ timestamp time.Time
50
+ err error
51
+}
52
+
53
+type planCacheSnapshot struct {
54
+ values map[string]planCacheInstanceMetrics
55
+ meta map[string]planCacheMetrics
56
+ timestamp time.Time
57
+ err error
58
+}
59
+
60
+type slowCache struct {
61
+ mu sync.RWMutex
62
+ messageQueues messageQueueSnapshot
63
+ jobQueues jobQueueSnapshot
64
+ outputQueues outputQueueSnapshot
65
+ subsystems subsystemSnapshot
66
+ planCache planCacheSnapshot
67
+ latency latencyCache
68
+}
69
+
70
+func (c *Collector) slowPathActive() bool {
71
+ return c != nil && c.slow.config.enabled && c.slow.client != nil
72
+}
73
+
74
+func (c *slowCache) beginLatencyCycle(ts time.Time) {
75
+ c.latency.beginCycle(ts)
76
+}
77
+
78
+func (c *slowCache) addLatency(name string, value int64) {
79
+ c.latency.add(name, value)
80
+}
81
+
82
+func (c *slowCache) setMessageQueues(snapshot messageQueueSnapshot) {
83
+ c.mu.Lock()
84
+ c.messageQueues = snapshot
85
+ c.mu.Unlock()
86
+}
87
+
88
+func (c *slowCache) setJobQueues(snapshot jobQueueSnapshot) {
89
+ c.mu.Lock()
90
+ c.jobQueues = snapshot
91
+ c.mu.Unlock()
92
+}
93
+
94
+func (c *slowCache) setOutputQueues(snapshot outputQueueSnapshot) {
95
+ c.mu.Lock()
96
+ c.outputQueues = snapshot
97
+ c.mu.Unlock()
98
+}
99
+
100
+func (c *slowCache) setSubsystems(snapshot subsystemSnapshot) {
101
+ c.mu.Lock()
102
+ c.subsystems = snapshot
103
+ c.mu.Unlock()
104
+}
105
+
106
+func (c *slowCache) setPlanCache(snapshot planCacheSnapshot) {
107
+ c.mu.Lock()
108
+ c.planCache = snapshot
109
+ c.mu.Unlock()
110
+}
111
+
112
+func (c *slowCache) getMessageQueues() messageQueueSnapshot {
113
+ c.mu.RLock()
114
+ defer c.mu.RUnlock()
115
+ return cloneMessageQueueSnapshot(c.messageQueues)
116
+}
117
+
118
+func (c *slowCache) getJobQueues() jobQueueSnapshot {
119
+ c.mu.RLock()
120
+ defer c.mu.RUnlock()
121
+ return cloneJobQueueSnapshot(c.jobQueues)
122
+}
123
+
124
+func (c *slowCache) getOutputQueues() outputQueueSnapshot {
125
+ c.mu.RLock()
126
+ defer c.mu.RUnlock()
127
+ return cloneOutputQueueSnapshot(c.outputQueues)
128
+}
129
+
130
+func (c *slowCache) getSubsystems() subsystemSnapshot {
131
+ c.mu.RLock()
132
+ defer c.mu.RUnlock()
133
+ return cloneSubsystemSnapshot(c.subsystems)
134
+}
135
+
136
+func (c *slowCache) getPlanCache() planCacheSnapshot {
137
+ c.mu.RLock()
138
+ defer c.mu.RUnlock()
139
+ return clonePlanCacheSnapshot(c.planCache)
140
+}
141
+
142
+func (c *slowCache) getLatencies() (map[string]int64, time.Time) {
143
+ return c.latency.snapshot()
144
+}
145
+
146
+func cloneMessageQueueSnapshot(src messageQueueSnapshot) messageQueueSnapshot {
147
+ dst := messageQueueSnapshot{
148
+ timestamp: src.timestamp,
149
+ err: src.err,
150
+ }
151
+ if src.metrics != nil {
152
+ dst.metrics = make(map[string]messageQueueInstanceMetrics, len(src.metrics))
153
+ for k, v := range src.metrics {
154
+ dst.metrics[k] = v
155
+ }
156
+ }
157
+ if src.meta != nil {
158
+ dst.meta = make(map[string]messageQueueMetrics, len(src.meta))
159
+ for k, v := range src.meta {
160
+ dst.meta[k] = v
161
+ }
162
+ }
163
+ return dst
164
+}
165
+
166
+func cloneJobQueueSnapshot(src jobQueueSnapshot) jobQueueSnapshot {
167
+ dst := jobQueueSnapshot{
168
+ timestamp: src.timestamp,
169
+ err: src.err,
170
+ }
171
+ if src.metrics != nil {
172
+ dst.metrics = make(map[string]jobQueueInstanceMetrics, len(src.metrics))
173
+ for k, v := range src.metrics {
174
+ dst.metrics[k] = v
175
+ }
176
+ }
177
+ if src.meta != nil {
178
+ dst.meta = make(map[string]jobQueueMetrics, len(src.meta))
179
+ for k, v := range src.meta {
180
+ dst.meta[k] = v
181
+ }
182
+ }
183
+ return dst
184
+}
185
+
186
+func cloneOutputQueueSnapshot(src outputQueueSnapshot) outputQueueSnapshot {
187
+ dst := outputQueueSnapshot{
188
+ timestamp: src.timestamp,
189
+ err: src.err,
190
+ }
191
+ if src.metrics != nil {
192
+ dst.metrics = make(map[string]outputQueueInstanceMetrics, len(src.metrics))
193
+ for k, v := range src.metrics {
194
+ dst.metrics[k] = v
195
+ }
196
+ }
197
+ if src.meta != nil {
198
+ dst.meta = make(map[string]outputQueueMetrics, len(src.meta))
199
+ for k, v := range src.meta {
200
+ dst.meta[k] = v
201
+ }
202
+ }
203
+ return dst
204
+}
205
+
206
+func cloneSubsystemSnapshot(src subsystemSnapshot) subsystemSnapshot {
207
+ dst := subsystemSnapshot{
208
+ timestamp: src.timestamp,
209
+ err: src.err,
210
+ }
211
+ if src.metrics != nil {
212
+ dst.metrics = make(map[string]subsystemInstanceMetrics, len(src.metrics))
213
+ for k, v := range src.metrics {
214
+ dst.metrics[k] = v
215
+ }
216
+ }
217
+ if src.meta != nil {
218
+ dst.meta = make(map[string]subsystemMetrics, len(src.meta))
219
+ for k, v := range src.meta {
220
+ dst.meta[k] = v
221
+ }
222
+ }
223
+ return dst
224
+}
225
+
226
+func clonePlanCacheSnapshot(src planCacheSnapshot) planCacheSnapshot {
227
+ dst := planCacheSnapshot{
228
+ timestamp: src.timestamp,
229
+ err: src.err,
230
+ }
231
+ if src.values != nil {
232
+ dst.values = make(map[string]planCacheInstanceMetrics, len(src.values))
233
+ for k, v := range src.values {
234
+ dst.values[k] = v
235
+ }
236
+ }
237
+ if src.meta != nil {
238
+ dst.meta = make(map[string]planCacheMetrics, len(src.meta))
239
+ for k, v := range src.meta {
240
+ dst.meta[k] = v
241
+ }
242
+ }
243
+ return dst
244
+}
245
+
246
+func (c *Collector) startSlowPath() error {
247
+ c.stopSlowPath()
248
+
249
+ cfg := slowPathConfig{
250
+ enabled: c.SlowPath,
251
+ interval: time.Duration(c.SlowPathUpdateEvery),
252
+ maxConnections: c.SlowPathMaxConnections,
253
+ }
254
+
255
+ if !cfg.enabled {
256
+ c.Debugf("slow path disabled; running sequential-only mode")
257
+ c.slow.config = cfg
258
+ return nil
259
+ }
260
+
261
+ if cfg.interval <= 0 {
262
+ cfg.interval = 30 * time.Second
263
+ }
264
+ if cfg.maxConnections <= 0 {
265
+ cfg.maxConnections = 1
266
+ }
267
+
268
+ fastInterval := time.Duration(c.fastPathIntervalSeconds()) * time.Second
269
+ if fastInterval <= 0 {
270
+ fastInterval = time.Second
271
+ }
272
+ if cfg.interval < fastInterval {
273
+ c.Warningf("slow path update every %s is shorter than main update %s; using %s", cfg.interval, fastInterval, fastInterval)
274
+ cfg.interval = fastInterval
275
+ }
276
+
277
+ clientCfg := as400proto.Config{
278
+ DSN: c.DSN,
279
+ Timeout: time.Duration(c.Timeout),
280
+ MaxOpenConns: cfg.maxConnections,
281
+ }
282
+
283
+ client := as400proto.NewClient(clientCfg)
284
+ ctx := context.Background()
285
+ if err := client.Connect(ctx); err != nil {
286
+ return fmt.Errorf("slow path: connect failed: %w", err)
287
+ }
288
+ if err := client.Ping(ctx); err != nil {
289
+ _ = client.Close()
290
+ return fmt.Errorf("slow path: ping failed: %w", err)
291
+ }
292
+
293
+ runCtx, cancel := context.WithCancel(context.Background())
294
+ c.slow.client = client
295
+ c.slow.cancel = cancel
296
+ c.slow.config = cfg
297
+ c.slow.wg.Add(1)
298
+ go c.runSlowPath(runCtx)
299
+ c.Infof("slow path worker started (interval=%s, max_conns=%d)", cfg.interval, cfg.maxConnections)
300
+ return nil
301
+}
302
+
303
+func (c *Collector) stopSlowPath() {
304
+ if c.slow.cancel != nil {
305
+ c.slow.cancel()
306
+ }
307
+ c.slow.wg.Wait()
308
+ if c.slow.client != nil {
309
+ if err := c.slow.client.Close(); err != nil {
310
+ c.Errorf("slow path: closing client failed: %v", err)
311
+ }
312
+ }
313
+ c.slow.cancel = nil
314
+ c.slow.client = nil
315
+ c.slow.config = slowPathConfig{}
316
+ c.slow.cache = slowCache{}
317
+}
318
+
319
+func (c *Collector) runSlowPath(ctx context.Context) {
320
+ defer c.slow.wg.Done()
321
+
322
+ interval := c.slow.config.interval
323
+ if interval <= 0 {
324
+ interval = 30 * time.Second
325
+ }
326
+
327
+ now := time.Now()
328
+ beat := now
329
+ c.runSlowCollectors(ctx, beat)
330
+ nextBeat := beat.Add(interval)
331
+
332
+ for {
333
+ sleep := time.Until(nextBeat)
334
+ if sleep > 0 {
335
+ timer := time.NewTimer(sleep)
336
+ select {
337
+ case <-ctx.Done():
338
+ timer.Stop()
339
+ return
340
+ case <-timer.C:
341
+ }
342
+ } else {
343
+ select {
344
+ case <-ctx.Done():
345
+ return
346
+ default:
347
+ }
348
+ }
349
+
350
+ beat = nextBeat
351
+ c.runSlowCollectors(ctx, beat)
352
+
353
+ nextBeat = nextBeat.Add(interval)
354
+ now = time.Now()
355
+ for nextBeat.Before(now) {
356
+ nextBeat = nextBeat.Add(interval)
357
+ }
358
+ }
359
+}
360
+
361
+func (c *Collector) runSlowCollectors(ctx context.Context, beat time.Time) {
362
+ if ctx.Err() != nil {
363
+ return
364
+ }
365
+
366
+ c.slow.cache.beginLatencyCycle(beat)
367
+
368
+ workCtx, cancel := context.WithCancel(ctx)
369
+ defer cancel()
370
+
371
+ group, groupCtx := errgroup.WithContext(workCtx)
372
+ group.SetLimit(c.slow.config.maxConnections)
373
+
374
+ group.Go(func() error {
375
+ snapshot, err := c.fetchMessageQueues(groupCtx, beat, c.slowDoQuery)
376
+ c.slow.cache.setMessageQueues(snapshot)
377
+ if err != nil {
378
+ return fmt.Errorf("message queues: %w", err)
379
+ }
380
+ return nil
381
+ })
382
+
383
+ group.Go(func() error {
384
+ snapshot, err := c.fetchJobQueues(groupCtx, beat, c.slowDoQuery)
385
+ c.slow.cache.setJobQueues(snapshot)
386
+ if err != nil {
387
+ return fmt.Errorf("job queues: %w", err)
388
+ }
389
+ return nil
390
+ })
391
+
392
+ group.Go(func() error {
393
+ snapshot, err := c.fetchOutputQueues(groupCtx, beat, c.slowDoQuery)
394
+ c.slow.cache.setOutputQueues(snapshot)
395
+ if err != nil {
396
+ return fmt.Errorf("output queues: %w", err)
397
+ }
398
+ return nil
399
+ })
400
+
401
+ group.Go(func() error {
402
+ snapshot, err := c.fetchSubsystems(groupCtx, beat, c.slowDoQuery, c.slowDoQueryRow)
403
+ c.slow.cache.setSubsystems(snapshot)
404
+ if err != nil {
405
+ return fmt.Errorf("subsystems: %w", err)
406
+ }
407
+ return nil
408
+ })
409
+
410
+ if c.CollectPlanCacheMetrics.IsEnabled() {
411
+ group.Go(func() error {
412
+ snapshot, err := c.fetchPlanCache(groupCtx, beat, c.slowExec, c.slowDoQuery)
413
+ c.slow.cache.setPlanCache(snapshot)
414
+ if err != nil {
415
+ return fmt.Errorf("plan cache: %w", err)
416
+ }
417
+ return nil
418
+ })
419
+ } else {
420
+ c.slow.cache.setPlanCache(planCacheSnapshot{
421
+ timestamp: beat,
422
+ err: nil,
423
+ values: make(map[string]planCacheInstanceMetrics),
424
+ meta: make(map[string]planCacheMetrics),
425
+ })
426
+ }
427
+
428
+ if err := group.Wait(); err != nil && !errors.Is(err, context.Canceled) {
429
+ c.logErrorOnce("slow_path_error", "slow path: %s", trimDriverMessage(err))
430
+ } else if err == nil {
431
+ c.clearErrorOnce("slow_path_error")
432
+ }
433
+}
434
+
435
+type queryFunc func(ctx context.Context, queryName, query string, assign func(column, value string, lineEnd bool)) error
436
+type queryRowFunc func(ctx context.Context, queryName, query string, assign func(column, value string)) error
437
+type execFunc func(ctx context.Context, query string) error
438
+
439
+func (c *Collector) fetchMessageQueues(ctx context.Context, beat time.Time, do queryFunc) (messageQueueSnapshot, error) {
440
+ snapshot := messageQueueSnapshot{
441
+ metrics: make(map[string]messageQueueInstanceMetrics),
442
+ meta: make(map[string]messageQueueMetrics),
443
+ timestamp: beat,
444
+ }
445
+
446
+ if len(c.messageQueueTargets) == 0 {
447
+ return snapshot, nil
448
+ }
449
+
450
+ var firstErr error
451
+
452
+ for _, target := range c.messageQueueTargets {
453
+ key := target.ID()
454
+ errorKey := "slow_message_queue_" + key
455
+ meta := messageQueueMetrics{
456
+ library: target.Library,
457
+ name: target.Name,
458
+ }
459
+ metrics := messageQueueInstanceMetrics{}
460
+
461
+ queryName := fmt.Sprintf("message_queue_%s_%s", target.Library, target.Name)
462
+ query := buildMessageQueueQuery(target, c.supportsMessageQueueTableFunction())
463
+ err := do(ctx, queryName, query, func(column, value string, lineEnd bool) {
464
+ switch column {
465
+ case "MESSAGE_COUNT":
466
+ metrics.Total = parseInt64OrZero(value)
467
+ case "INFORMATIONAL_MESSAGES":
468
+ metrics.Informational = parseInt64OrZero(value)
469
+ case "INQUIRY_MESSAGES":
470
+ metrics.Inquiry = parseInt64OrZero(value)
471
+ case "DIAGNOSTIC_MESSAGES":
472
+ metrics.Diagnostic = parseInt64OrZero(value)
473
+ case "ESCAPE_MESSAGES":
474
+ metrics.Escape = parseInt64OrZero(value)
475
+ case "NOTIFY_MESSAGES":
476
+ metrics.Notify = parseInt64OrZero(value)
477
+ case "SENDER_COPY_MESSAGES":
478
+ metrics.SenderCopy = parseInt64OrZero(value)
479
+ case "MAX_SEVERITY":
480
+ metrics.MaxSeverity = parseInt64OrZero(value)
481
+ }
482
+ })
483
+
484
+ if err != nil {
485
+ c.logQueryErrorOnce(errorKey, query, err)
486
+ if firstErr == nil {
487
+ firstErr = fmt.Errorf("message queue %s: %w", key, err)
488
+ }
489
+ continue
490
+ }
491
+ c.clearErrorOnce(errorKey)
492
+
493
+ snapshot.metrics[key] = metrics
494
+ snapshot.meta[key] = meta
495
+ }
496
+
497
+ snapshot.err = firstErr
498
+ return snapshot, firstErr
499
+}
500
+
501
+func (c *Collector) fetchJobQueues(ctx context.Context, beat time.Time, do queryFunc) (jobQueueSnapshot, error) {
502
+ snapshot := jobQueueSnapshot{
503
+ metrics: make(map[string]jobQueueInstanceMetrics),
504
+ meta: make(map[string]jobQueueMetrics),
505
+ timestamp: beat,
506
+ }
507
+
508
+ if len(c.jobQueueTargets) == 0 {
509
+ return snapshot, nil
510
+ }
511
+
512
+ var firstErr error
513
+
514
+ for _, target := range c.jobQueueTargets {
515
+ key := target.ID()
516
+ errorKey := "slow_job_queue_" + key
517
+ meta := jobQueueMetrics{
518
+ library: target.Library,
519
+ name: target.Name,
520
+ status: "UNKNOWN",
521
+ }
522
+ metrics := jobQueueInstanceMetrics{}
523
+ found := false
524
+
525
+ queryName := fmt.Sprintf("job_queue_%s_%s", target.Library, target.Name)
526
+ query := buildJobQueueQuery(target)
527
+ err := do(ctx, queryName, query, func(column, value string, lineEnd bool) {
528
+ switch column {
529
+ case "JOB_QUEUE_STATUS":
530
+ meta.status = strings.TrimSpace(value)
531
+ case "NUMBER_OF_JOBS":
532
+ metrics.NumberOfJobs = parseInt64OrZero(value)
533
+ case "RELEASED_JOBS":
534
+ meta.jobsWaiting = parseInt64OrZero(value)
535
+ case "SCHEDULED_JOBS":
536
+ meta.jobsScheduled = parseInt64OrZero(value)
537
+ case "HELD_JOBS":
538
+ meta.jobsHeld = parseInt64OrZero(value)
539
+ case "MAXIMUM_ACTIVE_JOBS":
540
+ meta.maxJobs = parseInt64OrZero(value)
541
+ }
542
+ if lineEnd {
543
+ found = true
544
+ }
545
+ })
546
+
547
+ if err != nil {
548
+ c.logQueryErrorOnce(errorKey, query, err)
549
+ if firstErr == nil {
550
+ firstErr = fmt.Errorf("job queue %s: %w", key, err)
551
+ }
552
+ continue
553
+ }
554
+ c.clearErrorOnce(errorKey)
555
+
556
+ if !found {
557
+ meta.status = "NOT_FOUND"
558
+ }
559
+
560
+ snapshot.metrics[key] = metrics
561
+ snapshot.meta[key] = meta
562
+ }
563
+
564
+ snapshot.err = firstErr
565
+ return snapshot, firstErr
566
+}
567
+
568
+func (c *Collector) fetchOutputQueues(ctx context.Context, beat time.Time, do queryFunc) (outputQueueSnapshot, error) {
569
+ snapshot := outputQueueSnapshot{
570
+ metrics: make(map[string]outputQueueInstanceMetrics),
571
+ meta: make(map[string]outputQueueMetrics),
572
+ timestamp: beat,
573
+ }
574
+
575
+ if len(c.outputQueueTargets) == 0 {
576
+ return snapshot, nil
577
+ }
578
+
579
+ var firstErr error
580
+
581
+ for _, target := range c.outputQueueTargets {
582
+ key := target.ID()
583
+ errorEntriesKey := "slow_output_queue_entries_" + key
584
+ errorInfoKey := "slow_output_queue_info_" + key
585
+ meta := outputQueueMetrics{
586
+ library: target.Library,
587
+ name: target.Name,
588
+ status: "UNKNOWN",
589
+ }
590
+
591
+ metrics := outputQueueInstanceMetrics{}
592
+ entriesCount := int64(0)
593
+ entriesUsed := false
594
+
595
+ queryName := fmt.Sprintf("output_queue_%s_%s", target.Library, target.Name)
596
+ entriesQuery := buildOutputQueueEntriesQuery(target)
597
+ err := do(ctx, queryName, entriesQuery, func(column, value string, lineEnd bool) {
598
+ if lineEnd {
599
+ entriesCount++
600
+ }
601
+ })
602
+ if err != nil {
603
+ c.logQueryErrorOnce(errorEntriesKey, entriesQuery, err)
604
+ if firstErr == nil {
605
+ firstErr = fmt.Errorf("output queue %s (entries): %w", key, err)
606
+ }
607
+ } else {
608
+ c.clearErrorOnce(errorEntriesKey)
609
+ entriesUsed = true
610
+ metrics.Files = entriesCount
611
+ }
612
+
613
+ infoQuery := buildOutputQueueInfoQuery(target)
614
+ viewErr := do(ctx, queryName+"_view", infoQuery, func(column, value string, lineEnd bool) {
615
+ switch column {
616
+ case "OUTPUT_QUEUE_STATUS":
617
+ meta.status = strings.TrimSpace(value)
618
+ case "NUMBER_OF_WRITERS":
619
+ metrics.Writers = parseInt64OrZero(value)
620
+ case "NUMBER_OF_FILES":
621
+ if !entriesUsed {
622
+ metrics.Files = parseInt64OrZero(value)
623
+ }
624
+ }
625
+ })
626
+ if viewErr != nil {
627
+ c.logQueryErrorOnce(errorInfoKey, infoQuery, viewErr)
628
+ if firstErr == nil {
629
+ firstErr = fmt.Errorf("output queue %s (info): %w", key, viewErr)
630
+ }
631
+ continue
632
+ }
633
+ c.clearErrorOnce(errorInfoKey)
634
+
635
+ metrics.Released = boolToInt(strings.EqualFold(meta.status, "RELEASED"))
636
+ snapshot.metrics[key] = metrics
637
+ snapshot.meta[key] = meta
638
+ }
639
+
640
+ snapshot.err = firstErr
641
+ return snapshot, firstErr
642
+}
643
+
644
+func (c *Collector) countSubsystemsWith(doRow queryRowFunc, ctx context.Context) (int, error) {
645
+ var count int64
646
+ err := doRow(ctx, "count_subsystems", queryCountSubsystems, func(column, value string) {
647
+ if column == "COUNT" {
648
+ if v, ok := c.parseInt64Value(value, 1); ok {
649
+ count = v
650
+ }
651
+ }
652
+ })
653
+ return int(count), err
654
+}
655
+
656
+func (c *Collector) fetchSubsystems(ctx context.Context, beat time.Time, do queryFunc, doRow queryRowFunc) (subsystemSnapshot, error) {
657
+ snapshot := subsystemSnapshot{
658
+ metrics: make(map[string]subsystemInstanceMetrics),
659
+ meta: make(map[string]subsystemMetrics),
660
+ timestamp: beat,
661
+ }
662
+
663
+ query := querySubsystems
664
+ if c.MaxSubsystems > 0 {
665
+ if total, err := c.countSubsystemsWith(doRow, ctx); err != nil {
666
+ c.logOnce("subsystem_count_failed", "failed to count subsystems before applying limit: %v", err)
667
+ } else if total > c.MaxSubsystems {
668
+ c.logOnce("subsystem_limit", "subsystem count (%d) exceeds limit (%d); truncating results", total, c.MaxSubsystems)
669
+ }
670
+ query = withFetchLimit(query, c.MaxSubsystems)
671
+ }
672
+
673
+ currentSubsystem := ""
674
+ err := do(ctx, "subsystems", query, func(column, value string, lineEnd bool) {
675
+ switch column {
676
+ case "SUBSYSTEM_NAME":
677
+ name := strings.TrimSpace(value)
678
+ if name == "" {
679
+ currentSubsystem = ""
680
+ return
681
+ }
682
+ if c.subsystemSelector != nil && !c.subsystemSelector.MatchString(name) {
683
+ currentSubsystem = ""
684
+ return
685
+ }
686
+ currentSubsystem = name
687
+ subsystem := subsystemMetrics{name: name, status: "ACTIVE"}
688
+ parts := strings.SplitN(name, "/", 2)
689
+ if len(parts) == 2 {
690
+ subsystem.library = parts[0]
691
+ subsystem.name = parts[1]
692
+ }
693
+ snapshot.meta[currentSubsystem] = subsystem
694
+ case "CURRENT_ACTIVE_JOBS":
695
+ if currentSubsystem != "" {
696
+ if v, ok := c.parseInt64Value(value, 1); ok {
697
+ if metrics, exists := snapshot.metrics[currentSubsystem]; exists {
698
+ metrics.CurrentActiveJobs = v
699
+ snapshot.metrics[currentSubsystem] = metrics
700
+ } else {
701
+ snapshot.metrics[currentSubsystem] = subsystemInstanceMetrics{CurrentActiveJobs: v}
702
+ }
703
+ }
704
+ }
705
+ case "MAXIMUM_ACTIVE_JOBS":
706
+ if currentSubsystem != "" {
707
+ if v, ok := c.parseInt64Value(value, 1); ok {
708
+ if metrics, exists := snapshot.metrics[currentSubsystem]; exists {
709
+ metrics.MaximumActiveJobs = v
710
+ snapshot.metrics[currentSubsystem] = metrics
711
+ } else {
712
+ snapshot.metrics[currentSubsystem] = subsystemInstanceMetrics{MaximumActiveJobs: v}
713
+ }
714
+ }
715
+ }
716
+ }
717
+
718
+ if lineEnd {
719
+ currentSubsystem = ""
720
+ }
721
+ })
722
+
723
+ if err != nil {
724
+ c.logQueryErrorOnce("slow_subsystems", query, err)
725
+ snapshot.err = err
726
+ return snapshot, err
727
+ }
728
+ c.clearErrorOnce("slow_subsystems")
729
+ snapshot.err = nil
730
+ return snapshot, nil
731
+}
732
+
733
+func (c *Collector) fetchPlanCache(ctx context.Context, beat time.Time, exec execFunc, do queryFunc) (planCacheSnapshot, error) {
734
+ snapshot := planCacheSnapshot{
735
+ values: make(map[string]planCacheInstanceMetrics),
736
+ meta: make(map[string]planCacheMetrics),
737
+ timestamp: beat,
738
+ }
739
+
740
+ if err := exec(ctx, callAnalyzePlanCache); err != nil {
741
+ c.logQueryErrorOnce("slow_plan_cache_analyze", callAnalyzePlanCache, err)
742
+ snapshot.err = fmt.Errorf("analyze plan cache: %w", err)
743
+ return snapshot, snapshot.err
744
+ }
745
+
746
+ var currentHeading string
747
+ err := do(ctx, "plan_cache_summary", queryPlanCacheSummary, func(column, value string, lineEnd bool) {
748
+ switch column {
749
+ case "HEADING":
750
+ currentHeading = strings.TrimSpace(value)
751
+ case "VALUE":
752
+ if currentHeading == "" {
753
+ return
754
+ }
755
+ key := planCacheMetricKey(currentHeading)
756
+ if key == "" {
757
+ return
758
+ }
759
+ if parsed, ok := c.parseInt64Value(value, precision); ok {
760
+ snapshot.values[key] = planCacheInstanceMetrics{Value: parsed}
761
+ snapshot.meta[key] = planCacheMetrics{heading: currentHeading}
762
+ }
763
+ }
764
+ if lineEnd {
765
+ currentHeading = ""
766
+ }
767
+ })
768
+
769
+ if err != nil {
770
+ c.logQueryErrorOnce("slow_plan_cache_summary", queryPlanCacheSummary, err)
771
+ snapshot.err = fmt.Errorf("plan cache summary: %w", err)
772
+ return snapshot, snapshot.err
773
+ }
774
+ c.clearErrorOnce("slow_plan_cache_summary")
775
+
776
+ return snapshot, nil
777
+}
778
+
779
+func (c *Collector) slowDoQuery(ctx context.Context, queryName, query string, assign func(column, value string, lineEnd bool)) error {
780
+ if c.slow.client == nil {
781
+ return errors.New("slow path client not initialised")
782
+ }
783
+
784
+ start := time.Now()
785
+ err := c.queryWithClient(ctx, c.slow.client, queryName, query, assign)
786
+ elapsed := time.Since(start)
787
+ c.slow.cache.addLatency(queryName, elapsed.Microseconds())
788
+ return err
789
+}
790
+
791
+func (c *Collector) slowDoQueryRow(ctx context.Context, queryName, query string, assign func(column, value string)) error {
792
+ if c.slow.client == nil {
793
+ return errors.New("slow path client not initialised")
794
+ }
795
+
796
+ start := time.Now()
797
+ err := c.queryRowWithClient(ctx, c.slow.client, queryName, query, assign)
798
+ elapsed := time.Since(start)
799
+ c.slow.cache.addLatency(queryName, elapsed.Microseconds())
800
+ return err
801
+}
802
+
803
+func (c *Collector) slowExec(ctx context.Context, query string) error {
804
+ if c.slow.client == nil {
805
+ return errors.New("slow path client not initialised")
806
+ }
807
+ start := time.Now()
808
+ err := c.execWithClient(ctx, c.slow.client, query)
809
+ elapsed := time.Since(start)
810
+ c.slow.cache.addLatency("analyze_plan_cache", elapsed.Microseconds())
811
+ return err
812
+}
813
+
814
+func (c *Collector) queryWithClient(ctx context.Context, client *as400proto.Client, queryName, query string, assign func(column, value string, lineEnd bool)) error {
815
+ return client.Query(ctx, query, func(columns []string, values []string) error {
816
+ for idx, col := range columns {
817
+ assign(col, values[idx], idx == len(columns)-1)
818
+ }
819
+ return nil
820
+ })
821
+}
822
+
823
+func (c *Collector) queryRowWithClient(ctx context.Context, client *as400proto.Client, queryName, query string, assign func(column, value string)) error {
824
+ return client.QueryWithLimit(ctx, query, 1, func(columns []string, values []string) error {
825
+ for idx, col := range columns {
826
+ assign(col, values[idx])
827
+ }
828
+ return nil
829
+ })
830
+}
831
+
832
+func (c *Collector) execWithClient(ctx context.Context, client *as400proto.Client, query string) error {
833
+ if err := client.Connect(ctx); err != nil {
834
+ return err
835
+ }
836
+ return client.Exec(ctx, query)
837
+}
src/go/plugin/ibm.d/modules/as400/sql_queries.go
+123
-101
@@ -5,6 +5,8 @@
5
6
package as400
7
8
+import "fmt"
9
+
10
const (
11
// VERIFIED: Comprehensive system status queries - works on IBM i 7.4+
12
// The reset variant matches legacy behaviour but clears global statistics.
@@ -92,6 +94,7 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
94
95
// System information queries for labels - only use verified columns
96
querySerialNumber = `SELECT SERIAL_NUMBER FROM TABLE(QSYS2.SYSTEM_STATUS()) X`
97
+ querySystemName = `SELECT HOST_NAME FROM QSYS2.SYSTEM_STATUS_INFO`
98
querySystemModel = `SELECT MACHINE_MODEL FROM TABLE(QSYS2.SYSTEM_STATUS()) X`
99
100
// Query for IBM i version - use SYSIBMADM.ENV_SYS_INFO which works across IBM i versions
@@ -142,35 +145,6 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
145
WHERE LINE_DESCRIPTION != '*LOOPBACK'
146
`
147
145
- queryCountMessageQueues = `
146
- SELECT COUNT(*) AS COUNT
147
- FROM (
148
- SELECT
149
- COALESCE(MESSAGE_QUEUE_LIBRARY, '') AS LIBRARY,
150
- COALESCE(MESSAGE_QUEUE_NAME, '') AS NAME
151
- FROM QSYS2.MESSAGE_QUEUE_INFO
152
- GROUP BY COALESCE(MESSAGE_QUEUE_LIBRARY, ''), COALESCE(MESSAGE_QUEUE_NAME, '')
153
- ) X
154
- `
155
-
156
- queryMessageQueueAggregates = `
157
- SELECT
158
- COALESCE(MESSAGE_QUEUE_LIBRARY, '*UNKNOWN') AS MESSAGE_QUEUE_LIBRARY,
159
- COALESCE(MESSAGE_QUEUE_NAME, '*UNKNOWN') AS MESSAGE_QUEUE_NAME,
160
- COUNT(*) AS MESSAGE_COUNT,
161
- SUM(CASE WHEN MESSAGE_TYPE = 'INFORMATIONAL' THEN 1 ELSE 0 END) AS INFORMATIONAL_MESSAGES,
162
- SUM(CASE WHEN MESSAGE_TYPE = 'INQUIRY' THEN 1 ELSE 0 END) AS INQUIRY_MESSAGES,
163
- SUM(CASE WHEN MESSAGE_TYPE = 'DIAGNOSTIC' THEN 1 ELSE 0 END) AS DIAGNOSTIC_MESSAGES,
164
- SUM(CASE WHEN MESSAGE_TYPE = 'ESCAPE' THEN 1 ELSE 0 END) AS ESCAPE_MESSAGES,
165
- SUM(CASE WHEN MESSAGE_TYPE = 'NOTIFY' THEN 1 ELSE 0 END) AS NOTIFY_MESSAGES,
166
- SUM(CASE WHEN MESSAGE_TYPE = 'SENDER COPY' THEN 1 ELSE 0 END) AS SENDER_COPY_MESSAGES,
167
- COALESCE(MAX(SEVERITY), 0) AS MAX_SEVERITY
168
- FROM QSYS2.MESSAGE_QUEUE_INFO
169
- GROUP BY MESSAGE_QUEUE_LIBRARY, MESSAGE_QUEUE_NAME
170
- ORDER BY MESSAGE_COUNT DESC
171
- FETCH FIRST %d ROWS ONLY
172
- `
173
-
148
// HTTP server monitoring
149
queryHTTPServerInfo = `
150
SELECT
@@ -217,23 +191,6 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
191
WHERE GLOBAL_BUCKET_NAME IS NOT NULL
192
`
193
220
- queryCountOutputQueues = `
221
- SELECT COUNT(*) AS COUNT
222
- FROM QSYS2.OUTPUT_QUEUE_INFO
223
- `
224
-
225
- queryOutputQueueInfo = `
226
- SELECT
227
- COALESCE(OUTPUT_QUEUE_LIBRARY_NAME, '*UNKNOWN') AS OUTPUT_QUEUE_LIBRARY_NAME,
228
- COALESCE(OUTPUT_QUEUE_NAME, '*UNKNOWN') AS OUTPUT_QUEUE_NAME,
229
- COALESCE(OUTPUT_QUEUE_STATUS, 'UNKNOWN') AS OUTPUT_QUEUE_STATUS,
230
- COALESCE(NUMBER_OF_FILES, 0) AS NUMBER_OF_FILES,
231
- COALESCE(NUMBER_OF_WRITERS, 0) AS NUMBER_OF_WRITERS
232
- FROM QSYS2.OUTPUT_QUEUE_INFO
233
- ORDER BY NUMBER_OF_FILES DESC
234
- FETCH FIRST %d ROWS ONLY
235
- `
236
-
194
// VERIFIED: Subsystem monitoring
195
// Works on both IBM i 7.4 and pub400.com
196
// Note: HELD_JOB_COUNT and STORAGE_USED_KB columns don't exist, removed per AS400.md verification
@@ -253,24 +210,29 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
210
WHERE STATUS = 'ACTIVE'
211
`
212
256
- // VERIFIED: Job queue monitoring
257
- // Works on both IBM i 7.4 and pub400.com
258
- // Note: HELD_JOB_COUNT column doesn't exist, removed per AS400.md verification
259
- queryJobQueues = `
260
- SELECT
261
- JOB_QUEUE_LIBRARY || '/' || JOB_QUEUE_NAME as QUEUE_NAME,
262
- NUMBER_OF_JOBS
263
- FROM QSYS2.JOB_QUEUE_INFO
264
- WHERE JOB_QUEUE_STATUS = 'RELEASED'
265
- ORDER BY NUMBER_OF_JOBS DESC
213
+ // Aggregated queue totals (expensive queries, run on batch path)
214
+ queryMessageQueueTotals = `
215
+ SELECT
216
+ COUNT(*) AS MESSAGE_COUNT,
217
+ COUNT(DISTINCT COALESCE(MESSAGE_QUEUE_LIBRARY, '') || '/' || COALESCE(MESSAGE_QUEUE_NAME, '')) AS QUEUE_COUNT
218
+ FROM QSYS2.MESSAGE_QUEUE_INFO
219
`
220
268
- queryCountJobQueues = `
269
- SELECT COUNT(*) as COUNT
221
+ queryJobQueueTotals = `
222
+ SELECT
223
+ COUNT(*) AS QUEUE_COUNT,
224
+ COALESCE(SUM(NUMBER_OF_JOBS), 0) AS JOB_COUNT
225
FROM QSYS2.JOB_QUEUE_INFO
226
WHERE JOB_QUEUE_STATUS = 'RELEASED'
227
`
228
229
+ queryOutputQueueTotals = `
230
+ SELECT
231
+ COUNT(*) AS QUEUE_COUNT,
232
+ COALESCE(SUM(NUMBER_OF_FILES), 0) AS FILE_COUNT
233
+ FROM QSYS2.OUTPUT_QUEUE_INFO
234
+ `
235
+
236
// Enhanced disk query with all metrics
237
queryDiskInstancesEnhanced = `
238
SELECT
@@ -292,44 +254,6 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
254
FROM QSYS2.SYSDISKSTAT
255
`
256
295
- // Top active jobs query using ACTIVE_JOB_INFO (requires IBM i 7.3+)
296
- queryTopActiveJobs = `
297
- SELECT
298
- JOB_NAME,
299
- JOB_STATUS,
300
- SUBSYSTEM,
301
- JOB_TYPE,
302
- ELAPSED_CPU_TIME,
303
- ELAPSED_TIME,
304
- TEMPORARY_STORAGE,
305
- CPU_PERCENTAGE,
306
- ELAPSED_INTERACTIVE_TRANSACTIONS,
307
- ELAPSED_TOTAL_DISK_IO_COUNT,
308
- THREAD_COUNT,
309
- RUN_PRIORITY
310
- FROM TABLE(QSYS2.ACTIVE_JOB_INFO(
311
- JOB_NAME_FILTER => '*ALL',
312
- SUBSYSTEM_LIST_FILTER => '*ALL',
313
- CURRENT_USER_LIST_FILTER => '*ALL',
314
- DETAILED_INFO => 'BASIC'
315
- )) X
316
- WHERE JOB_STATUS != '*JOBLOG PENDING'
317
- ORDER BY CPU_PERCENTAGE DESC
318
- FETCH FIRST %d ROWS ONLY
319
- `
320
-
321
- // Count active jobs for cardinality check
322
- queryCountActiveJobs = `
323
- SELECT COUNT(*) as COUNT
324
- FROM TABLE(QSYS2.ACTIVE_JOB_INFO(
325
- JOB_NAME_FILTER => '*ALL',
326
- SUBSYSTEM_LIST_FILTER => '*ALL',
327
- CURRENT_USER_LIST_FILTER => '*ALL',
328
- DETAILED_INFO => 'NONE'
329
- )) X
330
- WHERE JOB_STATUS != '*JOBLOG PENDING'
331
- `
332
-
257
// Remove all queries that reference non-existent tables/columns:
258
// - MESSAGE_QUEUE_INFO (doesn't exist)
259
// - SUBSYSTEM_INFO (columns don't exist)
@@ -344,10 +268,108 @@ FROM TABLE(QSYS2.SYSTEM_STATUS('NO','ALL'))`
268
// Plan cache analysis
269
callAnalyzePlanCache = `CALL QSYS2.ANALYZE_PLAN_CACHE('03', '', '', BX'', '%')`
270
queryPlanCacheSummary = `
347
- SELECT
348
- HEADING,
349
- VALUE
350
- FROM QTEMP.QDBOP00003
351
- WHERE VALUE IS NOT NULL
271
+ SELECT
272
+ HEADING,
273
+ VALUE
274
+ FROM QTEMP.QDBOP00003
275
+ WHERE VALUE IS NOT NULL
276
`
277
)
278
+
279
+func buildMessageQueueQuery(target queueTarget, useTableFunction bool) string {
280
+ if useTableFunction {
281
+ return fmt.Sprintf(`
282
+SELECT
283
+ '%[1]s' AS MESSAGE_QUEUE_LIBRARY,
284
+ '%[2]s' AS MESSAGE_QUEUE_NAME,
285
+ COUNT(*) AS MESSAGE_COUNT,
286
+ SUM(CASE WHEN MESSAGE_TYPE = 'INFORMATIONAL' THEN 1 ELSE 0 END) AS INFORMATIONAL_MESSAGES,
287
+ SUM(CASE WHEN MESSAGE_TYPE = 'INQUIRY' THEN 1 ELSE 0 END) AS INQUIRY_MESSAGES,
288
+ SUM(CASE WHEN MESSAGE_TYPE = 'DIAGNOSTIC' THEN 1 ELSE 0 END) AS DIAGNOSTIC_MESSAGES,
289
+ SUM(CASE WHEN MESSAGE_TYPE = 'ESCAPE' THEN 1 ELSE 0 END) AS ESCAPE_MESSAGES,
290
+ SUM(CASE WHEN MESSAGE_TYPE = 'NOTIFY' THEN 1 ELSE 0 END) AS NOTIFY_MESSAGES,
291
+ SUM(CASE WHEN MESSAGE_TYPE = 'SENDER COPY' THEN 1 ELSE 0 END) AS SENDER_COPY_MESSAGES,
292
+ COALESCE(MAX(SEVERITY), 0) AS MAX_SEVERITY
293
+FROM TABLE(QSYS2.MESSAGE_QUEUE_INFO(
294
+ QUEUE_LIBRARY => '%[1]s',
295
+ QUEUE_NAME => '%[2]s',
296
+ MESSAGE_FILTER => '*ALL'
297
+)) MQ
298
+`, target.Library, target.Name)
299
+ }
300
+
301
+ return fmt.Sprintf(`
302
+SELECT
303
+ COUNT(*) AS MESSAGE_COUNT,
304
+ SUM(CASE WHEN MESSAGE_TYPE = 'INFORMATIONAL' THEN 1 ELSE 0 END) AS INFORMATIONAL_MESSAGES,
305
+ SUM(CASE WHEN MESSAGE_TYPE = 'INQUIRY' THEN 1 ELSE 0 END) AS INQUIRY_MESSAGES,
306
+ SUM(CASE WHEN MESSAGE_TYPE = 'DIAGNOSTIC' THEN 1 ELSE 0 END) AS DIAGNOSTIC_MESSAGES,
307
+ SUM(CASE WHEN MESSAGE_TYPE = 'ESCAPE' THEN 1 ELSE 0 END) AS ESCAPE_MESSAGES,
308
+ SUM(CASE WHEN MESSAGE_TYPE = 'NOTIFY' THEN 1 ELSE 0 END) AS NOTIFY_MESSAGES,
309
+ SUM(CASE WHEN MESSAGE_TYPE = 'SENDER COPY' THEN 1 ELSE 0 END) AS SENDER_COPY_MESSAGES,
310
+ COALESCE(MAX(SEVERITY), 0) AS MAX_SEVERITY
311
+FROM QSYS2.MESSAGE_QUEUE_INFO
312
+WHERE MESSAGE_QUEUE_LIBRARY = '%[1]s'
313
+ AND MESSAGE_QUEUE_NAME = '%[2]s'
314
+`, target.Library, target.Name)
315
+}
316
+
317
+func buildActiveJobQuery(target activeJobTarget) string {
318
+ return fmt.Sprintf(`
319
+SELECT
320
+ JOB_NAME,
321
+ JOB_USER,
322
+ JOB_NUMBER,
323
+ JOB_STATUS,
324
+ SUBSYSTEM,
325
+ JOB_TYPE,
326
+ ELAPSED_CPU_TIME,
327
+ ELAPSED_TIME,
328
+ TEMPORARY_STORAGE,
329
+ CPU_PERCENTAGE,
330
+ ELAPSED_INTERACTIVE_TRANSACTIONS,
331
+ ELAPSED_TOTAL_DISK_IO_COUNT,
332
+ THREAD_COUNT
333
+FROM TABLE(QSYS2.ACTIVE_JOB_INFO(
334
+ JOB_NAME_FILTER => '%[3]s',
335
+ SUBSYSTEM_LIST_FILTER => '*ALL',
336
+ CURRENT_USER_LIST_FILTER => '%[2]s',
337
+ JOB_STATUS_FILTER => '*ALL',
338
+ JOB_TYPE_LIST_FILTER => '*ALL',
339
+ DETAILED_INFO => 'BASIC'
340
+)) X
341
+WHERE JOB_NUMBER = '%[1]s'
342
+`, target.Number, target.User, target.Name)
343
+}
344
+
345
+func buildJobQueueQuery(target queueTarget) string {
346
+ return fmt.Sprintf(`
347
+SELECT
348
+ JOB_QUEUE_STATUS,
349
+ COALESCE(NUMBER_OF_JOBS, 0) AS NUMBER_OF_JOBS,
350
+ COALESCE(RELEASED_JOBS, 0) AS RELEASED_JOBS,
351
+ COALESCE(SCHEDULED_JOBS, 0) AS SCHEDULED_JOBS,
352
+ COALESCE(HELD_JOBS, 0) AS HELD_JOBS
353
+FROM QSYS2.JOB_QUEUE_INFO
354
+WHERE JOB_QUEUE_LIBRARY = '%[1]s'
355
+ AND JOB_QUEUE_NAME = '%[2]s'
356
+`, target.Library, target.Name)
357
+}
358
+
359
+func buildOutputQueueEntriesQuery(target queueTarget) string {
360
+ return fmt.Sprintf(`SELECT * FROM TABLE(QSYS2.OUTPUT_QUEUE_ENTRIES('%[1]s', '%[2]s', '*NO'))`, target.Library, target.Name)
361
+}
362
+
363
+func buildOutputQueueInfoQuery(target queueTarget) string {
364
+ return fmt.Sprintf(`
365
+SELECT
366
+ OUTPUT_QUEUE_LIBRARY_NAME,
367
+ OUTPUT_QUEUE_NAME,
368
+ OUTPUT_QUEUE_STATUS,
369
+ COALESCE(NUMBER_OF_FILES, 0) AS NUMBER_OF_FILES,
370
+ COALESCE(NUMBER_OF_WRITERS, 0) AS NUMBER_OF_WRITERS
371
+FROM QSYS2.OUTPUT_QUEUE_INFO
372
+WHERE OUTPUT_QUEUE_LIBRARY_NAME = '%[1]s'
373
+ AND OUTPUT_QUEUE_NAME = '%[2]s'
374
+`, target.Library, target.Name)
375
+}
src/go/plugin/ibm.d/modules/db2/config_schema.json
+58
-29
@@ -4,183 +4,212 @@
4
"properties": {
5
"backup_history_days": {
6
"default": 30,
7
- "title": "BackupHistoryDays controls how many days of backup history are retrieved.",
7
+ "description": "BackupHistoryDays controls how many days of backup history are retrieved.",
8
+ "title": "Backup History Days",
9
"type": "integer"
10
},
11
"collect_bufferpool_metrics": {
12
"default": "auto",
13
+ "description": "CollectBufferpoolMetrics toggles buffer pool efficiency metrics.",
14
"enum": [
15
"auto",
16
"enabled",
17
"disabled"
18
],
17
- "title": "CollectBufferpoolMetrics toggles buffer pool efficiency metrics.",
19
+ "title": "Collect Bufferpool Metrics",
20
"type": "string"
21
},
22
"collect_bufferpools_matching": {
23
"default": "",
22
- "title": "CollectBufferpoolsMatching filters buffer pools by name using glob patterns.",
24
+ "description": "CollectBufferpoolsMatching filters buffer pools by name using glob patterns.",
25
+ "title": "Collect Bufferpools Matching",
26
"type": "string"
27
},
28
"collect_connection_metrics": {
29
"default": "auto",
30
+ "description": "CollectConnectionMetrics toggles per-connection activity metrics.",
31
"enum": [
32
"auto",
33
"enabled",
34
"disabled"
35
],
32
- "title": "CollectConnectionMetrics toggles per-connection activity metrics.",
36
+ "title": "Collect Connection Metrics",
37
"type": "string"
38
},
39
"collect_connections_matching": {
40
"default": "",
37
- "title": "CollectConnectionsMatching filters monitored connections by application ID.",
41
+ "description": "CollectConnectionsMatching filters monitored connections by application ID.",
42
+ "title": "Collect Connections Matching",
43
"type": "string"
44
},
45
"collect_database_metrics": {
46
"default": "auto",
47
+ "description": "CollectDatabaseMetrics toggles high-level database status metrics.",
48
"enum": [
49
"auto",
50
"enabled",
51
"disabled"
52
],
47
- "title": "CollectDatabaseMetrics toggles high-level database status metrics.",
53
+ "title": "Collect Database Metrics",
54
"type": "string"
55
},
56
"collect_databases_matching": {
57
"default": "",
52
- "title": "CollectDatabasesMatching filters databases by name using glob patterns.",
58
+ "description": "CollectDatabasesMatching filters databases by name using glob patterns.",
59
+ "title": "Collect Databases Matching",
60
"type": "string"
61
},
62
"collect_index_metrics": {
63
"default": "auto",
64
+ "description": "CollectIndexMetrics toggles index usage metrics.",
65
"enum": [
66
"auto",
67
"enabled",
68
"disabled"
69
],
62
- "title": "CollectIndexMetrics toggles index usage metrics.",
70
+ "title": "Collect Index Metrics",
71
"type": "string"
72
},
73
"collect_indexes_matching": {
74
"default": "",
67
- "title": "CollectIndexesMatching filters indexes by schema/name.",
75
+ "description": "CollectIndexesMatching filters indexes by schema/name.",
76
+ "title": "Collect Indexes Matching",
77
"type": "string"
78
},
79
"collect_lock_metrics": {
80
"default": "auto",
81
+ "description": "CollectLockMetrics toggles lock contention metrics.",
82
"enum": [
83
"auto",
84
"enabled",
85
"disabled"
86
],
77
- "title": "CollectLockMetrics toggles lock contention metrics.",
87
+ "title": "Collect Lock Metrics",
88
"type": "string"
89
},
90
"collect_memory_metrics": {
91
"default": true,
82
- "title": "CollectMemoryMetrics enables memory pool statistics.",
92
+ "description": "CollectMemoryMetrics enables memory pool statistics.",
93
+ "title": "Collect Memory Metrics",
94
"type": "boolean"
95
},
96
"collect_table_io_metrics": {
97
"default": true,
87
- "title": "CollectTableIOMetrics enables table I/O statistics when available.",
98
+ "description": "CollectTableIOMetrics enables table I/O statistics when available.",
99
+ "title": "Collect Table Io Metrics",
100
"type": "boolean"
101
},
102
"collect_table_metrics": {
103
"default": "auto",
104
+ "description": "CollectTableMetrics toggles table-level size and row metrics.",
105
"enum": [
106
"auto",
107
"enabled",
108
"disabled"
109
],
97
- "title": "CollectTableMetrics toggles table-level size and row metrics.",
110
+ "title": "Collect Table Metrics",
111
"type": "string"
112
},
113
"collect_tables_matching": {
114
"default": "",
102
- "title": "CollectTablesMatching filters tables by schema/name.",
115
+ "description": "CollectTablesMatching filters tables by schema/name.",
116
+ "title": "Collect Tables Matching",
117
"type": "string"
118
},
119
"collect_tablespace_metrics": {
120
"default": "auto",
121
+ "description": "CollectTablespaceMetrics toggles tablespace capacity metrics.",
122
"enum": [
123
"auto",
124
"enabled",
125
"disabled"
126
],
112
- "title": "CollectTablespaceMetrics toggles tablespace capacity metrics.",
127
+ "title": "Collect Tablespace Metrics",
128
"type": "string"
129
},
130
"collect_tablespaces_matching": {
131
"default": "",
117
- "title": "CollectTablespacesMatching filters tablespaces by name using glob patterns.",
132
+ "description": "CollectTablespacesMatching filters tablespaces by name using glob patterns.",
133
+ "title": "Collect Tablespaces Matching",
134
"type": "string"
135
},
136
"collect_wait_metrics": {
137
"default": true,
122
- "title": "CollectWaitMetrics enables wait time statistics (locks, logs, I/O).",
138
+ "description": "CollectWaitMetrics enables wait time statistics (locks, logs, I/O).",
139
+ "title": "Collect Wait Metrics",
140
"type": "boolean"
141
},
142
"dsn": {
143
"default": "",
127
- "title": "DSN provides a full DB2 connection string when manual control is required.",
144
+ "description": "DSN provides a full DB2 connection string when manual control is required.",
145
+ "title": "DSN",
146
"type": "string"
147
},
148
"max_bufferpools": {
149
"default": 20,
132
- "title": "MaxBufferpools caps the number of buffer pools charted.",
150
+ "description": "MaxBufferpools caps the number of buffer pools charted.",
151
+ "title": "Max Bufferpools",
152
"type": "integer"
153
},
154
"max_connections": {
155
"default": 200,
137
- "title": "MaxConnections caps the number of connection instances charted.",
156
+ "description": "MaxConnections caps the number of connection instances charted.",
157
+ "title": "Max Connections",
158
"type": "integer"
159
},
160
"max_databases": {
161
"default": 10,
142
- "title": "MaxDatabases caps the number of databases charted.",
162
+ "description": "MaxDatabases caps the number of databases charted.",
163
+ "title": "Max Databases",
164
"type": "integer"
165
},
166
"max_db_conns": {
167
"default": 1,
147
- "title": "MaxDbConns limits the connection pool size.",
168
+ "description": "MaxDbConns limits the connection pool size.",
169
+ "title": "Max Db Conns",
170
"type": "integer"
171
},
172
"max_db_life_time": {
173
"default": 600000000000,
152
- "title": "MaxDbLifeTime forces pooled connections to be recycled after the specified duration.",
174
+ "description": "MaxDbLifeTime forces pooled connections to be recycled after the specified duration.",
175
+ "title": "Max Db Life Time",
176
"type": "integer"
177
},
178
"max_indexes": {
179
"default": 100,
157
- "title": "MaxIndexes caps the number of indexes charted.",
180
+ "description": "MaxIndexes caps the number of indexes charted.",
181
+ "title": "Max Indexes",
182
"type": "integer"
183
},
184
"max_tables": {
185
"default": 50,
162
- "title": "MaxTables caps the number of tables charted.",
186
+ "description": "MaxTables caps the number of tables charted.",
187
+ "title": "Max Tables",
188
"type": "integer"
189
},
190
"max_tablespaces": {
191
"default": 100,
167
- "title": "MaxTablespaces caps the number of tablespaces charted.",
192
+ "description": "MaxTablespaces caps the number of tablespaces charted.",
193
+ "title": "Max Tablespaces",
194
"type": "integer"
195
},
196
"timeout": {
197
"default": 2000000000,
172
- "title": "Timeout controls how long DB2 RPCs may run before cancellation.",
198
+ "description": "Timeout controls how long DB2 RPCs may run before cancellation.",
199
+ "title": "Timeout",
200
"type": "integer"
201
},
202
"update_every": {
203
"default": 5,
204
+ "description": "Data collection frequency",
205
"minimum": 1,
178
- "title": "Data collection frequency",
206
+ "title": "Update Every",
207
"type": "integer"
208
},
209
"vnode": {
210
"default": "",
183
- "title": "Vnode allows binding the collector to a virtual node.",
211
+ "description": "Vnode allows binding the collector to a virtual node.",
212
+ "title": "Vnode",
213
"type": "string"
214
}
215
},
src/go/plugin/ibm.d/modules/mq/collect_sys_topics.go
+14
-2
@@ -2,6 +2,7 @@ package mq
2
3
import (
4
"github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/mq/contexts"
5
+ "github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/pcf"
6
)
7
8
// collectSysTopics collects resource metrics using IBM MQ's resource monitoring system
@@ -21,18 +22,29 @@ func (c *Collector) collectSysTopics() error {
22
return nil
23
}
24
25
+ if c.client.GetResourceStatus() == pcf.ResourceStatusFailed {
26
+ // Previous attempts already determined this isn't available; skip spamming logs
27
+ return nil
28
+ }
29
+
30
// Enable resource monitoring on first use
31
if err := c.client.EnableResourceMonitoring(); err != nil {
26
- c.Warningf("Failed to enable resource monitoring: %v", err)
32
+ c.warnOnce("resource_monitoring_enable", "Failed to enable resource monitoring: %v", err)
33
+ if c.client.GetResourceStatus() == pcf.ResourceStatusFailed {
34
+ // Stop trying in future iterations
35
+ c.Config.CollectSysTopics = false
36
+ }
37
return nil // Don't fail the entire collection
38
}
39
+ c.clearWarnOnce("resource_monitoring_enable")
40
41
// Get resource publications
42
result, err := c.client.GetResourcePublications()
43
if err != nil {
33
- c.Warningf("Failed to get resource publications: %v", err)
44
+ c.warnOnce("resource_monitoring_publications", "Failed to get resource publications: %v", err)
45
return nil // Don't fail the entire collection
46
}
47
+ c.clearWarnOnce("resource_monitoring_publications")
48
49
c.Debugf("Retrieved %d resource publications", result.Stats.Discovery.AvailableItems)
50
src/go/plugin/ibm.d/modules/mq/collector.go
+57
-7
@@ -1,6 +1,9 @@
1
package mq
2
3
import (
4
+ "sync"
5
+ "time"
6
+
7
"github.com/netdata/netdata/go/plugins/plugin/ibm.d/framework"
8
"github.com/netdata/netdata/go/plugins/plugin/ibm.d/modules/mq/contexts"
9
"github.com/netdata/netdata/go/plugins/plugin/ibm.d/protocols/pcf"
@@ -15,6 +18,39 @@ type Collector struct {
18
// Resolved effective intervals (auto-detected or user-configured)
19
effectiveStatisticsInterval int
20
effectiveSysTopicInterval int
21
+
22
+ // Warning throttling
23
+ warnMu sync.Mutex
24
+ warns map[string]time.Time
25
+}
26
+
27
+const warnThrottleInterval = 10 * time.Minute
28
+
29
+func (c *Collector) warnOnce(key string, format string, args ...interface{}) {
30
+ c.warnMu.Lock()
31
+ defer c.warnMu.Unlock()
32
+
33
+ if c.warns == nil {
34
+ c.warns = make(map[string]time.Time)
35
+ }
36
+
37
+ now := time.Now()
38
+ if last, ok := c.warns[key]; ok && now.Sub(last) < warnThrottleInterval {
39
+ return
40
+ }
41
+
42
+ c.Warningf(format, args...)
43
+ c.warns[key] = now
44
+}
45
+
46
+func (c *Collector) clearWarnOnce(key string) {
47
+ c.warnMu.Lock()
48
+ defer c.warnMu.Unlock()
49
+
50
+ if c.warns == nil {
51
+ return
52
+ }
53
+ delete(c.warns, key)
54
}
55
56
// CollectOnce is called by the framework to collect metrics.
@@ -39,48 +75,62 @@ func (c *Collector) CollectOnce() error {
75
76
// Collect queue manager metrics
77
if err := c.collectQueueManagerMetrics(); err != nil {
42
- return err
78
+ c.warnOnce("queue_manager_metrics", "failed to collect queue manager metrics: %v", err)
79
+ } else {
80
+ c.clearWarnOnce("queue_manager_metrics")
81
}
82
83
// Collect queue metrics
84
if c.Config.CollectQueues {
85
if err := c.collectQueueMetrics(); err != nil {
48
- c.Warningf("failed to collect queue metrics: %v", err)
86
+ c.warnOnce("queue_metrics", "failed to collect queue metrics: %v", err)
87
+ } else {
88
+ c.clearWarnOnce("queue_metrics")
89
}
90
}
91
92
// Collect channel metrics
93
if c.Config.CollectChannels {
94
if err := c.collectChannelMetrics(); err != nil {
55
- c.Warningf("failed to collect channel metrics: %v", err)
95
+ c.warnOnce("channel_metrics", "failed to collect channel metrics: %v", err)
96
+ } else {
97
+ c.clearWarnOnce("channel_metrics")
98
}
99
}
100
101
// Collect topic metrics
102
if c.Config.CollectTopics {
103
if err := c.collectTopicMetrics(); err != nil {
62
- c.Warningf("failed to collect topic metrics: %v", err)
104
+ c.warnOnce("topic_metrics", "failed to collect topic metrics: %v", err)
105
+ } else {
106
+ c.clearWarnOnce("topic_metrics")
107
}
108
}
109
110
// Collect listener metrics
111
if c.Config.CollectListeners {
112
if err := c.collectListenerMetrics(); err != nil {
69
- c.Warningf("failed to collect listener metrics: %v", err)
113
+ c.warnOnce("listener_metrics", "failed to collect listener metrics: %v", err)
114
+ } else {
115
+ c.clearWarnOnce("listener_metrics")
116
}
117
}
118
119
// Collect subscription metrics
120
if c.Config.CollectSubscriptions {
121
if err := c.collectSubscriptions(); err != nil {
76
- c.Warningf("failed to collect subscription metrics: %v", err)
122
+ c.warnOnce("subscription_metrics", "failed to collect subscription metrics: %v", err)
123
+ } else {
124
+ c.clearWarnOnce("subscription_metrics")
125
}
126
}
127
128
// Collect statistics queue metrics (advanced metrics) - every iteration
129
if c.Config.CollectStatisticsQueue {
130
if err := c.collectStatistics(); err != nil {
83
- c.Warningf("failed to collect statistics queue metrics: %v", err)
131
+ c.warnOnce("statistics_queue", "failed to collect statistics queue metrics: %v", err)
132
+ } else {
133
+ c.clearWarnOnce("statistics_queue")
134
}
135
}
136
src/go/plugin/ibm.d/modules/mq/config_schema.json
+64
-32
@@ -4,166 +4,198 @@
4
"properties": {
5
"channel": {
6
"default": "SYSTEM.DEF.SVRCONN",
7
- "title": "IBM MQ channel name for connection",
7
+ "description": "IBM MQ channel name for connection",
8
+ "title": "Channel",
9
"type": "string"
10
},
11
"channel_selector": {
12
"default": "",
12
- "title": "Pattern to filter channels (wildcards supported)",
13
+ "description": "Pattern to filter channels (wildcards supported)",
14
+ "title": "Channel Selector",
15
"type": "string"
16
},
17
"collect_channel_config": {
18
"default": true,
17
- "title": "Enable collection of channel configuration metrics",
19
+ "description": "Enable collection of channel configuration metrics",
20
+ "title": "Collect Channel Config",
21
"type": "boolean"
22
},
23
"collect_channels": {
24
"default": true,
22
- "title": "Enable collection of channel metrics",
25
+ "description": "Enable collection of channel metrics",
26
+ "title": "Collect Channels",
27
"type": "boolean"
28
},
29
"collect_listeners": {
30
"default": true,
27
- "title": "Enable collection of listener metrics",
31
+ "description": "Enable collection of listener metrics",
32
+ "title": "Collect Listeners",
33
"type": "boolean"
34
},
35
"collect_queue_config": {
36
"default": true,
32
- "title": "Enable collection of queue configuration metrics",
37
+ "description": "Enable collection of queue configuration metrics",
38
+ "title": "Collect Queue Config",
39
"type": "boolean"
40
},
41
"collect_queues": {
42
"default": true,
37
- "title": "Enable collection of queue metrics",
43
+ "description": "Enable collection of queue metrics",
44
+ "title": "Collect Queues",
45
"type": "boolean"
46
},
47
"collect_reset_queue_stats": {
48
"default": false,
42
- "title": "Enable collection of queue statistics (destructive operation)",
49
+ "description": "Enable collection of queue statistics (destructive operation)",
50
+ "title": "Collect Reset Queue Stats",
51
"type": "boolean"
52
},
53
"collect_statistics_queue": {
54
"default": false,
47
- "title": "Enable collection of statistics queue metrics (SYSTEM.ADMIN.STATISTICS.QUEUE provides advanced metrics like min/max depth)",
55
+ "description": "Enable collection of statistics queue metrics (SYSTEM.ADMIN.STATISTICS.QUEUE provides advanced metrics like min/max depth)",
56
+ "title": "Collect Statistics Queue",
57
"type": "boolean"
58
},
59
"collect_subscriptions": {
60
"default": true,
52
- "title": "Enable collection of subscription metrics",
61
+ "description": "Enable collection of subscription metrics",
62
+ "title": "Collect Subscriptions",
63
"type": "boolean"
64
},
65
"collect_sys_topics": {
66
"default": false,
57
- "title": "Enable collection of $SYS topic metrics (provides Queue Manager CPU, memory, and log utilization)",
67
+ "description": "Enable collection of $SYS topic metrics (provides Queue Manager CPU, memory, and log utilization)",
68
+ "title": "Collect Sys Topics",
69
"type": "boolean"
70
},
71
"collect_system_channels": {
72
"default": true,
62
- "title": "Enable collection of system channel metrics (SYSTEM.* channels show clustering and administrative health)",
73
+ "description": "Enable collection of system channel metrics (SYSTEM.* channels show clustering and administrative health)",
74
+ "title": "Collect System Channels",
75
"type": "boolean"
76
},
77
"collect_system_listeners": {
78
"default": true,
67
- "title": "Enable collection of system listener metrics (SYSTEM.* listeners show internal connectivity)",
79
+ "description": "Enable collection of system listener metrics (SYSTEM.* listeners show internal connectivity)",
80
+ "title": "Collect System Listeners",
81
"type": "boolean"
82
},
83
"collect_system_queues": {
84
"default": true,
72
- "title": "Enable collection of system queue metrics (SYSTEM.* queues provide critical infrastructure visibility)",
85
+ "description": "Enable collection of system queue metrics (SYSTEM.* queues provide critical infrastructure visibility)",
86
+ "title": "Collect System Queues",
87
"type": "boolean"
88
},
89
"collect_system_topics": {
90
"default": true,
77
- "title": "Enable collection of system topic metrics (SYSTEM.* topics show internal messaging patterns)",
91
+ "description": "Enable collection of system topic metrics (SYSTEM.* topics show internal messaging patterns)",
92
+ "title": "Collect System Topics",
93
"type": "boolean"
94
},
95
"collect_topics": {
96
"default": true,
82
- "title": "Enable collection of topic metrics",
97
+ "description": "Enable collection of topic metrics",
98
+ "title": "Collect Topics",
99
"type": "boolean"
100
},
101
"host": {
102
"default": "localhost",
87
- "title": "IBM MQ server hostname or IP address",
103
+ "description": "IBM MQ server hostname or IP address",
104
+ "title": "Host",
105
"type": "string"
106
},
107
"listener_selector": {
108
"default": "",
92
- "title": "Pattern to filter listeners (wildcards supported)",
109
+ "description": "Pattern to filter listeners (wildcards supported)",
110
+ "title": "Listener Selector",
111
"type": "string"
112
},
113
"max_channels": {
114
"default": 100,
97
- "title": "Maximum number of channels to collect (0 = no limit)",
115
+ "description": "Maximum number of channels to collect (0 = no limit)",
116
+ "title": "Max Channels",
117
"type": "integer"
118
},
119
"max_listeners": {
120
"default": 100,
102
- "title": "Maximum number of listeners to collect (0 = no limit)",
121
+ "description": "Maximum number of listeners to collect (0 = no limit)",
122
+ "title": "Max Listeners",
123
"type": "integer"
124
},
125
"max_queues": {
126
"default": 100,
107
- "title": "Maximum number of queues to collect (0 = no limit)",
127
+ "description": "Maximum number of queues to collect (0 = no limit)",
128
+ "title": "Max Queues",
129
"type": "integer"
130
},
131
"max_topics": {
132
"default": 100,
112
- "title": "Maximum number of topics to collect (0 = no limit)",
133
+ "description": "Maximum number of topics to collect (0 = no limit)",
134
+ "title": "Max Topics",
135
"type": "integer"
136
},
137
"password": {
138
"default": "",
139
+ "description": "Password for IBM MQ authentication",
140
"format": "password",
118
- "title": "Password for IBM MQ authentication",
141
+ "title": "Password",
142
"type": "string"
143
},
144
"port": {
145
"default": 1414,
146
+ "description": "IBM MQ server port number",
147
"maximum": 65535,
148
"minimum": 1,
125
- "title": "IBM MQ server port number",
149
+ "title": "Port",
150
"type": "integer"
151
},
152
"queue_manager": {
153
"default": "QM1",
130
- "title": "IBM MQ Queue Manager name to connect to",
154
+ "description": "IBM MQ Queue Manager name to connect to",
155
+ "title": "Queue Manager",
156
"type": "string"
157
},
158
"queue_selector": {
159
"default": "",
135
- "title": "Pattern to filter queues (wildcards supported)",
160
+ "description": "Pattern to filter queues (wildcards supported)",
161
+ "title": "Queue Selector",
162
"type": "string"
163
},
164
"statistics_interval,omitempty": {
165
"default": 60,
140
- "title": "Statistics collection interval in seconds (auto-detected STATINT overwrites this value)",
166
+ "description": "Statistics collection interval in seconds (auto-detected STATINT overwrites this value)",
167
+ "title": "Statistics Interval,omitempty",
168
"type": "integer"
169
},
170
"subscription_selector": {
171
"default": "",
145
- "title": "Pattern to filter subscriptions (wildcards supported)",
172
+ "description": "Pattern to filter subscriptions (wildcards supported)",
173
+ "title": "Subscription Selector",
174
"type": "string"
175
},
176
"sys_topic_interval,omitempty": {
177
"default": 10,
150
- "title": "$SYS topic collection interval in seconds (user override for customized MQ configurations)",
178
+ "description": "$SYS topic collection interval in seconds (user override for customized MQ configurations)",
179
+ "title": "Sys Topic Interval,omitempty",
180
"type": "integer"
181
},
182
"topic_selector": {
183
"default": "",
155
- "title": "Pattern to filter topics (wildcards supported)",
184
+ "description": "Pattern to filter topics (wildcards supported)",
185
+ "title": "Topic Selector",
186
"type": "string"
187
},
188
"update_every": {
189
"default": 10,
190
+ "description": "Data collection frequency",
191
"minimum": 1,
161
- "title": "Data collection frequency",
192
+ "title": "Update Every",
193
"type": "integer"
194
},
195
"user": {
196
"default": "",
166
- "title": "Username for IBM MQ authentication",
197
+ "description": "Username for IBM MQ authentication",
198
+ "title": "User",
199
"type": "string"
200
}
201
},
src/go/plugin/ibm.d/pkg/odbcbridge/connection.go
+16
-4
@@ -14,12 +14,14 @@ package odbcbridge
14
*/
15
import "C"
16
import (
17
+ "bytes"
18
"context"
19
"database/sql/driver"
20
"errors"
21
"fmt"
22
"io"
23
"strconv"
24
+ "strings"
25
"sync"
26
"unsafe"
27
)
@@ -30,6 +32,16 @@ type OptimizedConnection struct {
32
mu sync.Mutex
33
}
34
35
+func cleanErrorBuffer(buf []byte) string {
36
+ if len(buf) == 0 {
37
+ return ""
38
+ }
39
+ if idx := bytes.IndexByte(buf, 0); idx >= 0 {
40
+ buf = buf[:idx]
41
+ }
42
+ return strings.TrimSpace(string(buf))
43
+}
44
+
45
// ConnectOptimized establishes a new optimized ODBC connection
46
func ConnectOptimized(dsn string) (*OptimizedConnection, error) {
47
cDSN := C.CString(dsn)
@@ -39,7 +51,7 @@ func ConnectOptimized(dsn string) (*OptimizedConnection, error) {
51
handle := C.odbc_connect(cDSN, (*C.char)(unsafe.Pointer(&errorBuf[0])), C.int(len(errorBuf)))
52
53
if handle == nil {
42
- return nil, fmt.Errorf("connection failed: %s", string(errorBuf))
54
+ return nil, fmt.Errorf("connection failed: %s", cleanErrorBuffer(errorBuf))
55
}
56
57
return &OptimizedConnection{handle: handle}, nil
@@ -93,7 +105,7 @@ func (c *OptimizedConnection) QueryContext(ctx context.Context, query string) (*
105
ret := C.odbc_execute_direct(c.handle, cQuery, (*C.char)(unsafe.Pointer(&errorBuf[0])), C.int(len(errorBuf)))
106
107
if ret != Success {
96
- return nil, fmt.Errorf("%w: %s", ErrQueryFailed, string(errorBuf))
108
+ return nil, fmt.Errorf("%w: %s", ErrQueryFailed, cleanErrorBuffer(errorBuf))
109
}
110
111
// Get metadata
@@ -146,7 +158,7 @@ func (c *OptimizedConnection) PrepareContext(ctx context.Context, query string)
158
ret := C.odbc_prepare(c.handle, cQuery, (*C.char)(unsafe.Pointer(&errorBuf[0])), C.int(len(errorBuf)))
159
160
if ret != Success {
149
- return nil, fmt.Errorf("prepare failed: %s", string(errorBuf))
161
+ return nil, fmt.Errorf("prepare failed: %s", cleanErrorBuffer(errorBuf))
162
}
163
164
return &PreparedStatement{conn: c, ctx: ctx}, nil
@@ -299,7 +311,7 @@ func (s *PreparedStatement) Execute() (*OptimizedRows, error) {
311
ret := C.odbc_execute(s.conn.handle, (*C.char)(unsafe.Pointer(&errorBuf[0])), C.int(len(errorBuf)))
312
313
if ret != Success {
302
- return nil, fmt.Errorf("execute failed: %s", string(errorBuf))
314
+ return nil, fmt.Errorf("execute failed: %s", cleanErrorBuffer(errorBuf))
315
}
316
317
// Get metadata (same as QueryContext)
src/go/plugin/ibm.d/protocols/pcf/client_core.go
+54
@@ -7,6 +7,8 @@ package pcf
7
8
import (
9
"fmt"
10
+ "strings"
11
+ "sync"
12
"time"
13
14
"github.com/ibm-messaging/mq-golang/v5/ibmmq"
@@ -60,6 +62,10 @@ type Client struct {
62
resourceMonitoringEnabled bool // Track if resource monitoring should be enabled
63
resourceStatus ResourceStatus // Global status: disabled(0), enabled(1), failed(2)
64
metricsReady bool // Track if metrics connection is ready
65
+
66
+ // Warning throttling
67
+ warnMu sync.Mutex
68
+ warnLast map[string]time.Time
69
}
70
71
// Config is the configuration for the PCF client.
@@ -78,6 +84,7 @@ func NewClient(config Config, state *framework.CollectorState) *Client {
84
config: config,
85
protocol: framework.NewProtocolClient("pcf", state),
86
jobCreationTime: time.Now(), // Record job creation time for statistics filtering
87
+ warnLast: make(map[string]time.Time),
88
}
89
}
90
@@ -147,3 +154,50 @@ func (c *Client) GetResourceStatus() ResourceStatus {
154
func (c *Client) IsResourceMonitoringAvailable() bool {
155
return c.metricsReady
156
}
157
+
158
+const warnThrottleInterval = 10 * time.Minute
159
+
160
+// warnOnce logs a warning at most once per throttle interval for the given key
161
+func (c *Client) warnOnce(key string, format string, args ...interface{}) {
162
+ c.warnMu.Lock()
163
+ defer c.warnMu.Unlock()
164
+
165
+ if c.warnLast == nil {
166
+ c.warnLast = make(map[string]time.Time)
167
+ }
168
+
169
+ now := time.Now()
170
+ if last, ok := c.warnLast[key]; ok && now.Sub(last) < warnThrottleInterval {
171
+ return
172
+ }
173
+
174
+ c.protocol.Warningf(format, args...)
175
+ c.warnLast[key] = now
176
+}
177
+
178
+// clearWarn removes a previously logged warning key so it can fire again
179
+func (c *Client) clearWarn(key string) {
180
+ c.warnMu.Lock()
181
+ defer c.warnMu.Unlock()
182
+
183
+ if c.warnLast == nil {
184
+ return
185
+ }
186
+ delete(c.warnLast, key)
187
+}
188
+
189
+// clearWarnPrefix clears all warning keys that share a common prefix
190
+func (c *Client) clearWarnPrefix(prefix string) {
191
+ c.warnMu.Lock()
192
+ defer c.warnMu.Unlock()
193
+
194
+ if c.warnLast == nil {
195
+ return
196
+ }
197
+
198
+ for key := range c.warnLast {
199
+ if strings.HasPrefix(key, prefix) {
200
+ delete(c.warnLast, key)
201
+ }
202
+ }
203
+}
src/go/plugin/ibm.d/protocols/pcf/pcf_ibm_transport.go
+7
-1
@@ -124,7 +124,13 @@ func (c *Client) getPCFReply(correlId []byte) ([]*ibmmq.PCFParameter, error) {
124
params, err := c.parsePCFResponseInternal(buffer[:datalen])
125
if err != nil {
126
// In multi-message responses, log the error but continue reading
127
- c.protocol.Warningf("Error in multi-message response: %v", err)
127
+ if pcfErr, ok := err.(*PCFError); ok {
128
+ key := fmt.Sprintf("pcf_command_%d_reason_%d", cfh.Command, pcfErr.Code)
129
+ c.warnOnce(key, "PCF command %s(%d) failed: %s", mqcmdToString(cfh.Command), cfh.Command, pcfErr.Message)
130
+ } else {
131
+ key := fmt.Sprintf("pcf_command_%d_parse", cfh.Command)
132
+ c.warnOnce(key, "Error in multi-message response for %s(%d): %v", mqcmdToString(cfh.Command), cfh.Command, err)
133
+ }
134
135
// Check if this is the last message even with error
136
if cfh.Control == ibmmq.MQCFC_LAST {
src/go/plugin/ibm.d/protocols/pcf/statistics_queue.go
+1
-1
@@ -32,7 +32,7 @@ func (c *Client) GetStatisticsQueue() (*StatisticsCollectionResult, error) {
32
return nil, fmt.Errorf("not connected")
33
}
34
35
- c.protocol.Warningf("Statistics queue collection is not yet implemented in the IBM MQ Go library migration")
35
+ c.warnOnce("statistics_queue_unimplemented", "Statistics queue collection is not yet implemented in the IBM MQ Go library migration")
36
c.protocol.Debugf("GetStatisticsQueue returning empty results - feature needs implementation")
37
38
// Return empty successful result for now
src/go/plugin/ibm.d/protocols/pcf/stub.go
+12
@@ -81,6 +81,14 @@ type Client struct {
81
config Config
82
}
83
84
+type ResourceStatus int
85
+
86
+const (
87
+ ResourceStatusDisabled ResourceStatus = 0
88
+ ResourceStatusEnabled ResourceStatus = 1
89
+ ResourceStatusFailed ResourceStatus = 2
90
+)
91
+
92
func NewClient(config Config, state *framework.CollectorState) *Client {
93
return &Client{
94
config: config,
@@ -165,6 +173,10 @@ func (c *Client) IsResourceMonitoringSupported() bool {
173
return false
174
}
175
176
+func (c *Client) GetResourceStatus() ResourceStatus {
177
+ return ResourceStatusDisabled
178
+}
179
+
180
func (c *Client) EnableResourceMonitoring() error {
181
return errors.New("PCF protocol requires CGO support")
182
}
src/go/plugin/ibm.d/samples.d/pub400.org/as400/pub400/meta/config.json
+2
-2
@@ -31,9 +31,9 @@
31
"max_job_queues": 200,
32
"max_message_queues": 200,
33
"max_output_queues": 200,
34
- "max_active_jobs": 200,
34
+ "active_jobs": [],
35
"collect_disks_matching": "",
36
"collect_subsystems_matching": "",
37
"collect_job_queues_matching": ""
38
}
39
-}
\ No newline at end of file
39
+}
src/web/mcp/TODO-LIST.md
deleted
-426
@@ -1,426 +0,0 @@
1
-# MCP Implementation Plan
2
-
3
-## Overview
4
-
5
-This document outlines the complete plan for implementing the Model Context Protocol (MCP) system with clean separation between transport and business logic, supporting both HTTP and WebSocket transports.
6
-
7
-## Architecture Overview
8
-
9
-### Core Design Principles
10
-1. **Transport-agnostic MCP core** - Business logic separated from transport protocols
11
-2. **Registry-based tool system** - Single source of truth for all MCP tools
12
-3. **Netdata-compatible authorization** - Reuse existing HTTP_ACL and HTTP_ACCESS system
13
-4. **Multi-buffer responses** - Support ordered responses using libnetdata double-linked lists
14
-5. **Clean job-based execution** - Each request becomes a structured job
15
-
16
-## Phase 1 – Transport Decoupling (Current Focus)
17
-
18
-### Goals
19
-- Keep request parsing inside each adapter while handing a parsed `json_object *` to the core. [done]
20
-- Transform `MCP_CLIENT` into a session container with a per-request array of `BUFFER *` chunks instead of a single result buffer and JSON-RPC metadata. [done]
21
-- Provide helper APIs (e.g. `mcp_response_reset`, `mcp_response_add_json`, `mcp_response_add_text`, `mcp_response_finalize`) so namespace handlers build transport-neutral responses without touching envelopes. [done]
22
-- Ensure adapters own correlation data: WebSocket keeps JSON-RPC ids, future transports can pick their own tokens. [done]
23
-- Preserve existing namespace function signatures by passing the same `MCP_CLIENT *`, params object, and `MCP_REQUEST_ID` while changing only the response building helpers they call. [done]
24
-
25
-### Deliverables
26
-- Response buffer management implementation with request-level limits and ownership handled by `MCP_CLIENT`. [done]
27
-- Updated namespace implementations (initialize, ping, tools, resources, prompts, logging, completion, etc.) to use the new helper APIs. [done]
28
-- WebSocket adapter refactor that wraps/unwraps JSON-RPC entirely in adapter code, including batching and notifications. [done]
29
-- Documentation updates describing the new lifecycle and expectations for adapters. [done]
30
-
31
-### Open Questions / Checks
32
-- Confirm memory caps for accumulated response buffers and expose configuration knobs if required. [done]
33
-- Validate streaming semantics: adapters must never split a single `BUFFER`, but may send multiple buffers sequentially. [done]
34
-- Identify any shared utilities (UUID helpers, auth context) that should remain in core versus adapter. [done]
35
-
36
-Status:
37
-- [x] Response buffer helpers implemented in mcp.c (prepare, add_json/text, finalize via buffer_json_finalize in handlers)
38
-- [x] Namespaces updated to use helpers (initialize, ping, tools, resources, prompts, logging, completion)
39
-- [x] WebSocket adapter wraps JSON-RPC (batching, notifications) and converts MCP response chunks to JSON-RPC payloads
40
-- [x] Error handling unified via mcp_error_result and mcpc->error buffer
41
-
42
-## 1. Core MCP Architecture Refactoring
43
-
44
-### A. Job-Based Request Processing
45
-
46
-#### MCP_REQ_JOB Structure (Transport-Agnostic)
47
-```c
48
-typedef struct mcp_req_job {
49
- // Job identification
50
- nd_uuid_t job_id;
51
- char job_id_str[UUID_STR_LEN];
52
-
53
- // Request data (pure, no transport context)
54
- struct json_object *params; // Parsed JSON parameters
55
- const char *tool_name; // Tool to execute
56
- USER_AUTH *auth; // Authentication context
57
-
58
- // Response data (ordered list using libnetdata double-linked lists)
59
- MCP_RESPONSE_BUFFER *response_buffers; // Head of double-linked list
60
-
61
- // Status and metadata
62
- int status_code; // Overall job status
63
- const char *error_message; // Error description if failed
64
- bool completed; // Job completion status
65
- usec_t created_usec; // Creation timestamp
66
- usec_t completed_usec; // Completion timestamp
67
-
68
- // Pagination support
69
- const char *next_cursor; // For paginated responses
70
-} MCP_REQ_JOB;
71
-```
72
-
73
-#### MCP_RESPONSE_BUFFER Structure (Using BUFFER's Built-in HTTP Metadata)
74
-```c
75
-typedef struct mcp_response_buffer {
76
- BUFFER *buffer; // Uses existing BUFFER with HTTP metadata built-in
77
- const char *response_type; // "text", "error", "data", etc.
78
-
79
- // Double-linked list support using libnetdata macros
80
- struct mcp_response_buffer *prev;
81
- struct mcp_response_buffer *next;
82
-} MCP_RESPONSE_BUFFER;
83
-```
84
-
85
-### B. Adapter-Specific Job Structures
86
-
87
-#### HTTP Adapter Job
88
-```c
89
-typedef struct mcp_http_adapter_job {
90
- MCP_REQ_JOB req; // Core MCP job (no MCP_CLIENT field!)
91
-
92
- // HTTP-specific data
93
- struct web_client *web_client; // HTTP client context
94
- const char *url_path; // Original URL path
95
- const char *query_string; // URL query parameters
96
-} MCP_HTTP_ADAPTER_JOB;
97
-```
98
-
99
-#### WebSocket/JSON-RPC Adapter Job
100
-```c
101
-typedef struct mcp_jsonrpc_adapter_job {
102
- MCP_REQ_JOB req; // Core MCP job
103
-
104
- // JSON-RPC/WebSocket specific data
105
- MCP_CLIENT *mcpc; // WebSocket client context (adapter manages this)
106
- uint64_t jsonrpc_id; // JSON-RPC request ID
107
- const char *jsonrpc_method; // Original method name
108
-} MCP_JSONRPC_ADAPTER_JOB;
109
-```
110
-
111
-**Status**:
112
-- [ ] Implement MCP_REQ_JOB structure
113
-- [ ] Implement MCP_RESPONSE_BUFFER with libnetdata double-linked list support
114
-- [ ] Create adapter job structures
115
-- [ ] Implement job lifecycle management functions
116
-
117
-## 2. Registry-Based Tool System
118
-
119
-### A. Tool Registry Structure (Following web_api_command Pattern)
120
-
121
-```c
122
-typedef struct mcp_tool_registry_entry {
123
- // Tool identification (similar to web_api_command)
124
- const char *name; // Tool name (e.g., "execute_function")
125
- uint32_t hash; // Hash for fast lookup (like api_commands_v3)
126
-
127
- // Authorization (following Netdata pattern exactly)
128
- HTTP_ACL acl; // ACL requirements (e.g., HTTP_ACL_FUNCTIONS)
129
- HTTP_ACCESS access; // Access level requirements
130
-
131
- // Execution
132
- int (*execute)(MCP_REQ_JOB *job); // Function pointer (similar to callback)
133
-
134
- // MCP-specific metadata
135
- MCP_NAMESPACE namespace; // Which MCP namespace
136
- const char *title; // Human-readable title
137
- const char *description; // Tool description
138
- const char *input_schema_json; // JSON schema for parameters (static string)
139
-
140
- // Feature flags
141
- bool supports_pagination; // Whether tool supports cursor pagination
142
- bool supports_streaming; // Whether tool supports streaming responses (future)
143
-
144
- // Caching hints for adapters
145
- bool cacheable_schema; // Whether schema responses can be cached
146
- time_t schema_cache_duration; // How long to cache schema responses
147
-} MCP_TOOL_REGISTRY_ENTRY;
148
-```
149
-
150
-### B. Registry Implementation
151
-
152
-#### Global Static Registry (Following api_commands_v3 Pattern)
153
-```c
154
-MCP_TOOL_REGISTRY_ENTRY mcp_tools_registry[] = {
155
- // Function execution tools
156
- {
157
- .name = "execute_function",
158
- .hash = 0, // Will be calculated on init like api_commands_v3
159
- .acl = HTTP_ACL_FUNCTIONS, // Same as existing function APIs
160
- .access = HTTP_ACCESS_ANONYMOUS_DATA, // Same as api_v1_function
161
- .execute = mcp_tool_execute_function,
162
- .namespace = MCP_NAMESPACE_TOOLS,
163
- .title = "Execute Netdata Function",
164
- .description = "Execute live data collection functions on nodes",
165
- .input_schema_json = MCP_EXECUTE_FUNCTION_SCHEMA_JSON,
166
- .supports_pagination = true,
167
- .supports_streaming = false,
168
- .cacheable_schema = true,
169
- .schema_cache_duration = 3600,
170
- },
171
- // ... more tools
172
-
173
- // Terminator (like api_commands_v3)
174
- { .name = NULL }
175
-};
176
-```
177
-
178
-#### Registry Access Functions
179
-```c
180
-// Initialize registry (calculate hashes like web_client_api_request_v3)
181
-void mcp_tools_registry_init(void);
182
-
183
-// Tool lookup (similar to web_client_api_request_vX)
184
-const MCP_TOOL_REGISTRY_ENTRY *mcp_find_tool(const char *tool_name);
185
-
186
-// Get tools by namespace
187
-const MCP_TOOL_REGISTRY_ENTRY **mcp_get_tools_by_namespace(MCP_NAMESPACE namespace, size_t *count);
188
-```
189
-
190
-**Status**:
191
-- [ ] Define MCP_TOOL_REGISTRY_ENTRY structure
192
-- [ ] Implement static registry array with all current tools
193
-- [ ] Implement registry initialization and lookup functions
194
-- [ ] Add authorization checking using HTTP_ACL/HTTP_ACCESS
195
-
196
-## 3. Transport Adapters
197
-
198
-### A. HTTP Adapter (Integrated with Netdata Web Server)
199
-
200
-#### HTTP Routing Hooks
201
-```c
202
-// src/web/server/web_client.c
203
-else if (unlikely(hash == hash_mcp && strcmp(tok, "mcp") == 0)) {
204
- if (!http_can_access_dashboard(w))
205
- return web_client_permission_denied_acl(w);
206
- return mcp_http_handle_request(host, w);
207
-}
208
-else if (unlikely(hash == hash_sse && strcmp(tok, "sse") == 0)) {
209
- if (!http_can_access_dashboard(w))
210
- return web_client_permission_denied_acl(w);
211
- return mcp_sse_handle_request(host, w);
212
-}
213
-```
214
-
215
-`mcp_http_handle_request()` streams the accumulated MCP response as JSON (chunked when multiple buffers are present). `mcp_sse_handle_request()` produces Server-Sent Event frames and disables compression before returning.
216
-
217
-#### Authorization Integration
218
-```c
219
-static inline bool mcp_adapter_authorize(struct web_client *w, const MCP_TOOL_REGISTRY_ENTRY *tool) {
220
- if (!tool)
221
- return false;
222
- if (tool->acl != HTTP_ACL_NOCHECK && !(w->acl & tool->acl))
223
- return false;
224
- if (tool->access != HTTP_ACCESS_NONE && !web_client_can_access_with_auth(w, tool->access))
225
- return false;
226
- return true;
227
-}
228
-
229
-int mcp_http_handle_request(RRDHOST *host, struct web_client *w) {
230
- struct json_object *request = mcp_http_parse_request_body(w);
231
- const char *method = mcp_http_request_method(request);
232
- const MCP_TOOL_REGISTRY_ENTRY *tool = mcp_find_tool(method);
233
- if (!mcp_adapter_authorize(w, tool))
234
- return web_client_permission_denied_acl(w);
235
-
236
- MCP_CLIENT *mcpc = mcp_create_client(MCP_TRANSPORT_HTTP, w);
237
- MCP_RETURN_CODE rc = mcp_dispatch_method(mcpc, method, mcp_http_request_params(request), 1);
238
- return mcp_http_send_response(w, mcpc, rc);
239
-}
240
-```
241
-
242
-**Status**:
243
-- [ ] Add `/mcp` and `/sse` branches in `web_client_process_url()`
244
-- [ ] Implement HTTP JSON parsing helpers (`mcp_http_parse_request_body`, etc.)
245
-- [ ] Implement chunked JSON serializer (`mcp_http_send_response`)
246
-- [ ] Implement SSE serializer (`mcp_sse_send_response`)
247
-- [ ] Share authorization helpers between HTTP and SSE adapters
248
-
249
-### B. WebSocket/JSON-RPC Adapter (Manages MCP_CLIENT)
250
-
251
-#### Adapter Responsibilities
252
-- **MCP_CLIENT management** (WebSocket connections, stdio pipes)
253
-- **JSON-RPC protocol** wrapping/unwrapping
254
-- **Ping/pong handling** - MCP core not involved
255
-- **Client notifications**
256
-- **Connection lifecycle**
257
-
258
-#### JSON-RPC Implementation
259
-```c
260
-void mcp_jsonrpc_handle_tools_call(MCP_CLIENT *mcpc, struct json_object *request) {
261
- struct json_object *params = json_object_object_get(request, "params");
262
- const char *tool_name = json_object_get_string(json_object_object_get(params, "name"));
263
-
264
- // Look up in registry (same as HTTP)
265
- const MCP_TOOL_REGISTRY_ENTRY *tool = mcp_find_tool(tool_name);
266
- if (!tool) {
267
- mcp_jsonrpc_send_error(mcpc, extract_id_from_request(request), -32601, "Tool not found");
268
- return;
269
- }
270
-
271
- // Check ACL and access (same authorization as HTTP!)
272
- // ... authorization code
273
-
274
- // Execute (same execution path as HTTP)
275
- // ... execution code
276
-}
277
-
278
-// Adapter handles ping separately
279
-void mcp_jsonrpc_handle_ping(MCP_CLIENT *mcpc, struct json_object *request) {
280
- // Adapter handles ping/pong - MCP core not involved
281
- uint64_t id = extract_id_from_request(request);
282
- mcp_jsonrpc_send_pong_response(mcpc, id);
283
-}
284
-```
285
-
286
-**Status**:
287
-- [ ] Extract JSON-RPC code from current MCP implementation
288
-- [ ] Move MCP_CLIENT management to WebSocket adapter
289
-- [ ] Implement JSON-RPC request/response conversion
290
-- [ ] Handle ping/pong and notifications in adapter
291
-- [ ] Ensure same authorization as HTTP adapter
292
-
293
-## 4. Core MCP Interface (Transport-Agnostic)
294
-
295
-### A. Core Execution Function
296
-```c
297
-// Main MCP execution function - completely transport agnostic
298
-int mcp_execute_tool(MCP_REQ_JOB *job);
299
-
300
-// Response buffer management using libnetdata double-linked list macros
301
-MCP_RESPONSE_BUFFER *mcp_job_add_response_buffer(MCP_REQ_JOB *job, BUFFER *buffer, const char *response_type);
302
-void mcp_job_prepend_response_buffer(MCP_REQ_JOB *job, MCP_RESPONSE_BUFFER *item);
303
-void mcp_job_append_response_buffer(MCP_REQ_JOB *job, MCP_RESPONSE_BUFFER *item);
304
-
305
-// Helper functions for common response types
306
-MCP_RESPONSE_BUFFER *mcp_job_add_text_response(MCP_REQ_JOB *job, const char *text, int http_status, HTTP_CONTENT_TYPE content_type);
307
-MCP_RESPONSE_BUFFER *mcp_job_add_json_response(MCP_REQ_JOB *job, BUFFER *json_buffer);
308
-MCP_RESPONSE_BUFFER *mcp_job_add_error_response(MCP_REQ_JOB *job, const char *error_msg, int http_status);
309
-```
310
-
311
-### B. Tool Implementation Interface
312
-```c
313
-// Tool function signature - no client context needed
314
-typedef int (*mcp_tool_func_t)(MCP_REQ_JOB *job);
315
-
316
-// Example simplified tool implementation
317
-int mcp_tool_execute_function(MCP_REQ_JOB *job) {
318
- // Pure business logic - no transport awareness
319
- const char *node = json_object_get_string(json_object_object_get(job->params, "node"));
320
-
321
- if (!node) {
322
- mcp_job_add_error_response(job, "Missing required parameter: node", 400);
323
- return -1;
324
- }
325
-
326
- // Execute function using existing BUFFER with built-in HTTP metadata
327
- BUFFER *result = buffer_create(0, NULL);
328
- buffer_set_content_type(result, CT_APPLICATION_JSON);
329
- buffer_cacheable(result);
330
- result->expires = now_monotonic_usec() + 300 * USEC_PER_SEC;
331
-
332
- int status = execute_netdata_function(node, function, result);
333
-
334
- if (status == 0) {
335
- mcp_job_add_response_buffer(job, result, "text");
336
- } else {
337
- mcp_job_add_error_response(job, "Function execution failed", 500);
338
- buffer_free(result);
339
- }
340
-
341
- return status;
342
-}
343
-```
344
-
345
-**Status**:
346
-- [ ] Implement core execution function
347
-- [ ] Implement response buffer management with double-linked lists
348
-- [ ] Update all existing tools to use new job interface
349
-- [ ] Remove transport dependencies from tool implementations
350
-
351
-## 5. Directory Structure
352
-
353
-```
354
-src/web/mcp/
355
-├── core/
356
-│ ├── mcp-registry.c/h # Global tool registry
357
-│ ├── mcp-job.c/h # Job management
358
-│ ├── mcp-response.c/h # Response management
359
-│ ├── mcp-tools-*.c/h # Individual tool implementations
360
-│ └── mcp-core.c/h # Core execution (tool lookup + execute)
361
-├── adapters/
362
-│ ├── websocket/
363
-│ │ ├── mcp-jsonrpc-adapter.c/h # tools/list, tools/call implementation
364
-│ │ └── mcp-client.c/h # MCP_CLIENT management
365
-│ └── http/
366
-│ ├── mcp-http-adapter.c/h # /mcp chunked JSON responses
367
-│ └── mcp-sse-adapter.c/h # /sse server-sent events
368
-├── schemas/
369
-│ ├── execute_function.json # Static schema definitions
370
-│ ├── query_metrics.json
371
-│ └── list_metrics.json
372
-└── mcp.h # Public interfaces
373
-```
374
-
375
-**Status**:
376
-- [ ] Create directory structure
377
-- [ ] Move existing code to appropriate locations
378
-- [ ] Update build system for new structure
379
-
380
-## 6. Logs Tools Implementation (Advanced Features)
381
-
382
-### Progressive Logs Discovery Workflow
383
-
384
-#### Specialized Logs Tools
385
-1. **`list_logs_sources`** - Discovery: "What logs are available?"
386
-2. **`list_logs_fields`** - Schema discovery: "What fields exist in this source?"
387
-3. **`list_logs_fields_values`** - Facets analysis: "What values can these fields have and how common are they?"
388
-4. **`query_logs`** - Actual log retrieval with all the power of the current system
389
-
390
-#### Implementation Strategy
391
-- **Tools 1-2**: Use the `info=true` calls to logs functions to get metadata
392
-- **Tool 3**: Call logs functions with specific facets parameters, return just the facets counts
393
-- **Tool 4**: Current full query capability
394
-
395
-**Status**:
396
-- [x] Design the 4-tool architecture
397
-- [ ] Implement `list_logs_sources`
398
-- [ ] Implement `list_logs_fields`
399
-- [ ] Implement `list_logs_fields_values` (the key facets tool)
400
-- [ ] Implement `query_logs`
401
-- [ ] Update `execute_function` to exclude logs functions
402
-- [ ] Test the complete workflow
403
-
404
-## 7. Implementation Phases
405
-
406
-### Phase 1: Advanced Features (Priority: Medium)
407
-1. Specialized logs tools workflow.
408
-2. Enhanced error handling, status reporting, and potential job queue abstractions once multiple transports are stable.
409
-3. **Performance optimizations**
410
-4. **Comprehensive testing**
411
-
412
-### Phase 2: Future Enhancements (Priority: Low)
413
-1. **Streaming support for long-running operations**
414
-2. **Additional MCP namespaces (resources, prompts)**
415
-3. **Advanced caching strategies**
416
-4. **Monitoring and metrics**
417
-
418
-## Benefits of This Architecture
419
-
420
-1. ✅ **Code Reuse**: All existing MCP tools work unchanged after refactoring
421
-2. ✅ **Consistency**: Same functionality via HTTP and WebSocket
422
-3. ✅ **Integration**: Native part of Netdata web server
423
-4. ✅ **Authorization**: Reuses existing HTTP_ACL/HTTP_ACCESS system
424
-5. ✅ **Maintenance**: Single codebase for all MCP logic
425
-6. ✅ **Performance**: No extra proxy/adapter process
426
-7. ✅ **Scalability**: Clean separation enables easy addition of new tools and transports