Fix MDX build errors in integration docs and regenerate (#21749)
1. Fix gen_docs_integrations.py: clean_and_write() used broad string replacement ('" %}' -> '</summary>') that mangled {% relatedResource %} tags, causing MDX "unexpected lazy line" errors in 30+ docs. Replaced with targeted regex that only converts {% details %} tags. 2. Fix overview/collector.md template: used entry.meta.name (undefined for collectors) instead of entry.meta.monitored_instance.name, leaving related_resources sentences without a subject. 3. Fix DCGM metadata.yaml: update category from removed data-collection.hardware-devices-and-sensors to data-collection.hardware-and-iot. 4. Fix Python 3.13 DeprecationWarning: pass maxsplit as keyword argument in re.split() call. 5. Regenerate all integration docs.
Costa Tsaousis committed
Feb 12, 2026 at 12:38 UTC
3832c6b01ccd39d2b913a0bed0748866015a2880
37 files changed
+703
-198
integrations/gen_docs_integrations.py
+3
-4
@@ -37,9 +37,8 @@ def clean_and_write(md: str, path: Path):
37
"""
38
Convert custom {% details %} markers to HTML <details> and write file.
39
"""
40
- md = md.replace('{% details open=true summary="', "<details open><summary>")
41
- md = md.replace('{% details summary="', "<details><summary>")
42
- md = md.replace('" %}', "</summary>\n")
40
+ md = re.sub(r'\{% details open=true summary="(.*?)" %\}', r'<details open><summary>\1</summary>\n', md)
41
+ md = re.sub(r'\{% details summary="(.*?)" %\}', r'<details><summary>\1</summary>\n', md)
42
md = md.replace("{% /details %}", "</details>\n")
43
path.write_text(md, encoding="utf-8")
44
@@ -131,7 +130,7 @@ def create_overview(integration, filename: str, overview_key_name: str = "overvi
130
if not overview_key_name:
131
return f"# {integration['meta']['name']}\n\n<img src=\"https://netdata.cloud/img/{filename}\" width=\"150\"/>\n"
132
134
- split = re.split(r"(#.*\n)", integration[overview_key_name], 1)
133
+ split = re.split(r"(#.*\n)", integration[overview_key_name], maxsplit=1)
134
first_overview_part = split[1]
135
rest_overview_part = split[2]
136
integrations/templates/overview/collector.md
+1
-1
@@ -36,7 +36,7 @@ This collector only supports collecting metrics from a single instance of this i
36
[% endif %]
37
38
[% if related %]
39
-[[ entry.meta.name ]] can be monitored further using the following other integrations:
39
+[[ entry.meta.monitored_instance.name ]] can be monitored further using the following other integrations:
40
41
[% for res in related %]
42
- {% relatedResource id="[[ res.id ]]" %}[[ res.name ]]{% /relatedResource %}
src/collectors/diskspace.plugin/integrations/disk_space.md
+2
-3
@@ -31,10 +31,9 @@ This collector is supported on all platforms.
31
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
32
33
34
- can be monitored further using the following other integrations:
34
+Disk space can be monitored further using the following other integrations:
35
36
-- {% relatedResource id="ebpf.plugin-disk-eBPF_Disk</summary>
37
-eBPF Disk{% /relatedResource %}
36
+- {% relatedResource id="ebpf.plugin-disk-eBPF_Disk" %}eBPF Disk{% /relatedResource %}
37
38
### Default Behavior
39
src/collectors/ebpf.plugin/integrations/ebpf_cachestat.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF Cachestat can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_dcstat.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF DCstat can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_filedescriptor.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netdata sets necessary permissions during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF Filedescriptor can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_oomkill.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF OOMkill can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_processes.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF Processes can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_shm.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF SHM can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_socket.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF Socket can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_swap.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF SWAP can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/ebpf.plugin/integrations/ebpf_vfs.md
+3
-5
@@ -34,12 +34,10 @@ This collector supports collecting metrics from multiple instances of this integ
34
35
The plugin needs setuid because it loads data inside kernel. Netada sets necessary permission during installation time.
36
37
- can be monitored further using the following other integrations:
37
+eBPF VFS can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/collectors/guides/proxmox/integrations/proxmox_ve_monitoring.md
+14
-26
@@ -46,32 +46,20 @@ This collector is only supported on the following platforms:
46
This collector only supports collecting metrics from a single instance of this integration.
47
48
49
- can be monitored further using the following other integrations:
50
-
51
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Proxmox_VMs_and_Containers</summary>
52
-Proxmox VMs and Containers{% /relatedResource %}
53
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Systemd_Services</summary>
54
-Systemd Services{% /relatedResource %}
55
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
56
-Applications{% /relatedResource %}
57
-- {% relatedResource id="go.d.plugin-zfspool-ZFS_Pools</summary>
58
-ZFS Pools{% /relatedResource %}
59
-- {% relatedResource id="go.d.plugin-ceph-Ceph</summary>
60
-Ceph{% /relatedResource %}
61
-- {% relatedResource id="go.d.plugin-smartctl-S.M.A.R.T.</summary>
62
-S.M.A.R.T.{% /relatedResource %}
63
-- {% relatedResource id="go.d.plugin-sensors-Linux_Sensors</summary>
64
-Linux Sensors{% /relatedResource %}
65
-- {% relatedResource id="proc.plugin-/proc/net/dev-Network_interfaces</summary>
66
-Network interfaces{% /relatedResource %}
67
-- {% relatedResource id="proc.plugin-/proc/diskstats-Disk_Statistics</summary>
68
-Disk Statistics{% /relatedResource %}
69
-- {% relatedResource id="proc.plugin-/proc/stat-System_statistics</summary>
70
-System statistics{% /relatedResource %}
71
-- {% relatedResource id="proc.plugin-/proc/meminfo-Memory_Usage</summary>
72
-Memory Usage{% /relatedResource %}
73
-- {% relatedResource id="proc.plugin-/proc/spl/kstat/zfs/arcstats-ZFS_Adaptive_Replacement_Cache</summary>
74
-ZFS Adaptive Replacement Cache{% /relatedResource %}
49
+Proxmox VE Monitoring can be monitored further using the following other integrations:
50
+
51
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Proxmox_VMs_and_Containers" %}Proxmox VMs and Containers{% /relatedResource %}
52
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Systemd_Services" %}Systemd Services{% /relatedResource %}
53
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
54
+- {% relatedResource id="go.d.plugin-zfspool-ZFS_Pools" %}ZFS Pools{% /relatedResource %}
55
+- {% relatedResource id="go.d.plugin-ceph-Ceph" %}Ceph{% /relatedResource %}
56
+- {% relatedResource id="go.d.plugin-smartctl-S.M.A.R.T." %}S.M.A.R.T.{% /relatedResource %}
57
+- {% relatedResource id="go.d.plugin-sensors-Linux_Sensors" %}Linux Sensors{% /relatedResource %}
58
+- {% relatedResource id="proc.plugin-/proc/net/dev-Network_interfaces" %}Network interfaces{% /relatedResource %}
59
+- {% relatedResource id="proc.plugin-/proc/diskstats-Disk_Statistics" %}Disk Statistics{% /relatedResource %}
60
+- {% relatedResource id="proc.plugin-/proc/stat-System_statistics" %}System statistics{% /relatedResource %}
61
+- {% relatedResource id="proc.plugin-/proc/meminfo-Memory_Usage" %}Memory Usage{% /relatedResource %}
62
+- {% relatedResource id="proc.plugin-/proc/spl/kstat/zfs/arcstats-ZFS_Adaptive_Replacement_Cache" %}ZFS Adaptive Replacement Cache{% /relatedResource %}
63
64
### Default Behavior
65
src/go/plugin/go.d/collector/activemq/integrations/activemq.md
+3
-5
@@ -31,12 +31,10 @@ This collector is supported on all platforms.
31
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
32
33
34
- can be monitored further using the following other integrations:
34
+ActiveMQ can be monitored further using the following other integrations:
35
36
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
37
-HTTP Endpoints{% /relatedResource %}
38
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
39
-Applications{% /relatedResource %}
36
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
37
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
38
39
### Default Behavior
40
src/go/plugin/go.d/collector/apache/integrations/apache.md
+5
-8
@@ -34,14 +34,11 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
38
-
39
-- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files</summary>
40
-Web server log files{% /relatedResource %}
41
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
42
-HTTP Endpoints{% /relatedResource %}
43
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
44
-Applications{% /relatedResource %}
37
+Apache can be monitored further using the following other integrations:
38
+
39
+- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files" %}Web server log files{% /relatedResource %}
40
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
41
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
42
43
### Default Behavior
44
src/go/plugin/go.d/collector/apache/integrations/httpd.md
+5
-8
@@ -34,14 +34,11 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
38
-
39
-- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files</summary>
40
-Web server log files{% /relatedResource %}
41
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
42
-HTTP Endpoints{% /relatedResource %}
43
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
44
-Applications{% /relatedResource %}
37
+HTTPD can be monitored further using the following other integrations:
38
+
39
+- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files" %}Web server log files{% /relatedResource %}
40
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
41
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
42
43
### Default Behavior
44
src/go/plugin/go.d/collector/dcgm/integrations/nvidia_dcgm_exporter.md
+582
@@ -1,3 +1,585 @@
1
+<!--startmeta
2
+custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/dcgm/README.md"
3
+meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/plugin/go.d/collector/dcgm/metadata.yaml"
4
+sidebar_label: "NVIDIA DCGM Exporter"
5
+learn_status: "Published"
6
+learn_rel_path: "Collecting Metrics/Hardware and IoT"
7
+most_popular: False
8
+keywords: ['nvidia', 'gpu', 'dcgm', 'dcgm-exporter']
9
+message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE"
10
+endmeta-->
11
12
+# NVIDIA DCGM Exporter
13
+
14
+
15
+<img src="https://netdata.cloud/img/nvidia.svg" width="150"/>
16
+
17
+
18
+Plugin: go.d.plugin
19
+Module: dcgm
20
21
<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />
22
+
23
+## Overview
24
+
25
+This collector gathers NVIDIA GPU telemetry from a `dcgm-exporter` endpoint.
26
+It supports all numeric fields exposed by the exporter and maps them into Netdata-native contexts.
27
+
28
+
29
+It collects metrics by periodically scraping the exporter Prometheus endpoint over HTTP.
30
+
31
+
32
+This collector is supported on all platforms.
33
+
34
+This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
+
36
+
37
+### Default Behavior
38
+
39
+#### Auto-Detection
40
+
41
+This integration does not support auto-detection in v1.
42
+
43
+#### Limits
44
+
45
+The collector applies global and per-metric time series limits to prevent excessive cardinality.
46
+
47
+
48
+#### Performance Impact
49
+
50
+The impact depends on dcgm-exporter field selection and resulting series cardinality.
51
+
52
+## Metrics
53
+
54
+Metrics grouped by *scope*.
55
+
56
+The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.
57
+
58
+Metrics are grouped into static Netdata contexts. Contexts are created only when matching DCGM fields are present in the exporter output.
59
+
60
+
61
+### Per gpu
62
+
63
+These metrics refer to GPU device instances.
64
+
65
+Labels:
66
+
67
+| Label | Description |
68
+|:-----------|:----------------|
69
+| gpu | gpu label from exporter metrics. |
70
+| uuid | uuid label from exporter metrics. |
71
+
72
+Metrics:
73
+
74
+| Metric | Dimensions | Unit |
75
+|:------|:----------|:----|
76
+| dcgm.gpu.capability.support | cc_mode, cuda_compute_capability, gpm_support, mig_attributes, mig_ci_info, mig_gi_info, mig_max_slices, supported_clocks, supported_type_info | state |
77
+| dcgm.gpu.clock.frequency | app_mem_clock, app_sm_clock, max_mem_clock, max_sm_clock, max_video_clock, memory, sm, video_clock | MHz |
78
+| dcgm.gpu.compute.activity | dram, fp16, fp32, fp64, graphics_engine_active, integer, sm_active, sm_occupancy, tensor | % |
79
+| dcgm.gpu.compute.tensor.activity | tensor_dfma, tensor_hmma, tensor_imma | % |
80
+| dcgm.gpu.compute.media.activity | nvdec0_active, nvdec1_active, nvdec2_active, nvdec3_active, nvdec4_active, nvdec5_active, nvdec6_active, nvdec7_active, nvjpg0_active, nvjpg1_active, nvjpg2_active, nvjpg3_active, nvjpg4_active, nvjpg5_active, nvjpg6_active, nvjpg7_active, nvofa0_active, nvofa1_active | % |
81
+| dcgm.gpu.compute.cache.activity | hostmem_cache_hit, hostmem_cache_miss, peermem_cache_hit, peermem_cache_miss | events/s |
82
+| dcgm.gpu.compute.utilization | decoder, encoder, gpu, memory_copy | % |
83
+| dcgm.gpu.cpu.power | module_power_util_current, sysio_power_util_current | Watts |
84
+| dcgm.gpu.cpu.info | cpu_model, cpu_vendor | value |
85
+| dcgm.gpu.diagnostics.results | diag_diagnostic_result, diag_eud_result, diag_memory_bandwidth_result, diag_memory_result, diag_memtest_result, diag_nccl_tests_result, diag_nvbandwidth_result, diag_pulse_test_result, diag_software_result, diag_targeted_power_result, diag_targeted_stress_result | state |
86
+| dcgm.gpu.diagnostics.status | diag_status | state |
87
+| dcgm.gpu.health.status | imex_daemon_status, imex_domain_status | state |
88
+| dcgm.gpu.interconnect.connectx.error_status | connectx_correctable_err_mask, connectx_correctable_err_status, connectx_uncorrectable_err_mask, connectx_uncorrectable_err_severity, connectx_uncorrectable_err_status | state |
89
+| dcgm.gpu.interconnect.connectx.errors | connectx_correctable_err_mask, connectx_correctable_err_status, connectx_uncorrectable_err_mask, connectx_uncorrectable_err_severity, connectx_uncorrectable_err_status | errors/s |
90
+| dcgm.gpu.interconnect.connectx.link | connectx_active_pcie_link_speed, connectx_expect_pcie_link_speed | value |
91
+| dcgm.gpu.interconnect.connectx.status | connectx_health | state |
92
+| dcgm.gpu.interconnect.error_rate | c2c_link_error_intr, c2c_link_error_replay, c2c_link_error_replay_b2b | errors/s |
93
+| dcgm.gpu.interconnect.fabric | fabric_clique_id, fabric_cluster_uuid, fabric_health_mask, fabric_manager_error_code, fabric_manager_status | state |
94
+| dcgm.gpu.interconnect.nvlink.error_rate | gpu_nvlink_errors | errors/s |
95
+| dcgm.gpu.interconnect.pcie.error_rate | pcie_count_correctable_errors, pcie_replay | errors/s |
96
+| dcgm.gpu.interconnect.pcie.link.generation | link_gen, max_link_gen | generation |
97
+| dcgm.gpu.interconnect.pcie.link.width | connectx_active_pcie_link_width, connectx_expect_pcie_link_width, link_width, max_link_width | lanes |
98
+| dcgm.gpu.interconnect.state | c2c_link, c2c_link_power_state, c2c_link_status | state |
99
+| dcgm.gpu.interconnect.pcie.state | diag_pcie_result | state |
100
+| dcgm.gpu.interconnect.throughput | c2c_max_bandwidth, c2c_rx_all_bytes, c2c_rx_data_bytes, c2c_tx_all_bytes, c2c_tx_data_bytes | B/s |
101
+| dcgm.gpu.interconnect.pcie.throughput | pcie_rx, pcie_rx_throughput, pcie_tx, pcie_tx_throughput | B/s |
102
+| dcgm.gpu.interconnect.nvlink.throughput | nvlink_rx, nvlink_tx | B/s |
103
+| dcgm.gpu.interconnect.total.throughput | pcie, nvlink | B/s |
104
+| dcgm.gpu.internal.boundary | first_connectx_field_id, first_vgpu_field_id, internal_fields_0_end, internal_fields_0_start, last_connectx_field_id, last_vgpu_field_id | state |
105
+| dcgm.gpu.inventory.identity | brand, count, cuda_visible_devices_str, minor_number, name, nvml_index, serial, uuid | value |
106
+| dcgm.gpu.inventory.platform | platform_chassis_serial_number, platform_chassis_slot_number, platform_host_id, platform_infiniband_guid, platform_module_id, platform_peer_type, platform_tray_index | value |
107
+| dcgm.gpu.inventory.software | inforom_config_check, inforom_config_valid, inforom_image_ver, oem_inforom_ver, power_inforom_ver, process_name, vbios_version | value |
108
+| dcgm.gpu.memory.bar1_usage | free, used | B |
109
+| dcgm.gpu.memory.bar1_capacity | total | B |
110
+| dcgm.gpu.memory.ecc_error_rate | ecc_current, ecc_dbe_agg, ecc_dbe_agg_cbu, ecc_dbe_agg_dev, ecc_dbe_agg_l1, ecc_dbe_agg_l2, ecc_dbe_agg_reg, ecc_dbe_agg_shm, ecc_dbe_agg_srm, ecc_dbe_agg_tex, ecc_dbe_vol, ecc_dbe_vol_cbu, ecc_dbe_vol_dev, ecc_dbe_vol_l1, ecc_dbe_vol_l2, ecc_dbe_vol_reg, ecc_dbe_vol_shm, ecc_dbe_vol_srm, ecc_dbe_vol_tex, ecc_pending, ecc_sbe_agg, ecc_sbe_agg_cbu, ecc_sbe_agg_dev, ecc_sbe_agg_l1, ecc_sbe_agg_l2, ecc_sbe_agg_reg, ecc_sbe_agg_shm, ecc_sbe_agg_srm, ecc_sbe_agg_tex, ecc_sbe_vol, ecc_sbe_vol_cbu, ecc_sbe_vol_dev, ecc_sbe_vol_l1, ecc_sbe_vol_l2, ecc_sbe_vol_reg, ecc_sbe_vol_shm, ecc_sbe_vol_srm, ecc_sbe_vol_tex | errors/s |
111
+| dcgm.gpu.memory.ecc_errors | ecc_current, ecc_dbe_agg_cbu, ecc_dbe_agg_dev, ecc_dbe_agg_l1, ecc_dbe_agg_l2, ecc_dbe_agg_reg, ecc_dbe_agg_shm, ecc_dbe_agg_srm, ecc_dbe_agg_tex, ecc_dbe_vol_cbu, ecc_dbe_vol_dev, ecc_dbe_vol_l1, ecc_dbe_vol_l2, ecc_dbe_vol_reg, ecc_dbe_vol_shm, ecc_dbe_vol_srm, ecc_dbe_vol_tex, ecc_inforom_ver, ecc_pending, ecc_sbe_agg_cbu, ecc_sbe_agg_dev, ecc_sbe_agg_l1, ecc_sbe_agg_l2, ecc_sbe_agg_reg, ecc_sbe_agg_shm, ecc_sbe_agg_srm, ecc_sbe_agg_tex, ecc_sbe_vol_cbu, ecc_sbe_vol_dev, ecc_sbe_vol_l1, ecc_sbe_vol_l2, ecc_sbe_vol_reg, ecc_sbe_vol_shm, ecc_sbe_vol_srm, ecc_sbe_vol_tex | errors |
112
+| dcgm.gpu.memory.page_retirements | retired_dbe, retired_pending, retired_sbe | pages/s |
113
+| dcgm.gpu.memory.usage | free, reserved, used | B |
114
+| dcgm.gpu.memory.capacity | total | B |
115
+| dcgm.gpu.memory.utilization | used_percent | % |
116
+| dcgm.gpu.power.energy | total | mJ/s |
117
+| dcgm.gpu.power.profiles | enforced_power_profile_mask, requested_power_profile_mask, valid_power_profile_mask | state |
118
+| dcgm.gpu.power.smoothing | pwr_smoothing_active_preset_profile, pwr_smoothing_admin_override_percent_tmp_floor, pwr_smoothing_admin_override_ramp_down_hyst_val, pwr_smoothing_admin_override_ramp_down_rate, pwr_smoothing_admin_override_ramp_up_rate, pwr_smoothing_applied_tmp_ceil, pwr_smoothing_applied_tmp_floor, pwr_smoothing_enabled, pwr_smoothing_hw_circuitry_percent_lifetime_remaining, pwr_smoothing_imm_ramp_down_enabled, pwr_smoothing_max_num_preset_profiles, pwr_smoothing_max_percent_tmp_floor_setting, pwr_smoothing_min_percent_tmp_floor_setting, pwr_smoothing_priv_lvl, pwr_smoothing_profile_percent_tmp_floor, pwr_smoothing_profile_ramp_down_hyst_val, pwr_smoothing_profile_ramp_down_rate, pwr_smoothing_profile_ramp_up_rate | value |
119
+| dcgm.gpu.power.usage | draw, enforced_limit, power_mgmt_limit, power_mgmt_limit_def, power_mgmt_limit_max, power_mgmt_limit_min, power_usage_instant | Watts |
120
+| dcgm.gpu.reliability.memory_health | banks_remap_rows_avail_high, banks_remap_rows_avail_low, banks_remap_rows_avail_max, banks_remap_rows_avail_none, banks_remap_rows_avail_partial, memory_unrepairable_flag, threshold_srm | state |
121
+| dcgm.gpu.reliability.recovery_action | get_gpu_recovery_action | state |
122
+| dcgm.gpu.reliability.row_remap_events | correctable_remapped_rows, uncorrectable_remapped_rows | rows/s |
123
+| dcgm.gpu.reliability.row_remap_status | row_remap_failure, row_remap_pending | state |
124
+| dcgm.gpu.reliability.xid | xid | code |
125
+| dcgm.gpu.state.configuration | autoboost, compute_mode, persistence_mode, sync_boost, sync_boost_violation | state |
126
+| dcgm.gpu.state.performance | pstate | state |
127
+| dcgm.gpu.state.virtualization | mig_mode, virtual_mode | state |
128
+| dcgm.gpu.thermal.fan_speed | fan_speed | % |
129
+| dcgm.gpu.thermal.temperature | connectx_device_temperature, gpu, gpu_max_op_temp, gpu_temp_limit, mem_max_op_temp, memory, shutdown_temp, slowdown_temp | Celsius |
130
+| dcgm.gpu.throttle.reasons | clocks_event_reasons | bitmask |
131
+| dcgm.gpu.throttle.violations | board_limit_violation, hw_power_brake_slowdown, hw_therm_slowdown, low_utilization_violation, power_violation, reliability_violation, sw_power_cap, sw_therm_slowdown, sync_boost, thermal_violation, total_app_clocks_violation, total_base_clocks_violation | milliseconds/s |
132
+| dcgm.gpu.topology.affinity | cpu_affinity_0, cpu_affinity_1, cpu_affinity_2, cpu_affinity_3, gpu_topology_affinity, gpu_topology_pci, mem_affinity_0, mem_affinity_1, mem_affinity_2, mem_affinity_3, pci_busid, pci_combined_id, pci_subsys_id | value |
133
+| dcgm.gpu.virtualization.vgpu.frame_rate | vgpu_frame_rate_limit | fps |
134
+| dcgm.gpu.virtualization.vgpu.instance | vgpu_instance_ids, vgpu_pci_id, vgpu_uuid | value |
135
+| dcgm.gpu.virtualization.vgpu.license | vgpu_instance_license_state, vgpu_license_status, vgpu_type_license | state |
136
+| dcgm.gpu.virtualization.vgpu.memory | vgpu_memory_usage | B |
137
+| dcgm.gpu.virtualization.vgpu.sessions | vgpu_enc_sessions_info, vgpu_enc_stats, vgpu_fbc_sessions_info, vgpu_fbc_stats | value |
138
+| dcgm.gpu.virtualization.vgpu.software | vgpu_driver_version | value |
139
+| dcgm.gpu.virtualization.vgpu.type | creatable_vgpu_type_ids, supported_vgpu_type_ids, vgpu_type, vgpu_type_class, vgpu_type_info, vgpu_type_name | value |
140
+| dcgm.gpu.virtualization.vgpu.utilization | vgpu_per_process_utilization | % |
141
+| dcgm.gpu.virtualization.vgpu.vm | vgpu_vm_gpu_instance_id, vgpu_vm_id, vgpu_vm_name | value |
142
+| dcgm.gpu.workload.sessions | accounting_data, enc_stats, fbc_sessions_info, fbc_stats | value |
143
+
144
+### Per mig
145
+
146
+These metrics refer to MIG instances.
147
+
148
+Labels:
149
+
150
+| Label | Description |
151
+|:-----------|:----------------|
152
+| gpu | gpu label from exporter metrics. |
153
+| gpu_i_id | gpu_i_id label from exporter metrics. |
154
+| gpu_i_profile | gpu_i_profile label from exporter metrics. |
155
+
156
+Metrics:
157
+
158
+| Metric | Dimensions | Unit |
159
+|:------|:----------|:----|
160
+| dcgm.mig.clock.frequency | app_mem_clock, app_sm_clock, max_mem_clock, max_sm_clock, max_video_clock, memory, sm, video_clock | MHz |
161
+| dcgm.mig.compute.activity | dram, fp16, fp32, fp64, graphics_engine_active, integer, sm_active, sm_occupancy, tensor | % |
162
+| dcgm.mig.compute.tensor.activity | tensor_dfma, tensor_hmma, tensor_imma | % |
163
+| dcgm.mig.compute.media.activity | nvdec0_active, nvdec1_active, nvdec2_active, nvdec3_active, nvdec4_active, nvdec5_active, nvdec6_active, nvdec7_active, nvjpg0_active, nvjpg1_active, nvjpg2_active, nvjpg3_active, nvjpg4_active, nvjpg5_active, nvjpg6_active, nvjpg7_active, nvofa0_active, nvofa1_active | % |
164
+| dcgm.mig.compute.cache.activity | hostmem_cache_hit, hostmem_cache_miss, peermem_cache_hit, peermem_cache_miss | events/s |
165
+| dcgm.mig.compute.utilization | decoder, encoder, gpu, memory_copy | % |
166
+| dcgm.mig.interconnect.nvlink.ber | nvlink_count_effective_ber, nvlink_count_effective_ber_float, nvlink_count_symbol_ber, nvlink_count_symbol_ber_float | ratio |
167
+| dcgm.mig.interconnect.nvlink.congestion | nvlink_ppcnt_ibpc_port_xmit_wait | events/s |
168
+| dcgm.mig.interconnect.error_rate | c2c_link_error_intr, c2c_link_error_replay, c2c_link_error_replay_b2b | errors/s |
169
+| dcgm.mig.interconnect.nvlink.error_rate | gpu_nvlink_errors, nvlink_count_effective_errors, nvlink_count_fec_history_0, nvlink_count_fec_history_1, nvlink_count_fec_history_10, nvlink_count_fec_history_11, nvlink_count_fec_history_12, nvlink_count_fec_history_13, nvlink_count_fec_history_14, nvlink_count_fec_history_15, nvlink_count_fec_history_2, nvlink_count_fec_history_3, nvlink_count_fec_history_4, nvlink_count_fec_history_5, nvlink_count_fec_history_6, nvlink_count_fec_history_7, nvlink_count_fec_history_8, nvlink_count_fec_history_9, nvlink_count_link_recovery_events, nvlink_count_link_recovery_failed_events, nvlink_count_link_recovery_successful_events, nvlink_count_local_link_integrity_errors, nvlink_count_rx_buffer_overrun_errors, nvlink_count_rx_errors, nvlink_count_rx_general_errors, nvlink_count_rx_malformed_packet_errors, nvlink_count_rx_remote_errors, nvlink_count_rx_symbol_errors, nvlink_count_tx_discards, nvlink_crc_data_error, nvlink_crc_data_error_count_l0, nvlink_crc_data_error_count_l1, nvlink_crc_data_error_count_l10, nvlink_crc_data_error_count_l11, nvlink_crc_data_error_count_l12, nvlink_crc_data_error_count_l13, nvlink_crc_data_error_count_l14, nvlink_crc_data_error_count_l15, nvlink_crc_data_error_count_l16, nvlink_crc_data_error_count_l17, nvlink_crc_data_error_count_l2, nvlink_crc_data_error_count_l3, nvlink_crc_data_error_count_l4, nvlink_crc_data_error_count_l5, nvlink_crc_data_error_count_l6, nvlink_crc_data_error_count_l7, nvlink_crc_data_error_count_l8, nvlink_crc_data_error_count_l9, nvlink_crc_flit_error, nvlink_crc_flit_error_count_l0, nvlink_crc_flit_error_count_l1, nvlink_crc_flit_error_count_l10, nvlink_crc_flit_error_count_l11, nvlink_crc_flit_error_count_l12, nvlink_crc_flit_error_count_l13, nvlink_crc_flit_error_count_l14, nvlink_crc_flit_error_count_l15, nvlink_crc_flit_error_count_l16, nvlink_crc_flit_error_count_l17, nvlink_crc_flit_error_count_l2, nvlink_crc_flit_error_count_l3, nvlink_crc_flit_error_count_l4, nvlink_crc_flit_error_count_l5, nvlink_crc_flit_error_count_l6, nvlink_crc_flit_error_count_l7, nvlink_crc_flit_error_count_l8, nvlink_crc_flit_error_count_l9, nvlink_error_dl_crc, nvlink_error_dl_recovery, nvlink_error_dl_replay, nvlink_ppcnt_physical_successful_recovery_events, nvlink_ppcnt_plr_rcv_uncorrectable_code, nvlink_ppcnt_recovery_time_since_last, nvlink_ppcnt_recovery_total_successful_events, nvlink_pprm_oper_recovery, nvlink_recovery_error, nvlink_recovery_error_count_l0, nvlink_recovery_error_count_l1, nvlink_recovery_error_count_l10, nvlink_recovery_error_count_l11, nvlink_recovery_error_count_l12, nvlink_recovery_error_count_l13, nvlink_recovery_error_count_l14, nvlink_recovery_error_count_l15, nvlink_recovery_error_count_l16, nvlink_recovery_error_count_l17, nvlink_recovery_error_count_l2, nvlink_recovery_error_count_l3, nvlink_recovery_error_count_l4, nvlink_recovery_error_count_l5, nvlink_recovery_error_count_l6, nvlink_recovery_error_count_l7, nvlink_recovery_error_count_l8, nvlink_recovery_error_count_l9, nvlink_replay_error, nvlink_replay_error_count_l0, nvlink_replay_error_count_l1, nvlink_replay_error_count_l10, nvlink_replay_error_count_l11, nvlink_replay_error_count_l12, nvlink_replay_error_count_l13, nvlink_replay_error_count_l14, nvlink_replay_error_count_l15, nvlink_replay_error_count_l16, nvlink_replay_error_count_l17, nvlink_replay_error_count_l2, nvlink_replay_error_count_l3, nvlink_replay_error_count_l4, nvlink_replay_error_count_l5, nvlink_replay_error_count_l6, nvlink_replay_error_count_l7, nvlink_replay_error_count_l8, nvlink_replay_error_count_l9 | errors/s |
170
+| dcgm.mig.interconnect.pcie.error_rate | pcie_count_correctable_errors, pcie_replay | errors/s |
171
+| dcgm.mig.interconnect.nvlink.errors | nvlink_ppcnt_plr_rcv_uncorrectable_code | errors |
172
+| dcgm.mig.interconnect.fabric | fabric_clique_id, fabric_cluster_uuid, fabric_health_mask, fabric_manager_error_code, fabric_manager_status | state |
173
+| dcgm.mig.interconnect.pcie.link.generation | link_gen, max_link_gen | generation |
174
+| dcgm.mig.interconnect.pcie.link.width | link_width, max_link_width | lanes |
175
+| dcgm.mig.interconnect.state | c2c_link, c2c_link_power_state, c2c_link_status | state |
176
+| dcgm.mig.interconnect.pcie.state | diag_pcie_result | state |
177
+| dcgm.mig.interconnect.nvlink.state | gpu_topology_nvlink, nvlink_get_state, nvlink_ppcnt_physical_link_down_counter, nvlink_ppcnt_plr_rcv_code_err, nvlink_ppcnt_plr_sync_events, nvlink_ppcnt_plr_xmit_retry_events, p2p_nvlink_status | state |
178
+| dcgm.mig.interconnect.throughput | c2c_max_bandwidth, c2c_rx_all_bytes, c2c_rx_data_bytes, c2c_tx_all_bytes, c2c_tx_data_bytes | B/s |
179
+| dcgm.mig.interconnect.nvlink.throughput | nvlink_bandwidth_l0, nvlink_bandwidth_l1, nvlink_bandwidth_l10, nvlink_bandwidth_l11, nvlink_bandwidth_l12, nvlink_bandwidth_l13, nvlink_bandwidth_l14, nvlink_bandwidth_l15, nvlink_bandwidth_l16, nvlink_bandwidth_l17, nvlink_bandwidth_l2, nvlink_bandwidth_l3, nvlink_bandwidth_l4, nvlink_bandwidth_l5, nvlink_bandwidth_l6, nvlink_bandwidth_l7, nvlink_bandwidth_l8, nvlink_bandwidth_l9, nvlink_count_rx, nvlink_count_tx, nvlink_l0_rx, nvlink_l0_tx, nvlink_l10_rx, nvlink_l10_tx, nvlink_l11_rx, nvlink_l11_tx, nvlink_l12_rx, nvlink_l12_tx, nvlink_l13_rx, nvlink_l13_tx, nvlink_l14_rx, nvlink_l14_tx, nvlink_l15_rx, nvlink_l15_tx, nvlink_l16_rx, nvlink_l16_tx, nvlink_l17_rx, nvlink_l17_tx, nvlink_l1_rx, nvlink_l1_tx, nvlink_l2_rx, nvlink_l2_tx, nvlink_l3_rx, nvlink_l3_tx, nvlink_l4_rx, nvlink_l4_tx, nvlink_l5_rx, nvlink_l5_tx, nvlink_l6_rx, nvlink_l6_tx, nvlink_l7_rx, nvlink_l7_tx, nvlink_l8_rx, nvlink_l8_tx, nvlink_l9_rx, nvlink_l9_tx, nvlink_rx_bandwidth, nvlink_rx_bandwidth_l0, nvlink_rx_bandwidth_l1, nvlink_rx_bandwidth_l10, nvlink_rx_bandwidth_l11, nvlink_rx_bandwidth_l12, nvlink_rx_bandwidth_l13, nvlink_rx_bandwidth_l14, nvlink_rx_bandwidth_l15, nvlink_rx_bandwidth_l16, nvlink_rx_bandwidth_l17, nvlink_rx_bandwidth_l2, nvlink_rx_bandwidth_l3, nvlink_rx_bandwidth_l4, nvlink_rx_bandwidth_l5, nvlink_rx_bandwidth_l6, nvlink_rx_bandwidth_l7, nvlink_rx_bandwidth_l8, nvlink_rx_bandwidth_l9, nvlink_rx, nvlink_tx_bandwidth, nvlink_tx_bandwidth_l0, nvlink_tx_bandwidth_l1, nvlink_tx_bandwidth_l10, nvlink_tx_bandwidth_l11, nvlink_tx_bandwidth_l12, nvlink_tx_bandwidth_l13, nvlink_tx_bandwidth_l14, nvlink_tx_bandwidth_l15, nvlink_tx_bandwidth_l16, nvlink_tx_bandwidth_l17, nvlink_tx_bandwidth_l2, nvlink_tx_bandwidth_l3, nvlink_tx_bandwidth_l4, nvlink_tx_bandwidth_l5, nvlink_tx_bandwidth_l6, nvlink_tx_bandwidth_l7, nvlink_tx_bandwidth_l8, nvlink_tx_bandwidth_l9, nvlink_tx | B/s |
180
+| dcgm.mig.interconnect.pcie.throughput | pcie_rx, pcie_rx_throughput, pcie_tx, pcie_tx_throughput | B/s |
181
+| dcgm.mig.interconnect.total.throughput | pcie, nvlink | B/s |
182
+| dcgm.mig.interconnect.nvlink.traffic | nvlink_count_rx_packets, nvlink_count_tx_packets, nvlink_ppcnt_plr_rcv_codes, nvlink_ppcnt_plr_xmit_codes, nvlink_ppcnt_plr_xmit_retry_codes | events/s |
183
+| dcgm.mig.memory.bar1_usage | free, used | B |
184
+| dcgm.mig.memory.bar1_capacity | total | B |
185
+| dcgm.mig.memory.ecc_error_rate | ecc_current, ecc_dbe_agg, ecc_dbe_agg_cbu, ecc_dbe_agg_dev, ecc_dbe_agg_l1, ecc_dbe_agg_l2, ecc_dbe_agg_reg, ecc_dbe_agg_shm, ecc_dbe_agg_srm, ecc_dbe_agg_tex, ecc_dbe_vol, ecc_dbe_vol_cbu, ecc_dbe_vol_dev, ecc_dbe_vol_l1, ecc_dbe_vol_l2, ecc_dbe_vol_reg, ecc_dbe_vol_shm, ecc_dbe_vol_srm, ecc_dbe_vol_tex, ecc_pending, ecc_sbe_agg, ecc_sbe_agg_cbu, ecc_sbe_agg_dev, ecc_sbe_agg_l1, ecc_sbe_agg_l2, ecc_sbe_agg_reg, ecc_sbe_agg_shm, ecc_sbe_agg_srm, ecc_sbe_agg_tex, ecc_sbe_vol, ecc_sbe_vol_cbu, ecc_sbe_vol_dev, ecc_sbe_vol_l1, ecc_sbe_vol_l2, ecc_sbe_vol_reg, ecc_sbe_vol_shm, ecc_sbe_vol_srm, ecc_sbe_vol_tex, nvlink_ecc_data_error | errors/s |
186
+| dcgm.mig.memory.ecc_errors | ecc_current, ecc_dbe_agg_cbu, ecc_dbe_agg_dev, ecc_dbe_agg_l1, ecc_dbe_agg_l2, ecc_dbe_agg_reg, ecc_dbe_agg_shm, ecc_dbe_agg_srm, ecc_dbe_agg_tex, ecc_dbe_vol_cbu, ecc_dbe_vol_dev, ecc_dbe_vol_l1, ecc_dbe_vol_l2, ecc_dbe_vol_reg, ecc_dbe_vol_shm, ecc_dbe_vol_srm, ecc_dbe_vol_tex, ecc_inforom_ver, ecc_pending, ecc_sbe_agg_cbu, ecc_sbe_agg_dev, ecc_sbe_agg_l1, ecc_sbe_agg_l2, ecc_sbe_agg_reg, ecc_sbe_agg_shm, ecc_sbe_agg_srm, ecc_sbe_agg_tex, ecc_sbe_vol_cbu, ecc_sbe_vol_dev, ecc_sbe_vol_l1, ecc_sbe_vol_l2, ecc_sbe_vol_reg, ecc_sbe_vol_shm, ecc_sbe_vol_srm, ecc_sbe_vol_tex | errors |
187
+| dcgm.mig.memory.page_retirements | retired_dbe, retired_pending, retired_sbe | pages/s |
188
+| dcgm.mig.memory.usage | free, reserved, used | B |
189
+| dcgm.mig.memory.capacity | total | B |
190
+| dcgm.mig.memory.utilization | used_percent | % |
191
+| dcgm.mig.power.energy | total | mJ/s |
192
+| dcgm.mig.power.profiles | enforced_power_profile_mask, requested_power_profile_mask, valid_power_profile_mask | state |
193
+| dcgm.mig.power.smoothing | pwr_smoothing_active_preset_profile, pwr_smoothing_admin_override_percent_tmp_floor, pwr_smoothing_admin_override_ramp_down_hyst_val, pwr_smoothing_admin_override_ramp_down_rate, pwr_smoothing_admin_override_ramp_up_rate, pwr_smoothing_applied_tmp_ceil, pwr_smoothing_applied_tmp_floor, pwr_smoothing_enabled, pwr_smoothing_hw_circuitry_percent_lifetime_remaining, pwr_smoothing_imm_ramp_down_enabled, pwr_smoothing_max_num_preset_profiles, pwr_smoothing_max_percent_tmp_floor_setting, pwr_smoothing_min_percent_tmp_floor_setting, pwr_smoothing_priv_lvl, pwr_smoothing_profile_percent_tmp_floor, pwr_smoothing_profile_ramp_down_hyst_val, pwr_smoothing_profile_ramp_down_rate, pwr_smoothing_profile_ramp_up_rate | value |
194
+| dcgm.mig.power.usage | draw, enforced_limit, power_mgmt_limit, power_mgmt_limit_def, power_mgmt_limit_max, power_mgmt_limit_min, power_usage_instant | Watts |
195
+| dcgm.mig.reliability.memory_health | banks_remap_rows_avail_high, banks_remap_rows_avail_low, banks_remap_rows_avail_max, banks_remap_rows_avail_none, banks_remap_rows_avail_partial, memory_unrepairable_flag, threshold_srm | state |
196
+| dcgm.mig.reliability.recovery_action | get_gpu_recovery_action | state |
197
+| dcgm.mig.reliability.row_remap_events | correctable_remapped_rows, uncorrectable_remapped_rows | rows/s |
198
+| dcgm.mig.reliability.row_remap_status | row_remap_failure, row_remap_pending | state |
199
+| dcgm.mig.reliability.xid | xid | code |
200
+| dcgm.mig.state.configuration | autoboost, compute_mode, persistence_mode, sync_boost, sync_boost_violation | state |
201
+| dcgm.mig.state.performance | pstate | state |
202
+| dcgm.mig.state.virtualization | mig_mode, virtual_mode | state |
203
+| dcgm.mig.thermal.fan_speed | fan_speed | % |
204
+| dcgm.mig.thermal.temperature | gpu, gpu_max_op_temp, gpu_temp_limit, mem_max_op_temp, memory, shutdown_temp, slowdown_temp | Celsius |
205
+| dcgm.mig.throttle.reasons | clocks_event_reasons | bitmask |
206
+| dcgm.mig.throttle.violations | board_limit_violation, hw_power_brake_slowdown, hw_therm_slowdown, low_utilization_violation, power_violation, reliability_violation, sw_power_cap, sw_therm_slowdown, sync_boost, thermal_violation, total_app_clocks_violation, total_base_clocks_violation | milliseconds/s |
207
+
208
+### Per nvlink
209
+
210
+These metrics refer to NVLink link instances.
211
+
212
+Labels:
213
+
214
+| Label | Description |
215
+|:-----------|:----------------|
216
+| gpu | gpu label from exporter metrics. |
217
+| gpu_uuid | gpu_uuid label from exporter metrics. |
218
+| nvlink | nvlink label from exporter metrics. |
219
+
220
+Metrics:
221
+
222
+| Metric | Dimensions | Unit |
223
+|:------|:----------|:----|
224
+| dcgm.nvlink.interconnect.ber | nvlink_count_effective_ber, nvlink_count_effective_ber_float, nvlink_count_symbol_ber, nvlink_count_symbol_ber_float | ratio |
225
+| dcgm.nvlink.interconnect.congestion | nvlink_ppcnt_ibpc_port_xmit_wait | events/s |
226
+| dcgm.nvlink.interconnect.error_rate | gpu_nvlink_errors, nvlink_count_effective_errors, nvlink_count_fec_history_0, nvlink_count_fec_history_1, nvlink_count_fec_history_10, nvlink_count_fec_history_11, nvlink_count_fec_history_12, nvlink_count_fec_history_13, nvlink_count_fec_history_14, nvlink_count_fec_history_15, nvlink_count_fec_history_2, nvlink_count_fec_history_3, nvlink_count_fec_history_4, nvlink_count_fec_history_5, nvlink_count_fec_history_6, nvlink_count_fec_history_7, nvlink_count_fec_history_8, nvlink_count_fec_history_9, nvlink_count_link_recovery_events, nvlink_count_link_recovery_failed_events, nvlink_count_link_recovery_successful_events, nvlink_count_local_link_integrity_errors, nvlink_count_rx_buffer_overrun_errors, nvlink_count_rx_errors, nvlink_count_rx_general_errors, nvlink_count_rx_malformed_packet_errors, nvlink_count_rx_remote_errors, nvlink_count_rx_symbol_errors, nvlink_count_tx_discards, nvlink_crc_data_error, nvlink_crc_data_error_count_l0, nvlink_crc_data_error_count_l1, nvlink_crc_data_error_count_l10, nvlink_crc_data_error_count_l11, nvlink_crc_data_error_count_l12, nvlink_crc_data_error_count_l13, nvlink_crc_data_error_count_l14, nvlink_crc_data_error_count_l15, nvlink_crc_data_error_count_l16, nvlink_crc_data_error_count_l17, nvlink_crc_data_error_count_l2, nvlink_crc_data_error_count_l3, nvlink_crc_data_error_count_l4, nvlink_crc_data_error_count_l5, nvlink_crc_data_error_count_l6, nvlink_crc_data_error_count_l7, nvlink_crc_data_error_count_l8, nvlink_crc_data_error_count_l9, nvlink_crc_flit_error, nvlink_crc_flit_error_count_l0, nvlink_crc_flit_error_count_l1, nvlink_crc_flit_error_count_l10, nvlink_crc_flit_error_count_l11, nvlink_crc_flit_error_count_l12, nvlink_crc_flit_error_count_l13, nvlink_crc_flit_error_count_l14, nvlink_crc_flit_error_count_l15, nvlink_crc_flit_error_count_l16, nvlink_crc_flit_error_count_l17, nvlink_crc_flit_error_count_l2, nvlink_crc_flit_error_count_l3, nvlink_crc_flit_error_count_l4, nvlink_crc_flit_error_count_l5, nvlink_crc_flit_error_count_l6, nvlink_crc_flit_error_count_l7, nvlink_crc_flit_error_count_l8, nvlink_crc_flit_error_count_l9, nvlink_error_dl_crc, nvlink_error_dl_recovery, nvlink_error_dl_replay, nvlink_ppcnt_physical_successful_recovery_events, nvlink_ppcnt_plr_rcv_uncorrectable_code, nvlink_ppcnt_recovery_time_since_last, nvlink_ppcnt_recovery_total_successful_events, nvlink_pprm_oper_recovery, nvlink_recovery_error, nvlink_recovery_error_count_l0, nvlink_recovery_error_count_l1, nvlink_recovery_error_count_l10, nvlink_recovery_error_count_l11, nvlink_recovery_error_count_l12, nvlink_recovery_error_count_l13, nvlink_recovery_error_count_l14, nvlink_recovery_error_count_l15, nvlink_recovery_error_count_l16, nvlink_recovery_error_count_l17, nvlink_recovery_error_count_l2, nvlink_recovery_error_count_l3, nvlink_recovery_error_count_l4, nvlink_recovery_error_count_l5, nvlink_recovery_error_count_l6, nvlink_recovery_error_count_l7, nvlink_recovery_error_count_l8, nvlink_recovery_error_count_l9, nvlink_replay_error, nvlink_replay_error_count_l0, nvlink_replay_error_count_l1, nvlink_replay_error_count_l10, nvlink_replay_error_count_l11, nvlink_replay_error_count_l12, nvlink_replay_error_count_l13, nvlink_replay_error_count_l14, nvlink_replay_error_count_l15, nvlink_replay_error_count_l16, nvlink_replay_error_count_l17, nvlink_replay_error_count_l2, nvlink_replay_error_count_l3, nvlink_replay_error_count_l4, nvlink_replay_error_count_l5, nvlink_replay_error_count_l6, nvlink_replay_error_count_l7, nvlink_replay_error_count_l8, nvlink_replay_error_count_l9 | errors/s |
227
+| dcgm.nvlink.interconnect.errors | nvlink_ppcnt_plr_rcv_uncorrectable_code | errors |
228
+| dcgm.nvlink.interconnect.state | gpu_topology_nvlink, nvlink_get_state, nvlink_ppcnt_physical_link_down_counter, nvlink_ppcnt_plr_rcv_code_err, nvlink_ppcnt_plr_sync_events, nvlink_ppcnt_plr_xmit_retry_events, p2p_nvlink_status | state |
229
+| dcgm.nvlink.interconnect.throughput | nvlink_bandwidth, nvlink_bandwidth_l0, nvlink_bandwidth_l1, nvlink_bandwidth_l10, nvlink_bandwidth_l11, nvlink_bandwidth_l12, nvlink_bandwidth_l13, nvlink_bandwidth_l14, nvlink_bandwidth_l15, nvlink_bandwidth_l16, nvlink_bandwidth_l17, nvlink_bandwidth_l2, nvlink_bandwidth_l3, nvlink_bandwidth_l4, nvlink_bandwidth_l5, nvlink_bandwidth_l6, nvlink_bandwidth_l7, nvlink_bandwidth_l8, nvlink_bandwidth_l9, nvlink_count_rx, nvlink_count_tx, nvlink_l0_rx, nvlink_l0_tx, nvlink_l10_rx, nvlink_l10_tx, nvlink_l11_rx, nvlink_l11_tx, nvlink_l12_rx, nvlink_l12_tx, nvlink_l13_rx, nvlink_l13_tx, nvlink_l14_rx, nvlink_l14_tx, nvlink_l15_rx, nvlink_l15_tx, nvlink_l16_rx, nvlink_l16_tx, nvlink_l17_rx, nvlink_l17_tx, nvlink_l1_rx, nvlink_l1_tx, nvlink_l2_rx, nvlink_l2_tx, nvlink_l3_rx, nvlink_l3_tx, nvlink_l4_rx, nvlink_l4_tx, nvlink_l5_rx, nvlink_l5_tx, nvlink_l6_rx, nvlink_l6_tx, nvlink_l7_rx, nvlink_l7_tx, nvlink_l8_rx, nvlink_l8_tx, nvlink_l9_rx, nvlink_l9_tx, nvlink_rx_bandwidth, nvlink_rx_bandwidth_l0, nvlink_rx_bandwidth_l1, nvlink_rx_bandwidth_l10, nvlink_rx_bandwidth_l11, nvlink_rx_bandwidth_l12, nvlink_rx_bandwidth_l13, nvlink_rx_bandwidth_l14, nvlink_rx_bandwidth_l15, nvlink_rx_bandwidth_l16, nvlink_rx_bandwidth_l17, nvlink_rx_bandwidth_l2, nvlink_rx_bandwidth_l3, nvlink_rx_bandwidth_l4, nvlink_rx_bandwidth_l5, nvlink_rx_bandwidth_l6, nvlink_rx_bandwidth_l7, nvlink_rx_bandwidth_l8, nvlink_rx_bandwidth_l9, nvlink_rx, nvlink_tx_bandwidth, nvlink_tx_bandwidth_l0, nvlink_tx_bandwidth_l1, nvlink_tx_bandwidth_l10, nvlink_tx_bandwidth_l11, nvlink_tx_bandwidth_l12, nvlink_tx_bandwidth_l13, nvlink_tx_bandwidth_l14, nvlink_tx_bandwidth_l15, nvlink_tx_bandwidth_l16, nvlink_tx_bandwidth_l17, nvlink_tx_bandwidth_l2, nvlink_tx_bandwidth_l3, nvlink_tx_bandwidth_l4, nvlink_tx_bandwidth_l5, nvlink_tx_bandwidth_l6, nvlink_tx_bandwidth_l7, nvlink_tx_bandwidth_l8, nvlink_tx_bandwidth_l9, nvlink_tx | B/s |
230
+| dcgm.nvlink.interconnect.traffic | nvlink_count_rx_packets, nvlink_count_tx_packets, nvlink_ppcnt_plr_rcv_codes, nvlink_ppcnt_plr_xmit_codes, nvlink_ppcnt_plr_xmit_retry_codes | events/s |
231
+| dcgm.nvlink.internal.boundary | nvlink_ppcnt_recovery_time_between_last_two | state |
232
+| dcgm.nvlink.memory.ecc_error_rate | nvlink_ecc_data_error | errors/s |
233
+
234
+### Per nvswitch
235
+
236
+These metrics refer to NVSwitch instances.
237
+
238
+Labels:
239
+
240
+| Label | Description |
241
+|:-----------|:----------------|
242
+| nvswitch | nvswitch label from exporter metrics. |
243
+
244
+Metrics:
245
+
246
+| Metric | Dimensions | Unit |
247
+|:------|:----------|:----|
248
+| dcgm.nvswitch.interconnect.nvswitch.current | nvswitch_current_iddq, nvswitch_current_iddq_dvdd, nvswitch_current_iddq_rev | value |
249
+| dcgm.nvswitch.interconnect.nvswitch.errors | nvswitch_fatal_errors, nvswitch_link_crc_errors, nvswitch_link_crc_errors_lane0, nvswitch_link_crc_errors_lane1, nvswitch_link_crc_errors_lane2, nvswitch_link_crc_errors_lane3, nvswitch_link_crc_errors_lane4, nvswitch_link_crc_errors_lane5, nvswitch_link_crc_errors_lane6, nvswitch_link_crc_errors_lane7, nvswitch_link_fatal_errors, nvswitch_link_flit_errors, nvswitch_link_non_fatal_errors, nvswitch_link_recovery_errors, nvswitch_link_replay_errors, nvswitch_non_fatal_errors | errors/s |
250
+| dcgm.nvswitch.interconnect.nvswitch.latency | nvswitch_link_latency_count_vc0, nvswitch_link_latency_count_vc1, nvswitch_link_latency_count_vc2, nvswitch_link_latency_count_vc3, nvswitch_link_latency_high_vc0, nvswitch_link_latency_high_vc1, nvswitch_link_latency_high_vc2, nvswitch_link_latency_high_vc3, nvswitch_link_latency_low_vc0, nvswitch_link_latency_low_vc1, nvswitch_link_latency_low_vc2, nvswitch_link_latency_low_vc3, nvswitch_link_latency_medium_vc0, nvswitch_link_latency_medium_vc1, nvswitch_link_latency_medium_vc2, nvswitch_link_latency_medium_vc3, nvswitch_link_latency_panic_vc0, nvswitch_link_latency_panic_vc1, nvswitch_link_latency_panic_vc2, nvswitch_link_latency_panic_vc3 | events/s |
251
+| dcgm.nvswitch.interconnect.nvswitch.power | nvswitch_power_dvdd, nvswitch_power_hvdd, nvswitch_power_vdd | Watts |
252
+| dcgm.nvswitch.interconnect.nvswitch.status | nvswitch_link_status, nvswitch_link_type, nvswitch_reset_required | state |
253
+| dcgm.nvswitch.interconnect.nvswitch.throughput | nvswitch_link_throughput_rx, nvswitch_link_throughput_tx, nvswitch_throughput_rx, nvswitch_throughput_tx | B/s |
254
+| dcgm.nvswitch.interconnect.nvswitch.topology | nvswitch_device_uuid, nvswitch_link_device_link_id, nvswitch_link_device_link_sid, nvswitch_link_id, nvswitch_link_remote_pcie_bus, nvswitch_link_remote_pcie_device, nvswitch_link_remote_pcie_domain, nvswitch_link_remote_pcie_function, nvswitch_pcie_bus, nvswitch_pcie_device, nvswitch_pcie_domain, nvswitch_pcie_function, nvswitch_phys_id | value |
255
+| dcgm.nvswitch.interconnect.nvswitch.voltage | nvswitch_voltage_mvolt | mV |
256
+| dcgm.nvswitch.internal.boundary | first_nvswitch_field_id, last_nvswitch_field_id | state |
257
+| dcgm.nvswitch.memory.ecc_error_rate | nvswitch_link_ecc_errors, nvswitch_link_ecc_errors_lane0, nvswitch_link_ecc_errors_lane1, nvswitch_link_ecc_errors_lane2, nvswitch_link_ecc_errors_lane3, nvswitch_link_ecc_errors_lane4, nvswitch_link_ecc_errors_lane5, nvswitch_link_ecc_errors_lane6, nvswitch_link_ecc_errors_lane7 | errors/s |
258
+| dcgm.nvswitch.thermal.temperature | nvswitch_temperature_current, nvswitch_temperature_limit_shutdown, nvswitch_temperature_limit_slowdown | Celsius |
259
+
260
+### Per cpu
261
+
262
+These metrics refer to host CPU instances.
263
+
264
+Labels:
265
+
266
+| Label | Description |
267
+|:-----------|:----------------|
268
+| cpu | cpu label from exporter metrics. |
269
+
270
+Metrics:
271
+
272
+| Metric | Dimensions | Unit |
273
+|:------|:----------|:----|
274
+| dcgm.cpu.clock.frequency | cpu_clock_current | MHz |
275
+| dcgm.cpu.cpu.info | cpu_model, cpu_vendor | value |
276
+| dcgm.cpu.cpu.power | cpu_power_limit, cpu_power_util_current | Watts |
277
+| dcgm.cpu.cpu.temperature | cpu_temp_critical, cpu_temp_current, cpu_temp_warning | Celsius |
278
+| dcgm.cpu.cpu.utilization | cpu_util, cpu_util_irq, cpu_util_nice, cpu_util_sys, cpu_util_user | % |
279
+| dcgm.cpu.diagnostics.results | diag_cpu_eud_result | state |
280
+
281
+### Per cpu_core
282
+
283
+These metrics refer to host CPU core instances.
284
+
285
+Labels:
286
+
287
+| Label | Description |
288
+|:-----------|:----------------|
289
+| cpu | cpu label from exporter metrics. |
290
+| cpucore | cpucore label from exporter metrics. |
291
+
292
+Metrics:
293
+
294
+| Metric | Dimensions | Unit |
295
+|:------|:----------|:----|
296
+| dcgm.cpu_core.clock.frequency | cpu_clock_current | MHz |
297
+| dcgm.cpu_core.cpu.info | cpu_model, cpu_vendor | value |
298
+| dcgm.cpu_core.cpu.power | cpu_power_limit, cpu_power_util_current | Watts |
299
+| dcgm.cpu_core.cpu.temperature | cpu_temp_critical, cpu_temp_current, cpu_temp_warning | Celsius |
300
+| dcgm.cpu_core.cpu.utilization | cpu_util, cpu_util_irq, cpu_util_nice, cpu_util_sys, cpu_util_user | % |
301
+| dcgm.cpu_core.diagnostics.results | diag_cpu_eud_result | state |
302
+
303
+### Per exporter
304
+
305
+These metrics refer to exporter/global instances.
306
+
307
+Labels:
308
+
309
+| Label | Description |
310
+|:-----------|:----------------|
311
+| job | job label from exporter metrics. |
312
+
313
+Metrics:
314
+
315
+| Metric | Dimensions | Unit |
316
+|:------|:----------|:----|
317
+| dcgm.exporter.health.status | bind_unbind_event | state |
318
+| dcgm.exporter.inventory.software | cuda_driver_version, driver_version, nvml_version | value |
319
+
320
+
321
+
322
+## Alerts
323
+
324
+
325
+The following alerts are available:
326
+
327
+| Alert name | On metric | Description |
328
+|:------------|:----------|:------------|
329
+| [ dcgm_gpu_xid_errors ](https://github.com/netdata/netdata/blob/master/src/health/health.d/dcgm.conf) | dcgm.gpu.reliability.xid | NVIDIA driver reported GPU XID error on GPU ${label:gpu} |
330
+| [ dcgm_gpu_row_remap_failure ](https://github.com/netdata/netdata/blob/master/src/health/health.d/dcgm.conf) | dcgm.gpu.reliability.row_remap_status | GPU row remapping failed on GPU ${label:gpu} |
331
+| [ dcgm_gpu_uncorrectable_remapped_rows ](https://github.com/netdata/netdata/blob/master/src/health/health.d/dcgm.conf) | dcgm.gpu.reliability.row_remap_events | Uncorrectable remapped rows increased on GPU ${label:gpu} |
332
+| [ dcgm_gpu_power_violation ](https://github.com/netdata/netdata/blob/master/src/health/health.d/dcgm.conf) | dcgm.gpu.throttle.violations | Power throttling detected on GPU ${label:gpu} |
333
+| [ dcgm_gpu_thermal_violation ](https://github.com/netdata/netdata/blob/master/src/health/health.d/dcgm.conf) | dcgm.gpu.throttle.violations | Thermal throttling detected on GPU ${label:gpu} |
334
+
335
+
336
+## Setup
337
+
338
+
339
+You can configure the **dcgm** collector in two ways:
340
+
341
+| Method | Best for | How to |
342
+|-----------------------|------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|
343
+| [**UI**](#via-ui) | Fast setup without editing files | Go to **Nodes → Configure this node → Collectors → Jobs**, search for **dcgm**, then click **+** to add a job. |
344
+| [**File**](#via-file) | If you prefer configuring via file, or need to automate deployments (e.g., with Ansible) | Edit `go.d/dcgm.conf` and add a job. |
345
+
346
+:::important
347
+
348
+UI configuration requires paid Netdata Cloud plan.
349
+
350
+:::
351
+
352
+
353
+### Prerequisites
354
+
355
+#### Run dcgm-exporter
356
+
357
+Install DCGM and run `dcgm-exporter` so that a Prometheus endpoint is available (default `:9400/metrics`).
358
+
359
+#### Configure exporter field list
360
+
361
+The default exporter profile exposes a small subset of fields.
362
+Use the Netdata recommended profile:
363
+[`dcgm-exporter-netdata.csv`](https://github.com/netdata/netdata/blob/master/src/go/plugin/go.d/collector/dcgm/dcgm-exporter-netdata.csv)
364
+(raw download: `https://raw.githubusercontent.com/netdata/netdata/master/src/go/plugin/go.d/collector/dcgm/dcgm-exporter-netdata.csv`).
365
+
366
+The Netdata profile enables 127 fields by default and documents all remaining known DCGM fields as commented entries.
367
+To customize beyond the baseline, uncomment the field you need and comment one currently enabled field.
368
+
369
+Runtime validation artifact:
370
+`src/go/plugin/go.d/collector/dcgm/runtime-validation-driver-590.48.01-dcgm-exporter-4.4.1-4.5.2.md`
371
+and
372
+`src/go/plugin/go.d/collector/dcgm/runtime-validation-driver-590.48.01-dcgm-exporter-4.4.1-4.5.2.json`
373
+
374
+Validation is primarily version-scoped (NVIDIA driver + DCGM/DCGM-exporter versions), so treat it as a strong baseline rather than universal compatibility.
375
+
376
+Example:
377
+`dcgm-exporter -f /path/to/dcgm-exporter-netdata.csv`
378
+
379
+
380
+#### Keep collection intervals aligned
381
+
382
+Set Netdata `update_every` to the same value as dcgm-exporter collection interval (default 30 seconds).
383
+Example exporter interval: `dcgm-exporter -c 30000` and Netdata `update_every: 30`.
384
+
385
+
386
+#### Enable profiling capabilities (optional)
387
+
388
+Profiling fields may require additional privileges/capabilities in your runtime environment.
389
+
390
+
391
+### Configuration
392
+
393
+#### Options
394
+
395
+The following options can be defined globally: update_every, autodetection_retry.
396
+
397
+
398
+<details open><summary>Config options</summary>
399
+
400
+
401
+
402
+| Group | Option | Description | Default | Required |
403
+|:------|:-----|:------------|:--------|:---------:|
404
+| **Collection** | update_every | Data collection interval (seconds). Keep this aligned with dcgm-exporter collection interval. | 30 | no |
405
+| | autodetection_retry | Autodetection retry interval (seconds). Set 0 to disable. | 0 | no |
406
+| **Target** | url | DCGM exporter metrics endpoint URL. | http://127.0.0.1:9400/metrics | yes |
407
+| | timeout | HTTP request timeout (seconds). | 10 | no |
408
+| **Limits** | max_time_series | Global time series limit. If exceeded, collection is skipped for this cycle. | 2000 | no |
409
+| | max_time_series_per_metric | Per-metric time series limit. Metrics above this limit are skipped. | 200 | no |
410
+| **HTTP Auth** | username | Username for Basic HTTP authentication. | | no |
411
+| | password | Password for Basic HTTP authentication. | | no |
412
+| | bearer_token_file | Path to a file containing a bearer token. | | no |
413
+| **TLS** | tls_skip_verify | Skip TLS certificate and hostname verification (insecure). | no | no |
414
+| | tls_ca | Path to CA bundle used to validate the server certificate. | | no |
415
+| | tls_cert | Path to client TLS certificate (for mTLS). | | no |
416
+| | tls_key | Path to client TLS private key (for mTLS). | | no |
417
+| **Proxy** | proxy_url | HTTP proxy URL. | | no |
418
+| | proxy_username | Username for proxy authentication. | | no |
419
+| | proxy_password | Password for proxy authentication. | | no |
420
+| **Request** | headers | Additional HTTP headers to include in the request. | | no |
421
+| | method | HTTP method. | GET | no |
422
+| | body | HTTP request body. | | no |
423
+| | not_follow_redirects | Do not follow HTTP redirects. | no | no |
424
+| | force_http2 | Force HTTP/2 (including h2c over TCP). | no | no |
425
+| **Virtual Node** | vnode | Associate this job with a Virtual Node. | | no |
426
+
427
+
428
+</details>
429
+
430
+
431
+#### via UI
432
+
433
+Configure the **dcgm** collector from the Netdata web interface:
434
+
435
+1. Go to **Nodes**.
436
+2. Select the node **where you want the dcgm data-collection job to run** and click the :gear: (**Configure this node**). That node will run the data collection.
437
+3. The **Collectors → Jobs** view opens by default.
438
+4. In the Search box, type _dcgm_ (or scroll the list) to locate the **dcgm** collector.
439
+5. Click the **+** next to the **dcgm** collector to add a new job.
440
+6. Fill in the job fields, then click **Test** to verify the configuration and **Submit** to save.
441
+ - **Test** runs the job with the provided settings and shows whether data can be collected.
442
+ - If it fails, an error message appears with details (for example, connection refused, timeout, or command execution errors), so you can adjust and retest.
443
+
444
+
445
+#### via File
446
+
447
+The configuration file name for this integration is `go.d/dcgm.conf`.
448
+
449
+The file format is YAML. Generally, the structure is:
450
+
451
+```yaml
452
+update_every: 1
453
+autodetection_retry: 0
454
+jobs:
455
+ - name: some_name1
456
+ - name: some_name2
457
+```
458
+You can edit the configuration file using the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#edit-configuration-files) script from the
459
+Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#locate-your-config-directory).
460
+
461
+```bash
462
+cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
463
+sudo ./edit-config go.d/dcgm.conf
464
+```
465
+
466
+##### Examples
467
+
468
+###### Local exporter
469
+
470
+Collect metrics from a local dcgm-exporter endpoint.
471
+
472
+<details open><summary>Config</summary>
473
+
474
+```yaml
475
+jobs:
476
+ - name: local
477
+ url: http://127.0.0.1:9400/metrics
478
+ update_every: 30
479
+
480
+```
481
+</details>
482
+
483
+###### TLS endpoint
484
+
485
+Collect metrics over HTTPS with custom CA certificate.
486
+
487
+<details open><summary>Config</summary>
488
+
489
+```yaml
490
+jobs:
491
+ - name: secure
492
+ url: https://dcgm-exporter.example.com:9400/metrics
493
+ update_every: 30
494
+ tls_ca: /etc/netdata/certs/dcgm-ca.crt
495
+
496
+```
497
+</details>
498
+
499
+###### Increased cardinality limits
500
+
501
+Increase limits when collecting large field sets and multiple entities.
502
+
503
+<details open><summary>Config</summary>
504
+
505
+```yaml
506
+jobs:
507
+ - name: dcgm_large
508
+ url: http://127.0.0.1:9400/metrics
509
+ update_every: 30
510
+ max_time_series: 10000
511
+ max_time_series_per_metric: 2000
512
+
513
+```
514
+</details>
515
+
516
+
517
+
518
+## Troubleshooting
519
+
520
+### Debug Mode
521
+
522
+**Important**: Debug mode is not supported for data collection jobs created via the UI using the Dyncfg feature.
523
+
524
+To troubleshoot issues with the `dcgm` collector, run the `go.d.plugin` with the debug option enabled. The output
525
+should give you clues as to why the collector isn't working.
526
+
527
+- Navigate to the `plugins.d` directory, usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case on
528
+ your system, open `netdata.conf` and look for the `plugins` setting under `[directories]`.
529
+
530
+ ```bash
531
+ cd /usr/libexec/netdata/plugins.d/
532
+ ```
533
+
534
+- Switch to the `netdata` user.
535
+
536
+ ```bash
537
+ sudo -u netdata -s
538
+ ```
539
+
540
+- Run the `go.d.plugin` to debug the collector:
541
+
542
+ ```bash
543
+ ./go.d.plugin -d -m dcgm
544
+ ```
545
+
546
+ To debug a specific job:
547
+
548
+ ```bash
549
+ ./go.d.plugin -d -m dcgm -j jobName
550
+ ```
551
+
552
+### Getting Logs
553
+
554
+If you're encountering problems with the `dcgm` collector, follow these steps to retrieve logs and identify potential issues:
555
+
556
+- **Run the command** specific to your system (systemd, non-systemd, or Docker container).
557
+- **Examine the output** for any warnings or error messages that might indicate issues. These messages should provide clues about the root cause of the problem.
558
+
559
+#### System with systemd
560
+
561
+Use the following command to view logs generated since the last Netdata service restart:
562
+
563
+```bash
564
+journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep dcgm
565
+```
566
+
567
+#### System without systemd
568
+
569
+Locate the collector log file, typically at `/var/log/netdata/collector.log`, and use `grep` to filter for collector's name:
570
+
571
+```bash
572
+grep dcgm /var/log/netdata/collector.log
573
+```
574
+
575
+**Note**: This method shows logs from all restarts. Focus on the **latest entries** for troubleshooting current issues.
576
+
577
+#### Docker Container
578
+
579
+If your Netdata runs in a Docker container named "netdata" (replace if different), use this command:
580
+
581
+```bash
582
+docker logs netdata 2>&1 | grep dcgm
583
+```
584
+
585
+
src/go/plugin/go.d/collector/dcgm/metadata.yaml
+1
-1
@@ -9,7 +9,7 @@ modules:
9
link: https://github.com/NVIDIA/dcgm-exporter
10
icon_filename: nvidia.svg
11
categories:
12
- - data-collection.hardware-devices-and-sensors
12
+ - data-collection.hardware-and-iot
13
keywords:
14
- nvidia
15
- gpu
src/go/plugin/go.d/collector/elasticsearch/integrations/elasticsearch.md
+3
-5
@@ -43,12 +43,10 @@ This collector is supported on all platforms.
43
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
44
45
46
- can be monitored further using the following other integrations:
46
+Elasticsearch can be monitored further using the following other integrations:
47
48
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
49
-Applications{% /relatedResource %}
50
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
51
-Containers{% /relatedResource %}
48
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
49
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
50
51
### Default Behavior
52
src/go/plugin/go.d/collector/elasticsearch/integrations/opensearch.md
+3
-5
@@ -43,12 +43,10 @@ This collector is supported on all platforms.
43
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
44
45
46
- can be monitored further using the following other integrations:
46
+OpenSearch can be monitored further using the following other integrations:
47
48
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
49
-Applications{% /relatedResource %}
50
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
51
-Containers{% /relatedResource %}
48
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
49
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
50
51
### Default Behavior
52
src/go/plugin/go.d/collector/envoy/integrations/envoy.md
+2
-3
@@ -32,10 +32,9 @@ This collector is supported on all platforms.
32
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
33
34
35
- can be monitored further using the following other integrations:
35
+Envoy can be monitored further using the following other integrations:
36
37
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
38
-Applications{% /relatedResource %}
37
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
38
39
### Default Behavior
40
src/go/plugin/go.d/collector/geth/integrations/go-ethereum.md
+2
-3
@@ -32,10 +32,9 @@ This collector is supported on all platforms.
32
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
33
34
35
- can be monitored further using the following other integrations:
35
+Go-ethereum can be monitored further using the following other integrations:
36
37
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
38
-Applications{% /relatedResource %}
37
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
38
39
### Default Behavior
40
src/go/plugin/go.d/collector/k8s_apiserver/integrations/kubernetes_api_server.md
+5
-8
@@ -46,14 +46,11 @@ The ServiceAccount used must have permissions to access the `/metrics` endpoint.
46
In most clusters, this requires cluster-admin or a custom ClusterRole with metrics access.
47
48
49
- can be monitored further using the following other integrations:
50
-
51
-- {% relatedResource id="go.d.plugin-k8s_kubelet-Kubelet</summary>
52
-Kubelet{% /relatedResource %}
53
-- {% relatedResource id="go.d.plugin-k8s_kubeproxy-Kubeproxy</summary>
54
-Kubeproxy{% /relatedResource %}
55
-- {% relatedResource id="go.d.plugin-k8s_state-Kubernetes_Cluster_State</summary>
56
-Kubernetes Cluster State{% /relatedResource %}
49
+Kubernetes API Server can be monitored further using the following other integrations:
50
+
51
+- {% relatedResource id="go.d.plugin-k8s_kubelet-Kubelet" %}Kubelet{% /relatedResource %}
52
+- {% relatedResource id="go.d.plugin-k8s_kubeproxy-Kubeproxy" %}Kubeproxy{% /relatedResource %}
53
+- {% relatedResource id="go.d.plugin-k8s_state-Kubernetes_Cluster_State" %}Kubernetes Cluster State{% /relatedResource %}
54
55
### Default Behavior
56
src/go/plugin/go.d/collector/k8s_kubelet/integrations/kubelet.md
+2
-3
@@ -32,10 +32,9 @@ This collector is supported on all platforms.
32
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
33
34
35
- can be monitored further using the following other integrations:
35
+Kubelet can be monitored further using the following other integrations:
36
37
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
38
-Applications{% /relatedResource %}
37
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
38
39
### Default Behavior
40
src/go/plugin/go.d/collector/k8s_kubeproxy/integrations/kubeproxy.md
+2
-3
@@ -32,10 +32,9 @@ This collector is supported on all platforms.
32
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
33
34
35
- can be monitored further using the following other integrations:
35
+Kubeproxy can be monitored further using the following other integrations:
36
37
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
38
-Applications{% /relatedResource %}
37
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
38
39
### Default Behavior
40
src/go/plugin/go.d/collector/lighttpd/integrations/lighttpd.md
+5
-8
@@ -34,14 +34,11 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
38
-
39
-- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files</summary>
40
-Web server log files{% /relatedResource %}
41
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
42
-HTTP Endpoints{% /relatedResource %}
43
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
44
-Applications{% /relatedResource %}
37
+Lighttpd can be monitored further using the following other integrations:
38
+
39
+- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files" %}Web server log files{% /relatedResource %}
40
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
41
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
42
43
### Default Behavior
44
src/go/plugin/go.d/collector/mssql/integrations/microsoft_sql_server.md
+3
-5
@@ -53,12 +53,10 @@ SQL Agent job monitoring is part of collector startup, so access to
53
`msdb.dbo.sysjobs` is required.
54
55
56
- can be monitored further using the following other integrations:
56
+Microsoft SQL Server can be monitored further using the following other integrations:
57
58
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
59
-Applications{% /relatedResource %}
60
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
61
-Containers{% /relatedResource %}
58
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
59
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
60
61
### Default Behavior
62
src/go/plugin/go.d/collector/mysql/integrations/mariadb.md
+3
-5
@@ -43,12 +43,10 @@ This collector is supported on all platforms.
43
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
44
45
46
- can be monitored further using the following other integrations:
46
+MariaDB can be monitored further using the following other integrations:
47
48
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
49
-Applications{% /relatedResource %}
50
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
51
-Containers{% /relatedResource %}
48
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
49
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
50
51
### Default Behavior
52
src/go/plugin/go.d/collector/mysql/integrations/mysql.md
+3
-5
@@ -43,12 +43,10 @@ This collector is supported on all platforms.
43
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
44
45
46
- can be monitored further using the following other integrations:
46
+MySQL can be monitored further using the following other integrations:
47
48
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
49
-Applications{% /relatedResource %}
50
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
51
-Containers{% /relatedResource %}
48
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
49
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
50
51
### Default Behavior
52
src/go/plugin/go.d/collector/mysql/integrations/percona_mysql.md
+3
-5
@@ -43,12 +43,10 @@ This collector is supported on all platforms.
43
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
44
45
46
- can be monitored further using the following other integrations:
46
+Percona MySQL can be monitored further using the following other integrations:
47
48
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
49
-Applications{% /relatedResource %}
50
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
51
-Containers{% /relatedResource %}
48
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
49
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
50
51
### Default Behavior
52
src/go/plugin/go.d/collector/nginx/integrations/nginx.md
+6
-10
@@ -33,16 +33,12 @@ This collector is supported on all platforms.
33
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
34
35
36
- can be monitored further using the following other integrations:
37
-
38
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
39
-HTTP Endpoints{% /relatedResource %}
40
-- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files</summary>
41
-Web server log files{% /relatedResource %}
42
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
43
-Applications{% /relatedResource %}
44
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
45
-Containers{% /relatedResource %}
36
+NGINX can be monitored further using the following other integrations:
37
+
38
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
39
+- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files" %}Web server log files{% /relatedResource %}
40
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
41
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
42
43
### Default Behavior
44
src/go/plugin/go.d/collector/nginxvts/integrations/nginx_vts.md
+5
-8
@@ -34,14 +34,11 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
38
-
39
-- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files</summary>
40
-Web server log files{% /relatedResource %}
41
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
42
-HTTP Endpoints{% /relatedResource %}
43
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
44
-Applications{% /relatedResource %}
37
+NGINX VTS can be monitored further using the following other integrations:
38
+
39
+- {% relatedResource id="go.d.plugin-web_log-Web_server_log_files" %}Web server log files{% /relatedResource %}
40
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
41
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
42
43
### Default Behavior
44
src/go/plugin/go.d/collector/postgres/integrations/postgresql.md
+3
-5
@@ -34,12 +34,10 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
37
+PostgreSQL can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
41
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
42
-Containers{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
41
42
### Default Behavior
43
src/go/plugin/go.d/collector/pulsar/integrations/apache_pulsar.md
+2
-3
@@ -33,10 +33,9 @@ This collector is supported on all platforms.
33
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
34
35
36
- can be monitored further using the following other integrations:
36
+Apache Pulsar can be monitored further using the following other integrations:
37
38
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
39
-Applications{% /relatedResource %}
38
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
39
40
### Default Behavior
41
src/go/plugin/go.d/collector/redis/integrations/redis.md
+3
-5
@@ -36,12 +36,10 @@ This collector is supported on all platforms.
36
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
37
38
39
- can be monitored further using the following other integrations:
39
+Redis can be monitored further using the following other integrations:
40
41
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
42
-Applications{% /relatedResource %}
43
-- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers</summary>
44
-Containers{% /relatedResource %}
41
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
42
+- {% relatedResource id="cgroups.plugin-/sys/fs/cgroup-Containers" %}Containers{% /relatedResource %}
43
44
### Default Behavior
45
src/go/plugin/go.d/collector/rspamd/integrations/rspamd.md
+3
-5
@@ -33,12 +33,10 @@ This collector is supported on all platforms.
33
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
34
35
36
- can be monitored further using the following other integrations:
36
+Rspamd can be monitored further using the following other integrations:
37
38
-- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints</summary>
39
-HTTP Endpoints{% /relatedResource %}
40
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
41
-Applications{% /relatedResource %}
38
+- {% relatedResource id="go.d.plugin-httpcheck-HTTP_Endpoints" %}HTTP Endpoints{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
41
### Default Behavior
42
src/go/plugin/go.d/collector/zookeeper/integrations/zookeeper.md
+2
-3
@@ -34,10 +34,9 @@ This collector is supported on all platforms.
34
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
35
36
37
- can be monitored further using the following other integrations:
37
+ZooKeeper can be monitored further using the following other integrations:
38
39
-- {% relatedResource id="apps.plugin-apps-Applications</summary>
40
-Applications{% /relatedResource %}
39
+- {% relatedResource id="apps.plugin-apps-Applications" %}Applications{% /relatedResource %}
40
41
### Default Behavior
42