| 1 | # AE_FIELDS reference |
| 2 | |
| 3 | Verified field map for the agent-events journal namespace. |
| 4 | Every claim here is traceable to producer source at |
| 5 | `<repo>/src/daemon/status-file.c` (the schema is |
| 6 | `STATUS_FILE_VERSION = 28`, `src/daemon/status-file.h:14`). |
| 7 | |
| 8 | The .local draft `agent-events-journals.md` was found to have |
| 9 | 14 high-severity divergences (wrong enums, missing fields, |
| 10 | misattributed semantics). This file supersedes it. |
| 11 | |
| 12 | ## How journal field names are formed |
| 13 | |
| 14 | Producer JSON path -> journal field name: |
| 15 | |
| 16 | 1. Producer emits a JSON path (e.g. `agent.profile[0]`). |
| 17 | 2. log2journal applies `--prefix 'AE_'` (literal prepend; the |
| 18 | prefix is NOT transliterated -- |
| 19 | `src/collectors/log2journal/log2journal-help.c:108-109`). |
| 20 | 3. log2journal walks the JSON tree. For nested objects, it |
| 21 | joins parent + child with `_`. For arrays, it appends |
| 22 | `_<index>` (`log2journal-json.c:477-511`). |
| 23 | 4. Per-character transliteration applies a 256-entry map |
| 24 | (`log2journal.c:8-61`): lowercase -> uppercase, digits and |
| 25 | uppercase pass through, everything else (including `.`, |
| 26 | `-`, `:`, `/`, `@`, `_`, `[`, `]`) maps to `_`. Consecutive |
| 27 | underscores are collapsed |
| 28 | (`log2journal-json.c:395-396`). |
| 29 | |
| 30 | So: |
| 31 | - `agent.id` -> `AE_AGENT_ID` |
| 32 | - `agent.profile[0]` -> `AE_AGENT_PROFILE_0` |
| 33 | - `host.boot.id` -> `AE_HOST_BOOT_ID` |
| 34 | - `@timestamp` -> `AE__TIMESTAMP` (note: double underscore |
| 35 | because `@` -> `_`) |
| 36 | - `fatal.errno` -> `AE_FATAL_ERRNO` (NOT from a top-level |
| 37 | `fatal_errno`; the path is nested) |
| 38 | |
| 39 | ## Top-level POST-time fields (always present) |
| 40 | |
| 41 | Added by `post_status_file()` at `status-file.c:967-976`. These |
| 42 | sit at the JSON top level, so they have NO nested-path |
| 43 | prefix in the journal field name. |
| 44 | |
| 45 | | JSON path | Journal field | Type | Always? | Triage meaning | |
| 46 | |---|---|---|---|---| |
| 47 | | `exit_cause` | `AE_EXIT_CAUSE` | string | yes | Human-readable label for why this session ended. The first thing to look at. See enum below. | |
| 48 | | `message` | `AE_MESSAGE` | string | yes | One-line summary; subject of FTS search. | |
| 49 | | `priority` | `AE_PRIORITY` | int | yes | Syslog priority (lower = more severe). | |
| 50 | | `version_saved` | `AE_VERSION_SAVED` | uint | yes | The status file's own version (typically 28). | |
| 51 | | `agent_version_now` | `AE_AGENT_VERSION_NOW` | string | yes | Posting agent's version (the agent that did the POST = next session, NOT the one that crashed). | |
| 52 | | `agent_pid_now` | `AE_AGENT_PID_NOW` | uint | yes | PID of the agent that POSTed. | |
| 53 | | `host_memory_critical` | `AE_HOST_MEMORY_CRITICAL` | bool | yes | Was the host under memory pressure at POST time? | |
| 54 | | `host_memory_free_percent` | `AE_HOST_MEMORY_FREE_PERCENT` | uint | yes | % free RAM at POST time. | |
| 55 | | `agent_health` | `AE_AGENT_HEALTH` | string | yes | Health classification across restart history. See enum below. | |
| 56 | | `@timestamp` | `AE__TIMESTAMP` | RFC3339 | yes | When the captured session ended. | |
| 57 | | `version` | `AE_VERSION` | uint | yes | The schema version (`STATUS_FILE_VERSION`). Use to slice by schema. | |
| 58 | |
| 59 | These fields are **always present** on every record and are |
| 60 | the safest scoping anchors. `AE_EXIT_CAUSE`, `AE_AGENT_HEALTH`, |
| 61 | `AE_VERSION` are all good index-friendly facets. |
| 62 | |
| 63 | ## `agent.*` fields (`dsf_json_agent`, `status-file.c:99-156`) |
| 64 | |
| 65 | The previous (crashed) session's agent state. |
| 66 | |
| 67 | | JSON path | Journal field | Type | Version-gating | Triage meaning | |
| 68 | |---|---|---|---|---| |
| 69 | | `agent.id` | `AE_AGENT_ID` | UUID | always | Netdata machine GUID (per-install, persistent). The "agent identity". DIFFERENT from `host.id`. | |
| 70 | | `agent.since` | `AE_AGENT_SINCE` | RFC3339 | v>=24 | When this install was first registered. | |
| 71 | | `agent.ephemeral_id` | `AE_AGENT_EPHEMERAL_ID` | UUID | always | Unique per Netdata invocation (changes on every restart). Use to group multiple events from the same crashed session. | |
| 72 | | `agent.version` | `AE_AGENT_VERSION` | string | always | The version of the *crashed* session. **Slice on this for regression-spotting.** | |
| 73 | | `agent.uptime` | `AE_AGENT_UPTIME` | int seconds | always | Duration the crashed session ran. Short uptime + crash = startup bug. | |
| 74 | | `agent.node_id` | `AE_AGENT_NODE_ID` | UUID | always | Cloud node UUID (empty when agent isn't claimed). | |
| 75 | | `agent.claim_id` | `AE_AGENT_CLAIM_ID` | UUID | always | Cloud claim UUID (empty when not claimed). Presence -> cloud-connected agent. | |
| 76 | | `agent.restarts` | `AE_AGENT_RESTARTS` | uint | always | Total restart count for this install. High value + recent crash = agent loop. | |
| 77 | | `agent.crashes` | `AE_AGENT_CRASHES` | uint | v>=24 | Total crash count. | |
| 78 | | `agent.pid` | `AE_AGENT_PID` | uint | v>=27 | PID of the crashed session. | |
| 79 | | `agent.posts` | `AE_AGENT_POSTS` | uint | v>=22 | Total POSTs from this install. | |
| 80 | | `agent.aclk` | `AE_AGENT_ACLK` | enum | v>=22 | Cloud connection state. See enum. | |
| 81 | | `agent.profile[N]` | `AE_AGENT_PROFILE_0..N` | enum array | always | Bitmap rendered as array. Slice by parent vs child vs iot. See enum. | |
| 82 | | `agent.status` | `AE_AGENT_STATUS` | enum | always | DAEMON_STATUS at the time of capture. See enum. | |
| 83 | | `agent.exit_reason[N]` | `AE_AGENT_EXIT_REASON_0..N` | enum array | always | EXIT_REASON bitmap rendered as array. Empty array = no specific reason. See enum. | |
| 84 | | `agent.install_type` | `AE_AGENT_INSTALL_TYPE` | string | always | `kickstart`, `binpkg`, `static`, etc. **Slice on this for "is this a packaging issue?"** | |
| 85 | | `agent.db_mode` | `AE_AGENT_DB_MODE` | string | v>=14 | dbengine memory mode. | |
| 86 | | `agent.db_tiers` | `AE_AGENT_DB_TIERS` | uint | v>=14 | Number of dbengine tiers. | |
| 87 | | `agent.kubernetes` | `AE_AGENT_KUBERNETES` | bool | v>=14 | Kubernetes deployment? Slice on this for k8s-specific issues. | |
| 88 | | `agent.sentry_available` | `AE_AGENT_SENTRY_AVAILABLE` | bool | v>=16 | Is Sentry enabled? | |
| 89 | | `agent.reliability` | `AE_AGENT_RELIABILITY` | int | always | Signed reliability counter (positive = healthy run streak; negative = crash streak). `<= -2` -> `crash-loop`. | |
| 90 | | `agent.stack_traces` | `AE_AGENT_STACK_TRACES` | string | always | Backtrace backend name (`libbacktrace`, `none`). | |
| 91 | | `agent.timings.init` | `AE_AGENT_TIMINGS_INIT` | int seconds | always | How long startup took. Long init + crash = startup bug. | |
| 92 | | `agent.timings.exit` | `AE_AGENT_TIMINGS_EXIT` | int seconds | always | How long shutdown took. | |
| 93 | |
| 94 | ## `metrics.*` fields (`dsf_json_metrics`, `:158-193`) |
| 95 | |
| 96 | Snapshot of the database at the time of capture. Useful for |
| 97 | "big-database crashes" investigations. |
| 98 | |
| 99 | | JSON path | Journal field | |
| 100 | |---|---| |
| 101 | | `metrics.nodes.total` | `AE_METRICS_NODES_TOTAL` | |
| 102 | | `metrics.nodes.receiving` | `AE_METRICS_NODES_RECEIVING` | |
| 103 | | `metrics.nodes.sending` | `AE_METRICS_NODES_SENDING` | |
| 104 | | `metrics.nodes.archived` | `AE_METRICS_NODES_ARCHIVED` | |
| 105 | | `metrics.metrics.collected` | `AE_METRICS_METRICS_COLLECTED` | |
| 106 | | `metrics.metrics.available` | `AE_METRICS_METRICS_AVAILABLE` | |
| 107 | | `metrics.instances.collected` | `AE_METRICS_INSTANCES_COLLECTED` | |
| 108 | | `metrics.instances.available` | `AE_METRICS_INSTANCES_AVAILABLE` | |
| 109 | | `metrics.contexts.collected` | `AE_METRICS_CONTEXTS_COLLECTED` | |
| 110 | | `metrics.contexts.available` | `AE_METRICS_CONTEXTS_AVAILABLE` | |
| 111 | |
| 112 | ## `host.*` fields (`dsf_json_host`, `:195-253`) |
| 113 | |
| 114 | Host-level info, mostly stable across crashes on the same |
| 115 | host. |
| 116 | |
| 117 | | JSON path | Journal field | Triage meaning | |
| 118 | |---|---|---| |
| 119 | | `host.id` | `AE_HOST_ID` | OS-level `/etc/machine-id`. **Different from `AE_AGENT_ID`** (Netdata's own identifier). Use to spot multiple agents on the same host. | |
| 120 | | `host.architecture` | `AE_HOST_ARCHITECTURE` | `x86_64`, `aarch64`, `armv7l`, ... **Slice for arch-specific bugs.** | |
| 121 | | `host.virtualization` | `AE_HOST_VIRTUALIZATION` | `none`, `kvm`, `vmware`, `lxc`, `docker`, ... | |
| 122 | | `host.container` | `AE_HOST_CONTAINER` | `none`, `docker`, `kubernetes`, ... | |
| 123 | | `host.uptime` | `AE_HOST_UPTIME` | **MISLEADING NAME.** Stores boottime EPOCH (`status-file.c:202` writes `ds->boottime` from `now_boottime_sec()`). NOT a duration. Compute uptime via `now - AE_HOST_UPTIME`. | |
| 124 | | `host.timezone` | `AE_HOST_TIMEZONE` | string, v>=20 | |
| 125 | | `host.cloud_provider` | `AE_HOST_CLOUD_PROVIDER` | `aws`, `gcp`, `azure`, ..., v>=20 | |
| 126 | | `host.cloud_instance` | `AE_HOST_CLOUD_INSTANCE` | EC2 instance type etc., v>=20 | |
| 127 | | `host.cloud_region` | `AE_HOST_CLOUD_REGION` | v>=20 | |
| 128 | | `host.system_cpus` | `AE_HOST_SYSTEM_CPUS` | uint. **Slice for "low-cpu environment" bugs.** | |
| 129 | | `host.boot.id` | `AE_HOST_BOOT_ID` | UUID, changes on every host boot. | |
| 130 | | `host.memory.total` | `AE_HOST_MEMORY_TOTAL` | bytes, only when `OS_SYSTEM_MEMORY_OK` | |
| 131 | | `host.memory.free` | `AE_HOST_MEMORY_FREE` | bytes | |
| 132 | | `host.memory.netdata` | `AE_HOST_MEMORY_NETDATA` | bytes used by netdata, v>=21 | |
| 133 | | `host.memory.oom_protection` | `AE_HOST_MEMORY_OOM_PROTECTION` | uint, v>=21 | |
| 134 | | `host.disk.db.total` | `AE_HOST_DISK_DB_TOTAL` | bytes available to dbengine | |
| 135 | | `host.disk.db.free` | `AE_HOST_DISK_DB_FREE` | bytes free | |
| 136 | | `host.disk.db.inodes_total` | `AE_HOST_DISK_DB_INODES_TOTAL` | uint | |
| 137 | | `host.disk.db.inodes_free` | `AE_HOST_DISK_DB_INODES_FREE` | uint | |
| 138 | | `host.disk.db.read_only` | `AE_HOST_DISK_DB_READ_ONLY` | bool. True + crash -> "disk read-only" cause. | |
| 139 | | `host.disk.netdata.dbengine` | `AE_HOST_DISK_NETDATA_DBENGINE` | bytes used by dbengine files. | |
| 140 | | `host.disk.netdata.sqlite` | `AE_HOST_DISK_NETDATA_SQLITE` | bytes used by SQLite files. | |
| 141 | | `host.disk.netdata.other` | `AE_HOST_DISK_NETDATA_OTHER` | bytes used by other files. | |
| 142 | | `host.disk.netdata.last_updated` | `AE_HOST_DISK_NETDATA_LAST_UPDATED` | RFC3339. | |
| 143 | |
| 144 | ## `os.*` fields (`dsf_json_os`, `:255-266`) |
| 145 | |
| 146 | | JSON path | Journal field | Triage meaning | |
| 147 | |---|---|---| |
| 148 | | `os.type` | `AE_OS_TYPE` | enum: `unknown`, `linux`, `freebsd`, `macos`, `windows`. | |
| 149 | | `os.kernel` | `AE_OS_KERNEL` | Kernel version string. | |
| 150 | | `os.name` | `AE_OS_NAME` | Distro name (e.g. `Ubuntu`, `CentOS Stream`). | |
| 151 | | `os.version` | `AE_OS_VERSION` | Distro version. | |
| 152 | | `os.family` | `AE_OS_FAMILY` | `os_id` (e.g. `ubuntu`). **Slice for distro-specific issues.** | |
| 153 | | `os.platform` | `AE_OS_PLATFORM` | `os_id_like` (parent distro family, e.g. `debian`). NOT a rewrite of `AE_OS_FAMILY` -- they're independent producer fields. | |
| 154 | |
| 155 | ## `hw.*` fields (`dsf_json_hw`, `:268-319`) |
| 156 | |
| 157 | DMI / SMBIOS data. Useful for hardware-specific bug |
| 158 | investigation. Privacy-sensitive serials and asset_tags are |
| 159 | **commented out at the producer side** (`status-file.c:275-276, |
| 160 | :294-295, :304-305`) and never reach the journal. |
| 161 | |
| 162 | | JSON path | Journal field | Notes | |
| 163 | |---|---|---| |
| 164 | | `hw.sys.vendor` | `AE_HW_SYS_VENDOR` | BIOS / system vendor. | |
| 165 | | `hw.sys.uuid` | `AE_HW_SYS_UUID` | System UUID. | |
| 166 | | `hw.product.name` | `AE_HW_PRODUCT_NAME` | Product name (e.g. `MacBookPro18,3`). | |
| 167 | | `hw.product.version` | `AE_HW_PRODUCT_VERSION` | | |
| 168 | | `hw.product.sku` | `AE_HW_PRODUCT_SKU` | | |
| 169 | | `hw.product.family` | `AE_HW_PRODUCT_FAMILY` | | |
| 170 | | `hw.board.name` | `AE_HW_BOARD_NAME` | | |
| 171 | | `hw.board.version` | `AE_HW_BOARD_VERSION` | | |
| 172 | | `hw.board.vendor` | `AE_HW_BOARD_VENDOR` | | |
| 173 | | `hw.chassis.type` | `AE_HW_CHASSIS_TYPE` | Numeric (e.g. `6` = desktop, `9` = laptop). | |
| 174 | | `hw.chassis.vendor` | `AE_HW_CHASSIS_VENDOR` | | |
| 175 | | `hw.chassis.version` | `AE_HW_CHASSIS_VERSION` | | |
| 176 | | `hw.bios.date` | `AE_HW_BIOS_DATE` | | |
| 177 | | `hw.bios.release` | `AE_HW_BIOS_RELEASE` | | |
| 178 | | `hw.bios.version` | `AE_HW_BIOS_VERSION` | | |
| 179 | | `hw.bios.vendor` | `AE_HW_BIOS_VENDOR` | | |
| 180 | |
| 181 | ## `product.*` fields (`dsf_json_product`, `:321-329`) |
| 182 | |
| 183 | | JSON path | Journal field | |
| 184 | |---|---| |
| 185 | | `product.vendor` | `AE_PRODUCT_VENDOR` | |
| 186 | | `product.name` | `AE_PRODUCT_NAME` | |
| 187 | | `product.type` | `AE_PRODUCT_TYPE` | |
| 188 | |
| 189 | ## `fatal.*` fields (`dsf_json_fatal`, `:331-367`) |
| 190 | |
| 191 | Present on crashes and deliberate fatal conditions. Empty on |
| 192 | graceful exits. |
| 193 | |
| 194 | | JSON path | Journal field | Version-gating | Triage meaning | |
| 195 | |---|---|---|---| |
| 196 | | `fatal.line` | `AE_FATAL_LINE` | always | Source line of the panic. Combine with FILENAME and FUNCTION for de-dup. | |
| 197 | | `fatal.filename` | `AE_FATAL_FILENAME` | always | Source file. **Slice on this for "this file is buggy".** | |
| 198 | | `fatal.function` | `AE_FATAL_FUNCTION` | always | Function name (with demangled symbol). **Slice on this for "this function is buggy".** | |
| 199 | | `fatal.message` | `AE_FATAL_MESSAGE` | always | Panic message. Subject of FTS. | |
| 200 | | `fatal.errno` | `AE_FATAL_ERRNO` | always | errno string at panic. | |
| 201 | | `fatal.thread` | `AE_FATAL_THREAD` | always | Worker thread name (e.g. `CTXLOAD`, `STREAM:63`). | |
| 202 | | `fatal.thread_id` | `AE_FATAL_THREAD_ID` | always | POSIX TID. | |
| 203 | | `fatal.stack_trace` | `AE_FATAL_STACK_TRACE` | always | Backtrace. Real addresses preserved (anonymization is dedup-only, `status-file-dedup.c:26-36`). | |
| 204 | | `fatal.signal_code` | `AE_FATAL_SIGNAL_CODE` | v>=16 | `SIGNAL/SI_CODE` formatted (e.g. `SIGSEGV/SEGV_MAPERR`). Empty -> not a signal crash. **Primary signal-crash predicate.** See enum. | |
| 205 | | `fatal.sentry` | `AE_FATAL_SENTRY` | v>=17 | Was a Sentry submission attempted? | |
| 206 | | `fatal.fault_address` | `AE_FATAL_FAULT_ADDRESS` | v>=18 | Hex address of the fault. Empty when `signal_code == 0`. | |
| 207 | | `fatal.worker_job_id` | `AE_FATAL_WORKER_JOB_ID` | v>=23 | Worker job ID at panic. | |
| 208 | |
| 209 | ## Enum reference |
| 210 | |
| 211 | ### `AE_AGENT_STATUS` (DAEMON_STATUS) |
| 212 | |
| 213 | Source: `src/daemon/status-file.c:23-33`. |
| 214 | |
| 215 | | Value | Meaning for triage | |
| 216 | |---|---| |
| 217 | | `none` | No prior status (very first session). | |
| 218 | | `initializing` | Crashed during startup -> startup bug. Combine with `agent.timings.init` for context. | |
| 219 | | `running` | Crashed during normal operation -> the most "interesting" class. | |
| 220 | | `exiting` | Crashed during shutdown -> shutdown-path bug. | |
| 221 | | `exited` | Graceful exit (no crash). | |
| 222 | |
| 223 | ### `AE_AGENT_ACLK` (CLOUD_STATUS) |
| 224 | |
| 225 | Source: `src/claim/cloud-status.c:5-15`. |
| 226 | |
| 227 | | Value | Meaning for triage | |
| 228 | |---|---| |
| 229 | | `available` | Default; not yet attempted. | |
| 230 | | `online` | Connected to Cloud (ACLK up). | |
| 231 | | `indirect` | Connected via parent. | |
| 232 | | `banned` | Cloud rejected (claim issue). | |
| 233 | | `offline` | Disconnected (network or shutdown). | |
| 234 | |
| 235 | (The .local draft listed a `disabled` value -- it does NOT |
| 236 | exist in the producer source.) |
| 237 | |
| 238 | ### `AE_AGENT_HEALTH` |
| 239 | |
| 240 | Source: `src/daemon/status-file.c:929-952`. Computed by the |
| 241 | **agent** (not the ingestion server) at POST time across |
| 242 | restart history. Used to isolate crash classes. |
| 243 | |
| 244 | | Value | Meaning for triage | |
| 245 | |---|---| |
| 246 | | `healthy-first` | First run, no prior crashes. Boring (filter out). | |
| 247 | | `healthy-repeated` | Multiple healthy runs in a row. | |
| 248 | | `healthy-loop` | Reliability >= 2 consecutive healthy runs. | |
| 249 | | `healthy-recovered` | Was unhealthy, now healthy. | |
| 250 | | `crash-first` | First crash ever on this install. Interesting -- new bug? | |
| 251 | | `crash-entered` | Single crash, then recovered. | |
| 252 | | `crash-loop` | Reliability <= -2 (repeated crashes). **Highest-priority class.** | |
| 253 | | `crash-repeated` | Two or more crashes. | |
| 254 | |
| 255 | To find ALL crashes: `(AE_AGENT_HEALTH in crash-first, crash-loop, crash-repeated, crash-entered)`. |
| 256 | |
| 257 | ### `AE_AGENT_PROFILE_*` (ND_PROFILE bitmap) |
| 258 | |
| 259 | Source: `src/daemon/config/netdata-conf-profile.c:7-15`. |
| 260 | |
| 261 | | Value | Meaning | |
| 262 | |---|---| |
| 263 | | `standalone` | Single-node deployment. | |
| 264 | | `parent` | Streaming parent. **Slice for "parent-only" bugs.** | |
| 265 | | `child` | Streaming child. **Slice for "child-only" bugs.** | |
| 266 | | `iot` | IoT / lightweight profile. | |
| 267 | |
| 268 | (The .local draft listed `dopple` and `store-child` -- they do |
| 269 | NOT exist; `iot` was missing.) |
| 270 | |
| 271 | ### `AE_AGENT_EXIT_REASON_*` (EXIT_REASON bitmap) |
| 272 | |
| 273 | Source: `src/libnetdata/exit/exit_initiated.c:7-38`. The |
| 274 | EXIT_REASON bitmap renders as a JSON array. Empty bitmap -> |
| 275 | empty array (no `none` element). |
| 276 | |
| 277 | 20 distinct strings: |
| 278 | |
| 279 | | Value | Meaning | |
| 280 | |---|---| |
| 281 | | `signal-segmentation-fault` | SIGSEGV received. | |
| 282 | | `signal-bus-error` | SIGBUS received. | |
| 283 | | `signal-floating-point-exception` | SIGFPE received. | |
| 284 | | `signal-illegal-instruction` | SIGILL received. | |
| 285 | | `signal-abort` | SIGABRT received (assertion / abort()). | |
| 286 | | `signal-bad-system-call` | SIGSYS received. | |
| 287 | | `signal-cpu-time-limit-exceeded` | SIGXCPU received. | |
| 288 | | `signal-file-size-limit-exceeded` | SIGXFSZ received. | |
| 289 | | `signal-quit` | SIGQUIT received. | |
| 290 | | `signal-terminate` | SIGTERM received (graceful kill). | |
| 291 | | `signal-interrupt` | SIGINT received (Ctrl-C). | |
| 292 | | `out-of-memory` | OOM panic. | |
| 293 | | `already-running` | Another instance held the listen socket. | |
| 294 | | `fatal` | Generic fatal() call. | |
| 295 | | `api-quit` | API endpoint requested exit. | |
| 296 | | `cmd-exit` | Explicit `netdata --exit` invocation. | |
| 297 | | `service-stop` | Service manager (systemd) sent stop. | |
| 298 | | `system-shutdown` | Host shutting down. | |
| 299 | | `update` | Replaced by a new version. | |
| 300 | | `shutdown-timeout` | Shutdown took too long. | |
| 301 | |
| 302 | (The .local draft was significantly wrong here -- listed |
| 303 | ~10 invented values like `exit-called`, `exit-and-update`, |
| 304 | `cannot-allocate`, `oom`, `assertion-failed`, none of which |
| 305 | exist in producer source.) |
| 306 | |
| 307 | ### `AE_EXIT_CAUSE` (top-level) |
| 308 | |
| 309 | Source: `src/daemon/status-file.c:1097-1286`. Computed by the |
| 310 | **agent**, NOT the ingestion server. The most useful field for |
| 311 | classifying records. |
| 312 | |
| 313 | 26 distinct strings: |
| 314 | |
| 315 | **Initial / no prior state (1):** |
| 316 | |
| 317 | | Value | Meaning | |
| 318 | |---|---| |
| 319 | | `no last status` | First-ever start; no prior status file readable. | |
| 320 | |
| 321 | **Prior was EXITED (graceful) (7):** |
| 322 | |
| 323 | | Value | Meaning | |
| 324 | |---|---| |
| 325 | | `exit no reason` | Prior exited cleanly with no reason recorded. | |
| 326 | | `deadly signal and exit` | Got a deadly signal but exited normally. | |
| 327 | | `fatal and exit` | Hit a fatal but managed to exit. | |
| 328 | | `exit on system shutdown` | Host shutting down; agent stopped gracefully. | |
| 329 | | `exit to update` | Stopped to allow an update. | |
| 330 | | `exit and updated` | Stopped and was replaced by a new version. | |
| 331 | | `exit instructed` | `netdata --exit` or service stop. | |
| 332 | |
| 333 | **Prior was INITIALIZING (8):** |
| 334 | |
| 335 | | Value | Meaning | |
| 336 | |---|---| |
| 337 | | `abnormal power off` | Power loss during startup. | |
| 338 | | `deadly signal on start` | Signal during startup. | |
| 339 | | `out of memory` | OOM during startup. (.local draft says `cannot allocate` -- wrong.) | |
| 340 | | `already running` | Listen socket conflict at init. | |
| 341 | | `disk read-only` | Filesystem read-only at init. | |
| 342 | | `disk full` | Disk full at init. | |
| 343 | | `disk almost full` | Disk near capacity at init. | |
| 344 | | `fatal on start` | fatal() during startup. | |
| 345 | | `killed hard on start` | SIGKILL/SIGTERM during startup. | |
| 346 | |
| 347 | **Prior was EXITING (5):** |
| 348 | |
| 349 | | Value | Meaning | |
| 350 | |---|---| |
| 351 | | `deadly signal on exit` | Signal during shutdown. | |
| 352 | | `exit timeout` | Shutdown didn't complete in time. | |
| 353 | | `fatal on exit` | fatal() during shutdown. | |
| 354 | | `killed hard on shutdown` | SIGKILL during shutdown (host shutdown). | |
| 355 | | `killed hard on update` | SIGKILL during hot update. | |
| 356 | | `killed hard on exit` | SIGKILL during exit. | |
| 357 | |
| 358 | **Prior was RUNNING (6):** |
| 359 | |
| 360 | | Value | Meaning | |
| 361 | |---|---| |
| 362 | | `abnormal power off` | Power loss during normal operation. | |
| 363 | | `out of memory` | OOM during normal operation. | |
| 364 | | `deadly signal` | Signal received during normal operation. | |
| 365 | | `killed fatal` | SIGKILL after a fatal. | |
| 366 | | `killed hard low ram` | OOM-killed (RAM pressure). | |
| 367 | | `killed hard` | SIGKILL/SIGTERM from outside. | |
| 368 | |
| 369 | ### `AE_OS_TYPE` (DAEMON_OS_TYPE) |
| 370 | |
| 371 | Source: `src/daemon/status-file.c:35-45`. |
| 372 | |
| 373 | `unknown`, `linux`, `freebsd`, `macos`, `windows`. |
| 374 | |
| 375 | ### `AE_FATAL_SIGNAL_CODE` |
| 376 | |
| 377 | Format: `SIGNAL/SI_CODE` (e.g. `SIGSEGV/SEGV_MAPERR`). Sources: |
| 378 | `src/libnetdata/signals/signal-code.c:12-53` (signal name map), |
| 379 | `:97-184` (per-signal SI_CODE map). |
| 380 | |
| 381 | Most relevant for crash triage: |
| 382 | |
| 383 | | Value | Meaning | |
| 384 | |---|---| |
| 385 | | `SIGSEGV/SEGV_MAPERR` | Invalid memory map (NULL pointer, freed memory). | |
| 386 | | `SIGSEGV/SEGV_ACCERR` | Access violation (write to read-only page). | |
| 387 | | `SIGSEGV/SEGV_BNDERR` | Address bound check fault. | |
| 388 | | `SIGSEGV/SEGV_PKUERR` | Protection key fault. | |
| 389 | | `SIGBUS/BUS_ADRALN` | Alignment error. | |
| 390 | | `SIGBUS/BUS_ADRERR` | Non-existent physical address. | |
| 391 | | `SIGBUS/BUS_OBJERR` | Object-specific bus error. | |
| 392 | | `SIGFPE/FPE_INTDIV` | Integer divide by zero. | |
| 393 | | `SIGFPE/FPE_INTOVF` | Integer overflow. | |
| 394 | | `SIGFPE/FPE_FLTDIV` | Float divide by zero. | |
| 395 | | `SIGABRT/SI_TKILL` | abort() / assertion failure (typical SI_CODE for abort()). | |
| 396 | | `SIGTRAP/TRAP_BRKPT` | Breakpoint trap. | |
| 397 | | `SIGTRAP/TRAP_TRACE` | Trace trap. | |
| 398 | |
| 399 | Empty `AE_FATAL_SIGNAL_CODE` -> not a signal crash (it's a |
| 400 | deliberate fatal or a graceful exit). |
| 401 | |
| 402 | (The .local draft had `SIGABRT/ABRT` which is wrong: `ABRT` |
| 403 | is not a valid SI_CODE token. And `SIGTRAP/TRAP Trace` should |
| 404 | be `SIGTRAP/TRAP_TRACE`. And `SIGVTALRM/VTALRM` does not |
| 405 | exist as a per-signal SI_CODE.) |
| 406 | |
| 407 | ## Index-friendly facets (high-value) |
| 408 | |
| 409 | These fields are the **first-pass slicers** for queries. |
| 410 | Always include at least 1-2 of these in `selections` before |
| 411 | falling back to FTS: |
| 412 | |
| 413 | - `AE_AGENT_VERSION` -- regression / fix-detection. |
| 414 | - `AE_AGENT_HEALTH` -- crash class. |
| 415 | - `AE_EXIT_CAUSE` -- exit class. |
| 416 | - `AE_FATAL_SIGNAL_CODE` -- signal type. |
| 417 | - `AE_FATAL_FUNCTION` -- localize to a function. |
| 418 | - `AE_FATAL_FILENAME` -- localize to a file. |
| 419 | - `AE_HOST_ARCHITECTURE` -- arch-specific bugs. |
| 420 | - `AE_OS_FAMILY` -- distro-specific bugs. |
| 421 | - `AE_AGENT_PROFILE_0` (and `_1`, `_2`) -- parent / child / iot. |
| 422 | - `AE_AGENT_KUBERNETES` -- k8s-specific. |
| 423 | - `AE_AGENT_INSTALL_TYPE` -- packaging-specific. |
| 424 | |
| 425 | ## Privacy-sensitive fields |
| 426 | |
| 427 | Treat these as identifying. The `redact-events.sh` opt-in |
| 428 | filter masks them when sharing: |
| 429 | |
| 430 | - `AE_AGENT_ID` (machine GUID). |
| 431 | - `AE_HOST_ID` (OS machine-id). |
| 432 | - `AE_AGENT_NODE_ID`, `AE_AGENT_CLAIM_ID` (Cloud identifiers). |
| 433 | - `AE_HOST_BOOT_ID`, `AE_AGENT_EPHEMERAL_ID`. |
| 434 | - `AE_HW_SYS_UUID`. |
| 435 | - DMI fields (`AE_HW_*`) when correlated with serial-equivalent |
| 436 | identifiers. |
| 437 | |
| 438 | (Privacy-sensitive serials and asset_tags are already |
| 439 | commented out at the producer side and never reach the |
| 440 | journal -- `status-file.c:275-276, :294-295, :304-305`.) |
| 441 | |
| 442 | ## What is NOT in the journal |
| 443 | |
| 444 | Several producer fields are intentionally redacted at the |
| 445 | producer side (commented out in `dsf_json_hw`): |
| 446 | |
| 447 | - `hw.sys.serial`, `hw.sys.asset_tag` |
| 448 | - `hw.board.serial`, `hw.board.asset_tag` |
| 449 | - `hw.chassis.serial`, `hw.chassis.asset_tag` |
| 450 | |
| 451 | There is no `agent.happiness` field in the producer source at |
| 452 | any version. The `.local` draft mentioned it -- the field has |
| 453 | never existed. |
| 454 | |
| 455 | ## Stack trace addresses are NOT anonymized in the journal |
| 456 | |
| 457 | `status-file-dedup.c:26-36` zeroes out hex addresses ONLY when |
| 458 | computing the dedup hash. The journal-emitted |
| 459 | `AE_FATAL_STACK_TRACE` retains real addresses. Useful for |
| 460 | bug investigation; sensitive when sharing externally. |