| 1 | # Windows Events plugin |
| 2 | |
| 3 | [KEY FEATURES](#key-features) | [EVENTS SOURCES](#events-sources) | [EVENT FIELDS](#event-fields) | |
| 4 | [PLAY MODE](#play-mode) | [FULL TEXT SEARCH](#full-text-search) | [PERFORMANCE](#query-performance) | |
| 5 | [CONFIGURATION](#configuration-and-maintenance) | [FAQ](#faq) |
| 6 | |
| 7 | The Windows Events plugin by Netdata makes viewing, exploring and analyzing Windows Events simple and |
| 8 | efficient. |
| 9 | |
| 10 |  |
| 11 | |
| 12 | ## Key features |
| 13 | |
| 14 | - Supports **Windows Event Logs (WEL)**. |
| 15 | - Supports **Event Tracing for Windows (ETW)** and **TraceLogging (TL)**, when events are routed to Event Log. |
| 16 | - Allows filtering on all System Events fields. |
| 17 | - Allows **full text search** (`grep`) on all System and User fields. |
| 18 | - Provides a **histogram** for log entries over time, with a break down per field-value, for any System Event field and any |
| 19 | time-frame. |
| 20 | - Supports coloring log entries based on severity. |
| 21 | - In PLAY mode it "tails" all the Events, showing new log entries immediately after they are received. |
| 22 | |
| 23 | ### Prerequisites |
| 24 | |
| 25 | `windows-events.plugin` is a Netdata Function Plugin. |
| 26 | |
| 27 | To protect your privacy, as with all Netdata Functions, a free Netdata Cloud user account is required to access it. |
| 28 | For more information check [this discussion](https://github.com/netdata/netdata/discussions/16136). |
| 29 | |
| 30 | ## Events Sources |
| 31 | |
| 32 | The plugin automatically detects all the available channels and offers a list of "Event Channels". |
| 33 | |
| 34 | By default, it aggregates events from all event channels, providing a unified view of all events. |
| 35 | |
| 36 | > To improve query performance, we recommend selecting the relevant event channels, before doing more |
| 37 | > analysis on the events. |
| 38 | |
| 39 | In the list of events channels, several shortcuts are added, aggregating events according to various attributes: |
| 40 | |
| 41 | - `All`, aggregates events from all available channels. This provides a holistic view of all events in the system. |
| 42 | - `All-Admin`, `All-Operational`, `All-Analytic` and `All-Debug` aggregates events from channels marked `Admin`, `Operational`, `Analytic` and `Debug`, respectively. |
| 43 | - `All-Windows`, aggregates events from `Application`, `Security`, `System` and `Setup`. |
| 44 | - `All-Enabled` and `All-Disabled` aggregates events from channels depending on their status. |
| 45 | - `All-Forwarded` aggregates events from channels owned by `Microsoft-Windows-EventCollector`. |
| 46 | - `All-Classic` aggregates events from channels using the Classic Event Log API. |
| 47 | - `All-Of-X`, where `X` is a provider name, is offered for all providers having more than a channel. |
| 48 | - `All-In-X`, where `X` is `Backup-Mode`, `Overwrite-Mode`, `StopWhenFull-Mode` and `RetainAndBackup-Mode`, aggregate events based on their channel retention policy. |
| 49 | |
| 50 | Channels that are configured but are not queryable, and channels that do not have any events in them, are automatically excluded from the channels list. |
| 51 | |
| 52 | ## Event Fields |
| 53 | |
| 54 | Windows Events are structured with both system-defined fields and user-defined fields. |
| 55 | The Windows Events plugin primarily works with the system-defined fields, which are consistently available |
| 56 | across all event types. |
| 57 | |
| 58 | ### System-defined fields |
| 59 | |
| 60 | The system-defined fields are: |
| 61 | |
| 62 | 1. **EventRecordID** |
| 63 | A unique, sequential identifier for the event within the channel. This ID increases as new events are logged. |
| 64 | |
| 65 | 2. **Version** |
| 66 | The version of the event, indicating possible structural changes or updates to the event definition. |
| 67 | |
| 68 | Netdata adds this field automatically when it is not zero. |
| 69 | |
| 70 | 3. **Level** |
| 71 | The severity or importance of the event. Levels can include: |
| 72 | - 0: LogAlways (reserved) |
| 73 | - 1: Critical |
| 74 | - 2: Error |
| 75 | - 3: Warning |
| 76 | - 4: Informational |
| 77 | - 5: Verbose |
| 78 | |
| 79 | Additionally, applications may define their own levels. |
| 80 | |
| 81 | Netdata provides 2 fields: `Level` and `LevelID` for the text and numeric representation of it. |
| 82 | |
| 83 | 4. **Opcode** |
| 84 | The action or state within a provider when the event was logged. |
| 85 | |
| 86 | Netdata provides 2 fields: `Opcode` and `OpcodeID` for the text and numeric representation of it. |
| 87 | |
| 88 | 5. **EventID** |
| 89 | This identifies the event template, linking it to a specific message or event type. Event IDs are provider-specific. |
| 90 | |
| 91 | 6. **Task** |
| 92 | Defines a higher-level categorization or logical grouping for the event, often related to a specific function within the application or provider. |
| 93 | |
| 94 | Netdata provides 2 fields: `Task` and `TaskID` for the text and numeric representation of it. |
| 95 | |
| 96 | 7. **Qualifiers** |
| 97 | Provides additional detail for interpreting the event and is often specific to the event source. |
| 98 | |
| 99 | Netdata adds this field automatically when it is not zero. |
| 100 | |
| 101 | 8. **ProcessID** |
| 102 | The ID of the process that generated the event, useful for pinpointing the source of the event within the system. |
| 103 | |
| 104 | 9. **ThreadID** |
| 105 | The ID of the thread within the process that generated the event, which helps in more detailed debugging scenarios. |
| 106 | |
| 107 | 10. **Keywords** |
| 108 | A categorization field that can be used for event filtering. Keywords are bit flags that represent categories or purposes of the event, providing additional context. |
| 109 | |
| 110 | Netdata provides 2 fields: `Keywords` and `keywordsID` for the text and numeric representation of it. |
| 111 | |
| 112 | 11. **Provider** |
| 113 | The unique identifier (GUID) of the event provider. This is essential for knowing which application or system component generated the event. |
| 114 | |
| 115 | Netdata provides 2 fields: `Provider` and `ProviderGUID` for its name and GUID of it. |
| 116 | |
| 117 | 12. **ActivityID** |
| 118 | A GUID that correlates events generated as part of the same operation or transaction, helping to track activities across different components or stages. |
| 119 | |
| 120 | Netdata adds this field automatically when it is not zero. |
| 121 | |
| 122 | 13. **RelatedActivityID** |
| 123 | A GUID that links related operations or transactions, allowing for tracing complex workflows where one event triggers or relates to another. |
| 124 | |
| 125 | Netdata adds this field automatically when it is not zero. |
| 126 | |
| 127 | 14. **Timestamp** |
| 128 | The timestamp when the event was created. This provides precise timing information about when the event occurred. |
| 129 | |
| 130 | 15. **User** |
| 131 | The system user who logged this event. |
| 132 | |
| 133 | Netdata provides 3 fields: `UserAccount`, `UserDomain` and `UserSID`. |
| 134 | |
| 135 | ### User-defined fields |
| 136 | Each event log entry can include up to 100 user-defined fields (per event-id). |
| 137 | |
| 138 | Unfortunately, accessing these fields is significantly slower, to a level that is not practical to do so |
| 139 | when there are more than few thousand log entries to explore. So, Netdata presents them |
| 140 | with lazy loading. |
| 141 | |
| 142 | This prevents Netdata for offering filtering for user-defined fields, although Netdata does support |
| 143 | full text search on user-defined field values. |
| 144 | |
| 145 | ### Event fields as columns in the table |
| 146 | |
| 147 | The system fields mentioned above are offered as columns on the UI. Use the gear button above the table to |
| 148 | select visible columns. |
| 149 | |
| 150 | ### Event fields as filters |
| 151 | |
| 152 | The plugin presents the system fields as filters for the query, with counters for each of the possible values |
| 153 | for the field. This list can be used to quickly check which fields and values are available for the entire |
| 154 | time-frame of the query, across multiple providers and channels. |
| 155 | |
| 156 | ### Event fields as histogram sources |
| 157 | |
| 158 | The histogram can be based on any of the system fields that are available as filters. This allows you to |
| 159 | visualize the distribution of events over time based on different criteria such as Level, Provider, or EventID. |
| 160 | |
| 161 | ## PLAY mode |
| 162 | |
| 163 | The PLAY mode in this plugin allows real-time monitoring of new events as they are added to the Windows Event |
| 164 | Log. This feature works by continuously querying for new events and updating the display. |
| 165 | |
| 166 | ## Full-text search |
| 167 | |
| 168 | The plugin supports searching for text within all system and user fields of the events. This means that while |
| 169 | user-defined fields are not directly filterable, they are searchable through the full-text search feature. |
| 170 | |
| 171 | Keep in mind that query performance is slower while doing full text search, mainly because the plugin |
| 172 | needs to ask from the system to provide all the user fields values. |
| 173 | |
| 174 | ## Query performance |
| 175 | |
| 176 | The plugin is optimized to work efficiently with Event Logs. It uses several layers of caching and |
| 177 | similar techniques to offload as much work as possible from the system, offering quick responses even when |
| 178 | hundreds of thousands of events are within the visible timeframe. |
| 179 | |
| 180 | To achieve this level of efficiency, the plugin: |
| 181 | |
| 182 | - pre-loads ETW providers' manifests for resolving numeric Levels, Opcodes, Tasks and Keywords to text. |
| 183 | - caches number to text maps for Levels, Opcodes, Tasks and Keywords per provider for WEL providers. |
| 184 | - caches user SID to account and domain maps. |
| 185 | - lazy loads the "expensive" event Message and XML, so that the system is queried only for the visible events. |
| 186 | |
| 187 | For Full Text Search: |
| 188 | |
| 189 | - requests only the Message and the values of the user-fields from the system, avoiding the "expensive" XML call (which is still lazy-loaded). |
| 190 | |
| 191 | The result is a system that is highly efficient for working with moderate volumes (hundreds of thousands) of events. |
| 192 | |
| 193 | ## Configuration and maintenance |
| 194 | |
| 195 | This Netdata plugin does not require any specific configuration. It automatically detects available event logs |
| 196 | on the system. |
| 197 | |
| 198 | ## FAQ |
| 199 | |
| 200 | ### Can I use this plugin on event centralization servers? |
| 201 | |
| 202 | Yes. You can centralize your Windows Events using Windows Event Forwarding (WEF) or other event collection |
| 203 | mechanisms, and then install Netdata on this events centralization server to explore the events of all your |
| 204 | infrastructure. |
| 205 | |
| 206 | This plugin will automatically provide multi-node views of your events and also give you the ability to |
| 207 | combine the events of multiple servers, as you see fit. |
| 208 | |
| 209 | ### Can I use this plugin from a parent Netdata? |
| 210 | |
| 211 | Yes. When your nodes are connected to a Netdata parent, all their functions are available via the parent's UI. |
| 212 | So, from the parent UI, you can access the functions of all your nodes. |
| 213 | |
| 214 | Keep in mind that to protect your privacy, in order to access Netdata functions, you need a free Netdata Cloud |
| 215 | account. |
| 216 | |
| 217 | ### Is any of my data exposed to Netdata Cloud from this plugin? |
| 218 | |
| 219 | No. When you access the Agent directly, none of your data passes through Netdata Cloud. You need a free Netdata |
| 220 | Cloud account only to verify your identity and enable the use of Netdata Functions. Once this is done, all the |
| 221 | data flow directly from your Netdata Agent to your web browser. |
| 222 | |
| 223 | When you access Netdata via https://app.netdata.cloud, your data travel via Netdata Cloud, but they are not stored |
| 224 | in Netdata Cloud. This is to allow you access your Netdata Agents from anywhere. All communication from/to |
| 225 | Netdata Cloud is encrypted. |
| 226 | |
| 227 | ### What are the different types of event logs supported by this plugin? |
| 228 | |
| 229 | The plugin supports all the kinds of event logs currently supported by the Windows Event Viewer: |
| 230 | |
| 231 | - Windows Event Logs (WEL): The traditional event logging system in Windows. |
| 232 | - Event Tracing for Windows (ETW): A more detailed and efficient event tracing system. |
| 233 | - TraceLogging (TL): An extension of ETW that simplifies the process of adding events to your code. |
| 234 | |
| 235 | The plugin can access all of these when they are routed to the Windows Event Log. |
| 236 | |
| 237 | ### How does this plugin handle user-defined fields in Windows Events? |
| 238 | |
| 239 | User-defined fields are not directly exposed as table columns or filters in the plugin interface. However, |
| 240 | they are included in the XML representation of each event, which can be viewed in the info sidebar when |
| 241 | clicking on an event entry. Additionally, the full-text search feature does search through these |
| 242 | user-defined fields, allowing you to find specific information even if it's not in the main system fields. |
| 243 | |
| 244 | ### Can I use this plugin to monitor real-time events? |
| 245 | |
| 246 | Yes, the plugin supports a PLAY mode that allows you to monitor events in real-time. When activated, it |
| 247 | continuously updates to show new events as they are logged, similar to the "tail" functionality in |
| 248 | Unix-like systems. |
| 249 | |
| 250 | ### How does the plugin handle large volumes of events? |
| 251 | |
| 252 | The plugin is designed to handle moderate volumes of events (hundreds of thousands of events) efficiently. |
| 253 | |
| 254 | It is in our roadmap to port the `systemd-journal` sampling techniques to it, for working with very large |
| 255 | datasets to provide quick responses while still giving accurate representations of the data. However, for |
| 256 | the best performance, we recommend querying smaller time frames or using more specific filters when dealing |
| 257 | with extremely large event volumes. |
| 258 | |
| 259 | ### Can I use this plugin to analyze events from multiple servers? |
| 260 | |
| 261 | Yes, if you have set up Windows Event Forwarding (WEF) or another method of centralizing your Windows Events, |
| 262 | you can use this plugin on the central server to analyze events from multiple sources. The plugin will |
| 263 | automatically detect the available event sources. |
| 264 | |
| 265 | ### How does the histogram feature work in this plugin? |
| 266 | |
| 267 | The histogram feature provides a visual representation of event frequency over time. You can base the |
| 268 | histogram on any of the system fields available as filters (such as Level, Provider, or EventID). This |
| 269 | allows you to quickly identify patterns or anomalies in your event logs. |
| 270 | |
| 271 | ### Is it possible to export or share the results from this plugin? |
| 272 | |
| 273 | While the plugin doesn't have a direct export feature, you can use browser-based methods to save or share |
| 274 | the results. This could include taking screenshots, using browser print/save as PDF functionality, or |
| 275 | copying data from the table view. For more advanced data export needs, you might need to use the Windows |
| 276 | Event Log API directly or other Windows administrative tools. |
| 277 | |
| 278 | ### How often does the plugin update its data? |
| 279 | |
| 280 | The plugin updates its data in real-time when in PLAY mode. In normal mode, it refreshes data based on the |
| 281 | query you've submitted. The plugin is designed to provide the most up-to-date information available in the |
| 282 | Windows Event Logs at the time of the query. |
| 283 | |
| 284 | ## TODO |
| 285 | |
| 286 | 1. Support Sampling, so that the plugin can respond faster even on very busy systems (millions of events visible). |
| 287 | 2. Support exploring events from live Tracing sessions. |
| 288 | 3. Support exploring events in saved Event Trace Log files (`.etl` files). |
| 289 | 4. Support exploring events in saved Event Logs files (`.evtx` files). |