| 1 | <!--startmeta |
| 2 | custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/integrations/decapsulation.md" |
| 3 | meta_yaml: "https://github.com/netdata/netdata/edit/master/src/crates/netflow-plugin/metadata.yaml" |
| 4 | sidebar_label: "Decapsulation" |
| 5 | learn_status: "Published" |
| 6 | learn_rel_path: "Network Flows/Enrichment Methods" |
| 7 | keywords: ['decapsulation', 'srv6', 'vxlan', 'tunnel', 'overlay', 'inner packet', 'dataLinkFrameSection', 'layer2packetSectionData', 'ipfix ie 315', 'netflow v9 ie 104', 'sampled header'] |
| 8 | message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE FLOWS' metadata.yaml FILE" |
| 9 | endmeta--> |
| 10 | |
| 11 | <!-- markdownlint-disable-file --> |
| 12 | |
| 13 | # Decapsulation |
| 14 | |
| 15 | |
| 16 | <img src="https://netdata.cloud/img/network-wired.svg" width="150"/> |
| 17 | |
| 18 | |
| 19 | Plugin: netflow-plugin |
| 20 | Module: decapsulation |
| 21 | |
| 22 | <img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> |
| 23 | |
| 24 | ## Overview |
| 25 | |
| 26 | Enrich network flows with inner source and destination endpoints from VXLAN or |
| 27 | SRv6 encapsulated traffic. Decapsulation rewrites the 5-tuple of a flow record |
| 28 | from the **outer** (tunnel) endpoints to the **inner** (real) endpoints when the |
| 29 | exporter ships the encapsulated frame bytes alongside the standard fields. Without |
| 30 | it, every VM-to-VM conversation on a VXLAN fabric or every inner flow on an SRv6 |
| 31 | transit shows up as the same tunnel pair (hypervisor-to-hypervisor or PE-to-PE), |
| 32 | which tells you nothing about the actual traffic. |
| 33 | |
| 34 | Two encapsulations are supported, selected globally for the plugin. The mode set is |
| 35 | `none`, `srv6`, or `vxlan`; the default is `none`. |
| 36 | |
| 37 | | Mode | Strips | Surfaces | |
| 38 | |---------|--------------------------------------------------|--------------------------------------------------| |
| 39 | | `none` | nothing | the outer-header view (default) | |
| 40 | | `srv6` | IPv6 outer + extension headers + SRH (type 4) | the inner IPv4 (next-header 4) or IPv6 (41) | |
| 41 | | `vxlan` | outer Eth/IP + UDP/4789 + 8-byte VXLAN header | the inner Ethernet frame, then its L3/L4 | |
| 42 | |
| 43 | The plugin only reaches the decap path when the flow record carries the inner |
| 44 | frame bytes in a Layer-2 packet section field. Three transport-level paths feed it: |
| 45 | |
| 46 | - **NetFlow v9 IE 104** -- `Layer2packetSectionData` (RFC 5102 / RFC 7270). |
| 47 | - **IPFIX IE 315** -- `dataLinkFrameSection` (RFC 7133). |
| 48 | - **sFlow `SampledHeader`** -- always present in header-sampling mode. |
| 49 | |
| 50 | When decap succeeds, the inner 5-tuple replaces the outer one in the resulting |
| 51 | journal record: `SRC_ADDR`, `DST_ADDR`, `SRC_PORT`, `DST_PORT`, `PROTOCOL`, |
| 52 | `ETYPE`, `IPTOS`, `IPTTL`, `IPV6_FLOW_LABEL`, `TCP_FLAGS`, `IP_FRAGMENT_ID`, |
| 53 | `IP_FRAGMENT_OFFSET`, ICMP type/code, MPLS labels (if present), and `BYTES` |
| 54 | (set to the inner L3 length so byte counts represent inner payload, not outer |
| 55 | overhead). For VXLAN, `SRC_MAC`, `DST_MAC`, `SRC_VLAN`, `DST_VLAN` come from the |
| 56 | inner Ethernet frame -- the outer MACs and VLANs are lost. The VXLAN VNI is |
| 57 | parsed but not exposed as a journal field; pure VNI-based segmentation is |
| 58 | not visible. |
| 59 | |
| 60 | The `vxlan` parser matches **only UDP destination port 4789** (RFC 7348). |
| 61 | VXLAN-GPE on 4790 and any vendor-custom port are not recognised. The `srv6` |
| 62 | parser walks IPv6 extension headers and the Routing Header type 4 (SRH), then |
| 63 | surfaces the inner IPv4 or IPv6 packet pointed to by next-header 4 or 41. |
| 64 | |
| 65 | For the cross-cutting concept (how decap composes with the rest of the enrichment |
| 66 | pipeline, the non-tunnel "drop, do not fall back" semantics, and per-source |
| 67 | dependence on the L2-section path), see |
| 68 | [Decapsulation](https://learn.netdata.cloud/docs/network-flows/enrichment). |
| 69 | |
| 70 | |
| 71 | Set `protocols.decapsulation_mode` in `netflow.yaml` to `srv6` or `vxlan`. The |
| 72 | plugin then runs the inner-packet parser whenever a flow record carries an L2 |
| 73 | frame section (NetFlow v9 IE 104 / IPFIX IE 315 / sFlow `SampledHeader`). Plain |
| 74 | NetFlow / IPFIX records that do **not** carry an L2 section pass through |
| 75 | unchanged regardless of the setting -- so enabling decap will not break a mixed |
| 76 | stream where only a subset of exporters ship the frame bytes. Enabling decap is |
| 77 | half the work; the exporter must also be configured to ship the inner frame. |
| 78 | |
| 79 | |
| 80 | This integration is only supported on the following platforms: |
| 81 | |
| 82 | - Linux |
| 83 | |
| 84 | This integration runs as a single instance per Netdata Agent. |
| 85 | |
| 86 | |
| 87 | ### Default Behavior |
| 88 | |
| 89 | #### Auto-Detection |
| 90 | |
| 91 | Disabled by default (`decapsulation_mode: none`). You must opt in by setting the mode explicitly. |
| 92 | |
| 93 | #### Limits |
| 94 | |
| 95 | One mode is active at a time -- the plugin cannot decap VXLAN and SRv6 simultaneously. If your fleet mixes both, choose the one your L2-section-bearing exporters carry exclusively. |
| 96 | |
| 97 | #### Performance Impact |
| 98 | |
| 99 | Decapsulation runs in the flow hot path for records carrying L2 frame sections. It adds protocol parsing work and drops L2-section records that do not match the configured tunnel mode. |
| 100 | |
| 101 | ## Setup |
| 102 | |
| 103 | |
| 104 | ### Prerequisites |
| 105 | |
| 106 | #### Confirm your exporter ships the L2 frame section |
| 107 | |
| 108 | Decapsulation requires the exporter to include the inner frame bytes. Verify |
| 109 | before enabling the mode: |
| 110 | |
| 111 | - **NetFlow v9** -- the template must contain field type 104 |
| 112 | (`Layer2packetSectionData`). Capture a packet with `tcpdump`, decode with |
| 113 | Wireshark, and inspect the template. |
| 114 | - **IPFIX** -- the template must contain Information Element 315 |
| 115 | (`dataLinkFrameSection`, RFC 7133). Same verification path. |
| 116 | - **sFlow** -- header sampling is the default for sFlow agents and ships the |
| 117 | truncated raw packet inside `SampledHeader`. No special configuration is |
| 118 | required beyond enabling sFlow. |
| 119 | |
| 120 | Section length matters. VXLAN over Ethernet over IPv4 fits in roughly 96-128 |
| 121 | bytes of inner-frame capture; SRv6 with a Routing Header type 4 needs more |
| 122 | (256 bytes is a safe starting point). Truncated captures fail the inner |
| 123 | parser and the flow is dropped (see the failure modes on the concept page). |
| 124 | |
| 125 | |
| 126 | #### Configure your exporter to emit the L2 section |
| 127 | |
| 128 | Vendor support varies. Recommended exporter paths are: |
| 129 | |
| 130 | - **Juniper inline-monitoring (IPFIX 315)** on platforms supporting |
| 131 | `services { inline-monitoring { ... } }` -- the template includes |
| 132 | `datalink-frame-size` and a `maximum-clip-length` controls how much of the |
| 133 | frame is captured. Reference recipe in the |
| 134 | [Akvorado documentation](https://demo.akvorado.net/docs/operations#ipfix-315). |
| 135 | - **sFlow** with header sampling -- supported by Juniper QFX, Arista EOS, |
| 136 | Mellanox/NVIDIA, MikroTik, and others. Header sampling is the default for |
| 137 | most agents; verify the agent emits `SampledHeader` rather than only |
| 138 | `SampledIPv4` / `SampledIPv6` records (the latter do not carry inner |
| 139 | bytes). |
| 140 | |
| 141 | Cisco IOS-XE and IOS-XR Flexible NetFlow support for Layer-2 frame |
| 142 | sections is platform-dependent. Before deploying Cisco decapsulation, |
| 143 | inspect the exported template and look for IE 104 (v9) or IE 315 (IPFIX). |
| 144 | Do not copy `collect datalink frame-section` snippets into production |
| 145 | unless the platform template confirms that the L2 section is exported. |
| 146 | |
| 147 | |
| 148 | |
| 149 | ### Configuration |
| 150 | |
| 151 | #### Options |
| 152 | |
| 153 | Decapsulation has a single configuration knob -- `protocols.decapsulation_mode`. |
| 154 | |
| 155 | |
| 156 | <details open><summary>Config options</summary> |
| 157 | |
| 158 | |
| 159 | |
| 160 | | Option | Description | Default | Required | |
| 161 | |:-----|:------------|:--------|:---------:| |
| 162 | | protocols.decapsulation_mode | One of `none` (default), `srv6`, `vxlan`. The mode applies globally; the plugin cannot decode both VXLAN and SRv6 in the same instance. Setting this only affects records that travel through the L2-section path (NetFlow v9 IE 104 / IPFIX IE 315 / sFlow `SampledHeader`); regular flow records are unaffected. When the mode is set and the inner packet does not match the configured tunnel, the record is dropped -- there is no "fall back to outer view". | none | no | |
| 163 | |
| 164 | |
| 165 | </details> |
| 166 | |
| 167 | |
| 168 | |
| 169 | #### via File |
| 170 | |
| 171 | The configuration file name for this integration is `netflow.yaml`. |
| 172 | |
| 173 | |
| 174 | You can edit the configuration file using the [`edit-config`](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#edit-configuration-files) script from the |
| 175 | Netdata [config directory](https://github.com/netdata/netdata/blob/master/docs/netdata-agent/configuration/README.md#locate-your-config-directory). |
| 176 | |
| 177 | ```bash |
| 178 | cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata |
| 179 | sudo ./edit-config netflow.yaml |
| 180 | ``` |
| 181 | |
| 182 | ##### Examples |
| 183 | |
| 184 | ###### VXLAN-based VTEP fleet (decode the inner tenant traffic) |
| 185 | |
| 186 | Hypervisors emit sFlow with header sampling on the underlay. Every flow |
| 187 | between two VTEP loopbacks decodes to the inner VM-to-VM 5-tuple instead |
| 188 | of the underlay tunnel pair. |
| 189 | |
| 190 | |
| 191 | ```yaml |
| 192 | protocols: |
| 193 | decapsulation_mode: vxlan |
| 194 | |
| 195 | ``` |
| 196 | ###### SRv6 transit network (see the inner service traffic) |
| 197 | |
| 198 | Provider-edge routers export IPFIX with IE 315 (`dataLinkFrameSection`) |
| 199 | via Juniper inline-monitoring. The plugin walks the IPv6 extension chain, |
| 200 | strips the SRH, and surfaces the inner IPv4 or IPv6 5-tuple. |
| 201 | |
| 202 | |
| 203 | <details open><summary>Config</summary> |
| 204 | |
| 205 | ```yaml |
| 206 | protocols: |
| 207 | decapsulation_mode: srv6 |
| 208 | |
| 209 | ``` |
| 210 | </details> |
| 211 | |
| 212 | ###### Default (no decapsulation) |
| 213 | |
| 214 | Outer-header view only. Tunnel endpoints appear as the source and |
| 215 | destination of every flow. |
| 216 | |
| 217 | |
| 218 | <details open><summary>Config</summary> |
| 219 | |
| 220 | ```yaml |
| 221 | protocols: |
| 222 | decapsulation_mode: none |
| 223 | |
| 224 | ``` |
| 225 | </details> |
| 226 | |
| 227 | |
| 228 | |
| 229 | ### Decap mode set but tunnels still show outer endpoints |
| 230 | |
| 231 | The exporter is not shipping the L2 frame section. Plain NetFlow / IPFIX |
| 232 | flow records (no IE 104 / IE 315) take the regular path and are unaffected |
| 233 | by `decapsulation_mode`. Inspect the template -- look for field type 104 |
| 234 | on NetFlow v9 or IE 315 on IPFIX. For sFlow, confirm the agent is sending |
| 235 | `SampledHeader` records rather than only `SampledIPv4` / `SampledIPv6`. |
| 236 | |
| 237 | |
| 238 | ### Records disappear after enabling decap |
| 239 | |
| 240 | When `decapsulation_mode` is set and a record arrives via the L2-section |
| 241 | path with a payload that does not match the configured tunnel, the record |
| 242 | is **dropped**. There is no fall back to the outer view. For sFlow with decap on, only `SampledHeader` records are processed; |
| 243 | `SampledIPv4`, `SampledIPv6`, `SampledEthernet`, `ExtendedSwitch`, |
| 244 | `ExtendedRouter`, `ExtendedGateway` records are skipped. If the same |
| 245 | exporter mixes tunnel and non-tunnel traffic on the L2-section path, you |
| 246 | will lose the non-tunnel records. |
| 247 | |
| 248 | |
| 249 | ### VXLAN on a non-default UDP port goes undetected |
| 250 | |
| 251 | The VXLAN parser matches only UDP destination port 4789. VXLAN-GPE on 4790 and any |
| 252 | vendor-custom port are not recognised and the record is dropped under |
| 253 | `decapsulation_mode: vxlan`. |
| 254 | |
| 255 | |
| 256 | ### Frame section truncated, inner parsing fails |
| 257 | |
| 258 | The exporter's clip / section size is shorter than the outer headers plus |
| 259 | the inner L3/L4 needed to populate the 5-tuple. Increase the section size |
| 260 | -- 128 bytes for VXLAN over IPv4, 256 bytes or more for SRv6 with extension |
| 261 | headers. On Juniper inline-monitoring, the knob is `maximum-clip-length`. |
| 262 | |
| 263 | |
| 264 | ### VNI-based segmentation invisible |
| 265 | |
| 266 | Bytes 4-6 of the VXLAN header (the VNI) are not exposed as journal |
| 267 | fields. If the inner Ethernet carries a VLAN tag, that VLAN reaches |
| 268 | `SRC_VLAN` / `DST_VLAN` and works for segmentation -- pure VNI does not. |
| 269 | No workaround inside the plugin; either VLAN-tag the inner traffic or |
| 270 | filter at query time using the tunnel-endpoint pair before decap. |
| 271 | |
| 272 | |
| 273 | ### One mode at a time |
| 274 | |
| 275 | The plugin cannot decode VXLAN and SRv6 in the same instance. If exporter A |
| 276 | ships VXLAN tenant traffic and exporter B ships SRv6 transit traffic, you |
| 277 | must choose the mode that matches the traffic this Netdata Agent receives. |
| 278 | |
| 279 | |
| 280 |