Docs: explain CoPilot ↔ Shuffle integration (#733)
* docs: add docs.socfortress.co link to README * docs: add CoPilot-Shuffle integration page --------- Co-authored-by: Clawdbot <clawdbot@Clawdbots-Mac-mini.local>
taylorcopilot committed
Feb 20, 2026 at 12:54 UTC
0c7ed97c680c283479a2889eebede3e787835b01
2 files changed
+157
docs/user/shuffle-integration.md
new
+156
@@ -0,0 +1,156 @@
1
+# CoPilot ↔ Shuffle Integration (Admin/Operator)
2
+
3
+CoPilot integrates with **Shuffle** so you can attach **automation and external notifications** (Teams/Slack/Jira/Email/VT enrichment, etc.) to alerts and cases **without** SOCFortress having to ship a bespoke integration for every downstream tool.
4
+
5
+At a high level:
6
+
7
+- **Graylog** creates the *SIEM event/alert* (via Event Definitions)
8
+- **CoPilot** polls Graylog for *new, unprocessed* events and turns them into **CoPilot Alerts/Cases**
9
+- **CoPilot** sends a structured payload to **Shuffle**, which runs a workflow you choose
10
+
11
+---
12
+
13
+## Why we do it this way
14
+
15
+### 1) Scale integrations without bloating CoPilot
16
+CoPilot is the “hub” for the stack, but the long tail of integrations (Teams, Jira, PagerDuty, ServiceNow, custom webhooks, etc.) is endless.
17
+
18
+Shuffle is purpose-built for this:
19
+
20
+- hundreds of apps/connectors
21
+- easy HTTP/webhook support for anything else
22
+- workflows/playbooks can be owned by the operator team (not necessarily product engineering)
23
+
24
+### 2) Keep automation close to your environment (minimize exposure)
25
+Shuffle supports an architecture where the **UI/orchestration can be cloud-hosted** while execution happens via an **on‑prem “location/worker”** inside your network.
26
+
27
+That’s valuable for SIEM stacks because you often want to:
28
+
29
+- reach internal services (Wazuh Manager, Indexer/OpenSearch, AD, internal APIs)
30
+- avoid poking inbound firewall holes
31
+- keep credentials and execution inside your boundary
32
+
33
+### 3) Strong separation of concerns
34
+- **Graylog**: detection logic (queries, thresholds, correlation)
35
+- **CoPilot**: multi-tenant alert/case lifecycle + normalization + UI + API
36
+- **Shuffle**: automation/orchestration + outbound integrations
37
+
38
+---
39
+
40
+## Data flow (what triggers what)
41
+
42
+### Alerts (automatic)
43
+1. A Graylog **Event Definition** triggers.
44
+2. Graylog writes an event into its events index (e.g., `gl-events_*`).
45
+3. CoPilot’s scheduler job **polls** for new events where `copilot_alert_id = none`.
46
+4. CoPilot creates a **CoPilot Alert** and then updates the Graylog event so it won’t be reprocessed.
47
+5. If a Shuffle notification workflow is configured for that customer, CoPilot invokes it and includes:
48
+ - alert/case metadata
49
+ - customer code
50
+ - asset identifier
51
+ - **alert context** (see below)
52
+
53
+### Cases (manual trigger)
54
+Cases can be triggered into Shuffle **manually** from the CoPilot UI. This is intentional so case automation isn’t accidentally run without an analyst decision.
55
+
56
+---
57
+
58
+## What CoPilot sends to Shuffle (payload model)
59
+
60
+CoPilot sends a JSON payload that includes:
61
+
62
+- **Alert ID / Case ID** (CoPilot identifiers)
63
+- **Customer code** (multi-tenant routing)
64
+- **Asset name** (e.g., Wazuh agent name, O365 mailbox/user, etc.)
65
+- **Alert Context**: a small, curated set of fields you choose per Source
66
+
67
+### Alert Context comes from *Sources*
68
+The most important concept for operators/admins:
69
+
70
+> **The fields that appear in Shuffle’s `alert_context` are defined by the CoPilot Source configuration.**
71
+
72
+That’s why CoPilot can support wildly different log types:
73
+
74
+- Wazuh endpoint events → asset might be `agent.name`
75
+- O365 events → asset might be `o365.recipients` or `user_id`
76
+
77
+You choose which fields are “headline context” for that Source, and those same fields become the default “automation payload” for Shuffle.
78
+
79
+---
80
+
81
+## Setup: enabling CoPilot → Shuffle
82
+
83
+### Step 1 — Get a Shuffle API key
84
+In Shuffle:
85
+
86
+1. Go to **User Account / Settings**
87
+2. Copy the **API key**
88
+
89
+### Step 2 — Add the Shuffle API key to CoPilot
90
+In CoPilot:
91
+
92
+1. Navigate to **Connectors**
93
+2. Locate **Shuffle**
94
+3. Paste the API key and **verify**
95
+
96
+### Step 3 — Create a Shuffle workflow to receive CoPilot notifications
97
+Create a workflow in Shuffle (example: `SIEM Alert — <customer>`). The workflow will receive the CoPilot payload as input.
98
+
99
+### Step 4 — Set the notification workflow per customer
100
+In CoPilot:
101
+
102
+1. Open the **Customer**
103
+2. Go to **Notification Workflow**
104
+3. Create/configure the notification and set the **Shuffle Workflow ID**
105
+
106
+**Result:**
107
+- New alerts for that customer can invoke the workflow automatically
108
+- Cases can invoke the workflow when manually triggered
109
+
110
+---
111
+
112
+## How this ties back to Graylog + Indexer/OpenSearch
113
+
114
+A quick mental model (this matters when debugging):
115
+
116
+- **Indexer/OpenSearch (Wazuh Indexer)** stores the actual event data.
117
+- **Graylog** manages streams/index sets and runs event definitions (alerting) by querying the indexer.
118
+- **CoPilot** reads Graylog’s generated events and then enriches/normalizes them into CoPilot alerts.
119
+
120
+So when you want to automate enrichment in Shuffle (e.g., look up related events, run searches, pull agent state):
121
+
122
+- Query **Indexer/OpenSearch** for raw events/search
123
+- Use **Wazuh Manager API** for agent lifecycle state
124
+- Use **CoPilot API** when you want CoPilot’s normalized view (alerts/cases/customers)
125
+
126
+---
127
+
128
+## Practical examples (common workflows)
129
+
130
+- Post to **Microsoft Teams** when a High severity alert arrives
131
+- Enrich hashes/domains with **VirusTotal** before an analyst sees the alert
132
+- Create a **Jira** ticket with alert context and links back to CoPilot
133
+- Invoke an internal **FastAPI microservice** (e.g., disable AD user) with an approval step
134
+
135
+---
136
+
137
+## Troubleshooting checklist
138
+
139
+1. **Is the Graylog Event Definition firing?**
140
+ - Confirm the event exists in `gl-events_*`.
141
+2. **Is CoPilot creating the alert?**
142
+ - If `copilot_alert_id` remains `none`, CoPilot likely can’t map the event to a Source.
143
+3. **Is the Source configured correctly?**
144
+ - Ensure the Source matches the event’s `syslog_type` and correct field mapping.
145
+4. **Is the Shuffle connector verified in CoPilot?**
146
+5. **Is the customer’s Shuffle Workflow ID set?**
147
+6. **Check Shuffle runs** for the workflow.
148
+
149
+---
150
+
151
+## Related docs
152
+
153
+- [Incident Sources](ui/incident-sources.md)
154
+- [Alerts (SIEM)](ui/alerts-siem.md)
155
+- [Connectors](ui/connectors.md)
156
+- [External Services](ui/external-services.md)
mkdocs.yml
+1
@@ -77,6 +77,7 @@ nav:
77
- Quickstart (Admins/Engineers): user/admins-quickstart.md
78
- Customer Provisioning (Tenancy): user/customer-provisioning.md
79
- Features by Area: user/features.md
80
+ - CoPilot ↔ Shuffle Integration (Admin/Operator): user/shuffle-integration.md
81
- Navigation Guide (UI): user/navigation.md
82
- UI Guide (mirrors menu):
83
- Start here: user/ui/README.md