| 1 | # CoPilot ↔ Shuffle Integration (Admin/Operator) |
| 2 | |
| 3 | CoPilot integrates with **Shuffle** so you can attach **automation and external notifications** (Teams/Slack/Jira/Email/VT enrichment, etc.) to alerts and cases **without** SOCFortress having to ship a bespoke integration for every downstream tool. |
| 4 | |
| 5 | At a high level: |
| 6 | |
| 7 | - **Graylog** creates the *SIEM event/alert* (via Event Definitions) |
| 8 | - **CoPilot** polls Graylog for *new, unprocessed* events and turns them into **CoPilot Alerts/Cases** |
| 9 | - **CoPilot** sends a structured payload to **Shuffle**, which runs a workflow you choose |
| 10 | |
| 11 | --- |
| 12 | |
| 13 | ## Why we do it this way |
| 14 | |
| 15 | ### 1) Scale integrations without bloating CoPilot |
| 16 | CoPilot is the “hub” for the stack, but the long tail of integrations (Teams, Jira, PagerDuty, ServiceNow, custom webhooks, etc.) is endless. |
| 17 | |
| 18 | Shuffle is purpose-built for this: |
| 19 | |
| 20 | - hundreds of apps/connectors |
| 21 | - easy HTTP/webhook support for anything else |
| 22 | - workflows/playbooks can be owned by the operator team (not necessarily product engineering) |
| 23 | |
| 24 | ### 2) Keep automation close to your environment (minimize exposure) |
| 25 | Shuffle supports an architecture where the **UI/orchestration can be cloud-hosted** while execution happens via an **on‑prem “location/worker”** inside your network. |
| 26 | |
| 27 | That’s valuable for SIEM stacks because you often want to: |
| 28 | |
| 29 | - reach internal services (Wazuh Manager, Indexer/OpenSearch, AD, internal APIs) |
| 30 | - avoid poking inbound firewall holes |
| 31 | - keep credentials and execution inside your boundary |
| 32 | |
| 33 | ### 3) Strong separation of concerns |
| 34 | - **Graylog**: detection logic (queries, thresholds, correlation) |
| 35 | - **CoPilot**: multi-tenant alert/case lifecycle + normalization + UI + API |
| 36 | - **Shuffle**: automation/orchestration + outbound integrations |
| 37 | |
| 38 | --- |
| 39 | |
| 40 | ## Data flow (what triggers what) |
| 41 | |
| 42 | ### Alerts (automatic) |
| 43 | 1. A Graylog **Event Definition** triggers. |
| 44 | 2. Graylog writes an event into its events index (e.g., `gl-events_*`). |
| 45 | 3. CoPilot’s scheduler job **polls** for new events where `copilot_alert_id = none`. |
| 46 | 4. CoPilot creates a **CoPilot Alert** and then updates the Graylog event so it won’t be reprocessed. |
| 47 | 5. If a Shuffle notification workflow is configured for that customer, CoPilot invokes it and includes: |
| 48 | - alert/case metadata |
| 49 | - customer code |
| 50 | - asset identifier |
| 51 | - **alert context** (see below) |
| 52 | |
| 53 | ### Cases (manual trigger) |
| 54 | Cases can be triggered into Shuffle **manually** from the CoPilot UI. This is intentional so case automation isn’t accidentally run without an analyst decision. |
| 55 | |
| 56 | --- |
| 57 | |
| 58 | ## What CoPilot sends to Shuffle (payload model) |
| 59 | |
| 60 | CoPilot sends a JSON payload that includes: |
| 61 | |
| 62 | - **Alert ID / Case ID** (CoPilot identifiers) |
| 63 | - **Customer code** (multi-tenant routing) |
| 64 | - **Asset name** (e.g., Wazuh agent name, O365 mailbox/user, etc.) |
| 65 | - **Alert Context**: a small, curated set of fields you choose per Source |
| 66 | |
| 67 | ### Alert Context comes from *Sources* |
| 68 | The most important concept for operators/admins: |
| 69 | |
| 70 | > **The fields that appear in Shuffle’s `alert_context` are defined by the CoPilot Source configuration.** |
| 71 | |
| 72 | That’s why CoPilot can support wildly different log types: |
| 73 | |
| 74 | - Wazuh endpoint events → asset might be `agent_name` |
| 75 | - O365 events → asset might be `data_o365_recipients` or `user_id` |
| 76 | |
| 77 | You choose which fields are “headline context” for that Source, and those same fields become the default “automation payload” for Shuffle. |
| 78 | |
| 79 | --- |
| 80 | |
| 81 | ## Setup: enabling CoPilot → Shuffle |
| 82 | |
| 83 | ### Step 1 — Get a Shuffle API key |
| 84 | In Shuffle: |
| 85 | |
| 86 | 1. Go to **User Account / Settings** |
| 87 | 2. Copy the **API key** |
| 88 | |
| 89 | ### Step 2 — Add the Shuffle API key to CoPilot |
| 90 | In CoPilot: |
| 91 | |
| 92 | 1. Navigate to **Connectors** |
| 93 | 2. Locate **Shuffle** |
| 94 | 3. Paste the API key and **verify** |
| 95 | |
| 96 | ### Step 3 — Create a Shuffle workflow to receive CoPilot notifications |
| 97 | Create a workflow in Shuffle (example: `SIEM Alert — <customer>`). The workflow will receive the CoPilot payload as input. |
| 98 | |
| 99 | ### Step 4 — Set the notification workflow per customer |
| 100 | In CoPilot: |
| 101 | |
| 102 | 1. Open the **Customer** |
| 103 | 2. Go to **Notification Workflow** |
| 104 | 3. Create/configure the notification and set the **Shuffle Workflow ID** |
| 105 | |
| 106 | **Result:** |
| 107 | - New alerts for that customer can invoke the workflow automatically |
| 108 | - Cases can invoke the workflow when manually triggered |
| 109 | |
| 110 | --- |
| 111 | |
| 112 | ## How this ties back to Graylog + Indexer/OpenSearch |
| 113 | |
| 114 | A quick mental model (this matters when debugging): |
| 115 | |
| 116 | - **Indexer/OpenSearch (Wazuh Indexer)** stores the actual event data. |
| 117 | - **Graylog** manages streams/index sets and runs event definitions (alerting) by querying the indexer. |
| 118 | - **CoPilot** reads Graylog’s generated events and then enriches/normalizes them into CoPilot alerts. |
| 119 | |
| 120 | So when you want to automate enrichment in Shuffle (e.g., look up related events, run searches, pull agent state): |
| 121 | |
| 122 | - Query **Indexer/OpenSearch** for raw events/search |
| 123 | - Use **Wazuh Manager API** for agent lifecycle state |
| 124 | - Use **CoPilot API** when you want CoPilot’s normalized view (alerts/cases/customers) |
| 125 | |
| 126 | --- |
| 127 | |
| 128 | ## Practical examples (common workflows) |
| 129 | |
| 130 | - Post to **Microsoft Teams** when a High severity alert arrives |
| 131 | - Enrich hashes/domains with **VirusTotal** before an analyst sees the alert |
| 132 | - Create a **Jira** ticket with alert context and links back to CoPilot |
| 133 | - Invoke an internal **FastAPI microservice** (e.g., disable AD user) with an approval step |
| 134 | |
| 135 | --- |
| 136 | |
| 137 | ## Troubleshooting checklist |
| 138 | |
| 139 | 1. **Is the Graylog Event Definition firing?** |
| 140 | - Confirm the event exists in `gl-events_*`. |
| 141 | 2. **Is CoPilot creating the alert?** |
| 142 | - If `copilot_alert_id` remains `none`, CoPilot likely can’t map the event to a Source. |
| 143 | 3. **Is the Source configured correctly?** |
| 144 | - Ensure the Source matches the event’s `syslog_type` and correct field mapping. |
| 145 | 4. **Is the Shuffle connector verified in CoPilot?** |
| 146 | 5. **Is the customer’s Shuffle Workflow ID set?** |
| 147 | 6. **Check Shuffle runs** for the workflow. |
| 148 | |
| 149 | --- |
| 150 | |
| 151 | ## Related docs |
| 152 | |
| 153 | - [Incident Sources](ui/incident-sources.md) |
| 154 | - [Alerts (SIEM)](ui/alerts-siem.md) |
| 155 | - [Connectors](ui/connectors.md) |
| 156 | - [External Services](ui/external-services.md) |