| 1 | --- |
| 2 | title: AI Analyst — Notification routing |
| 3 | description: Send Talon's investigation reports to the customer's Slack, Outlook, Teams, or any of Shuffle's 3,000+ apps based on per-customer routing rules. |
| 4 | --- |
| 5 | |
| 6 | Every Talon investigation produces a report. The notification routing |
| 7 | feature sends those reports out — to a Slack channel, an email |
| 8 | distribution list, a Teams card, a ServiceNow ticket, or any of |
| 9 | Shuffle's 3,000+ authenticated apps — based on rules a SOC admin |
| 10 | configures per customer. |
| 11 | |
| 12 | This page is for **operators** (SOC admins setting up notifications). |
| 13 | For the architecture and dispatch internals, see |
| 14 | [AI Analyst (Talon)](/power-features/ai-analyst). |
| 15 | |
| 16 | --- |
| 17 | |
| 18 | ## Why route notifications |
| 19 | |
| 20 | Without routing, investigation reports live in CoPilot's UI and |
| 21 | nowhere else. Routing pushes the result to wherever the SOC team |
| 22 | already operates: |
| 23 | |
| 24 | - A Slack channel where the on-call sees high-severity findings |
| 25 | - An email distribution list for after-hours escalation |
| 26 | - A Teams card for the IR team's daily standup feed |
| 27 | - A ServiceNow ticket auto-created on Critical alerts |
| 28 | - Anything else Shuffle's catalog can reach |
| 29 | |
| 30 | CoPilot owns the routing logic (which destinations to fan out to, |
| 31 | filtered by severity); Shuffle handles the actual delivery to each |
| 32 | app's API. |
| 33 | |
| 34 | --- |
| 35 | |
| 36 | ## Where it lives in the UI |
| 37 | |
| 38 | Per-customer config lives under **Customers → [your customer] → |
| 39 | Details → AI Notifications**. Three sub-tabs: |
| 40 | |
| 41 | | Sub-tab | Purpose | |
| 42 | |---------|---------| |
| 43 | | **Routes** | The actual notification rules — what severity threshold, which Shuffle integration, which app, where to send | |
| 44 | | **Shuffle integrations** | Per-customer Shuffle org configurations + the "Manage apps" drawer for authenticating new Shuffle apps | |
| 45 | | **Dispatch log** | Read-only audit of every notification attempt — `sent`, `failed`, `skipped`, latency, error messages | |
| 46 | |
| 47 | --- |
| 48 | |
| 49 | ## One-time prerequisite: deployment Shuffle connector |
| 50 | |
| 51 | Before any customer can use notifications, your CoPilot deployment |
| 52 | needs the Shuffle connector configured. This is admin-tier infra |
| 53 | that's set up once for the whole platform, not per-customer. |
| 54 | |
| 55 | 1. **Connectors → Shuffle** (or via SQL: `SELECT * FROM connectors |
| 56 | WHERE connector_name='Shuffle'`) |
| 57 | 2. Set: |
| 58 | - `connector_url` — your Shuffle region (e.g. `https://shuffler.io`, |
| 59 | `https://california.shuffler.io`) |
| 60 | - `connector_api_key` — admin Bearer token from your Shuffle account |
| 61 | (Shuffle profile → API key, or |
| 62 | `POST /api/v1/users/generateapikey`) |
| 63 | - `connector_extra_data` — your parent Org-Id (optional but helps |
| 64 | in some flows) |
| 65 | - `connector_enabled` = 1, `connector_verified` = 1 |
| 66 | |
| 67 | CoPilot's notification engine reads the URL + API key fresh on every |
| 68 | dispatch, so a key rotation takes effect without restarting the |
| 69 | backend. |
| 70 | |
| 71 | --- |
| 72 | |
| 73 | ## Step-by-step setup for one customer |
| 74 | |
| 75 | End-to-end walkthrough — first time setting up notifications for a |
| 76 | customer that's not yet configured. |
| 77 | |
| 78 | ### Step 1 — Add a Shuffle integration |
| 79 | |
| 80 | The integration tells CoPilot **which Shuffle org** belongs to this |
| 81 | customer. Each customer gets one or more integration rows. |
| 82 | |
| 83 | 1. Open the customer's **AI Notifications** tab |
| 84 | 2. Click **Shuffle integrations** |
| 85 | 3. Click **Add Shuffle integration** |
| 86 | 4. Fill out: |
| 87 | - **Display name** — human label, e.g. `Acme Production Shuffle` |
| 88 | - **Shuffle org** — dropdown populated from your deployment's |
| 89 | Shuffle connector. Picks the org for this customer (top-level or |
| 90 | sub-org). Sub-orgs show a `· sub-org` hint |
| 91 | - **Enabled** — leave checked |
| 92 | 5. Click **Add integration** |
| 93 | |
| 94 | After creating, click the **checkmark** icon on the new row to |
| 95 | **Test connection** — should turn green with the count of authenticated |
| 96 | apps in that org. |
| 97 | |
| 98 | > If the org dropdown is empty or fails to load, the Shuffle connector |
| 99 | > isn't configured correctly. See [troubleshooting](#troubleshooting). |
| 100 | |
| 101 | ### Step 2 — Authenticate the apps you want to use |
| 102 | |
| 103 | Each Shuffle org needs the apps you plan to notify through to be |
| 104 | **authenticated** within Shuffle. CoPilot doesn't do the authentication |
| 105 | itself — Shuffle owns that flow. You can authenticate apps either: |
| 106 | |
| 107 | - **Inside Shuffle's UI directly** (shuffler.io → Apps → connect Slack/ |
| 108 | Outlook/etc.) |
| 109 | - **From CoPilot's "Manage apps" drawer** — clicks through to |
| 110 | Shuffle's OAuth dance |
| 111 | |
| 112 | To use the drawer: |
| 113 | |
| 114 | 1. **Shuffle integrations** sub-tab → click the **catalog icon** on |
| 115 | the integration row |
| 116 | 2. The drawer opens with Shuffle's app picker scoped to this |
| 117 | customer's org |
| 118 | 3. Search for the app you want (`Slack`, `Outlook`, `Teams`, `Gmail`, |
| 119 | `ServiceNow`, etc.) |
| 120 | 4. Click the app — Shuffle opens an OAuth window in a new tab |
| 121 | 5. Authenticate in Shuffle, return to CoPilot |
| 122 | 6. The new app is now available for routes that target this org |
| 123 | |
| 124 | > Apps authenticated inside Shuffle directly will also appear here — |
| 125 | > the drawer is a convenience, not a requirement. |
| 126 | |
| 127 | ### Step 3 — Create a notification route |
| 128 | |
| 129 | The route is the actual rule that says "when an AI investigation |
| 130 | matches X, send to Y." |
| 131 | |
| 132 | 1. Open **Routes** sub-tab → **Add route** |
| 133 | 2. Fill out: |
| 134 | - **Name** — human label, e.g. `SOC Slack #alerts` |
| 135 | - **Minimum severity** — only investigations at this tier or |
| 136 | higher fire this route. Pick from `Critical (only)`, `High and |
| 137 | above`, `Medium and above`, `Low and above`, `Informational and |
| 138 | above (everything)` |
| 139 | - **Channel** — locked to **Shuffle** |
| 140 | - **Shuffle integration** — pick the integration you created in |
| 141 | Step 1 |
| 142 | - **Shuffle app** — pick from the dropdown of apps authenticated |
| 143 | in this org (Step 2) |
| 144 | - **Destination hint** — free-form text that's prepended to the |
| 145 | outgoing message as a `Send to <hint>: …` instruction so |
| 146 | Shuffle's app agent knows where to deliver inside that app: |
| 147 | - For Slack: `#soc-alerts` or `#general` |
| 148 | - For Outlook / Gmail: `soc@example.com` |
| 149 | - For Teams: a channel name or webhook target |
| 150 | - For ServiceNow: ticket queue or assignee |
| 151 | - **Custom message template** — leave empty for the default body. |
| 152 | If you set one, include `{{summary}}` somewhere or the AI's |
| 153 | finding will be dropped from the message |
| 154 | - **Enabled** — leave checked |
| 155 | 3. Click **Create route** |
| 156 | |
| 157 | ### Step 4 — Test by triggering an investigation |
| 158 | |
| 159 | The notification fires automatically every time Talon completes an |
| 160 | investigation that matches the route's filter. To test without |
| 161 | waiting for a real alert: |
| 162 | |
| 163 | 1. Pick any alert in **Incident Management → Alerts** for this |
| 164 | customer |
| 165 | 2. Click the **AI Analyst** tab on the alert |
| 166 | 3. Click **Investigate with AI Analyst** if no report exists |
| 167 | 4. Wait ~30–90 seconds for Talon to finish |
| 168 | 5. Check the route's destination — Slack channel, mailbox, etc. |
| 169 | |
| 170 | You can also fire a dispatch manually for any alert via: |
| 171 | |
| 172 | ```bash |
| 173 | curl -X POST "${COPILOT_URL}/api/notifications/dispatch" \ |
| 174 | -H "Authorization: Bearer ${COPILOT_API_KEY}" \ |
| 175 | -H "Content-Type: application/json" \ |
| 176 | -d '{ |
| 177 | "customer_code": "00001", |
| 178 | "alert_id": 147, |
| 179 | "trigger": "investigation_complete", |
| 180 | "severity_assessment": "Critical", |
| 181 | "summary": "Test notification — ignore", |
| 182 | "alert_name": "Manual smoke test" |
| 183 | }' |
| 184 | ``` |
| 185 | |
| 186 | The response shows `routes_matched`, `dispatched`, `failed`, and |
| 187 | per-route outcomes including the Shuffle `execution_id` for forensics. |
| 188 | |
| 189 | ### Step 5 — Verify in the dispatch log |
| 190 | |
| 191 | **Dispatch log** sub-tab shows every notification attempt for this |
| 192 | customer: |
| 193 | |
| 194 | - **When** — timestamp |
| 195 | - **Alert** — alert ID |
| 196 | - **Trigger** — the event type (`investigation_complete`) |
| 197 | - **Status** — `sent` (success), `failed` (provider error), or |
| 198 | `skipped` (idempotency hit — same alert already dispatched for the |
| 199 | same route+trigger) |
| 200 | - **Latency** — provider call duration |
| 201 | - **Error / Preview** — error message on failures, or the first 500 |
| 202 | chars of the rendered body |
| 203 | |
| 204 | Click any failed row to see why; cross-reference the |
| 205 | `shuffle_execution_id` (if present) in Shuffle's UI to see what |
| 206 | happened on Shuffle's side. |
| 207 | |
| 208 | --- |
| 209 | |
| 210 | ## End-to-end example: Slack #soc-alerts for High+ findings |
| 211 | |
| 212 | Putting it all together for customer code `acme`: |
| 213 | |
| 214 | 1. **Shuffle prerequisite:** Slack workspace authenticated in Acme's |
| 215 | Shuffle org as the `Slack` app (done either in Shuffle's UI or |
| 216 | via Step 2 above) |
| 217 | 2. **Add Shuffle integration:** |
| 218 | - Display name: `Acme Shuffle` |
| 219 | - Shuffle org: `Acme Corp (3a8c…)` (picked from dropdown) |
| 220 | 3. **Add notification route:** |
| 221 | - Name: `SOC Slack #soc-alerts` |
| 222 | - Minimum severity: `High and above` |
| 223 | - Shuffle integration: `Acme Shuffle` |
| 224 | - Shuffle app: `Slack` |
| 225 | - Destination hint: `#soc-alerts` |
| 226 | - Custom template: empty |
| 227 | 4. **Trigger test investigation** on a High-severity alert |
| 228 | 5. **Dispatch log** shows `sent` row with ~300 ms latency, |
| 229 | `shuffle_execution_id=exec-…` |
| 230 | 6. **Slack #soc-alerts** receives a message with the alert summary |
| 231 | and severity |
| 232 | |
| 233 | Repeat steps 2–4 for additional destinations (e.g. an Outlook email |
| 234 | distribution list for Critical-only, a Teams card for the daily |
| 235 | standup feed). |
| 236 | |
| 237 | --- |
| 238 | |
| 239 | ## Common patterns |
| 240 | |
| 241 | | Pattern | How to configure | |
| 242 | |---------|------------------| |
| 243 | | **One channel, all severities** | Single route, `min_severity = Informational` | |
| 244 | | **High-tier paging only** | Single route, `min_severity = High` (fires on High and Critical) | |
| 245 | | **Critical to one channel, everything else to another** | Two routes — Critical-only Slack DM to on-call, separate `Informational+` route to a noisy `#all-investigations` channel | |
| 246 | | **Multiple destinations for the same severity** | Multiple routes with the same `min_severity`, different `shuffle_app` / `destination_hint`. Each fires independently | |
| 247 | | **Customer with no notifications** | No routes configured. Investigations still write back to CoPilot's DB; nothing fans out | |
| 248 | |
| 249 | --- |
| 250 | |
| 251 | ## How filtering actually works |
| 252 | |
| 253 | A route fires when **all** of these are true: |
| 254 | |
| 255 | 1. Route is `enabled` |
| 256 | 2. Route's `trigger` matches the dispatch's event type |
| 257 | (currently always `investigation_complete`) |
| 258 | 3. Investigation's severity is at-or-above the route's |
| 259 | `min_severity` (`Informational < Low < Medium < High < Critical`) |
| 260 | |
| 261 | If any check fails, the route is skipped (no row in the dispatch log |
| 262 | for skipped-by-filter — those only show idempotency hits). |
| 263 | |
| 264 | **Idempotency:** the same investigation cannot fire the same route |
| 265 | twice. The dispatch log has a unique constraint on `(customer_code, |
| 266 | alert_id, route_id, trigger)`. Re-running an investigation re-fires |
| 267 | the route only after the initial dispatch failed (the `failed` row |
| 268 | gets overwritten on retry; `sent` rows are immutable). |
| 269 | |
| 270 | --- |
| 271 | |
| 272 | ## Custom message templates |
| 273 | |
| 274 | By default the notification body is a simple AI-generated summary |
| 275 | plus alert metadata. If you need a different format — branded |
| 276 | language, specific recipient phrasing, internal ticket numbers — set |
| 277 | a custom **format_template** on the route. |
| 278 | |
| 279 | Templates support these substitution tokens: |
| 280 | |
| 281 | | Token | Replaced with | |
| 282 | |-------|---------------| |
| 283 | | `{{customer_code}}` | The customer's code | |
| 284 | | `{{alert_id}}` | The integer alert ID | |
| 285 | | `{{alert_name}}` | The original alert title | |
| 286 | | `{{severity}}` | `Critical` / `High` / `Medium` / `Low` / `Informational` | |
| 287 | | `{{summary}}` | The AI investigation summary | |
| 288 | | `{{report_url}}` | Deep link to the report in CoPilot | |
| 289 | |
| 290 | Example for a Slack-style banner: |
| 291 | |
| 292 | ``` |
| 293 | :rotating_light: *{{severity}}* finding on alert #{{alert_id}} |
| 294 | |
| 295 | {{summary}} |
| 296 | |
| 297 | <{{report_url}}|Open in CoPilot> |
| 298 | ``` |
| 299 | |
| 300 | > **Watch out:** if your template doesn't reference `{{summary}}`, |
| 301 | > the actual investigation finding gets dropped. The route fires but |
| 302 | > recipients see only what's in the template literal. Always include |
| 303 | > `{{summary}}` somewhere unless you genuinely want a content-free |
| 304 | > teaser. |
| 305 | |
| 306 | --- |
| 307 | |
| 308 | ## Troubleshooting |
| 309 | |
| 310 | | Symptom | Likely cause | Fix | |
| 311 | |---------|--------------|-----| |
| 312 | | Org dropdown is empty when adding a Shuffle integration | Deployment's Shuffle connector misconfigured or unreachable | Check `Connectors → Shuffle`. Confirm `connector_url`, `connector_api_key`, `connector_enabled=1`. Try `curl ${SHUFFLE_URL}/api/v1/orgs -H "Authorization: Bearer ${KEY}"` from the CoPilot host | |
| 313 | | "Test connection" stays neutral / turns red | Org-Id is invalid or the API key doesn't have access to that org | Verify the Org-Id in Shuffle's UI. Confirm your API key is admin-scoped or has access to that org | |
| 314 | | Route saves but dispatch log says `routes_matched: 0` | Route's `min_severity` is higher than the alert's actual severity | Edit the route, lower `min_severity`. Or trigger an investigation on a higher-severity alert | |
| 315 | | Dispatch log shows `status=failed` with `Shuffle returned 4xx` | The Shuffle app isn't authenticated in the customer's org, or the destination is wrong | Open Shuffle UI → Apps → check the app is authenticated. For unrecognized destinations, check Shuffle's app-specific docs (e.g. Slack channel must include `#`) | |
| 316 | | Dispatch log shows `status=failed` with `ReadTimeout` | Shuffle backend was slow or temporarily unreachable | Re-trigger the investigation. The dispatch loop overwrites the `failed` row on retry | |
| 317 | | Recipient gets a notification with the wrong content | A custom `format_template` is set on the route and doesn't include `{{summary}}` | Edit the route, either add `{{summary}}` to the template or clear the template field entirely | |
| 318 | | Investigation completes but nothing in the dispatch log | Talon didn't call `DispatchNotificationsTool`. Either no routes configured or Talon's `notifications.md` not deployed | Check Talon container logs for the `tool_use: DispatchNotificationsTool` line. Confirm at least one route exists for the customer | |
| 319 | |
| 320 | --- |
| 321 | |
| 322 | ## Safety & guardrails |
| 323 | |
| 324 | - **Per-customer scope** — routes attached to customer A's integrations |
| 325 | only fire on customer A's alerts. CoPilot's dispatcher refuses to |
| 326 | cross tenant boundaries even if a route is hand-edited to point at |
| 327 | another customer's integration. |
| 328 | - **Best-effort delivery** — a notification dispatch failure never |
| 329 | fails the underlying investigation. Talon's report is written |
| 330 | before the dispatch attempt; failures are logged but don't |
| 331 | propagate. |
| 332 | - **No automatic retries** — a `failed` dispatch waits for the next |
| 333 | Talon run on the same alert (or a manual re-trigger) to overwrite |
| 334 | the log row. Phase 4 may add automatic retry semantics for |
| 335 | transient failures. |
| 336 | - **Admin-tier auth** — only users with `admin` or `analyst` scope |
| 337 | can read or modify routes. Operator-tier users see investigation |
| 338 | reports but not the routing config. |
| 339 | - **Org auth tokens visible to admins** — the Manage apps drawer |
| 340 | fetches a per-org auth token from Shuffle so the embedded picker |
| 341 | can show authenticated apps. This token is exposed to the admin's |
| 342 | browser session for the duration of the drawer being open. Don't |
| 343 | share screenshots of the drawer with non-admins. |
| 344 | |
| 345 | --- |
| 346 | |
| 347 | ## Related |
| 348 | |
| 349 | - [AI Analyst (Talon)](/power-features/ai-analyst) — the investigation |
| 350 | engine that produces the reports being routed |
| 351 | - [AI Analyst — Analyst review workflow](/power-features/ai-analyst-review) |
| 352 | — review and grade the reports after they're produced |