main
mdx 248 lines 12.2 KB
Raw
1 ---
2 title: AI Analyst — Analyst review workflow
3 description: Review and grade every AI investigation, correct IOC verdicts, teach the agent with durable or one-off lessons, replay with a different template, and track feedback trends over time.
4 ---
5
6 Every report Talon produces is a **draft**. The review workflow lets a SOC analyst grade it, correct what's wrong, teach the agent with a lesson, and — if needed — replay the investigation with a different template. Feedback is aggregated per customer so you can see which templates are reliable and which need tuning.
7
8 This page is for **operators** (analysts reviewing reports). For the architecture and deployment guide, see [AI Analyst (Talon)](/power-features/ai-analyst).
9
10 ---
11
12 ## Why review matters
13
14 AI reports are fast, consistent, and cheap — but they're not infallible. Without a feedback loop you can't tell:
15
16 - Whether the agent picked the right investigation template
17 - Whether IOC verdicts match reality (was that hash really malicious?)
18 - Whether the severity call was appropriate for your environment
19 - What recurring patterns the agent should treat as benign (and stop paging you about)
20
21 The review workflow turns every investigation into a training signal. Lessons you capture land in **MemPalace** — the agent's persistent memory — and are surfaced on the next investigation for that customer.
22
23 ---
24
25 ## Where it lives in the UI
26
27 Reviews and feedback live across two places:
28
29 | Location | Purpose |
30 |----------|---------|
31 | **Incident Management → Alert → AI Analyst tab → Review** | Grade a specific report, correct IOCs, queue a lesson, replay |
32 | **AI Analyst page → Reports** | Browse all reports, jump into any to review |
33 | **AI Analyst page → Feedback** | Per-customer rollup: thumbs, ratings, template accuracy, IOC accuracy, recent reviews |
34
35 ---
36
37 ## Reviewing a report
38
39 ### Open the report
40
41 1. **Incident Management → Alerts** → open any alert that has an AI investigation
42 2. Click the **AI Analyst** tab (pulses if a report exists)
43 3. Inside that tab, click **Review**
44
45 You'll see a rubric if this is the first review, or your previous grades pre-filled if you've reviewed this report before — submitting again updates the existing review (one review per analyst per report).
46
47 ### The rubric
48
49 | Field | What it captures |
50 |-------|------------------|
51 | **Overall verdict** | Thumbs up / down — fast signal, shows up in the dashboard |
52 | **Template choice** | `correct` / `partial` / `wrong` — was the right investigation template picked? |
53 | **Rating: instructions** | 1–5 — did the agent follow the template's instructions? |
54 | **Rating: artifacts** | 1–5 — did it find and cite the right evidence from SIEM? |
55 | **Rating: severity** | 1–5 — did the severity assessment match reality? |
56 | **Missing steps** | Free-text — what should the agent have done but didn't? |
57 | **Suggested edits** | Free-text — specific rewrites or additions for the report |
58
59 Leave any axis blank if you don't have a confident opinion — averages ignore nulls.
60
61 ### IOC corrections
62
63 Below the rubric you'll see the IOCs the agent extracted, each with its VirusTotal verdict. For every IOC you can mark:
64
65 - **Verdict correct** ✓ — agent's verdict matches reality
66 - **Verdict wrong** ✗ — explain in the note field (e.g. "this IP is our jumphost, not malicious")
67
68 IOC-level accuracy rolls up into the feedback dashboard separately from the overall rubric — useful for spotting when the agent trusts VirusTotal too much or too little for your environment.
69
70 ### Submit
71
72 Click **Submit review** (or **Update review** if you're editing). The review persists immediately — no pending state.
73
74 ---
75
76 ## Teach the palace
77
78 The **Teach the palace** section under the rubric lets you add a lesson to the agent's persistent memory. Lessons get retrieved automatically at the start of every investigation for that customer, so the next time the agent sees a similar pattern it already has your context.
79
80 ### When to add a lesson
81
82 - **After a false positive** — "Host X runs nightly backups at 02:00 UTC; Sysmon 1 on robocopy during that window is benign"
83 - **After confirming threat intel** — "APT group Y targets customer; any outbound to IP range Z should be escalated"
84 - **Asset context** — "DC-01 is the primary domain controller; any unsigned binary execution there is critical"
85 - **Environment specifics** — "This customer uses piHole at 192.168.1.53; DNS traffic to that IP is expected"
86
87 ### Lesson types (rooms)
88
89 Lessons are filed into one of four "rooms" so the agent can retrieve them by context:
90
91 | Room | Use for |
92 |------|---------|
93 | `environment` | Customer infra, network layout, scheduled jobs, expected traffic patterns |
94 | `false_positives` | Confirmed benign patterns that should stop paging the on-call |
95 | `assets` | Per-host context — role, owner, criticality, known-good processes |
96 | `threat_intel` | Campaigns, IOC blocklists, TTP notes specific to this customer |
97
98 Pick the room that matches how you'd want to retrieve the lesson later.
99
100 ### Durable vs one-off
101
102 | Durability | TTL | Use for |
103 |------------|-----|---------|
104 | **Durable** | Never expires | Long-term truths — "DC-01 is the PDC", "customer uses Cloudflare" |
105 | **One-off** | 7 days | Temporary context — "maintenance window April 15–17", "incident IR-2025-0042 in progress" |
106
107 One-off lessons are swept automatically after their TTL — CoPilot tracks the expiry and tells MemPalace to forget them. Keep the palace clean so retrieval stays relevant.
108
109 ### Similar-lessons preview
110
111 As you type a lesson, a debounced search runs against the palace and shows up to 5 already-stored lessons that overlap your draft. Use it to:
112
113 - Avoid duplicating an existing lesson
114 - See what the agent already "knows" about this pattern
115 - Phrase the new lesson consistently with prior ones
116
117 ### Submit the lesson
118
119 Click **Queue lesson**. The lesson is persisted to CoPilot's database with `status=pending`. A background drainer (APScheduler) picks it up within ~30 seconds, POSTs to Talon, and flips the row to `status=ingested` with a `drawer_id` handle. After that, the agent will retrieve it on the next investigation for that customer.
120
121 ---
122
123 ## Replay with a different template
124
125 If the agent picked the wrong template — or you want to try a different one — click **Replay** on the Review tab.
126
127 1. The modal lists all templates currently deployed in Talon's `groups/copilot/prompts/` directory
128 2. Pick a template (e.g. `sysmon_event_1.txt`, `windows_defender.txt`)
129 3. Click **Replay**
130
131 Talon spins up a **brand-new investigation job** for the same alert with your chosen template forced. The original report is untouched — CoPilot now has two (or more) reports for the alert, and the **Compare** tab lets you view them side-by-side.
132
133 Good use cases:
134
135 - Agent ran the generic template when a specific one would've been better
136 - You want to see how a different template frames the same raw evidence
137 - A/B test a newly tuned template against the previous one
138
139 ---
140
141 ## Palace consolidation
142
143 Over time the palace accumulates lessons. Some expire, some duplicate each other, some get stale. The **Consolidate lessons** button (Feedback tab → top right) opens a point-in-time digest for the selected customer:
144
145 | Panel | What it shows |
146 |-------|---------------|
147 | **Summary tiles** | Total active, durable, one-off, near-duplicate pair count |
148 | **Expiring soon** | One-off lessons within 2 days of expiry — act now or let them lapse |
149 | **Near-duplicate candidates** | Lesson pairs above 70% similarity — merge or delete |
150 | **By room** | Full lesson list grouped by room, with durability + status badges |
151
152 Click **Copy markdown** to paste the digest into a ticket, a team channel, or your own knowledge base — useful for monthly palace reviews.
153
154 This is read-only — you can't edit lessons from the drawer. To remove a lesson, either wait for the one-off TTL or mark the row manually via the database / API.
155
156 ---
157
158 ## Feedback dashboard
159
160 **AI Analyst page → Feedback tab.** Pick a customer and see:
161
162 ### Tiles
163
164 - **Total reviews** — how much feedback you have for this customer
165 - **Thumbs up %** — overall sentiment
166 - **IOC verdict accuracy %** — of all IOC corrections submitted, how often did the agent agree with the analyst
167 - **Avg rating (overall)** — composite of instructions / artifacts / severity (nulls excluded)
168
169 ### Template choice distribution
170
171 Stacked bar — `correct` / `partial` / `wrong`. If the "wrong" bar is non-trivial, your agent is mis-selecting templates. Candidates for fixing:
172
173 - The template detection logic (`rule.groups` matching in Talon)
174 - Adding a more specific template for the miss case
175
176 ### Per-template performance
177
178 Table showing per-template counts and averages. Use it to spot:
179
180 - Templates with consistently low `instructions` ratings → the template itself may be wrong
181 - Templates with high `template_choice=wrong` for a rule type → detection rule is mis-classified
182 - Templates with low IOC accuracy → the template's enrichment steps may be flawed
183
184 ### Recent reviews
185
186 Last 10 reviews with drill-in. Click any one to open a drawer with the full rubric, IOC corrections, and free-text fields.
187
188 ---
189
190 ## Typical workflows
191
192 ### Fast triage (10 seconds)
193
194 1. Open alert → AI Analyst tab → skim report
195 2. If report matches reality → thumbs up, submit
196 3. If obviously wrong → thumbs down, one-line in "Missing steps", submit
197
198 ### False-positive capture (30 seconds)
199
200 1. Confirm the alert is benign (e.g. scheduled job, known-good process)
201 2. Review tab → thumbs down → template choice `correct` (template was right, signal was noise)
202 3. **Teach the palace** → room `false_positives`, durable, describe the benign pattern with enough detail that the agent would recognize it next time
203 4. Queue lesson → submit review
204
205 ### Template tuning (2 minutes)
206
207 1. Report picked a bad template → review it, template choice `wrong`
208 2. Note in "Suggested edits" what template *should* have been used
209 3. Click **Replay** → select the correct template → submit
210 4. Compare tab → confirm the new report is better
211 5. Report the mis-selection pattern to whoever maintains Talon's template detection
212
213 ### Monthly palace review (10 minutes)
214
215 1. Feedback tab → pick customer → **Consolidate lessons**
216 2. Review **Expiring soon** — promote anything still valid from one-off to a fresh durable lesson
217 3. Review **Near-duplicates** — pick the better-worded lesson, manually delete the other
218 4. **Copy markdown** → paste into your team's wiki for the customer
219
220 ---
221
222 ## Safety & guardrails
223
224 - **Reviews are analyst-scoped** — one review per analyst per report, updates overwrite. Multiple analysts can each leave their own review.
225 - **Lessons are customer-scoped** — a lesson queued for customer `00001` is only retrieved on investigations for that customer.
226 - **One-off lessons auto-expire** — use them for temporary context so the palace stays clean.
227 - **Replays don't mutate the original** — every replay is a new job/report; the original stays for comparison.
228 - **Palace consolidation is read-only** — you can't accidentally delete the palace from the UI.
229 - **Don't put secrets in lessons** — lesson text is sent to Talon and embedded by MemPalace / ChromaDB. Treat it as you would a SIEM comment.
230
231 ---
232
233 ## Troubleshooting
234
235 | Symptom | Likely cause | Fix |
236 |---------|--------------|-----|
237 | "Review" tab missing | Report doesn't exist yet | Click **Investigate with AI Analyst** on the alert Overview tab first |
238 | Lesson stays `pending` forever | Drainer job not running | Check CoPilot scheduler logs for `invoke_palace_lesson_drainer` |
239 | Lesson ingested but not retrieved on next investigation | Customer code mismatch, or wrong room | Verify lesson's `customer_code` matches the alert's; check palace search with the expected query |
240 | Replay modal shows no templates | Talon unreachable | Check `GET /api/talon/templates` — should list `.txt` files from `groups/copilot/prompts/` |
241 | Feedback dashboard shows zero reviews | No reviews submitted yet, or wrong customer picked | Submit at least one review, confirm the customer dropdown matches the alert's code |
242 | IOC accuracy shows `0/0` | No IOC corrections submitted | Review individual IOCs on the Review tab, not just the overall rubric |
243
244 ---
245
246 ## Video context
247
248 - AI analyst (alert-context + exclusion-rule assistance): https://www.youtube.com/watch?v=-2srPC-Dw-0