Your SOC generates tens of thousands of alerts a week and staffs a rota that can properly investigate a single-digit percentage of them. This framework sets out the four components of a production AI analyst, the governance gates that keep it honest, and the nine-month sequence that gets there.
The trap most SOCs walked into: buy more detection, connect more telemetry, and let the queue absorb the difference, so tens of thousands of weekly alerts meet twenty analysts on a 24-hour rota, most are suppressed or silently expire, and the team calls that arithmetic tuning.
What the teams doing better do instead: move the volume to an agent with six scoped tools, a calibrated confidence threshold and a policy gate the model cannot reach, then keep every judgement human and spend the analyst hour on escalations, detections and purple-team injects.
Not Wildcard Credentials
The agent is defined by its tool registry, not its prompt, and the registry should be small and deliberately boring. A capped read-only SIEM query, an EDR lookup, an identity graph read, a threat intelligence call, a ticketing write, and containment held separate and gated. One wildcard credential makes the agent the estate's most attractive lateral-movement target.
Close as benign, suppress as duplicate, enrich and escalate, or page a human. That boundary belongs in the orchestration layer as deterministic, version-controlled policy, evaluated after the model's output. Asking a model politely to escalate anything touching a domain controller is not a control. A gate that refuses closure on a tier-0 asset tag is one.
Persist the alert identifier, every tool call with its arguments and raw response, the model and prompt versions, the reasoning summary, the confidence value, the policy verdict, the action taken and the human who accepted or overrode it. The test is reconstructing why alert 481,922 was closed nine months later, in front of counsel.
Alert volume by class, the true investigation rate, MTTD and MTTR per class rather than averaged, cost per investigated alert, and analyst attrition over twelve months. Two weeks of work, and the only comparison you will ever get.
The agent's verdict, confidence and evidence written to a store the analysts cannot see, against real alerts. Spend the review on disagreements rather than matches. Agreement near 91% is common, and the cases where the agent was right are the finding.
High volume, low consequence, every action reversible, with a named analyst sampling daily for the first month. Build fifty purple-team injects mapped to your detection classes first, and gate every model and policy change on them.
One command revokes every agent credential and drains the queue to humans. Test it on a working day. Then reconstruct five random closed decisions quarterly as though counsel had asked, and fix the logging before adding the next alert class.
No. The agent reads your existing SIEM, EDR and identity tooling through scoped, read-only connectors and writes to your ticketing system. What changes is the orchestration and policy layer above them, plus an evidence store you own. The detection estate stays exactly where it is, and so does the licence.
Three controls rather than one. A calibrated threshold so low-confidence alerts escalate with everything the agent gathered. A policy gate that refuses closure
on tier-0 asset tags regardless of model output. And blind re-opening of a random 3% of closures weekly, against a permanent human-only holdout queue.
Fully, until the pipeline is fixed. Filenames, process command lines, hostnames, user-agent strings, email bodies, DNS TXT records and certificate fields are all
adversary-writable and already in your pipeline. OWASP ranks injection and excessive agency as the top LLM application risks. The fix is structural, so no field content ever authorises a tool call.
Redistribution is the honest framing. Volume moves to the agent, judgement stays human, and tier-1 stops existing as a career stage. Analysts move to escalations, detection engineering and purple-team work, which is the job most of them wanted. Teams that brief this as a cut lose their best people first.
Around three months. Two weeks of baselining, eight weeks of shadow mode against real alerts, then a measured comparison against how your analysts closed the same queue. Agreement rates near 91% are common. The disagreements are where the value sits, because some of them will show an alert class you have been closing wrongly.
CISOs and heads of security operations who have already bought the detection estate and still cannot move dwell time. It assumes you run a 24-hour rota, you know what proportion of your alerts get a real investigation, and you now have to answer for the rest.
Drop your details and we'll send 258 Days To Contain A Breach Is Not A Tooling Problem straight to your inbox - no spam, unsubscribe anytime.
Run a scoped AI analyst against one of your real alert classes for two weeks, with a written comparison against how your analysts closed the same alerts.
Start a trial sprint