whitepaper

Benchmarking AI Security.

Ask an AppSec team whether its AI features are secured and the answer is usually yes, since a guardrail product sits in front of the chat box. Ask which systems call which tools on which credentials, which corpora accept writes from outside, and who last tried to break any of it, and the room goes quiet. This instrument settles the question with artefacts. Twenty controls, four domains, one hundred points, and a row scores only when a test result or a document is on the table.

Download whitepaper Talk to Our Engineers

In depth

An Untested Control Is A Design Intention With A Budget Line.

Questionnaires measure what was designed, not what survived.

A security questionnaire asks whether a control exists and accepts a yes, which is why teams with a guardrail product and a written standard score well on paper and badly under a red team. This instrument inverts the test: every row demands the artefact or the test result behind it, and a roadmap item scores zero. So does a supplier assurance nobody has checked.

In shortSo does a supplier assurance nobody has checked

The weights follow what decides an engagement.

Attack surface knowledge carries 30 points and architectural controls another 30, since an attacker picks the system you forgot and the credential you over-scoped. Adversarial assurance takes 20 and detection and response takes 20, which is where most competent teams lose the run of it. Nothing here is weighted by what it costs to build.

In shortNothing here is weighted by what it costs to build

A control that holds on the flagship holds nothing.

An attacker picks the other eight applications, the ones built by a team that read the OWASP list once and shipped under a demo deadline, so score across the estate and take the weakest reading. The same applies to the AI arriving inside software you bought, which almost nobody scores at all. Coverage is the difference between a score and a story.

In shortCoverage is the difference between a score and a story

Self-scoring proves nothing to an attacker.

It maps the exposure and orders the work, and it can tell you which four fixes take the most time off the containment clock for the least outlay. What it cannot do is call a system safe. That verdict belongs to whoever tries hardest to break it, so read the total as a hypothesis with a date on it.

In shortread the total as a hypothesis with a date on it

The detail

Where The Hundred Points Sit, And Which Ones Go Missing.

Attack surface knowledge carries 30 points, architectural controls 30, adversarial assurance 20, detection and response 20. The three below are where a typical run bleeds, and each failed row gets a name, a weight and a date it can be closed by.

Zone · 01

Attack surface knowledge

Thirty points for knowing what exists: a single register of AI systems including the unapproved ones, every tool a system can call with its credential and scope, retrieval corpora listed with their write paths, models pinned by digest rather than tag, and non-human identities with a named owner and a rotation date that has passed at least once.

Zone · 02

Adversarial assurance

Twenty points, and the domain that most often reads zero. An external test against the OWASP Top 10 for LLM Applications inside twelve months, an evaluation suite in CI that can fail a build, findings tracked to a retest rather than a ticket closure, scope that reached the tools and the corpora, and testers who do not report to the people who built it.

Zone · 03

Containment

measured not estimated

Twenty points for detection and response, where the expensive row is the drill. Full request traces held in storage you control, an AI incident playbook that is not the outage runbook, cost breakers tripped on purpose, and a dated record of how many minutes passed from the first anomalous trace to the component being isolated.

By the numbers

The figures that make it a board-level conversation.

46 / 100

the score a competent engineering team typically reaches against these twenty controls

40%

of organisations control access to their AI models and data at all

$1,100

added to the bill for every hour a breach runs before containment

Inside the report

What you'll take away.

Step 1 - Score one live system, not the architecture you intended

Pick an AI feature already serving traffic and bring whoever drew it. Take the whole weight on a row or none of it, and write down every artefact that turns out not to exist.

Step 2 - Read the distribution before you read the total

Architecture usually scores best and buys nothing you can prove, while adversarial assurance reads near zero. A high total with an empty Domain C means the design has never met a hostile caller.

Step 3 - Rank the fixes by hours taken off the containment clock

Two inventories built from configuration files that already exist, a playbook written from the incident classes, and one half-day drill moved a real score from 46 to 67 inside a quarter.

Step 4 - Put the saved hours against $1,100 and fund the work

One drill took the path from 212 hours to 68, and the 144 hours removed are worth about $158,400 against a build cost near $42,000. That is a funding case, not an assurance.

Questions

Frequently asked.

What counts as evidence for a row, and what does not?

An artefact on the table or a test result you can point at, there and then. A roadmap item scores zero. So does a supplier assurance nobody has verified, and so does a control that holds on the flagship application but not on the other eight, since an attacker picks the eight.

What is a realistic score for a competent engineering team?

Forty-six out of a hundred is the figure a worked example reached: a B2B software company with nine AI features in production, a capable platform team and a guardrail product in front of two of them. Architecture scored 17 of 30. Adversarial assurance scored 7 of 20.

We scored under 55. Where do we start?

The register and the credential map, in that order, and both are built from configuration files you already hold. Under 55 puts you with the 92% whose AI-related breach met no real access control, so the first job is knowing what exists and what each thing is allowed to reach.

Does a high score mean our AI systems are secure?

No. Self-scoring maps exposure and orders the work, and it cannot call a system safe, because that verdict belongs to whoever tries hardest to break it. Above 80 the useful move is to hand an enterprise customer a test report instead of a policy, then widen coverage to the AI inside software you bought.

Why does adversarial assurance carry only 20 points if it matters most?

Weight follows what decides an engagement, and the first two domains decide more of it. An untested control still bounds damage; an untested system with no inventory and shared credentials does not. Scoring 55 to 79 almost always means the architecture is sound and the assurance is thin, and the external test is the next cheque to write.

How does this differ from your other AI security resources?

This one scores what you already run. AI Security: What Buyers Should Ask turns the same ground on a supplier before you sign. The AI Security Report takes the 2026 cost curve to a board. Score yourself with this, interrogate vendors with that, and fund the work with the third.