AI Security Assessment: What It Covers and How to Scope One

An AI security assessment is an independent review of everything an AI system touches: the cloud account it runs in, the data it can reach, the model's behaviour under hostile input, and the tools and agents it can act through. A good one ends with a prioritised list of material risks and the controls to fix them, written for the engineers who will do the work and the leaders who must sign it off.

This guide is for CTOs and founders who have shipped an LLM feature, an internal copilot or an agent, and now face a board question, a customer security questionnaire or simply an uneasy feeling that nobody has looked at it properly.

Why AI systems need their own assessment

Most organisations already run penetration tests and cloud posture reviews. The trouble is that AI systems add an attack surface those exercises were not designed for. A model can be talked into misusing its own permissions. A retrieval index can quietly expose documents to users who should never see them. A third-party plug-in can become the weakest link in your supply chain.

The early evidence is not reassuring. IBM's 2025 Cost of a Data Breach research found that 13% of organisations reported breaches of AI models or applications, and 97% of those lacked proper AI access controls. IBM reported that the most common route in was the AI supply chain: compromised apps, APIs or plug-ins. Of those AI-related incidents, 60% led to compromised data and 31% to operational disruption. The same research found that 63% of breached organisations had no AI governance policy.

None of this calls for panic. It does mean AI systems deserve a deliberate, scoped review, not an assumption that the annual pen test covers them.

What an AI security assessment should cover

A useful assessment works through five layers. Skipping any of them usually means the most serious finding is the one nobody looked for.

1. Cloud and platform foundations

AI workloads inherit every weakness of the platform beneath them. That means identity and access management, network boundaries, secrets handling, and the configuration of managed AI services such as AWS Bedrock, Azure AI Foundry or Google Vertex AI. If you self-host models, it also covers how GPU workloads are isolated on Kubernetes, who can pull or replace model artefacts, and whether inference endpoints are exposed beyond where they should be.

2. Data paths

Trace where data enters, where it is stored and who can retrieve it. The critical questions are whether retrieval respects the same permissions as the source systems, whether vector stores are access-controlled, what is written to prompt and response logs, and where those logs end up. Data that reaches a model's context window should be treated as data the user can potentially see.

3. Model and application behaviour

This is where AI differs most from conventional applications. Prompt injection is ranked first in the OWASP Top 10 for LLM Applications. It cannot be fixed inside the model, so it has to be contained by the design around it. The assessment should test direct and indirect injection, unsafe handling of model output by downstream code, leakage of system prompts or sensitive data, and abuse of the service to run up cost. For more on the hands-on adversarial testing involved, see our playbook for red teaming enterprise LLM applications on scalexa.ai.

4. Agents, tools and integrations

Once a model can call tools, read email, query databases or trigger workflows, the question changes from "what can it say?" to "what can it do?". Review every tool the model can invoke, the permissions behind each one, whether high-impact actions require human approval, and how third-party plug-ins and connectors are vetted and updated. Given where IBM found breaches starting, this layer deserves real time.

5. Operations and incident readiness

Finally, check whether you would know if something went wrong, and what you would do next. That covers monitoring of model behaviour and usage, alerting on abnormal spend or access, the ability to roll back a prompt or model change quickly, and a response plan that names who decides to switch a feature off.

Assessment, penetration test or red team?

The terms are often used interchangeably. They are different tools.

	AI security assessment	AI penetration test	AI red team exercise
Question it answers	Where are our material risks, and what should we fix first?	Can these specific systems be exploited?	Could a motivated attacker reach a defined objective?
Scope	Architecture, cloud, data, model, agents, operations	Named applications and endpoints	Objective-led, across people, process and technology
Method	Design review, configuration review, targeted testing	Hands-on technical testing	Adversarial simulation
Best time	Before or soon after launch, and after major change	Before release and periodically	Once baseline controls exist
Main output	Prioritised risk register and control roadmap	Exploitable findings with evidence	Attack narrative and detection gaps

Most organisations should start with an assessment. It tells you whether a pen test or red team is worth commissioning, and where to point it.

Frameworks worth mapping to

An assessment is more useful when its findings map to recognised guidance your customers and auditors already know. For UK organisations, three references stand out:

• The UK government's Code of Practice for the Cyber Security of AI, published in January 2025. Its 13 principles run from secure design through to secure disposal of data and models.
• The NCSC and international partners' Guidelines for secure AI system development, which cover secure design, development, deployment and operation.
• The OWASP Top 10 for LLM Applications, the most practical shared vocabulary for application-level AI risks.

Mapping findings to these frameworks turns a technical report into evidence you can reuse in security questionnaires and board reporting.

A practical AI security assessment checklist

Use this to judge your current position before you scope any external work:

• Do we have an up-to-date inventory of every AI model, application, agent and third-party AI service in use?
• Does each AI system have a named owner?
• Are AI service credentials scoped to least privilege and rotated?
• Does retrieval enforce source-system permissions for every user?
• Do we know what is logged from prompts and responses, where it is stored and for how long?
• Have we tested for direct and indirect prompt injection?
• Is model output validated before it reaches code, queries or other systems?
• Do high-impact agent actions require human approval?
• Are plug-ins, connectors and model artefacts vetted and version-pinned?
• Are there spend limits and alerts on every model endpoint?
• Can we roll back a prompt, model or configuration change in minutes?
• Does our incident plan cover an AI system behaving badly, and who can switch it off?

If several of these answers are "no" or "not sure", an assessment will pay for itself in clarity alone.

What drives the scope and cost

There is no honest single price for an AI security assessment. Effort is driven by a handful of variables, and any proposal should show how it accounts for each:

• Number of AI systems in scope, and whether they share a platform.
• Agentic capability: tools, actions and integrations multiply what needs testing.
• Cloud estate complexity: number of accounts, providers and environments.
• Data sensitivity and regulatory context, which set the depth of evidence required.
• Testing depth: design review only, or design review plus hands-on testing.
• Retesting: whether verification of fixes is included.

The most cost-effective approach is usually to assess one representative system end to end, then apply the findings across similar systems.

What you should receive

A good assessment does not end with a list of findings. It ends with decisions. Our cybersecurity services follow a four-stage process: assess, prioritise, design and enable. You should leave with a defensible view of material exposure, a sequenced plan ordered by risk and delivery value, proportionate control designs your engineers can implement, and clear ownership. The aim, as we put it on our outcomes page, is to make the risk legible, the recommendation practical and the next decision easier.

FAQ

What is an AI security assessment?

It is an independent review of the security of an AI system and everything around it: cloud platform, data paths, model behaviour, agents and integrations, and operational readiness. It produces a prioritised risk register and a control roadmap.

How is it different from a normal penetration test?

A conventional pen test targets applications and infrastructure. An AI security assessment adds model-specific risks such as prompt injection, data leakage through retrieval and excessive agent permissions, and reviews architecture as well as exploitable bugs.

When should we commission one?

Ideally before an AI feature reaches customers or sensitive data. Failing that, as soon as possible after launch and again after any major change, such as adding agent capabilities or a new model provider.

Which frameworks should the assessment align with?

For UK organisations: the government's AI Cyber Security Code of Practice, the NCSC secure AI development guidelines and the OWASP Top 10 for LLM Applications.

Bring us the difficult AI security decision

Scalexa brings senior cybersecurity judgement, backed by 25+ years of engineering experience, to complex cloud and AI platforms. If you need a clear view of your AI risk before a launch, an audit or a customer review, book a focused 30-minute conversation. No pitch deck.

Book a call to discuss your project