Short answer
An AI agent risk assessment examines what an agent is allowed to do, how it can fail, what data it touches and who intervenes when it does. Done before go-live, it turns vague worries into specific controls, an accepted residual risk and a named owner.
On this page
Why classic IT risk assessments miss agent risks
Most organizations already assess new software for security and data protection. Agents break some of the assumptions behind those templates. Their behaviour is not fully specified in code, they interpret instructions from documents and emails, they call tools with real permissions, and the same input can produce slightly different outputs. A checklist that asks "is data encrypted at rest?" is still necessary, but it says nothing about an agent that is tricked by a forwarded email into sending a supplier list to an outside address.
An agent-specific assessment fills that gap. It should be short enough to be done for every agent, and rigorous enough that an auditor, works council or data protection officer can follow the reasoning.
Our method in six steps
1. Describe the mandate
Write down in plain language what the agent owns, which outcome it is measured by and which decisions it must never make. A good mandate fits in a paragraph. If it does not, the agent is probably doing too much.
2. Map tools and permissions
List every system the agent can read from and write to, with the exact scope: which mailbox, which SharePoint libraries, which CRM objects, which API endpoints. Mark each write permission as reversible or irreversible. Irreversible actions, such as sending external messages, booking payments or deleting records, are where most of the risk lives.
3. Identify failure modes
For each tool, ask how things could go wrong:
- Wrong output: hallucinated facts, wrong calculation, outdated source.
- Wrong action: correct reasoning, but the wrong record or recipient.
- Manipulation: prompt injection through documents, emails or web pages the agent reads.
- Data leakage: personal or confidential data sent to a model, log or recipient that should not receive it.
- Silent failure: the agent stops working or loops and nobody notices.
- Scope creep: the agent is later given new tools without a fresh assessment.
4. Rate likelihood and impact
We use a simple three-level scale for each and discuss the ratings with the business owner. Precision matters less than a shared, documented judgement.
5. Assign controls
Typical controls for agents include least-privilege permissions, human approval for irreversible actions, allowlists for recipients and domains, input sanitization, confidence thresholds that trigger escalation, rate limits, full action logs and alerting on unusual behaviour. For personal data, anonymizing text before it reaches a model is one of the most effective controls; Textwash Pro does this locally, offline or as a privacy layer in front of ChatGPT and Claude.
6. Accept residual risk and name an owner
The assessment ends with a decision: the remaining risk is accepted by a named person with the authority to do so, with a review date tied to changes in scope. Without that signature there is no go-live.
The template
Our one-workbook template has these sections:
| Section | Content |
|---|---|
| Mandate | Task, outcome metric, forbidden decisions |
| Systems | Read and write permissions per system, reversible or not |
| Data | Categories of data, personal data, legal basis reference |
| Failure modes | Scenario, likelihood, impact, existing controls |
| Controls | Measure, implemented by, how it is tested |
| Oversight | Who approves what, escalation path, response times |
| Evidence | Where logs and test results are stored |
| Decision | Residual risk, owner, conditions, next review |
We hand the template over so your teams can run it themselves for future agents.
Testing, not just documenting
A risk assessment is only credible if the controls are tested. Before go-live our AI Code Engineers run scenario tests against the agent: injected instructions hidden in documents, ambiguous requests, edge-case records and simulated outages of connected systems. Results are attached to the assessment as evidence.
How the platform helps
On the kalAIDoCreate platform, many controls from the template are configuration rather than custom work: governed agents with explicit permissions, SSO, audit trails, integrations with SAP, CRM, SharePoint and Slack, and EU data residency. That keeps assessments consistent across dozens of agents. See also how we structure audit trails for AI agents.
Fit with regulation
The assessment supports, but does not replace, a data protection impact assessment and the classification required under the EU AI Act. We link the documents so the same facts are not maintained twice. Our AI governance and compliance team runs the process with you, and our managed AI agents operations team keeps assessments current after launch, because the riskiest moment is often the quiet change three months later. For the regulatory side, read EU AI Act for AI agents.
FAQ
How long does an AI agent risk assessment take?
For a well-scoped agent, a focused workshop plus testing is usually enough. Agents with many write permissions or personal data need more time.
Who should take part?
The business owner of the process, IT security, the data protection officer and the engineers building the agent. For agents affecting employees, involve the works council early.
Do we need a new assessment when the agent changes?
Yes, whenever it gains new tools, permissions, data categories or user groups. Minor prompt adjustments can be covered by a lighter review.
Can we use the template without your services?
Yes. We hand it over so your teams can run it independently.