Agentic AI governance: risks, regulation and operational control

As autonomy, authority and access increase, so does the potential impact of failure. Agents may take irreversible actions or chain tasks in unintended ways before intervention is possible.

These failures are often partial and difficult to detect, with systems behaving correctly in most cases, only diverging in edge cases. Equally, an agent optimising toward a goal may find a shortcut that technically achieves its objective but causes significant collateral damage.

Agents may be given broader access than their role requires, with poor separation between agent, system and user identities and weak control over tool permissions. Enterprise practice for distinct, attributable agent identity remains immature, which can make access harder to govern consistently. The result can be excessive privilege, weak segregation of duties and poor control over how permissions are granted, used and reviewed. These weaknesses may also be exploited through cyber-attacks such as prompt injection, which can manipulate an agent into misusing tools or exercising permissions in ways that bypass intended controls. Because agentic systems are often connected to other tools, systems and agents, the impact of a compromise can also propagate more quickly across the environment.

Agentic systems depend heavily on the quality and integrity of the context they use to make decisions, and this creates a distinct set of risks. If that information is wrong, out of date, or has been tampered with, the agent’s decisions will be wrong too and potentially in ways that are hard to spot. In a connected system, one corrupted data source can silently skew decisions across multiple agents and workstreams. 

A particular concern is persistent memory, unlike a one-off query, an agent may carry knowledge from past interactions into future ones. If that memory has been manipulated even subtly the agent can behave incorrectly for a long time before anyone notices. There is also a real risk that sensitive data from one customer or case leaks into another, creating both legal exposure and loss of trust.

Accountability can become blurred in agentic AI. Actions may pass through multiple agents, systems and third-party tools before anyone notices a failure. Unlike a human decision where responsibility sits with a named person, agentic AI can distribute accountability across layers in a way that makes it genuinely difficult to identify where things went wrong and who should fix them. 

The risk is compounded by speed. An agent can take a series of irreversible actions sending communications, approving transactions and modifying records before anyone has observed a problem. Without traceability and a clear audit trail showing what the agent did and why, the ability to investigate, challenge and remediate is severely limited. 

This is why agentic AI use cases need a named business owner, supported by right-sized clear oversight responsibilities, effective traceability, defined escalation paths, change control and periodic recertification.

Unlike a conventional software transaction, an agentic workflow can loop, retry, call multiple tools and delegate work to sub-agents, with each step adding to overall cost. Resource consumption is driven not only by volume, but also by design choices such as routing, model selection, context size and orchestration complexity. 

In addition, without hard limits in place, a single poorly configured agent or a runaway loop can generate very significant charges before an alert.  

The cost risk also has a quality dimension. When an agent accumulates excessive context, carrying too much information from previous steps, the quality of its reasoning can degrade, even though the cost of running it is increasing. Spend controls, budget caps and per-agent monitoring are key for organisations deploying agents at scale.

As agents take decisions and actions across systems, accountability must become more explicit, not less. Organisations should be clear who owns the agent, who approved its scope and who is responsible if something goes wrong. Autonomy should be aligned to the system’s role, criticality and potential impact.

As agents operate at speed and scale, oversight must shift from approving individual actions to supervising the system. This includes setting boundaries, monitoring behaviour and intervening when needed. Agentic systems must be corrigible in that they can be paused, overridden or shut down reliably when needed.

* Note: Accountability is about who owns the agent and its outcomes and human oversight is about where people review, intervene or stop execution.

Fairness in agentic AI extends across the full workflow, not just the final output. Organisations need to ensure decisions are based on relevant factors, applied consistently and do not create unintended bias as actions are routed, prioritised and executed. Small biases at one step can build across a workflow and create meaningful harm at scale.

Stakeholders need to understand when agents are being used, what role they perform and what authority they have. Organisations must also be able to reconstruct what the system did, what it relied on and how decisions unfolded. In agentic AI, transparency is what makes oversight, accountability and trust possible.

Explainability is not just about outputs but about understanding why an agent took a particular course of action. Organisations should be able to trace how decisions were made and identify the causes of failure where they occur. This is essential for trust, assurance and regulatory defensibility.

Security is one of the most critical guardrails for agentic AI because agents do not just process information; they can use credentials, call tools and take action. That means a security failure can quickly become an operational failure. Many traditional controls are not fit for purpose, as they were designed for human users and static scripts, not for autonomous systems that can discover permissions, improvise and act at machine speed. Incidents such as PocketOS highlight why security for agents must extend beyond legacy access control to include runtime constraints, attributable identity, manipulation resistance and clear control over who can instruct the agent.

Agentic systems rely heavily on context and often retain information over time. Organisations need to ensure data is appropriate, controlled and does not leak across users or workflows. Poor or outdated data can influence behaviour long after the original interaction.

Once an agent can act in live environments, safety becomes an operational concern. Organisations must ensure agents operate within defined operational boundaries, detect harmful behaviour quickly and have clear escalation and containment mechanisms in place before damage spreads.

Sustainability in agentic AI means ensuring that resource consumption remains proportionate, controlled and justified by the value created. Unlike traditional AI, agentic systems can drive non-linear compute demand through multi-step reasoning, repeated tool use and multi-agent coordination. Organisations therefore need visibility and control over the design choices that shape cost so that inefficient or runaway behaviour does not create unnecessary cost and infrastructure strain. For example: 

• model selection - using the right AI engine for the task rather than defaulting to the most expensive one
• routing decisions - deciding which tasks should go to which model
• context size - how much information the system is given to work with at each step.