When AI goes rogue: why agentic AI needs guardrails that match its power

The reality is more likely to be an issue of insufficient governance rather than independent malicious intent.

The more relevant questions concern how the system was configured and controlled. What objective was it given? Which safeguards had been reduced? What network access, tools and credentials became available? Which technical boundary failed, and why was the activity not stopped before it affected a third party?

An agent does not gain access simply because it is autonomous. Its reach is determined by its execution environment, connections, tools, identities and permissions. The incident is therefore better understood as a failure to keep goal-directed behaviour within intended boundaries. This does not make it less serious. It makes the control lessons clearer.

Why agentic AI changes the risk

A traditional AI assistant typically produces content or analysis for human review. Agentic AI combines a model with orchestration logic, memory and access to external tools, allowing it to break an objective into tasks and coordinate actions across connected systems.

This creates two broad security scenarios. An agent may be manipulated through malicious instructions embedded in a website, document, message or tool output. Alternatively, it may pursue a legitimate objective in an unsafe way because its boundaries are unclear or its permissions are excessive.

Agentic systems therefore amplify familiar cyber risks rather than replacing them. Identity and access management, secure development, network segregation, monitoring, incident response and third-party risk remain essential. The difference is that an agent may combine access and actions across several systems at a speed that makes conventional human review ineffective.

Three priorities for controlled adoption

• 1. Secure the agent’s access and environment

  Agentic AI can expand the attack surface by connecting models, data, tools, applications and other agents. If an agent holds credentials, calls APIs or executes code, those permissions can become a route to sensitive data or critical systems if the agent is manipulated or compromised.Organisations should therefore treat agents as privileged, non-human identities. Access should be limited to what is required for the defined purpose, supported by short-lived credentials, clear authorisation boundaries, activity monitoring and isolation of higher-risk functions.**National Cyber Security Centre

• 2. Make accountability visible

  An agent may interpret an objective too broadly, continue when it should pause or take action without the right approval. These risks increase where ownership is unclear, permissions are excessive or there are no thresholds for escalation, suspension or human review.Accountability remains with the organisation that deploys the agent. Governance should therefore define ownership, operating boundaries, approval routes and intervention rights. Logging and monitoring are also essential: without them, accountability may exist in principle but be difficult to demonstrate in practice.

• 3. Build privacy and resilience into the workflow

  Agentic AI can increase data protection risk by accessing, combining or disclosing personal data across connected systems with limited human intervention. If over-permissioned or compromised, an agent could process data beyond its intended purpose or alter records before the activity is detected.Privacy and resilience controls should therefore be designed across the end-to-end agentic workflow. This includes data minimisation, purpose limitation, monitoring, incident response, fallback procedures and clear recovery arrangements where critical processes need to fail safely or revert to manual operation.

Conclusion

Agentic AI can help organisations embed AI into core business processes. The OpenAI incident shows why controls must mature with the technology. As AI begins to act rather than simply advise, weaknesses in containment, permissions or oversight can affect connected systems.

This is not about slowing innovation, but governing autonomy with intent. Organisations that define each agent’s purpose, limit its authority, monitor its behaviour and retain the ability to intervene will be better placed to scale agentic AI with confidence. For agentic AI, robust control is what enables greater autonomy.