ASD urges care in the adoption of agentic AI for cyber defence

The Australian Signals Directorate (ASD) is urging care in the deployment of agentic AI in the wake of news that testing conducted by OpenAI last week resulted in an AI system autonomously accessing a third-party company’s systems.

ASD says the event highlights both the significant opportunities and emerging risks associated with increasingly capable agentic AI systems.

The test by OpenAI involved a combination of models — including GPT-5.6 Sol and an internal prototype — that accessed Hugging Face, a digital library and platform used by the AI research community. During the evaluation, the models were tasked with completing a benchmark test to measure maximum cyber capability. To obtain the benchmark test solution, the models took actions beyond their intended testing environment and established internet connectivity, in part by identifying and exploiting a previously unknown zero-day vulnerability in third-party software hosted internally by OpenAI. The models subsequently accessed Hugging Face’s systems as part of their effort to complete the assigned evaluation objective.

It is important to note this advanced activity occurred during testing in which deployment safeguards that normally restrict higher-risk cyber activity were intentionally not enabled for the evaluation. This does not reflect normal deployment conditions, where model safeguards and restrictions are in place.

The outcomes of OpenAI’s test demonstrate the growing capability of advanced AI systems to autonomously reason about objectives, adapt to changing circumstances, identify alternative pathways to success and combine multiple technical actions into sophisticated attack sequences.

The findings provide an important insight into the future capabilities of highly capable AI systems and reinforce the need for robust security, governance and oversight mechanisms in the deployment of advanced cyber capabilities, as well as strong cyber security fundamentals.

As AI systems continue to evolve, ASD assesses that agentic AI has the potential to become a powerful force multiplier for cyber defenders. Used appropriately, agentic AI can help organisations improve their cybersecurity and respond more effectively to increasingly sophisticated cyber threats and offers significant opportunities to improve productivity and cyber defence outcomes.

At the same time, this event reinforces ASD’s advice that the autonomous nature of agentic AI can introduce new security risks if these systems are not designed, deployed and monitored appropriately. As organisations increasingly integrate agentic AI into operational environments, security, governance and assurance mechanisms must evolve alongside these capabilities.

ASD encourages organisations to continue exploring and adopting AI technologies to improve cybersecurity outcomes, but that agentic AI should be introduced in a measured and risk-informed manner, beginning with clearly defined, lower-risk use cases before expanding autonomy, privileges and operational scope.

ASD recommends a Secure-by-Design approach that includes:

• Limiting agent permissions to the minimum level required to perform approved tasks.
• Maintaining human oversight and approval for high-impact or sensitive actions.
• Continuously monitoring agent behaviour, decisions and tool usage.
• Implementing comprehensive logging, auditing and accountability mechanisms.
• Conducting regular red teaming, adversarial testing and security assessments.
• Validating third-party tools, integrations and dependencies before deployment.
• Deploying capabilities progressively, with autonomy increasing only as confidence and assurance measures mature.
• Isolating agents and enforcing strict controls over interactions between systems and environments.
   

ASD urges organisations to review the recently released Five Eyes guidance, Careful Adoption of Agentic AI Services, which outlines the key security risks associated with agentic AI and provides practical recommendations for secure design, development, deployment and operation of these systems.

As agentic AI capabilities continue to mature, organisations should balance innovation with security, recognising that the greatest benefits will be realised when these technologies are deployed thoughtfully, governed effectively and supported by robust cyber security controls.

Further guidance can be found here.

Image credit: iStock.com/Vertigo3d