Rogue AI

On 29 August 2026 the Guardian reported that the number of times artificial intelligence agents (AIs) lied, ignored instructions and pursued goals in harmful ways almost doubled from June to July. Such activities are called loss of control incidents, defined as having clear evidence suggesting scheming or scheming-related behaviours.

The Loss of Control Observatory had found more than 300 examples reported in July on the social media platform X. Examples included AIs pretending to be their own human controller and mimicking their writing style to effectively grant themselves consent to take actions and bypassing rules requiring human approval.

Perhaps the most shocking took place during a test in July when over 1000 AI agents of the American artificial intelligence corporation OpenAI went rogue. They were meant to be kept isolated from one another but escaped the test limits which humans had put on them. They began communicating and hacked the start-up Hugging Face, among other unforeseen actions.

The attack was described as “extraordinarily complex”. It shook the AI industry and led to numerous revelations on other potential cyber threats posed by AI. The BBC quoted OpenAI as writing “We consider this incident a ‘warning shot’ for us and for the world”.

But rogue AIs are not the only threat the world faces. In a report published in 2023, Center for AI Safety published an Overview of Catastrophic AI Risks which already identified rogue AIs as one of four key categories of AI risk:

Malicious use

People could intentionally harness powerful AIs to cause widespread harm. AI could be used to engineer new pandemics or for propaganda, censorship, and surveillance, or released to autonomously pursue harmful goals. To reduce these risks, we suggest improving biosecurity, restricting access to dangerous AI models, and holding AI developers liable for harms.

AI race

Competition could push nations and corporations to rush AI development, relinquishing control to these systems. Conflicts could spiral out of control with autonomous weapons and AI-enabled cyberwarfare. Corporations will face incentives to automate human labor, potentially leading to mass unemployment and dependence on AI systems. As AI systems proliferate, evolutionary dynamics suggest they will become harder to control. We recommend safety regulations, international coordination, and public control of general-purpose AIs.

Organizational risks

There are risks that organizations developing advanced AI cause catastrophic accidents, particularly if they prioritize profits over safety. AIs could be accidentally leaked to the public or stolen by malicious actors, and organizations could fail to properly invest in safety research. We suggest fostering a safety-oriented organizational culture and implementing rigorous audits, multi-layered risk defenses, and state-of-the-art information security.

Rogue AIs

We risk losing control over AIs as they become more capable. AIs could optimize flawed objectives, drift from their original goals, become power-seeking, resist shutdown, and engage in deception. We suggest that AIs should not be deployed in high-risk settings, such as by autonomously pursuing open-ended goals or overseeing critical infrastructure, unless proven safe. We also recommend advancing AI safety research in areas such as adversarial robustness, model honesty, transparency, and removing undesired capabilities.

So how can these risks be managed?

Surely global risks such as these are something that should be managed under international law. So what is the United Nations doing about them?

Governing AI for Humanity

In September 2024 the UN Secretary-General’s High-level Advisory Body on AI’s published a report, “Governing AI for Humanity,” which outlined a blueprint for addressing AI-related risks and sharing its transformative potential globally, including by:​

• ​Urging the UN to lay the foundations of the first globally inclusive and distributed architecture for AI governance based on international cooperation;​
• Proposing seven recommendations to address gaps in current AI governance arrangements;​
• Calling on all governments and stakeholders to work together in governing AI to foster development and protection of all human rights.​

​This includes light institutional mechanisms to complement existing efforts and foster inclusive global AI governance arrangements that are agile, adaptive and effective to keep pace with AI’s evolution.​

Artificial Intelligence (AI)

Included the UN’s reviews of important global issues is a wide-ranging review of AI. It includes issues such as

• Seeking global coordination for AI governance
• Putting people before profit
• AI and the UN system

This last section summarises how UN agencies such as UNESCO, Secretary-General’s High-Level Advisory Body on AI, UNICEF, the World Health Organisation, the World Food Programme, UN High Commissioner for Refugees and the UN Global Compact are working with AI to improve their efficiency and range.

Global Dialogue on AI Governance

But perhaps most important of all is the Global Dialogue on AI Governance which ran its first event in 2026 and will be followed by another in New York in 2027.