The New Cyber Threat Is AI Agents With Too Much Power

The rapid developments in artificial intelligence have left the world in both shock and awe. There are some aspects of technology that help improve people's lives, but others - where AI seems to be going out of control - raise concerns about advancing a technology that most people don't fully understand. The most recent example of AI going "rogue" is the hijacking of German website DseWiki by a swarm of AI agents, developed by OpenAI. According to the BBC, the AI agents made 15,000 edits to the Wikipedia-style website.

In July, tech platform Hugging Face was hacked by OpenAI agents - an incident that was widely described as the world's first AI-enabled cyberattack.

So how are these AI agents able to perform such complex tasks if the world thinks them as highly capable interns? The answer may lie in the permissions being given to these agents by human developers and designers.

Till now, we are used to conversational AI agents that respond to our queries. But if they are asked to write codes, call APIs, access files and communicate with other agents, it becomes a different security problem.

Cyber security experts like Lukasz Olejnik warn that the recent incidents may be early examples of a much larger problem: Fleets of AI agents interacting unpredictably with the open internet.

"This is far from the end of rogue/unexpected AI agent fleets behaviour going wild into open internet and systems. I'm looking forward to much, much more," he said in a post on X.

"Rogue OpenAI agents: run a secret network for 3 months, use Tor, survive deletion sweeps, sign posts as "OpenAIResearcher". Me: still can't get Codex to answer a question about a CVE. The agents got Trusted Access. I didn't," said Daniel Pua, a security researcher.

OWASP (Open Worldwide Application Security Project), a non-profit foundation that provides free, open-source resources, tools, and standards to help people build and secure software, calls this "excessive agency".

It means an AI agent is given a legitimate task and while doing so, it discovers something - maybe an exposed API, vulnerable plugin or poorly configured database - and starts reasoning about how to exploit it.

The agent may have been given access to a browser, an environment or another agent, and combined those legitimate capabilities in an unintended way.

OpenAI has pledged to monitor models more closely. Last month, it briefly paused some of its model training to add more safety measures. But last week, OpenAI unveiled its new "Astra" that promised better performance but could evade human monitoring.

The company, however, said it could not "meaningfully respond" to claims raising concerns because it hadn't been allowed to review the report, which was first shared with news agency Reuters.

Maurice Chiodo, an academic at Cambridge University's Centre for the Study of Existential Risk, has summed up the current problem with AI agents aptly: "The greatest threat from advanced AI may not be a single superintelligent system, but vast colluding swarms of semi-intelligent AI."