OpenAI Builds A 'Kill Switch' To Stop Risky AI Model Work

OpenAI has unveiled what could be described as an emergency brake for its AI research models. It's called Control Room and the centralised system is designed to stop model output within 35 seconds of a decision to intervene when a safety or security concern arises.

As AI models become more capable and autonomous, ensuring researchers can quickly halt potentially risky behaviour becomes increasingly important. The emergency brake comes after an internal research model found a way around internet restrictions in its training sandbox, exposing a gap in OpenAI's safety systems.

In September, an internal research model from OpenAI found a way to reach an external chatbot through a gap in its training sandbox's internet restrictions.

The ChatGPT maker's monitoring system detected the behaviour and a human reviewer acknowledged the alert.

However, the research run could not be stopped automatically. It had to be manually shut down around 2.5 hours later.

"The run did not stop automatically as expected, leading to confusion around whether it should have been stopped," OpenAI had revealed.

OpenAI subsequently paused certain training, evaluation and inference activities involving tool use for its most capable models.

How Does Control Room Work?

Every research workload gets a unique job ID and the model's inference engine checks with 'Control Room' before generating output. If a job is flagged and stopped, the system denies further requests from that job.
It does not depend on the research workload itself to obey a shutdown instruction.
"The maximum time between the moment when Control Room receives an instruction to stop a job, and the moment when further progress is no longer possible, is 35 seconds," OpenAI said.
It explained that the 35-second lag comes from a caching layer for inference authorisation, to support the required volume of traffic. "We are working to drive this number down to zero so that we can end jobs instantly," it added.

Control Room is already deployed across OpenAI's research infrastructure. It is one layer in a wider safety system, working alongside monitoring for misaligned behaviour and security breaches.
However, it does not prevent a model from attempting something risky in the first place; it is designed to help stop further output once an intervention decision is made.
"Control Room is just one part of our holistic approach to safe frontier model training and deployment," OpenAI said.