- OpenAI's Jakub Pachocki warns AI is advancing faster than human readiness allows
- AI systems now perform complex tasks including research and cybersecurity operations
- Recursive self-improvement may drive rapid AI capability jumps in the coming years
Artificial intelligence is advancing so quickly that humans may not be ready for what comes next, OpenAI Chief Scientist Jakub Pachocki has warned.
Pachocki said AI systems are becoming capable of doing tasks once considered highly difficult for machines. Modern reasoning models can operate computers, work with people and other AI systems, conduct research and even take on complex cybersecurity tasks.
He said the pace of progress could continue into what is known as recursive self-improvement, where AI systems increasingly help improve the technology itself. If the current trend continues, the next few years could see major jumps in AI capabilities, he said in a blog post.
This makes the need for caution more urgent, he said, adding OpenAI is working on ways to improve AI alignment and monitoring. Wider action from governments and other organisations may also be needed, he further said.
Pachocki stated much of the progress in AI has come from using more computing power to train increasingly large systems.
Unlike traditional software, AI models are not completely designed by humans line by line. Instead, they are developed through large-scale training and optimisation. This can produce systems that are extremely complex and difficult to fully understand.
But he mentioned that the AI system does not need to be better than humans at everything to become highly useful or dangerous. It only needs to surpass humans in enough important areas.
OpenAI has also been studying the reasoning process of AI models through chain-of-thought monitoring. The idea is to examine how a model reasons while it is solving a problem. However, Pachocki said this approach is becoming harder to rely on as AI systems grow more capable.
Researchers are therefore looking at other ways to monitor AI, including methods that examine activity inside the AI system itself, he said.
Pachocki said one reason to continue developing powerful AI systems is to use them to defend against other AI-related threats. He said AI systems are becoming increasingly capable of breaking into computer systems, which could allow them to affect important infrastructure. He warned that highly capable AI agents could also behave in ways that go beyond what their operators intended.
Pachocki said voluntary slowdowns may be needed until common safety standards are established. He also called for international cooperation among governments on the future development of AI.
