Google DeepMind Unveils Genie 4: All About Real-Time World Building With Built-In Sound

Google DeepMind has launched Genie 4, which is the latest version of its experimental AI technology that lets users generate fully interactive 3D environments. In a post on X (formerly Twitter), Google CEO Sundar Pichai called it their "most realistic and interactive world model".

"We've started to roll out as a research preview to trusted testers. Coming soon to Google AI Ultra subscribers in Project Genie and early access partners via the Gemini API and AI Studio," Pichai clarified. 

According to Google, Genie 4 builds virtual spaces frame by frame as the users explore them. Users can navigate environments using keyboard controls while issuing natural language commands to change the scene.

"With Genie 4, you can set an event in motion and steer scenes in real time using natural language, like moving the camera, changing the weather, or introducing new characters. Every world also generates native audio on the fly, including ambient sounds, music, and interactive dialogue," said Google in a blog post.

Excited to share Genie 4 today our most realistic and interactive world model: - steer dynamic environments in real time, shaping events, camera angles, and actions - generate native audio matched to the world, from realistic background sounds to characters who can converse in real time! - simulate photorealistic scenes and real-world physics - create more coherent multi-step events through reasoning from Gemini We've started to roll out as a research preview to trusted testers. Coming soon to Google AI Ultra subscribers in Project Genie and early access partners via the Gemini API and AI Studio.

— Sundar Pichai (@sundarpichai) October 27, 2026

Also read | Downfall Of US Among 3 Chilling Predictions For 2028 By 'America's Nostradamus'

What is new in Genie 4?

The model brings several major upgrades over its predecessors, expanding both visual realism and how developers interact with generated spaces. 

Simulation in real-time: Genie 4 is DeepMind's first real-time world model that generates audio alongside the visuals. "Gemini will contextually adapt your prompt to match the surrounding scene. Between your inputs, Gemini also monitors and steers the scene to keep the world progressing cohesively," an article by DeepMind mentioned.

Real-time text steering: The user can type instructions during an active session, instead of starting over if they want to change an environment. A connected Gemini model interprets these prompts in context, making sure adjustments feel natural.

"Genie 4 is our first real-time world model to generate audio natively. Every world carries the ambient sound of its own setting, from wind in the trees to waves crashing at the beach," the article mentioned.

"The true frontier, however, is human speech. Genie 4 introduces interactive dialogue that adapts on the fly. Characters converse and react to your inputs or events in the world, all in real-time."

Also read | Donald Trump Pushes To Ban The Term 'AI' In Government Documents, Here's Why

Physics and human movement: DeepMind improved how the model handles collisions, aerodynamics, and body movements, leading to far more realistic human reactions and interactions.

"The most dramatic leap is how Genie 4 simulates people, creating characters that move naturally, respond organically to their environments, and can be interactively steered in real-time," the article noted.

Longer scene memory: According to TechCrunch, Genie 4 can maintain a consistent visual scene for up to a few minutes (some reports claim that it is 2 minutes), double the capacity of Genie 3. But there are continuity problems.

'Crucial aspect of building AGI'

"Understanding and simulating the world is a crucial aspect of building AGI. Genie 4 advances our ability to create safe, infinitely variable simulations that can help with everything from developing training environments for robots and agents to unlocking the creation of new interactive experiences," the article noted.

But users need to understand that the technology is still in an early research stage despite demonstrations highlighting impressive creative possibilities.