Elon Musk’s artificial intelligence start-up, xAI, is levelling up. The company is now working on so-called world models, a new breed of AI systems designed to understand and interact with the physical world, not just text and images. The ambitious project puts Musk’s AI venture in direct competition with giants like Meta and Google, both of which are racing to make machines that can learn from the real world and even build new ones. This sounds similar to AGI (artificial general intelligence), but the report did not mention the term.

According to a report by the Financial Times, xAI has been quietly hiring top researchers from Nvidia over the past few months to accelerate its world-model efforts. These next-generation systems train on video footage and robotic data to grasp the dynamics of the real world, physics, movement, cause and effect, something that current text-based large language models like ChatGPT and Grok can only imagine.

From text prompts to real-world physics

While today’s most popular AI tools predict words or pixels, world models go a step further. They aim to understand how objects behave in three-dimensional environments, how a ball bounces, how light moves, or how a robot might navigate a cluttered room. In essence, they give AI a sense of physical intuition.

Two people familiar with xAI’s plans told the FT that the company is developing world models that could initially be used for gaming, allowing AI to generate immersive and dynamic 3D environments. One of them added that the same technology could eventually power robots, teaching machines to understand, and even design, real spaces.

To make this happen, xAI has hired Zeeshan Patel and Ethan He, two former Nvidia researchers with experience in world modelling. Nvidia, for its part, has been a pioneer in this field through its Omniverse platform, which lets developers simulate realistic digital worlds.

AI-generated game by next year

Musk confirmed on X (formerly Twitter) that xAI plans to release “a great AI-generated game before the end of next year”. The billionaire first teased the idea last year, and the new hires suggest that development is well underway.

This week, xAI also launched its latest image and video generation model, boasting “massive upgrades” and offering it free to users. The company is now on a hiring spree for its so-called “omni team,” which focuses on creating “magical AI experiences beyond text” — covering images, video, and audio. Salaries range from $180,000 to $440,000 a year, and one listing for a “video games tutor” promises $45–$100 per hour to help train Grok, xAI’s chatbot, in game design.

A trillion-dollar potential, but a massive challenge

The excitement around world models extends beyond Musk’s company. Nvidia recently told the Financial Times that the potential market for such technology could be as large as the entire global economy. By enabling AI to understand the real world, tech firms could unlock applications across robotics, industrial automation, and even autonomous vehicles.

But for all the hype, the challenge remains enormous. Training AI systems to accurately simulate the real world requires colossal amounts of data, computing power, and careful modelling of physics and human behaviour — something even the most advanced labs have yet to fully master.

Game industry veterans, meanwhile, remain sceptical. Michael Douse, head of publishing at Larian Studios (the studio behind Baldur’s Gate 3), said on X that AI won’t solve the “big problem” in gaming, which, in his view, is “leadership and vision”. He argued that the industry doesn’t need “more mathematically produced, psychologically trained gameplay loops,” but rather “more expressions of worlds that folks are engaged with, or want to engage with.”

As xAI pushes forward, Musk’s bet on world models marks his boldest move yet to fuse AI with the physical and digital realms. If it works, these systems could transform how machines and people experience reality. If not, it’ll be another fascinating chapter in Musk’s ever-expanding sci-fi experiment.

– Ends

Tune In