AI's next frontier is what? Fei-Fei Li's answer is space intelligence.

In the most recent episode of the a16z interview, Fei-Fei Li, the creator of ImageNet and the founder and CEO of World Labs, engaged with host Martin Casado and jointly with Li Yunzhu, co-founder of SceniX, on the topic of why AI cannot remain limited to processing text and images but must learn to understand the three-dimensional world.

Fei-Fei Li and her colleagues are now equipping AI with spatial awareness. In this way, AI will understand the basic rules of the physical world, such as why a cup will break if it drops from a table, and how much force is needed when picking up an object. Consequently, the AI will be able to see, reason, and act in three-dimensional space.

Recently, they pulled off a significant undertaking by acquiring Li Yunzhu's robotics company SceniX, successfully connecting the concepts of spatial intelligence and robotics into a unified framework. In this interview, they discussed why the merger was pursued, whether simulation-based training can be deemed reliable, whether humanoid robots have been overhyped, and what they aim to achieve with the endeavor in two years.

1

Why SceniX was acquired: Robots are stuck in the “data desert”

Martin Casado: I am pleased to welcome two today. Fei-Fei Li, first please introduce to the listeners who are not familiar with World Labs what exactly you are doing?

Li Fei-Fei: World Labs is a startup company founded two years ago. What we do is the next frontier in AI, which we call spatial intelligence. Spatial intelligence is the capability of AI to understand, generate, reason, and interact with space, whether in the physical world or the virtual one. One important path to realizing spatial intelligence is the construction of large world models, which is the core work of World Labs at present.

Martin Casado: From the very beginning you have been talking about the capabilities of machines to perceive and reason in space and act within it. But I have always thought that "acting within it" was something far in the future, so when you acquired a robotics company, does that mean this timeline has been advanced?

Li Fei-Fei: First, acting in space is not limited to robots as a single means. In creative fields such as VFX, games, and design, everything operates as interaction within virtual space. World Labs has always held the view that the world we live in can be a multi-universe, and we create technology that lets developers and builders act in different spaces. But in return, the capability to act in physical space is one of the most exciting and important capabilities for future AI, with robots being the core of it. So, the acquisition of SceniX is part of our long-term vision, and we have always believed that robots are a key application scenario for spatial intelligence and world models.

Martin Casado: Li Yunzhu, please introduce yourself and SceniX.

Li Yunzhu: I am Li Yunzhu, co-founder of SceniX and assistant professor at Columbia University. My research started with my doctorate at MIT and continued later as a postdoctoral fellow with Fei-Fei Li at Stanford. My target has always been simple: to make robots work really in real physical environments.

SceniX identified the core problem facing general-purpose robots as two key bottlenecks, training and evaluation. Therefore, they are developing a pipeline known as “real-simulation-real” that maps high-fidelity real-world environments to the digital realm so that events occurring in the digital world happen as closely as possible in reality. In this way, massive amounts of data generated in the digital world can be used to replace the expensive and scarce data found in the real world. A team was assembled that is strong in robotics, robot learning, simulation, and rendering with the goal of overcoming these bottlenecks.

Martin Casado: I hear that you two previously collaborated.

Li Fei-Fei: There is a rather interesting story. Everyone probably thinks that because Li Yunzhu was once my postdoc, we must have already planned the merger. In fact, it is not the case. The real trigger was that last winter we released the first version of the generative model Marble, after which SceniX registered and used it as a customer. Only after I took a look did I realize that it was actually Li Yunzhu's company. After I called him and had a chat, it turned out that the technical directions and synergies between the two parties were very strong.

Martin Casado: Fei-Fei Li, could you also introduce Marble to us?

李飞飞表示，Marble是World Labs一直在训练的基础模型代号。它目前的核心能力是，当提供一个提示，包括文本或者一张或几张图片时，它就可以生成一个几何上自洽的3D世界，并用高斯泼溅或网格等形式表示。而李昀竹他们要解决的是机器人领域一个极其困难的问题，即数据不足。训练数据不够，评估数据也不够。这和语言模型完全不同，因为语言模型能从互联网上获得海量文本数据。但如果机器人要发展，我们必须想办法把规模法则引入机器人领域，问题就在于数据从哪里来。这是整个行业都在面对的难题。

Martin Casado: How complementary are the talents on both sides?

Fei-Fei Li: The short answer is that the partnership is highly complementary and the missions align perfectly. The three technical co-founders of SceniX are outstanding. In addition to Li Yunzhu’s full-stack robotics capability, there is Changxi Zheng, the other Columbia professor who is a world-class expert in simulation and who has previously worked at Weta and Tencent; and Sunny, a very experienced engineering lead whose previous company was acquired by Amazon and who has rich experience in computer vision and large-scale technology stacks. World Labs is strong in generative models, computer vision, and 3D reconstruction, which is exactly what SceniX needs; while SceniX’s capabilities in full-stack robotics and simulation are precisely what World Labs has been missing. Together the two sides make the overall technology landscape more complete.

Martin Casado: Li Yunzhu, from your perspective, why did you decide to join World Labs?

Li Yunzhu: At first we also considered going it alone. But after speaking with Fei-Fei Li, I saw the many synergies in the middle. A strong combination is a very natural choice. SceniX’s real-to-sim-to-real is essentially environment reconstruction: it requires reconstruction of appearance, geometry, and also dynamics, that is, how the environment changes after actions are applied. The cost of such reconstruction is currently high. World Labs has very strong capabilities in sparse reconstruction and generative modeling, such as Marble. We can make use of these capabilities to make environment modeling more efficient and more scalable.

2

机器人基础模型长什么样

Martin Casado: Will World Labs build a foundation model for robots?

World Labs is building foundational models itself and these models are multimodal. A foundational model in the robot domain is likely to include actions as both input and output. It must be able to predict how the environment will change under specific actions and also be able to output the corresponding actions based on goals. Such a versatile model can provide great value to the robot community. It can serve as the backbone for environment modeling and also as the backbone for execution strategies. Finally, it can be fine-tuned for specific applications to meet the requirements of customers for reliability and efficiency.

Li Yunzhu: A foundational model for robots must be multimodal to simultaneously process inputs such as video frames, text, images, depth maps and more, with actions serving as one of the most critical modalities. If state and action serve as inputs, the model functions like a forward simulator that predicts how the environment will change under the applied action; if action serves as the output, it functions more like a policy model that, given a goal, decides which action to take in the real environment. This kind of fully multimodal model holds substantial value for the robot field, enabling it to both understand the environment and act within it, while also serving as a foundation that can be further fine-tuned.

马丁·卡萨多：很多机器人公司走的是视频模型路线，你们这套3D+仿真的路线和它最大的区别是什么？

Li Yunzhu: The core lies in consistency. Robots require a structured world in which to learn, and the key requirement for this world is consistency—space, time, changes in perspective, and modes of interaction all must remain consistent. Many video prediction models cause objects to vanish from nowhere, which violates physical laws, and robots cannot learn correct behavioral signals from them. Marble generates worlds that provide this foundation of consistency.

Of course, video models are also advancing. We believe a more reasonable path is to integrate these capabilities into a complete data flywheel. From simulation-driven models to policy models that execute real tasks, and then feed real data back into the system so that the entire framework grows continuously stronger. This system is not purely physical or purely learning-based; instead, it finds a balance between the two. It captures the core structure of the problem while continuously expanding with accumulating data.

Martin Casado: Li Yunzhu, what is your core objective?

Li Yunzhu: Very simply put, to make robots work really in real physical environments. During the time at Stanford with Fei-Fei Li, we conducted a survey and collected over a thousand tasks that the general public wants robots to assist with, one-third of which were related to cleaning. What people simply do not want is the dirty and laborious work. What I care about is how to bring robots into these real and demanded scenes.

I very much admire the pragmatic approach of SceniX. Even though many of them come from academia, their first reaction to robotics is to seek out design partners, identify real customers, and set up industrial labs in warehouses and electronics assembly. This focus on practical demands excites me.

3

仿真不需要“完美”

Martin Casado: Creative scenarios permit imprecision, but robots demand high precision. How is this contradiction resolved?

Li Yunzhu: Robots do require higher precision than creative applications, but simulation does not have to be absolutely perfect. Many robot systems have historically depended on models for aircraft, drones, sweepers, quadrupeds, and bipeds. They are able to transfer from simulation to reality not because simulation fully reproduces every detail, but because they capture the most critical structure of the problem and then enhance robustness through systematic randomization. What we truly need to address is how high a fidelity level is required for the real world at scale, so that robots trained in the digital world can successfully migrate back to reality.

Martin Casado: Researchers have said that simulation will always diverge from reality and that real data is the only foundation. What do you think?

Li Yunzhu: These two are not contradictory. Simulation is essentially the prediction of how the environment will change under action and it serves as a world model that can be not purely physical but a combination of physics and learning. We will collect real data and we will use real data but the focus will be different at different stages. In the initial phase we rely more on physical simulation to ensure structure and consistency and with the accumulation of data and customer cooperation we will gradually introduce more learning-based modeling. The truly important thing is how to combine physics, geometry, consistency, and data, computing power, and learning ability.

李飞飞表示：我想从哲学角度补充，仿真和真实数据并不构成二选一的局面。人类本身就在大脑里开展“仿真”工作，这被称为反事实推理，也就是推演那些尚未发生或不可能发生的事情并从中学习。在机器人领域，这一点尤其重要，因为我们永远不可能拥有足够真实的训练数据来覆盖所有情况。自动驾驶就是典型例子，Waymo公开表示他们使用了数十亿小时的仿真数据，仿真数据远超真实路测。连汽车这种“最简单”的机器人尚且如此依赖仿真，可见仿真在机器人学习中的巨大作用。

Li Yunzhu: I would like to elaborate on two points in greater detail. Simulation can provide reliability and efficiency. On the aspect of reliability, robots must operate steadily in real environments, which requires the coverage of sufficiently abundant state spaces and variations. Simulation allows systematic control over various variables such as lighting, friction, geometry, and physical parameters, enabling robots to cover more scenarios and become more robust.

效率方面，遥操作采集数据的速度比人亲自做任务还慢，但很多客户想要的是比人更快。真实世界里不能简单让机器人加速，因为重力不会配合。而在仿真里可以系统性加速行为过程，同时让模型学习速度变化带来的环境动态影响。这些是仿真能直接帮助客户的地方。

Martin Casado: How specifically do customers use your platform?

Li Yunzhu: The main challenges that remain are training and evaluation. Evaluation is often underestimated. After a model has been trained, one must know how well it performs, whether the success rate stands at 95 percent or 99.9 percent. This figure serves as the basis for iteration. When testing takes place in the real environment, the process is slow, expensive, and carries safety risks, so iteration proceeds at a pace that is several orders of magnitude slower than that of language models. But with a digital world that is highly aligned to the real environment, clients can conduct evaluation faster, more safely, and on a larger scale entirely within the simulation. So long as the simulation results correlate closely with the real results, clients can confidently iterate based on the simulation feedback.

Training data collection in real environments is not only slow but also limited by the number of robots, equipment, and personnel involved. In simulation, everything is controllable and can be systematically managed, allowing you to clearly know exactly which distributions and scenarios you have covered, thereby establishing greater confidence that the robots will function reliably within these distributions and scenarios.

Li Fei-Fei: Even prior to the merger, Marble was already receiving numerous inquiries from robotics companies, although it was not yet in a position to serve those clients. A demand for this technology does exist from the early stages of model development to downstream practical applications.

Martin Casado: When people hear that you are entering the field of robotics, they might think that you are going to build hardware and robot brains on your own. But that is not actually the case. Can you explain clearly what role you play in the robot ecosystem?

Li Yunzhu said: What we are doing is not to directly build a robot. Instead, we are constructing a set of infrastructure. This infrastructure is a set that allows others to build digital worlds, train robots, and evaluate robots. This set of infrastructure is inherently unrelated to the specific form of any robot. Customers have single-arm robots, dual-arm robots, fixed-arm robots, mobile operation platforms, and different types of grippers. Our platform can connect these different forms of robots into the digital world for training and evaluation. We are equally not bound to a single model route. Whether customers train from scratch or perform post-training on existing models, we can provide the environment, data, and evaluation. For us, the most important thing is to make robots ultimately able to reliably and efficiently complete tasks in the real world.

Martin Casado: You mentioned that the outside world has some overly optimistic predictions about humanoid robots. What is your specific take on this?

Li Yunzhu said: Robots usually follow a clear path toward deployment. This path runs from fully structured environments such as factories, where complete control over configurations is available and automation has already been practiced for decades, to semi-structured environments such as warehouses and restaurants, where only a portion of conditions can be controlled while many objects remain outside one’s reach, and finally to unstructured environments such as homes, which represent the genuine challenge. A more realistic route consists of first mastering semi-structured environments and then gradually moving toward unstructured ones, which is more steady and sustainable.

Fei-Fei Li: Humanoid robots also have one point that is easy to overlook: the human body is shaped as it is now because evolution has adapted it to unstructured environments. Our bodies are highly generic, but this does not mean they offer the optimal solution for every task. If the scene is sufficiently clear and sufficiently narrow, then a more specialized form is actually more effective. From the perspective of commerce and technical implementation, the combination of a “generic body plus unstructured environment” is actually the most difficult problem and is not necessarily the most reasonable starting point. The challenge faced by SceniX lies in allowing the infrastructure to adapt to different forms and different semi-structured environments.

4

离“像人一样高效”还有多远？

Martin Casado: Language models can produce text and code more quickly and more cheaply than humans. However, humans have already become very efficient at moving and grasping in three-dimensional space. Can we build robots in the foreseeable future that approach human energy efficiency? In five years? Or will it never be possible?

Li Yunzhu: I think it will take a very long time. Robots in the real world will always be a system, with hardware, software, algorithms, and even the friction coefficient of the fingers all needing careful design and a large amount of iteration. But I must also say that the development speed of robot learning is often faster than I expected. The problems I am researching today are completely different from when I just started my PhD, and the entire ecosystem is indeed advancing rapidly. It is just that we need to maintain a sense of restraint and calibration in our timeline.

Li Fei-Fei: One of the most challenging tasks for AI today is maintaining a prudent optimism. Even large language models remain far short of the energy efficiency of the human brain, which consumes only 30 watts, and the gap is even wider in the field of robotics. Moreover, however powerful the language models become, we cannot fully entrust them to directly handle tasks such as booking flights or hotels without human review of the outputs. Robots are different, however. Once deployed in real-world environments, they must operate reliably right out of the box. Today we still lack the supporting infrastructure and data systems for such reliability. Therefore, the ability to construct scalable digital worlds in which robots can be trained and evaluated is of great significance. It can replace the expensive, unsafe, and low-efficiency data processes that exist in the real world.

Martin Casado: In many company integrations there is a choice between a rapid merger and maintaining independence along a long-term route. What is your thinking on this?

Li Fei-Fei said we will take this very seriously but without rushing. SceniX already possesses a technical stack that is clear and relatively complete, along with its own client base and product direction. We will not immediately merge the codebases or reorganize the organizations in full. Instead, we will proceed with gradual collaboration beginning from the simulation and potential action-conditioned foundation model directions, which they are already employing Marble as an internal customer for. Integration will occur, but not in the manner of one sudden mix of all elements into a single blend.

来源：李飞飞：AI的下一个前沿不是聊天，是“让机器人看见空间” | 具身研习社