Evaluating robots in the real world can be very hard. It gets worse when robots face changing settings, like homes or offices. A new approach tries to solve this problem. Researchers are learning a simulator from raw sensor data. This lets them test robots across many scenarios without making manual assets.
They gathered thousands of hours of data on Eve humanoids. These robots did tasks in homes and offices and interacted with people. They combined video and action data to train a world model. This model can predict future video from what the robot sees and does.
The main value of this project comes from simulating object interactions. For example, the robots can now fold a t-shirt. This helps make robot training more realistic and scalable. However, some technical challenges remain. The community is invited to help solve these through competitions.
Interestingly, when they placed Eve in front of a mirror, the robot did not recognize itself. In another test, the model showed an understanding of physical properties, like when a spoon fell on a table. But, it still had some errors. For example, sometimes objects didn’t fall when they should have.
To address these issues, they have set up various challenges. One is the compression challenge, which aims to minimize training loss on the robot dataset. The prize for this challenge is $10,000. There are also upcoming sampling and evaluation challenges.
In other news, Microsoft and BlackRock are creating a $30 billion AI infrastructure fund. They plan to raise money from investors, which could reach $100 billion. The goal is to put more funding into data centers. They are working with Global Infrastructure Partners and MGX to raise private equity from asset owners and corporations.
This is surprising because many thought AI infrastructure investment might slow down. Microsoft has already done the $100 billion Stargate project. Other companies like Oracle and Nvidia are also heavily investing in AI. It seems there is still a lot more to come in terms of AI investments.