How Lightwheel is Opening Fridge Doors to Improve Physical AI

“A cup’s really easy,” says Jonathan Stephens. “It’s a solid object. There’s not really much happening there.” Lightwheel’s chief evangelist gives me the quick and dirty of generating one in simulation using a sensor. A refrigerator door, on the other hand, can be a surprisingly complex bit of work.

“Everyone has opened one of those and it takes a lot of force because it has got that seal,” he adds. “But when that seal breaks, all the force it takes to open that door is almost gone. It just swings open.”

Lightwheel is capturing that data so you – and your robots – don’t have to. Stephens alludes to deployment of robot arms tasked with opening and closing fridge doors, over and over again, generating sensor data to hue as closely to the real world experience as possible when mapped out in simulation. Limiting the real-to-sim gap to help eliminate the sim-to-real gap to better train robots that execute real world tasks continue to refine that world through realy-world data collection.

Founded in 2023, Lightwheel is one of a growing number of startups that doesn’t build the simulators that train the robots, so much as populate them with as accurate an approximate as possible of the assets in which those systems will interact.

“Think of NVIDIA as a company that creates 75 % of what you need because everyone has some different problem,” says Stephens “Why would they want to create all the assets for simulation when they don't know what asset you need? That should be up to you. But then we're finding companies so they don't have the expertise to create physics ready assets, so they come to people like us who have been doing it for a while now and have it way as repeatable and predictable.”

We talk a lot about simulation and synthetic data in these pages, these worlds require a basis in real-world data, which is where companies like Lightwheel come in. It’s a kind of brute force approach, in which data collected by specialty robots can form digital twin assets. The key is getting the measures as accurate to the physics as possible in a way that allows for massive troves of data to be collected and imported quickly. If the goal is generalized models, it will likely require massive troves of diverse data sets.

ROI Calculator

Discover the potential cost savings of robotic automation over a 20-year system life

This calculator compares your current manual labor costs against the total cost of owning and operating a robotic system over its 20-year lifespan.

EXPLORE TODAY

One seeming consensus is that robots will be trained on a broad range of data sources – the proportion of each, however, remains a source of debate. Lightwheel has diversified its own data sources, which include, EgoSuite, which utilizes human videos to help train robots in the real world, using, “action-centric, contact-rich, multimodal recordings across diverse real environments.” There are also exoskeleton and other wearables, designed to collect data in a human embodiment for robot training.

And while true edge cases are, by their nature, difficult (or perhaps impossible) to fully predict for, Stephens points out that training in simulation affords companies the ability to recreate scenarios companies want to stay far away from in the real world. “You can replicate in simulation edge cases that I don't necessarily want to create in real life,” he explains. “That’s what autonomous driving's been doing for a while, too. You have a kid running out in front of the road. You want the car to stop. Maybe a ball rolls out across the road. You're going to stop for the ball, but you want that car to stay and wait. Because there might be a kid coming after that. But not the same thing for a human as well. Like, what happens if something happens that's an edge case? We want it to know what to do, but we don't necessarily want to the edge cases in life. Maybe it's a fire, you don't want to just light a house on fire and see if the robot can make it out.”