Super Models: Skild.AI is Building a Big Robot Brain with $1.4B in Fresh Funding

Skild.AI hasn’t taken the easy road. Abhinav Gupta acknowledges that generalized AI will be a significantly harder nut to crack than bespoke models tailored to different robot embodiments.  Ultimately, however, the startup’s cofounder and president doesn’t believe there’s much choice with this particular question. 

“Learning a common model across different form factors is a necessity,” he explains. 
“This is the only way we believe that this problem is going to get solved.” 

Gupta has been steeped in questions about robot intelligence for much longer than most of the industry. In addition to research stints with Meta, Google, and the Allen Institute for AI (AI2), he’s served as a professor at Carnegie Mellon’s Robotics Institute for more than 16 years. With papers published on topics like reinforcement learning, grasping, and navigation, he’s well aware that there are still plenty of challenges ahead. 

While a one-size-fits-all robot model isn’t exactly industry consensus, a growing number of startups are banking on such a generalized approach.  

The notion of a one-size-fits-all model for robotics isn’t what you would call industry consensus, but a growing number of AI startups are banking on this generalized approach. Gupta cites two primary drivers for his own thinking on the subject. The first comes down to the broader problem of data. One thing everyone in the space seems to agree about is that there just isn’t enough of it. Like, not even close.  

Various methods have been deployed to generate that data, with a wide range of efficacy. One of the best ways is to let robots interact with the real world. It’s plausible that, one day in the not-so-distant future,we’ll have a sort of critical mass of systems operating in the real, gathering and refining datasets in the process. But is there a way to achieve scale that doesn’t require having systems that are already fully trained on that data? 

Gupta contends that by focusing on data collected by a specific embodiment to train only that model, researchers may be unnecessarily bottlenecking a valuable source of shared information.  

“As robotics researchers, beggars are not choosers,” he explains. “We want as much data as we can get from the real world. We should not be choosing and saying, ‘This is for us and this is not for us.’  You have to actually build a common model across all the form factors because data is going to be very diverse.” 

Hardware diversity, meanwhile, goes beyond the form factor a robot ships with. The last time I spoke with the Gupta and cofounder/CEO Deepak Pathak, the pair recounted a humanoid demo they conducted for some deep pocketed investors. As important demos are wont to do, things went pear shaped when the system broke an ankle. Rather than forcing the startup to pack it up, however, the humanoid soldiered on, according to the founders, essentially limping the rest of the way. 

This speaks to Gupta’s second point. What happens when a bipedal robot suddenly becomes a one-and-half-pedal robot? “At the end of the day, these robots are machines with gears and motors,” he says. “Gears wear down, motors breakdown. There are multiple bodies in the world, and these bodies are also changing continuously. Wear and tear is happening, motors are breaking down. You can never assume bodies are fixed.” 

The Skild team refers to this ability to this adaptability as, “in context learning.” The feature is still in its relatively early stages, though it can be seen at work in some of the company’s demo videos. The key to building an even more powerful version of the tool is, naturally, to feed the system even more data.  

“It is using the data of its past and in context figuring it out that oh, now the body has changed,” says Gupta. “ ‘Should we be making different decisions?’  We are the first ones to bring this in context learning.  We do believe it's a great start, I mean, but of course needs much more data to be more explicit in different environments and so on. But that is where we believe the robustness would come from.” 

Much like the data and the form factors, methods for training robots remain diverse. They have different strengths and weaknesses, relative to the data and the efficacy with which its gathered. Video is proving to be a powerful tool for pre-training systems. It can offer robots valuable information about the world and how to effectively behave within it. And thankfully, human beings upload a lot of the stuff.  

Simulation can then further improve the pre-training process, as it gives robots a better sense of physics such as force information, that simply doesn’t exist on YouTube, regardless of resolution. Tele-op, while lacking the scalability of other applications, remains a valuable tool for post-training, to better refine these systems as they make their way into the world.  

“We started with this basically in 2015,” says Gupta. “I started with scaling up on robotic robots themselves, collecting data on physical robots themselves. Then we tried tele-op in 2018 through research. Recently we have been working on videos before starting Skild. And then [Pathak]’s work on simulation pioneering, where uses large scale simulations to train models in simulation that transfer to the real world.” 

The Skild team believes that, while generalized models can make the learning process more difficult, they streamline deployment. And while growing the Skild Brain is an on-going work in process, the company says it’s already begun to deploy smaller scale versions of its platform with real world customers.  

“The general perception in the public is that these robotic companies are just doing videos, there's not much deployments and so on,” says Gupta. “We, on the other hand, have been deploying these robots already. You might not have crossed it yet, but it has been in public spaces like La Guardia Airport.” 

The company says it currently has eight partners working with the Skild Brain, including a Fortune 500 company.  

“The next NVIDIA factory is also going to have Skild.AI as the brain for [a] few robots that are deployed,” says Gupta. “Our goal this year is to double the deployments, double the revenue,and so on. That’s the next phase. Right now we are confident that our Brain is ready to scale up and it will keep scaling up as more and more it trains more and more.”