Skild’s Brain Helps Injured Robots Learn To Limp

“Have you seen any of these famous humanoid companies showing stairs?” Deepak Pathak asks, toward the end of our video chat.

I give him a non-comital answer about seeing a few in simulation, and a vague memory of one company doing as much as much in at least one video.

Pathak counters that videos of humanoids climbing stairs have -- up to now -- largely been limited to short clips confined to controlled laboratory settings.

Pathak isn’t trying to single out any company, so much as make a broader point. Climbing stairs is one of those tasks that inhabits the world of Moravec's paradox – something that seems simple for non-disabled humans and is far more complex for their robotic counterparts.

To punctuate the point, the embodied AI startup just released a compilation featuring a humanoid robot – powered by Skild Brain – tromping up and down a wide variety of stair sizes in various settings outside the lab. Indeed, the Unitree robot starring in the video is game for a range of challenges, being yanked back by a rope in one and stepping onto and off a wheeled platform truck in another. It accomplishes each of these with an almost human-like movement, in most aspects down to the occasional stumble and recovery.

It's the second Skild video in as many weeks, following what amounts to a sizzle reel released last Tuesday. Pathak promises that further videos will continue to expand upon the original’s quick clips, offering addition context into the glimpses.

That first media offering served as a flag-planting moment for the Pittsburgh-based firm. Skild wasn’t in stealth, exactly, announcing a massive $300 million round last year, but the startup had thus far been playing things close to the vest. Last week, however, marked the official unveiling of Skild Brain.

The name derives from a mission statement Pathak describes as, “any task, any robot, one brain.” It’s a pithy tagline that nods to both a shared knowledgebase and the hardware agnosticism the industry has been working to crack for so long. Pathak and cofounder, Abhinav Gupta, spent years working on this and other ways to improve how robots perceive and move through the world.

ROI Calculator

Discover the potential cost savings of robotic automation over a 20-year system life

This calculator compares your current manual labor costs against the total cost of owning and operating a robotic system over its 20-year lifespan.

EXPLORE TODAY

Skild’s work leverages three familiar modes of robot learning in a bid to substantially scale the shared data sets on which these systems are trained. This, too, presents challenges. Robots need to navigate in the real world to collect data. They also need data to navigate in the real world – a bit of a chicken and egg conundrum (or, perhaps, egg and chicken).

The Skild brain leverages familiar training mechanisms, using simulation and videos of humans performing tasks to pre-train robots. Tele-operation is used in the post-training phase. Gupta notes that while he, Pathak, and countless other researchers have utilized such methods to train robots in a lab setting, scaling for deployment is another issue entirely.

“The scale with which you need to train these does not exist,” he tells me. “Some of these ideas in academia have been more proof of concept. What we have been focusing on here is taking them to the next level.”

In addition to hardware agnosticism and unstructured real-world settings, another key challenge arrives in the form of unexpected changes to the hardware itself. If we’re truly going to task these systems with ‘round the clock dull, dirty, and dangerous jobs, they’re inevitably going to take a beating.

Pathnak and Gupta have a glint in their eyes when they relate a story to illustrate the issue. It’s a good story, because it just as easily could have turned out an unmitigated disaster.  The first part, at least, will sound familiar to anyone who’s played a part in a well-funded robotics startup. Some high-profile guests are visiting the offices to check out your progress, and then suddenly the demo goes sideways.

In this case, the robot breaks an ankle. Rather than ending the demo – or worse yet, falling – the robot limps its way to the finish line. It’s never a great idea to humanize a robot, but there’s something extremely relatable in the way the system instantly determines how to best redistribute its weight.

That adjustment is, in part, the product of a system designed to work across different hardware form factors. In this example, however, it’s the same hardware having an exceptionally rough Monday morning.

Gupta stresses that the result wouldn’t have been the same, had the company gone the standard route of building for one specific set of parameters. It likely wouldn’t have ended up the kind of story you tell a reporter on a Teams call.

“Learning an AI model that only works on one hardware is a bad idea,” says Gupta. “It will never generalize and you need generalizations because physical world has no guarantees.”